2024-09-17T05:09:45.764 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T05:09:45.786 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T05:09:45.880 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689 branch: wip-athakkar-testing-2024-09-16-2351 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: athakkar@redhat.com first_in_suite: false job_id: '7908689' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-athakkar-testing-2024-09-16-2351 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 820f760e4779dafd30d668ecea7aba6355dfd3d8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 820f760e4779dafd30d668ecea7aba6355dfd3d8 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-athakkar-testing-2024-09-16-2351 sha1: 820f760e4779dafd30d668ecea7aba6355dfd3d8 owner: scheduled_avan@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 448 sha1: 820f760e4779dafd30d668ecea7aba6355dfd3d8 sleep_before_teardown: 0 subset: 1/20 suite: fs:volumes suite_branch: wip-athakkar-testing-2024-09-16-2351 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_820f760e4779dafd30d668ecea7aba6355dfd3d8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 820f760e4779dafd30d668ecea7aba6355dfd3d8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBCk2fv5ikwml16vxOA0Y2e90NM9G4J5usuURDz37ms5URmAafpco6DKPEUcrvq/rnZxUxJSgvK9rHW7SSyrR5E= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgU/LFxku8av9JJyCv9GHIp42FmzgHX5JGRPjQRQ4JaRQ2RMaQUX/wJeJVDXF9uq+vRKG1JjH6CQVPRChUb0Ao= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_05:02:16 tube: smithi user: avan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T05:09:45.880 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_820f760e4779dafd30d668ecea7aba6355dfd3d8/qa; will attempt to use it 2024-09-17T05:09:45.881 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_820f760e4779dafd30d668ecea7aba6355dfd3d8/qa/tasks 2024-09-17T05:09:45.881 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T05:09:45.882 INFO:teuthology.task.internal:Checking packages... 2024-09-17T05:09:45.906 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '820f760e4779dafd30d668ecea7aba6355dfd3d8' 2024-09-17T05:09:45.906 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T05:09:45.906 INFO:teuthology.packaging:ref: None 2024-09-17T05:09:45.906 INFO:teuthology.packaging:tag: None 2024-09-17T05:09:45.906 INFO:teuthology.packaging:branch: wip-athakkar-testing-2024-09-16-2351 2024-09-17T05:09:45.906 INFO:teuthology.packaging:sha1: 820f760e4779dafd30d668ecea7aba6355dfd3d8 2024-09-17T05:09:45.906 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-athakkar-testing-2024-09-16-2351 2024-09-17T05:09:46.049 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4963-g820f760e-1jammy 2024-09-17T05:09:46.051 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T05:09:46.058 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T05:09:46.058 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T05:09:46.103 INFO:teuthology.task.internal:Saving configuration 2024-09-17T05:09:46.121 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T05:09:46.127 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T05:09:46.153 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 05:04:06.577107', 'locked_by': 'scheduled_avan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDUnW16qLPmMpFaER0hvgal1F1Xx/3qMTQTBX9rKl/Gm'} 2024-09-17T05:09:46.174 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 05:04:06.578526', 'locked_by': 'scheduled_avan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB07P4R6EBSZOr/e1mje4CbsCqFXy8V26UWAYYVp4qGB'} 2024-09-17T05:09:46.174 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T05:09:46.180 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-17T05:09:46.180 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-17T05:09:46.180 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T05:09:46.260 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5ffdccfbe0>, signals=[15]) 2024-09-17T05:09:46.261 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T05:09:46.267 INFO:teuthology.task.internal:Opening connections... 2024-09-17T05:09:46.267 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:09:46.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:09:46.347 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:09:46.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:09:46.429 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T05:09:46.435 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-17T05:09:46.440 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-17T05:09:46.440 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-17T05:09:46.486 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T05:09:46.486 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2024-09-17T05:09:46.486 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="22.04" 2024-09-17T05:09:46.486 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T05:09:46.486 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=jammy 2024-09-17T05:09:46.487 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2024-09-17T05:09:46.487 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2024-09-17T05:09:46.487 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T05:09:46.487 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T05:09:46.487 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T05:09:46.487 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T05:09:46.487 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=jammy 2024-09-17T05:09:46.488 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-17T05:09:46.510 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-09-17T05:09:46.515 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-09-17T05:09:46.515 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-09-17T05:09:46.562 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T05:09:46.562 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="Ubuntu" 2024-09-17T05:09:46.562 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="22.04" 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_CODENAME=jammy 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:ID=ubuntu 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE=debian 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T05:09:46.563 INFO:teuthology.orchestra.run.smithi173.stdout:UBUNTU_CODENAME=jammy 2024-09-17T05:09:46.563 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-09-17T05:09:46.585 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T05:09:46.595 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T05:09:46.628 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T05:09:46.628 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-17T05:09:46.630 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-09-17T05:09:46.635 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T05:09:46.672 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T05:09:46.673 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-17T05:09:46.676 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-09-17T05:09:46.764 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T05:09:46.821 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T05:09:46.835 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-17T05:09:46.836 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-17T05:09:46.836 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-17T05:09:46.837 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T05:09:46.837 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T05:09:46.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:09:46.842 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-17T05:09:46.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:09:46.845 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-17T05:09:47.007 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:09:47.008 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T05:09:47.008 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-09-17T05:09:47.013 INFO:teuthology.orchestra.run.smithi173.stdout:5.15.0-56-generic 2024-09-17T05:09:47.013 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:09:47.014 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T05:09:47.014 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-17T05:09:47.014 DEBUG:teuthology.orchestra.run.smithi173:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-17T05:09:47.019 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:09:47.019 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T05:09:47.019 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-17T05:09:47.024 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-56-generic 2024-09-17T05:09:47.025 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:09:47.025 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T05:09:47.025 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-17T05:09:47.025 DEBUG:teuthology.orchestra.run.smithi122:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-17T05:09:47.069 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-17T05:09:47.083 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-17T05:09:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-17T05:09:47.222 INFO:teuthology.orchestra.run.smithi122.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-17T05:09:47.223 INFO:teuthology.orchestra.run.smithi122.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-17T05:09:47.223 INFO:teuthology.orchestra.run.smithi122.stderr:hint: 2024-09-17T05:09:47.223 INFO:teuthology.orchestra.run.smithi122.stderr:hint: git config --global init.defaultBranch 2024-09-17T05:09:47.223 INFO:teuthology.orchestra.run.smithi122.stderr:hint: 2024-09-17T05:09:47.223 INFO:teuthology.orchestra.run.smithi122.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-17T05:09:47.223 INFO:teuthology.orchestra.run.smithi122.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-17T05:09:47.223 INFO:teuthology.orchestra.run.smithi122.stderr:hint: 2024-09-17T05:09:47.223 INFO:teuthology.orchestra.run.smithi122.stderr:hint: git branch -m 2024-09-17T05:09:47.224 INFO:teuthology.orchestra.run.smithi122.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-17T05:09:47.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-17T05:09:47.227 INFO:teuthology.orchestra.run.smithi173.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-17T05:09:47.227 INFO:teuthology.orchestra.run.smithi173.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-17T05:09:47.227 INFO:teuthology.orchestra.run.smithi173.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-17T05:09:47.227 INFO:teuthology.orchestra.run.smithi173.stderr:hint: 2024-09-17T05:09:47.227 INFO:teuthology.orchestra.run.smithi173.stderr:hint: git config --global init.defaultBranch 2024-09-17T05:09:47.227 INFO:teuthology.orchestra.run.smithi173.stderr:hint: 2024-09-17T05:09:47.228 INFO:teuthology.orchestra.run.smithi173.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-17T05:09:47.228 INFO:teuthology.orchestra.run.smithi173.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-17T05:09:47.228 INFO:teuthology.orchestra.run.smithi173.stderr:hint: 2024-09-17T05:09:47.228 INFO:teuthology.orchestra.run.smithi173.stderr:hint: git branch -m 2024-09-17T05:09:47.228 INFO:teuthology.orchestra.run.smithi173.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-17T05:09:47.230 DEBUG:teuthology.orchestra.run.smithi173:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-17T05:09:47.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-17T05:09:47.298 DEBUG:teuthology.orchestra.run.smithi173:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-17T05:09:47.352 DEBUG:teuthology.orchestra.run.smithi122:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-17T05:09:47.360 DEBUG:teuthology.orchestra.run.smithi173:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-17T05:10:25.484 INFO:teuthology.orchestra.run.smithi173.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-17T05:10:25.484 INFO:teuthology.orchestra.run.smithi173.stderr: * [new branch] main -> origin/main 2024-09-17T05:10:25.520 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190312 -> 20190312 2024-09-17T05:10:25.520 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190416 -> 20190416 2024-09-17T05:10:25.520 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190514 -> 20190514 2024-09-17T05:10:25.520 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190618 -> 20190618 2024-09-17T05:10:25.521 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190717 -> 20190717 2024-09-17T05:10:25.521 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190815 -> 20190815 2024-09-17T05:10:25.521 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190923 -> 20190923 2024-09-17T05:10:25.521 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20191022 -> 20191022 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20191215 -> 20191215 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200122 -> 20200122 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200316 -> 20200316 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200421 -> 20200421 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200519 -> 20200519 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200619 -> 20200619 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200721 -> 20200721 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200817 -> 20200817 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200918 -> 20200918 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20201022 -> 20201022 2024-09-17T05:10:25.523 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20201118 -> 20201118 2024-09-17T05:10:25.524 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20201218 -> 20201218 2024-09-17T05:10:25.524 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20210208 -> 20210208 2024-09-17T05:10:25.524 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20210315 -> 20210315 2024-09-17T05:10:25.524 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20210511 -> 20210511 2024-09-17T05:10:25.524 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20210716 -> 20210716 2024-09-17T05:10:25.524 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20210818 -> 20210818 2024-09-17T05:10:25.524 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20210919 -> 20210919 2024-09-17T05:10:25.524 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20211027 -> 20211027 2024-09-17T05:10:25.527 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20211216 -> 20211216 2024-09-17T05:10:25.527 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20220209 -> 20220209 2024-09-17T05:10:25.527 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20220310 -> 20220310 2024-09-17T05:10:25.527 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20220411 -> 20220411 2024-09-17T05:10:25.527 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20220509 -> 20220509 2024-09-17T05:10:25.527 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20220610 -> 20220610 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20220708 -> 20220708 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20220815 -> 20220815 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20220913 -> 20220913 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20221012 -> 20221012 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20221109 -> 20221109 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20221214 -> 20221214 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20230117 -> 20230117 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20230210 -> 20230210 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20230310 -> 20230310 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20230404 -> 20230404 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20230515 -> 20230515 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20230625 -> 20230625 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20230804 -> 20230804 2024-09-17T05:10:25.528 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20230919 -> 20230919 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20231030 -> 20231030 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20231111 -> 20231111 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20231211 -> 20231211 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240115 -> 20240115 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240220 -> 20240220 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240312 -> 20240312 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240410 -> 20240410 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240513 -> 20240513 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240610 -> 20240610 2024-09-17T05:10:25.529 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240709 -> 20240709 2024-09-17T05:10:25.530 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240811 -> 20240811 2024-09-17T05:10:25.530 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20240909 -> 20240909 2024-09-17T05:10:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-17T05:10:26.083 INFO:teuthology.orchestra.run.smithi122.stderr: * [new branch] main -> origin/main 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20190312 -> 20190312 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20190416 -> 20190416 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20190514 -> 20190514 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20190618 -> 20190618 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20190717 -> 20190717 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20190815 -> 20190815 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20190923 -> 20190923 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20191022 -> 20191022 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20191215 -> 20191215 2024-09-17T05:10:26.120 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20200122 -> 20200122 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20200316 -> 20200316 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20200421 -> 20200421 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20200519 -> 20200519 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20200619 -> 20200619 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20200721 -> 20200721 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20200817 -> 20200817 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20200918 -> 20200918 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20201022 -> 20201022 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20201118 -> 20201118 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20201218 -> 20201218 2024-09-17T05:10:26.123 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20210208 -> 20210208 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20210315 -> 20210315 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20210511 -> 20210511 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20210716 -> 20210716 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20210818 -> 20210818 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20210919 -> 20210919 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20211027 -> 20211027 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20211216 -> 20211216 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20220209 -> 20220209 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20220310 -> 20220310 2024-09-17T05:10:26.124 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20220411 -> 20220411 2024-09-17T05:10:26.128 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20220509 -> 20220509 2024-09-17T05:10:26.128 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20220610 -> 20220610 2024-09-17T05:10:26.128 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20220708 -> 20220708 2024-09-17T05:10:26.128 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20220815 -> 20220815 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20220913 -> 20220913 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20221012 -> 20221012 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20221109 -> 20221109 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20221214 -> 20221214 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20230117 -> 20230117 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20230210 -> 20230210 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20230310 -> 20230310 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20230404 -> 20230404 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20230515 -> 20230515 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20230625 -> 20230625 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20230804 -> 20230804 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20230919 -> 20230919 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20231030 -> 20231030 2024-09-17T05:10:26.129 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20231111 -> 20231111 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20231211 -> 20231211 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240115 -> 20240115 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240220 -> 20240220 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240312 -> 20240312 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240410 -> 20240410 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240513 -> 20240513 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240610 -> 20240610 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240709 -> 20240709 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240811 -> 20240811 2024-09-17T05:10:26.130 INFO:teuthology.orchestra.run.smithi122.stderr: * [new tag] 20240909 -> 20240909 2024-09-17T05:10:30.125 INFO:teuthology.orchestra.run.smithi173.stderr:Updating files: 41% (1453/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2076/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 69% (2429/3520) Updating files: 69% (2434/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 82% (2887/3520) Updating files: 82% (2902/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-17T05:10:30.129 INFO:teuthology.orchestra.run.smithi173.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-17T05:10:30.152 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T05:10:30.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:10:30.295 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:10:30.296 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-17T05:10:30.299 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi173.front.sepia.ceph.com to complete... 2024-09-17T05:10:30.725 INFO:teuthology.orchestra.run.smithi122.stderr:Updating files: 33% (1167/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2072/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 69% (2429/3520) Updating files: 69% (2433/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 82% (2887/3520) Updating files: 82% (2902/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-17T05:10:30.730 INFO:teuthology.orchestra.run.smithi122.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-17T05:10:30.751 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T05:10:30.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T05:10:30.893 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:10:30.893 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-17T05:10:30.896 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi122.front.sepia.ceph.com to complete... 2024-09-17T05:10:31.373 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17 05:10:31 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-17T05:10:31.373 INFO:teuthology.orchestra.run.smithi173.stderr:FINISHED --2024-09-17 05:10:31-- 2024-09-17T05:10:31.373 INFO:teuthology.orchestra.run.smithi173.stderr:Total wall clock time: 1.0s 2024-09-17T05:10:31.373 INFO:teuthology.orchestra.run.smithi173.stderr:Downloaded: 1 files, 43M in 0.9s (49.7 MB/s) 2024-09-17T05:10:31.377 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T05:10:31.377 DEBUG:teuthology.orchestra.run.smithi173:> sudo dpkg -i /tmp/linux-image.deb 2024-09-17T05:10:31.631 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-17T05:10:32.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17 05:10:32 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-17T05:10:32.638 INFO:teuthology.orchestra.run.smithi122.stderr:FINISHED --2024-09-17 05:10:32-- 2024-09-17T05:10:32.638 INFO:teuthology.orchestra.run.smithi122.stderr:Total wall clock time: 1.7s 2024-09-17T05:10:32.638 INFO:teuthology.orchestra.run.smithi122.stderr:Downloaded: 1 files, 43M in 1.4s (30.5 MB/s) 2024-09-17T05:10:32.642 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T05:10:32.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo dpkg -i /tmp/linux-image.deb 2024-09-17T05:10:32.951 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-17T05:10:33.615 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-17T05:10:33.620 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-17T05:10:33.726 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T05:10:34.978 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-17T05:10:34.982 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-17T05:10:35.077 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T05:10:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T05:10:37.860 INFO:teuthology.orchestra.run.smithi173.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:10:39.035 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T05:10:39.150 INFO:teuthology.orchestra.run.smithi122.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:00.516 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub' 2024-09-17T05:11:00.538 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T05:11:00.561 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2024-09-17T05:11:00.847 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:00.867 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:01.157 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T05:11:01.182 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T05:11:01.245 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T05:11:01.250 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T05:11:01.398 INFO:teuthology.orchestra.run.smithi173.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T05:11:01.398 INFO:teuthology.orchestra.run.smithi173.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T05:11:01.398 INFO:teuthology.orchestra.run.smithi173.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T05:11:01.449 INFO:teuthology.orchestra.run.smithi173.stderr:done 2024-09-17T05:11:01.621 DEBUG:teuthology.orchestra.run.smithi173:> dpkg-deb -c /tmp/linux-image.deb 2024-09-17T05:11:01.628 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-17T05:11:01.629 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-17T05:11:01.629 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:01.643 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:01.644 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-17T05:11:01.666 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T05:11:01.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T05:11:01.668 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T05:11:01.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-17T05:11:01.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T05:11:01.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T05:11:01.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T05:11:01.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-17T05:11:01.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T05:11:01.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T05:11:01.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T05:11:01.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T05:11:01.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T05:11:01.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T05:11:01.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T05:11:01.697 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-17T05:11:01.698 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T05:11:01.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-17T05:11:01.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-17T05:11:01.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-17T05:11:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-17T05:11:01.703 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-17T05:11:01.703 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T05:11:01.756 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-17T05:11:01.756 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T05:11:01.757 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T05:11:01.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-17T05:11:01.759 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-17T05:11:01.760 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-17T05:11:01.761 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-17T05:11:01.762 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T05:11:01.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-17T05:11:01.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T05:11:01.765 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T05:11:01.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T05:11:01.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-17T05:11:01.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T05:11:01.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T05:11:01.770 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-17T05:11:01.771 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T05:11:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T05:11:01.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T05:11:01.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T05:11:01.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T05:11:01.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T05:11:01.777 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T05:11:01.778 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T05:11:01.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-17T05:11:01.780 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-17T05:11:01.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T05:11:01.782 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T05:11:01.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T05:11:01.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T05:11:01.785 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T05:11:01.786 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-17T05:11:01.787 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-17T05:11:01.788 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-17T05:11:01.789 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-17T05:11:01.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-17T05:11:01.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T05:11:01.792 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T05:11:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-17T05:11:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-17T05:11:01.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T05:11:01.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-17T05:11:01.797 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T05:11:01.798 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T05:11:01.799 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T05:11:01.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T05:11:01.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T05:11:01.802 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T05:11:01.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-17T05:11:01.804 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T05:11:01.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T05:11:01.806 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-17T05:11:01.808 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T05:11:01.808 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-17T05:11:01.808 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T05:11:01.809 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T05:11:01.810 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T05:11:01.811 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T05:11:01.812 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-17T05:11:01.813 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T05:11:01.814 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-17T05:11:01.814 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T05:11:01.814 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-17T05:11:01.814 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T05:11:01.814 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-17T05:11:01.814 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T05:11:01.814 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-17T05:11:01.814 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T05:11:01.881 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T05:11:01.882 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-17T05:11:01.883 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T05:11:01.884 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T05:11:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-17T05:11:01.886 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T05:11:01.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T05:11:01.888 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T05:11:01.889 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-17T05:11:01.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T05:11:01.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T05:11:01.892 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T05:11:01.893 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T05:11:01.894 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-17T05:11:01.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T05:11:01.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T05:11:01.897 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T05:11:01.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-17T05:11:01.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-17T05:11:01.900 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-17T05:11:01.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-17T05:11:01.902 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T05:11:01.903 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-17T05:11:01.904 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T05:11:01.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T05:11:01.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-17T05:11:01.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T05:11:01.908 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T05:11:01.909 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T05:11:01.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T05:11:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T05:11:01.912 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-17T05:11:01.913 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-17T05:11:01.914 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T05:11:01.915 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T05:11:01.916 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-17T05:11:01.917 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T05:11:01.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T05:11:01.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-17T05:11:01.920 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T05:11:01.921 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-17T05:11:01.922 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-17T05:11:01.923 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-17T05:11:01.924 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-17T05:11:01.925 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-17T05:11:01.926 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-17T05:11:01.927 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T05:11:01.928 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T05:11:01.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-17T05:11:01.930 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T05:11:01.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T05:11:01.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T05:11:01.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T05:11:01.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T05:11:01.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-17T05:11:01.936 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-17T05:11:01.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-17T05:11:01.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T05:11:01.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T05:11:01.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-17T05:11:01.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T05:11:01.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T05:11:01.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T05:11:01.944 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-17T05:11:01.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-17T05:11:01.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-17T05:11:01.947 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T05:11:01.948 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T05:11:01.949 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T05:11:01.950 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T05:11:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-17T05:11:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T05:11:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T05:11:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-17T05:11:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T05:11:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T05:11:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-17T05:11:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T05:11:01.991 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-17T05:11:01.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T05:11:01.991 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-17T05:11:01.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T05:11:01.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T05:11:01.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T05:11:01.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T05:11:01.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T05:11:01.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T05:11:01.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-17T05:11:01.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-17T05:11:01.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-17T05:11:01.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T05:11:01.998 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-17T05:11:01.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T05:11:02.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T05:11:02.001 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-17T05:11:02.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T05:11:02.003 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T05:11:02.004 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T05:11:02.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T05:11:02.006 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T05:11:02.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T05:11:02.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T05:11:02.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T05:11:02.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T05:11:02.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T05:11:02.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T05:11:02.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T05:11:02.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T05:11:02.015 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-17T05:11:02.016 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T05:11:02.017 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T05:11:02.018 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T05:11:02.019 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T05:11:02.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T05:11:02.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T05:11:02.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-17T05:11:02.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T05:11:02.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T05:11:02.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T05:11:02.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T05:11:02.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T05:11:02.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T05:11:02.029 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T05:11:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T05:11:02.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T05:11:02.032 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T05:11:02.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T05:11:02.034 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T05:11:02.035 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T05:11:02.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-17T05:11:02.037 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-17T05:11:02.038 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:11:02.038 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-17T05:11:02.038 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-17T05:11:02.042 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:02.043 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:02.043 DEBUG:teuthology.orchestra.run.smithi173:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-17T05:11:02.049 INFO:teuthology.orchestra.run.smithi173.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:11:02.049 INFO:teuthology.orchestra.run.smithi173.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:11:02.049 INFO:teuthology.orchestra.run.smithi173.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:11:02.050 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-17T05:11:02.050 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:02.050 DEBUG:teuthology.orchestra.run.smithi173:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-17T05:11:02.095 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi173.front.sepia.ceph.com to complete... 2024-09-17T05:11:02.154 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub' 2024-09-17T05:11:02.166 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T05:11:02.195 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-17T05:11:02.451 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub' 2024-09-17T05:11:02.452 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T05:11:02.466 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2024-09-17T05:11:02.504 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:02.528 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:02.639 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:02.644 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:02.857 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T05:11:02.870 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T05:11:02.877 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T05:11:02.882 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T05:11:02.918 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T05:11:02.923 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T05:11:02.928 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T05:11:02.934 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T05:11:03.075 INFO:teuthology.orchestra.run.smithi173.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T05:11:03.076 INFO:teuthology.orchestra.run.smithi173.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T05:11:03.076 INFO:teuthology.orchestra.run.smithi173.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T05:11:03.087 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T05:11:03.087 INFO:teuthology.orchestra.run.smithi122.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T05:11:03.087 INFO:teuthology.orchestra.run.smithi122.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T05:11:03.091 INFO:teuthology.orchestra.run.smithi173.stderr:done 2024-09-17T05:11:03.125 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-17T05:11:03.303 DEBUG:teuthology.orchestra.run.smithi122:> dpkg-deb -c /tmp/linux-image.deb 2024-09-17T05:11:03.310 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-17T05:11:03.310 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-17T05:11:03.311 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:03.325 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:03.325 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:03.347 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-17T05:11:03.347 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-17T05:11:03.347 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-17T05:11:03.347 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-17T05:11:03.347 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-17T05:11:03.347 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-17T05:11:03.347 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-17T05:11:03.347 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T05:11:03.348 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T05:11:03.349 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T05:11:03.349 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T05:11:03.349 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T05:11:03.349 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T05:11:03.349 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T05:11:03.349 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T05:11:03.349 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T05:11:03.349 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T05:11:03.350 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-17T05:11:03.351 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-17T05:11:03.351 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-17T05:11:03.351 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T05:11:03.351 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-17T05:11:03.351 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-17T05:11:03.351 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-17T05:11:03.351 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T05:11:03.351 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T05:11:03.450 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T05:11:03.450 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-17T05:11:03.450 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-17T05:11:03.450 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-17T05:11:03.450 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-17T05:11:03.450 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T05:11:03.450 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T05:11:03.450 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T05:11:03.451 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-17T05:11:03.451 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-17T05:11:03.451 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-17T05:11:03.451 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T05:11:03.451 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T05:11:03.451 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T05:11:03.451 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T05:11:03.451 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T05:11:03.452 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T05:11:03.453 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T05:11:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T05:11:03.455 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T05:11:03.456 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T05:11:03.457 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T05:11:03.458 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T05:11:03.459 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T05:11:03.460 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T05:11:03.461 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T05:11:03.462 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T05:11:03.462 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T05:11:03.462 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T05:11:03.462 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T05:11:03.462 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T05:11:03.462 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T05:11:03.462 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T05:11:03.462 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T05:11:03.463 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T05:11:03.463 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T05:11:03.463 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T05:11:03.463 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T05:11:03.463 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T05:11:03.463 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T05:11:03.463 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-17T05:11:03.463 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T05:11:03.464 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-17T05:11:03.464 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T05:11:03.464 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-17T05:11:03.464 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-17T05:11:03.464 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T05:11:03.464 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-17T05:11:03.464 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T05:11:03.464 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T05:11:03.465 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-17T05:11:03.465 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T05:11:03.465 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T05:11:03.465 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-17T05:11:03.465 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T05:11:03.465 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T05:11:03.465 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T05:11:03.465 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T05:11:03.466 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T05:11:03.467 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-17T05:11:03.468 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-17T05:11:03.469 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-17T05:11:03.469 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T05:11:03.469 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T05:11:03.469 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-17T05:11:03.469 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-17T05:11:03.469 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-17T05:11:03.469 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-17T05:11:03.469 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T05:11:03.470 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-17T05:11:03.471 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T05:11:03.472 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T05:11:03.472 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T05:11:03.472 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-17T05:11:03.472 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T05:11:03.472 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-17T05:11:03.472 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T05:11:03.472 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-17T05:11:03.472 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T05:11:03.473 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T05:11:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T05:11:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T05:11:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T05:11:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T05:11:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T05:11:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T05:11:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T05:11:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-17T05:11:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-17T05:11:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-17T05:11:03.477 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-17T05:11:03.478 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-17T05:11:03.479 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T05:11:03.480 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T05:11:03.481 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T05:11:03.482 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T05:11:03.483 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T05:11:03.484 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T05:11:03.485 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T05:11:03.486 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T05:11:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T05:11:03.488 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T05:11:03.489 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T05:11:03.490 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-17T05:11:03.491 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T05:11:03.492 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T05:11:03.493 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T05:11:03.494 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T05:11:03.495 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T05:11:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T05:11:03.497 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T05:11:03.498 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T05:11:03.499 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T05:11:03.499 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T05:11:03.499 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T05:11:03.499 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T05:11:03.499 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T05:11:03.499 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T05:11:03.499 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-17T05:11:03.499 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T05:11:03.500 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-17T05:11:03.501 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T05:11:03.502 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T05:11:03.503 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T05:11:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T05:11:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T05:11:03.506 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-17T05:11:03.507 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-17T05:11:03.508 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-17T05:11:03.509 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-17T05:11:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-17T05:11:03.511 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T05:11:03.512 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-17T05:11:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-17T05:11:03.514 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T05:11:03.515 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T05:11:03.516 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T05:11:03.517 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-17T05:11:03.518 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-17T05:11:03.519 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T05:11:03.520 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-17T05:11:03.521 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-17T05:11:03.522 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-17T05:11:03.523 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-17T05:11:03.524 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-17T05:11:03.525 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-17T05:11:03.526 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T05:11:03.527 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-17T05:11:03.528 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T05:11:03.529 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-17T05:11:03.529 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T05:11:03.529 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-17T05:11:03.529 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T05:11:03.529 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T05:11:03.529 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T05:11:03.529 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-17T05:11:03.529 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T05:11:03.530 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-17T05:11:03.531 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-17T05:11:03.532 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-17T05:11:03.533 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-17T05:11:03.534 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-17T05:11:03.535 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T05:11:03.536 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-17T05:11:03.537 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T05:11:03.537 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T05:11:03.537 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-17T05:11:03.537 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T05:11:03.537 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T05:11:03.537 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T05:11:03.537 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-17T05:11:03.538 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-17T05:11:03.538 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T05:11:03.538 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T05:11:03.538 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-17T05:11:03.538 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-17T05:11:03.538 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T05:11:03.677 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-17T05:11:03.677 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-17T05:11:03.677 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-17T05:11:03.677 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-17T05:11:03.677 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T05:11:03.677 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-17T05:11:03.677 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-17T05:11:03.678 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-17T05:11:03.679 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T05:11:03.680 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-17T05:11:03.681 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-17T05:11:03.682 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-17T05:11:03.683 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-17T05:11:03.684 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T05:11:03.685 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T05:11:03.686 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T05:11:03.687 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T05:11:03.688 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T05:11:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T05:11:03.690 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-17T05:11:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-17T05:11:03.692 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T05:11:03.693 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T05:11:03.694 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-17T05:11:03.695 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-17T05:11:03.696 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-17T05:11:03.697 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T05:11:03.698 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T05:11:03.699 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-17T05:11:03.700 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-17T05:11:03.701 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-17T05:11:03.702 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T05:11:03.703 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T05:11:03.704 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T05:11:03.705 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T05:11:03.706 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T05:11:03.707 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T05:11:03.708 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-17T05:11:03.709 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T05:11:03.709 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-17T05:11:03.709 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T05:11:03.709 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T05:11:03.709 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T05:11:03.710 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T05:11:03.711 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T05:11:03.712 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T05:11:03.713 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T05:11:03.714 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-17T05:11:03.715 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-17T05:11:03.716 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-17T05:11:03.716 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T05:11:03.716 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-17T05:11:03.716 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-17T05:11:03.716 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T05:11:03.716 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-17T05:11:03.716 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-17T05:11:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-17T05:11:03.718 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T05:11:03.719 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-17T05:11:03.720 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-17T05:11:03.721 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-17T05:11:03.722 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-17T05:11:03.723 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-17T05:11:03.724 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T05:11:03.725 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T05:11:03.726 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-17T05:11:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T05:11:03.728 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T05:11:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T05:11:03.730 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-17T05:11:03.731 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T05:11:03.732 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-17T05:11:03.733 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-17T05:11:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-17T05:11:03.735 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-17T05:11:03.736 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-17T05:11:03.737 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T05:11:03.738 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T05:11:03.739 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T05:11:03.740 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T05:11:03.741 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-17T05:11:03.742 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-17T05:11:03.743 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-17T05:11:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-17T05:11:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-17T05:11:03.746 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T05:11:03.747 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-17T05:11:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T05:11:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T05:11:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T05:11:03.751 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T05:11:03.751 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T05:11:03.751 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T05:11:03.751 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T05:11:03.751 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T05:11:03.751 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T05:11:03.751 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-17T05:11:03.752 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T05:11:03.753 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T05:11:03.753 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T05:11:03.753 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T05:11:03.753 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T05:11:03.753 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-17T05:11:03.753 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-17T05:11:03.753 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-17T05:11:03.754 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T05:11:03.754 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-17T05:11:03.754 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T05:11:03.754 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-17T05:11:03.754 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-17T05:11:03.754 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-17T05:11:03.754 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-17T05:11:03.754 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-17T05:11:03.755 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-17T05:11:03.755 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-17T05:11:03.755 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-17T05:11:03.755 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-17T05:11:03.755 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T05:11:03.755 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-17T05:11:03.755 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T05:11:03.755 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-17T05:11:03.756 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-17T05:11:03.756 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T05:11:03.756 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-17T05:11:03.756 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T05:11:03.756 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T05:11:03.756 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-17T05:11:03.756 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T05:11:03.757 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-17T05:11:03.757 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T05:11:03.757 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-17T05:11:03.757 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T05:11:03.757 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-17T05:11:03.757 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T05:11:03.757 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-17T05:11:03.757 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T05:11:03.758 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-17T05:11:03.758 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-17T05:11:03.758 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-17T05:11:03.758 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-17T05:11:03.758 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-17T05:11:03.758 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T05:11:03.758 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-17T05:11:03.758 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T05:11:03.759 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-17T05:11:03.759 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T05:11:03.759 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-17T05:11:03.759 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T05:11:03.759 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T05:11:03.759 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-17T05:11:03.759 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T05:11:03.759 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T05:11:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-17T05:11:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T05:11:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T05:11:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-17T05:11:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T05:11:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-17T05:11:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-17T05:11:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-17T05:11:03.761 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-17T05:11:03.761 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-17T05:11:03.761 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T05:11:03.761 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-17T05:11:03.761 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-17T05:11:03.761 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T05:11:03.762 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-17T05:11:03.762 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-17T05:11:03.762 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-17T05:11:03.762 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-17T05:11:03.762 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T05:11:03.762 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T05:11:03.762 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T05:11:03.763 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-17T05:11:03.763 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-17T05:11:03.763 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-17T05:11:03.763 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T05:11:03.763 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-17T05:11:03.763 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-17T05:11:03.763 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T05:11:03.763 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T05:11:03.765 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T05:11:03.765 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T05:11:03.765 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T05:11:03.765 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T05:11:03.765 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-17T05:11:03.765 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T05:11:03.765 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-17T05:11:03.765 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T05:11:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-17T05:11:03.767 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T05:11:03.767 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T05:11:03.767 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T05:11:03.767 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T05:11:03.767 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T05:11:03.767 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T05:11:03.767 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T05:11:03.767 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T05:11:03.768 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T05:11:03.768 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T05:11:03.768 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T05:11:03.768 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T05:11:03.768 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T05:11:03.768 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T05:11:03.768 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T05:11:03.769 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T05:11:03.769 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T05:11:03.769 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T05:11:03.769 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T05:11:03.769 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T05:11:03.769 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T05:11:03.769 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T05:11:03.769 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T05:11:03.770 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T05:11:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-17T05:11:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T05:11:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T05:11:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T05:11:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T05:11:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T05:11:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-17T05:11:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-17T05:11:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-17T05:11:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-17T05:11:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T05:11:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T05:11:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T05:11:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T05:11:03.773 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T05:11:03.774 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T05:11:03.775 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T05:11:03.776 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-17T05:11:03.777 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T05:11:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T05:11:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T05:11:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T05:11:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-17T05:11:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-17T05:11:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T05:11:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-17T05:11:03.778 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T05:11:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T05:11:03.780 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T05:11:03.780 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T05:11:03.780 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T05:11:03.780 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T05:11:03.780 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T05:11:03.780 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T05:11:03.780 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T05:11:03.780 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T05:11:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-17T05:11:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T05:11:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-17T05:11:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T05:11:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T05:11:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T05:11:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T05:11:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T05:11:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T05:11:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T05:11:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-17T05:11:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-17T05:11:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-17T05:11:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T05:11:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-17T05:11:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-17T05:11:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T05:11:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-17T05:11:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T05:11:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-17T05:11:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-17T05:11:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T05:11:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T05:11:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T05:11:03.784 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T05:11:03.785 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T05:11:03.785 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T05:11:03.785 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T05:11:03.785 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T05:11:03.785 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-17T05:11:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T05:11:03.787 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T05:11:03.787 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T05:11:03.787 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T05:11:03.787 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T05:11:03.787 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T05:11:03.787 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T05:11:03.787 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T05:11:03.787 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T05:11:03.788 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T05:11:03.788 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T05:11:03.788 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T05:11:03.788 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T05:11:03.788 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T05:11:03.788 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T05:11:03.788 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T05:11:03.788 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T05:11:03.789 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T05:11:03.790 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T05:11:03.790 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T05:11:03.790 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T05:11:03.790 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T05:11:03.790 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T05:11:03.790 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T05:11:03.790 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T05:11:03.790 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T05:11:03.791 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T05:11:03.791 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T05:11:03.791 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T05:11:03.791 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T05:11:03.791 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T05:11:03.791 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T05:11:03.791 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T05:11:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T05:11:03.793 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T05:11:03.794 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T05:11:03.795 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T05:11:03.796 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T05:11:03.797 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T05:11:03.798 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T05:11:03.799 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T05:11:03.799 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T05:11:03.799 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T05:11:03.799 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T05:11:03.799 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T05:11:03.799 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T05:11:03.799 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T05:11:03.800 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T05:11:03.800 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T05:11:03.800 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T05:11:03.800 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T05:11:03.800 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T05:11:03.800 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T05:11:03.800 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T05:11:03.801 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-17T05:11:03.802 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T05:11:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T05:11:03.804 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T05:11:03.805 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-17T05:11:03.806 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T05:11:03.807 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T05:11:03.808 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-17T05:11:03.809 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-17T05:11:03.814 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:03.814 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:03.815 DEBUG:teuthology.orchestra.run.smithi122:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-17T05:11:03.821 INFO:teuthology.orchestra.run.smithi122.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:11:03.821 INFO:teuthology.orchestra.run.smithi122.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:11:03.822 INFO:teuthology.orchestra.run.smithi122.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T05:11:03.822 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-17T05:11:03.822 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:03.822 DEBUG:teuthology.orchestra.run.smithi122:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-17T05:11:03.867 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi122.front.sepia.ceph.com to complete... 2024-09-17T05:11:04.232 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub' 2024-09-17T05:11:04.234 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T05:11:04.250 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-17T05:11:04.433 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:04.439 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:11:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T05:11:04.682 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T05:11:04.725 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T05:11:04.731 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T05:11:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T05:11:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T05:11:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T05:11:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-17T05:11:34.233 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-17T05:11:34.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T05:11:34.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:11:35.968 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-17T05:11:35.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-17T05:11:35.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:11:52.637 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-17T05:11:54.429 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-17T05:12:01.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T05:12:01.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:12:03.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-17T05:12:03.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:12:04.701 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-17T05:12:06.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-17T05:12:16.709 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T05:12:16.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:12:18.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-17T05:12:18.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:12:35.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-09-17T05:12:36.929 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-17T05:12:50.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T05:12:50.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:12:50.690 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-09-17T05:12:51.639 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T05:12:51.639 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T05:12:51.640 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-09-17T05:12:51.694 INFO:teuthology.orchestra.run.smithi173.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:12:51.694 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:12:51.694 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T05:12:51.694 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T05:12:51.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-17T05:12:51.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:12:52.221 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-17T05:12:52.697 DEBUG:teuthology.parallel:result is None 2024-09-17T05:12:53.512 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-17T05:12:53.512 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T05:12:53.512 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-17T05:12:53.566 INFO:teuthology.orchestra.run.smithi122.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:12:53.567 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T05:12:53.567 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T05:12:53.567 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T05:12:54.570 DEBUG:teuthology.parallel:result is None 2024-09-17T05:12:54.570 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T05:12:54.577 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T05:12:54.577 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T05:12:54.580 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T05:12:54.590 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T05:12:54.689 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T05:12:54.737 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T05:12:54.738 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T05:12:54.741 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T05:12:54.956 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T05:12:54.962 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T05:12:54.963 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T05:12:54.966 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T05:12:55.083 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:12:55.105 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:12:55.194 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:12:55.218 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T05:12:55.220 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T05:12:55.231 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T05:12:55.231 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T05:12:55.234 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T05:12:55.255 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T05:12:55.303 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T05:12:55.304 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T05:12:55.307 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T05:12:55.315 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T05:12:55.366 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T05:12:55.418 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:12:55.418 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T05:12:55.488 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T05:12:55.497 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T05:12:55.546 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:12:55.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T05:12:55.618 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-17T05:12:55.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-09-17T05:12:55.728 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T05:12:55.735 INFO:teuthology.task.internal:Starting timer... 2024-09-17T05:12:55.735 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T05:12:55.814 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T05:12:55.902 DEBUG:teuthology.task.selinux:Excluding smithi122: OS 'ubuntu' does not support SELinux 2024-09-17T05:12:55.902 DEBUG:teuthology.task.selinux:Excluding smithi173: OS 'ubuntu' does not support SELinux 2024-09-17T05:12:55.902 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T05:12:55.902 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-17T05:12:55.902 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T05:12:55.902 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T05:12:55.954 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T05:12:55.966 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T05:12:55.967 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi122.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T05:21:06.597 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-09-17T05:21:06.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-17T05:21:06.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:21:06.680 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-17T05:21:06.787 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-17T05:21:06.788 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T05:21:06.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T05:21:06.873 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-09-17T05:21:06.999 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-09-17T05:21:06.999 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T05:21:07.009 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T05:21:07.009 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T05:21:07.010 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:21:07.013 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T05:21:07.013 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:21:07.047 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T05:21:07.048 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Command line: ntpd -gq 2024-09-17T05:21:07.048 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: ---------------------------------------------------- 2024-09-17T05:21:07.048 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T05:21:07.048 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T05:21:07.048 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: corporation. Support and training for ntp-4 are 2024-09-17T05:21:07.048 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: available at https://www.nwtime.org/support 2024-09-17T05:21:07.048 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: ---------------------------------------------------- 2024-09-17T05:21:07.049 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: proto: precision = 0.042 usec (-24) 2024-09-17T05:21:07.049 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: basedate set to 2022-02-04 2024-09-17T05:21:07.049 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: gps base set to 2022-02-06 (week 2196) 2024-09-17T05:21:07.049 INFO:teuthology.orchestra.run.smithi122.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T05:21:07.049 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T05:21:07.050 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T05:21:07.050 INFO:teuthology.orchestra.run.smithi122.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T05:21:07.051 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T05:21:07.051 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T05:21:07.051 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T05:21:07.051 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Listen normally on 3 enp3s0f1 172.21.15.122:123 2024-09-17T05:21:07.051 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Listen normally on 4 lo [::1]:123 2024-09-17T05:21:07.051 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d515%5]:123 2024-09-17T05:21:07.051 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:07 ntpd[18463]: Listening on routing socket on fd #22 for interface updates 2024-09-17T05:21:07.078 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T05:21:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Command line: ntpd -gq 2024-09-17T05:21:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: ---------------------------------------------------- 2024-09-17T05:21:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T05:21:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T05:21:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: corporation. Support and training for ntp-4 are 2024-09-17T05:21:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: available at https://www.nwtime.org/support 2024-09-17T05:21:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: ---------------------------------------------------- 2024-09-17T05:21:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: proto: precision = 0.042 usec (-24) 2024-09-17T05:21:07.080 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: basedate set to 2022-02-04 2024-09-17T05:21:07.080 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: gps base set to 2022-02-06 (week 2196) 2024-09-17T05:21:07.080 INFO:teuthology.orchestra.run.smithi173.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T05:21:07.080 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T05:21:07.081 INFO:teuthology.orchestra.run.smithi173.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T05:21:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T05:21:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T05:21:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T05:21:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T05:21:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Listen normally on 3 enp3s0f1 172.21.15.173:123 2024-09-17T05:21:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Listen normally on 4 lo [::1]:123 2024-09-17T05:21:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7259%5]:123 2024-09-17T05:21:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:07 ntpd[18474]: Listening on routing socket on fd #22 for interface updates 2024-09-17T05:21:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:08 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:08 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:08 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:08 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:08 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:08 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:08.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:08 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:08.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:08 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:08.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:08 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:08.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:08 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:08.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:08 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:08.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:08 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:09 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:09 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:09 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:09 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:09 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:09 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:10.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:10 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:10.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:10 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:10.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:10 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:10.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:10 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:10.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:10 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:10.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:10 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:10.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:10 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:10.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:10 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:10.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:10.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:10.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:10.081 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:10 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:11 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:11 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:11 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:11 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:11 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:11 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:11.078 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:11 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:11.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:11 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:11.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:11 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:11.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:11 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:11.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:11 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:11.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:11 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:12.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:12.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:12.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:12.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:12 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:12.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:12.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:12 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:13.048 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:13 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:13.048 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:13 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:13.048 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:13 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:13.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:13 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:13.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:13 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:13.049 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:13 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:13.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:13 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:13.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:13 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:13.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:13 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:13.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:13 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:13.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:13 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:13.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:13 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:14.051 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:14.051 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:14.051 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:14.051 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:14.051 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:14.051 INFO:teuthology.orchestra.run.smithi122.stdout:17 Sep 05:21:14 ntpd[18463]: ntpd: time slew -0.002850 s 2024-09-17T05:21:14.052 INFO:teuthology.orchestra.run.smithi122.stdout:ntpd: time slew -0.002850s 2024-09-17T05:21:14.052 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T05:21:14.053 INFO:teuthology.orchestra.run.smithi122.stderr:17 Sep 05:21:14 ntpd[18463]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T05:21:14.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:14.079 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:14.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:14.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:14.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T05:21:14.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T05:21:14.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T05:21:14.080 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T05:21:14.081 INFO:teuthology.orchestra.run.smithi173.stdout:17 Sep 05:21:14 ntpd[18474]: ntpd: time slew -0.002994 s 2024-09-17T05:21:14.081 INFO:teuthology.orchestra.run.smithi173.stdout:ntpd: time slew -0.002994s 2024-09-17T05:21:14.081 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T05:21:14.081 INFO:teuthology.orchestra.run.smithi173.stderr:17 Sep 05:21:14 ntpd[18474]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T05:21:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:21:14.150 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-17T05:21:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:21:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:21:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:21:14.151 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:21:14.175 INFO:teuthology.orchestra.run.smithi173.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:21:14.175 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================== 2024-09-17T05:21:14.175 INFO:teuthology.orchestra.run.smithi173.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:21:14.175 INFO:teuthology.orchestra.run.smithi173.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:21:14.176 INFO:teuthology.orchestra.run.smithi173.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:21:14.176 INFO:teuthology.orchestra.run.smithi173.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T05:21:14.177 INFO:teuthology.run_tasks:Running task install... 2024-09-17T05:21:14.190 DEBUG:teuthology.task.install:project ceph 2024-09-17T05:21:14.190 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8'}} 2024-09-17T05:21:14.191 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8'} 2024-09-17T05:21:14.191 INFO:teuthology.task.install:Using flavor: default 2024-09-17T05:21:14.199 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T05:21:14.200 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-17T05:21:14.200 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-key list | grep Ceph 2024-09-17T05:21:14.201 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-key list | grep Ceph 2024-09-17T05:21:14.304 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T05:21:14.320 INFO:teuthology.orchestra.run.smithi173.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T05:21:14.355 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T05:21:14.355 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T05:21:14.356 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T05:21:14.356 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T05:21:14.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=820f760e4779dafd30d668ecea7aba6355dfd3d8 2024-09-17T05:21:14.371 INFO:teuthology.orchestra.run.smithi173.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T05:21:14.371 INFO:teuthology.orchestra.run.smithi173.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T05:21:14.373 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T05:21:14.373 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T05:21:14.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=820f760e4779dafd30d668ecea7aba6355dfd3d8 2024-09-17T05:21:14.499 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default/ 2024-09-17T05:21:14.500 INFO:teuthology.task.install.deb:Package version is 19.3.0-4963-g820f760e-1jammy 2024-09-17T05:21:14.533 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default/ 2024-09-17T05:21:14.533 INFO:teuthology.task.install.deb:Package version is 19.3.0-4963-g820f760e-1jammy 2024-09-17T05:21:14.610 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:21:14.610 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T05:21:14.630 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-17T05:21:14.644 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:21:14.645 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T05:21:14.664 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get update 2024-09-17T05:21:14.815 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T05:21:14.815 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T05:21:14.821 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T05:21:14.839 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T05:21:14.848 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T05:21:14.848 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T05:21:14.856 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T05:21:14.872 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T05:21:14.872 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T05:21:14.895 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T05:21:14.898 INFO:teuthology.orchestra.run.smithi173.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T05:21:14.917 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T05:21:14.919 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T05:21:14.947 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-17T05:21:14.947 INFO:teuthology.orchestra.run.smithi173.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T05:21:14.969 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-17T05:21:14.980 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-17T05:21:15.005 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-17T05:21:15.412 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 29.3 kB in 1s (52.9 kB/s) 2024-09-17T05:21:15.453 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 29.3 kB in 1s (52.1 kB/s) 2024-09-17T05:21:16.465 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-17T05:21:16.480 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4963-g820f760e-1jammy cephadm=19.3.0-4963-g820f760e-1jammy ceph-mds=19.3.0-4963-g820f760e-1jammy ceph-mgr=19.3.0-4963-g820f760e-1jammy ceph-common=19.3.0-4963-g820f760e-1jammy ceph-fuse=19.3.0-4963-g820f760e-1jammy ceph-test=19.3.0-4963-g820f760e-1jammy ceph-volume=19.3.0-4963-g820f760e-1jammy radosgw=19.3.0-4963-g820f760e-1jammy python3-rados=19.3.0-4963-g820f760e-1jammy python3-rgw=19.3.0-4963-g820f760e-1jammy python3-cephfs=19.3.0-4963-g820f760e-1jammy python3-rbd=19.3.0-4963-g820f760e-1jammy libcephfs2=19.3.0-4963-g820f760e-1jammy libcephfs-dev=19.3.0-4963-g820f760e-1jammy librados2=19.3.0-4963-g820f760e-1jammy librbd1=19.3.0-4963-g820f760e-1jammy rbd-fuse=19.3.0-4963-g820f760e-1jammy python3-cephfs=19.3.0-4963-g820f760e-1jammy cephfs-shell=19.3.0-4963-g820f760e-1jammy cephfs-top=19.3.0-4963-g820f760e-1jammy cephfs-mirror=19.3.0-4963-g820f760e-1jammy 2024-09-17T05:21:16.494 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-09-17T05:21:16.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4963-g820f760e-1jammy cephadm=19.3.0-4963-g820f760e-1jammy ceph-mds=19.3.0-4963-g820f760e-1jammy ceph-mgr=19.3.0-4963-g820f760e-1jammy ceph-common=19.3.0-4963-g820f760e-1jammy ceph-fuse=19.3.0-4963-g820f760e-1jammy ceph-test=19.3.0-4963-g820f760e-1jammy ceph-volume=19.3.0-4963-g820f760e-1jammy radosgw=19.3.0-4963-g820f760e-1jammy python3-rados=19.3.0-4963-g820f760e-1jammy python3-rgw=19.3.0-4963-g820f760e-1jammy python3-cephfs=19.3.0-4963-g820f760e-1jammy python3-rbd=19.3.0-4963-g820f760e-1jammy libcephfs2=19.3.0-4963-g820f760e-1jammy libcephfs-dev=19.3.0-4963-g820f760e-1jammy librados2=19.3.0-4963-g820f760e-1jammy librbd1=19.3.0-4963-g820f760e-1jammy rbd-fuse=19.3.0-4963-g820f760e-1jammy python3-cephfs=19.3.0-4963-g820f760e-1jammy cephfs-shell=19.3.0-4963-g820f760e-1jammy cephfs-top=19.3.0-4963-g820f760e-1jammy cephfs-mirror=19.3.0-4963-g820f760e-1jammy 2024-09-17T05:21:16.557 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-17T05:21:16.586 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-09-17T05:21:16.746 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-17T05:21:16.747 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-17T05:21:16.769 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-09-17T05:21:16.769 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-09-17T05:21:16.941 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T05:21:16.942 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T05:21:16.942 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T05:21:16.942 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T05:21:16.942 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T05:21:16.942 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T05:21:16.942 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T05:21:16.943 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-17T05:21:16.943 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T05:21:16.944 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T05:21:16.945 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T05:21:16.945 INFO:teuthology.orchestra.run.smithi122.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T05:21:16.945 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T05:21:16.945 INFO:teuthology.orchestra.run.smithi122.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T05:21:16.946 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T05:21:16.947 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T05:21:16.947 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T05:21:16.947 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T05:21:16.947 INFO:teuthology.orchestra.run.smithi122.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T05:21:16.947 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-17T05:21:16.947 INFO:teuthology.orchestra.run.smithi122.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T05:21:16.947 INFO:teuthology.orchestra.run.smithi122.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T05:21:16.947 INFO:teuthology.orchestra.run.smithi122.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T05:21:16.948 INFO:teuthology.orchestra.run.smithi122.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T05:21:16.948 INFO:teuthology.orchestra.run.smithi122.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T05:21:16.948 INFO:teuthology.orchestra.run.smithi122.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T05:21:16.948 INFO:teuthology.orchestra.run.smithi122.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T05:21:16.948 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-17T05:21:16.948 INFO:teuthology.orchestra.run.smithi122.stdout: btrfs-tools 2024-09-17T05:21:16.956 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T05:21:16.956 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T05:21:16.956 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T05:21:16.957 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T05:21:16.957 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T05:21:16.957 INFO:teuthology.orchestra.run.smithi173.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T05:21:16.957 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T05:21:16.958 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2024-09-17T05:21:16.958 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T05:21:16.960 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T05:21:16.960 INFO:teuthology.orchestra.run.smithi173.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T05:21:16.960 INFO:teuthology.orchestra.run.smithi173.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T05:21:16.960 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T05:21:16.960 INFO:teuthology.orchestra.run.smithi173.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T05:21:16.960 INFO:teuthology.orchestra.run.smithi173.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T05:21:16.960 INFO:teuthology.orchestra.run.smithi173.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T05:21:16.961 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T05:21:16.962 INFO:teuthology.orchestra.run.smithi173.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T05:21:16.963 INFO:teuthology.orchestra.run.smithi173.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T05:21:16.963 INFO:teuthology.orchestra.run.smithi173.stdout:Recommended packages: 2024-09-17T05:21:16.963 INFO:teuthology.orchestra.run.smithi173.stdout: btrfs-tools 2024-09-17T05:21:17.099 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-17T05:21:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T05:21:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T05:21:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T05:21:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T05:21:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T05:21:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T05:21:17.100 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T05:21:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T05:21:17.101 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T05:21:17.102 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T05:21:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T05:21:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T05:21:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T05:21:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T05:21:17.103 INFO:teuthology.orchestra.run.smithi122.stdout: socat unzip xmlstarlet zip 2024-09-17T05:21:17.118 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2024-09-17T05:21:17.119 INFO:teuthology.orchestra.run.smithi173.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T05:21:17.119 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T05:21:17.119 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T05:21:17.119 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T05:21:17.120 INFO:teuthology.orchestra.run.smithi173.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T05:21:17.121 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T05:21:17.122 INFO:teuthology.orchestra.run.smithi173.stdout: socat unzip xmlstarlet zip 2024-09-17T05:21:17.169 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-17T05:21:17.169 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 244 MB of archives. 2024-09-17T05:21:17.169 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 910 MB of additional disk space will be used. 2024-09-17T05:21:17.169 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T05:21:17.187 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T05:21:17.189 INFO:teuthology.orchestra.run.smithi173.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-17T05:21:17.189 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 244 MB of archives. 2024-09-17T05:21:17.189 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 910 MB of additional disk space will be used. 2024-09-17T05:21:17.189 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T05:21:17.197 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T05:21:17.206 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T05:21:17.217 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T05:21:17.238 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T05:21:17.257 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T05:21:17.259 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T05:21:17.262 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T05:21:17.268 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4963-g820f760e-1jammy [3,634 kB] 2024-09-17T05:21:17.274 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T05:21:17.277 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T05:21:17.279 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4963-g820f760e-1jammy [3,634 kB] 2024-09-17T05:21:17.281 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T05:21:17.291 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T05:21:17.313 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T05:21:17.316 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T05:21:17.322 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T05:21:17.324 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T05:21:17.325 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T05:21:17.326 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T05:21:17.330 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T05:21:17.332 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T05:21:17.332 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T05:21:17.333 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T05:21:17.335 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T05:21:17.336 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T05:21:17.339 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T05:21:17.342 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T05:21:17.343 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T05:21:17.343 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T05:21:17.349 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T05:21:17.349 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T05:21:17.350 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T05:21:17.351 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T05:21:17.352 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T05:21:17.353 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T05:21:17.354 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T05:21:17.355 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T05:21:17.355 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T05:21:17.366 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T05:21:17.367 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T05:21:17.367 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T05:21:17.368 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T05:21:17.368 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T05:21:17.368 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T05:21:17.369 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T05:21:17.370 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T05:21:17.371 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T05:21:17.372 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T05:21:17.382 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T05:21:17.384 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T05:21:17.384 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T05:21:17.384 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T05:21:17.385 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T05:21:17.386 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T05:21:17.387 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T05:21:17.387 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T05:21:17.389 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T05:21:17.390 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T05:21:17.400 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T05:21:17.400 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T05:21:17.401 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T05:21:17.403 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T05:21:17.403 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T05:21:17.404 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T05:21:17.405 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T05:21:17.405 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T05:21:17.406 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T05:21:17.406 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T05:21:17.417 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T05:21:17.417 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T05:21:17.418 INFO:teuthology.orchestra.run.smithi173.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T05:21:17.418 INFO:teuthology.orchestra.run.smithi173.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T05:21:17.419 INFO:teuthology.orchestra.run.smithi173.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T05:21:17.422 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T05:21:17.422 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T05:21:17.423 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T05:21:17.428 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T05:21:17.428 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T05:21:17.433 INFO:teuthology.orchestra.run.smithi173.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T05:21:17.434 INFO:teuthology.orchestra.run.smithi173.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T05:21:17.435 INFO:teuthology.orchestra.run.smithi173.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T05:21:17.439 INFO:teuthology.orchestra.run.smithi173.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T05:21:17.439 INFO:teuthology.orchestra.run.smithi173.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T05:21:17.440 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T05:21:17.440 INFO:teuthology.orchestra.run.smithi122.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T05:21:17.441 INFO:teuthology.orchestra.run.smithi122.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T05:21:17.443 INFO:teuthology.orchestra.run.smithi122.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T05:21:17.444 INFO:teuthology.orchestra.run.smithi122.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T05:21:17.450 INFO:teuthology.orchestra.run.smithi173.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T05:21:17.451 INFO:teuthology.orchestra.run.smithi173.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T05:21:17.452 INFO:teuthology.orchestra.run.smithi173.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T05:21:17.453 INFO:teuthology.orchestra.run.smithi173.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T05:21:17.454 INFO:teuthology.orchestra.run.smithi173.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T05:21:17.457 INFO:teuthology.orchestra.run.smithi122.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T05:21:17.457 INFO:teuthology.orchestra.run.smithi122.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T05:21:17.459 INFO:teuthology.orchestra.run.smithi122.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T05:21:17.460 INFO:teuthology.orchestra.run.smithi122.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T05:21:17.461 INFO:teuthology.orchestra.run.smithi122.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T05:21:17.467 INFO:teuthology.orchestra.run.smithi173.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T05:21:17.468 INFO:teuthology.orchestra.run.smithi173.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T05:21:17.470 INFO:teuthology.orchestra.run.smithi173.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T05:21:17.470 INFO:teuthology.orchestra.run.smithi173.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T05:21:17.471 INFO:teuthology.orchestra.run.smithi173.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T05:21:17.474 INFO:teuthology.orchestra.run.smithi122.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T05:21:17.484 INFO:teuthology.orchestra.run.smithi122.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4963-g820f760e-1jammy [3,334 kB] 2024-09-17T05:21:17.485 INFO:teuthology.orchestra.run.smithi173.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T05:21:17.520 INFO:teuthology.orchestra.run.smithi122.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T05:21:17.520 INFO:teuthology.orchestra.run.smithi122.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T05:21:17.522 INFO:teuthology.orchestra.run.smithi122.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T05:21:17.522 INFO:teuthology.orchestra.run.smithi122.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T05:21:17.524 INFO:teuthology.orchestra.run.smithi122.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T05:21:17.528 INFO:teuthology.orchestra.run.smithi173.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T05:21:17.529 INFO:teuthology.orchestra.run.smithi173.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T05:21:17.530 INFO:teuthology.orchestra.run.smithi173.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T05:21:17.531 INFO:teuthology.orchestra.run.smithi173.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T05:21:17.532 INFO:teuthology.orchestra.run.smithi173.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T05:21:17.542 INFO:teuthology.orchestra.run.smithi122.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T05:21:17.544 INFO:teuthology.orchestra.run.smithi122.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T05:21:17.545 INFO:teuthology.orchestra.run.smithi122.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T05:21:17.550 INFO:teuthology.orchestra.run.smithi173.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T05:21:17.552 INFO:teuthology.orchestra.run.smithi173.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T05:21:17.553 INFO:teuthology.orchestra.run.smithi173.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T05:21:17.561 INFO:teuthology.orchestra.run.smithi122.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T05:21:17.561 INFO:teuthology.orchestra.run.smithi122.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T05:21:17.562 INFO:teuthology.orchestra.run.smithi122.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T05:21:17.562 INFO:teuthology.orchestra.run.smithi122.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T05:21:17.563 INFO:teuthology.orchestra.run.smithi122.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T05:21:17.564 INFO:teuthology.orchestra.run.smithi122.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T05:21:17.565 INFO:teuthology.orchestra.run.smithi122.stdout:Get:70 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4963-g820f760e-1jammy [722 kB] 2024-09-17T05:21:17.568 INFO:teuthology.orchestra.run.smithi173.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T05:21:17.569 INFO:teuthology.orchestra.run.smithi173.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T05:21:17.569 INFO:teuthology.orchestra.run.smithi173.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T05:21:17.570 INFO:teuthology.orchestra.run.smithi173.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T05:21:17.571 INFO:teuthology.orchestra.run.smithi173.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T05:21:17.571 INFO:teuthology.orchestra.run.smithi173.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T05:21:17.574 INFO:teuthology.orchestra.run.smithi122.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T05:21:17.575 INFO:teuthology.orchestra.run.smithi122.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T05:21:17.578 INFO:teuthology.orchestra.run.smithi122.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T05:21:17.581 INFO:teuthology.orchestra.run.smithi173.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T05:21:17.582 INFO:teuthology.orchestra.run.smithi173.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T05:21:17.586 INFO:teuthology.orchestra.run.smithi173.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T05:21:17.588 INFO:teuthology.orchestra.run.smithi122.stdout:Get:74 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4963-g820f760e-1jammy [357 kB] 2024-09-17T05:21:17.592 INFO:teuthology.orchestra.run.smithi122.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T05:21:17.593 INFO:teuthology.orchestra.run.smithi122.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T05:21:17.594 INFO:teuthology.orchestra.run.smithi122.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T05:21:17.597 INFO:teuthology.orchestra.run.smithi122.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T05:21:17.599 INFO:teuthology.orchestra.run.smithi173.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T05:21:17.600 INFO:teuthology.orchestra.run.smithi122.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T05:21:17.601 INFO:teuthology.orchestra.run.smithi122.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T05:21:17.601 INFO:teuthology.orchestra.run.smithi173.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T05:21:17.601 INFO:teuthology.orchestra.run.smithi173.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T05:21:17.601 INFO:teuthology.orchestra.run.smithi122.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T05:21:17.604 INFO:teuthology.orchestra.run.smithi173.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T05:21:17.607 INFO:teuthology.orchestra.run.smithi173.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T05:21:17.607 INFO:teuthology.orchestra.run.smithi173.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T05:21:17.608 INFO:teuthology.orchestra.run.smithi173.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T05:21:17.609 INFO:teuthology.orchestra.run.smithi122.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T05:21:17.610 INFO:teuthology.orchestra.run.smithi122.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4963-g820f760e-1jammy [32.8 kB] 2024-09-17T05:21:17.613 INFO:teuthology.orchestra.run.smithi122.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T05:21:17.614 INFO:teuthology.orchestra.run.smithi122.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T05:21:17.616 INFO:teuthology.orchestra.run.smithi173.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T05:21:17.620 INFO:teuthology.orchestra.run.smithi173.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T05:21:17.621 INFO:teuthology.orchestra.run.smithi173.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T05:21:17.621 INFO:teuthology.orchestra.run.smithi122.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4963-g820f760e-1jammy [180 kB] 2024-09-17T05:21:17.626 INFO:teuthology.orchestra.run.smithi122.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T05:21:17.627 INFO:teuthology.orchestra.run.smithi122.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T05:21:17.627 INFO:teuthology.orchestra.run.smithi122.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T05:21:17.628 INFO:teuthology.orchestra.run.smithi122.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T05:21:17.629 INFO:teuthology.orchestra.run.smithi122.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T05:21:17.630 INFO:teuthology.orchestra.run.smithi122.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T05:21:17.633 INFO:teuthology.orchestra.run.smithi173.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T05:21:17.636 INFO:teuthology.orchestra.run.smithi173.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T05:21:17.637 INFO:teuthology.orchestra.run.smithi173.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T05:21:17.637 INFO:teuthology.orchestra.run.smithi173.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T05:21:17.637 INFO:teuthology.orchestra.run.smithi173.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T05:21:17.637 INFO:teuthology.orchestra.run.smithi122.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T05:21:17.637 INFO:teuthology.orchestra.run.smithi173.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T05:21:17.643 INFO:teuthology.orchestra.run.smithi173.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T05:21:17.644 INFO:teuthology.orchestra.run.smithi122.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T05:21:17.644 INFO:teuthology.orchestra.run.smithi122.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T05:21:17.646 INFO:teuthology.orchestra.run.smithi122.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T05:21:17.650 INFO:teuthology.orchestra.run.smithi173.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T05:21:17.651 INFO:teuthology.orchestra.run.smithi173.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T05:21:17.652 INFO:teuthology.orchestra.run.smithi122.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4963-g820f760e-1jammy [72.8 kB] 2024-09-17T05:21:17.652 INFO:teuthology.orchestra.run.smithi173.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T05:21:17.655 INFO:teuthology.orchestra.run.smithi122.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4963-g820f760e-1jammy [338 kB] 2024-09-17T05:21:17.661 INFO:teuthology.orchestra.run.smithi122.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T05:21:17.667 INFO:teuthology.orchestra.run.smithi173.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T05:21:17.685 INFO:teuthology.orchestra.run.smithi122.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4963-g820f760e-1jammy [7,966 kB] 2024-09-17T05:21:17.719 INFO:teuthology.orchestra.run.smithi173.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4963-g820f760e-1jammy [3,334 kB] 2024-09-17T05:21:17.933 INFO:teuthology.orchestra.run.smithi122.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4963-g820f760e-1jammy [112 kB] 2024-09-17T05:21:17.976 INFO:teuthology.orchestra.run.smithi122.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4963-g820f760e-1jammy [473 kB] 2024-09-17T05:21:18.027 INFO:teuthology.orchestra.run.smithi122.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4963-g820f760e-1jammy [28.2 MB] 2024-09-17T05:21:18.837 INFO:teuthology.orchestra.run.smithi122.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4963-g820f760e-1jammy [5,206 kB] 2024-09-17T05:21:19.017 INFO:teuthology.orchestra.run.smithi122.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4963-g820f760e-1jammy [252 kB] 2024-09-17T05:21:19.036 INFO:teuthology.orchestra.run.smithi122.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4963-g820f760e-1jammy [124 kB] 2024-09-17T05:21:19.054 INFO:teuthology.orchestra.run.smithi122.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4963-g820f760e-1jammy [1,562 kB] 2024-09-17T05:21:19.072 INFO:teuthology.orchestra.run.smithi122.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4963-g820f760e-1jammy [6,309 kB] 2024-09-17T05:21:19.348 INFO:teuthology.orchestra.run.smithi122.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4963-g820f760e-1jammy [23.1 MB] 2024-09-17T05:21:20.152 INFO:teuthology.orchestra.run.smithi122.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4963-g820f760e-1jammy [14.0 kB] 2024-09-17T05:21:20.158 INFO:teuthology.orchestra.run.smithi122.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4963-g820f760e-1jammy [880 kB] 2024-09-17T05:21:20.291 INFO:teuthology.orchestra.run.smithi122.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4963-g820f760e-1jammy [2,454 kB] 2024-09-17T05:21:20.415 INFO:teuthology.orchestra.run.smithi122.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4963-g820f760e-1jammy [813 kB] 2024-09-17T05:21:20.454 INFO:teuthology.orchestra.run.smithi122.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4963-g820f760e-1jammy [160 kB] 2024-09-17T05:21:20.724 INFO:teuthology.orchestra.run.smithi122.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4963-g820f760e-1jammy [61.5 MB] 2024-09-17T05:21:22.840 INFO:teuthology.orchestra.run.smithi122.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4963-g820f760e-1jammy [8,625 kB] 2024-09-17T05:21:23.194 INFO:teuthology.orchestra.run.smithi122.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4963-g820f760e-1jammy [14.2 kB] 2024-09-17T05:21:23.195 INFO:teuthology.orchestra.run.smithi173.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4963-g820f760e-1jammy [722 kB] 2024-09-17T05:21:23.223 INFO:teuthology.orchestra.run.smithi122.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4963-g820f760e-1jammy [48.5 MB] 2024-09-17T05:21:24.679 INFO:teuthology.orchestra.run.smithi173.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4963-g820f760e-1jammy [357 kB] 2024-09-17T05:21:24.701 INFO:teuthology.orchestra.run.smithi122.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4963-g820f760e-1jammy [130 kB] 2024-09-17T05:21:24.722 INFO:teuthology.orchestra.run.smithi122.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4963-g820f760e-1jammy [233 kB] 2024-09-17T05:21:24.743 INFO:teuthology.orchestra.run.smithi122.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4963-g820f760e-1jammy [33.9 kB] 2024-09-17T05:21:24.762 INFO:teuthology.orchestra.run.smithi173.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4963-g820f760e-1jammy [32.8 kB] 2024-09-17T05:21:24.763 INFO:teuthology.orchestra.run.smithi173.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4963-g820f760e-1jammy [180 kB] 2024-09-17T05:21:24.765 INFO:teuthology.orchestra.run.smithi173.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4963-g820f760e-1jammy [72.8 kB] 2024-09-17T05:21:24.860 INFO:teuthology.orchestra.run.smithi122.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4963-g820f760e-1jammy [23.4 kB] 2024-09-17T05:21:24.861 INFO:teuthology.orchestra.run.smithi173.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4963-g820f760e-1jammy [338 kB] 2024-09-17T05:21:24.871 INFO:teuthology.orchestra.run.smithi122.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4963-g820f760e-1jammy [39.0 kB] 2024-09-17T05:21:24.872 INFO:teuthology.orchestra.run.smithi173.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4963-g820f760e-1jammy [7,966 kB] 2024-09-17T05:21:24.904 INFO:teuthology.orchestra.run.smithi122.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4963-g820f760e-1jammy [15.8 MB] 2024-09-17T05:21:25.387 INFO:teuthology.orchestra.run.smithi122.stdout:Get:125 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4963-g820f760e-1jammy [91.2 kB] 2024-09-17T05:21:25.464 INFO:teuthology.orchestra.run.smithi173.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4963-g820f760e-1jammy [112 kB] 2024-09-17T05:21:25.465 INFO:teuthology.orchestra.run.smithi173.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4963-g820f760e-1jammy [473 kB] 2024-09-17T05:21:25.471 INFO:teuthology.orchestra.run.smithi173.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4963-g820f760e-1jammy [28.2 MB] 2024-09-17T05:21:25.943 INFO:teuthology.orchestra.run.smithi173.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4963-g820f760e-1jammy [5,206 kB] 2024-09-17T05:21:26.020 INFO:teuthology.orchestra.run.smithi173.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4963-g820f760e-1jammy [252 kB] 2024-09-17T05:21:26.023 INFO:teuthology.orchestra.run.smithi173.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4963-g820f760e-1jammy [124 kB] 2024-09-17T05:21:26.025 INFO:teuthology.orchestra.run.smithi173.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4963-g820f760e-1jammy [1,562 kB] 2024-09-17T05:21:26.044 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 244 MB in 8s (29.5 MB/s) 2024-09-17T05:21:26.051 INFO:teuthology.orchestra.run.smithi173.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4963-g820f760e-1jammy [6,309 kB] 2024-09-17T05:21:26.155 INFO:teuthology.orchestra.run.smithi173.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4963-g820f760e-1jammy [23.1 MB] 2024-09-17T05:21:26.209 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T05:21:26.272 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-17T05:21:26.276 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T05:21:26.316 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T05:21:26.541 INFO:teuthology.orchestra.run.smithi173.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4963-g820f760e-1jammy [14.0 kB] 2024-09-17T05:21:26.541 INFO:teuthology.orchestra.run.smithi173.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4963-g820f760e-1jammy [880 kB] 2024-09-17T05:21:26.549 INFO:teuthology.orchestra.run.smithi173.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4963-g820f760e-1jammy [2,454 kB] 2024-09-17T05:21:26.589 INFO:teuthology.orchestra.run.smithi173.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4963-g820f760e-1jammy [813 kB] 2024-09-17T05:21:26.601 INFO:teuthology.orchestra.run.smithi173.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4963-g820f760e-1jammy [160 kB] 2024-09-17T05:21:26.605 INFO:teuthology.orchestra.run.smithi173.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4963-g820f760e-1jammy [61.5 MB] 2024-09-17T05:21:26.636 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T05:21:26.650 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T05:21:26.694 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T05:21:26.989 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T05:21:27.003 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T05:21:27.039 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T05:21:27.393 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T05:21:27.406 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T05:21:27.493 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T05:21:27.632 INFO:teuthology.orchestra.run.smithi173.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4963-g820f760e-1jammy [8,625 kB] 2024-09-17T05:21:27.757 INFO:teuthology.orchestra.run.smithi173.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4963-g820f760e-1jammy [14.2 kB] 2024-09-17T05:21:27.757 INFO:teuthology.orchestra.run.smithi173.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4963-g820f760e-1jammy [48.5 MB] 2024-09-17T05:21:27.830 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T05:21:27.843 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T05:21:27.922 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T05:21:28.225 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T05:21:28.239 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T05:21:28.284 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T05:21:28.565 INFO:teuthology.orchestra.run.smithi173.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4963-g820f760e-1jammy [130 kB] 2024-09-17T05:21:28.566 INFO:teuthology.orchestra.run.smithi173.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4963-g820f760e-1jammy [233 kB] 2024-09-17T05:21:28.568 INFO:teuthology.orchestra.run.smithi173.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4963-g820f760e-1jammy [33.9 kB] 2024-09-17T05:21:28.568 INFO:teuthology.orchestra.run.smithi173.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4963-g820f760e-1jammy [23.4 kB] 2024-09-17T05:21:28.569 INFO:teuthology.orchestra.run.smithi173.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4963-g820f760e-1jammy [39.0 kB] 2024-09-17T05:21:28.569 INFO:teuthology.orchestra.run.smithi173.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4963-g820f760e-1jammy [15.8 MB] 2024-09-17T05:21:28.721 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T05:21:28.735 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:21:28.795 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:28.821 INFO:teuthology.orchestra.run.smithi173.stdout:Get:125 https://4.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-2351/820f760e4779dafd30d668ecea7aba6355dfd3d8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4963-g820f760e-1jammy [91.2 kB] 2024-09-17T05:21:29.241 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T05:21:29.256 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:21:29.299 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:29.475 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 244 MB in 12s (20.9 MB/s) 2024-09-17T05:21:29.578 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T05:21:29.592 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:21:29.618 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T05:21:29.659 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:29.685 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-17T05:21:29.688 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T05:21:29.992 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T05:21:30.006 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T05:21:30.021 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T05:21:30.065 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T05:21:30.236 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T05:21:30.250 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T05:21:30.286 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T05:21:30.418 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librados2. 2024-09-17T05:21:30.432 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../010-librados2_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:30.476 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librados2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:30.581 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T05:21:30.595 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T05:21:30.639 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T05:21:30.913 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libnbd0. 2024-09-17T05:21:30.927 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T05:21:30.962 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T05:21:31.017 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T05:21:31.031 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T05:21:31.117 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T05:21:31.274 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librbd1. 2024-09-17T05:21:31.289 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../012-librbd1_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:31.332 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librbd1 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:31.454 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T05:21:31.469 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T05:21:31.512 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T05:21:31.786 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T05:21:31.800 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:31.844 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:31.857 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T05:21:31.872 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T05:21:31.915 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T05:21:32.105 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rados. 2024-09-17T05:21:32.119 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:32.155 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rados (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:32.327 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T05:21:32.342 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:21:32.385 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:32.475 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T05:21:32.489 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:21:32.566 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-argparse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:32.788 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T05:21:32.803 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:21:32.819 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T05:21:32.834 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:32.846 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:32.877 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cephfs (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:33.166 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T05:21:33.181 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T05:21:33.214 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T05:21:33.228 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:21:33.249 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:21:33.281 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-common (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:33.536 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T05:21:33.551 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T05:21:33.586 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T05:21:33.621 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T05:21:33.632 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T05:21:33.701 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T05:21:33.888 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librados2. 2024-09-17T05:21:33.903 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../010-librados2_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:33.937 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T05:21:33.946 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librados2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:33.951 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:33.987 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rbd (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:34.374 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T05:21:34.388 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T05:21:34.399 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libnbd0. 2024-09-17T05:21:34.414 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T05:21:34.432 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T05:21:34.482 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T05:21:34.769 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librbd1. 2024-09-17T05:21:34.784 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../012-librbd1_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:34.819 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librbd1 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:35.016 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T05:21:35.032 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T05:21:35.226 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T05:21:35.429 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T05:21:35.444 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:35.488 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcephfs2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:35.555 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T05:21:35.570 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T05:21:35.613 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T05:21:35.799 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rados. 2024-09-17T05:21:35.814 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:35.857 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rados (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:35.936 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua5.1. 2024-09-17T05:21:35.950 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T05:21:35.994 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T05:21:36.168 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T05:21:36.183 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:21:36.226 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-ceph-argparse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:36.314 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-any. 2024-09-17T05:21:36.328 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T05:21:36.364 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T05:21:36.478 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T05:21:36.493 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:36.537 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cephfs (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:36.600 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package zip. 2024-09-17T05:21:36.614 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T05:21:36.700 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T05:21:36.806 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T05:21:36.821 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:21:36.856 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-ceph-common (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:37.012 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package unzip. 2024-09-17T05:21:37.027 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T05:21:37.070 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T05:21:37.201 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T05:21:37.216 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T05:21:37.259 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T05:21:37.474 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package luarocks. 2024-09-17T05:21:37.489 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T05:21:37.503 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T05:21:37.518 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:37.532 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T05:21:37.562 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rbd (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:37.847 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T05:21:37.863 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T05:21:37.925 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T05:21:37.950 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librgw2. 2024-09-17T05:21:37.964 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../028-librgw2_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:38.008 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librgw2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:38.209 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T05:21:38.224 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T05:21:38.267 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T05:21:38.553 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T05:21:38.568 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:38.569 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T05:21:38.584 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T05:21:38.611 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rgw (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:38.628 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T05:21:38.922 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T05:21:38.937 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T05:21:38.956 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua5.1. 2024-09-17T05:21:38.971 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T05:21:38.997 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T05:21:39.023 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T05:21:39.301 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T05:21:39.316 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:39.334 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-any. 2024-09-17T05:21:39.349 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T05:21:39.360 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libradosstriper1 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:39.392 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T05:21:39.678 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package zip. 2024-09-17T05:21:39.693 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T05:21:39.736 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T05:21:39.813 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-common. 2024-09-17T05:21:39.828 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:39.871 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-common (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:40.048 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package unzip. 2024-09-17T05:21:40.063 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T05:21:40.106 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T05:21:40.610 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package luarocks. 2024-09-17T05:21:40.625 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T05:21:40.660 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T05:21:41.019 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-base. 2024-09-17T05:21:41.034 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:41.076 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librgw2. 2024-09-17T05:21:41.086 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-base (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:41.090 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../028-librgw2_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:41.133 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librgw2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:41.632 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T05:21:41.645 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T05:21:41.653 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T05:21:41.668 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:41.690 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T05:21:41.702 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rgw (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:42.035 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T05:21:42.049 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T05:21:42.055 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T05:21:42.071 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T05:21:42.093 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T05:21:42.113 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T05:21:42.425 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T05:21:42.438 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T05:21:42.440 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:42.453 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T05:21:42.483 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libradosstriper1 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:42.496 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T05:21:42.775 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T05:21:42.788 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T05:21:42.824 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T05:21:42.903 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-common. 2024-09-17T05:21:42.918 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:42.961 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-common (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:43.153 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T05:21:43.167 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T05:21:43.211 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T05:21:43.514 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T05:21:43.529 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T05:21:43.597 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T05:21:43.900 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-portend. 2024-09-17T05:21:43.915 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T05:21:43.959 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T05:21:44.092 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-base. 2024-09-17T05:21:44.107 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:44.159 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-base (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:44.237 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T05:21:44.252 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T05:21:44.287 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T05:21:44.590 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T05:21:44.605 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T05:21:44.649 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T05:21:44.712 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T05:21:44.726 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T05:21:44.762 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T05:21:45.003 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T05:21:45.017 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T05:21:45.061 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T05:21:45.073 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T05:21:45.089 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T05:21:45.132 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T05:21:45.398 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T05:21:45.412 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T05:21:45.451 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T05:21:45.456 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T05:21:45.466 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T05:21:45.510 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T05:21:45.742 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T05:21:45.757 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T05:21:45.801 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T05:21:45.996 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T05:21:46.011 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T05:21:46.054 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T05:21:46.129 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-mako. 2024-09-17T05:21:46.144 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T05:21:46.179 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T05:21:46.351 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T05:21:46.366 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T05:21:46.409 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T05:21:46.516 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T05:21:46.531 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T05:21:46.574 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T05:21:46.711 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T05:21:46.727 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T05:21:46.770 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T05:21:46.886 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T05:21:46.900 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T05:21:46.944 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T05:21:47.081 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-portend. 2024-09-17T05:21:47.096 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T05:21:47.139 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T05:21:47.272 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webob. 2024-09-17T05:21:47.287 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T05:21:47.322 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T05:21:47.400 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T05:21:47.415 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T05:21:47.451 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T05:21:47.676 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T05:21:47.690 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T05:21:47.728 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T05:21:47.736 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T05:21:47.752 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T05:21:47.795 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T05:21:48.046 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T05:21:48.058 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T05:21:48.096 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T05:21:48.148 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T05:21:48.163 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T05:21:48.223 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T05:21:48.432 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-paste. 2024-09-17T05:21:48.444 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T05:21:48.482 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T05:21:48.492 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T05:21:48.508 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T05:21:48.543 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T05:21:48.811 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T05:21:48.823 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T05:21:48.828 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T05:21:48.844 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T05:21:48.861 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T05:21:48.887 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T05:21:49.164 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T05:21:49.173 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-mako. 2024-09-17T05:21:49.176 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T05:21:49.188 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T05:21:49.214 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T05:21:49.231 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T05:21:49.550 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T05:21:49.565 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T05:21:49.576 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T05:21:49.591 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T05:21:49.600 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T05:21:49.634 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T05:21:49.862 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T05:21:49.875 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T05:21:49.904 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T05:21:49.918 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T05:21:49.945 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T05:21:49.954 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T05:21:50.981 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webob. 2024-09-17T05:21:50.997 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T05:21:51.040 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T05:21:51.151 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T05:21:51.164 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T05:21:51.209 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T05:21:51.384 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T05:21:51.399 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T05:21:51.445 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T05:21:51.554 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-idna. 2024-09-17T05:21:51.568 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T05:21:51.612 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T05:21:51.737 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T05:21:51.752 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T05:21:51.795 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T05:21:51.924 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T05:21:51.937 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T05:21:51.982 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T05:21:52.098 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-paste. 2024-09-17T05:21:52.113 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T05:21:52.148 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T05:21:52.311 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests. 2024-09-17T05:21:52.324 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T05:21:52.361 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T05:21:52.476 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T05:21:52.491 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T05:21:52.526 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T05:21:52.697 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T05:21:52.711 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T05:21:52.756 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T05:21:52.795 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T05:21:52.812 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T05:21:52.853 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T05:21:53.042 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T05:21:53.056 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:21:53.101 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:53.157 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T05:21:53.172 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T05:21:53.215 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T05:21:53.506 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T05:21:53.520 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:53.576 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T05:21:53.591 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T05:21:53.634 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T05:21:53.893 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T05:21:53.906 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T05:21:53.951 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T05:21:53.989 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T05:21:54.004 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T05:21:54.047 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T05:21:54.229 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T05:21:54.242 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T05:21:54.287 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T05:21:54.342 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-idna. 2024-09-17T05:21:54.356 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T05:21:54.391 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T05:21:54.641 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T05:21:54.654 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:54.699 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:54.703 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T05:21:54.718 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T05:21:54.761 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T05:21:55.072 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-requests. 2024-09-17T05:21:55.087 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T05:21:55.094 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T05:21:55.111 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:55.131 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T05:21:55.153 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mon (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:55.450 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T05:21:55.465 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T05:21:55.508 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T05:21:55.820 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T05:21:55.835 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:21:55.878 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:55.973 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T05:21:55.988 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:56.057 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-osd (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:56.408 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T05:21:56.423 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:56.457 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:56.752 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T05:21:56.768 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T05:21:56.810 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T05:21:56.844 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph. 2024-09-17T05:21:56.858 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../069-ceph_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:56.902 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:57.113 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T05:21:57.128 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T05:21:57.171 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T05:21:57.230 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T05:21:57.244 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:57.279 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-fuse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:57.508 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T05:21:57.523 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:57.566 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:57.591 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T05:21:57.605 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:57.641 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mds (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:57.952 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T05:21:57.967 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:58.002 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mon (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:58.037 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephadm. 2024-09-17T05:21:58.051 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../072-cephadm_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:58.095 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephadm (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:58.406 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T05:21:58.418 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T05:21:58.456 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T05:21:58.572 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T05:21:58.587 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:58.622 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-osd (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:58.767 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T05:21:58.779 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T05:21:58.809 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T05:21:59.375 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph. 2024-09-17T05:21:59.389 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../069-ceph_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:59.433 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:59.786 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T05:21:59.801 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:21:59.839 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-babel. 2024-09-17T05:21:59.844 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-fuse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:21:59.850 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T05:21:59.889 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T05:22:00.197 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T05:22:00.212 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:00.234 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T05:22:00.245 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T05:22:00.255 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mds (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:00.284 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T05:22:00.591 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephadm. 2024-09-17T05:22:00.606 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../072-cephadm_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:00.649 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephadm (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:00.663 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T05:22:00.674 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:00.721 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:00.994 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T05:22:01.009 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T05:22:01.053 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T05:22:01.066 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T05:22:01.077 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T05:22:01.116 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T05:22:01.372 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T05:22:01.387 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T05:22:01.430 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T05:22:01.436 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-routes. 2024-09-17T05:22:01.447 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T05:22:01.486 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T05:22:01.848 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T05:22:01.859 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:01.898 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:02.443 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-babel. 2024-09-17T05:22:02.457 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T05:22:02.501 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T05:22:02.821 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T05:22:02.834 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T05:22:02.871 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T05:22:03.207 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T05:22:03.220 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:03.256 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:03.602 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T05:22:03.615 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T05:22:03.660 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T05:22:03.946 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-routes. 2024-09-17T05:22:03.960 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T05:22:04.004 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T05:22:04.263 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T05:22:04.275 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T05:22:04.313 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T05:22:04.340 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T05:22:04.354 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:04.399 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:04.792 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T05:22:04.805 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T05:22:04.927 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T05:22:05.341 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T05:22:05.355 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T05:22:05.399 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T05:22:05.702 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T05:22:05.714 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T05:22:05.752 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T05:22:06.445 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T05:22:06.458 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:06.512 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:06.887 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T05:22:06.902 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T05:22:06.945 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T05:22:07.448 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T05:22:07.449 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T05:22:07.462 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T05:22:07.463 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T05:22:07.507 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T05:22:07.584 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T05:22:07.818 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T05:22:07.832 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T05:22:07.877 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T05:22:07.929 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T05:22:07.943 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T05:22:07.987 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T05:22:08.206 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T05:22:08.219 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T05:22:08.264 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T05:22:08.349 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T05:22:08.363 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T05:22:08.415 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T05:22:08.609 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T05:22:08.623 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T05:22:08.667 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T05:22:09.021 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T05:22:09.035 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T05:22:09.057 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T05:22:09.072 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:09.079 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T05:22:09.115 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:09.450 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T05:22:09.465 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T05:22:09.525 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T05:22:09.903 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T05:22:09.917 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T05:22:09.970 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T05:22:10.044 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T05:22:10.060 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T05:22:10.094 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T05:22:10.340 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T05:22:10.355 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T05:22:10.380 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T05:22:10.396 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T05:22:10.399 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T05:22:10.438 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T05:22:10.774 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T05:22:10.791 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T05:22:10.833 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T05:22:11.152 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T05:22:11.169 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T05:22:11.211 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T05:22:11.455 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T05:22:11.468 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:11.505 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:11.681 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T05:22:11.697 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T05:22:11.739 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T05:22:11.825 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T05:22:11.837 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T05:22:11.874 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T05:22:12.059 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T05:22:12.075 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T05:22:12.117 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T05:22:12.211 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T05:22:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T05:22:12.270 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T05:22:12.462 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T05:22:12.478 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T05:22:12.520 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T05:22:12.541 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package jq. 2024-09-17T05:22:12.555 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T05:22:12.599 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T05:22:12.823 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T05:22:12.839 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T05:22:12.852 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package socat. 2024-09-17T05:22:12.866 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T05:22:12.881 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T05:22:12.902 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T05:22:13.205 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T05:22:13.221 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T05:22:13.255 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T05:22:13.495 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T05:22:13.510 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:13.578 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:13.584 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-test. 2024-09-17T05:22:13.597 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:13.659 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-test (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:13.897 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T05:22:13.912 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T05:22:13.956 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T05:22:14.309 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T05:22:14.323 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T05:22:14.367 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T05:22:14.611 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package jq. 2024-09-17T05:22:14.626 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T05:22:14.669 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T05:22:14.939 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package socat. 2024-09-17T05:22:14.955 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T05:22:14.997 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T05:22:15.300 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T05:22:15.316 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T05:22:15.350 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T05:22:15.630 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T05:22:15.643 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:15.679 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-test. 2024-09-17T05:22:15.688 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-volume (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:15.694 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:15.737 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-test (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:16.042 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T05:22:16.056 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:16.100 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-mirror (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:16.428 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T05:22:16.442 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T05:22:16.486 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T05:22:16.756 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T05:22:16.769 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:16.813 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-shell (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:17.075 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T05:22:17.089 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:17.150 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-top (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:17.393 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T05:22:17.407 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:17.443 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs-dev (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:17.779 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T05:22:17.793 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T05:22:17.838 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T05:22:17.883 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T05:22:17.899 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:17.933 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-volume (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:18.184 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T05:22:18.198 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T05:22:18.267 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T05:22:18.295 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T05:22:18.311 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:18.353 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephfs-mirror (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:18.596 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T05:22:18.613 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T05:22:18.654 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T05:22:18.656 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T05:22:18.672 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T05:22:18.714 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T05:22:19.009 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T05:22:19.025 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:19.067 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephfs-shell (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:19.093 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package pkg-config. 2024-09-17T05:22:19.107 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T05:22:19.151 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T05:22:19.320 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T05:22:19.336 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4963-g820f760e-1jammy_all.deb ... 2024-09-17T05:22:19.378 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephfs-top (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:19.437 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T05:22:19.452 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T05:22:19.487 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T05:22:19.605 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T05:22:19.621 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:19.663 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcephfs-dev (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:19.890 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T05:22:19.904 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T05:22:19.940 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T05:22:19.975 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T05:22:19.991 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T05:22:20.033 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T05:22:20.276 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T05:22:20.291 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T05:22:20.334 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T05:22:20.396 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T05:22:20.412 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T05:22:20.453 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T05:22:20.662 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T05:22:20.677 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T05:22:20.720 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T05:22:20.824 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T05:22:20.840 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T05:22:20.882 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T05:22:21.041 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T05:22:21.055 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T05:22:21.098 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T05:22:21.296 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package pkg-config. 2024-09-17T05:22:21.312 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T05:22:21.354 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T05:22:21.459 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-py. 2024-09-17T05:22:21.474 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T05:22:21.518 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T05:22:21.615 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T05:22:21.631 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T05:22:21.665 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T05:22:21.871 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T05:22:21.886 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T05:22:21.929 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T05:22:22.093 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T05:22:22.108 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T05:22:22.151 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T05:22:22.408 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T05:22:22.422 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T05:22:22.429 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T05:22:22.444 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T05:22:22.458 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T05:22:22.487 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T05:22:22.744 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-toml. 2024-09-17T05:22:22.759 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T05:22:22.802 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T05:22:22.924 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T05:22:22.940 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T05:22:22.976 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T05:22:23.121 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T05:22:23.137 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T05:22:23.180 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T05:22:23.268 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T05:22:23.284 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T05:22:23.326 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T05:22:23.533 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T05:22:23.548 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T05:22:23.591 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T05:22:23.648 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-py. 2024-09-17T05:22:23.664 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T05:22:23.706 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T05:22:23.852 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T05:22:23.867 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T05:22:23.910 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T05:22:24.069 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T05:22:24.085 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T05:22:24.126 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T05:22:24.523 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T05:22:24.537 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T05:22:24.581 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T05:22:24.614 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package radosgw. 2024-09-17T05:22:24.629 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../123-radosgw_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:24.689 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking radosgw (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:24.883 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-toml. 2024-09-17T05:22:24.899 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T05:22:24.942 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T05:22:25.270 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T05:22:25.286 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T05:22:25.328 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T05:22:25.368 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T05:22:25.383 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:25.426 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking rbd-fuse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:25.674 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T05:22:25.690 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T05:22:25.720 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T05:22:25.732 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T05:22:25.993 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T05:22:26.010 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T05:22:26.031 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T05:22:26.051 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T05:22:26.157 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T05:22:26.415 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T05:22:26.732 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package radosgw. 2024-09-17T05:22:26.746 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../123-radosgw_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:26.789 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking radosgw (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:26.845 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T05:22:27.461 INFO:teuthology.orchestra.run.smithi122.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T05:22:27.475 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T05:22:27.488 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4963-g820f760e-1jammy_amd64.deb ... 2024-09-17T05:22:27.517 INFO:teuthology.orchestra.run.smithi122.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-17T05:22:27.525 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking rbd-fuse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:27.676 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephadm (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:27.853 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T05:22:28.115 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T05:22:28.240 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T05:22:28.278 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user cephadm....done 2024-09-17T05:22:28.440 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T05:22:28.494 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T05:22:28.708 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T05:22:28.887 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T05:22:28.976 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T05:22:29.093 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T05:22:29.370 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T05:22:29.460 INFO:teuthology.orchestra.run.smithi173.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T05:22:29.512 INFO:teuthology.orchestra.run.smithi173.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-17T05:22:29.559 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephadm (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:29.646 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T05:22:29.772 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T05:22:30.078 INFO:teuthology.orchestra.run.smithi173.stdout:Adding system user cephadm....done 2024-09-17T05:22:30.082 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T05:22:30.265 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T05:22:30.442 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T05:22:30.542 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T05:22:30.718 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T05:22:30.801 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T05:22:30.927 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T05:22:30.978 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T05:22:31.164 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T05:22:31.288 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T05:22:31.438 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T05:22:31.556 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T05:22:31.565 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T05:22:31.840 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T05:22:31.875 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T05:22:32.108 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T05:22:32.244 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T05:22:32.342 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-argparse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:32.503 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T05:22:32.593 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T05:22:32.719 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T05:22:32.755 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T05:22:32.870 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T05:22:32.987 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T05:22:33.040 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T05:22:33.239 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T05:22:33.316 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T05:22:33.390 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T05:22:33.601 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T05:22:33.649 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T05:22:34.003 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T05:22:34.030 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T05:22:34.088 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T05:22:34.152 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T05:22:34.246 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-ceph-argparse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:34.279 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T05:22:34.505 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T05:22:34.631 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T05:22:34.638 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T05:22:34.757 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T05:22:34.764 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T05:22:34.884 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T05:22:35.040 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T05:22:35.151 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T05:22:35.269 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T05:22:35.333 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T05:22:35.504 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T05:22:35.634 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T05:22:35.785 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T05:22:35.874 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T05:22:35.901 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T05:22:35.966 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T05:22:36.092 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T05:22:36.460 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T05:22:36.470 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T05:22:36.569 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T05:22:36.738 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T05:22:36.821 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T05:22:36.881 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T05:22:37.089 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T05:22:37.182 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T05:22:37.398 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T05:22:37.499 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T05:22:37.525 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T05:22:37.901 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T05:22:38.202 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T05:22:38.402 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T05:22:38.603 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T05:22:38.645 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T05:22:38.746 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T05:22:38.888 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T05:22:38.889 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T05:22:39.206 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T05:22:39.206 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T05:22:39.566 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T05:22:39.608 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T05:22:39.817 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T05:22:39.868 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T05:22:39.942 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-common (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:40.261 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T05:22:40.294 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T05:22:40.554 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T05:22:40.636 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T05:22:40.880 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T05:22:41.232 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T05:22:41.491 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T05:22:41.618 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-ceph-common (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:41.842 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T05:22:41.936 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T05:22:42.111 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:22:42.245 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T05:22:42.262 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T05:22:42.512 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T05:22:42.647 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T05:22:42.764 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T05:22:43.083 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T05:22:43.375 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T05:22:43.493 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T05:22:43.643 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:22:43.712 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:22:43.794 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T05:22:43.871 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T05:22:44.112 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T05:22:44.148 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T05:22:44.274 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T05:22:44.380 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T05:22:44.400 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T05:22:44.656 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T05:22:44.710 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T05:22:44.924 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T05:22:44.978 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T05:22:45.183 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T05:22:45.221 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:22:45.347 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T05:22:45.484 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T05:22:45.610 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T05:22:45.657 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T05:22:45.903 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T05:22:45.925 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T05:22:46.037 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T05:22:46.201 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T05:22:46.339 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T05:22:46.460 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T05:22:46.649 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T05:22:46.703 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T05:22:46.916 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T05:22:46.980 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T05:22:47.025 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T05:22:47.106 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T05:22:47.136 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T05:22:47.256 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T05:22:47.390 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T05:22:47.494 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T05:22:47.533 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T05:22:47.621 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T05:22:47.867 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T05:22:48.022 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T05:22:48.169 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T05:22:48.290 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T05:22:48.420 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T05:22:48.432 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T05:22:48.554 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T05:22:48.680 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T05:22:48.692 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:22:48.769 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T05:22:48.818 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T05:22:49.016 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T05:22:49.142 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T05:22:49.169 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T05:22:49.504 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T05:22:49.543 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T05:22:49.811 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T05:22:49.873 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T05:22:49.920 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T05:22:50.007 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T05:22:50.188 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T05:22:50.300 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T05:22:50.314 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T05:22:50.568 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T05:22:50.641 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T05:22:50.853 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T05:22:50.959 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T05:22:51.246 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T05:22:51.345 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T05:22:51.471 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T05:22:51.539 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T05:22:51.740 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T05:22:51.884 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T05:22:52.009 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librados2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:52.016 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T05:22:52.136 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:52.264 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T05:22:52.293 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T05:22:52.623 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:52.672 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T05:22:52.749 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libradosstriper1 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:52.875 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T05:22:52.946 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T05:22:53.185 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librbd1 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:53.299 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T05:22:53.311 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-modules-core (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:53.425 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librados2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:53.436 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-fuse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:53.551 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:53.677 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T05:22:53.680 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T05:22:53.680 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T05:22:54.022 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcephfs2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.148 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libradosstriper1 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.274 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T05:22:54.296 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs-dev (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.422 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rados (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.548 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librgw2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.568 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librbd1 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.694 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-modules-core (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.699 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-top (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.820 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-fuse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:54.825 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T05:22:55.012 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T05:22:55.012 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T05:22:55.689 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcephfs-dev (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:55.815 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rados (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:56.079 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rbd (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:56.196 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up rbd-fuse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:56.292 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librgw2 (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:56.306 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rgw (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:56.434 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephfs-top (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:56.465 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cephfs (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:56.561 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T05:22:56.591 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-shell (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:56.717 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-common (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:57.321 INFO:teuthology.orchestra.run.smithi122.stdout:Adding group ceph....done 2024-09-17T05:22:57.747 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user ceph....done 2024-09-17T05:22:57.831 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rbd (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:57.915 INFO:teuthology.orchestra.run.smithi122.stdout:Setting system user ceph properties....done 2024-09-17T05:22:57.932 INFO:teuthology.orchestra.run.smithi122.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T05:22:57.957 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up rbd-fuse (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:58.074 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T05:22:58.083 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rgw (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:58.213 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cephfs (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:58.335 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephfs-shell (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:58.461 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T05:22:58.461 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-common (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:59.118 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-mirror (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:59.328 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T05:22:59.328 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T05:22:59.595 INFO:teuthology.orchestra.run.smithi173.stdout:Adding group ceph....done 2024-09-17T05:22:59.928 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-test (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:22:59.996 INFO:teuthology.orchestra.run.smithi173.stdout:Adding system user ceph....done 2024-09-17T05:23:00.054 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up radosgw (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:00.283 INFO:teuthology.orchestra.run.smithi173.stdout:Setting system user ceph properties....done 2024-09-17T05:23:00.299 INFO:teuthology.orchestra.run.smithi173.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T05:23:00.443 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T05:23:00.553 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T05:23:00.553 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T05:23:00.846 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T05:23:01.172 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-base (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:01.510 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephfs-mirror (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:01.545 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T05:23:01.718 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T05:23:01.719 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T05:23:02.199 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mds (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:02.336 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-test (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:02.437 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up radosgw (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:02.882 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T05:23:02.882 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T05:23:02.982 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T05:23:02.982 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T05:23:03.551 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:03.623 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-base (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:03.771 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T05:23:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T05:23:03.984 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T05:23:04.428 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-osd (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:04.650 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mds (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:04.759 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T05:23:04.759 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T05:23:04.889 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T05:23:04.889 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T05:23:05.346 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:05.472 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:05.511 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:05.648 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mon (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:05.714 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T05:23:05.715 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T05:23:05.859 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T05:23:05.860 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T05:23:06.329 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-osd (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:06.475 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-cephadm (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:06.651 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:06.686 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T05:23:06.686 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T05:23:06.811 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-dashboard (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:06.937 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-volume (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:07.297 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:07.367 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T05:23:07.423 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:07.531 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T05:23:07.567 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mon (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:07.774 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T05:23:07.775 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T05:23:07.831 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T05:23:08.401 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-cephadm (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:08.527 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:08.636 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-dashboard (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:08.787 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-volume (19.3.0-4963-g820f760e-1jammy) ... 2024-09-17T05:23:09.149 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T05:23:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T05:23:09.273 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T05:23:09.518 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T05:23:09.852 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T05:23:09.856 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T05:23:09.932 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-17T05:23:10.117 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-17T05:23:10.117 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-17T05:23:10.306 INFO:teuthology.orchestra.run.smithi122.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T05:23:10.307 INFO:teuthology.orchestra.run.smithi122.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T05:23:10.307 INFO:teuthology.orchestra.run.smithi122.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T05:23:10.307 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T05:23:10.307 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T05:23:10.307 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T05:23:10.307 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T05:23:10.308 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T05:23:10.308 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T05:23:10.308 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T05:23:10.308 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-17T05:23:10.308 INFO:teuthology.orchestra.run.smithi122.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T05:23:10.308 INFO:teuthology.orchestra.run.smithi122.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T05:23:10.309 INFO:teuthology.orchestra.run.smithi122.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T05:23:10.309 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T05:23:10.309 INFO:teuthology.orchestra.run.smithi122.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T05:23:10.311 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-17T05:23:10.311 INFO:teuthology.orchestra.run.smithi122.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T05:23:10.311 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T05:23:10.311 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T05:23:10.366 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-17T05:23:10.366 INFO:teuthology.orchestra.run.smithi122.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T05:23:10.367 INFO:teuthology.orchestra.run.smithi122.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T05:23:10.367 INFO:teuthology.orchestra.run.smithi122.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T05:23:10.367 INFO:teuthology.orchestra.run.smithi122.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T05:23:10.367 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T05:23:10.368 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T05:23:10.368 INFO:teuthology.orchestra.run.smithi122.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T05:23:10.368 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-09-17T05:23:10.369 INFO:teuthology.orchestra.run.smithi122.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T05:23:10.554 INFO:teuthology.orchestra.run.smithi122.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-17T05:23:10.554 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 55.8 MB of archives. 2024-09-17T05:23:10.555 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T05:23:10.555 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T05:23:10.745 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T05:23:11.177 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T05:23:11.193 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T05:23:11.317 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T05:23:11.318 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T05:23:11.319 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T05:23:11.348 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T05:23:11.349 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T05:23:11.356 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T05:23:11.356 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T05:23:11.357 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T05:23:11.358 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T05:23:11.421 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T05:23:11.422 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T05:23:11.422 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T05:23:11.445 INFO:teuthology.orchestra.run.smithi173.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T05:23:11.449 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T05:23:11.505 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T05:23:11.526 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-09-17T05:23:11.709 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-09-17T05:23:11.710 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-09-17T05:23:11.897 INFO:teuthology.orchestra.run.smithi173.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T05:23:11.897 INFO:teuthology.orchestra.run.smithi173.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T05:23:11.897 INFO:teuthology.orchestra.run.smithi173.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T05:23:11.897 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T05:23:11.897 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T05:23:11.898 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T05:23:11.898 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T05:23:11.898 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T05:23:11.898 INFO:teuthology.orchestra.run.smithi173.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T05:23:11.898 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T05:23:11.899 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2024-09-17T05:23:11.899 INFO:teuthology.orchestra.run.smithi173.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T05:23:11.899 INFO:teuthology.orchestra.run.smithi173.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T05:23:11.899 INFO:teuthology.orchestra.run.smithi173.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T05:23:11.899 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T05:23:11.900 INFO:teuthology.orchestra.run.smithi173.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T05:23:11.900 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2024-09-17T05:23:11.900 INFO:teuthology.orchestra.run.smithi173.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T05:23:11.900 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T05:23:11.900 INFO:teuthology.orchestra.run.smithi173.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T05:23:11.955 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2024-09-17T05:23:11.955 INFO:teuthology.orchestra.run.smithi173.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T05:23:11.955 INFO:teuthology.orchestra.run.smithi173.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T05:23:11.956 INFO:teuthology.orchestra.run.smithi173.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T05:23:11.956 INFO:teuthology.orchestra.run.smithi173.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T05:23:11.956 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T05:23:11.956 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T05:23:11.956 INFO:teuthology.orchestra.run.smithi173.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T05:23:11.957 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages will be upgraded: 2024-09-17T05:23:11.958 INFO:teuthology.orchestra.run.smithi173.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T05:23:12.014 INFO:teuthology.orchestra.run.smithi173.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-17T05:23:12.014 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 55.8 MB of archives. 2024-09-17T05:23:12.014 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T05:23:12.015 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T05:23:12.147 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T05:23:12.151 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T05:23:12.178 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T05:23:12.179 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T05:23:12.179 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T05:23:12.187 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T05:23:12.187 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T05:23:12.188 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T05:23:12.189 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T05:23:12.189 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T05:23:12.191 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T05:23:12.192 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T05:23:12.192 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T05:23:12.192 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T05:23:12.193 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T05:23:12.193 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T05:23:12.197 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T05:23:12.199 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T05:23:12.200 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T05:23:12.200 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T05:23:12.201 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T05:23:12.217 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T05:23:12.246 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T05:23:12.246 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T05:23:12.325 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T05:23:12.329 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T05:23:12.340 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T05:23:12.416 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T05:23:12.420 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T05:23:12.420 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T05:23:12.421 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T05:23:12.421 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T05:23:12.427 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T05:23:12.429 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T05:23:12.429 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T05:23:12.432 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T05:23:12.432 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T05:23:12.436 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T05:23:12.438 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T05:23:12.443 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T05:23:12.445 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T05:23:12.445 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T05:23:12.446 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T05:23:12.467 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T05:23:12.467 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T05:23:12.477 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T05:23:12.497 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T05:23:12.500 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T05:23:12.501 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T05:23:12.501 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T05:23:12.502 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T05:23:12.513 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T05:23:12.515 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T05:23:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T05:23:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T05:23:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T05:23:12.661 INFO:teuthology.orchestra.run.smithi173.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T05:23:12.662 INFO:teuthology.orchestra.run.smithi173.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T05:23:12.662 INFO:teuthology.orchestra.run.smithi173.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T05:23:12.674 INFO:teuthology.orchestra.run.smithi173.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T05:23:12.675 INFO:teuthology.orchestra.run.smithi173.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T05:23:13.008 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T05:23:13.008 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T05:23:13.009 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T05:23:13.009 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T05:23:13.009 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T05:23:13.009 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T05:23:13.059 INFO:teuthology.orchestra.run.smithi173.stdout:Preconfiguring packages ... 2024-09-17T05:23:13.072 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T05:23:13.073 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T05:23:13.212 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 55.8 MB in 1s (78.7 MB/s) 2024-09-17T05:23:13.455 INFO:teuthology.orchestra.run.smithi122.stdout:Preconfiguring packages ... 2024-09-17T05:23:13.464 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143295 files and directories currently installed.) 2024-09-17T05:23:13.468 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:23:13.610 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:23:13.613 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 55.8 MB in 3s (20.7 MB/s) 2024-09-17T05:23:13.924 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143295 files and directories currently installed.) 2024-09-17T05:23:13.928 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:23:14.070 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:23:14.279 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T05:23:14.705 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T05:23:15.378 INFO:teuthology.orchestra.run.smithi173.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T05:23:15.618 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package flex. 2024-09-17T05:23:15.688 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143295 files and directories currently installed.) 2024-09-17T05:23:15.692 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T05:23:15.733 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T05:23:15.817 INFO:teuthology.orchestra.run.smithi122.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T05:23:16.093 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package flex. 2024-09-17T05:23:16.163 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143295 files and directories currently installed.) 2024-09-17T05:23:16.166 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T05:23:16.209 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T05:23:16.211 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T05:23:16.382 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T05:23:16.745 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T05:23:16.815 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:23:16.914 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T05:23:17.295 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:23:17.361 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143368 files and directories currently installed.) 2024-09-17T05:23:17.365 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:23:17.506 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:23:17.877 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:23:17.877 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143368 files and directories currently installed.) 2024-09-17T05:23:17.881 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:23:18.004 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:23:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:23:18.329 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T05:23:18.427 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T05:23:18.554 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T05:23:18.688 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package bison. 2024-09-17T05:23:18.763 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143368 files and directories currently installed.) 2024-09-17T05:23:18.767 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T05:23:18.815 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T05:23:18.935 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T05:23:19.233 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T05:23:19.248 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T05:23:19.255 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package bison. 2024-09-17T05:23:19.292 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T05:23:19.323 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143368 files and directories currently installed.) 2024-09-17T05:23:19.327 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T05:23:19.366 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T05:23:19.628 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package dump. 2024-09-17T05:23:19.644 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T05:23:19.685 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T05:23:19.783 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T05:23:19.798 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T05:23:19.841 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T05:23:19.973 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package indent. 2024-09-17T05:23:19.989 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:20.031 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T05:23:20.220 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package dump. 2024-09-17T05:23:20.234 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T05:23:20.278 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T05:23:20.283 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T05:23:20.300 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T05:23:20.334 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T05:23:20.565 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package indent. 2024-09-17T05:23:20.580 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:20.648 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T05:23:20.662 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T05:23:20.679 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T05:23:20.721 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T05:23:20.859 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T05:23:20.874 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T05:23:20.918 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T05:23:20.982 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T05:23:20.998 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T05:23:21.040 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T05:23:21.272 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T05:23:21.286 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T05:23:21.330 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T05:23:21.360 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T05:23:21.376 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T05:23:21.410 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T05:23:21.599 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T05:23:21.614 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T05:23:21.646 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T05:23:21.658 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T05:23:21.662 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T05:23:21.704 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T05:23:21.995 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T05:23:22.009 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T05:23:22.053 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T05:23:22.082 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T05:23:22.098 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T05:23:22.140 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T05:23:22.306 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T05:23:22.320 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T05:23:22.389 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T05:23:22.451 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T05:23:22.468 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T05:23:22.510 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T05:23:22.642 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T05:23:22.656 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T05:23:22.700 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T05:23:23.096 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T05:23:23.111 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T05:23:23.154 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T05:23:23.763 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T05:23:23.779 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T05:23:23.813 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:23:24.082 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T05:23:24.098 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T05:23:24.140 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:23:24.459 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T05:23:24.475 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T05:23:24.518 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:23:24.532 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T05:23:24.549 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T05:23:24.590 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:23:24.796 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T05:23:24.812 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T05:23:24.843 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T05:23:24.845 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T05:23:24.859 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T05:23:24.901 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:23:25.124 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T05:23:25.140 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T05:23:25.215 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T05:23:25.221 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T05:23:25.237 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T05:23:25.279 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:23:25.559 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T05:23:25.576 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:25.617 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:25.640 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T05:23:25.657 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T05:23:25.699 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T05:23:25.937 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T05:23:25.953 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:25.987 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:23:26.027 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T05:23:26.042 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T05:23:26.085 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T05:23:26.241 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T05:23:26.256 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:26.289 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:23:26.430 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T05:23:26.445 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:26.513 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:26.567 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T05:23:26.582 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T05:23:26.626 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:23:26.849 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T05:23:26.865 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:26.907 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:23:27.112 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T05:23:27.128 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T05:23:27.171 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T05:23:27.210 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T05:23:27.227 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:27.277 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:23:27.465 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package modemmanager. 2024-09-17T05:23:27.481 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T05:23:27.515 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:23:27.521 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T05:23:27.537 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T05:23:27.580 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:23:28.066 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T05:23:28.082 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T05:23:28.125 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T05:23:28.180 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package network-manager. 2024-09-17T05:23:28.197 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T05:23:28.247 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T05:23:28.512 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package modemmanager. 2024-09-17T05:23:28.528 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T05:23:28.570 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:23:28.812 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ppp. 2024-09-17T05:23:28.829 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T05:23:29.165 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T05:23:29.386 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package network-manager. 2024-09-17T05:23:29.402 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T05:23:29.442 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T05:23:29.452 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T05:23:29.457 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T05:23:29.492 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T05:23:29.762 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T05:23:29.776 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T05:23:29.820 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T05:23:30.123 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T05:23:30.138 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T05:23:30.181 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T05:23:30.202 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ppp. 2024-09-17T05:23:30.217 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T05:23:30.502 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T05:23:30.518 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:30.561 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T05:23:30.562 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:30.874 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T05:23:30.888 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T05:23:30.932 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T05:23:31.098 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T05:23:31.113 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T05:23:31.165 INFO:teuthology.orchestra.run.smithi173.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T05:23:31.196 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-common (238) ... 2024-09-17T05:23:31.226 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T05:23:31.241 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T05:23:31.276 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T05:23:31.571 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T05:23:31.586 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T05:23:31.629 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T05:23:31.663 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T05:23:31.679 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:31.724 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:31.950 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T05:23:31.964 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:32.008 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:32.605 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T05:23:32.620 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T05:23:32.680 INFO:teuthology.orchestra.run.smithi122.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T05:23:32.711 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-common (238) ... 2024-09-17T05:23:32.808 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql. 2024-09-17T05:23:32.824 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T05:23:32.941 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql (14+238) ... 2024-09-17T05:23:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T05:23:33.184 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T05:23:33.212 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T05:23:33.228 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T05:23:33.228 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T05:23:33.273 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:33.471 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T05:23:33.487 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T05:23:33.529 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T05:23:33.849 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T05:23:33.865 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T05:23:33.906 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T05:23:34.201 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package tcl. 2024-09-17T05:23:34.217 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T05:23:34.255 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T05:23:34.612 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T05:23:34.628 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T05:23:34.673 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T05:23:34.782 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql. 2024-09-17T05:23:34.797 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T05:23:34.840 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql (14+238) ... 2024-09-17T05:23:35.024 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T05:23:35.042 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T05:23:35.084 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T05:23:35.101 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T05:23:35.115 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T05:23:35.150 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T05:23:35.369 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T05:23:35.384 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T05:23:35.427 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T05:23:35.428 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T05:23:35.444 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T05:23:35.485 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T05:23:35.762 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T05:23:35.772 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T05:23:35.787 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T05:23:35.830 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T05:23:35.974 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T05:23:36.101 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T05:23:36.133 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package tcl. 2024-09-17T05:23:36.148 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T05:23:36.195 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T05:23:36.227 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:36.353 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T05:23:36.512 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T05:23:36.544 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T05:23:36.559 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T05:23:36.602 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T05:23:36.638 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:36.965 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T05:23:36.980 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T05:23:37.023 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T05:23:37.410 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T05:23:37.425 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T05:23:37.476 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T05:23:37.591 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T05:23:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T05:23:37.761 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T05:23:37.982 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T05:23:38.098 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T05:23:38.098 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T05:23:38.109 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T05:23:38.243 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:38.369 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T05:23:38.520 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T05:23:38.646 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:38.805 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T05:23:38.917 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T05:23:39.049 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:23:39.174 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T05:23:39.309 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T05:23:39.435 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T05:23:39.561 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T05:23:39.603 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T05:23:39.686 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T05:23:39.744 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T05:23:40.197 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T05:23:40.197 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T05:23:40.792 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T05:23:40.897 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T05:23:40.918 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T05:23:41.013 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T05:23:41.022 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T05:23:41.078 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:23:41.123 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T05:23:41.204 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T05:23:41.241 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T05:23:41.330 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T05:23:41.400 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T05:23:41.455 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T05:23:41.526 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T05:23:41.598 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T05:23:41.663 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T05:23:41.680 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T05:23:41.750 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T05:23:41.787 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T05:23:41.876 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T05:23:42.052 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T05:23:42.178 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:23:42.304 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T05:23:42.464 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T05:23:42.476 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T05:23:42.581 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T05:23:42.602 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T05:23:42.696 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T05:23:42.762 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:23:42.896 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T05:23:43.047 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T05:23:43.165 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T05:23:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:23:43.187 INFO:teuthology.orchestra.run.smithi173.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T05:23:43.187 INFO:teuthology.orchestra.run.smithi173.stdout:which means they are currently configured by ifupdown: 2024-09-17T05:23:43.187 INFO:teuthology.orchestra.run.smithi173.stdout:- enp3s0f1 2024-09-17T05:23:43.187 INFO:teuthology.orchestra.run.smithi173.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T05:23:43.187 INFO:teuthology.orchestra.run.smithi173.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T05:23:43.187 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:23:43.274 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T05:23:43.320 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T05:23:43.364 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T05:23:43.434 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T05:23:43.560 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T05:23:43.729 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T05:23:43.770 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T05:23:43.896 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:23:44.022 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T05:23:44.131 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T05:23:44.223 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T05:23:44.333 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T05:23:44.792 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:23:44.935 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T05:23:45.061 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T05:23:45.187 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-common (238) ... 2024-09-17T05:23:45.257 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:23:45.257 INFO:teuthology.orchestra.run.smithi122.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T05:23:45.257 INFO:teuthology.orchestra.run.smithi122.stdout:which means they are currently configured by ifupdown: 2024-09-17T05:23:45.257 INFO:teuthology.orchestra.run.smithi122.stdout:- enp3s0f1 2024-09-17T05:23:45.257 INFO:teuthology.orchestra.run.smithi122.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T05:23:45.257 INFO:teuthology.orchestra.run.smithi122.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T05:23:45.257 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:23:45.390 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T05:23:45.779 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T05:23:46.180 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T05:23:46.491 INFO:teuthology.orchestra.run.smithi173.stdout:Adding user postgres to group ssl-cert 2024-09-17T05:23:46.734 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:23:46.734 INFO:teuthology.orchestra.run.smithi173.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T05:23:46.877 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T05:23:46.994 INFO:teuthology.orchestra.run.smithi173.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T05:23:46.994 INFO:teuthology.orchestra.run.smithi173.stdout:Removing obsolete dictionary files: 2024-09-17T05:23:47.029 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T05:23:47.173 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T05:23:47.291 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-common (238) ... 2024-09-17T05:23:47.742 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T05:23:48.205 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:48.251 INFO:teuthology.orchestra.run.smithi122.stdout:Adding user postgres to group ssl-cert 2024-09-17T05:23:48.534 INFO:teuthology.orchestra.run.smithi173.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T05:23:48.543 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:23:48.543 INFO:teuthology.orchestra.run.smithi122.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T05:23:48.552 INFO:teuthology.orchestra.run.smithi173.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T05:23:48.602 INFO:teuthology.orchestra.run.smithi173.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T05:23:48.602 INFO:teuthology.orchestra.run.smithi173.stdout:This user must also own the server process. 2024-09-17T05:23:48.602 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:23:48.602 INFO:teuthology.orchestra.run.smithi173.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T05:23:48.602 INFO:teuthology.orchestra.run.smithi173.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T05:23:48.602 INFO:teuthology.orchestra.run.smithi173.stdout:The default text search configuration will be set to "english". 2024-09-17T05:23:48.602 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:23:48.602 INFO:teuthology.orchestra.run.smithi173.stdout:Data page checksums are disabled. 2024-09-17T05:23:48.603 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:23:48.603 INFO:teuthology.orchestra.run.smithi173.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T05:23:48.603 INFO:teuthology.orchestra.run.smithi173.stdout:creating subdirectories ... ok 2024-09-17T05:23:48.603 INFO:teuthology.orchestra.run.smithi173.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T05:23:48.627 INFO:teuthology.orchestra.run.smithi173.stdout:selecting default max_connections ... 100 2024-09-17T05:23:48.660 INFO:teuthology.orchestra.run.smithi173.stdout:selecting default shared_buffers ... 128MB 2024-09-17T05:23:48.675 INFO:teuthology.orchestra.run.smithi173.stdout:selecting default time zone ... Etc/UTC 2024-09-17T05:23:48.676 INFO:teuthology.orchestra.run.smithi173.stdout:creating configuration files ... ok 2024-09-17T05:23:48.817 INFO:teuthology.orchestra.run.smithi122.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T05:23:48.818 INFO:teuthology.orchestra.run.smithi122.stdout:Removing obsolete dictionary files: 2024-09-17T05:23:48.838 INFO:teuthology.orchestra.run.smithi173.stdout:running bootstrap script ... ok 2024-09-17T05:23:49.481 INFO:teuthology.orchestra.run.smithi173.stdout:performing post-bootstrap initialization ... ok 2024-09-17T05:23:49.548 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T05:23:49.992 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T05:23:50.056 INFO:teuthology.orchestra.run.smithi173.stdout:syncing data to disk ... ok 2024-09-17T05:23:50.376 INFO:teuthology.orchestra.run.smithi122.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T05:23:50.395 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout:This user must also own the server process. 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout:The default text search configuration will be set to "english". 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout:Data page checksums are disabled. 2024-09-17T05:23:50.442 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:23:50.443 INFO:teuthology.orchestra.run.smithi122.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T05:23:50.443 INFO:teuthology.orchestra.run.smithi122.stdout:creating subdirectories ... ok 2024-09-17T05:23:50.443 INFO:teuthology.orchestra.run.smithi122.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T05:23:50.467 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default max_connections ... 100 2024-09-17T05:23:50.500 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default shared_buffers ... 128MB 2024-09-17T05:23:50.513 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default time zone ... Etc/UTC 2024-09-17T05:23:50.514 INFO:teuthology.orchestra.run.smithi122.stdout:creating configuration files ... ok 2024-09-17T05:23:50.560 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T05:23:50.676 INFO:teuthology.orchestra.run.smithi122.stdout:running bootstrap script ... ok 2024-09-17T05:23:51.298 INFO:teuthology.orchestra.run.smithi122.stdout:performing post-bootstrap initialization ... ok 2024-09-17T05:23:51.941 INFO:teuthology.orchestra.run.smithi122.stdout:syncing data to disk ... ok 2024-09-17T05:23:52.491 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T05:23:53.563 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:23:53.689 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T05:23:53.815 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql (14+238) ... 2024-09-17T05:23:53.941 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:23:54.084 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:23:54.344 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T05:23:54.344 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T05:23:55.087 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T05:23:55.170 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T05:23:55.442 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:23:55.531 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T05:23:55.568 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T05:23:55.666 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T05:23:55.694 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql (14+238) ... 2024-09-17T05:23:55.837 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T05:23:55.954 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T05:23:56.224 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T05:23:56.224 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T05:23:56.416 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T05:23:56.541 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T05:23:56.957 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T05:23:57.066 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T05:23:57.088 INFO:teuthology.orchestra.run.smithi173.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T05:23:57.095 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:57.360 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T05:23:57.545 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T05:23:58.915 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T05:23:59.015 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T05:23:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T05:23:59.591 DEBUG:teuthology.parallel:result is None 2024-09-17T05:23:59.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=820f760e4779dafd30d668ecea7aba6355dfd3d8 2024-09-17T05:23:59.734 DEBUG:teuthology.orchestra.run.smithi122:> dpkg-query -W -f '${Version}' ceph 2024-09-17T05:23:59.756 INFO:teuthology.orchestra.run.smithi122.stdout:19.3.0-4963-g820f760e-1jammy 2024-09-17T05:23:59.756 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4963-g820f760e-1jammy 2024-09-17T05:23:59.756 INFO:teuthology.task.install:The correct ceph version 19.3.0-4963-g820f760e-1jammy is installed. 2024-09-17T05:23:59.759 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=820f760e4779dafd30d668ecea7aba6355dfd3d8 2024-09-17T05:23:59.902 DEBUG:teuthology.orchestra.run.smithi173:> dpkg-query -W -f '${Version}' ceph 2024-09-17T05:23:59.925 INFO:teuthology.orchestra.run.smithi173.stdout:19.3.0-4963-g820f760e-1jammy 2024-09-17T05:23:59.926 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4963-g820f760e-1jammy 2024-09-17T05:23:59.926 INFO:teuthology.task.install:The correct ceph version 19.3.0-4963-g820f760e-1jammy is installed. 2024-09-17T05:23:59.928 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T05:23:59.928 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:23:59.928 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T05:23:59.948 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:23:59.948 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T05:23:59.990 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T05:23:59.990 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:23:59.990 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T05:24:00.009 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T05:24:00.069 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:00.069 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T05:24:00.091 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T05:24:00.153 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T05:24:00.153 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:00.153 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T05:24:00.172 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T05:24:00.232 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:00.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T05:24:00.253 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T05:24:00.321 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T05:24:00.321 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:00.321 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T05:24:00.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T05:24:00.400 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:00.400 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T05:24:00.421 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T05:24:00.485 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T05:24:00.568 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T05:24:00.568 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-09-17T05:24:00.572 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /var/log/ceph 2024-09-17T05:24:00.592 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T05:24:00.592 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T05:24:00.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T05:24:00.653 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T05:24:00.654 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T05:24:00.693 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T05:24:00.723 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-17T05:24:00.723 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-17T05:24:00.723 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-17T05:24:00.723 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:00.723 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T05:24:00.757 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T05:24:00.842 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:00.842 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T05:24:00.851 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T05:24:00.946 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T05:24:00.947 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T05:24:00.947 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689', 'branch': 'wip-athakkar-testing-2024-09-16-2351', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': 'athakkar@redhat.com', 'first_in_suite': False, 'job_id': '7908689', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-athakkar-testing-2024-09-16-2351'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-athakkar-testing-2024-09-16-2351', 'sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8'}}, 'owner': 'scheduled_avan@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 448, 'sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8', 'sleep_before_teardown': 0, 'subset': '1/20', 'suite': 'fs:volumes', 'suite_branch': 'wip-athakkar-testing-2024-09-16-2351', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_820f760e4779dafd30d668ecea7aba6355dfd3d8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBCk2fv5ikwml16vxOA0Y2e90NM9G4J5usuURDz37ms5URmAafpco6DKPEUcrvq/rnZxUxJSgvK9rHW7SSyrR5E=', 'smithi173.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgU/LFxku8av9JJyCv9GHIp42FmzgHX5JGRPjQRQ4JaRQ2RMaQUX/wJeJVDXF9uq+vRKG1JjH6CQVPRChUb0Ao='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '820f760e4779dafd30d668ecea7aba6355dfd3d8', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_05:02:16', 'tube': 'smithi', 'user': 'avan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T05:24:00.948 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T05:24:00.951 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T05:24:00.995 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T05:24:01.000 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T05:24:01.057 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:01.057 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T05:24:01.066 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T05:24:01.066 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-17T05:24:01.118 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T05:24:01.118 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:24:01.118 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 893 Links: 1 2024-09-17T05:24:01.118 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:24:01.118 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-17 05:17:21.861247655 +0000 2024-09-17T05:24:01.118 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-17 05:17:21.857247691 +0000 2024-09-17T05:24:01.118 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-17 05:17:21.857247691 +0000 2024-09-17T05:24:01.118 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-17 05:17:21.857247691 +0000 2024-09-17T05:24:01.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T05:24:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-17T05:24:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-17T05:24:01.183 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000678357 s, 755 kB/s 2024-09-17T05:24:01.184 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T05:24:01.239 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-17T05:24:01.289 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T05:24:01.290 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:24:01.290 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 911 Links: 1 2024-09-17T05:24:01.290 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:24:01.290 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-17 05:17:22.673240402 +0000 2024-09-17T05:24:01.290 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-17 05:17:22.401242831 +0000 2024-09-17T05:24:01.290 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-17 05:17:22.401242831 +0000 2024-09-17T05:24:01.290 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-17 05:17:22.401242831 +0000 2024-09-17T05:24:01.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T05:24:01.346 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-17T05:24:01.346 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-17T05:24:01.346 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000342224 s, 1.5 MB/s 2024-09-17T05:24:01.348 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T05:24:01.399 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-17T05:24:01.454 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T05:24:01.454 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:24:01.454 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 927 Links: 1 2024-09-17T05:24:01.454 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:24:01.454 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-17 05:17:22.969237757 +0000 2024-09-17T05:24:01.454 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-17 05:17:22.965237794 +0000 2024-09-17T05:24:01.454 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-17 05:17:22.965237794 +0000 2024-09-17T05:24:01.454 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-17 05:17:22.965237794 +0000 2024-09-17T05:24:01.454 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T05:24:01.514 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-17T05:24:01.514 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-17T05:24:01.514 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000357451 s, 1.4 MB/s 2024-09-17T05:24:01.516 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T05:24:01.567 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-17T05:24:01.621 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T05:24:01.621 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:24:01.622 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 942 Links: 1 2024-09-17T05:24:01.622 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:24:01.622 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-17 05:17:23.785230470 +0000 2024-09-17T05:24:01.622 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-17 05:17:23.553232542 +0000 2024-09-17T05:24:01.622 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-17 05:17:23.553232542 +0000 2024-09-17T05:24:01.622 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-17 05:17:23.553232542 +0000 2024-09-17T05:24:01.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T05:24:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-17T05:24:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-17T05:24:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000386155 s, 1.3 MB/s 2024-09-17T05:24:01.680 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T05:24:01.731 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:01.731 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:01.731 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T05:24:01.741 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T05:24:01.741 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-09-17T05:24:01.793 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T05:24:01.793 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:24:01.793 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 891 Links: 1 2024-09-17T05:24:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:24:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-17 05:17:22.182956278 +0000 2024-09-17T05:24:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-17 05:17:21.906958743 +0000 2024-09-17T05:24:01.794 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-17 05:17:21.906958743 +0000 2024-09-17T05:24:01.794 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-17 05:17:21.906958743 +0000 2024-09-17T05:24:01.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T05:24:01.855 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-17T05:24:01.855 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-17T05:24:01.855 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000414248 s, 1.2 MB/s 2024-09-17T05:24:01.857 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T05:24:01.913 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-09-17T05:24:01.969 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T05:24:01.969 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:24:01.970 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 908 Links: 1 2024-09-17T05:24:01.970 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:24:01.970 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-17 05:17:22.754951168 +0000 2024-09-17T05:24:01.970 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-17 05:17:22.482953597 +0000 2024-09-17T05:24:01.970 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-17 05:17:22.482953597 +0000 2024-09-17T05:24:01.970 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-17 05:17:22.482953597 +0000 2024-09-17T05:24:01.970 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T05:24:02.035 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-17T05:24:02.035 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-17T05:24:02.035 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.0003849 s, 1.3 MB/s 2024-09-17T05:24:02.037 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T05:24:02.095 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-09-17T05:24:02.145 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T05:24:02.145 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:24:02.145 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 926 Links: 1 2024-09-17T05:24:02.145 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:24:02.145 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-17 05:17:23.318946131 +0000 2024-09-17T05:24:02.145 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-17 05:17:23.042948596 +0000 2024-09-17T05:24:02.145 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-17 05:17:23.042948596 +0000 2024-09-17T05:24:02.146 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-17 05:17:23.042948596 +0000 2024-09-17T05:24:02.146 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T05:24:02.211 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-17T05:24:02.211 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-17T05:24:02.211 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000382239 s, 1.3 MB/s 2024-09-17T05:24:02.213 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T05:24:02.267 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-09-17T05:24:02.322 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T05:24:02.322 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:24:02.322 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 942 Links: 1 2024-09-17T05:24:02.322 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:24:02.322 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-09-17 05:17:23.610943522 +0000 2024-09-17T05:24:02.322 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-09-17 05:17:23.606943558 +0000 2024-09-17T05:24:02.322 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-09-17 05:17:23.606943558 +0000 2024-09-17T05:24:02.322 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-09-17 05:17:23.606943558 +0000 2024-09-17T05:24:02.322 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T05:24:02.387 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-09-17T05:24:02.387 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-09-17T05:24:02.387 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000396315 s, 1.3 MB/s 2024-09-17T05:24:02.390 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T05:24:02.443 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:02.443 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T05:24:02.443 INFO:tasks.ceph:Generating config... 2024-09-17T05:24:02.445 INFO:tasks.ceph:[client] pid file = 2024-09-17T05:24:02.445 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T05:24:02.445 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T05:24:02.446 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T05:24:02.447 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T05:24:02.447 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T05:24:02.447 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T05:24:02.447 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T05:24:02.489 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T05:24:02.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T05:24:02.592 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:24:02.611 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.122'), ('mon.b', '172.21.15.173'), ('mon.c', '[v2:172.21.15.173:3301,v1:172.21.15.173:6790]')] 2024-09-17T05:24:02.611 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.122,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-17T05:24:02.612 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:02.612 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T05:24:02.662 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.122 --add b 172.21.15.173 --addv c '[v2:172.21.15.173:3301,v1:172.21.15.173:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:02.747 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: generated fsid 51547930-c98d-41c6-b8ce-f14d7623b707 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = quincy 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 0 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:fsid 51547930-c98d-41c6-b8ce-f14d7623b707 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:last_changed 2024-09-17T05:24:02.749270+0000 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-09-17T05:24:02.749270+0000 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:min_mon_release 17 (quincy) 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:election_strategy: 1 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:2: [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] mon.c 2024-09-17T05:24:02.748 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T05:24:02.750 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T05:24:02.758 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 51547930-c98d-41c6-b8ce-f14d7623b707... 2024-09-17T05:24:02.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T05:24:02.804 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T05:24:02.843 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-17T05:24:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-09-17T05:24:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T05:24:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-09-17T05:24:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-09-17T05:24:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T05:24:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-09-17T05:24:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.122,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on skipped message = False 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 51547930-c98d-41c6-b8ce-f14d7623b707 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.846 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-09-17T05:24:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: bdev async discard = True 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: bdev enable discard = True 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore allocator = bitmap 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 1/20 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 1/20 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 4/10 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-09-17T05:24:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: client mount timeout = 30 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: debug client = 20 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: debug finisher = 20 2024-09-17T05:24:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-09-17T05:24:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: mon down mkfs grace = 300 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = "" 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-09-17T05:24:02.853 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-09-17T05:24:02.854 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-09-17T05:24:02.854 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-09-17T05:24:02.854 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c] 2024-09-17T05:24:02.854 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-09-17T05:24:02.854 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d] 2024-09-17T05:24:02.854 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2024-09-17T05:24:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2024-09-17T05:24:02.854 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.855 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2024-09-17T05:24:02.856 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2024-09-17T05:24:02.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.122,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 51547930-c98d-41c6-b8ce-f14d7623b707 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: osd mclock profile = high_recovery_ops 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2024-09-17T05:24:02.858 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug trim objects = true 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.859 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2024-09-17T05:24:02.860 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 1/20 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 1/20 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 20 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 4/10 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2024-09-17T05:24:02.861 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 30 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: debug finisher = 20 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2024-09-17T05:24:02.862 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.863 INFO:teuthology.orchestra.run.smithi173.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: # 1m isn't quite enough 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: mon_down_mkfs_grace = 2m 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_filestore_osds = false 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: mon down mkfs grace = 300 2024-09-17T05:24:02.864 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2024-09-17T05:24:02.865 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2024-09-17T05:24:02.867 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T05:24:02.867 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T05:24:02.966 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T05:24:02.966 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:02.966 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T05:24:02.975 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:02.975 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T05:24:03.026 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:24:03.026 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:03.026 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T05:24:03.026 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:24:03.100 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:03.100 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:03.156 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:24:03.156 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:03.156 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T05:24:03.156 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:24:03.191 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:03.191 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:03.241 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T05:24:03.241 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T05:24:03.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T05:24:03.294 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T05:24:03.299 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T05:24:03.358 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T05:24:03.374 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T05:24:03.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T05:24:03.428 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T05:24:03.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-17T05:24:03.452 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T05:24:03.546 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T05:24:03.550 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-17T05:24:03.569 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T05:24:03.627 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T05:24:03.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-17T05:24:03.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T05:24:03.750 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T05:24:03.756 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-17T05:24:03.775 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T05:24:03.775 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T05:24:03.817 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T05:24:03.835 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T05:24:03.918 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-17T05:24:03.935 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T05:24:03.976 INFO:teuthology.orchestra.run.smithi173.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T05:24:03.993 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-17T05:24:04.078 INFO:teuthology.orchestra.run.smithi173.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-17T05:24:04.096 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T05:24:04.096 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T05:24:04.096 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T05:24:04.115 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:04.115 INFO:tasks.ceph:role: osd.0 2024-09-17T05:24:04.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:24:04.115 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:24:04.184 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:24:04.584 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-17T05:24:04.587 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-17T05:24:04.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T05:24:04.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T05:24:04.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:24:04.701 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:24:04.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T05:24:04.760 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:04.761 INFO:tasks.ceph:role: osd.1 2024-09-17T05:24:04.761 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:24:04.761 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T05:24:04.828 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:24:04.828 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:24:04.828 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:24:04.828 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:24:04.828 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:24:04.829 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:24:04.829 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:24:04.829 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:24:04.829 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:24:04.829 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:24:05.233 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-17T05:24:05.236 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-17T05:24:05.236 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T05:24:05.370 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T05:24:05.386 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:24:05.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:24:05.387 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T05:24:05.454 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:05.454 INFO:tasks.ceph:role: osd.2 2024-09-17T05:24:05.454 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:24:05.454 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T05:24:05.525 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:24:05.525 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:24:05.525 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:24:05.525 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:24:05.526 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:24:05.526 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:24:05.526 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:24:05.526 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:24:05.526 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:24:05.526 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:24:05.938 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-17T05:24:05.941 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-17T05:24:05.941 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T05:24:06.066 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T05:24:06.081 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:24:06.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:24:06.083 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T05:24:06.149 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:06.149 INFO:tasks.ceph:role: osd.3 2024-09-17T05:24:06.149 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:24:06.149 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T05:24:06.216 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:24:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:24:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:24:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:24:06.217 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:24:06.217 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:24:06.217 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:24:06.217 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:24:06.217 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:24:06.217 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:24:06.582 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-17T05:24:06.585 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-17T05:24:06.585 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T05:24:06.658 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T05:24:06.721 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:24:06.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:24:06.723 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:06.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:06.841+0000 7f98ed6d97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T05:24:06.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:06.841+0000 7f98ed6d97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T05:24:06.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:06.841+0000 7f98ed6d97c0 -1 bdev(0x555df02d3800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T05:24:06.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:06.841+0000 7f98ed6d97c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T05:24:10.068 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T05:24:10.088 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:10.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:10.181+0000 7f212b3b67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T05:24:10.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:10.181+0000 7f212b3b67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T05:24:10.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:10.181+0000 7f212b3b67c0 -1 bdev(0x5611db40f800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T05:24:10.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:10.181+0000 7f212b3b67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T05:24:13.644 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T05:24:13.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:13.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:13.753+0000 7f83e58b27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T05:24:13.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:13.753+0000 7f83e58b27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T05:24:13.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:13.753+0000 7f83e58b27c0 -1 bdev(0x55d90cfd3800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T05:24:13.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:13.753+0000 7f83e58b27c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T05:24:17.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T05:24:17.023 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:17.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:17.113+0000 7fa36685b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T05:24:17.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:17.113+0000 7fa36685b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T05:24:17.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:17.113+0000 7fa36685b7c0 -1 bdev(0x562b72353800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T05:24:17.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:17.113+0000 7fa36685b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T05:24:20.332 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T05:24:20.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T05:24:20.374 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:20.374 INFO:tasks.ceph:role: osd.4 2024-09-17T05:24:20.374 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:24:20.374 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:24:20.445 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:24:20.853 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-09-17T05:24:20.857 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-09-17T05:24:20.857 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T05:24:20.996 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T05:24:21.013 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:24:21.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:24:21.014 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T05:24:21.077 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:21.077 INFO:tasks.ceph:role: osd.5 2024-09-17T05:24:21.077 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:24:21.077 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T05:24:21.148 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:24:21.148 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:24:21.148 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:24:21.148 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:24:21.148 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:24:21.148 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:24:21.148 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:24:21.149 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:24:21.149 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:24:21.149 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:24:21.520 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-09-17T05:24:21.523 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-09-17T05:24:21.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T05:24:21.600 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T05:24:21.661 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:24:21.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:24:21.662 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T05:24:21.724 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:21.725 INFO:tasks.ceph:role: osd.6 2024-09-17T05:24:21.725 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:24:21.725 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T05:24:21.796 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:24:21.796 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:24:21.796 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:24:21.796 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:24:21.796 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:24:21.796 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:24:21.796 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:24:21.796 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:24:21.797 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:24:21.797 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:24:22.196 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-09-17T05:24:22.199 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-09-17T05:24:22.199 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T05:24:22.295 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T05:24:22.311 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:24:22.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:24:22.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T05:24:22.375 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:24:22.376 INFO:tasks.ceph:role: osd.7 2024-09-17T05:24:22.376 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:24:22.376 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T05:24:22.443 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:24:22.444 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:24:22.844 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-09-17T05:24:22.848 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-09-17T05:24:22.848 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T05:24:22.972 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T05:24:22.989 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:24:22.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:24:22.990 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:23.111 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:23.108+0000 7f3e00f4a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T05:24:23.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:23.116+0000 7f3e00f4a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T05:24:23.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:23.116+0000 7f3e00f4a7c0 -1 bdev(0x55d52dae7800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T05:24:23.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:23.116+0000 7f3e00f4a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T05:24:26.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T05:24:26.309 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:26.406 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:26.404+0000 7fcf8a7a47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T05:24:26.406 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:26.404+0000 7fcf8a7a47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T05:24:26.407 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:26.404+0000 7fcf8a7a47c0 -1 bdev(0x562c3eebf800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T05:24:26.407 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:26.404+0000 7fcf8a7a47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T05:24:29.590 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T05:24:29.611 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:29.704 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:29.700+0000 7f891d4e47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T05:24:29.705 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:29.704+0000 7f891d4e47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T05:24:29.705 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:29.704+0000 7f891d4e47c0 -1 bdev(0x564c4a28f800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T05:24:29.705 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:29.704+0000 7f891d4e47c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T05:24:30.949 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:24:30.953 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:24:32.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T05:24:32.895 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:32.990 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:32.987+0000 7fb318e607c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T05:24:32.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:32.987+0000 7fb318e607c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T05:24:32.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:32.987+0000 7fb318e607c0 -1 bdev(0x556460be7800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T05:24:32.991 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:32.987+0000 7fb318e607c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T05:24:36.194 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T05:24:36.214 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T05:24:36.214 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:36.214 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T05:24:36.235 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:36.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-17T05:24:36.301 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:36.302 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-17T05:24:36.365 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:36.365 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T05:24:36.425 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:36.425 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T05:24:36.485 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:36.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T05:24:36.545 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:36.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T05:24:36.609 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:36.609 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T05:24:36.631 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:36.631 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-17T05:24:36.692 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:36.693 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-17T05:24:36.757 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:36.757 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T05:24:36.820 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:36.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T05:24:36.879 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:36.880 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T05:24:36.943 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:36.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T05:24:37.003 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:37.004 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T05:24:37.012 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:37.012 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-17T05:24:37.062 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:37.062 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T05:24:37.070 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:37.071 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-17T05:24:37.120 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T05:24:37.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T05:24:37.123 DEBUG:teuthology.orchestra.run.smithi173:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T05:24:37.168 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.y] 2024-09-17T05:24:37.168 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg== 2024-09-17T05:24:37.168 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-09-17T05:24:37.168 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug== 2024-09-17T05:24:37.168 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c] 2024-09-17T05:24:37.168 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA== 2024-09-17T05:24:37.168 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA== 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA== 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA== 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw== 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg== 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ== 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d] 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw== 2024-09-17T05:24:37.169 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q== 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ== 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.6] 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ== 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.7] 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ== 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout:[client.2] 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A== 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout:[client.1] 2024-09-17T05:24:37.170 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ== 2024-09-17T05:24:37.171 INFO:teuthology.orchestra.run.smithi122.stdout:[client.3] 2024-09-17T05:24:37.171 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA== 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.y] 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg== 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug== 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA== 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.0] 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA== 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.1] 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA== 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.2] 2024-09-17T05:24:37.182 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA== 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.3] 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw== 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.x] 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg== 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ== 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw== 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.4] 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q== 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.5] 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ== 2024-09-17T05:24:37.183 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.6] 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ== 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.7] 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ== 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout:[client.0] 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout:[client.2] 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A== 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout:[client.1] 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ== 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout:[client.3] 2024-09-17T05:24:37.184 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA== 2024-09-17T05:24:37.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:24:37.220 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:24:37.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:24:37.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:24:37.345 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:24:37.347 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:24:37.403 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.406 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.462 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.515 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.572 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.628 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:24:37.631 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:24:37.685 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:24:37.688 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:24:37.744 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:24:37.747 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:24:37.803 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.859 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.863 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.916 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.919 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.972 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:37.975 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:24:38.029 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:24:38.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:24:38.089 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:24:38.092 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:24:38.148 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:24:38.152 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:24:38.208 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:24:38.211 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:24:38.267 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T05:24:38.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T05:24:38.287 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:24:38.396 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T05:24:38.415 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T05:24:38.435 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:24:38.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T05:24:38.566 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T05:24:38.630 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:24:38.738 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T05:24:38.757 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:38.760 DEBUG:teuthology.orchestra.run.smithi173:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:24:38.812 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T05:24:38.812 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T05:24:38.812 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T05:24:38.815 INFO:tasks.ceph.mon.a:Started 2024-09-17T05:24:38.815 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T05:24:38.815 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T05:24:38.855 INFO:tasks.ceph.mon.b:Started 2024-09-17T05:24:38.855 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T05:24:38.855 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T05:24:38.858 INFO:tasks.ceph.mon.c:Started 2024-09-17T05:24:38.858 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T05:24:38.858 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T05:24:38.858 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T05:24:38.861 INFO:tasks.ceph.mgr.y:Started 2024-09-17T05:24:38.861 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T05:24:38.862 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T05:24:38.865 INFO:tasks.ceph.mgr.x:Started 2024-09-17T05:24:38.865 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-17T05:24:38.865 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-17T05:24:38.868 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-17T05:24:38.868 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-17T05:24:38.917 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-17T05:24:38.917 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-17T05:24:38.969 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-17T05:24:38.969 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-17T05:24:39.016 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-17T05:24:39.016 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-17T05:24:39.065 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-17T05:24:39.065 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-17T05:24:39.113 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-17T05:24:39.113 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-17T05:24:39.164 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-17T05:24:39.164 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-17T05:24:39.210 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-17T05:24:39.210 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-17T05:24:39.265 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-17T05:24:39.265 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-17T05:24:39.313 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-17T05:24:39.313 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-17T05:24:39.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7f5fbb600640 1 -- 172.21.15.122:0/798919001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb40ca2d0 msgr2=0x7f5fb40ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7f5fbb600640 1 --2- 172.21.15.122:0/798919001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb40ca2d0 0x7f5fb40ca780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4071160 tx=0x7f5fac010f70 comp rx=0 tx=0).stop 2024-09-17T05:24:39.369 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-17T05:24:39.369 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-17T05:24:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7ff2c7400640 1 -- 172.21.15.122:0/3927442161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2c01ea210 msgr2=0x7ff2c0155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7ff2c7400640 1 --2- 172.21.15.122:0/3927442161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2c01ea210 0x7ff2c0155780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80093c0 tx=0x7ff2b8010da0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.364+0000 7ff2c7400640 1 -- 172.21.15.122:0/3927442161 shutdown_connections 2024-09-17T05:24:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.364+0000 7ff2c7400640 1 --2- 172.21.15.122:0/3927442161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2c01ea210 0x7ff2c0155780 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.364+0000 7ff2c7400640 1 --2- 172.21.15.122:0/3927442161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2c00b18f0 0x7ff2c0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.364+0000 7ff2c7400640 1 --2- 172.21.15.122:0/3927442161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2a4003a60 0x7ff2c009ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.364+0000 7ff2c7400640 1 -- 172.21.15.122:0/3927442161 >> 172.21.15.122:0/3927442161 conn(0x7ff2c00cd740 msgr2=0x7ff2c00719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7f5fbb600640 1 -- 172.21.15.122:0/798919001 shutdown_connections 2024-09-17T05:24:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7f5fbb600640 1 --2- 172.21.15.122:0/798919001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb40ca2d0 0x7f5fb40ca780 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7f5fbb600640 1 --2- 172.21.15.122:0/798919001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fb4149cf0 0x7f5fb415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7f5fbb600640 1 --2- 172.21.15.122:0/798919001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f9c003a60 0x7f5fb415cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.360+0000 7f5fbb600640 1 -- 172.21.15.122:0/798919001 >> 172.21.15.122:0/798919001 conn(0x7f5fb406ecd0 msgr2=0x7f5fb40cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.364+0000 7ff2c7400640 1 -- 172.21.15.122:0/3927442161 shutdown_connections 2024-09-17T05:24:39.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.368+0000 7ff2c7400640 1 -- 172.21.15.122:0/3927442161 wait complete. 2024-09-17T05:24:39.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.368+0000 7f5fbb600640 1 -- 172.21.15.122:0/798919001 shutdown_connections 2024-09-17T05:24:39.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.368+0000 7f5fbb600640 1 -- 172.21.15.122:0/798919001 wait complete. 2024-09-17T05:24:39.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.368+0000 7f1b1a000640 1 -- 172.21.15.122:0/677007022 <== mon.2 v2:172.21.15.173:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f1b10026480 con 0x7f1b1c005a10 2024-09-17T05:24:39.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.372+0000 7f3d18c00640 1 -- 172.21.15.122:0/4050560567 <== mon.2 v2:172.21.15.173:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f3d080bc520 con 0x7f3d1c0ce530 2024-09-17T05:24:39.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.372+0000 7f3d18c00640 1 -- 172.21.15.122:0/4050560567 <== mon.2 v2:172.21.15.173:3301/0 11 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v5)=0 v5) ==== 114+0+0 (secure 0 0 0) 0x7f3d08019340 con 0x7f3d1c0ce530 2024-09-17T05:24:39.377 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-17T05:24:39.377 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-17T05:24:39.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.372+0000 7f135a000640 1 -- 172.21.15.122:0/2991950191 <== mon.1 v2:172.21.15.173:3300/0 9 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f135400d400 con 0x7f136406fd30 2024-09-17T05:24:39.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.388+0000 7f3d23c00640 1 -- 172.21.15.122:0/4050560567 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d1c0ce530 msgr2=0x7f3d1c06ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.388+0000 7f3d23c00640 1 --2- 172.21.15.122:0/4050560567 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d1c0ce530 0x7f3d1c06ef30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3d08006e90 tx=0x7f3d04005840 comp rx=0 tx=0).stop 2024-09-17T05:24:39.393 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-17T05:24:39.393 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-17T05:24:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.396+0000 7f3d23c00640 1 -- 172.21.15.122:0/4050560567 shutdown_connections 2024-09-17T05:24:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.396+0000 7f3d23c00640 1 --2- 172.21.15.122:0/4050560567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d1c06f470 0x7f3d1c0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.396+0000 7f3d23c00640 1 --2- 172.21.15.122:0/4050560567 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d1c0ce530 0x7f3d1c06ef30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.396+0000 7f3d23c00640 1 --2- 172.21.15.122:0/4050560567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d1c0c36b0 0x7f3d1c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.396+0000 7f3d23c00640 1 -- 172.21.15.122:0/4050560567 >> 172.21.15.122:0/4050560567 conn(0x7f3d1c0b2060 msgr2=0x7f3d1c0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.409 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-17T05:24:39.409 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-17T05:24:39.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.408+0000 7f3d23c00640 1 -- 172.21.15.122:0/4050560567 shutdown_connections 2024-09-17T05:24:39.415 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-17T05:24:39.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-17T05:24:39.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.408+0000 7f3d23c00640 1 -- 172.21.15.122:0/4050560567 wait complete. 2024-09-17T05:24:39.436 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-17T05:24:39.437 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-17T05:24:39.499 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-17T05:24:39.499 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-17T05:24:39.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.520+0000 7f32ae400640 1 -- 172.21.15.122:0/1778604032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a0035f10 msgr2=0x7f32a00363a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.520+0000 7f32ae400640 1 --2- 172.21.15.122:0/1778604032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a0035f10 0x7f32a00363a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3294009900 tx=0x7f329401c8d0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.520+0000 7f1b23200640 1 -- 172.21.15.122:0/677007022 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f1af8005320 con 0x7f1b1c005a10 2024-09-17T05:24:39.529 DEBUG:tasks.ceph:set 18 configs 2024-09-17T05:24:39.529 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.529 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.529 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.529 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.529 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.528+0000 7f32ae400640 1 -- 172.21.15.122:0/1778604032 shutdown_connections 2024-09-17T05:24:39.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.528+0000 7f32ae400640 1 --2- 172.21.15.122:0/1778604032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a0035f10 0x7f32a00363a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.528+0000 7f32ae400640 1 --2- 172.21.15.122:0/1778604032 >> v2:172.21.15.173:3300/0 conn(0x7f32a00e2ee0 0x7f32a0034340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.528+0000 7f32ae400640 1 -- 172.21.15.122:0/1778604032 >> 172.21.15.122:0/1778604032 conn(0x7f32a001d780 msgr2=0x7f32a0054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.528+0000 7f32ae400640 1 -- 172.21.15.122:0/1778604032 shutdown_connections 2024-09-17T05:24:39.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.528+0000 7f32ae400640 1 -- 172.21.15.122:0/1778604032 wait complete. 2024-09-17T05:24:39.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.528+0000 7f32ae400640 1 Processor -- start 2024-09-17T05:24:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.532+0000 7f32ae400640 1 -- start start 2024-09-17T05:24:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32ae400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a0005a10 0x7f32a00c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32ae400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a0035f10 0x7f32a004fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32ae400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a00e2ee0 0x7f32a00dca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32ae400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0033900 con 0x7f32a0035f10 2024-09-17T05:24:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32ae400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f32a001dc50 con 0x7f32a0005a10 2024-09-17T05:24:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32ae400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f32a0033a80 con 0x7f32a00e2ee0 2024-09-17T05:24:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a00e2ee0 0x7f32a00dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a00e2ee0 0x7f32a00dca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:40958/0 (socket says 172.21.15.122:40958) 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 -- 172.21.15.122:0/741067931 learned_addr learned my addr 172.21.15.122:0/741067931 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7400640 1 --2- 172.21.15.122:0/741067931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a0005a10 0x7f32a00c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 -- 172.21.15.122:0/741067931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a0005a10 msgr2=0x7f32a00c05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 --2- 172.21.15.122:0/741067931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a0005a10 0x7f32a00c05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 -- 172.21.15.122:0/741067931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a0035f10 msgr2=0x7f32a004fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 --2- 172.21.15.122:0/741067931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a0035f10 0x7f32a004fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 -- 172.21.15.122:0/741067931 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a0050920 con 0x7f32a00e2ee0 2024-09-17T05:24:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.536+0000 7f32a7e00640 1 --2- 172.21.15.122:0/741067931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a00e2ee0 0x7f32a00dca00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f32a806b760 tx=0x7f32a806a190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.540+0000 7f1b1a000640 1 -- 172.21.15.122:0/677007022 <== mon.2 v2:172.21.15.173:3301/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b10005020 con 0x7f1b1c005a10 2024-09-17T05:24:39.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.540+0000 7f1b1a000640 1 -- 172.21.15.122:0/677007022 <== mon.2 v2:172.21.15.173:3301/0 12 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f1b10005360 con 0x7f1b1c005a10 2024-09-17T05:24:39.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.540+0000 7f329b400640 1 -- 172.21.15.122:0/741067931 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a808f070 con 0x7f32a00e2ee0 2024-09-17T05:24:39.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.540+0000 7f329b400640 1 -- 172.21.15.122:0/741067931 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a8064350 con 0x7f32a00e2ee0 2024-09-17T05:24:39.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.540+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a00c1150 con 0x7f32a00e2ee0 2024-09-17T05:24:39.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.540+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a0050640 con 0x7f32a00e2ee0 2024-09-17T05:24:39.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.540+0000 7f134f400640 1 -- 172.21.15.122:0/2991950191 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f13640cc420 con 0x7f136406fd30 2024-09-17T05:24:39.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.544+0000 7f329b400640 1 -- 172.21.15.122:0/741067931 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a806a4d0 con 0x7f32a00e2ee0 2024-09-17T05:24:39.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.544+0000 7f329b400640 1 -- 172.21.15.122:0/741067931 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f32a8063d10 con 0x7f32a00e2ee0 2024-09-17T05:24:39.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.544+0000 7f329b400640 1 -- 172.21.15.122:0/741067931 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f32a8094900 con 0x7f32a00e2ee0 2024-09-17T05:24:39.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.540+0000 7f135a000640 1 -- 172.21.15.122:0/2991950191 <== mon.1 v2:172.21.15.173:3300/0 10 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f135400d620 con 0x7f136406fd30 2024-09-17T05:24:39.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.544+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a00363a0 con 0x7f32a00e2ee0 2024-09-17T05:24:39.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 -- 172.21.15.122:0/677007022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c005a10 msgr2=0x7f1b1c032690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 --2- 172.21.15.122:0/677007022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c005a10 0x7f1b1c032690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000d950 tx=0x7f1b1000de10 comp rx=0 tx=0).stop 2024-09-17T05:24:39.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 -- 172.21.15.122:0/677007022 shutdown_connections 2024-09-17T05:24:39.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 --2- 172.21.15.122:0/677007022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b1c0e2ee0 0x7f1b1c0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 --2- 172.21.15.122:0/677007022 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b1c035f10 0x7f1b1c0d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 --2- 172.21.15.122:0/677007022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c005a10 0x7f1b1c032690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 -- 172.21.15.122:0/677007022 >> 172.21.15.122:0/677007022 conn(0x7f1b1c01d780 msgr2=0x7f1b1c0e1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 -- 172.21.15.122:0/677007022 shutdown_connections 2024-09-17T05:24:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.552+0000 7f1b23200640 1 -- 172.21.15.122:0/677007022 wait complete. 2024-09-17T05:24:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.556+0000 7f329b400640 1 -- 172.21.15.122:0/741067931 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f32a8094b00 con 0x7f32a00e2ee0 2024-09-17T05:24:39.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.572+0000 7f135a000640 1 -- 172.21.15.122:0/2991950191 <== mon.1 v2:172.21.15.173:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f1354013030 con 0x7f136406fd30 2024-09-17T05:24:39.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.580+0000 7f53b1a00640 1 -- 172.21.15.122:0/2706034646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53ac070420 msgr2=0x7f53ac0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.580+0000 7f53b1a00640 1 --2- 172.21.15.122:0/2706034646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53ac070420 0x7f53ac0cebf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0751a0 tx=0x7f53a401cc60 comp rx=0 tx=0).stop 2024-09-17T05:24:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 -- 172.21.15.122:0/2991950191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f136406fd30 msgr2=0x7f13640c3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 --2- 172.21.15.122:0/2991950191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f136406fd30 0x7f13640c3560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f135400b560 tx=0x7f135400ba20 comp rx=0 tx=0).stop 2024-09-17T05:24:39.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f53b1a00640 1 -- 172.21.15.122:0/2706034646 shutdown_connections 2024-09-17T05:24:39.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f53b1a00640 1 --2- 172.21.15.122:0/2706034646 >> v2:172.21.15.122:3300/0 conn(0x7f53ac0cc5f0 0x7f53ac0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f53b1a00640 1 --2- 172.21.15.122:0/2706034646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53ac070420 0x7f53ac0cebf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f53b1a00640 1 -- 172.21.15.122:0/2706034646 >> 172.21.15.122:0/2706034646 conn(0x7f53ac0b18f0 msgr2=0x7f53ac0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 -- 172.21.15.122:0/2991950191 shutdown_connections 2024-09-17T05:24:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 --2- 172.21.15.122:0/2991950191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13640cb140 0x7f13640b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 --2- 172.21.15.122:0/2991950191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1364072c80 0x7f13640c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 --2- 172.21.15.122:0/2991950191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f136406fd30 0x7f13640c3560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 -- 172.21.15.122:0/2991950191 >> 172.21.15.122:0/2991950191 conn(0x7f13640b1600 msgr2=0x7f136415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 -- 172.21.15.122:0/2991950191 shutdown_connections 2024-09-17T05:24:39.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.588+0000 7f1368c00640 1 -- 172.21.15.122:0/2991950191 wait complete. 2024-09-17T05:24:39.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.592+0000 7f53b1a00640 1 -- 172.21.15.122:0/2706034646 shutdown_connections 2024-09-17T05:24:39.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.596+0000 7f53b1a00640 1 -- 172.21.15.122:0/2706034646 wait complete. 2024-09-17T05:24:39.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.596+0000 7f53b1a00640 1 Processor -- start 2024-09-17T05:24:39.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.596+0000 7f53b1a00640 1 -- start start 2024-09-17T05:24:39.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.600+0000 7f53b1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53ac070420 0x7f53ac15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.600+0000 7f53b1a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53ac144830 0x7f53ac0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.600+0000 7f53b1a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53ac0ad940 0x7f53ac1fb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.604+0000 7f53aaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53ac070420 0x7f53ac15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.604+0000 7f53aaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53ac070420 0x7f53ac15d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45724/0 (socket says 172.21.15.122:45724) 2024-09-17T05:24:39.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.604+0000 7f53aa000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53ac144830 0x7f53ac0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.604+0000 7f53aa000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53ac144830 0x7f53ac0ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:40966/0 (socket says 172.21.15.122:40966) 2024-09-17T05:24:39.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.604+0000 7f53aaa00640 1 -- 172.21.15.122:0/488923721 learned_addr learned my addr 172.21.15.122:0/488923721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.604+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac1684e0 con 0x7f53ac070420 2024-09-17T05:24:39.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.604+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac168360 con 0x7f53ac0ad940 2024-09-17T05:24:39.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.604+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f53ac168660 con 0x7f53ac144830 2024-09-17T05:24:39.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53aaa00640 1 -- 172.21.15.122:0/488923721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53ac144830 msgr2=0x7f53ac0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53aaa00640 1 --2- 172.21.15.122:0/488923721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53ac144830 0x7f53ac0ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53aaa00640 1 -- 172.21.15.122:0/488923721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53ac0ad940 msgr2=0x7f53ac1fb7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:24:39.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53aaa00640 1 --2- 172.21.15.122:0/488923721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53ac0ad940 0x7f53ac1fb7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53aaa00640 1 -- 172.21.15.122:0/488923721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac1fc4a0 con 0x7f53ac070420 2024-09-17T05:24:39.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53aaa00640 1 --2- 172.21.15.122:0/488923721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53ac070420 0x7f53ac15d710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53a4000f90 tx=0x7f53a4009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53a2a00640 1 -- 172.21.15.122:0/488923721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a40bb070 con 0x7f53ac070420 2024-09-17T05:24:39.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53a2a00640 1 -- 172.21.15.122:0/488923721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53a4004d60 con 0x7f53ac070420 2024-09-17T05:24:39.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.608+0000 7f53a2a00640 1 -- 172.21.15.122:0/488923721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a40b2040 con 0x7f53ac070420 2024-09-17T05:24:39.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.612+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ac1332f0 con 0x7f53ac070420 2024-09-17T05:24:39.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.612+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ac15dc50 con 0x7f53ac070420 2024-09-17T05:24:39.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.612+0000 7f53a2a00640 1 -- 172.21.15.122:0/488923721 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53a4004260 con 0x7f53ac070420 2024-09-17T05:24:39.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.612+0000 7f53a2a00640 1 -- 172.21.15.122:0/488923721 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53a4005490 con 0x7f53ac070420 2024-09-17T05:24:39.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.616+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ac0add50 con 0x7f53ac070420 2024-09-17T05:24:39.628 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.632+0000 7f53a2a00640 1 -- 172.21.15.122:0/488923721 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f53ac0add50 con 0x7f53ac070420 2024-09-17T05:24:39.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.648+0000 7f3c44600640 1 -- 172.21.15.122:0/1566428708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c06ede0 msgr2=0x7f3c3c15c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.648+0000 7f3c44600640 1 --2- 172.21.15.122:0/1566428708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c06ede0 0x7f3c3c15c970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009900 tx=0x7f3c2801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.648+0000 7f3c44600640 1 -- 172.21.15.122:0/1566428708 shutdown_connections 2024-09-17T05:24:39.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.648+0000 7f3c44600640 1 --2- 172.21.15.122:0/1566428708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c06ede0 0x7f3c3c15c970 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009900 tx=0x7f3c2801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.648+0000 7f3c44600640 1 -- 172.21.15.122:0/1566428708 >> 172.21.15.122:0/1566428708 conn(0x7f3c3c0b18f0 msgr2=0x7f3c3c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.652+0000 7f3c44600640 1 -- 172.21.15.122:0/1566428708 shutdown_connections 2024-09-17T05:24:39.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.652+0000 7f3c44600640 1 -- 172.21.15.122:0/1566428708 wait complete. 2024-09-17T05:24:39.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.652+0000 7f3c44600640 1 Processor -- start 2024-09-17T05:24:39.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.656+0000 7f3c44600640 1 -- start start 2024-09-17T05:24:39.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c44600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3c06ea00 0x7f3c3c072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c44600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c15ceb0 0x7f3c3c074210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c44600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c148220 0x7f3c3c073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c15ceb0 0x7f3c3c074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3c06ea00 0x7f3c3c072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c15ceb0 0x7f3c3c074210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:53058/0 (socket says 172.21.15.122:53058) 2024-09-17T05:24:39.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c44600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c160ff0 con 0x7f3c3c148220 2024-09-17T05:24:39.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 -- 172.21.15.122:0/2250211536 learned_addr learned my addr 172.21.15.122:0/2250211536 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3c160e70 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3c3c161170 con 0x7f3c3c06ea00 2024-09-17T05:24:39.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c42600640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c148220 0x7f3c3c073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 -- 172.21.15.122:0/2250211536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3c06ea00 msgr2=0x7f3c3c072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3c06ea00 0x7f3c3c072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 -- 172.21.15.122:0/2250211536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c148220 msgr2=0x7f3c3c073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c148220 0x7f3c3c073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 -- 172.21.15.122:0/2250211536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3c14fd20 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41c00640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3c06ea00 0x7f3c3c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:39.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c42600640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c148220 0x7f3c3c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:39.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c41200640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c15ceb0 0x7f3c3c074210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c2801af90 tx=0x7f3c280a5720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c39600640 1 -- 172.21.15.122:0/2250211536 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c280b2040 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c39600640 1 -- 172.21.15.122:0/2250211536 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c28004230 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c39600640 1 -- 172.21.15.122:0/2250211536 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28003600 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3c150000 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3c1486d0 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c39600640 1 -- 172.21.15.122:0/2250211536 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c280bf020 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.660+0000 7f3c39600640 1 -- 172.21.15.122:0/2250211536 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3c280a6020 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.668+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c3c1616a0 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.668+0000 7f3c39600640 1 -- 172.21.15.122:0/2250211536 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3c280c6020 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 -- 172.21.15.122:0/1881087510 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb800ae4d0 msgr2=0x7feb800ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 --2- 172.21.15.122:0/1881087510 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb800ae4d0 0x7feb800ae910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb800c3330 tx=0x7feb7801cda0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 -- 172.21.15.122:0/1881087510 shutdown_connections 2024-09-17T05:24:39.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 --2- 172.21.15.122:0/1881087510 >> v2:172.21.15.173:3300/0 conn(0x7feb8015eeb0 0x7feb800cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 --2- 172.21.15.122:0/1881087510 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb800ae4d0 0x7feb800ae910 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 --2- 172.21.15.122:0/1881087510 >> v2:172.21.15.122:3300/0 conn(0x7feb800cd5d0 0x7feb800ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 -- 172.21.15.122:0/1881087510 >> 172.21.15.122:0/1881087510 conn(0x7feb800cfa00 msgr2=0x7feb800cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 -- 172.21.15.122:0/1881087510 shutdown_connections 2024-09-17T05:24:39.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.692+0000 7feb84c00640 1 -- 172.21.15.122:0/1881087510 wait complete. 2024-09-17T05:24:39.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.696+0000 7feb84c00640 1 Processor -- start 2024-09-17T05:24:39.702 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.700+0000 7feb84c00640 1 -- start start 2024-09-17T05:24:39.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.700+0000 7feb84c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb800ae4d0 0x7feb80152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb84c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb800cd5d0 0x7feb800c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb84c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb8015eeb0 0x7feb800b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb84c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb80154770 con 0x7feb800cd5d0 2024-09-17T05:24:39.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb84c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb801545f0 con 0x7feb800ae4d0 2024-09-17T05:24:39.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb84c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7feb801548f0 con 0x7feb8015eeb0 2024-09-17T05:24:39.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb800cd5d0 0x7feb800c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb800cd5d0 0x7feb800c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45758/0 (socket says 172.21.15.122:45758) 2024-09-17T05:24:39.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7ea00640 1 -- 172.21.15.122:0/1677969705 learned_addr learned my addr 172.21.15.122:0/1677969705 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7fe00640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb8015eeb0 0x7feb800b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7f400640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb800ae4d0 0x7feb80152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7fe00640 1 -- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb800ae4d0 msgr2=0x7feb80152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7fe00640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb800ae4d0 0x7feb80152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7fe00640 1 -- 172.21.15.122:0/1677969705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb800cd5d0 msgr2=0x7feb800c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7fe00640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb800cd5d0 0x7feb800c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7fe00640 1 -- 172.21.15.122:0/1677969705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb801558d0 con 0x7feb8015eeb0 2024-09-17T05:24:39.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb7fe00640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb8015eeb0 0x7feb800b0880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feb780ac000 tx=0x7feb78005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb77400640 1 -- 172.21.15.122:0/1677969705 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780ba020 con 0x7feb8015eeb0 2024-09-17T05:24:39.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb77400640 1 -- 172.21.15.122:0/1677969705 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb78004250 con 0x7feb8015eeb0 2024-09-17T05:24:39.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb77400640 1 -- 172.21.15.122:0/1677969705 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780b2070 con 0x7feb8015eeb0 2024-09-17T05:24:39.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb80155bb0 con 0x7feb8015eeb0 2024-09-17T05:24:39.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb80156170 con 0x7feb8015eeb0 2024-09-17T05:24:39.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb77400640 1 -- 172.21.15.122:0/1677969705 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb780bd020 con 0x7feb8015eeb0 2024-09-17T05:24:39.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.708+0000 7feb77400640 1 -- 172.21.15.122:0/1677969705 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb780add30 con 0x7feb8015eeb0 2024-09-17T05:24:39.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.716+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb800ae910 con 0x7feb8015eeb0 2024-09-17T05:24:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.716+0000 7feb77400640 1 -- 172.21.15.122:0/1677969705 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7feb780c4030 con 0x7feb8015eeb0 2024-09-17T05:24:39.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.716+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f32a00da570 con 0x7f32a00e2ee0 2024-09-17T05:24:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.736+0000 7f9bee600640 1 -- 172.21.15.122:0/2222844362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be00e2b90 msgr2=0x7f9be0005a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.736+0000 7f9bee600640 1 --2- 172.21.15.122:0/2222844362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be00e2b90 0x7f9be0005a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9be000baf0 tx=0x7f9bd401c890 comp rx=0 tx=0).stop 2024-09-17T05:24:39.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.740+0000 7f9bee600640 1 -- 172.21.15.122:0/2222844362 shutdown_connections 2024-09-17T05:24:39.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.740+0000 7f9bee600640 1 --2- 172.21.15.122:0/2222844362 >> v2:172.21.15.122:3300/0 conn(0x7f9be001d9c0 0x7f9be000ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.740+0000 7f9bee600640 1 --2- 172.21.15.122:0/2222844362 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9be00e2f70 0x7f9be001d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.740+0000 7f9bee600640 1 --2- 172.21.15.122:0/2222844362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be00e2b90 0x7f9be0005a70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.740+0000 7f9bee600640 1 -- 172.21.15.122:0/2222844362 >> 172.21.15.122:0/2222844362 conn(0x7f9be0054570 msgr2=0x7f9be0054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.740+0000 7f9bee600640 1 -- 172.21.15.122:0/2222844362 shutdown_connections 2024-09-17T05:24:39.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.740+0000 7f9bee600640 1 -- 172.21.15.122:0/2222844362 wait complete. 2024-09-17T05:24:39.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.740+0000 7f9bee600640 1 Processor -- start 2024-09-17T05:24:39.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.744+0000 7f9bee600640 1 -- start start 2024-09-17T05:24:39.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.748+0000 7f329b400640 1 -- 172.21.15.122:0/741067931 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f32a8094ce0 con 0x7f32a00e2ee0 2024-09-17T05:24:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bee600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be001d9c0 0x7f9be001c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bee600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9be00e2b90 0x7f9be0021990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bee600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be00e2f70 0x7f9be001e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bee600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9be00e3770 con 0x7f9be00e2f70 2024-09-17T05:24:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bee600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9be00e35f0 con 0x7f9be001d9c0 2024-09-17T05:24:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bee600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9be00e38f0 con 0x7f9be00e2b90 2024-09-17T05:24:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bed200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be001d9c0 0x7f9be001c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9be00e2b90 0x7f9be0021990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bed200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be001d9c0 0x7f9be001c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:53104/0 (socket says 172.21.15.122:53104) 2024-09-17T05:24:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bed200640 1 -- 172.21.15.122:0/57476394 learned_addr learned my addr 172.21.15.122:0/57476394 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be7e00640 1 -- 172.21.15.122:0/57476394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be001d9c0 msgr2=0x7f9be001c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be7e00640 1 --2- 172.21.15.122:0/57476394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be001d9c0 0x7f9be001c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be7e00640 1 -- 172.21.15.122:0/57476394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be00e2f70 msgr2=0x7f9be001e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be7e00640 1 --2- 172.21.15.122:0/57476394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be00e2f70 0x7f9be001e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be7e00640 1 -- 172.21.15.122:0/57476394 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be0020c20 con 0x7f9be00e2b90 2024-09-17T05:24:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be7e00640 1 --2- 172.21.15.122:0/57476394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9be00e2b90 0x7f9be0021990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9be806b800 tx=0x7f9be806a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be4c00640 1 -- 172.21.15.122:0/57476394 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8090070 con 0x7f9be00e2b90 2024-09-17T05:24:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be4c00640 1 -- 172.21.15.122:0/57476394 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9be8074700 con 0x7f9be00e2b90 2024-09-17T05:24:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be4c00640 1 -- 172.21.15.122:0/57476394 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8074b90 con 0x7f9be00e2b90 2024-09-17T05:24:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9bee600640 1 -- 172.21.15.122:0/57476394 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be001f4c0 con 0x7f9be00e2b90 2024-09-17T05:24:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.752+0000 7f9be4c00640 1 -- 172.21.15.122:0/57476394 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9be8093070 con 0x7f9be00e2b90 2024-09-17T05:24:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a00e2ee0 msgr2=0x7f32a00dca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 --2- 172.21.15.122:0/741067931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a00e2ee0 0x7f32a00dca00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f32a806b760 tx=0x7f32a806a190 comp rx=0 tx=0).stop 2024-09-17T05:24:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 shutdown_connections 2024-09-17T05:24:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 --2- 172.21.15.122:0/741067931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a00e2ee0 0x7f32a00dca00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 --2- 172.21.15.122:0/741067931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a0035f10 0x7f32a004fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 --2- 172.21.15.122:0/741067931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a0005a10 0x7f32a00c05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 >> 172.21.15.122:0/741067931 conn(0x7f32a001d780 msgr2=0x7f32a00dd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 shutdown_connections 2024-09-17T05:24:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f32ae400640 1 -- 172.21.15.122:0/741067931 wait complete. 2024-09-17T05:24:39.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f9bee600640 1 -- 172.21.15.122:0/57476394 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be0021190 con 0x7f9be00e2b90 2024-09-17T05:24:39.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f9be4c00640 1 -- 172.21.15.122:0/57476394 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9be8074d30 con 0x7f9be00e2b90 2024-09-17T05:24:39.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.760+0000 7f9bda000640 1 -- 172.21.15.122:0/57476394 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be0005a70 con 0x7f9be00e2b90 2024-09-17T05:24:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f3620600640 1 -- 172.21.15.122:0/898671809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610053180 msgr2=0x7f36100e30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f3620600640 1 --2- 172.21.15.122:0/898671809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610053180 0x7f36100e30e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3610054480 tx=0x7f360c020260 comp rx=0 tx=0).stop 2024-09-17T05:24:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f3620600640 1 -- 172.21.15.122:0/898671809 shutdown_connections 2024-09-17T05:24:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f3620600640 1 --2- 172.21.15.122:0/898671809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610053180 0x7f36100e30e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.756+0000 7f3620600640 1 -- 172.21.15.122:0/898671809 >> 172.21.15.122:0/898671809 conn(0x7f361001d7b0 msgr2=0x7f36100545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.760+0000 7f3620600640 1 -- 172.21.15.122:0/898671809 shutdown_connections 2024-09-17T05:24:39.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.760+0000 7f3620600640 1 -- 172.21.15.122:0/898671809 wait complete. 2024-09-17T05:24:39.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.764+0000 7f9be4c00640 1 -- 172.21.15.122:0/57476394 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9be809f440 con 0x7f9be00e2b90 2024-09-17T05:24:39.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.768+0000 7f3620600640 1 Processor -- start 2024-09-17T05:24:39.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.768+0000 7f3620600640 1 -- start start 2024-09-17T05:24:39.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.768+0000 7f3620600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610006150 0x7f36100d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.768+0000 7f3620600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3610053180 0x7f3610020790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.768+0000 7f3620600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3610053f20 0x7f3610020cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.768+0000 7f3620600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3610007020 con 0x7f3610053f20 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.768+0000 7f3620600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3610054ec0 con 0x7f3610006150 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.768+0000 7f3620600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f36100071a0 con 0x7f3610053180 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.772+0000 7f361dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610006150 0x7f36100d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.772+0000 7f361dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610006150 0x7f36100d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:53118/0 (socket says 172.21.15.122:53118) 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.772+0000 7f361dc00640 1 -- 172.21.15.122:0/2739819162 learned_addr learned my addr 172.21.15.122:0/2739819162 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.772+0000 7f361d200640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3610053180 0x7f3610020790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.772+0000 7f361dc00640 1 -- 172.21.15.122:0/2739819162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3610053180 msgr2=0x7f3610020790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.772+0000 7f361dc00640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3610053180 0x7f3610020790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.772+0000 7f361dc00640 1 -- 172.21.15.122:0/2739819162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3610053f20 msgr2=0x7f3610020cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:24:39.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.772+0000 7f361e600640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3610053f20 0x7f3610020cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.776+0000 7f361dc00640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3610053f20 0x7f3610020cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.776+0000 7f361dc00640 1 -- 172.21.15.122:0/2739819162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36100215a0 con 0x7f3610006150 2024-09-17T05:24:39.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.776+0000 7f361e600640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3610053f20 0x7f3610020cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:39.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.776+0000 7f361dc00640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610006150 0x7f36100d3cb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3618066720 tx=0x7f361806a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7f3615600640 1 -- 172.21.15.122:0/2739819162 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618090020 con 0x7f3610006150 2024-09-17T05:24:39.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7f3615600640 1 -- 172.21.15.122:0/2739819162 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3618072700 con 0x7f3610006150 2024-09-17T05:24:39.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7f3615600640 1 -- 172.21.15.122:0/2739819162 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618072bd0 con 0x7f3610006150 2024-09-17T05:24:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7fcd4e200640 1 -- 172.21.15.122:0/4232187710 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd2c0053d0 msgr2=0x7fcd2c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7fcd4e200640 1 --2- 172.21.15.122:0/4232187710 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd2c0053d0 0x7fcd2c00ba10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009890 tx=0x7fcd3401c6c0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7fcd4e200640 1 -- 172.21.15.122:0/4232187710 shutdown_connections 2024-09-17T05:24:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7fcd4e200640 1 --2- 172.21.15.122:0/4232187710 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd2c0053d0 0x7fcd2c00ba10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7fcd4e200640 1 -- 172.21.15.122:0/4232187710 >> 172.21.15.122:0/4232187710 conn(0x7fcd480b18f0 msgr2=0x7fcd480b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3610021880 con 0x7f3610006150 2024-09-17T05:24:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3610049520 con 0x7f3610006150 2024-09-17T05:24:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7fcd4e200640 1 -- 172.21.15.122:0/4232187710 shutdown_connections 2024-09-17T05:24:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7f3615600640 1 -- 172.21.15.122:0/2739819162 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3618093020 con 0x7f3610006150 2024-09-17T05:24:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.780+0000 7f3615600640 1 -- 172.21.15.122:0/2739819162 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f36180943b0 con 0x7f3610006150 2024-09-17T05:24:39.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.784+0000 7fcd4e200640 1 -- 172.21.15.122:0/4232187710 wait complete. 2024-09-17T05:24:39.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.784+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36100e30e0 con 0x7f3610006150 2024-09-17T05:24:39.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.784+0000 7fcd4e200640 1 Processor -- start 2024-09-17T05:24:39.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.788+0000 7f3615600640 1 -- 172.21.15.122:0/2739819162 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f361809a070 con 0x7f3610006150 2024-09-17T05:24:39.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.792+0000 7fcd4e200640 1 -- start start 2024-09-17T05:24:39.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.792+0000 7fcd4e200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd2c0053d0 0x7fcd480741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.792+0000 7fcd4e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd4814d990 0x7fcd4814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.792+0000 7fcd4e200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd4814d440 0x7fcd4815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.792+0000 7fcd47400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd2c0053d0 0x7fcd480741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.792+0000 7fcd47400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd2c0053d0 0x7fcd480741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:53128/0 (socket says 172.21.15.122:53128) 2024-09-17T05:24:39.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.792+0000 7fcd46a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd4814d990 0x7fcd4814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.792+0000 7fcd46a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd4814d990 0x7fcd4814cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45814/0 (socket says 172.21.15.122:45814) 2024-09-17T05:24:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd4e200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd480b40a0 con 0x7fcd4814d990 2024-09-17T05:24:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd4e200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd480b3f20 con 0x7fcd2c0053d0 2024-09-17T05:24:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd4e200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcd480b4220 con 0x7fcd4814d440 2024-09-17T05:24:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd4814d440 0x7fcd4815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd4814d440 0x7fcd4815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:41040/0 (socket says 172.21.15.122:41040) 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 -- 172.21.15.122:0/1301364729 learned_addr learned my addr 172.21.15.122:0/1301364729 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 -- 172.21.15.122:0/1301364729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd2c0053d0 msgr2=0x7fcd480741b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd2c0053d0 0x7fcd480741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 -- 172.21.15.122:0/1301364729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd4814d990 msgr2=0x7fcd4814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd4814d990 0x7fcd4814cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 -- 172.21.15.122:0/1301364729 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd480746f0 con 0x7fcd4814d440 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd47e00640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd4814d440 0x7fcd4815bbc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd3401c4f0 tx=0x7fcd340a7f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd3f400640 1 -- 172.21.15.122:0/1301364729 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34005030 con 0x7fcd4814d440 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd3f400640 1 -- 172.21.15.122:0/1301364729 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd340bd070 con 0x7fcd4814d440 2024-09-17T05:24:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd3f400640 1 -- 172.21.15.122:0/1301364729 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34005540 con 0x7fcd4814d440 2024-09-17T05:24:39.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4814d240 con 0x7fcd4814d440 2024-09-17T05:24:39.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd48153750 con 0x7fcd4814d440 2024-09-17T05:24:39.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd3f400640 1 -- 172.21.15.122:0/1301364729 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd34004710 con 0x7fcd4814d440 2024-09-17T05:24:39.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.796+0000 7fcd3f400640 1 -- 172.21.15.122:0/1301364729 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd340c2900 con 0x7fcd4814d440 2024-09-17T05:24:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.804+0000 7fcd47400640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd2c0053d0 0x7fcd480741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-17T05:24:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.804+0000 7fcd46a00640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd4814d990 0x7fcd4814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-17T05:24:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.804+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4815cc60 con 0x7fcd4814d440 2024-09-17T05:24:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.804+0000 7fcd3f400640 1 -- 172.21.15.122:0/1301364729 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcd4814d240 con 0x7fcd4814d440 2024-09-17T05:24:39.815 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.824+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f3c3c074b30 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.828+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f53ac15e890 con 0x7f53ac070420 2024-09-17T05:24:39.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7efd47200640 1 -- 172.21.15.122:0/1108143273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd48074540 msgr2=0x7efd480ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7efd47200640 1 --2- 172.21.15.122:0/1108143273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd48074540 0x7efd480ca050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd34009900 tx=0x7efd3401c8d0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7f53a2a00640 1 -- 172.21.15.122:0/488923721 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f53a40c4020 con 0x7f53ac070420 2024-09-17T05:24:39.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7efd47200640 1 -- 172.21.15.122:0/1108143273 shutdown_connections 2024-09-17T05:24:39.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7efd47200640 1 --2- 172.21.15.122:0/1108143273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd48074540 0x7efd480ca050 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7efd47200640 1 --2- 172.21.15.122:0/1108143273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd4815e300 0x7efd480cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7efd47200640 1 -- 172.21.15.122:0/1108143273 >> 172.21.15.122:0/1108143273 conn(0x7efd480cfa00 msgr2=0x7efd480cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7efd47200640 1 -- 172.21.15.122:0/1108143273 shutdown_connections 2024-09-17T05:24:39.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.840+0000 7efd47200640 1 -- 172.21.15.122:0/1108143273 wait complete. 2024-09-17T05:24:39.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53ac070420 msgr2=0x7f53ac15d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7f53b1a00640 1 --2- 172.21.15.122:0/488923721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53ac070420 0x7f53ac15d710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53a4000f90 tx=0x7f53a4009340 comp rx=0 tx=0).stop 2024-09-17T05:24:39.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7f3c39600640 1 -- 172.21.15.122:0/2250211536 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f3c280164d0 con 0x7f3c3c15ceb0 2024-09-17T05:24:39.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 shutdown_connections 2024-09-17T05:24:39.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7f53b1a00640 1 --2- 172.21.15.122:0/488923721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53ac0ad940 0x7f53ac1fb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7f53b1a00640 1 --2- 172.21.15.122:0/488923721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53ac144830 0x7f53ac0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7f53b1a00640 1 --2- 172.21.15.122:0/488923721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53ac070420 0x7f53ac15d710 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 >> 172.21.15.122:0/488923721 conn(0x7f53ac0b18f0 msgr2=0x7f53ac0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.844+0000 7efd47200640 1 Processor -- start 2024-09-17T05:24:39.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.848+0000 7efd47200640 1 -- start start 2024-09-17T05:24:39.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd47200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd48074540 0x7efd48158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd47200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd480cd5d0 0x7efd481fd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd47200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd4815e300 0x7efd480c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd480cd5d0 0x7efd481fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd46800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd4815e300 0x7efd480c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd480cd5d0 0x7efd481fd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:53138/0 (socket says 172.21.15.122:53138) 2024-09-17T05:24:39.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd46800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd4815e300 0x7efd480c2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45818/0 (socket says 172.21.15.122:45818) 2024-09-17T05:24:39.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd48074540 0x7efd48158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45400640 1 -- 172.21.15.122:0/3364264427 learned_addr learned my addr 172.21.15.122:0/3364264427 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c15ceb0 msgr2=0x7f3c3c074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7f3c44600640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c15ceb0 0x7f3c3c074210 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c2801af90 tx=0x7f3c280a5720 comp rx=0 tx=0).stop 2024-09-17T05:24:39.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 shutdown_connections 2024-09-17T05:24:39.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7f3c44600640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c3c148220 0x7f3c3c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7f3c44600640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c3c15ceb0 0x7f3c3c074210 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7f3c44600640 1 --2- 172.21.15.122:0/2250211536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3c06ea00 0x7f3c3c072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 >> 172.21.15.122:0/2250211536 conn(0x7f3c3c0b18f0 msgr2=0x7f3c3c15a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 shutdown_connections 2024-09-17T05:24:39.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7f3c44600640 1 -- 172.21.15.122:0/2250211536 wait complete. 2024-09-17T05:24:39.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 shutdown_connections 2024-09-17T05:24:39.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7f53b1a00640 1 -- 172.21.15.122:0/488923721 wait complete. 2024-09-17T05:24:39.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd481682d0 con 0x7efd4815e300 2024-09-17T05:24:39.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45e00640 1 --2- 172.21.15.122:0/3364264427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd48074540 0x7efd48158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:41056/0 (socket says 172.21.15.122:41056) 2024-09-17T05:24:39.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd48168150 con 0x7efd480cd5d0 2024-09-17T05:24:39.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efd48168450 con 0x7efd48074540 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45e00640 1 -- 172.21.15.122:0/3364264427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd480cd5d0 msgr2=0x7efd481fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45e00640 1 --2- 172.21.15.122:0/3364264427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd480cd5d0 0x7efd481fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45e00640 1 -- 172.21.15.122:0/3364264427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd4815e300 msgr2=0x7efd480c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45e00640 1 --2- 172.21.15.122:0/3364264427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd4815e300 0x7efd480c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45e00640 1 -- 172.21.15.122:0/3364264427 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4815ce50 con 0x7efd48074540 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.852+0000 7efd45e00640 1 --2- 172.21.15.122:0/3364264427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd48074540 0x7efd48158ad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd38009f50 tx=0x7efd3800b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7efd3e000640 1 -- 172.21.15.122:0/3364264427 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38026020 con 0x7efd48074540 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7efd3e000640 1 -- 172.21.15.122:0/3364264427 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd38022640 con 0x7efd48074540 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7efd3e000640 1 -- 172.21.15.122:0/3364264427 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38022ab0 con 0x7efd48074540 2024-09-17T05:24:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4815d130 con 0x7efd48074540 2024-09-17T05:24:39.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.856+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4815d6f0 con 0x7efd48074540 2024-09-17T05:24:39.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.860+0000 7efd3e000640 1 -- 172.21.15.122:0/3364264427 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd38029020 con 0x7efd48074540 2024-09-17T05:24:39.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.860+0000 7efd3e000640 1 -- 172.21.15.122:0/3364264427 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd3802a740 con 0x7efd48074540 2024-09-17T05:24:39.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.860+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd480ca050 con 0x7efd48074540 2024-09-17T05:24:39.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.864+0000 7efd3e000640 1 -- 172.21.15.122:0/3364264427 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7efd38030020 con 0x7efd48074540 2024-09-17T05:24:39.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.876+0000 7fcc9ae00640 1 -- 172.21.15.122:0/1370520842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc7c0053d0 msgr2=0x7fcc7c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.876+0000 7fcc9ae00640 1 --2- 172.21.15.122:0/1370520842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc7c0053d0 0x7fcc7c00b9d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcc94072250 tx=0x7fcc8001eba0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.876+0000 7fcc9a400640 1 -- 172.21.15.122:0/1370520842 reap_dead start 2024-09-17T05:24:39.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.876+0000 7fcc9ae00640 1 -- 172.21.15.122:0/1370520842 shutdown_connections 2024-09-17T05:24:39.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.876+0000 7fcc9ae00640 1 -- 172.21.15.122:0/1370520842 >> 172.21.15.122:0/1370520842 conn(0x7fcc940cfa00 msgr2=0x7fcc940cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.880+0000 7fcc9ae00640 1 -- 172.21.15.122:0/1370520842 shutdown_connections 2024-09-17T05:24:39.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.880+0000 7fcc9ae00640 1 -- 172.21.15.122:0/1370520842 wait complete. 2024-09-17T05:24:39.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.884+0000 7fcc9ae00640 1 Processor -- start 2024-09-17T05:24:39.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9ae00640 1 -- start start 2024-09-17T05:24:39.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9ae00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc941f3fd0 0x7fcc940ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc99a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc941f3fd0 0x7fcc940ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc99a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc941f3fd0 0x7fcc940ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45828/0 (socket says 172.21.15.122:45828) 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc99a00640 1 -- 172.21.15.122:0/3471174566 learned_addr learned my addr 172.21.15.122:0/3471174566 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9ae00640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc941f43b0 0x7fcc9415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9ae00640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc9415e8f0 0x7fcc9415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9406f060 con 0x7fcc941f3fd0 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9406eee0 con 0x7fcc941f43b0 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcc9406f1e0 con 0x7fcc9415e8f0 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9a400640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc9415e8f0 0x7fcc9415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc99000640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc941f43b0 0x7fcc9415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9a400640 1 -- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc941f43b0 msgr2=0x7fcc9415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9a400640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc941f43b0 0x7fcc9415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9a400640 1 -- 172.21.15.122:0/3471174566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc941f3fd0 msgr2=0x7fcc940ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9a400640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc941f3fd0 0x7fcc940ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.892+0000 7fcc9a400640 1 -- 172.21.15.122:0/3471174566 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9415aed0 con 0x7fcc9415e8f0 2024-09-17T05:24:39.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.896+0000 7fcc9a400640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc9415e8f0 0x7fcc9415ed70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc800ae000 tx=0x7fcc800afc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.896+0000 7fcc91600640 1 -- 172.21.15.122:0/3471174566 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8000a5f0 con 0x7fcc9415e8f0 2024-09-17T05:24:39.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.896+0000 7fcc91600640 1 -- 172.21.15.122:0/3471174566 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc80000e50 con 0x7fcc9415e8f0 2024-09-17T05:24:39.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.896+0000 7fcc91600640 1 -- 172.21.15.122:0/3471174566 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80004280 con 0x7fcc9415e8f0 2024-09-17T05:24:39.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.896+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9415b170 con 0x7fcc9415e8f0 2024-09-17T05:24:39.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.900+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9415ce90 con 0x7fcc9415e8f0 2024-09-17T05:24:39.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.900+0000 7fcc91600640 1 -- 172.21.15.122:0/3471174566 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc80005020 con 0x7fcc9415e8f0 2024-09-17T05:24:39.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.900+0000 7fcc91600640 1 -- 172.21.15.122:0/3471174566 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc80004b40 con 0x7fcc9415e8f0 2024-09-17T05:24:39.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.904+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc940741d0 con 0x7fcc9415e8f0 2024-09-17T05:24:39.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.908+0000 7fcc91600640 1 -- 172.21.15.122:0/3471174566 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcc800188b0 con 0x7fcc9415e8f0 2024-09-17T05:24:39.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.916+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7feb80070440 con 0x7feb8015eeb0 2024-09-17T05:24:39.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.928+0000 7f036fe00640 1 -- 172.21.15.122:0/1574090790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0370072610 msgr2=0x7f037006eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.928+0000 7f036fe00640 1 --2- 172.21.15.122:0/1574090790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0370072610 0x7f037006eb90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03700768a0 tx=0x7f036801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.932+0000 7f036fe00640 1 -- 172.21.15.122:0/1574090790 shutdown_connections 2024-09-17T05:24:39.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.932+0000 7f036fe00640 1 --2- 172.21.15.122:0/1574090790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0370072610 0x7f037006eb90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.932+0000 7f036fe00640 1 --2- 172.21.15.122:0/1574090790 >> v2:172.21.15.122:3300/0 conn(0x7f03700cac60 0x7f037006e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.932+0000 7f036fe00640 1 -- 172.21.15.122:0/1574090790 >> 172.21.15.122:0/1574090790 conn(0x7f03700cfa00 msgr2=0x7f03700cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.932+0000 7f036fe00640 1 -- 172.21.15.122:0/1574090790 shutdown_connections 2024-09-17T05:24:39.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.932+0000 7feb77400640 1 -- 172.21.15.122:0/1677969705 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7feb78016990 con 0x7feb8015eeb0 2024-09-17T05:24:39.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.932+0000 7f036fe00640 1 -- 172.21.15.122:0/1574090790 wait complete. 2024-09-17T05:24:39.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb8015eeb0 msgr2=0x7feb800b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7feb84c00640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb8015eeb0 0x7feb800b0880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feb780ac000 tx=0x7feb78005c80 comp rx=0 tx=0).stop 2024-09-17T05:24:39.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 shutdown_connections 2024-09-17T05:24:39.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7feb84c00640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb8015eeb0 0x7feb800b0880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7f8feba00640 1 -- 172.21.15.122:0/3590957761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe4072c80 msgr2=0x7f8fe406fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7f8feba00640 1 --2- 172.21.15.122:0/3590957761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe4072c80 0x7f8fe406fd30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40af390 tx=0x7f8fdc01cd70 comp rx=0 tx=0).stop 2024-09-17T05:24:39.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7feb84c00640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb800cd5d0 0x7feb800c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7feb84c00640 1 --2- 172.21.15.122:0/1677969705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb800ae4d0 0x7feb80152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.936+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 >> 172.21.15.122:0/1677969705 conn(0x7feb800cfa00 msgr2=0x7feb80160630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f036fe00640 1 Processor -- start 2024-09-17T05:24:39.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f036fe00640 1 -- start start 2024-09-17T05:24:39.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 shutdown_connections 2024-09-17T05:24:39.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7feb84c00640 1 -- 172.21.15.122:0/1677969705 wait complete. 2024-09-17T05:24:39.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f8feba00640 1 -- 172.21.15.122:0/3590957761 shutdown_connections 2024-09-17T05:24:39.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f8feba00640 1 --2- 172.21.15.122:0/3590957761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe4072c80 0x7f8fe406fd30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f8feba00640 1 --2- 172.21.15.122:0/3590957761 >> v2:172.21.15.122:3300/0 conn(0x7f8fe40ca5c0 0x7f8fe406f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f8feba00640 1 --2- 172.21.15.122:0/3590957761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe40cb140 0x7f8fe40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f8feba00640 1 -- 172.21.15.122:0/3590957761 >> 172.21.15.122:0/3590957761 conn(0x7f8fe40b1600 msgr2=0x7f8fe40b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f8feba00640 1 -- 172.21.15.122:0/3590957761 shutdown_connections 2024-09-17T05:24:39.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f8feba00640 1 -- 172.21.15.122:0/3590957761 wait complete. 2024-09-17T05:24:39.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f036fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0370072610 0x7f0370151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f036fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03700aecb0 0x7f03700c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.940+0000 7f036fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03700cac60 0x7f03701525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0370072610 0x7f0370151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0370072610 0x7f0370151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:41116/0 (socket says 172.21.15.122:41116) 2024-09-17T05:24:39.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03700aecb0 0x7f03700c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03700aecb0 0x7f03700c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:53194/0 (socket says 172.21.15.122:53194) 2024-09-17T05:24:39.950 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.950 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 -- 172.21.15.122:0/4265184213 learned_addr learned my addr 172.21.15.122:0/4265184213 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036f400640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03700cac60 0x7f03701525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036fe00640 1 -- 172.21.15.122:0/4265184213 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f037015ca10 con 0x7f03700cac60 2024-09-17T05:24:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036fe00640 1 -- 172.21.15.122:0/4265184213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f037015c890 con 0x7f03700aecb0 2024-09-17T05:24:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036fe00640 1 -- 172.21.15.122:0/4265184213 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f037015cb90 con 0x7f0370072610 2024-09-17T05:24:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 -- 172.21.15.122:0/4265184213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03700aecb0 msgr2=0x7f03700c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03700aecb0 0x7f03700c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 -- 172.21.15.122:0/4265184213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03700cac60 msgr2=0x7f03701525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03700cac60 0x7f03701525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 -- 172.21.15.122:0/4265184213 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0370153290 con 0x7f0370072610 2024-09-17T05:24:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036f400640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03700cac60 0x7f03701525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036e000640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03700aecb0 0x7f03700c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036ea00640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0370072610 0x7f0370151360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f036000c4c0 tx=0x7f0360001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f0366a00640 1 -- 172.21.15.122:0/4265184213 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360017070 con 0x7f0370072610 2024-09-17T05:24:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f0366a00640 1 -- 172.21.15.122:0/4265184213 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0360003600 con 0x7f0370072610 2024-09-17T05:24:39.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f0366a00640 1 -- 172.21.15.122:0/4265184213 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360003a50 con 0x7f0370072610 2024-09-17T05:24:39.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036fe00640 1 -- 172.21.15.122:0/4265184213 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03700c7b80 con 0x7f0370072610 2024-09-17T05:24:39.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.944+0000 7f036fe00640 1 -- 172.21.15.122:0/4265184213 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0370151df0 con 0x7f0370072610 2024-09-17T05:24:39.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.948+0000 7f8feba00640 1 Processor -- start 2024-09-17T05:24:39.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.948+0000 7f8feba00640 1 -- start start 2024-09-17T05:24:39.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8feba00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe4072c80 0x7f8fe40b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.948+0000 7f0366a00640 1 -- 172.21.15.122:0/4265184213 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f036001a070 con 0x7f0370072610 2024-09-17T05:24:39.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.948+0000 7f0366a00640 1 -- 172.21.15.122:0/4265184213 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0360012740 con 0x7f0370072610 2024-09-17T05:24:39.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f035be00640 1 -- 172.21.15.122:0/4265184213 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0370070190 con 0x7f0370072610 2024-09-17T05:24:39.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f9bda000640 1 -- 172.21.15.122:0/57476394 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f9be000ab60 con 0x7f9be00e2b90 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8feba00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe40ca5c0 0x7f8fe40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8feba00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe40cb140 0x7f8fe40c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8feba00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe40cc440 con 0x7f8fe4072c80 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8feba00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe415f330 con 0x7f8fe40ca5c0 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8feba00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe40cc5c0 con 0x7f8fe40cb140 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe4072c80 0x7f8fe40b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe4072c80 0x7f8fe40b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45860/0 (socket says 172.21.15.122:45860) 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9000640 1 -- 172.21.15.122:0/1551222274 learned_addr learned my addr 172.21.15.122:0/1551222274 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9a00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe40cb140 0x7f8fe40c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe3e00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe40ca5c0 0x7f8fe40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9a00640 1 -- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe40ca5c0 msgr2=0x7f8fe40c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f361001dcb0 con 0x7f3610006150 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9a00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe40ca5c0 0x7f8fe40c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9a00640 1 -- 172.21.15.122:0/1551222274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe4072c80 msgr2=0x7f8fe40b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9a00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe4072c80 0x7f8fe40b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9a00640 1 -- 172.21.15.122:0/1551222274 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe40c4ce0 con 0x7f8fe40cb140 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9000640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe4072c80 0x7f8fe40b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe3e00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe40ca5c0 0x7f8fe40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.952+0000 7f8fe9a00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe40cb140 0x7f8fe40c8330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0ac000 tx=0x7f8fdc0ae640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f8fe0c00640 1 -- 172.21.15.122:0/1551222274 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0ba020 con 0x7f8fe40cb140 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe40c4fc0 con 0x7f8fe40cb140 2024-09-17T05:24:39.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f0366a00640 1 -- 172.21.15.122:0/4265184213 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0360021070 con 0x7f0370072610 2024-09-17T05:24:39.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe40b2d80 con 0x7f8fe40cb140 2024-09-17T05:24:39.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f8fe0c00640 1 -- 172.21.15.122:0/1551222274 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fdc004d50 con 0x7f8fe40cb140 2024-09-17T05:24:39.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f8fe0c00640 1 -- 172.21.15.122:0/1551222274 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0b2070 con 0x7f8fe40cb140 2024-09-17T05:24:39.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f8fe0c00640 1 -- 172.21.15.122:0/1551222274 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8fdc0bd020 con 0x7f8fe40cb140 2024-09-17T05:24:39.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f8fe0c00640 1 -- 172.21.15.122:0/1551222274 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8fdc004720 con 0x7f8fe40cb140 2024-09-17T05:24:39.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.956+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc0052a0 con 0x7f8fe40cb140 2024-09-17T05:24:39.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3615600640 1 -- 172.21.15.122:0/2739819162 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v12)=0 v12) ==== 119+0+0 (secure 0 0 0) 0x7f361809a070 con 0x7f3610006150 2024-09-17T05:24:39.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3a44c00640 1 -- 172.21.15.122:0/74193058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a40073d40 msgr2=0x7f3a4006f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3a44c00640 1 --2- 172.21.15.122:0/74193058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a40073d40 0x7f3a4006f650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009900 tx=0x7f3a2c01c8d0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3a44c00640 1 -- 172.21.15.122:0/74193058 shutdown_connections 2024-09-17T05:24:39.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3a44c00640 1 --2- 172.21.15.122:0/74193058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a40073d40 0x7f3a4006f650 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3a44c00640 1 --2- 172.21.15.122:0/74193058 >> v2:172.21.15.122:3300/0 conn(0x7f3a4015acb0 0x7f3a400ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3a44c00640 1 -- 172.21.15.122:0/74193058 >> 172.21.15.122:0/74193058 conn(0x7f3a400d0ab0 msgr2=0x7f3a400d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3a44c00640 1 -- 172.21.15.122:0/74193058 shutdown_connections 2024-09-17T05:24:39.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.960+0000 7f3a44c00640 1 -- 172.21.15.122:0/74193058 wait complete. 2024-09-17T05:24:39.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f8fe0c00640 1 -- 172.21.15.122:0/1551222274 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8fdc0c4030 con 0x7f8fe40cb140 2024-09-17T05:24:39.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610006150 msgr2=0x7f36100d3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610006150 0x7f36100d3cb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3618066720 tx=0x7f361806a110 comp rx=0 tx=0).stop 2024-09-17T05:24:39.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 shutdown_connections 2024-09-17T05:24:39.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3610053f20 0x7f3610020cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3610053180 0x7f3610020790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 --2- 172.21.15.122:0/2739819162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3610006150 0x7f36100d3cb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 >> 172.21.15.122:0/2739819162 conn(0x7f361001d7b0 msgr2=0x7f361000ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 shutdown_connections 2024-09-17T05:24:39.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3620600640 1 -- 172.21.15.122:0/2739819162 wait complete. 2024-09-17T05:24:39.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3a44c00640 1 Processor -- start 2024-09-17T05:24:39.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.964+0000 7f3a44c00640 1 -- start start 2024-09-17T05:24:39.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a44c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a40073d40 0x7f3a400b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a44c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a400ccc90 0x7f3a401fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a44c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a4015acb0 0x7f3a400b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3ea00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a400ccc90 0x7f3a401fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3ea00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a400ccc90 0x7f3a401fa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:41136/0 (socket says 172.21.15.122:41136) 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a44c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a400cd2a0 con 0x7f3a40073d40 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3ea00640 1 -- 172.21.15.122:0/4163265490 learned_addr learned my addr 172.21.15.122:0/4163265490 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a400cd120 con 0x7f3a4015acb0 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4006d430 con 0x7f3a400ccc90 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3fe00640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a4015acb0 0x7f3a400b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3f400640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a40073d40 0x7f3a400b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:39.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3fe00640 1 -- 172.21.15.122:0/4163265490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a400ccc90 msgr2=0x7f3a401fa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3fe00640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a400ccc90 0x7f3a401fa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3fe00640 1 -- 172.21.15.122:0/4163265490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a40073d40 msgr2=0x7f3a400b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3fe00640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a40073d40 0x7f3a400b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3fe00640 1 -- 172.21.15.122:0/4163265490 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a40147130 con 0x7f3a4015acb0 2024-09-17T05:24:39.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3f400640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a40073d40 0x7f3a400b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:39.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a3fe00640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a4015acb0 0x7f3a400b4ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a400cc380 tx=0x7f3a2c0a5720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:39.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a37400640 1 -- 172.21.15.122:0/4163265490 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c0b2040 con 0x7f3a4015acb0 2024-09-17T05:24:39.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f9be4c00640 1 -- 172.21.15.122:0/57476394 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v13)=0 v13) ==== 120+0+0 (secure 0 0 0) 0x7f9be8095b30 con 0x7f9be00e2b90 2024-09-17T05:24:39.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a37400640 1 -- 172.21.15.122:0/4163265490 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a2c004230 con 0x7f3a4015acb0 2024-09-17T05:24:39.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.968+0000 7f3a37400640 1 -- 172.21.15.122:0/4163265490 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c003600 con 0x7f3a4015acb0 2024-09-17T05:24:39.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a40147410 con 0x7f3a4015acb0 2024-09-17T05:24:39.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a401479d0 con 0x7f3a4015acb0 2024-09-17T05:24:39.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f3a37400640 1 -- 172.21.15.122:0/4163265490 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a2c0bf020 con 0x7f3a4015acb0 2024-09-17T05:24:39.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f3a37400640 1 -- 172.21.15.122:0/4163265490 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a2c0b8cb0 con 0x7f3a4015acb0 2024-09-17T05:24:39.980 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi122') 2024-09-17T05:24:39.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fcd48074b30 con 0x7fcd4814d440 2024-09-17T05:24:39.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 -- 172.21.15.122:0/57476394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9be00e2b90 msgr2=0x7f9be0021990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 --2- 172.21.15.122:0/57476394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9be00e2b90 0x7f9be0021990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9be806b800 tx=0x7f9be806a050 comp rx=0 tx=0).stop 2024-09-17T05:24:39.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 -- 172.21.15.122:0/57476394 shutdown_connections 2024-09-17T05:24:39.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 --2- 172.21.15.122:0/57476394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be00e2f70 0x7f9be001e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 --2- 172.21.15.122:0/57476394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9be00e2b90 0x7f9be0021990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 --2- 172.21.15.122:0/57476394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9be001d9c0 0x7f9be001c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 -- 172.21.15.122:0/57476394 >> 172.21.15.122:0/57476394 conn(0x7f9be0054570 msgr2=0x7f9be0033fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 -- 172.21.15.122:0/57476394 shutdown_connections 2024-09-17T05:24:39.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4006d9c0 con 0x7f3a4015acb0 2024-09-17T05:24:39.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.976+0000 7f9bda000640 1 -- 172.21.15.122:0/57476394 wait complete. 2024-09-17T05:24:39.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.980+0000 7fcd3f400640 1 -- 172.21.15.122:0/1301364729 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7fcd4814d240 con 0x7fcd4814d440 2024-09-17T05:24:39.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.980+0000 7f3a37400640 1 -- 172.21.15.122:0/4163265490 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3a2c0164f0 con 0x7f3a4015acb0 2024-09-17T05:24:39.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd4814d440 msgr2=0x7fcd4815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:39.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd4814d440 0x7fcd4815bbc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd3401c4f0 tx=0x7fcd340a7f30 comp rx=0 tx=0).stop 2024-09-17T05:24:39.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 shutdown_connections 2024-09-17T05:24:39.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd4814d440 0x7fcd4815bbc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd4814d990 0x7fcd4814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 --2- 172.21.15.122:0/1301364729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd2c0053d0 0x7fcd480741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:39.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 >> 172.21.15.122:0/1301364729 conn(0x7fcd480b18f0 msgr2=0x7fcd4815ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:39.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 shutdown_connections 2024-09-17T05:24:39.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:39.988+0000 7fcd4e200640 1 -- 172.21.15.122:0/1301364729 wait complete. 2024-09-17T05:24:40.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.024+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7efd480d06a0 con 0x7efd48074540 2024-09-17T05:24:40.029 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi122') 2024-09-17T05:24:40.029 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi122') 2024-09-17T05:24:40.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.028+0000 7efd3e000640 1 -- 172.21.15.122:0/3364264427 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7efd3802aa50 con 0x7efd48074540 2024-09-17T05:24:40.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.028+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd48074540 msgr2=0x7efd48158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.032+0000 7efd47200640 1 --2- 172.21.15.122:0/3364264427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd48074540 0x7efd48158ad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd38009f50 tx=0x7efd3800b870 comp rx=0 tx=0).stop 2024-09-17T05:24:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.032+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 shutdown_connections 2024-09-17T05:24:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.032+0000 7efd47200640 1 --2- 172.21.15.122:0/3364264427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd4815e300 0x7efd480c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.032+0000 7efd47200640 1 --2- 172.21.15.122:0/3364264427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd480cd5d0 0x7efd481fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.032+0000 7efd47200640 1 --2- 172.21.15.122:0/3364264427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd48074540 0x7efd48158ad0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.032+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 >> 172.21.15.122:0/3364264427 conn(0x7efd480cfa00 msgr2=0x7efd48073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.032+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 shutdown_connections 2024-09-17T05:24:40.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.032+0000 7efd47200640 1 -- 172.21.15.122:0/3364264427 wait complete. 2024-09-17T05:24:40.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.052+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fcc940d06a0 con 0x7fcc9415e8f0 2024-09-17T05:24:40.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.056+0000 7fcc91600640 1 -- 172.21.15.122:0/3471174566 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v16)=0 v16) ==== 120+0+0 (secure 0 0 0) 0x7fcc800054f0 con 0x7fcc9415e8f0 2024-09-17T05:24:40.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.056+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc9415e8f0 msgr2=0x7fcc9415ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.056+0000 7fcc9ae00640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc9415e8f0 0x7fcc9415ed70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc800ae000 tx=0x7fcc800afc30 comp rx=0 tx=0).stop 2024-09-17T05:24:40.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.060+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 shutdown_connections 2024-09-17T05:24:40.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.060+0000 7fcc9ae00640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc9415e8f0 0x7fcc9415ed70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.060+0000 7fcc9ae00640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc941f43b0 0x7fcc9415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.060+0000 7fcc9ae00640 1 --2- 172.21.15.122:0/3471174566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc941f3fd0 0x7fcc940ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.060+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 >> 172.21.15.122:0/3471174566 conn(0x7fcc940cfa00 msgr2=0x7fcc940af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:40.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.060+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 shutdown_connections 2024-09-17T05:24:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.060+0000 7fcc9ae00640 1 -- 172.21.15.122:0/3471174566 wait complete. 2024-09-17T05:24:40.065 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi122') 2024-09-17T05:24:40.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.072+0000 7f035be00640 1 -- 172.21.15.122:0/4265184213 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f037015d0a0 con 0x7f0370072610 2024-09-17T05:24:40.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.072+0000 7f0366a00640 1 -- 172.21.15.122:0/4265184213 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f0370153290 con 0x7f0370072610 2024-09-17T05:24:40.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 -- 172.21.15.122:0/4265184213 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0370072610 msgr2=0x7f0370151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0370072610 0x7f0370151360 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f036000c4c0 tx=0x7f0360001e80 comp rx=0 tx=0).stop 2024-09-17T05:24:40.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 -- 172.21.15.122:0/4265184213 shutdown_connections 2024-09-17T05:24:40.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03700cac60 0x7f03701525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03700aecb0 0x7f03700c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 --2- 172.21.15.122:0/4265184213 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0370072610 0x7f0370151360 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 -- 172.21.15.122:0/4265184213 >> 172.21.15.122:0/4265184213 conn(0x7f03700cfa00 msgr2=0x7f03700cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:40.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 -- 172.21.15.122:0/4265184213 shutdown_connections 2024-09-17T05:24:40.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.076+0000 7f035be00640 1 -- 172.21.15.122:0/4265184213 wait complete. 2024-09-17T05:24:40.090 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi122') 2024-09-17T05:24:40.090 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi122') 2024-09-17T05:24:40.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.088+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f8fbc005590 con 0x7f8fe40cb140 2024-09-17T05:24:40.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8fe0c00640 1 -- 172.21.15.122:0/1551222274 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f8fdc0169b0 con 0x7f8fe40cb140 2024-09-17T05:24:40.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe40cb140 msgr2=0x7f8fe40c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe40cb140 0x7f8fe40c8330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc0ac000 tx=0x7f8fdc0ae640 comp rx=0 tx=0).stop 2024-09-17T05:24:40.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 shutdown_connections 2024-09-17T05:24:40.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe40cb140 0x7f8fe40c8330 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fe40ca5c0 0x7f8fe40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 --2- 172.21.15.122:0/1551222274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe4072c80 0x7f8fe40b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 >> 172.21.15.122:0/1551222274 conn(0x7f8fe40b1600 msgr2=0x7f8fe40c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:40.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 shutdown_connections 2024-09-17T05:24:40.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.092+0000 7f8feba00640 1 -- 172.21.15.122:0/1551222274 wait complete. 2024-09-17T05:24:40.099 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi122') 2024-09-17T05:24:40.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.112+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f3a400d18e0 con 0x7f3a4015acb0 2024-09-17T05:24:40.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.116+0000 7f3a37400640 1 -- 172.21.15.122:0/4163265490 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v19)=0 v19) ==== 125+0+0 (secure 0 0 0) 0x7f3a2c003b80 con 0x7f3a4015acb0 2024-09-17T05:24:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a4015acb0 msgr2=0x7f3a400b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a4015acb0 0x7f3a400b4ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a400cc380 tx=0x7f3a2c0a5720 comp rx=0 tx=0).stop 2024-09-17T05:24:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 shutdown_connections 2024-09-17T05:24:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a4015acb0 0x7f3a400b4ed0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a400ccc90 0x7f3a401fa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 --2- 172.21.15.122:0/4163265490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a40073d40 0x7f3a400b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 >> 172.21.15.122:0/4163265490 conn(0x7f3a400d0ab0 msgr2=0x7f3a400ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:40.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 shutdown_connections 2024-09-17T05:24:40.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.120+0000 7f3a44c00640 1 -- 172.21.15.122:0/4163265490 wait complete. 2024-09-17T05:24:40.144 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi122') 2024-09-17T05:24:40.144 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T05:24:40.144 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T05:24:40.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- 172.21.15.122:0/3613282491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c40ceb40 msgr2=0x7f07c40cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 --2- 172.21.15.122:0/3613282491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c40ceb40 0x7f07c40cb790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009c80 tx=0x7f07bc01d800 comp rx=0 tx=0).stop 2024-09-17T05:24:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- 172.21.15.122:0/3613282491 shutdown_connections 2024-09-17T05:24:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 --2- 172.21.15.122:0/3613282491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c40ceb40 0x7f07c40cb790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 --2- 172.21.15.122:0/3613282491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c4070500 0x7f07c40caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- 172.21.15.122:0/3613282491 >> 172.21.15.122:0/3613282491 conn(0x7f07c40742f0 msgr2=0x7f07c4074700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- 172.21.15.122:0/3613282491 shutdown_connections 2024-09-17T05:24:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- 172.21.15.122:0/3613282491 wait complete. 2024-09-17T05:24:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 Processor -- start 2024-09-17T05:24:40.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- start start 2024-09-17T05:24:40.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c4070500 0x7f07c415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:40.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c40ccb30 0x7f07c40c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:40.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07c40ceb40 0x7f07c4164040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:40.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4168380 con 0x7f07c40ceb40 2024-09-17T05:24:40.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4168200 con 0x7f07c40ccb30 2024-09-17T05:24:40.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.324+0000 7f07c9200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f07c4168500 con 0x7f07c4070500 2024-09-17T05:24:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07c40ceb40 0x7f07c4164040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07c40ceb40 0x7f07c4164040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45874/0 (socket says 172.21.15.122:45874) 2024-09-17T05:24:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 -- 172.21.15.122:0/1656366935 learned_addr learned my addr 172.21.15.122:0/1656366935 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c2a00640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c40ccb30 0x7f07c40c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3400640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c4070500 0x7f07c415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 -- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c4070500 msgr2=0x7f07c415aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c4070500 0x7f07c415aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 -- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c40ccb30 msgr2=0x7f07c40c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c40ccb30 0x7f07c40c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 -- 172.21.15.122:0/1656366935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c40ca090 con 0x7f07c40ceb40 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3400640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c4070500 0x7f07c415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c3e00640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07c40ceb40 0x7f07c4164040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009bd0 tx=0x7f07bc005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07bb400640 1 -- 172.21.15.122:0/1656366935 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0b2040 con 0x7f07c40ceb40 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07bb400640 1 -- 172.21.15.122:0/1656366935 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07bc004b40 con 0x7f07c40ceb40 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c40ca370 con 0x7f07c40ceb40 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07bb400640 1 -- 172.21.15.122:0/1656366935 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0ad5f0 con 0x7f07c40ceb40 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c40c92f0 con 0x7f07c40ceb40 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07bb400640 1 -- 172.21.15.122:0/1656366935 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f07bc0be060 con 0x7f07c40ceb40 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07bb400640 1 -- 172.21.15.122:0/1656366935 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f07bc004780 con 0x7f07c40ceb40 2024-09-17T05:24:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.328+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c40caad0 con 0x7f07c40ceb40 2024-09-17T05:24:40.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.332+0000 7f07bb400640 1 -- 172.21.15.122:0/1656366935 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f07bc0d7860 con 0x7f07c40ceb40 2024-09-17T05:24:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.392+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f07c4075100 con 0x7f07c40ceb40 2024-09-17T05:24:40.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07bb400640 1 -- 172.21.15.122:0/1656366935 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f07bc0c5070 con 0x7f07c40ceb40 2024-09-17T05:24:40.399 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-09-17T05:24:40.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07c40ceb40 msgr2=0x7f07c4164040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07c40ceb40 0x7f07c4164040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009bd0 tx=0x7f07bc005ac0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 shutdown_connections 2024-09-17T05:24:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07c40ceb40 0x7f07c4164040 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c40ccb30 0x7f07c40c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 --2- 172.21.15.122:0/1656366935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c4070500 0x7f07c415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 >> 172.21.15.122:0/1656366935 conn(0x7f07c40742f0 msgr2=0x7f07c40b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 shutdown_connections 2024-09-17T05:24:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:40.396+0000 7f07c9200640 1 -- 172.21.15.122:0/1656366935 wait complete. 2024-09-17T05:24:40.419 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T05:24:40.419 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T05:24:40.420 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:40.420 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T05:24:40.439 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:40.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T05:24:40.501 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:40.501 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T05:24:40.561 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:24:40.561 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T05:24:40.621 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:40.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T05:24:40.641 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:40.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T05:24:40.703 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:40.703 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T05:24:40.763 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:24:40.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T05:24:40.823 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new de145692-b14c-4df1-9dd6-975e867d1f5a 0 2024-09-17T05:24:41.004 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:40.999+0000 7fb908c00640 1 -- 172.21.15.173:0/443528010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb904151310 msgr2=0x7fb9040cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:40.999+0000 7fb908c00640 1 --2- 172.21.15.173:0/443528010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb904151310 0x7fb9040cc8c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40098a0 tx=0x7fb8f401c860 comp rx=0 tx=0).stop 2024-09-17T05:24:41.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:40.999+0000 7fb908c00640 1 -- 172.21.15.173:0/443528010 shutdown_connections 2024-09-17T05:24:41.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:40.999+0000 7fb908c00640 1 --2- 172.21.15.173:0/443528010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb904151310 0x7fb9040cc8c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:40.999+0000 7fb908c00640 1 -- 172.21.15.173:0/443528010 >> 172.21.15.173:0/443528010 conn(0x7fb9040d04f0 msgr2=0x7fb9040d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:41.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:40.999+0000 7fb908c00640 1 -- 172.21.15.173:0/443528010 shutdown_connections 2024-09-17T05:24:41.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:40.999+0000 7fb908c00640 1 -- 172.21.15.173:0/443528010 wait complete. 2024-09-17T05:24:41.005 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 Processor -- start 2024-09-17T05:24:41.006 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 -- start start 2024-09-17T05:24:41.007 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb90406d330 0x7fb9041545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.007 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb904151310 0x7fb9041f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.007 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90415a960 0x7fb9040c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb904068fa0 con 0x7fb904151310 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb904068e20 con 0x7fb90415a960 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb904168b10 con 0x7fb90406d330 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb90406d330 0x7fb9041545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb90406d330 0x7fb9041545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:57286/0 (socket says 172.21.15.173:57286) 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb902800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90415a960 0x7fb9040c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 -- 172.21.15.173:0/3407870529 learned_addr learned my addr 172.21.15.173:0/3407870529 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb8fbe00640 1 --2- 172.21.15.173:0/3407870529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb904151310 0x7fb9041f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 -- 172.21.15.173:0/3407870529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90415a960 msgr2=0x7fb9040c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 --2- 172.21.15.173:0/3407870529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90415a960 0x7fb9040c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 -- 172.21.15.173:0/3407870529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb904151310 msgr2=0x7fb9041f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 --2- 172.21.15.173:0/3407870529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb904151310 0x7fb9041f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.008 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 -- 172.21.15.173:0/3407870529 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb904150540 con 0x7fb90406d330 2024-09-17T05:24:41.010 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb901e00640 1 --2- 172.21.15.173:0/3407870529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb90406d330 0x7fb9041545b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb8f401a490 tx=0x7fb8f40a55c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:41.010 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb8fa000640 1 -- 172.21.15.173:0/3407870529 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b2040 con 0x7fb90406d330 2024-09-17T05:24:41.010 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb8fa000640 1 -- 172.21.15.173:0/3407870529 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8f4004430 con 0x7fb90406d330 2024-09-17T05:24:41.010 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb904150820 con 0x7fb90406d330 2024-09-17T05:24:41.010 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb8fa000640 1 -- 172.21.15.173:0/3407870529 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4005280 con 0x7fb90406d330 2024-09-17T05:24:41.010 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.003+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9041f89c0 con 0x7fb90406d330 2024-09-17T05:24:41.010 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.007+0000 7fb8fa000640 1 -- 172.21.15.173:0/3407870529 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb8f40be060 con 0x7fb90406d330 2024-09-17T05:24:41.011 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.007+0000 7fb8fa000640 1 -- 172.21.15.173:0/3407870529 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fb8f4003940 con 0x7fb90406d330 2024-09-17T05:24:41.011 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.007+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb904150d40 con 0x7fb90406d330 2024-09-17T05:24:41.011 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.007+0000 7fb8fa000640 1 -- 172.21.15.173:0/3407870529 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb8f40d7860 con 0x7fb90406d330 2024-09-17T05:24:41.080 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.075+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "de145692-b14c-4df1-9dd6-975e867d1f5a", "id": 0} v 0) -- 0x7fb9040cc8c0 con 0x7fb90406d330 2024-09-17T05:24:41.084 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.079+0000 7fb8fa000640 1 -- 172.21.15.173:0/3407870529 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "de145692-b14c-4df1-9dd6-975e867d1f5a", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fb8f40c5030 con 0x7fb90406d330 2024-09-17T05:24:41.084 INFO:teuthology.orchestra.run.smithi173.stdout:0 2024-09-17T05:24:41.085 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb90406d330 msgr2=0x7fb9041545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.086 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 --2- 172.21.15.173:0/3407870529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb90406d330 0x7fb9041545b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb8f401a490 tx=0x7fb8f40a55c0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.086 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 shutdown_connections 2024-09-17T05:24:41.086 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 --2- 172.21.15.173:0/3407870529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90415a960 0x7fb9040c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.086 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 --2- 172.21.15.173:0/3407870529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb904151310 0x7fb9041f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.086 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 --2- 172.21.15.173:0/3407870529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb90406d330 0x7fb9041545b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.086 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 >> 172.21.15.173:0/3407870529 conn(0x7fb9040d04f0 msgr2=0x7fb9040b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:41.086 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 shutdown_connections 2024-09-17T05:24:41.086 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.083+0000 7fb908c00640 1 -- 172.21.15.173:0/3407870529 wait complete. 2024-09-17T05:24:41.104 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 53a4f200-92b6-4a22-9c59-14a46e26f354 1 2024-09-17T05:24:41.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.243+0000 7f0c18c00640 1 -- 172.21.15.173:0/2276310548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 msgr2=0x7f0c140b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.243+0000 7f0c18c00640 1 --2- 172.21.15.173:0/2276310548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 0x7f0c140b15e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009840 tx=0x7f0c0401f010 comp rx=0 tx=0).stop 2024-09-17T05:24:41.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.243+0000 7f0c18c00640 1 -- 172.21.15.173:0/2276310548 shutdown_connections 2024-09-17T05:24:41.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.243+0000 7f0c18c00640 1 --2- 172.21.15.173:0/2276310548 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c140d29b0 0x7f0c140b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.243+0000 7f0c18c00640 1 --2- 172.21.15.173:0/2276310548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 0x7f0c140b15e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.243+0000 7f0c18c00640 1 -- 172.21.15.173:0/2276310548 >> 172.21.15.173:0/2276310548 conn(0x7f0c140d1e90 msgr2=0x7f0c140d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:41.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.243+0000 7f0c18c00640 1 -- 172.21.15.173:0/2276310548 shutdown_connections 2024-09-17T05:24:41.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.243+0000 7f0c18c00640 1 -- 172.21.15.173:0/2276310548 wait complete. 2024-09-17T05:24:41.249 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 Processor -- start 2024-09-17T05:24:41.249 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 -- start start 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 0x7f0c140c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c140d04d0 0x7f0c140cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c140d29b0 0x7f0c140caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140b0ed0 con 0x7f0c140d04d0 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140b0d50 con 0x7f0c140d00f0 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140b04f0 con 0x7f0c140d29b0 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 0x7f0c140c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 0x7f0c140c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:50488/0 (socket says 172.21.15.173:50488) 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 -- 172.21.15.173:0/3802499234 learned_addr learned my addr 172.21.15.173:0/3802499234 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c12800640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c140d29b0 0x7f0c140caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c0be00640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c140d04d0 0x7f0c140cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 -- 172.21.15.173:0/3802499234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c140d29b0 msgr2=0x7f0c140caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c140d29b0 0x7f0c140caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 -- 172.21.15.173:0/3802499234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c140d04d0 msgr2=0x7f0c140cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c140d04d0 0x7f0c140cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 -- 172.21.15.173:0/3802499234 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c140c7f20 con 0x7f0c140d00f0 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c11e00640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 0x7f0c140c8f60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009790 tx=0x7f0c04005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c0a000640 1 -- 172.21.15.173:0/3802499234 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040b5040 con 0x7f0c140d00f0 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c0a000640 1 -- 172.21.15.173:0/3802499234 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c04005220 con 0x7f0c140d00f0 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c140c8200 con 0x7f0c140d00f0 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c0a000640 1 -- 172.21.15.173:0/3802499234 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040046d0 con 0x7f0c140d00f0 2024-09-17T05:24:41.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c140c94a0 con 0x7f0c140d00f0 2024-09-17T05:24:41.254 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c0a000640 1 -- 172.21.15.173:0/3802499234 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0c040c1060 con 0x7f0c140d00f0 2024-09-17T05:24:41.254 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.247+0000 7f0c0a000640 1 -- 172.21.15.173:0/3802499234 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f0c040c3450 con 0x7f0c140d00f0 2024-09-17T05:24:41.254 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.251+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c140b15e0 con 0x7f0c140d00f0 2024-09-17T05:24:41.255 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.251+0000 7f0c0a000640 1 -- 172.21.15.173:0/3802499234 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0c04018b70 con 0x7f0c140d00f0 2024-09-17T05:24:41.323 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.319+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "53a4f200-92b6-4a22-9c59-14a46e26f354", "id": 1} v 0) -- 0x7f0c140cc370 con 0x7f0c140d00f0 2024-09-17T05:24:41.329 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c0a000640 1 -- 172.21.15.173:0/3802499234 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "53a4f200-92b6-4a22-9c59-14a46e26f354", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f0c04003600 con 0x7f0c140d00f0 2024-09-17T05:24:41.329 INFO:teuthology.orchestra.run.smithi173.stdout:1 2024-09-17T05:24:41.330 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 msgr2=0x7f0c140c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.330 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 0x7f0c140c8f60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009790 tx=0x7f0c04005dd0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.330 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 shutdown_connections 2024-09-17T05:24:41.331 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c140d29b0 0x7f0c140caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.331 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c140d04d0 0x7f0c140cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.331 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 --2- 172.21.15.173:0/3802499234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c140d00f0 0x7f0c140c8f60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.331 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 >> 172.21.15.173:0/3802499234 conn(0x7f0c140d1e90 msgr2=0x7f0c140b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:41.331 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 shutdown_connections 2024-09-17T05:24:41.331 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.327+0000 7f0c18c00640 1 -- 172.21.15.173:0/3802499234 wait complete. 2024-09-17T05:24:41.349 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 68b85d73-8530-454d-94a0-ceeb46b20285 2 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 -- 172.21.15.173:0/714844462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d00f0 msgr2=0x7f6e400b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 --2- 172.21.15.173:0/714844462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d00f0 0x7f6e400b15e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e400739c0 tx=0x7f6e3401f0b0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 -- 172.21.15.173:0/714844462 shutdown_connections 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 --2- 172.21.15.173:0/714844462 >> v2:172.21.15.122:3300/0 conn(0x7f6e400d04d0 0x7f6e400b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 --2- 172.21.15.173:0/714844462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e400d29b0 0x7f6e400b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 --2- 172.21.15.173:0/714844462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d00f0 0x7f6e400b15e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 -- 172.21.15.173:0/714844462 >> 172.21.15.173:0/714844462 conn(0x7f6e400d1e90 msgr2=0x7f6e400d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 -- 172.21.15.173:0/714844462 shutdown_connections 2024-09-17T05:24:41.490 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 -- 172.21.15.173:0/714844462 wait complete. 2024-09-17T05:24:41.491 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 Processor -- start 2024-09-17T05:24:41.491 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 -- start start 2024-09-17T05:24:41.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e400d00f0 0x7f6e400c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e400d04d0 0x7f6e400cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d29b0 0x7f6e400cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.487+0000 7f6e47800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4015ff60 con 0x7f6e400d00f0 2024-09-17T05:24:41.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e47800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4015fde0 con 0x7f6e400d29b0 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e47800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6e401600e0 con 0x7f6e400d04d0 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d29b0 0x7f6e400cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e44e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e400d00f0 0x7f6e400c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d29b0 0x7f6e400cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:50502/0 (socket says 172.21.15.173:50502) 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 -- 172.21.15.173:0/3210946021 learned_addr learned my addr 172.21.15.173:0/3210946021 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e3fe00640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e400d04d0 0x7f6e400cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 -- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e400d04d0 msgr2=0x7f6e400cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e400d04d0 0x7f6e400cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 -- 172.21.15.173:0/3210946021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e400d00f0 msgr2=0x7f6e400c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e400d00f0 0x7f6e400c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 -- 172.21.15.173:0/3210946021 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e400cc3c0 con 0x7f6e400d29b0 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e44e00640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e400d00f0 0x7f6e400c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:41.494 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e3fe00640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e400d04d0 0x7f6e400cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:41.495 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e45800640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d29b0 0x7f6e400cdb90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800e990 tx=0x7f6e3800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:41.495 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e3cc00640 1 -- 172.21.15.173:0/3210946021 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38004d30 con 0x7f6e400d29b0 2024-09-17T05:24:41.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e3cc00640 1 -- 172.21.15.173:0/3210946021 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e3800cb00 con 0x7f6e400d29b0 2024-09-17T05:24:41.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e3cc00640 1 -- 172.21.15.173:0/3210946021 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38005020 con 0x7f6e400d29b0 2024-09-17T05:24:41.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e401f2a50 con 0x7f6e400d29b0 2024-09-17T05:24:41.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e401f3010 con 0x7f6e400d29b0 2024-09-17T05:24:41.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e3cc00640 1 -- 172.21.15.173:0/3210946021 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e380051c0 con 0x7f6e400d29b0 2024-09-17T05:24:41.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e3cc00640 1 -- 172.21.15.173:0/3210946021 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f6e3801a400 con 0x7f6e400d29b0 2024-09-17T05:24:41.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.491+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4015e2c0 con 0x7f6e400d29b0 2024-09-17T05:24:41.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.495+0000 7f6e3cc00640 1 -- 172.21.15.173:0/3210946021 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6e4015e2c0 con 0x7f6e400d29b0 2024-09-17T05:24:41.566 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.563+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "68b85d73-8530-454d-94a0-ceeb46b20285", "id": 2} v 0) -- 0x7f6e400b0670 con 0x7f6e400d29b0 2024-09-17T05:24:41.573 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e3cc00640 1 -- 172.21.15.173:0/3210946021 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68b85d73-8530-454d-94a0-ceeb46b20285", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f6e38016030 con 0x7f6e400d29b0 2024-09-17T05:24:41.573 INFO:teuthology.orchestra.run.smithi173.stdout:2 2024-09-17T05:24:41.574 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d29b0 msgr2=0x7f6e400cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.574 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d29b0 0x7f6e400cdb90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800e990 tx=0x7f6e3800ee50 comp rx=0 tx=0).stop 2024-09-17T05:24:41.574 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 shutdown_connections 2024-09-17T05:24:41.574 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e400d29b0 0x7f6e400cdb90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.574 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e400d04d0 0x7f6e400cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.574 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 --2- 172.21.15.173:0/3210946021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e400d00f0 0x7f6e400c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.574 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 >> 172.21.15.173:0/3210946021 conn(0x7f6e400d1e90 msgr2=0x7f6e400b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:41.575 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 shutdown_connections 2024-09-17T05:24:41.575 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.571+0000 7f6e47800640 1 -- 172.21.15.173:0/3210946021 wait complete. 2024-09-17T05:24:41.593 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c 3 2024-09-17T05:24:41.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.731+0000 7f69b7400640 1 -- 172.21.15.173:0/3382340207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d00f0 msgr2=0x7f69b00b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.731+0000 7f69b7400640 1 --2- 172.21.15.173:0/3382340207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d00f0 0x7f69b00b15e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f69b00739c0 tx=0x7f69a401f0b0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 -- 172.21.15.173:0/3382340207 shutdown_connections 2024-09-17T05:24:41.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 --2- 172.21.15.173:0/3382340207 >> v2:172.21.15.122:3300/0 conn(0x7f69b00d29b0 0x7f69b00b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 --2- 172.21.15.173:0/3382340207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d00f0 0x7f69b00b15e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 -- 172.21.15.173:0/3382340207 >> 172.21.15.173:0/3382340207 conn(0x7f69b00d1e90 msgr2=0x7f69b00d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:41.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 -- 172.21.15.173:0/3382340207 shutdown_connections 2024-09-17T05:24:41.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 -- 172.21.15.173:0/3382340207 wait complete. 2024-09-17T05:24:41.738 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 Processor -- start 2024-09-17T05:24:41.738 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 -- start start 2024-09-17T05:24:41.739 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b00d00f0 0x7f69b01f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.739 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b00d04d0 0x7f69b01f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d29b0 0x7f69b0155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69b00b0cc0 con 0x7f69b00d00f0 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69b00b0b40 con 0x7f69b00d29b0 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b7400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f69b00b0e40 con 0x7f69b00d04d0 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69afe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b00d04d0 0x7f69b01f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d29b0 0x7f69b0155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b00d00f0 0x7f69b01f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b00d00f0 0x7f69b01f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.173:57718/0 (socket says 172.21.15.173:57718) 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d29b0 0x7f69b0155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:50514/0 (socket says 172.21.15.173:50514) 2024-09-17T05:24:41.740 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b4a00640 1 -- 172.21.15.173:0/1179764272 learned_addr learned my addr 172.21.15.173:0/1179764272 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-17T05:24:41.741 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b4a00640 1 -- 172.21.15.173:0/1179764272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b00d04d0 msgr2=0x7f69b01f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b4a00640 1 --2- 172.21.15.173:0/1179764272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b00d04d0 0x7f69b01f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b4a00640 1 -- 172.21.15.173:0/1179764272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d29b0 msgr2=0x7f69b0155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b4a00640 1 --2- 172.21.15.173:0/1179764272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d29b0 0x7f69b0155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.735+0000 7f69b4a00640 1 -- 172.21.15.173:0/1179764272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b01f5e70 con 0x7f69b00d00f0 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69b4a00640 1 --2- 172.21.15.173:0/1179764272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b00d00f0 0x7f69b01f5930 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f69a40b3040 tx=0x7f69a4005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69acc00640 1 -- 172.21.15.173:0/1179764272 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40b5040 con 0x7f69b00d00f0 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69acc00640 1 -- 172.21.15.173:0/1179764272 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69a40038f0 con 0x7f69b00d00f0 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69acc00640 1 -- 172.21.15.173:0/1179764272 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4003d20 con 0x7f69b00d00f0 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b00cf780 con 0x7f69b00d00f0 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b00c96e0 con 0x7f69b00d00f0 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69acc00640 1 -- 172.21.15.173:0/1179764272 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f69a40c0060 con 0x7f69b00d00f0 2024-09-17T05:24:41.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69acc00640 1 -- 172.21.15.173:0/1179764272 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f69a40c1880 con 0x7f69b00d00f0 2024-09-17T05:24:41.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.739+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b00b23e0 con 0x7f69b00d00f0 2024-09-17T05:24:41.746 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.743+0000 7f69acc00640 1 -- 172.21.15.173:0/1179764272 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f69b01f5e70 con 0x7f69b00d00f0 2024-09-17T05:24:41.815 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.811+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c", "id": 3} v 0) -- 0x7f69b00b76e0 con 0x7f69b00d00f0 2024-09-17T05:24:41.819 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.815+0000 7f69acc00640 1 -- 172.21.15.173:0/1179764272 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f69a4018c10 con 0x7f69b00d00f0 2024-09-17T05:24:41.819 INFO:teuthology.orchestra.run.smithi173.stdout:3 2024-09-17T05:24:41.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b00d00f0 msgr2=0x7f69b01f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:41.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 --2- 172.21.15.173:0/1179764272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b00d00f0 0x7f69b01f5930 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f69a40b3040 tx=0x7f69a4005e20 comp rx=0 tx=0).stop 2024-09-17T05:24:41.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 shutdown_connections 2024-09-17T05:24:41.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 --2- 172.21.15.173:0/1179764272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b00d29b0 0x7f69b0155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 --2- 172.21.15.173:0/1179764272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b00d04d0 0x7f69b01f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 --2- 172.21.15.173:0/1179764272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b00d00f0 0x7f69b01f5930 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:41.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 >> 172.21.15.173:0/1179764272 conn(0x7f69b00d1e90 msgr2=0x7f69b00b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:41.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 shutdown_connections 2024-09-17T05:24:41.822 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:41.819+0000 7f69b7400640 1 -- 172.21.15.173:0/1179764272 wait complete. 2024-09-17T05:24:41.839 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 633269aa-0f64-4cf3-93a1-39fc33040a34 4 2024-09-17T05:24:42.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.011+0000 7f4847200640 1 -- 172.21.15.173:0/3325845251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4828004e50 msgr2=0x7f48280052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.011+0000 7f4847200640 1 --2- 172.21.15.173:0/3325845251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4828004e50 0x7f48280052d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f48340094d0 tx=0x7f483401e990 comp rx=0 tx=0).stop 2024-09-17T05:24:42.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.011+0000 7f4846800640 1 -- 172.21.15.173:0/3325845251 reap_dead start 2024-09-17T05:24:42.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.011+0000 7f4847200640 1 -- 172.21.15.173:0/3325845251 shutdown_connections 2024-09-17T05:24:42.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.011+0000 7f4847200640 1 -- 172.21.15.173:0/3325845251 >> 172.21.15.173:0/3325845251 conn(0x7f48400cfa00 msgr2=0x7f48400cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:42.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.011+0000 7f4847200640 1 -- 172.21.15.173:0/3325845251 shutdown_connections 2024-09-17T05:24:42.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.011+0000 7f4847200640 1 -- 172.21.15.173:0/3325845251 wait complete. 2024-09-17T05:24:42.018 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.015+0000 7f4847200640 1 Processor -- start 2024-09-17T05:24:42.018 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.015+0000 7f4847200640 1 -- start start 2024-09-17T05:24:42.023 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4847200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4828004e50 0x7f48400b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.023 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4847200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4840159340 0x7f484014bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.023 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4847200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4840158d80 0x7f484015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.023 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4847200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4840164fe0 con 0x7f4840158d80 2024-09-17T05:24:42.024 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4847200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4840164e60 con 0x7f4828004e50 2024-09-17T05:24:42.024 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4847200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4840165160 con 0x7f4840159340 2024-09-17T05:24:42.025 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4846800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4840158d80 0x7f484015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.026 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4846800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4840158d80 0x7f484015af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.173:57748/0 (socket says 172.21.15.173:57748) 2024-09-17T05:24:42.026 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4846800640 1 -- 172.21.15.173:0/3023299586 learned_addr learned my addr 172.21.15.173:0/3023299586 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-17T05:24:42.026 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4845e00640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4828004e50 0x7f48400b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.026 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.019+0000 7f4845400640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4840159340 0x7f484014bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.026 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f4846800640 1 -- 172.21.15.173:0/3023299586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4840159340 msgr2=0x7f484014bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.026 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f4846800640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4840159340 0x7f484014bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.026 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f4846800640 1 -- 172.21.15.173:0/3023299586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4828004e50 msgr2=0x7f48400b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.027 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f4846800640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4828004e50 0x7f48400b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.027 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f4846800640 1 -- 172.21.15.173:0/3023299586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48401f6f10 con 0x7f4840158d80 2024-09-17T05:24:42.027 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f4846800640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4840158d80 0x7f484015af70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f48340b2040 tx=0x7f48340a8f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:42.028 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f483a000640 1 -- 172.21.15.173:0/3023299586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48340c0070 con 0x7f4840158d80 2024-09-17T05:24:42.028 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f483a000640 1 -- 172.21.15.173:0/3023299586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4834000e50 con 0x7f4840158d80 2024-09-17T05:24:42.029 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f483a000640 1 -- 172.21.15.173:0/3023299586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48340b4070 con 0x7f4840158d80 2024-09-17T05:24:42.029 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48401f71f0 con 0x7f4840158d80 2024-09-17T05:24:42.029 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48401f77b0 con 0x7f4840158d80 2024-09-17T05:24:42.029 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f483a000640 1 -- 172.21.15.173:0/3023299586 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f48340c3070 con 0x7f4840158d80 2024-09-17T05:24:42.029 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.023+0000 7f483a000640 1 -- 172.21.15.173:0/3023299586 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f48340c59a0 con 0x7f4840158d80 2024-09-17T05:24:42.030 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.027+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484015ea00 con 0x7f4840158d80 2024-09-17T05:24:42.034 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.031+0000 7f483a000640 1 -- 172.21.15.173:0/3023299586 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f48340186a0 con 0x7f4840158d80 2024-09-17T05:24:42.103 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.099+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "633269aa-0f64-4cf3-93a1-39fc33040a34", "id": 4} v 0) -- 0x7f4840069890 con 0x7f4840158d80 2024-09-17T05:24:42.107 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f483a000640 1 -- 172.21.15.173:0/3023299586 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "633269aa-0f64-4cf3-93a1-39fc33040a34", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f48340c5bf0 con 0x7f4840158d80 2024-09-17T05:24:42.107 INFO:teuthology.orchestra.run.smithi173.stdout:4 2024-09-17T05:24:42.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4840158d80 msgr2=0x7f484015af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4840158d80 0x7f484015af70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f48340b2040 tx=0x7f48340a8f70 comp rx=0 tx=0).stop 2024-09-17T05:24:42.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 shutdown_connections 2024-09-17T05:24:42.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4840158d80 0x7f484015af70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4840159340 0x7f484014bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.109 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 --2- 172.21.15.173:0/3023299586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4828004e50 0x7f48400b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 >> 172.21.15.173:0/3023299586 conn(0x7f48400cfa00 msgr2=0x7f48400cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:42.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 shutdown_connections 2024-09-17T05:24:42.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.103+0000 7f4847200640 1 -- 172.21.15.173:0/3023299586 wait complete. 2024-09-17T05:24:42.126 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 40d2464e-0cf8-448c-841a-794fc74d7d72 5 2024-09-17T05:24:42.270 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 -- 172.21.15.173:0/1989474466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218151960 msgr2=0x7ff218151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.270 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 --2- 172.21.15.173:0/1989474466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218151960 0x7ff218151d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff208009840 tx=0x7ff20801c820 comp rx=0 tx=0).stop 2024-09-17T05:24:42.270 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 -- 172.21.15.173:0/1989474466 shutdown_connections 2024-09-17T05:24:42.270 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 --2- 172.21.15.173:0/1989474466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218151960 0x7ff218151d40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.270 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 -- 172.21.15.173:0/1989474466 >> 172.21.15.173:0/1989474466 conn(0x7ff2180d0da0 msgr2=0x7ff2180d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:42.271 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 -- 172.21.15.173:0/1989474466 shutdown_connections 2024-09-17T05:24:42.271 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 -- 172.21.15.173:0/1989474466 wait complete. 2024-09-17T05:24:42.271 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 Processor -- start 2024-09-17T05:24:42.272 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.267+0000 7ff21dc00640 1 -- start start 2024-09-17T05:24:42.273 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218070720 0x7ff21814fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.273 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2180d00a0 0x7ff2180c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff218151960 0x7ff218153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2180ccea0 con 0x7ff218070720 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff2180ccd20 con 0x7ff218151960 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff2180cd020 con 0x7ff2180d00a0 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2180d00a0 0x7ff2180c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff217400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff218151960 0x7ff218153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2180d00a0 0x7ff2180c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:57392/0 (socket says 172.21.15.173:57392) 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20fe00640 1 -- 172.21.15.173:0/2844703862 learned_addr learned my addr 172.21.15.173:0/2844703862 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff216a00640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218070720 0x7ff21814fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff217400640 1 -- 172.21.15.173:0/2844703862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2180d00a0 msgr2=0x7ff2180c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff217400640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2180d00a0 0x7ff2180c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.274 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff217400640 1 -- 172.21.15.173:0/2844703862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218070720 msgr2=0x7ff21814fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.275 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff217400640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218070720 0x7ff21814fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.275 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff217400640 1 -- 172.21.15.173:0/2844703862 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff218150340 con 0x7ff218151960 2024-09-17T05:24:42.275 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20fe00640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2180d00a0 0x7ff2180c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:42.275 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff216a00640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218070720 0x7ff21814fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:42.275 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff217400640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff218151960 0x7ff218153ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff21000cad0 tx=0x7ff21000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:42.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20ea00640 1 -- 172.21.15.173:0/2844703862 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210016020 con 0x7ff218151960 2024-09-17T05:24:42.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20ea00640 1 -- 172.21.15.173:0/2844703862 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff21000ab60 con 0x7ff218151960 2024-09-17T05:24:42.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20ea00640 1 -- 172.21.15.173:0/2844703862 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000ce10 con 0x7ff218151960 2024-09-17T05:24:42.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2181f60f0 con 0x7ff218151960 2024-09-17T05:24:42.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181f66b0 con 0x7ff218151960 2024-09-17T05:24:42.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20ea00640 1 -- 172.21.15.173:0/2844703862 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff210019020 con 0x7ff218151960 2024-09-17T05:24:42.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff20ea00640 1 -- 172.21.15.173:0/2844703862 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7ff21001b400 con 0x7ff218151960 2024-09-17T05:24:42.276 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.271+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff218151d40 con 0x7ff218151960 2024-09-17T05:24:42.278 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.275+0000 7ff20ea00640 1 -- 172.21.15.173:0/2844703862 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff210020020 con 0x7ff218151960 2024-09-17T05:24:42.293 INFO:tasks.ceph.mgr.x.smithi173.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T05:24:42.293 INFO:tasks.ceph.mgr.x.smithi173.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T05:24:42.293 INFO:tasks.ceph.mgr.x.smithi173.stderr: from numpy import show_config as show_numpy_config 2024-09-17T05:24:42.348 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.343+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "40d2464e-0cf8-448c-841a-794fc74d7d72", "id": 5} v 0) -- 0x7ff21815ad60 con 0x7ff218151960 2024-09-17T05:24:42.355 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff20ea00640 1 -- 172.21.15.173:0/2844703862 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "40d2464e-0cf8-448c-841a-794fc74d7d72", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7ff21001ba00 con 0x7ff218151960 2024-09-17T05:24:42.355 INFO:teuthology.orchestra.run.smithi173.stdout:5 2024-09-17T05:24:42.356 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff218151960 msgr2=0x7ff218153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.357 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff218151960 0x7ff218153ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff21000cad0 tx=0x7ff21000cbc0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.357 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 shutdown_connections 2024-09-17T05:24:42.357 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff218151960 0x7ff218153ec0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.357 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2180d00a0 0x7ff2180c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.357 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 --2- 172.21.15.173:0/2844703862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff218070720 0x7ff21814fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.357 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 >> 172.21.15.173:0/2844703862 conn(0x7ff2180d0da0 msgr2=0x7ff21806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:42.357 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 shutdown_connections 2024-09-17T05:24:42.357 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.351+0000 7ff21dc00640 1 -- 172.21.15.173:0/2844703862 wait complete. 2024-09-17T05:24:42.374 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 97f6f4e4-4eae-445b-b8ea-5ec911516ef2 6 2024-09-17T05:24:42.515 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 -- 172.21.15.173:0/3136666181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa90006fff0 msgr2=0x7fa90006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.515 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 --2- 172.21.15.173:0/3136666181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa90006fff0 0x7fa90006d160 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009c80 tx=0x7fa8f801cda0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.515 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 -- 172.21.15.173:0/3136666181 shutdown_connections 2024-09-17T05:24:42.516 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 --2- 172.21.15.173:0/3136666181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa90006fff0 0x7fa90006d160 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.516 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 --2- 172.21.15.173:0/3136666181 >> v2:172.21.15.122:3300/0 conn(0x7fa9000b6130 0x7fa90006cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.516 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 -- 172.21.15.173:0/3136666181 >> 172.21.15.173:0/3136666181 conn(0x7fa9000d0da0 msgr2=0x7fa9000d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:42.516 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 -- 172.21.15.173:0/3136666181 shutdown_connections 2024-09-17T05:24:42.516 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 -- 172.21.15.173:0/3136666181 wait complete. 2024-09-17T05:24:42.521 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 Processor -- start 2024-09-17T05:24:42.521 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.511+0000 7fa906e00640 1 -- start start 2024-09-17T05:24:42.521 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa90006fff0 0x7fa9000b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.522 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9000b6130 0x7fa900154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.522 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9000b6510 0x7fa900159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.522 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa900168e40 con 0x7fa90006fff0 2024-09-17T05:24:42.522 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa900168cc0 con 0x7fa9000b6130 2024-09-17T05:24:42.522 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa900168fc0 con 0x7fa9000b6510 2024-09-17T05:24:42.523 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9000b6130 0x7fa900154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.523 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa904e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9000b6510 0x7fa900159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.523 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9000b6130 0x7fa900154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:50588/0 (socket says 172.21.15.173:50588) 2024-09-17T05:24:42.523 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8ff400640 1 -- 172.21.15.173:0/2101719868 learned_addr learned my addr 172.21.15.173:0/2101719868 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-17T05:24:42.523 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8ffe00640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa90006fff0 0x7fa9000b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.524 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa904e00640 1 -- 172.21.15.173:0/2101719868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9000b6130 msgr2=0x7fa900154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.524 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa904e00640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9000b6130 0x7fa900154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.524 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa904e00640 1 -- 172.21.15.173:0/2101719868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa90006fff0 msgr2=0x7fa9000b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.524 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa904e00640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa90006fff0 0x7fa9000b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.524 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa904e00640 1 -- 172.21.15.173:0/2101719868 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90015a440 con 0x7fa9000b6510 2024-09-17T05:24:42.524 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8ff400640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9000b6130 0x7fa900154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:42.525 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa904e00640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9000b6510 0x7fa900159510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80ac000 tx=0x7fa8f8009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:42.525 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8f7e00640 1 -- 172.21.15.173:0/2101719868 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80bb070 con 0x7fa9000b6510 2024-09-17T05:24:42.525 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8f7e00640 1 -- 172.21.15.173:0/2101719868 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8f8004d40 con 0x7fa9000b6510 2024-09-17T05:24:42.525 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9001557a0 con 0x7fa9000b6510 2024-09-17T05:24:42.525 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8f7e00640 1 -- 172.21.15.173:0/2101719868 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80b2040 con 0x7fa9000b6510 2024-09-17T05:24:42.525 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa900157040 con 0x7fa9000b6510 2024-09-17T05:24:42.526 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8f7e00640 1 -- 172.21.15.173:0/2101719868 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa8f80be070 con 0x7fa9000b6510 2024-09-17T05:24:42.526 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa8f7e00640 1 -- 172.21.15.173:0/2101719868 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fa8f8005620 con 0x7fa9000b6510 2024-09-17T05:24:42.526 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.515+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90015bdc0 con 0x7fa9000b6510 2024-09-17T05:24:42.526 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.519+0000 7fa8f7e00640 1 -- 172.21.15.173:0/2101719868 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa8f80d8860 con 0x7fa9000b6510 2024-09-17T05:24:42.590 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.587+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "97f6f4e4-4eae-445b-b8ea-5ec911516ef2", "id": 6} v 0) -- 0x7fa90015bae0 con 0x7fa9000b6510 2024-09-17T05:24:42.596 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa8f7e00640 1 -- 172.21.15.173:0/2101719868 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "97f6f4e4-4eae-445b-b8ea-5ec911516ef2", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fa8f80c6030 con 0x7fa9000b6510 2024-09-17T05:24:42.597 INFO:teuthology.orchestra.run.smithi173.stdout:6 2024-09-17T05:24:42.598 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9000b6510 msgr2=0x7fa900159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.598 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9000b6510 0x7fa900159510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80ac000 tx=0x7fa8f8009330 comp rx=0 tx=0).stop 2024-09-17T05:24:42.598 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 shutdown_connections 2024-09-17T05:24:42.599 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9000b6510 0x7fa900159510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.599 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9000b6130 0x7fa900154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.599 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 --2- 172.21.15.173:0/2101719868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa90006fff0 0x7fa9000b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.599 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 >> 172.21.15.173:0/2101719868 conn(0x7fa9000d0da0 msgr2=0x7fa90015d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:42.599 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 shutdown_connections 2024-09-17T05:24:42.599 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.595+0000 7fa906e00640 1 -- 172.21.15.173:0/2101719868 wait complete. 2024-09-17T05:24:42.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 8c603b20-eccf-447c-9117-dba8ff763577 7 2024-09-17T05:24:42.735 INFO:tasks.ceph.mgr.y.smithi122.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T05:24:42.735 INFO:tasks.ceph.mgr.y.smithi122.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T05:24:42.735 INFO:tasks.ceph.mgr.y.smithi122.stderr: from numpy import show_config as show_numpy_config 2024-09-17T05:24:42.760 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.755+0000 7fac08a00640 1 -- 172.21.15.173:0/3165706357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 msgr2=0x7fac04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.760 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.755+0000 7fac08a00640 1 --2- 172.21.15.173:0/3165706357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 0x7fac04068960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009870 tx=0x7fabf401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.760 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.755+0000 7fac08a00640 1 -- 172.21.15.173:0/3165706357 shutdown_connections 2024-09-17T05:24:42.760 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.755+0000 7fac08a00640 1 --2- 172.21.15.173:0/3165706357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 0x7fac04068960 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.761 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.755+0000 7fac08a00640 1 -- 172.21.15.173:0/3165706357 >> 172.21.15.173:0/3165706357 conn(0x7fac040d04f0 msgr2=0x7fac040d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:42.761 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.755+0000 7fac08a00640 1 -- 172.21.15.173:0/3165706357 shutdown_connections 2024-09-17T05:24:42.761 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.755+0000 7fac08a00640 1 -- 172.21.15.173:0/3165706357 wait complete. 2024-09-17T05:24:42.761 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 Processor -- start 2024-09-17T05:24:42.761 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 -- start start 2024-09-17T05:24:42.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0406d330 0x7fac041f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac04070770 0x7fac0415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 0x7fac0415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:42.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac0416a0f0 con 0x7fac04070770 2024-09-17T05:24:42.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fac04169f70 con 0x7fac0406d330 2024-09-17T05:24:42.763 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fac0416a270 con 0x7fac04156590 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0406d330 0x7fac041f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0406d330 0x7fac041f8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:50590/0 (socket says 172.21.15.173:50590) 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 -- 172.21.15.173:0/3412424358 learned_addr learned my addr 172.21.15.173:0/3412424358 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac02800640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 0x7fac0415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fabfbe00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac04070770 0x7fac0415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 -- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 msgr2=0x7fac0415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 0x7fac0415e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 -- 172.21.15.173:0/3412424358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac04070770 msgr2=0x7fac0415e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac04070770 0x7fac0415e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 -- 172.21.15.173:0/3412424358 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0415d9c0 con 0x7fac0406d330 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fabfbe00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac04070770 0x7fac0415e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:42.764 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac02800640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 0x7fac0415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:24:42.765 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac01e00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0406d330 0x7fac041f8ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000c10 tx=0x7fabf4005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:42.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fabfa000640 1 -- 172.21.15.173:0/3412424358 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf40b2040 con 0x7fac0406d330 2024-09-17T05:24:42.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fabfa000640 1 -- 172.21.15.173:0/3412424358 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabf40043e0 con 0x7fac0406d330 2024-09-17T05:24:42.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fabfa000640 1 -- 172.21.15.173:0/3412424358 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf40052d0 con 0x7fac0406d330 2024-09-17T05:24:42.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0415d400 con 0x7fac0406d330 2024-09-17T05:24:42.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac040c5130 con 0x7fac0406d330 2024-09-17T05:24:42.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.759+0000 7fabfa000640 1 -- 172.21.15.173:0/3412424358 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fabf4004a60 con 0x7fac0406d330 2024-09-17T05:24:42.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.763+0000 7fabfa000640 1 -- 172.21.15.173:0/3412424358 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fabf4004d50 con 0x7fac0406d330 2024-09-17T05:24:42.766 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.763+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac04068960 con 0x7fac0406d330 2024-09-17T05:24:42.767 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.763+0000 7fabfa000640 1 -- 172.21.15.173:0/3412424358 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fabf4016450 con 0x7fac0406d330 2024-09-17T05:24:42.836 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.831+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8c603b20-eccf-447c-9117-dba8ff763577", "id": 7} v 0) -- 0x7fac040703c0 con 0x7fac0406d330 2024-09-17T05:24:42.843 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fabfa000640 1 -- 172.21.15.173:0/3412424358 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8c603b20-eccf-447c-9117-dba8ff763577", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fabf4005470 con 0x7fac0406d330 2024-09-17T05:24:42.844 INFO:teuthology.orchestra.run.smithi173.stdout:7 2024-09-17T05:24:42.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0406d330 msgr2=0x7fac041f8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:42.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0406d330 0x7fac041f8ab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000c10 tx=0x7fabf4005d90 comp rx=0 tx=0).stop 2024-09-17T05:24:42.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 shutdown_connections 2024-09-17T05:24:42.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac04156590 0x7fac0415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.845 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac04070770 0x7fac0415e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.846 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 --2- 172.21.15.173:0/3412424358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac0406d330 0x7fac041f8ab0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:42.846 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 >> 172.21.15.173:0/3412424358 conn(0x7fac040d04f0 msgr2=0x7fac040b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:42.846 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 shutdown_connections 2024-09-17T05:24:42.846 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:24:42.839+0000 7fac08a00640 1 -- 172.21.15.173:0/3412424358 wait complete. 2024-09-17T05:24:42.863 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T05:24:42.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T05:24:42.866 INFO:tasks.ceph.osd.0:Started 2024-09-17T05:24:42.867 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T05:24:42.867 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T05:24:42.870 INFO:tasks.ceph.osd.1:Started 2024-09-17T05:24:42.870 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T05:24:42.870 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T05:24:42.874 INFO:tasks.ceph.osd.2:Started 2024-09-17T05:24:42.874 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T05:24:42.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T05:24:42.879 INFO:tasks.ceph.osd.3:Started 2024-09-17T05:24:42.879 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T05:24:42.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T05:24:42.882 INFO:tasks.ceph.osd.4:Started 2024-09-17T05:24:42.882 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T05:24:42.882 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T05:24:42.885 INFO:tasks.ceph.osd.5:Started 2024-09-17T05:24:42.886 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T05:24:42.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T05:24:42.888 INFO:tasks.ceph.osd.6:Started 2024-09-17T05:24:42.889 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T05:24:42.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T05:24:42.892 INFO:tasks.ceph.osd.7:Started 2024-09-17T05:24:42.893 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:24:43.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.072+0000 7f4385200640 1 -- 172.21.15.122:0/1954525667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800d28e0 msgr2=0x7f43800cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:43.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.072+0000 7f4385200640 1 --2- 172.21.15.122:0/1954525667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800d28e0 0x7f43800cebb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43800745d0 tx=0x7f437801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:24:43.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.072+0000 7f4385200640 1 -- 172.21.15.122:0/1954525667 shutdown_connections 2024-09-17T05:24:43.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.072+0000 7f4385200640 1 --2- 172.21.15.122:0/1954525667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800d28e0 0x7f43800cebb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:43.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.072+0000 7f4385200640 1 --2- 172.21.15.122:0/1954525667 >> v2:172.21.15.173:3300/0 conn(0x7f438015bea0 0x7f43800ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:43.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.072+0000 7f4385200640 1 -- 172.21.15.122:0/1954525667 >> 172.21.15.122:0/1954525667 conn(0x7f43800d1dc0 msgr2=0x7f43800d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:43.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.072+0000 7f4385200640 1 -- 172.21.15.122:0/1954525667 shutdown_connections 2024-09-17T05:24:43.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.072+0000 7f4385200640 1 -- 172.21.15.122:0/1954525667 wait complete. 2024-09-17T05:24:43.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 Processor -- start 2024-09-17T05:24:43.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 -- start start 2024-09-17T05:24:43.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800cdb30 0x7f4380156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:43.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43800d28e0 0x7f43801f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f438015bea0 0x7f43801f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4380168fc0 con 0x7f43800cdb30 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4380168e40 con 0x7f43800d28e0 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4380169140 con 0x7f438015bea0 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800cdb30 0x7f4380156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800cdb30 0x7f4380156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55532/0 (socket says 172.21.15.122:55532) 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 -- 172.21.15.122:0/3717498267 learned_addr learned my addr 172.21.15.122:0/3717498267 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4377e00640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43800d28e0 0x7f43801f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437e800640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f438015bea0 0x7f43801f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 -- 172.21.15.122:0/3717498267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f438015bea0 msgr2=0x7f43801f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f438015bea0 0x7f43801f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 -- 172.21.15.122:0/3717498267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43800d28e0 msgr2=0x7f43801f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43800d28e0 0x7f43801f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 -- 172.21.15.122:0/3717498267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43801f86c0 con 0x7f43800cdb30 2024-09-17T05:24:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f437de00640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800cdb30 0x7f4380156bf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f437000e7a0 tx=0x7f437000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:43.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4376000640 1 -- 172.21.15.122:0/3717498267 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437000ccc0 con 0x7f43800cdb30 2024-09-17T05:24:43.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4376000640 1 -- 172.21.15.122:0/3717498267 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4370010430 con 0x7f43800cdb30 2024-09-17T05:24:43.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43801f89a0 con 0x7f43800cdb30 2024-09-17T05:24:43.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4376000640 1 -- 172.21.15.122:0/3717498267 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370010860 con 0x7f43800cdb30 2024-09-17T05:24:43.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.076+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438006e220 con 0x7f43800cdb30 2024-09-17T05:24:43.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.080+0000 7f4376000640 1 -- 172.21.15.122:0/3717498267 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4370010a00 con 0x7f43800cdb30 2024-09-17T05:24:43.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.080+0000 7f4376000640 1 -- 172.21.15.122:0/3717498267 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4370010cf0 con 0x7f43800cdb30 2024-09-17T05:24:43.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.080+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43800cc9d0 con 0x7f43800cdb30 2024-09-17T05:24:43.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.080+0000 7f4376000640 1 -- 172.21.15.122:0/3717498267 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4370018bf0 con 0x7f43800cdb30 2024-09-17T05:24:43.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43800b0ac0 con 0x7f43800cdb30 2024-09-17T05:24:43.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4376000640 1 -- 172.21.15.122:0/3717498267 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4370018dd0 con 0x7f43800cdb30 2024-09-17T05:24:43.145 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:43.145 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":10,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:24:42.838555+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800cdb30 msgr2=0x7f4380156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800cdb30 0x7f4380156bf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f437000e7a0 tx=0x7f437000ec60 comp rx=0 tx=0).stop 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 shutdown_connections 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f438015bea0 0x7f43801f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43800d28e0 0x7f43801f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 --2- 172.21.15.122:0/3717498267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43800cdb30 0x7f4380156bf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 >> 172.21.15.122:0/3717498267 conn(0x7f43800d1dc0 msgr2=0x7f43800b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 shutdown_connections 2024-09-17T05:24:43.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:43.140+0000 7f4385200640 1 -- 172.21.15.122:0/3717498267 wait complete. 2024-09-17T05:24:43.164 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T05:24:43.164 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T05:24:43.971 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:24:44.104 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- 172.21.15.122:0/3145700813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 msgr2=0x7f20100cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:44.104 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 --2- 172.21.15.122:0/3145700813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 0x7f20100cecc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f201006f290 tx=0x7f1ff801c830 comp rx=0 tx=0).stop 2024-09-17T05:24:44.104 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- 172.21.15.122:0/3145700813 shutdown_connections 2024-09-17T05:24:44.104 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 --2- 172.21.15.122:0/3145700813 >> v2:172.21.15.173:3300/0 conn(0x7f20100ccf90 0x7f20100cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:44.104 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 --2- 172.21.15.122:0/3145700813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 0x7f20100cecc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:44.104 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- 172.21.15.122:0/3145700813 >> 172.21.15.122:0/3145700813 conn(0x7f20100d1d30 msgr2=0x7f20100d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:44.104 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- 172.21.15.122:0/3145700813 shutdown_connections 2024-09-17T05:24:44.105 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- 172.21.15.122:0/3145700813 wait complete. 2024-09-17T05:24:44.105 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 Processor -- start 2024-09-17T05:24:44.105 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- start start 2024-09-17T05:24:44.108 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 0x7f2010158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:44.108 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20100ccf90 0x7f20100c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:44.108 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f201015e0e0 0x7f20100ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:44.108 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f201016b0d0 con 0x7f201006e740 2024-09-17T05:24:44.108 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f201016af50 con 0x7f201015e0e0 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f2017000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f201016b250 con 0x7f20100ccf90 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.100+0000 7f200fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 0x7f2010158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 0x7f2010158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55550/0 (socket says 172.21.15.122:55550) 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200fe00640 1 -- 172.21.15.122:0/148663854 learned_addr learned my addr 172.21.15.122:0/148663854 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f2007e00640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20100ccf90 0x7f20100c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f2015000640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f201015e0e0 0x7f20100ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200fe00640 1 -- 172.21.15.122:0/148663854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20100ccf90 msgr2=0x7f20100c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200fe00640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20100ccf90 0x7f20100c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200fe00640 1 -- 172.21.15.122:0/148663854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f201015e0e0 msgr2=0x7f20100ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:44.109 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200fe00640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f201015e0e0 0x7f20100ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200fe00640 1 -- 172.21.15.122:0/148663854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101554e0 con 0x7f201006e740 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200fe00640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 0x7f2010158bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f200000c3f0 tx=0x7f200000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200cc00640 1 -- 172.21.15.122:0/148663854 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000017070 con 0x7f201006e740 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200cc00640 1 -- 172.21.15.122:0/148663854 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20000053c0 con 0x7f201006e740 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20101557c0 con 0x7f201006e740 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200cc00640 1 -- 172.21.15.122:0/148663854 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000003860 con 0x7f201006e740 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20100cabe0 con 0x7f201006e740 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200cc00640 1 -- 172.21.15.122:0/148663854 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f200001a070 con 0x7f201006e740 2024-09-17T05:24:44.110 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f200cc00640 1 -- 172.21.15.122:0/148663854 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2000011c20 con 0x7f201006e740 2024-09-17T05:24:44.111 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.104+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20100cd3c0 con 0x7f201006e740 2024-09-17T05:24:44.111 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.108+0000 7f200cc00640 1 -- 172.21.15.122:0/148663854 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2000021020 con 0x7f201006e740 2024-09-17T05:24:44.170 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.164+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20100d1870 con 0x7f201006e740 2024-09-17T05:24:44.170 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f200cc00640 1 -- 172.21.15.122:0/148663854 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2000011590 con 0x7f201006e740 2024-09-17T05:24:44.171 INFO:teuthology.misc.health.smithi122.stdout: 2024-09-17T05:24:44.171 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":10,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:24:42.838555+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:24:44.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 msgr2=0x7f2010158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:44.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 0x7f2010158bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f200000c3f0 tx=0x7f200000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:24:44.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 shutdown_connections 2024-09-17T05:24:44.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f201015e0e0 0x7f20100ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:44.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20100ccf90 0x7f20100c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:44.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 --2- 172.21.15.122:0/148663854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f201006e740 0x7f2010158bf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:44.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 >> 172.21.15.122:0/148663854 conn(0x7f20100d1d30 msgr2=0x7f20100b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:44.173 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 shutdown_connections 2024-09-17T05:24:44.173 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:44.168+0000 7f2017000640 1 -- 172.21.15.122:0/148663854 wait complete. 2024-09-17T05:24:44.188 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T05:24:44.552 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-17T05:24:44.548+0000 7f959f6547c0 -1 Falling back to public interface 2024-09-17T05:24:44.579 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-17T05:24:44.576+0000 7fc44302b7c0 -1 Falling back to public interface 2024-09-17T05:24:44.591 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-17T05:24:44.588+0000 7f7f1102b7c0 -1 Falling back to public interface 2024-09-17T05:24:44.626 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-09-17T05:24:44.623+0000 7f2f0a9e87c0 -1 Falling back to public interface 2024-09-17T05:24:44.776 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-17T05:24:44.772+0000 7fd6150e67c0 -1 Falling back to public interface 2024-09-17T05:24:44.797 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-09-17T05:24:44.795+0000 7fe4528de7c0 -1 Falling back to public interface 2024-09-17T05:24:44.809 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-09-17T05:24:44.807+0000 7f8052be17c0 -1 Falling back to public interface 2024-09-17T05:24:44.930 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-09-17T05:24:44.927+0000 7ff7ac6cf7c0 -1 Falling back to public interface 2024-09-17T05:24:45.450 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:24:45.447+0000 7fc0c3400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T05:24:45.452 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:24:45.447+0000 7fc0c3400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T05:24:48.861 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-09-17T05:24:48.859+0000 7f2f0a9e87c0 -1 osd.5 0 log_to_monitors true 2024-09-17T05:24:48.943 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-17T05:24:48.943+0000 7fd6150e67c0 -1 osd.1 0 log_to_monitors true 2024-09-17T05:24:49.082 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-17T05:24:49.079+0000 7fc44302b7c0 -1 osd.0 0 log_to_monitors true 2024-09-17T05:24:49.105 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-17T05:24:49.103+0000 7f7f1102b7c0 -1 osd.2 0 log_to_monitors true 2024-09-17T05:24:49.112 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-17T05:24:49.111+0000 7f959f6547c0 -1 osd.3 0 log_to_monitors true 2024-09-17T05:24:49.172 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-09-17T05:24:49.167+0000 7ff7ac6cf7c0 -1 osd.6 0 log_to_monitors true 2024-09-17T05:24:49.293 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-09-17T05:24:49.291+0000 7f8052be17c0 -1 osd.7 0 log_to_monitors true 2024-09-17T05:24:49.711 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-09-17T05:24:49.707+0000 7fe4528de7c0 -1 osd.4 0 log_to_monitors true 2024-09-17T05:24:50.999 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:24:51.164 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- 172.21.15.122:0/2674727136 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d80d16f0 msgr2=0x7f16d80d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:51.164 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 --2- 172.21.15.122:0/2674727136 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d80d16f0 0x7f16d80d1b90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009bc0 tx=0x7f16d001ca80 comp rx=0 tx=0).stop 2024-09-17T05:24:51.164 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- 172.21.15.122:0/2674727136 shutdown_connections 2024-09-17T05:24:51.164 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 --2- 172.21.15.122:0/2674727136 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d80d16f0 0x7f16d80d1b90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.164 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 --2- 172.21.15.122:0/2674727136 >> v2:172.21.15.173:3300/0 conn(0x7f16d80b4bc0 0x7f16d81623d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.164 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- 172.21.15.122:0/2674727136 >> 172.21.15.122:0/2674727136 conn(0x7f16d80d3fc0 msgr2=0x7f16d80d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:51.164 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- 172.21.15.122:0/2674727136 shutdown_connections 2024-09-17T05:24:51.164 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- 172.21.15.122:0/2674727136 wait complete. 2024-09-17T05:24:51.165 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 Processor -- start 2024-09-17T05:24:51.165 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- start start 2024-09-17T05:24:51.167 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16d80d16f0 0x7f16d814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:51.167 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16d81614b0 0x7f16d80cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:51.167 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d8150c00 0x7f16d8150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:51.167 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f16d816b390 con 0x7f16d81614b0 2024-09-17T05:24:51.167 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f16d816b210 con 0x7f16d80d16f0 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dee00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f16d816b510 con 0x7f16d8150c00 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16d7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16d81614b0 0x7f16d80cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16d7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16d81614b0 0x7f16d80cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55610/0 (socket says 172.21.15.122:55610) 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16d7400640 1 -- 172.21.15.122:0/1723578857 learned_addr learned my addr 172.21.15.122:0/1723578857 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16dce00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d8150c00 0x7f16d8150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.163+0000 7f16d7e00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16d80d16f0 0x7f16d814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dce00640 1 -- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16d80d16f0 msgr2=0x7f16d814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dce00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16d80d16f0 0x7f16d814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dce00640 1 -- 172.21.15.122:0/1723578857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16d81614b0 msgr2=0x7f16d80cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dce00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16d81614b0 0x7f16d80cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dce00640 1 -- 172.21.15.122:0/1723578857 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d81f82f0 con 0x7f16d8150c00 2024-09-17T05:24:51.168 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16d7400640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16d81614b0 0x7f16d80cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:51.169 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dce00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d8150c00 0x7f16d8150240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f16d80cf740 tx=0x7f16d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:51.169 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16cfe00640 1 -- 172.21.15.122:0/1723578857 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d00bb070 con 0x7f16d8150c00 2024-09-17T05:24:51.169 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16cfe00640 1 -- 172.21.15.122:0/1723578857 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16d0004d40 con 0x7f16d8150c00 2024-09-17T05:24:51.169 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16cfe00640 1 -- 172.21.15.122:0/1723578857 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d00b2040 con 0x7f16d8150c00 2024-09-17T05:24:51.169 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d81f85d0 con 0x7f16d8150c00 2024-09-17T05:24:51.170 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d814fa40 con 0x7f16d8150c00 2024-09-17T05:24:51.170 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.167+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d80d1b90 con 0x7f16d8150c00 2024-09-17T05:24:51.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.171+0000 7f16cfe00640 1 -- 172.21.15.122:0/1723578857 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f16d0004220 con 0x7f16d8150c00 2024-09-17T05:24:51.172 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.171+0000 7f16cfe00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f16b8073170 0x7f16b8075630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:51.173 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.171+0000 7f16cfe00640 1 -- 172.21.15.122:0/1723578857 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f16d012d1c0 con 0x7f16d8150c00 2024-09-17T05:24:51.173 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.171+0000 7f16d7e00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f16b8073170 0x7f16b8075630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:51.173 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.171+0000 7f16d7e00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f16b8073170 0x7f16b8075630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f16d815a180 tx=0x7f16c80013c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:51.199 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-09-17T05:24:51.195+0000 7f2f05800640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T05:24:51.210 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.207+0000 7f16cfe00640 1 -- 172.21.15.122:0/1723578857 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f16d0016790 con 0x7f16d8150c00 2024-09-17T05:24:51.378 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-09-17T05:24:51.375+0000 7f2efb800640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:24:51.431 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.431+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16d80b9f40 con 0x7f16d8150c00 2024-09-17T05:24:51.432 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.431+0000 7f16cfe00640 1 -- 172.21.15.122:0/1723578857 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f16d00f89f0 con 0x7f16d8150c00 2024-09-17T05:24:51.432 INFO:teuthology.misc.health.smithi122.stdout: 2024-09-17T05:24:51.433 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":13,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:24:50.921400+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:24:51.435 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f16b8073170 msgr2=0x7f16b8075630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:51.435 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f16b8073170 0x7f16b8075630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f16d815a180 tx=0x7f16c80013c0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.435 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d8150c00 msgr2=0x7f16d8150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:51.435 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d8150c00 0x7f16d8150240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f16d80cf740 tx=0x7f16d0009260 comp rx=0 tx=0).stop 2024-09-17T05:24:51.435 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 shutdown_connections 2024-09-17T05:24:51.435 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f16b8073170 0x7f16b8075630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.435 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16d8150c00 0x7f16d8150240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.436 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16d81614b0 0x7f16d80cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.436 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 --2- 172.21.15.122:0/1723578857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16d80d16f0 0x7f16d814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:51.436 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 >> 172.21.15.122:0/1723578857 conn(0x7f16d80d3fc0 msgr2=0x7f16d80d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:51.436 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 shutdown_connections 2024-09-17T05:24:51.436 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:51.435+0000 7f16dee00640 1 -- 172.21.15.122:0/1723578857 wait complete. 2024-09-17T05:24:51.456 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T05:24:52.723 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-09-17T05:24:52.719+0000 7fe44d800640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T05:24:52.754 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-09-17T05:24:52.751+0000 7fe443800640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:24:52.786 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-09-17T05:24:52.783+0000 7ff7a7600640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T05:24:52.791 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-09-17T05:24:52.787+0000 7f804da00640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T05:24:52.810 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-09-17T05:24:52.807+0000 7ff79d600640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:24:52.817 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-09-17T05:24:52.815+0000 7f8043a00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:24:53.195 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-17T05:24:53.195+0000 7f7f0be00640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T05:24:53.211 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-17T05:24:53.211+0000 7fc43d400640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T05:24:53.211 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-17T05:24:53.211+0000 7fd610000640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T05:24:53.232 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-17T05:24:53.231+0000 7f7f01400640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:24:53.233 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-17T05:24:53.231+0000 7f959a600640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T05:24:53.248 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-17T05:24:53.247+0000 7fc433400640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:24:53.256 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-17T05:24:53.255+0000 7fd606000640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:24:53.259 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-17T05:24:53.259+0000 7f9590600640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:24:58.263 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:24:58.405 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 -- 172.21.15.122:0/3180835215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f788415bcf0 msgr2=0x7f78840b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.405 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 --2- 172.21.15.122:0/3180835215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f788415bcf0 0x7f78840b39e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78740098f0 tx=0x7f787401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.405 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 -- 172.21.15.122:0/3180835215 shutdown_connections 2024-09-17T05:24:58.405 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 --2- 172.21.15.122:0/3180835215 >> v2:172.21.15.173:3300/0 conn(0x7f78840b3520 0x7f78840b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.405 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 --2- 172.21.15.122:0/3180835215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f788415bcf0 0x7f78840b39e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.406 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 -- 172.21.15.122:0/3180835215 >> 172.21.15.122:0/3180835215 conn(0x7f7884069360 msgr2=0x7f788409aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:58.406 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 -- 172.21.15.122:0/3180835215 shutdown_connections 2024-09-17T05:24:58.406 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 -- 172.21.15.122:0/3180835215 wait complete. 2024-09-17T05:24:58.406 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 Processor -- start 2024-09-17T05:24:58.406 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.403+0000 7f788b400640 1 -- start start 2024-09-17T05:24:58.407 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78840b12f0 0x7f788414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:58.407 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78840b3520 0x7f788414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:58.408 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f788415bcf0 0x7f78840c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:58.408 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7884169330 con 0x7f78840b12f0 2024-09-17T05:24:58.408 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f78841691b0 con 0x7f78840b3520 2024-09-17T05:24:58.408 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f78841694b0 con 0x7f788415bcf0 2024-09-17T05:24:58.408 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78840b12f0 0x7f788414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:58.408 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78840b12f0 0x7f788414e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53142/0 (socket says 172.21.15.122:53142) 2024-09-17T05:24:58.408 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 -- 172.21.15.122:0/360185083 learned_addr learned my addr 172.21.15.122:0/360185083 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:58.409 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f787be00640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78840b3520 0x7f788414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:58.409 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 -- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f788415bcf0 msgr2=0x7f78840c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.409 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7889400640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f788415bcf0 0x7f78840c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:58.409 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f788415bcf0 0x7f78840c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.409 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 -- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78840b3520 msgr2=0x7f788414dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78840b3520 0x7f788414dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 -- 172.21.15.122:0/360185083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78840c61e0 con 0x7f78840b12f0 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f787be00640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78840b3520 0x7f788414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7888a00640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78840b12f0 0x7f788414e1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c10 tx=0x7f7874005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7881600640 1 -- 172.21.15.122:0/360185083 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78740b2040 con 0x7f78840b12f0 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7881600640 1 -- 172.21.15.122:0/360185083 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7874003600 con 0x7f78840b12f0 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78841595a0 con 0x7f78840b12f0 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f7881600640 1 -- 172.21.15.122:0/360185083 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874003a10 con 0x7f78840b12f0 2024-09-17T05:24:58.410 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7884159b60 con 0x7f78840b12f0 2024-09-17T05:24:58.411 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.407+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788415f8a0 con 0x7f78840b12f0 2024-09-17T05:24:58.411 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.411+0000 7f7881600640 1 -- 172.21.15.122:0/360185083 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f78740bd030 con 0x7f78840b12f0 2024-09-17T05:24:58.412 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.411+0000 7f7881600640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7860073230 0x7f78600756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:58.412 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.411+0000 7f7881600640 1 -- 172.21.15.122:0/360185083 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f787412de40 con 0x7f78840b12f0 2024-09-17T05:24:58.412 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.411+0000 7f787be00640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7860073230 0x7f78600756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:58.413 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.411+0000 7f787be00640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7860073230 0x7f78600756f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7884150c40 tx=0x7f786c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:58.416 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.415+0000 7f7881600640 1 -- 172.21.15.122:0/360185083 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f78740c4060 con 0x7f78840b12f0 2024-09-17T05:24:58.589 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7884072210 con 0x7f78840b12f0 2024-09-17T05:24:58.589 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f7881600640 1 -- 172.21.15.122:0/360185083 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12956 (secure 0 0 0) 0x7f78740164d0 con 0x7f78840b12f0 2024-09-17T05:24:58.590 INFO:teuthology.misc.health.smithi122.stdout: 2024-09-17T05:24:58.590 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":18,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:24:56.737364+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7860073230 msgr2=0x7f78600756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7860073230 0x7f78600756f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7884150c40 tx=0x7f786c009040 comp rx=0 tx=0).stop 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78840b12f0 msgr2=0x7f788414e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78840b12f0 0x7f788414e1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c10 tx=0x7f7874005d00 comp rx=0 tx=0).stop 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 shutdown_connections 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7860073230 0x7f78600756f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f788415bcf0 0x7f78840c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78840b3520 0x7f788414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 --2- 172.21.15.122:0/360185083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78840b12f0 0x7f788414e1d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.591 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 >> 172.21.15.122:0/360185083 conn(0x7f7884069360 msgr2=0x7f7884153320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:58.592 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 shutdown_connections 2024-09-17T05:24:58.592 INFO:teuthology.misc.health.smithi122.stderr:2024-09-17T05:24:58.587+0000 7f788b400640 1 -- 172.21.15.122:0/360185083 wait complete. 2024-09-17T05:24:58.607 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T05:24:58.608 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T05:24:58.608 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:24:58.608 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:24:58.611 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:24:58.612 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:24:58.612 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:24:58.614 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:24:58.614 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:24:58.615 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:24:58.618 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:24:58.618 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:24:58.618 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:24:58.621 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:24:58.621 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-17T05:24:58.621 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:24:58.711 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:24:58.713 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:24:58.723 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:24:58.724 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:24:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 -- 172.21.15.122:0/1839274388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200cdb30 msgr2=0x7fed200cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 --2- 172.21.15.122:0/1839274388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200cdb30 0x7fed200cebb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009900 tx=0x7fed0c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 -- 172.21.15.122:0/1839274388 shutdown_connections 2024-09-17T05:24:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 --2- 172.21.15.122:0/1839274388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200cdb30 0x7fed200cebb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 --2- 172.21.15.122:0/1839274388 >> v2:172.21.15.173:3300/0 conn(0x7fed2015bea0 0x7fed200ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 -- 172.21.15.122:0/1839274388 >> 172.21.15.122:0/1839274388 conn(0x7fed200d1dc0 msgr2=0x7fed200d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 -- 172.21.15.122:0/1839274388 shutdown_connections 2024-09-17T05:24:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 -- 172.21.15.122:0/1839274388 wait complete. 2024-09-17T05:24:58.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 Processor -- start 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.771+0000 7fed24a00640 1 -- start start 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed200cdb30 0x7fed200cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200d28e0 0x7fed20141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2015bea0 0x7fed201565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed20168fc0 con 0x7fed2015bea0 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed20168e40 con 0x7fed200cdb30 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fed20169140 con 0x7fed200d28e0 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2015bea0 0x7fed201565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1d400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200d28e0 0x7fed20141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed200cdb30 0x7fed200cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed200cdb30 0x7fed200cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43020/0 (socket says 172.21.15.122:43020) 2024-09-17T05:24:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2015bea0 0x7fed201565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53174/0 (socket says 172.21.15.122:53174) 2024-09-17T05:24:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1d400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200d28e0 0x7fed20141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:49048/0 (socket says 172.21.15.122:49048) 2024-09-17T05:24:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 -- 172.21.15.122:0/2755214201 learned_addr learned my addr 172.21.15.122:0/2755214201 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:24:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 -- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200d28e0 msgr2=0x7fed20141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200d28e0 0x7fed20141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 -- 172.21.15.122:0/2755214201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2015bea0 msgr2=0x7fed201565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2015bea0 0x7fed201565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 -- 172.21.15.122:0/2755214201 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed200b2ee0 con 0x7fed200cdb30 2024-09-17T05:24:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1e800640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2015bea0 0x7fed201565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:24:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed1de00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed200cdb30 0x7fed200cb7c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e7a0 tx=0x7fed1000ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed16000640 1 -- 172.21.15.122:0/2755214201 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1000ccc0 con 0x7fed200cdb30 2024-09-17T05:24:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed16000640 1 -- 172.21.15.122:0/2755214201 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed10010430 con 0x7fed200cdb30 2024-09-17T05:24:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed200b2120 con 0x7fed200cdb30 2024-09-17T05:24:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed16000640 1 -- 172.21.15.122:0/2755214201 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10010860 con 0x7fed200cdb30 2024-09-17T05:24:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed200b26e0 con 0x7fed200cdb30 2024-09-17T05:24:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.775+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed200cc9d0 con 0x7fed200cdb30 2024-09-17T05:24:58.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.779+0000 7fed16000640 1 -- 172.21.15.122:0/2755214201 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fed10010a00 con 0x7fed200cdb30 2024-09-17T05:24:58.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.779+0000 7fed16000640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed00073200 0x7fed000756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:24:58.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.779+0000 7fed16000640 1 -- 172.21.15.122:0/2755214201 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fed10094bb0 con 0x7fed200cdb30 2024-09-17T05:24:58.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.779+0000 7fed1d400640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed00073200 0x7fed000756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:24:58.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.779+0000 7fed1d400640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed00073200 0x7fed000756c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0af000 tx=0x7fed0c01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:24:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.783+0000 7fed16000640 1 -- 172.21.15.122:0/2755214201 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fed1005f940 con 0x7fed200cdb30 2024-09-17T05:24:58.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.983+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed200af4f0 con 0x7fed200cdb30 2024-09-17T05:24:58.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.983+0000 7fed16000640 1 -- 172.21.15.122:0/2755214201 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fed10064370 con 0x7fed200cdb30 2024-09-17T05:24:58.987 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:24:58.987 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"btime":"2024-09-17T05:24:58:724450+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T05:24:58.987 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 2 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed00073200 msgr2=0x7fed000756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed00073200 0x7fed000756c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0af000 tx=0x7fed0c01b120 comp rx=0 tx=0).stop 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed200cdb30 msgr2=0x7fed200cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed200cdb30 0x7fed200cb7c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e7a0 tx=0x7fed1000ec60 comp rx=0 tx=0).stop 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 shutdown_connections 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed00073200 0x7fed000756c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2015bea0 0x7fed201565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed200d28e0 0x7fed20141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 --2- 172.21.15.122:0/2755214201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed200cdb30 0x7fed200cb7c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:24:58.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 >> 172.21.15.122:0/2755214201 conn(0x7fed200d1dc0 msgr2=0x7fed200d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:24:58.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 shutdown_connections 2024-09-17T05:24:58.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:24:58.987+0000 7fed24a00640 1 -- 172.21.15.122:0/2755214201 wait complete. 2024-09-17T05:25:01.007 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:01.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.147+0000 7fc079400640 1 -- 172.21.15.122:0/3082973288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 msgr2=0x7fc0740ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.147+0000 7fc079400640 1 --2- 172.21.15.122:0/3082973288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 0x7fc0740ce2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc0740b26f0 tx=0x7fc06c01cb50 comp rx=0 tx=0).stop 2024-09-17T05:25:01.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.147+0000 7fc079400640 1 -- 172.21.15.122:0/3082973288 shutdown_connections 2024-09-17T05:25:01.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.147+0000 7fc079400640 1 --2- 172.21.15.122:0/3082973288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 0x7fc0740ce2e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.147+0000 7fc079400640 1 --2- 172.21.15.122:0/3082973288 >> v2:172.21.15.173:3300/0 conn(0x7fc07415f9a0 0x7fc07406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.147+0000 7fc079400640 1 -- 172.21.15.122:0/3082973288 >> 172.21.15.122:0/3082973288 conn(0x7fc0740b18f0 msgr2=0x7fc0740b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:01.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.147+0000 7fc079400640 1 -- 172.21.15.122:0/3082973288 shutdown_connections 2024-09-17T05:25:01.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.147+0000 7fc079400640 1 -- 172.21.15.122:0/3082973288 wait complete. 2024-09-17T05:25:01.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 Processor -- start 2024-09-17T05:25:01.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 -- start start 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0740cf1d0 0x7fc074148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc07415f9a0 0x7fc0740b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 0x7fc0740b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc074169150 con 0x7fc074160490 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc074168fd0 con 0x7fc07415f9a0 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc0741692d0 con 0x7fc0740cf1d0 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 0x7fc0740b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 0x7fc0740b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53190/0 (socket says 172.21.15.122:53190) 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 -- 172.21.15.122:0/2189244597 learned_addr learned my addr 172.21.15.122:0/2189244597 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc072800640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0740cf1d0 0x7fc074148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 -- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0740cf1d0 msgr2=0x7fc074148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0740cf1d0 0x7fc074148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 -- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc07415f9a0 msgr2=0x7fc0740b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc07415f9a0 0x7fc0740b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 -- 172.21.15.122:0/2189244597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0740aeeb0 con 0x7fc074160490 2024-09-17T05:25:01.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc073200640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 0x7fc0740b0df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc06c01a780 tx=0x7fc06c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:01.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc06aa00640 1 -- 172.21.15.122:0/2189244597 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0bb070 con 0x7fc074160490 2024-09-17T05:25:01.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc06aa00640 1 -- 172.21.15.122:0/2189244597 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc06c004d40 con 0x7fc074160490 2024-09-17T05:25:01.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc06aa00640 1 -- 172.21.15.122:0/2189244597 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b2040 con 0x7fc074160490 2024-09-17T05:25:01.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0740af170 con 0x7fc074160490 2024-09-17T05:25:01.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.151+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc074158840 con 0x7fc074160490 2024-09-17T05:25:01.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.155+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074069410 con 0x7fc074160490 2024-09-17T05:25:01.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.155+0000 7fc06aa00640 1 -- 172.21.15.122:0/2189244597 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc06c0a5b60 con 0x7fc074160490 2024-09-17T05:25:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.155+0000 7fc06aa00640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc054073230 0x7fc0540756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.155+0000 7fc06aa00640 1 -- 172.21.15.122:0/2189244597 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fc06c12dc70 con 0x7fc074160490 2024-09-17T05:25:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.155+0000 7fc072800640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc054073230 0x7fc0540756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.155+0000 7fc072800640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc054073230 0x7fc0540756f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc064009b70 tx=0x7fc0640091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.159+0000 7fc06aa00640 1 -- 172.21.15.122:0/2189244597 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc06c016770 con 0x7fc074160490 2024-09-17T05:25:01.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc07406e7a0 con 0x7fc074160490 2024-09-17T05:25:01.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc06aa00640 1 -- 172.21.15.122:0/2189244597 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3912 (secure 0 0 0) 0x7fc06c0f8f50 con 0x7fc074160490 2024-09-17T05:25:01.365 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:01.365 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"btime":"2024-09-17T05:24:59:725917+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4332,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-17T05:25:01.365 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 3 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc054073230 msgr2=0x7fc0540756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc054073230 0x7fc0540756f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc064009b70 tx=0x7fc0640091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 msgr2=0x7fc0740b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 0x7fc0740b0df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc06c01a780 tx=0x7fc06c009220 comp rx=0 tx=0).stop 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 shutdown_connections 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc054073230 0x7fc0540756f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc074160490 0x7fc0740b0df0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc07415f9a0 0x7fc0740b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 --2- 172.21.15.122:0/2189244597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0740cf1d0 0x7fc074148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 >> 172.21.15.122:0/2189244597 conn(0x7fc0740b18f0 msgr2=0x7fc07406f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:01.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 shutdown_connections 2024-09-17T05:25:01.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.363+0000 7fc079400640 1 -- 172.21.15.122:0/2189244597 wait complete. 2024-09-17T05:25:01.379 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:25:01.382 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:25:01.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- 172.21.15.122:0/1987473317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 msgr2=0x7f59a80d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 --2- 172.21.15.122:0/1987473317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 0x7f59a80d01b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59a80c56a0 tx=0x7f599c01c830 comp rx=0 tx=0).stop 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- 172.21.15.122:0/1987473317 shutdown_connections 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 --2- 172.21.15.122:0/1987473317 >> v2:172.21.15.173:3300/0 conn(0x7f59a80cefb0 0x7f59a80cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 --2- 172.21.15.122:0/1987473317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a80ccaf0 0x7f59a80b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 --2- 172.21.15.122:0/1987473317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 0x7f59a80d01b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- 172.21.15.122:0/1987473317 >> 172.21.15.122:0/1987473317 conn(0x7f59a809aa10 msgr2=0x7f59a809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- 172.21.15.122:0/1987473317 shutdown_connections 2024-09-17T05:25:01.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- 172.21.15.122:0/1987473317 wait complete. 2024-09-17T05:25:01.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 Processor -- start 2024-09-17T05:25:01.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- start start 2024-09-17T05:25:01.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a80ccaf0 0x7f59a815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a80cefb0 0x7f59a80c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 0x7f59a80ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59a815ff80 con 0x7f59a815f000 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59a815fe00 con 0x7f59a80ccaf0 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59aec00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f59a8160100 con 0x7f59a80cefb0 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59ae200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 0x7f59a80ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59ae200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 0x7f59a80ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36550/0 (socket says 172.21.15.122:36550) 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59ae200640 1 -- 172.21.15.122:0/712721984 learned_addr learned my addr 172.21.15.122:0/712721984 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59ad800640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a80ccaf0 0x7f59a815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.567+0000 7f59ace00640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a80cefb0 0x7f59a80c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59ae200640 1 -- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a80cefb0 msgr2=0x7f59a80c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59ae200640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a80cefb0 0x7f59a80c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59ae200640 1 -- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a80ccaf0 msgr2=0x7f59a815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59ae200640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a80ccaf0 0x7f59a815b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59ae200640 1 -- 172.21.15.122:0/712721984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a815bb80 con 0x7f59a815f000 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59ae200640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 0x7f59a80ca7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e990 tx=0x7f59a000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59a5600640 1 -- 172.21.15.122:0/712721984 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0004d30 con 0x7f59a815f000 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59a5600640 1 -- 172.21.15.122:0/712721984 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59a000cb00 con 0x7f59a815f000 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59a5600640 1 -- 172.21.15.122:0/712721984 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0005020 con 0x7f59a815f000 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a80c5760 con 0x7f59a815f000 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a80c5d20 con 0x7f59a815f000 2024-09-17T05:25:01.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a80cc5b0 con 0x7f59a815f000 2024-09-17T05:25:01.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.571+0000 7f59a5600640 1 -- 172.21.15.122:0/712721984 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f59a00051c0 con 0x7f59a815f000 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.575+0000 7f59a5600640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5988073200 0x7f59880756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.575+0000 7f59a5600640 1 -- 172.21.15.122:0/712721984 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f59a00946a0 con 0x7f59a815f000 2024-09-17T05:25:01.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.575+0000 7f59ad800640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5988073200 0x7f59880756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.575+0000 7f59ad800640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5988073200 0x7f59880756c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f599c0af000 tx=0x7f599c01b070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.579+0000 7f59a5600640 1 -- 172.21.15.122:0/712721984 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f59a0016030 con 0x7f59a815f000 2024-09-17T05:25:01.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f59a80d02e0 con 0x7f59a815f000 2024-09-17T05:25:01.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59a5600640 1 -- 172.21.15.122:0/712721984 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f59a005f440 con 0x7f59a815f000 2024-09-17T05:25:01.748 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:01.748 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:25:01.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:01.748 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:25:01.748 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:25:01.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:01.748 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5988073200 msgr2=0x7f59880756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5988073200 0x7f59880756c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f599c0af000 tx=0x7f599c01b070 comp rx=0 tx=0).stop 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 msgr2=0x7f59a80ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 0x7f59a80ca7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f59a000e990 tx=0x7f59a000ee50 comp rx=0 tx=0).stop 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 shutdown_connections 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5988073200 0x7f59880756c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a815f000 0x7f59a80ca7c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a80cefb0 0x7f59a80c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 --2- 172.21.15.122:0/712721984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a80ccaf0 0x7f59a815b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 >> 172.21.15.122:0/712721984 conn(0x7f59a809aa10 msgr2=0x7f59a80b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:01.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 shutdown_connections 2024-09-17T05:25:01.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.747+0000 7f59aec00640 1 -- 172.21.15.122:0/712721984 wait complete. 2024-09-17T05:25:01.769 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:25:01.769 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:25:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.911+0000 7f2208000640 1 -- 172.21.15.122:0/3473498552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22000b3440 msgr2=0x7f22000cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.911+0000 7f2208000640 1 --2- 172.21.15.122:0/3473498552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22000b3440 0x7f22000cd300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22000b4240 tx=0x7f21f801caf0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.911+0000 7f2208000640 1 -- 172.21.15.122:0/3473498552 shutdown_connections 2024-09-17T05:25:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.911+0000 7f2208000640 1 --2- 172.21.15.122:0/3473498552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22000b3440 0x7f22000cd300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.911+0000 7f2208000640 1 --2- 172.21.15.122:0/3473498552 >> v2:172.21.15.173:3300/0 conn(0x7f2200068960 0x7f22000ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.911+0000 7f2208000640 1 -- 172.21.15.122:0/3473498552 >> 172.21.15.122:0/3473498552 conn(0x7f22000b2140 msgr2=0x7f22000b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.911+0000 7f2208000640 1 -- 172.21.15.122:0/3473498552 shutdown_connections 2024-09-17T05:25:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.911+0000 7f2208000640 1 -- 172.21.15.122:0/3473498552 wait complete. 2024-09-17T05:25:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 Processor -- start 2024-09-17T05:25:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 -- start start 2024-09-17T05:25:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2200068960 0x7f22001577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22000b3440 0x7f220014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220015be90 0x7f220014c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2200168cc0 con 0x7f2200068960 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22000b6150 con 0x7f220015be90 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2200168e40 con 0x7f22000b3440 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2200068960 0x7f22001577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2200068960 0x7f22001577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36568/0 (socket says 172.21.15.122:36568) 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 -- 172.21.15.122:0/2660096857 learned_addr learned my addr 172.21.15.122:0/2660096857 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2206000640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220015be90 0x7f220014c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2204c00640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22000b3440 0x7f220014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 -- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22000b3440 msgr2=0x7f220014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22000b3440 0x7f220014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 -- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220015be90 msgr2=0x7f220014c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220015be90 0x7f220014c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 -- 172.21.15.122:0/2660096857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22001f87e0 con 0x7f2200068960 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2206000640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220015be90 0x7f220014c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:01.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2204c00640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22000b3440 0x7f220014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2205600640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2200068960 0x7f22001577f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21f4001ec0 tx=0x7f21f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f21fd600640 1 -- 172.21.15.122:0/2660096857 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4017070 con 0x7f2200068960 2024-09-17T05:25:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f21fd600640 1 -- 172.21.15.122:0/2660096857 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21f4003600 con 0x7f2200068960 2024-09-17T05:25:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22001481f0 con 0x7f2200068960 2024-09-17T05:25:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f21fd600640 1 -- 172.21.15.122:0/2660096857 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4003a50 con 0x7f2200068960 2024-09-17T05:25:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001f7eb0 con 0x7f2200068960 2024-09-17T05:25:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.915+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220006f140 con 0x7f2200068960 2024-09-17T05:25:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.919+0000 7f21fd600640 1 -- 172.21.15.122:0/2660096857 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21f401a070 con 0x7f2200068960 2024-09-17T05:25:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.919+0000 7f21fd600640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21e0073200 0x7f21e00756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.919+0000 7f21fd600640 1 -- 172.21.15.122:0/2660096857 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f21f4093100 con 0x7f2200068960 2024-09-17T05:25:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.919+0000 7f2204c00640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21e0073200 0x7f21e00756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.919+0000 7f2204c00640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21e0073200 0x7f21e00756c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f220015f5f0 tx=0x7f21ec0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:01.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:01.923+0000 7f21fd600640 1 -- 172.21.15.122:0/2660096857 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f21f405e590 con 0x7f2200068960 2024-09-17T05:25:02.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.091+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2200152480 con 0x7f2200068960 2024-09-17T05:25:02.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.115+0000 7f21fd600640 1 -- 172.21.15.122:0/2660096857 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f21f405de90 con 0x7f2200068960 2024-09-17T05:25:02.118 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:25:02.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21e0073200 msgr2=0x7f21e00756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21e0073200 0x7f21e00756c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f220015f5f0 tx=0x7f21ec0091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2200068960 msgr2=0x7f22001577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2200068960 0x7f22001577f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21f4001ec0 tx=0x7f21f400c890 comp rx=0 tx=0).stop 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 shutdown_connections 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21e0073200 0x7f21e00756c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220015be90 0x7f220014c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22000b3440 0x7f220014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 --2- 172.21.15.122:0/2660096857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2200068960 0x7f22001577f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 >> 172.21.15.122:0/2660096857 conn(0x7f22000b2140 msgr2=0x7f220015f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 shutdown_connections 2024-09-17T05:25:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.119+0000 7f2208000640 1 -- 172.21.15.122:0/2660096857 wait complete. 2024-09-17T05:25:02.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:25:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.291+0000 7f58e9000640 1 -- 172.21.15.122:0/734598656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c8003c20 msgr2=0x7f58c8004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.291+0000 7f58e9000640 1 --2- 172.21.15.122:0/734598656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c8003c20 0x7f58c8004020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009b90 tx=0x7f58d401d270 comp rx=0 tx=0).stop 2024-09-17T05:25:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.291+0000 7f58e2800640 1 -- 172.21.15.122:0/734598656 reap_dead start 2024-09-17T05:25:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.291+0000 7f58e9000640 1 -- 172.21.15.122:0/734598656 shutdown_connections 2024-09-17T05:25:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.291+0000 7f58e9000640 1 -- 172.21.15.122:0/734598656 >> 172.21.15.122:0/734598656 conn(0x7f58e40d1dd0 msgr2=0x7f58e40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.291+0000 7f58e9000640 1 -- 172.21.15.122:0/734598656 shutdown_connections 2024-09-17T05:25:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.291+0000 7f58e9000640 1 -- 172.21.15.122:0/734598656 wait complete. 2024-09-17T05:25:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 Processor -- start 2024-09-17T05:25:02.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 -- start start 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58c8003c20 0x7f58e40cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e40b7710 0x7f58e4157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58e415ec20 0x7f58e415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40b1220 con 0x7f58e40b7710 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40b0ee0 con 0x7f58c8003c20 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40b13a0 con 0x7f58e415ec20 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e40b7710 0x7f58e4157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e40b7710 0x7f58e4157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36582/0 (socket says 172.21.15.122:36582) 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 -- 172.21.15.122:0/2996411974 learned_addr learned my addr 172.21.15.122:0/2996411974 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e2800640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58e415ec20 0x7f58e415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e1e00640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58c8003c20 0x7f58e40cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 -- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58e415ec20 msgr2=0x7f58e415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58e415ec20 0x7f58e415f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 -- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58c8003c20 msgr2=0x7f58e40cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58c8003c20 0x7f58e40cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 -- 172.21.15.122:0/2996411974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e40ce5e0 con 0x7f58e40b7710 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58dbe00640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e40b7710 0x7f58e4157220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004da0 tx=0x7f58cc001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58da000640 1 -- 172.21.15.122:0/2996411974 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc010890 con 0x7f58e40b7710 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58da000640 1 -- 172.21.15.122:0/2996411974 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58cc010b40 con 0x7f58e40b7710 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e41f7f30 con 0x7f58e40b7710 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58da000640 1 -- 172.21.15.122:0/2996411974 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc011460 con 0x7f58e40b7710 2024-09-17T05:25:02.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.295+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41f84f0 con 0x7f58e40b7710 2024-09-17T05:25:02.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.299+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e40affc0 con 0x7f58e40b7710 2024-09-17T05:25:02.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.299+0000 7f58da000640 1 -- 172.21.15.122:0/2996411974 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f58cc011600 con 0x7f58e40b7710 2024-09-17T05:25:02.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.299+0000 7f58da000640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58c0073230 0x7f58c00756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:02.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.299+0000 7f58da000640 1 -- 172.21.15.122:0/2996411974 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f58cc0954d0 con 0x7f58e40b7710 2024-09-17T05:25:02.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.299+0000 7f58e1e00640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58c0073230 0x7f58c00756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.303+0000 7f58e1e00640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58c0073230 0x7f58c00756f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f58d40ae000 tx=0x7f58d401a010 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:02.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.303+0000 7f58da000640 1 -- 172.21.15.122:0/2996411974 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f58cc060180 con 0x7f58e40b7710 2024-09-17T05:25:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:02.471+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f58e40d1080 con 0x7f58e40b7710 2024-09-17T05:25:03.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.123+0000 7f58da000640 1 -- 172.21.15.122:0/2996411974 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f58cc0670d0 con 0x7f58e40b7710 2024-09-17T05:25:03.126 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:25:03.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58c0073230 msgr2=0x7f58c00756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58c0073230 0x7f58c00756f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f58d40ae000 tx=0x7f58d401a010 comp rx=0 tx=0).stop 2024-09-17T05:25:03.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e40b7710 msgr2=0x7f58e4157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e40b7710 0x7f58e4157220 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004da0 tx=0x7f58cc001810 comp rx=0 tx=0).stop 2024-09-17T05:25:03.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 shutdown_connections 2024-09-17T05:25:03.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58c0073230 0x7f58c00756f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58e415ec20 0x7f58e415f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e40b7710 0x7f58e4157220 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004da0 tx=0x7f58cc001810 comp rx=0 tx=0).stop 2024-09-17T05:25:03.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 --2- 172.21.15.122:0/2996411974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58c8003c20 0x7f58e40cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 >> 172.21.15.122:0/2996411974 conn(0x7f58e40d1dd0 msgr2=0x7f58e415be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:03.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 shutdown_connections 2024-09-17T05:25:03.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.131+0000 7f58e9000640 1 -- 172.21.15.122:0/2996411974 wait complete. 2024-09-17T05:25:03.175 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 -- 172.21.15.122:0/1966933229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340b2a00 msgr2=0x7f85340b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 --2- 172.21.15.122:0/1966933229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340b2a00 0x7f85340b39e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85340738e0 tx=0x7f851c01c6b0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 -- 172.21.15.122:0/1966933229 shutdown_connections 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 --2- 172.21.15.122:0/1966933229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340b2a00 0x7f85340b39e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 --2- 172.21.15.122:0/1966933229 >> v2:172.21.15.173:3300/0 conn(0x7f853415f340 0x7f85340b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 -- 172.21.15.122:0/1966933229 >> 172.21.15.122:0/1966933229 conn(0x7f85340d1e00 msgr2=0x7f85340d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:03.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 -- 172.21.15.122:0/1966933229 shutdown_connections 2024-09-17T05:25:03.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 -- 172.21.15.122:0/1966933229 wait complete. 2024-09-17T05:25:03.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 Processor -- start 2024-09-17T05:25:03.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.319+0000 7f853ba00640 1 -- start start 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85340b2a00 0x7f853415b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340ccf10 0x7f853406f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f853415f340 0x7f85340b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85341690c0 con 0x7f85340ccf10 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8534168f40 con 0x7f85340b2a00 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8534169240 con 0x7f853415f340 2024-09-17T05:25:03.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340ccf10 0x7f853406f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340ccf10 0x7f853406f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36612/0 (socket says 172.21.15.122:36612) 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 -- 172.21.15.122:0/3652934420 learned_addr learned my addr 172.21.15.122:0/3652934420 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f8539000640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85340b2a00 0x7f853415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f8539a00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f853415f340 0x7f85340b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 -- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f853415f340 msgr2=0x7f85340b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f853415f340 0x7f85340b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 -- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85340b2a00 msgr2=0x7f853415b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85340b2a00 0x7f853415b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 -- 172.21.15.122:0/3652934420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85341fb9a0 con 0x7f85340ccf10 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f8539000640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85340b2a00 0x7f853415b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f8539a00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f853415f340 0x7f85340b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f852be00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340ccf10 0x7f853406f750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b0040 tx=0x7f851c0a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:03.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f8531600640 1 -- 172.21.15.122:0/3652934420 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0b2040 con 0x7f85340ccf10 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f8531600640 1 -- 172.21.15.122:0/3652934420 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f851c0044d0 con 0x7f85340ccf10 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f8531600640 1 -- 172.21.15.122:0/3652934420 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c005280 con 0x7f85340ccf10 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85341fbc80 con 0x7f85340ccf10 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853406fc90 con 0x7f85340ccf10 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85340cd340 con 0x7f85340ccf10 2024-09-17T05:25:03.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.323+0000 7f8531600640 1 -- 172.21.15.122:0/3652934420 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f851c005420 con 0x7f85340ccf10 2024-09-17T05:25:03.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.327+0000 7f8531600640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8510073200 0x7f85100756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:03.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.327+0000 7f8531600640 1 -- 172.21.15.122:0/3652934420 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f851c12d9e0 con 0x7f85340ccf10 2024-09-17T05:25:03.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.327+0000 7f8539000640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8510073200 0x7f85100756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:03.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.327+0000 7f8539000640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8510073200 0x7f85100756c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8524006fd0 tx=0x7f8524008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:03.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.331+0000 7f8531600640 1 -- 172.21.15.122:0/3652934420 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f85340d2c90 con 0x7f85340ccf10 2024-09-17T05:25:03.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:03.531+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f85341fc660 con 0x7f85340ccf10 2024-09-17T05:25:04.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.126+0000 7f8531600640 1 -- 172.21.15.122:0/3652934420 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f851c0f8ad0 con 0x7f85340ccf10 2024-09-17T05:25:04.128 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 2 and data pool 3 2024-09-17T05:25:04.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8510073200 msgr2=0x7f85100756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8510073200 0x7f85100756c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8524006fd0 tx=0x7f8524008040 comp rx=0 tx=0).stop 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340ccf10 msgr2=0x7f853406f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340ccf10 0x7f853406f750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b0040 tx=0x7f851c0a5690 comp rx=0 tx=0).stop 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 shutdown_connections 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8510073200 0x7f85100756c0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8524006fd0 tx=0x7f8524008040 comp rx=0 tx=0).stop 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f853415f340 0x7f85340b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85340ccf10 0x7f853406f750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 --2- 172.21.15.122:0/3652934420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85340b2a00 0x7f853415b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 >> 172.21.15.122:0/3652934420 conn(0x7f85340d1e00 msgr2=0x7f85340709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 shutdown_connections 2024-09-17T05:25:04.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.130+0000 7f853ba00640 1 -- 172.21.15.122:0/3652934420 wait complete. 2024-09-17T05:25:04.176 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:04.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.314+0000 7eff0b600640 1 -- 172.21.15.122:0/960697106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0415bea0 msgr2=0x7eff040ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.314+0000 7eff0b600640 1 --2- 172.21.15.122:0/960697106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0415bea0 0x7eff040ce670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7eff040cc3a0 tx=0x7efef401c830 comp rx=0 tx=0).stop 2024-09-17T05:25:04.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.314+0000 7eff0b600640 1 -- 172.21.15.122:0/960697106 shutdown_connections 2024-09-17T05:25:04.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.314+0000 7eff0b600640 1 --2- 172.21.15.122:0/960697106 >> v2:172.21.15.173:3300/0 conn(0x7eff040d28e0 0x7eff040cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.314+0000 7eff0b600640 1 --2- 172.21.15.122:0/960697106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0415bea0 0x7eff040ce670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.314+0000 7eff0b600640 1 -- 172.21.15.122:0/960697106 >> 172.21.15.122:0/960697106 conn(0x7eff040d1dc0 msgr2=0x7eff040d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.314+0000 7eff0b600640 1 -- 172.21.15.122:0/960697106 shutdown_connections 2024-09-17T05:25:04.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.314+0000 7eff0b600640 1 -- 172.21.15.122:0/960697106 wait complete. 2024-09-17T05:25:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 Processor -- start 2024-09-17T05:25:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 -- start start 2024-09-17T05:25:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040cdb30 0x7eff04159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff040d28e0 0x7eff04150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff0415bea0 0x7eff04150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff04168fc0 con 0x7eff040cdb30 2024-09-17T05:25:04.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff04168e40 con 0x7eff0415bea0 2024-09-17T05:25:04.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7eff04169140 con 0x7eff040d28e0 2024-09-17T05:25:04.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040cdb30 0x7eff04159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040cdb30 0x7eff04159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36630/0 (socket says 172.21.15.122:36630) 2024-09-17T05:25:04.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 -- 172.21.15.122:0/347843695 learned_addr learned my addr 172.21.15.122:0/347843695 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff09600640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff0415bea0 0x7eff04150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7efefbe00640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff040d28e0 0x7eff04150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 -- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff040d28e0 msgr2=0x7eff04150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff040d28e0 0x7eff04150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 -- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff0415bea0 msgr2=0x7eff04150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff0415bea0 0x7eff04150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 -- 172.21.15.122:0/347843695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff041f7e80 con 0x7eff040cdb30 2024-09-17T05:25:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff08c00640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040cdb30 0x7eff04159b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efef40b0040 tx=0x7efef40a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff01600640 1 -- 172.21.15.122:0/347843695 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef40b2040 con 0x7eff040cdb30 2024-09-17T05:25:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff01600640 1 -- 172.21.15.122:0/347843695 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efef4004430 con 0x7eff040cdb30 2024-09-17T05:25:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff041f8160 con 0x7eff040cdb30 2024-09-17T05:25:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff01600640 1 -- 172.21.15.122:0/347843695 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4003860 con 0x7eff040cdb30 2024-09-17T05:25:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff041f8720 con 0x7eff040cdb30 2024-09-17T05:25:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.318+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff040cc9d0 con 0x7eff040cdb30 2024-09-17T05:25:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.322+0000 7eff01600640 1 -- 172.21.15.122:0/347843695 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efef4003a00 con 0x7eff040cdb30 2024-09-17T05:25:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.322+0000 7eff01600640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efee0073230 0x7efee00756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.322+0000 7eff01600640 1 -- 172.21.15.122:0/347843695 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efef412df80 con 0x7eff040cdb30 2024-09-17T05:25:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.322+0000 7efefbe00640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efee0073230 0x7efee00756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.322+0000 7efefbe00640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efee0073230 0x7efee00756f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff040b4630 tx=0x7efeec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.326+0000 7eff01600640 1 -- 172.21.15.122:0/347843695 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efef40bd060 con 0x7eff040cdb30 2024-09-17T05:25:04.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff04071be0 con 0x7eff040cdb30 2024-09-17T05:25:04.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff01600640 1 -- 172.21.15.122:0/347843695 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16742 (secure 0 0 0) 0x7efef40164b0 con 0x7eff040cdb30 2024-09-17T05:25:04.497 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:04.497 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:03.536455+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efee0073230 msgr2=0x7efee00756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efee0073230 0x7efee00756f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7eff040b4630 tx=0x7efeec009040 comp rx=0 tx=0).stop 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040cdb30 msgr2=0x7eff04159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040cdb30 0x7eff04159b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efef40b0040 tx=0x7efef40a56d0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 shutdown_connections 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efee0073230 0x7efee00756f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff0415bea0 0x7eff04150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff040d28e0 0x7eff04150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 --2- 172.21.15.122:0/347843695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040cdb30 0x7eff04159b30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 >> 172.21.15.122:0/347843695 conn(0x7eff040d1dc0 msgr2=0x7eff04072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 shutdown_connections 2024-09-17T05:25:04.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.494+0000 7eff0b600640 1 -- 172.21.15.122:0/347843695 wait complete. 2024-09-17T05:25:04.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- 172.21.15.122:0/4167765839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bf70 msgr2=0x7f4dc80b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/4167765839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bf70 0x7f4dc80b53e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009c80 tx=0x7f4dc001cda0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- 172.21.15.122:0/4167765839 shutdown_connections 2024-09-17T05:25:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/4167765839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bf70 0x7f4dc80b53e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/4167765839 >> v2:172.21.15.173:3300/0 conn(0x7f4dc80d2940 0x7f4dc8160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- 172.21.15.122:0/4167765839 >> 172.21.15.122:0/4167765839 conn(0x7f4dc80d1d90 msgr2=0x7f4dc80d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- 172.21.15.122:0/4167765839 shutdown_connections 2024-09-17T05:25:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- 172.21.15.122:0/4167765839 wait complete. 2024-09-17T05:25:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 Processor -- start 2024-09-17T05:25:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- start start 2024-09-17T05:25:04.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc80d2940 0x7f4dc8152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bb90 0x7f4dc81fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dc815bf70 0x7f4dc814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8168ca0 con 0x7f4dc80d2940 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc80b65e0 con 0x7f4dc815bf70 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dcdc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc8168e20 con 0x7f4dc815bb90 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dc6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc80d2940 0x7f4dc8152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dc6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc80d2940 0x7f4dc8152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36642/0 (socket says 172.21.15.122:36642) 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.658+0000 7f4dc6a00640 1 -- 172.21.15.122:0/2838566922 learned_addr learned my addr 172.21.15.122:0/2838566922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dbe000640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bb90 0x7f4dc81fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dc7400640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dc815bf70 0x7f4dc814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dc6a00640 1 -- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bb90 msgr2=0x7f4dc81fbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dc6a00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bb90 0x7f4dc81fbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dc6a00640 1 -- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dc815bf70 msgr2=0x7f4dc814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dc6a00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dc815bf70 0x7f4dc814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dc6a00640 1 -- 172.21.15.122:0/2838566922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc81fc020 con 0x7f4dc80d2940 2024-09-17T05:25:04.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dbe000640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bb90 0x7f4dc81fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:04.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dc7400640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dc815bf70 0x7f4dc814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:04.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dc6a00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc80d2940 0x7f4dc8152460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4db800c430 tx=0x7f4db800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dbf400640 1 -- 172.21.15.122:0/2838566922 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8017070 con 0x7f4dc80d2940 2024-09-17T05:25:04.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dbf400640 1 -- 172.21.15.122:0/2838566922 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4db80053c0 con 0x7f4dc80d2940 2024-09-17T05:25:04.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc8152a90 con 0x7f4dc80d2940 2024-09-17T05:25:04.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dbf400640 1 -- 172.21.15.122:0/2838566922 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8003880 con 0x7f4dc80d2940 2024-09-17T05:25:04.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc8153050 con 0x7f4dc80d2940 2024-09-17T05:25:04.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc80b53e0 con 0x7f4dc80d2940 2024-09-17T05:25:04.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dbf400640 1 -- 172.21.15.122:0/2838566922 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4db801a070 con 0x7f4dc80d2940 2024-09-17T05:25:04.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dbf400640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4da4073200 0x7f4da40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:04.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.662+0000 7f4dbf400640 1 -- 172.21.15.122:0/2838566922 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4db8093650 con 0x7f4dc80d2940 2024-09-17T05:25:04.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.666+0000 7f4dbe000640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4da4073200 0x7f4da40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:04.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.666+0000 7f4dbe000640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4da4073200 0x7f4da40756c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8151aa0 tx=0x7f4db00091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.666+0000 7f4dbf400640 1 -- 172.21.15.122:0/2838566922 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4db8001130 con 0x7f4dc80d2940 2024-09-17T05:25:04.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.834+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4dc8070270 con 0x7f4dc80d2940 2024-09-17T05:25:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dbf400640 1 -- 172.21.15.122:0/2838566922 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16742 (secure 0 0 0) 0x7f4db805e670 con 0x7f4dc80d2940 2024-09-17T05:25:04.840 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:04.840 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:03.536455+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:04.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4da4073200 msgr2=0x7f4da40756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4da4073200 0x7f4da40756c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8151aa0 tx=0x7f4db00091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc80d2940 msgr2=0x7f4dc8152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc80d2940 0x7f4dc8152460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4db800c430 tx=0x7f4db800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 shutdown_connections 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4da4073200 0x7f4da40756c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dc815bf70 0x7f4dc814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dc815bb90 0x7f4dc81fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 --2- 172.21.15.122:0/2838566922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dc80d2940 0x7f4dc8152460 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 >> 172.21.15.122:0/2838566922 conn(0x7f4dc80d1d90 msgr2=0x7f4dc80cd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 shutdown_connections 2024-09-17T05:25:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:04.838+0000 7f4dcdc00640 1 -- 172.21.15.122:0/2838566922 wait complete. 2024-09-17T05:25:04.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:25:05.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.002+0000 7fa23be00640 1 -- 172.21.15.122:0/2014296220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 msgr2=0x7fa2340b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:05.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.002+0000 7fa23be00640 1 --2- 172.21.15.122:0/2014296220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 0x7fa2340b2740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa220009840 tx=0x7fa22001c820 comp rx=0 tx=0).stop 2024-09-17T05:25:05.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.002+0000 7fa23be00640 1 -- 172.21.15.122:0/2014296220 shutdown_connections 2024-09-17T05:25:05.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.002+0000 7fa23be00640 1 --2- 172.21.15.122:0/2014296220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 0x7fa2340b2740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.002+0000 7fa23be00640 1 -- 172.21.15.122:0/2014296220 >> 172.21.15.122:0/2014296220 conn(0x7fa2340cd740 msgr2=0x7fa2340cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:05.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.002+0000 7fa23be00640 1 -- 172.21.15.122:0/2014296220 shutdown_connections 2024-09-17T05:25:05.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.002+0000 7fa23be00640 1 -- 172.21.15.122:0/2014296220 wait complete. 2024-09-17T05:25:05.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 Processor -- start 2024-09-17T05:25:05.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 -- start start 2024-09-17T05:25:05.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2340b2c80 0x7fa2340cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:05.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 0x7fa2340a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:05.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa23415ac70 0x7fa23415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:05.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa234169080 con 0x7fa2340cf2a0 2024-09-17T05:25:05.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa234168f00 con 0x7fa23415ac70 2024-09-17T05:25:05.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa234169200 con 0x7fa2340b2c80 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 0x7fa2340a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 0x7fa2340a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36648/0 (socket says 172.21.15.122:36648) 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 -- 172.21.15.122:0/3717000046 learned_addr learned my addr 172.21.15.122:0/3717000046 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa239400640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2340b2c80 0x7fa2340cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa239e00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa23415ac70 0x7fa23415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 -- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2340b2c80 msgr2=0x7fa2340cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2340b2c80 0x7fa2340cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 -- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa23415ac70 msgr2=0x7fa23415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa23415ac70 0x7fa23415b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 -- 172.21.15.122:0/3717000046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa234156390 con 0x7fa2340cf2a0 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa239e00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa23415ac70 0x7fa23415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa238a00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 0x7fa2340a09b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa220009790 tx=0x7fa2200040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:05.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa231600640 1 -- 172.21.15.122:0/3717000046 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2200bb070 con 0x7fa2340cf2a0 2024-09-17T05:25:05.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa231600640 1 -- 172.21.15.122:0/3717000046 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa220005030 con 0x7fa2340cf2a0 2024-09-17T05:25:05.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa231600640 1 -- 172.21.15.122:0/3717000046 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2200b2040 con 0x7fa2340cf2a0 2024-09-17T05:25:05.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa234156670 con 0x7fa2340cf2a0 2024-09-17T05:25:05.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.006+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23415d0f0 con 0x7fa2340cf2a0 2024-09-17T05:25:05.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.010+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2340b2740 con 0x7fa2340cf2a0 2024-09-17T05:25:05.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.010+0000 7fa231600640 1 -- 172.21.15.122:0/3717000046 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa2200be070 con 0x7fa2340cf2a0 2024-09-17T05:25:05.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.010+0000 7fa231600640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa214073200 0x7fa2140756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:05.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.010+0000 7fa231600640 1 -- 172.21.15.122:0/3717000046 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa22012d9c0 con 0x7fa2340cf2a0 2024-09-17T05:25:05.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.010+0000 7fa239400640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa214073200 0x7fa2140756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:05.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.010+0000 7fa239400640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa214073200 0x7fa2140756c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa228007fd0 tx=0x7fa228009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:05.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.014+0000 7fa231600640 1 -- 172.21.15.122:0/3717000046 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa22012fe20 con 0x7fa2340cf2a0 2024-09-17T05:25:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:05.222+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa234156840 con 0x7fa2340cf2a0 2024-09-17T05:25:06.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.154+0000 7fa231600640 1 -- 172.21.15.122:0/3717000046 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fa220016410 con 0x7fa2340cf2a0 2024-09-17T05:25:06.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa214073200 msgr2=0x7fa2140756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:06.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa214073200 0x7fa2140756c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa228007fd0 tx=0x7fa228009040 comp rx=0 tx=0).stop 2024-09-17T05:25:06.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 msgr2=0x7fa2340a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:06.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 0x7fa2340a09b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa220009790 tx=0x7fa2200040c0 comp rx=0 tx=0).stop 2024-09-17T05:25:06.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 shutdown_connections 2024-09-17T05:25:06.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa214073200 0x7fa2140756c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:06.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa23415ac70 0x7fa23415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:06.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2340cf2a0 0x7fa2340a09b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:06.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 --2- 172.21.15.122:0/3717000046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2340b2c80 0x7fa2340cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:06.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 >> 172.21.15.122:0/3717000046 conn(0x7fa2340cd740 msgr2=0x7fa23406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:06.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 shutdown_connections 2024-09-17T05:25:06.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.158+0000 7fa23be00640 1 -- 172.21.15.122:0/3717000046 wait complete. 2024-09-17T05:25:06.184 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:25:06.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:25:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.334+0000 7fb2f1e00640 1 -- 172.21.15.122:0/854415105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 msgr2=0x7fb2d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.334+0000 7fb2f1e00640 1 --2- 172.21.15.122:0/854415105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 0x7fb2d0004c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c2e0 tx=0x7fb2d401f730 comp rx=0 tx=0).stop 2024-09-17T05:25:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.334+0000 7fb2eb400640 1 -- 172.21.15.122:0/854415105 reap_dead start 2024-09-17T05:25:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.334+0000 7fb2f1e00640 1 -- 172.21.15.122:0/854415105 shutdown_connections 2024-09-17T05:25:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.334+0000 7fb2f1e00640 1 -- 172.21.15.122:0/854415105 >> 172.21.15.122:0/854415105 conn(0x7fb2ec0d1dd0 msgr2=0x7fb2ec0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.334+0000 7fb2f1e00640 1 -- 172.21.15.122:0/854415105 shutdown_connections 2024-09-17T05:25:06.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.334+0000 7fb2f1e00640 1 -- 172.21.15.122:0/854415105 wait complete. 2024-09-17T05:25:06.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 Processor -- start 2024-09-17T05:25:06.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 -- start start 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 0x7fb2ec0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2ec0cb2c0 0x7fb2ec1f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2ec1f4f10 0x7fb2ec0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec0b0e50 con 0x7fb2d0004810 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec0b0cd0 con 0x7fb2ec0cb2c0 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ec0b1220 con 0x7fb2ec1f4f10 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 0x7fb2ec0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 0x7fb2ec0cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36664/0 (socket says 172.21.15.122:36664) 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 -- 172.21.15.122:0/2460532915 learned_addr learned my addr 172.21.15.122:0/2460532915 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:06.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eb400640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2ec1f4f10 0x7fb2ec0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:06.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2e2000640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2ec0cb2c0 0x7fb2ec1f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:06.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 -- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2ec1f4f10 msgr2=0x7fb2ec0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:06.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2ec1f4f10 0x7fb2ec0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:06.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 -- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2ec0cb2c0 msgr2=0x7fb2ec1f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:06.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2ec0cb2c0 0x7fb2ec1f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:06.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 -- 172.21.15.122:0/2460532915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec1f5390 con 0x7fb2d0004810 2024-09-17T05:25:06.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2eaa00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 0x7fb2ec0cb8e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc011d10 tx=0x7fb2dc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:06.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2e3e00640 1 -- 172.21.15.122:0/2460532915 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc003b90 con 0x7fb2d0004810 2024-09-17T05:25:06.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2e3e00640 1 -- 172.21.15.122:0/2460532915 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2dc003d30 con 0x7fb2d0004810 2024-09-17T05:25:06.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ec1f5670 con 0x7fb2d0004810 2024-09-17T05:25:06.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2e3e00640 1 -- 172.21.15.122:0/2460532915 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0042b0 con 0x7fb2d0004810 2024-09-17T05:25:06.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ec0c6d30 con 0x7fb2d0004810 2024-09-17T05:25:06.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.338+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2ec0d0690 con 0x7fb2d0004810 2024-09-17T05:25:06.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.342+0000 7fb2e3e00640 1 -- 172.21.15.122:0/2460532915 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb2dc004450 con 0x7fb2d0004810 2024-09-17T05:25:06.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.342+0000 7fb2e3e00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2c8073200 0x7fb2c80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:06.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.342+0000 7fb2e3e00640 1 -- 172.21.15.122:0/2460532915 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb2dc095610 con 0x7fb2d0004810 2024-09-17T05:25:06.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.342+0000 7fb2e2000640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2c8073200 0x7fb2c80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:06.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.342+0000 7fb2e2000640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2c8073200 0x7fb2c80756c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40b0000 tx=0x7fb2d401c580 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:06.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.346+0000 7fb2e3e00640 1 -- 172.21.15.122:0/2460532915 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb2dc0606b0 con 0x7fb2d0004810 2024-09-17T05:25:06.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:06.550+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb2ec0d1080 con 0x7fb2d0004810 2024-09-17T05:25:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.158+0000 7fb2e3e00640 1 -- 172.21.15.122:0/2460532915 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fb2dc05ffb0 con 0x7fb2d0004810 2024-09-17T05:25:07.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2c8073200 msgr2=0x7fb2c80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2c8073200 0x7fb2c80756c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40b0000 tx=0x7fb2d401c580 comp rx=0 tx=0).stop 2024-09-17T05:25:07.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 msgr2=0x7fb2ec0cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 0x7fb2ec0cb8e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc011d10 tx=0x7fb2dc00f790 comp rx=0 tx=0).stop 2024-09-17T05:25:07.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 shutdown_connections 2024-09-17T05:25:07.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2c8073200 0x7fb2c80756c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2ec1f4f10 0x7fb2ec0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2ec0cb2c0 0x7fb2ec1f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 --2- 172.21.15.122:0/2460532915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2d0004810 0x7fb2ec0cb8e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 >> 172.21.15.122:0/2460532915 conn(0x7fb2ec0d1dd0 msgr2=0x7fb2ec0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:07.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 shutdown_connections 2024-09-17T05:25:07.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.162+0000 7fb2f1e00640 1 -- 172.21.15.122:0/2460532915 wait complete. 2024-09-17T05:25:07.186 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:25:07.187 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:07.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.326+0000 7f533b600640 1 -- 172.21.15.122:0/928445307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f533415bea0 msgr2=0x7f53340ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.326+0000 7f533b600640 1 --2- 172.21.15.122:0/928445307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f533415bea0 0x7f53340ce670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5334073d10 tx=0x7f532401c830 comp rx=0 tx=0).stop 2024-09-17T05:25:07.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.326+0000 7f533b600640 1 -- 172.21.15.122:0/928445307 shutdown_connections 2024-09-17T05:25:07.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.326+0000 7f533b600640 1 --2- 172.21.15.122:0/928445307 >> v2:172.21.15.173:3300/0 conn(0x7f53340cdb30 0x7f53340cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.326+0000 7f533b600640 1 --2- 172.21.15.122:0/928445307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f533415bea0 0x7f53340ce670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.326+0000 7f533b600640 1 -- 172.21.15.122:0/928445307 >> 172.21.15.122:0/928445307 conn(0x7f53340d1dc0 msgr2=0x7f53340d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:07.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.326+0000 7f533b600640 1 -- 172.21.15.122:0/928445307 shutdown_connections 2024-09-17T05:25:07.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.326+0000 7f533b600640 1 -- 172.21.15.122:0/928445307 wait complete. 2024-09-17T05:25:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 Processor -- start 2024-09-17T05:25:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 -- start start 2024-09-17T05:25:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53340cdb30 0x7f53341661f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53340d28e0 0x7f5334155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f533415bea0 0x7f5334156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5334168fc0 con 0x7f53340cdb30 2024-09-17T05:25:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5334168e40 con 0x7f53340d28e0 2024-09-17T05:25:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5334169140 con 0x7f533415bea0 2024-09-17T05:25:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53340cdb30 0x7f53341661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53340cdb30 0x7f53341661f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36690/0 (socket says 172.21.15.122:36690) 2024-09-17T05:25:07.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 -- 172.21.15.122:0/3822839401 learned_addr learned my addr 172.21.15.122:0/3822839401 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:07.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 -- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f533415bea0 msgr2=0x7f5334156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f533415bea0 0x7f5334156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 -- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53340d28e0 msgr2=0x7f5334155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53340d28e0 0x7f5334155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 -- 172.21.15.122:0/3822839401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533406dd30 con 0x7f53340cdb30 2024-09-17T05:25:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5338c00640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53340cdb30 0x7f53341661f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f53240b0040 tx=0x7f53240a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5331600640 1 -- 172.21.15.122:0/3822839401 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53240b2040 con 0x7f53340cdb30 2024-09-17T05:25:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5331600640 1 -- 172.21.15.122:0/3822839401 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53240043b0 con 0x7f53340cdb30 2024-09-17T05:25:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f5331600640 1 -- 172.21.15.122:0/3822839401 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324005280 con 0x7f53340cdb30 2024-09-17T05:25:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533406e010 con 0x7f53340cdb30 2024-09-17T05:25:07.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.330+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533406e5d0 con 0x7f53340cdb30 2024-09-17T05:25:07.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.334+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53340cc9d0 con 0x7f53340cdb30 2024-09-17T05:25:07.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.334+0000 7f5331600640 1 -- 172.21.15.122:0/3822839401 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5324005420 con 0x7f53340cdb30 2024-09-17T05:25:07.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.334+0000 7f5331600640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5310073200 0x7f53100756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.334+0000 7f5331600640 1 -- 172.21.15.122:0/3822839401 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f532412ddc0 con 0x7f53340cdb30 2024-09-17T05:25:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.334+0000 7f532be00640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5310073200 0x7f53100756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.334+0000 7f532be00640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5310073200 0x7f53100756c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f531c0058d0 tx=0x7f531c005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.338+0000 7f5331600640 1 -- 172.21.15.122:0/3822839401 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f53240164b0 con 0x7f53340cdb30 2024-09-17T05:25:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.538+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53341557c0 con 0x7f53340cdb30 2024-09-17T05:25:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.538+0000 7f5331600640 1 -- 172.21.15.122:0/3822839401 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f53240f8de0 con 0x7f53340cdb30 2024-09-17T05:25:07.541 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:07.542 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:03.536393+0000","modified":"2024-09-17T05:25:06.553594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4332":{"gid":4332,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4332,"qdb_cluster":[4332]},"id":1}]} 2024-09-17T05:25:07.542 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5310073200 msgr2=0x7f53100756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5310073200 0x7f53100756c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f531c0058d0 tx=0x7f531c005860 comp rx=0 tx=0).stop 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53340cdb30 msgr2=0x7f53341661f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53340cdb30 0x7f53341661f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f53240b0040 tx=0x7f53240a5680 comp rx=0 tx=0).stop 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 shutdown_connections 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5310073200 0x7f53100756c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f533415bea0 0x7f5334156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53340d28e0 0x7f5334155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 --2- 172.21.15.122:0/3822839401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53340cdb30 0x7f53341661f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 >> 172.21.15.122:0/3822839401 conn(0x7f53340d1dc0 msgr2=0x7f53340b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:07.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 shutdown_connections 2024-09-17T05:25:07.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.542+0000 7f533b600640 1 -- 172.21.15.122:0/3822839401 wait complete. 2024-09-17T05:25:07.562 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.698+0000 7fd615c00640 1 -- 172.21.15.122:0/898935388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100ced80 msgr2=0x7fd6100ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.698+0000 7fd615c00640 1 --2- 172.21.15.122:0/898935388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100ced80 0x7fd6100ae9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009900 tx=0x7fd5fc01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.698+0000 7fd615c00640 1 -- 172.21.15.122:0/898935388 shutdown_connections 2024-09-17T05:25:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.698+0000 7fd615c00640 1 --2- 172.21.15.122:0/898935388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6100cdb30 0x7fd6100aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.698+0000 7fd615c00640 1 --2- 172.21.15.122:0/898935388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100ced80 0x7fd6100ae9b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.698+0000 7fd615c00640 1 -- 172.21.15.122:0/898935388 >> 172.21.15.122:0/898935388 conn(0x7fd6100d1dc0 msgr2=0x7fd6100d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.698+0000 7fd615c00640 1 -- 172.21.15.122:0/898935388 shutdown_connections 2024-09-17T05:25:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.698+0000 7fd615c00640 1 -- 172.21.15.122:0/898935388 wait complete. 2024-09-17T05:25:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 Processor -- start 2024-09-17T05:25:07.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 -- start start 2024-09-17T05:25:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100cdb30 0x7fd61006fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6100ced80 0x7fd6101fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd61015bea0 0x7fd6100c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd610169340 con 0x7fd6100cdb30 2024-09-17T05:25:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6101691c0 con 0x7fd61015bea0 2024-09-17T05:25:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd6101694c0 con 0x7fd6100ced80 2024-09-17T05:25:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100cdb30 0x7fd61006fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100cdb30 0x7fd61006fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36704/0 (socket says 172.21.15.122:36704) 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 -- 172.21.15.122:0/1512323700 learned_addr learned my addr 172.21.15.122:0/1512323700 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60e000640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6100ced80 0x7fd6101fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60f400640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd61015bea0 0x7fd6100c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 -- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6100ced80 msgr2=0x7fd6101fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6100ced80 0x7fd6101fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 -- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd61015bea0 msgr2=0x7fd6100c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd61015bea0 0x7fd6100c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 -- 172.21.15.122:0/1512323700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6101fbe80 con 0x7fd6100cdb30 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60f400640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd61015bea0 0x7fd6100c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd60ea00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100cdb30 0x7fd61006fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd60000c3c0 tx=0x7fd60000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd606a00640 1 -- 172.21.15.122:0/1512323700 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600017070 con 0x7fd6100cdb30 2024-09-17T05:25:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd606a00640 1 -- 172.21.15.122:0/1512323700 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6000053c0 con 0x7fd6100cdb30 2024-09-17T05:25:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6101560d0 con 0x7fd6100cdb30 2024-09-17T05:25:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd606a00640 1 -- 172.21.15.122:0/1512323700 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600003860 con 0x7fd6100cdb30 2024-09-17T05:25:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.702+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6100b1610 con 0x7fd6100cdb30 2024-09-17T05:25:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.706+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6100ce670 con 0x7fd6100cdb30 2024-09-17T05:25:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.706+0000 7fd606a00640 1 -- 172.21.15.122:0/1512323700 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd60001a070 con 0x7fd6100cdb30 2024-09-17T05:25:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.706+0000 7fd606a00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5f0073200 0x7fd5f00756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.706+0000 7fd606a00640 1 -- 172.21.15.122:0/1512323700 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd6000934d0 con 0x7fd6100cdb30 2024-09-17T05:25:07.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.706+0000 7fd60e000640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5f0073200 0x7fd5f00756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:07.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.706+0000 7fd60e000640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5f0073200 0x7fd5f00756c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0af000 tx=0x7fd5fc01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.710+0000 7fd606a00640 1 -- 172.21.15.122:0/1512323700 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd6101fbe80 con 0x7fd6100cdb30 2024-09-17T05:25:07.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.878+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6100b0e90 con 0x7fd6100cdb30 2024-09-17T05:25:07.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.878+0000 7fd606a00640 1 -- 172.21.15.122:0/1512323700 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7fd60005df20 con 0x7fd6100cdb30 2024-09-17T05:25:07.882 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:07.882 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":22,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:05.140074+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:07.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5f0073200 msgr2=0x7fd5f00756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5f0073200 0x7fd5f00756c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0af000 tx=0x7fd5fc01b120 comp rx=0 tx=0).stop 2024-09-17T05:25:07.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100cdb30 msgr2=0x7fd61006fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:07.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100cdb30 0x7fd61006fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd60000c3c0 tx=0x7fd60000c880 comp rx=0 tx=0).stop 2024-09-17T05:25:07.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 shutdown_connections 2024-09-17T05:25:07.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5f0073200 0x7fd5f00756c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd61015bea0 0x7fd6100c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6100ced80 0x7fd6101fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 --2- 172.21.15.122:0/1512323700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6100cdb30 0x7fd61006fa00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:07.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 >> 172.21.15.122:0/1512323700 conn(0x7fd6100d1dc0 msgr2=0x7fd6100b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:07.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 shutdown_connections 2024-09-17T05:25:07.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:07.882+0000 7fd615c00640 1 -- 172.21.15.122:0/1512323700 wait complete. 2024-09-17T05:25:07.903 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:25:07.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- 172.21.15.122:0/2416574548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 msgr2=0x7f8a0c0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 --2- 172.21.15.122:0/2416574548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 0x7f8a0c0ae9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0cf340 tx=0x7f89f401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- 172.21.15.122:0/2416574548 shutdown_connections 2024-09-17T05:25:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 --2- 172.21.15.122:0/2416574548 >> v2:172.21.15.173:3300/0 conn(0x7f8a0c0cdb30 0x7f8a0c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 --2- 172.21.15.122:0/2416574548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 0x7f8a0c0ae9b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- 172.21.15.122:0/2416574548 >> 172.21.15.122:0/2416574548 conn(0x7f8a0c0d1dc0 msgr2=0x7f8a0c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- 172.21.15.122:0/2416574548 shutdown_connections 2024-09-17T05:25:08.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- 172.21.15.122:0/2416574548 wait complete. 2024-09-17T05:25:08.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 Processor -- start 2024-09-17T05:25:08.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- start start 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c0cdb30 0x7f8a0c15c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 0x7f8a0c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c15bea0 0x7f8a0c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1692f0 con 0x7f8a0c0ced80 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c169170 con 0x7f8a0c0cdb30 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a11400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c169470 con 0x7f8a0c15bea0 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a01e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 0x7f8a0c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a01e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 0x7f8a0c0c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36730/0 (socket says 172.21.15.122:36730) 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a01e00640 1 -- 172.21.15.122:0/2945021023 learned_addr learned my addr 172.21.15.122:0/2945021023 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a0a800640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c0cdb30 0x7f8a0c15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a0b200640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c15bea0 0x7f8a0c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a01e00640 1 -- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c15bea0 msgr2=0x7f8a0c06f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a01e00640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c15bea0 0x7f8a0c06f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a01e00640 1 -- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c0cdb30 msgr2=0x7f8a0c15c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a01e00640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c0cdb30 0x7f8a0c15c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a01e00640 1 -- 172.21.15.122:0/2945021023 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c06fbc0 con 0x7f8a0c0ced80 2024-09-17T05:25:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.042+0000 7f8a0a800640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c0cdb30 0x7f8a0c15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a01e00640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 0x7f8a0c0c7d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89f40b0040 tx=0x7f89f40a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a03400640 1 -- 172.21.15.122:0/2945021023 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f40b2040 con 0x7f8a0c0ced80 2024-09-17T05:25:08.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a03400640 1 -- 172.21.15.122:0/2945021023 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89f4005030 con 0x7f8a0c0ced80 2024-09-17T05:25:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a03400640 1 -- 172.21.15.122:0/2945021023 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f40044f0 con 0x7f8a0c0ced80 2024-09-17T05:25:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c0c90e0 con 0x7f8a0c0ced80 2024-09-17T05:25:08.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c0c96a0 con 0x7f8a0c0ced80 2024-09-17T05:25:08.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c0ce670 con 0x7f8a0c0ced80 2024-09-17T05:25:08.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a03400640 1 -- 172.21.15.122:0/2945021023 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89f40bd060 con 0x7f8a0c0ced80 2024-09-17T05:25:08.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a03400640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89e8073200 0x7f89e80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.046+0000 7f8a03400640 1 -- 172.21.15.122:0/2945021023 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f89f412d710 con 0x7f8a0c0ced80 2024-09-17T05:25:08.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.050+0000 7f8a0a800640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89e8073200 0x7f89e80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.050+0000 7f8a0a800640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89e8073200 0x7f89e80756c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0016b0 tx=0x7f89fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.050+0000 7f8a03400640 1 -- 172.21.15.122:0/2945021023 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f89f40164b0 con 0x7f8a0c0ced80 2024-09-17T05:25:08.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.214+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a0c0c8430 con 0x7f89e8073200 2024-09-17T05:25:08.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.222+0000 7f8a03400640 1 -- 172.21.15.122:0/2945021023 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245685 (secure 0 0 0) 0x7f8a0c0c8430 con 0x7f89e8073200 2024-09-17T05:25:08.227 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:08.229 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89e8073200 msgr2=0x7f89e80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89e8073200 0x7f89e80756c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0016b0 tx=0x7f89fc009040 comp rx=0 tx=0).stop 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 msgr2=0x7f8a0c0c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 0x7f8a0c0c7d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89f40b0040 tx=0x7f89f40a56c0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 shutdown_connections 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89e8073200 0x7f89e80756c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c15bea0 0x7f8a0c06f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a0c0ced80 0x7f8a0c0c7d60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 --2- 172.21.15.122:0/2945021023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c0cdb30 0x7f8a0c15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 >> 172.21.15.122:0/2945021023 conn(0x7f8a0c0d1dc0 msgr2=0x7f8a0c15ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 shutdown_connections 2024-09-17T05:25:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.226+0000 7f8a11400640 1 -- 172.21.15.122:0/2945021023 wait complete. 2024-09-17T05:25:08.248 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T05:25:06.441960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219104,"kb_used_data":2624,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755616,"statfs":{"total":773094113280,"available":772869750784,"internally_reserved":0,"allocated":2686976,"data_stored":1649216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":221236790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.319572"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139696+0000","last_change":"2024-09-17T05:25:03.139696+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139696+0000","last_undegraded":"2024-09-17T05:25:03.139696+0000","last_fullsized":"2024-09-17T05:25:03.139696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141593+0000","last_change":"2024-09-17T05:25:03.141593+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141593+0000","last_undegraded":"2024-09-17T05:25:03.141593+0000","last_fullsized":"2024-09-17T05:25:03.141593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139805+0000","last_change":"2024-09-17T05:25:03.139805+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139805+0000","last_undegraded":"2024-09-17T05:25:03.139805+0000","last_fullsized":"2024-09-17T05:25:03.139805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139985+0000","last_change":"2024-09-17T05:25:03.139985+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139985+0000","last_undegraded":"2024-09-17T05:25:03.139985+0000","last_fullsized":"2024-09-17T05:25:03.139985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.137568+0000","last_change":"2024-09-17T05:25:03.137568+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.137568+0000","last_undegraded":"2024-09-17T05:25:03.137568+0000","last_fullsized":"2024-09-17T05:25:03.137568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141669+0000","last_change":"2024-09-17T05:25:03.141669+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141669+0000","last_undegraded":"2024-09-17T05:25:03.141669+0000","last_fullsized":"2024-09-17T05:25:03.141669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136500+0000","last_change":"2024-09-17T05:25:03.136500+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136500+0000","last_undegraded":"2024-09-17T05:25:03.136500+0000","last_fullsized":"2024-09-17T05:25:03.136500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139381+0000","last_change":"2024-09-17T05:25:03.139381+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139381+0000","last_undegraded":"2024-09-17T05:25:03.139381+0000","last_fullsized":"2024-09-17T05:25:03.139381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136937+0000","last_change":"2024-09-17T05:25:03.136937+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136937+0000","last_undegraded":"2024-09-17T05:25:03.136937+0000","last_fullsized":"2024-09-17T05:25:03.136937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138718+0000","last_change":"2024-09-17T05:25:03.138718+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138718+0000","last_undegraded":"2024-09-17T05:25:03.138718+0000","last_fullsized":"2024-09-17T05:25:03.138718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141990+0000","last_change":"2024-09-17T05:25:03.141990+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141990+0000","last_undegraded":"2024-09-17T05:25:03.141990+0000","last_fullsized":"2024-09-17T05:25:03.141990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139885+0000","last_change":"2024-09-17T05:25:03.139885+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139885+0000","last_undegraded":"2024-09-17T05:25:03.139885+0000","last_fullsized":"2024-09-17T05:25:03.139885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139387+0000","last_change":"2024-09-17T05:25:03.139387+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139387+0000","last_undegraded":"2024-09-17T05:25:03.139387+0000","last_fullsized":"2024-09-17T05:25:03.139387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133893+0000","last_change":"2024-09-17T05:25:03.133893+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133893+0000","last_undegraded":"2024-09-17T05:25:03.133893+0000","last_fullsized":"2024-09-17T05:25:03.133893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138629+0000","last_change":"2024-09-17T05:25:03.138629+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138629+0000","last_undegraded":"2024-09-17T05:25:03.138629+0000","last_fullsized":"2024-09-17T05:25:03.138629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136431+0000","last_change":"2024-09-17T05:25:03.136431+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136431+0000","last_undegraded":"2024-09-17T05:25:03.136431+0000","last_fullsized":"2024-09-17T05:25:03.136431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143548+0000","last_change":"2024-09-17T05:25:03.143548+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143548+0000","last_undegraded":"2024-09-17T05:25:03.143548+0000","last_fullsized":"2024-09-17T05:25:03.143548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143441+0000","last_change":"2024-09-17T05:25:03.143441+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143441+0000","last_undegraded":"2024-09-17T05:25:03.143441+0000","last_fullsized":"2024-09-17T05:25:03.143441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138842+0000","last_change":"2024-09-17T05:25:03.138842+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138842+0000","last_undegraded":"2024-09-17T05:25:03.138842+0000","last_fullsized":"2024-09-17T05:25:03.138842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143104+0000","last_change":"2024-09-17T05:25:03.143104+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143104+0000","last_undegraded":"2024-09-17T05:25:03.143104+0000","last_fullsized":"2024-09-17T05:25:03.143104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139789+0000","last_change":"2024-09-17T05:25:03.139789+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139789+0000","last_undegraded":"2024-09-17T05:25:03.139789+0000","last_fullsized":"2024-09-17T05:25:03.139789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141592+0000","last_change":"2024-09-17T05:25:03.141592+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141592+0000","last_undegraded":"2024-09-17T05:25:03.141592+0000","last_fullsized":"2024-09-17T05:25:03.141592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136665+0000","last_change":"2024-09-17T05:25:03.136665+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136665+0000","last_undegraded":"2024-09-17T05:25:03.136665+0000","last_fullsized":"2024-09-17T05:25:03.136665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133784+0000","last_change":"2024-09-17T05:25:03.133784+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133784+0000","last_undegraded":"2024-09-17T05:25:03.133784+0000","last_fullsized":"2024-09-17T05:25:03.133784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141020+0000","last_change":"2024-09-17T05:25:03.141020+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141020+0000","last_undegraded":"2024-09-17T05:25:03.141020+0000","last_fullsized":"2024-09-17T05:25:03.141020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141848+0000","last_change":"2024-09-17T05:25:03.141848+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141848+0000","last_undegraded":"2024-09-17T05:25:03.141848+0000","last_fullsized":"2024-09-17T05:25:03.141848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:25:02.131735+0000","last_change":"2024-09-17T05:25:02.131735+0000","last_active":"2024-09-17T05:25:02.108690+0000","last_peered":"2024-09-17T05:25:02.108690+0000","last_clean":"2024-09-17T05:25:02.108690+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:02.131735+0000","last_undegraded":"2024-09-17T05:25:02.131735+0000","last_fullsized":"2024-09-17T05:25:02.131735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.140257+0000","last_change":"2024-09-17T05:25:03.140257+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.140257+0000","last_undegraded":"2024-09-17T05:25:03.140257+0000","last_fullsized":"2024-09-17T05:25:03.140257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.146360+0000","last_change":"2024-09-17T05:25:03.138686+0000","last_active":"2024-09-17T05:25:03.146360+0000","last_peered":"2024-09-17T05:25:03.146360+0000","last_clean":"2024-09-17T05:25:03.146360+0000","last_became_active":"2024-09-17T05:25:03.138321+0000","last_became_peered":"2024-09-17T05:25:03.138321+0000","last_unstale":"2024-09-17T05:25:03.146360+0000","last_undegraded":"2024-09-17T05:25:03.146360+0000","last_fullsized":"2024-09-17T05:25:03.146360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.165802+0000","last_change":"2024-09-17T05:25:03.161161+0000","last_active":"2024-09-17T05:25:03.165802+0000","last_peered":"2024-09-17T05:25:03.165802+0000","last_clean":"2024-09-17T05:25:03.165802+0000","last_became_active":"2024-09-17T05:25:03.160876+0000","last_became_peered":"2024-09-17T05:25:03.160876+0000","last_unstale":"2024-09-17T05:25:03.165802+0000","last_undegraded":"2024-09-17T05:25:03.165802+0000","last_fullsized":"2024-09-17T05:25:03.165802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.158113+0000","last_change":"2024-09-17T05:25:03.156886+0000","last_active":"2024-09-17T05:25:03.158113+0000","last_peered":"2024-09-17T05:25:03.158113+0000","last_clean":"2024-09-17T05:25:03.158113+0000","last_became_active":"2024-09-17T05:25:03.156595+0000","last_became_peered":"2024-09-17T05:25:03.156595+0000","last_unstale":"2024-09-17T05:25:03.158113+0000","last_undegraded":"2024-09-17T05:25:03.158113+0000","last_fullsized":"2024-09-17T05:25:03.158113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:44.043946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139049+0000","last_change":"2024-09-17T05:25:03.139049+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139049+0000","last_undegraded":"2024-09-17T05:25:03.139049+0000","last_fullsized":"2024-09-17T05:25:03.139049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163284+0000","last_change":"2024-09-17T05:25:03.160352+0000","last_active":"2024-09-17T05:25:03.163284+0000","last_peered":"2024-09-17T05:25:03.163284+0000","last_clean":"2024-09-17T05:25:03.163284+0000","last_became_active":"2024-09-17T05:25:03.160030+0000","last_became_peered":"2024-09-17T05:25:03.160030+0000","last_unstale":"2024-09-17T05:25:03.163284+0000","last_undegraded":"2024-09-17T05:25:03.163284+0000","last_fullsized":"2024-09-17T05:25:03.163284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:45.753119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.142678+0000","last_change":"2024-09-17T05:25:03.142678+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.142678+0000","last_undegraded":"2024-09-17T05:25:03.142678+0000","last_fullsized":"2024-09-17T05:25:03.142678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159013+0000","last_change":"2024-09-17T05:25:03.155547+0000","last_active":"2024-09-17T05:25:03.159013+0000","last_peered":"2024-09-17T05:25:03.159013+0000","last_clean":"2024-09-17T05:25:03.159013+0000","last_became_active":"2024-09-17T05:25:03.155270+0000","last_became_peered":"2024-09-17T05:25:03.155270+0000","last_unstale":"2024-09-17T05:25:03.159013+0000","last_undegraded":"2024-09-17T05:25:03.159013+0000","last_fullsized":"2024-09-17T05:25:03.159013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157636+0000","last_change":"2024-09-17T05:25:03.154476+0000","last_active":"2024-09-17T05:25:03.157636+0000","last_peered":"2024-09-17T05:25:03.157636+0000","last_clean":"2024-09-17T05:25:03.157636+0000","last_became_active":"2024-09-17T05:25:03.153306+0000","last_became_peered":"2024-09-17T05:25:03.153306+0000","last_unstale":"2024-09-17T05:25:03.157636+0000","last_undegraded":"2024-09-17T05:25:03.157636+0000","last_fullsized":"2024-09-17T05:25:03.157636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135989+0000","last_change":"2024-09-17T05:25:03.135989+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135989+0000","last_undegraded":"2024-09-17T05:25:03.135989+0000","last_fullsized":"2024-09-17T05:25:03.135989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156191+0000","last_change":"2024-09-17T05:25:03.152426+0000","last_active":"2024-09-17T05:25:03.156191+0000","last_peered":"2024-09-17T05:25:03.156191+0000","last_clean":"2024-09-17T05:25:03.156191+0000","last_became_active":"2024-09-17T05:25:03.152129+0000","last_became_peered":"2024-09-17T05:25:03.152129+0000","last_unstale":"2024-09-17T05:25:03.156191+0000","last_undegraded":"2024-09-17T05:25:03.156191+0000","last_fullsized":"2024-09-17T05:25:03.156191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135542+0000","last_change":"2024-09-17T05:25:03.135542+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135542+0000","last_undegraded":"2024-09-17T05:25:03.135542+0000","last_fullsized":"2024-09-17T05:25:03.135542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.160537+0000","last_change":"2024-09-17T05:25:03.156640+0000","last_active":"2024-09-17T05:25:03.160537+0000","last_peered":"2024-09-17T05:25:03.160537+0000","last_clean":"2024-09-17T05:25:03.160537+0000","last_became_active":"2024-09-17T05:25:03.156338+0000","last_became_peered":"2024-09-17T05:25:03.156338+0000","last_unstale":"2024-09-17T05:25:03.160537+0000","last_undegraded":"2024-09-17T05:25:03.160537+0000","last_fullsized":"2024-09-17T05:25:03.160537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.140939+0000","last_change":"2024-09-17T05:25:03.140939+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.140939+0000","last_undegraded":"2024-09-17T05:25:03.140939+0000","last_fullsized":"2024-09-17T05:25:03.140939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.138842+0000","last_change":"2024-09-17T05:25:03.136217+0000","last_active":"2024-09-17T05:25:03.138842+0000","last_peered":"2024-09-17T05:25:03.138842+0000","last_clean":"2024-09-17T05:25:03.138842+0000","last_became_active":"2024-09-17T05:25:03.135857+0000","last_became_peered":"2024-09-17T05:25:03.135857+0000","last_unstale":"2024-09-17T05:25:03.138842+0000","last_undegraded":"2024-09-17T05:25:03.138842+0000","last_fullsized":"2024-09-17T05:25:03.138842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.132975+0000","last_change":"2024-09-17T05:25:03.132975+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.132975+0000","last_undegraded":"2024-09-17T05:25:03.132975+0000","last_fullsized":"2024-09-17T05:25:03.132975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.153179+0000","last_change":"2024-09-17T05:25:03.147144+0000","last_active":"2024-09-17T05:25:03.153179+0000","last_peered":"2024-09-17T05:25:03.153179+0000","last_clean":"2024-09-17T05:25:03.153179+0000","last_became_active":"2024-09-17T05:25:03.146907+0000","last_became_peered":"2024-09-17T05:25:03.146907+0000","last_unstale":"2024-09-17T05:25:03.153179+0000","last_undegraded":"2024-09-17T05:25:03.153179+0000","last_fullsized":"2024-09-17T05:25:03.153179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143555+0000","last_change":"2024-09-17T05:25:03.143555+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143555+0000","last_undegraded":"2024-09-17T05:25:03.143555+0000","last_fullsized":"2024-09-17T05:25:03.143555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156107+0000","last_change":"2024-09-17T05:25:03.151397+0000","last_active":"2024-09-17T05:25:03.156107+0000","last_peered":"2024-09-17T05:25:03.156107+0000","last_clean":"2024-09-17T05:25:03.156107+0000","last_became_active":"2024-09-17T05:25:03.151076+0000","last_became_peered":"2024-09-17T05:25:03.151076+0000","last_unstale":"2024-09-17T05:25:03.156107+0000","last_undegraded":"2024-09-17T05:25:03.156107+0000","last_fullsized":"2024-09-17T05:25:03.156107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135522+0000","last_change":"2024-09-17T05:25:03.135522+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135522+0000","last_undegraded":"2024-09-17T05:25:03.135522+0000","last_fullsized":"2024-09-17T05:25:03.135522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163289+0000","last_change":"2024-09-17T05:25:03.159880+0000","last_active":"2024-09-17T05:25:03.163289+0000","last_peered":"2024-09-17T05:25:03.163289+0000","last_clean":"2024-09-17T05:25:03.163289+0000","last_became_active":"2024-09-17T05:25:03.159636+0000","last_became_peered":"2024-09-17T05:25:03.159636+0000","last_unstale":"2024-09-17T05:25:03.163289+0000","last_undegraded":"2024-09-17T05:25:03.163289+0000","last_fullsized":"2024-09-17T05:25:03.163289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.142221+0000","last_change":"2024-09-17T05:25:03.142221+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.142221+0000","last_undegraded":"2024-09-17T05:25:03.142221+0000","last_fullsized":"2024-09-17T05:25:03.142221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.165529+0000","last_change":"2024-09-17T05:25:03.161164+0000","last_active":"2024-09-17T05:25:03.165529+0000","last_peered":"2024-09-17T05:25:03.165529+0000","last_clean":"2024-09-17T05:25:03.165529+0000","last_became_active":"2024-09-17T05:25:03.160875+0000","last_became_peered":"2024-09-17T05:25:03.160875+0000","last_unstale":"2024-09-17T05:25:03.165529+0000","last_undegraded":"2024-09-17T05:25:03.165529+0000","last_fullsized":"2024-09-17T05:25:03.165529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.134230+0000","last_change":"2024-09-17T05:25:03.134230+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.134230+0000","last_undegraded":"2024-09-17T05:25:03.134230+0000","last_fullsized":"2024-09-17T05:25:03.134230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:25:02.131668+0000","last_change":"2024-09-17T05:25:02.131668+0000","last_active":"2024-09-17T05:25:02.108690+0000","last_peered":"2024-09-17T05:25:02.108690+0000","last_clean":"2024-09-17T05:25:02.108690+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:02.131668+0000","last_undegraded":"2024-09-17T05:25:02.131668+0000","last_fullsized":"2024-09-17T05:25:02.131668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133863+0000","last_change":"2024-09-17T05:25:03.133863+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133863+0000","last_undegraded":"2024-09-17T05:25:03.133863+0000","last_fullsized":"2024-09-17T05:25:03.133863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.155834+0000","last_change":"2024-09-17T05:25:03.151849+0000","last_active":"2024-09-17T05:25:03.155834+0000","last_peered":"2024-09-17T05:25:03.155834+0000","last_clean":"2024-09-17T05:25:03.155834+0000","last_became_active":"2024-09-17T05:25:03.151518+0000","last_became_peered":"2024-09-17T05:25:03.151518+0000","last_unstale":"2024-09-17T05:25:03.155834+0000","last_undegraded":"2024-09-17T05:25:03.155834+0000","last_fullsized":"2024-09-17T05:25:03.155834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.131630+0000","last_change":"2024-09-17T05:25:03.131630+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.131630+0000","last_undegraded":"2024-09-17T05:25:03.131630+0000","last_fullsized":"2024-09-17T05:25:03.131630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.145884+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:03.145884+0000","last_peered":"2024-09-17T05:25:03.145884+0000","last_clean":"2024-09-17T05:25:03.145884+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:03.145884+0000","last_undegraded":"2024-09-17T05:25:03.145884+0000","last_fullsized":"2024-09-17T05:25:03.145884+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157330+0000","last_change":"2024-09-17T05:25:03.154924+0000","last_active":"2024-09-17T05:25:03.157330+0000","last_peered":"2024-09-17T05:25:03.157330+0000","last_clean":"2024-09-17T05:25:03.157330+0000","last_became_active":"2024-09-17T05:25:03.154274+0000","last_became_peered":"2024-09-17T05:25:03.154274+0000","last_unstale":"2024-09-17T05:25:03.157330+0000","last_undegraded":"2024-09-17T05:25:03.157330+0000","last_fullsized":"2024-09-17T05:25:03.157330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:40.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139080+0000","last_change":"2024-09-17T05:25:03.139080+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139080+0000","last_undegraded":"2024-09-17T05:25:03.139080+0000","last_fullsized":"2024-09-17T05:25:03.139080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156480+0000","last_change":"2024-09-17T05:25:03.154006+0000","last_active":"2024-09-17T05:25:03.156480+0000","last_peered":"2024-09-17T05:25:03.156480+0000","last_clean":"2024-09-17T05:25:03.156480+0000","last_became_active":"2024-09-17T05:25:03.153578+0000","last_became_peered":"2024-09-17T05:25:03.153578+0000","last_unstale":"2024-09-17T05:25:03.156480+0000","last_undegraded":"2024-09-17T05:25:03.156480+0000","last_fullsized":"2024-09-17T05:25:03.156480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138448+0000","last_change":"2024-09-17T05:25:03.138448+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138448+0000","last_undegraded":"2024-09-17T05:25:03.138448+0000","last_fullsized":"2024-09-17T05:25:03.138448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.160481+0000","last_change":"2024-09-17T05:25:03.154866+0000","last_active":"2024-09-17T05:25:03.160481+0000","last_peered":"2024-09-17T05:25:03.160481+0000","last_clean":"2024-09-17T05:25:03.160481+0000","last_became_active":"2024-09-17T05:25:03.154116+0000","last_became_peered":"2024-09-17T05:25:03.154116+0000","last_unstale":"2024-09-17T05:25:03.160481+0000","last_undegraded":"2024-09-17T05:25:03.160481+0000","last_fullsized":"2024-09-17T05:25:03.160481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152473+0000","last_change":"2024-09-17T05:25:03.144930+0000","last_active":"2024-09-17T05:25:03.152473+0000","last_peered":"2024-09-17T05:25:03.152473+0000","last_clean":"2024-09-17T05:25:03.152473+0000","last_became_active":"2024-09-17T05:25:03.144665+0000","last_became_peered":"2024-09-17T05:25:03.144665+0000","last_unstale":"2024-09-17T05:25:03.152473+0000","last_undegraded":"2024-09-17T05:25:03.152473+0000","last_fullsized":"2024-09-17T05:25:03.152473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135607+0000","last_change":"2024-09-17T05:25:03.135607+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135607+0000","last_undegraded":"2024-09-17T05:25:03.135607+0000","last_fullsized":"2024-09-17T05:25:03.135607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:25:02.130426+0000","last_change":"2024-09-17T05:25:02.130426+0000","last_active":"2024-09-17T05:25:02.108690+0000","last_peered":"2024-09-17T05:25:02.108690+0000","last_clean":"2024-09-17T05:25:02.108690+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:02.130426+0000","last_undegraded":"2024-09-17T05:25:02.130426+0000","last_fullsized":"2024-09-17T05:25:02.130426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143119+0000","last_change":"2024-09-17T05:25:03.143119+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143119+0000","last_undegraded":"2024-09-17T05:25:03.143119+0000","last_fullsized":"2024-09-17T05:25:03.143119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152911+0000","last_change":"2024-09-17T05:25:03.150695+0000","last_active":"2024-09-17T05:25:03.152911+0000","last_peered":"2024-09-17T05:25:03.152911+0000","last_clean":"2024-09-17T05:25:03.152911+0000","last_became_active":"2024-09-17T05:25:03.150452+0000","last_became_peered":"2024-09-17T05:25:03.150452+0000","last_unstale":"2024-09-17T05:25:03.152911+0000","last_undegraded":"2024-09-17T05:25:03.152911+0000","last_fullsized":"2024-09-17T05:25:03.152911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139456+0000","last_change":"2024-09-17T05:25:03.139456+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139456+0000","last_undegraded":"2024-09-17T05:25:03.139456+0000","last_fullsized":"2024-09-17T05:25:03.139456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157738+0000","last_change":"2024-09-17T05:25:03.153186+0000","last_active":"2024-09-17T05:25:03.157738+0000","last_peered":"2024-09-17T05:25:03.157738+0000","last_clean":"2024-09-17T05:25:03.157738+0000","last_became_active":"2024-09-17T05:25:03.152579+0000","last_became_peered":"2024-09-17T05:25:03.152579+0000","last_unstale":"2024-09-17T05:25:03.157738+0000","last_undegraded":"2024-09-17T05:25:03.157738+0000","last_fullsized":"2024-09-17T05:25:03.157738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143495+0000","last_change":"2024-09-17T05:25:03.143495+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143495+0000","last_undegraded":"2024-09-17T05:25:03.143495+0000","last_fullsized":"2024-09-17T05:25:03.143495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152271+0000","last_change":"2024-09-17T05:25:03.144098+0000","last_active":"2024-09-17T05:25:03.152271+0000","last_peered":"2024-09-17T05:25:03.152271+0000","last_clean":"2024-09-17T05:25:03.152271+0000","last_became_active":"2024-09-17T05:25:03.137827+0000","last_became_peered":"2024-09-17T05:25:03.137827+0000","last_unstale":"2024-09-17T05:25:03.152271+0000","last_undegraded":"2024-09-17T05:25:03.152271+0000","last_fullsized":"2024-09-17T05:25:03.152271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.146805+0000","last_change":"2024-09-17T05:25:03.146805+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.146805+0000","last_undegraded":"2024-09-17T05:25:03.146805+0000","last_fullsized":"2024-09-17T05:25:03.146805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159052+0000","last_change":"2024-09-17T05:25:03.155817+0000","last_active":"2024-09-17T05:25:03.159052+0000","last_peered":"2024-09-17T05:25:03.159052+0000","last_clean":"2024-09-17T05:25:03.159052+0000","last_became_active":"2024-09-17T05:25:03.155300+0000","last_became_peered":"2024-09-17T05:25:03.155300+0000","last_unstale":"2024-09-17T05:25:03.159052+0000","last_undegraded":"2024-09-17T05:25:03.159052+0000","last_fullsized":"2024-09-17T05:25:03.159052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.134779+0000","last_change":"2024-09-17T05:25:03.134779+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.134779+0000","last_undegraded":"2024-09-17T05:25:03.134779+0000","last_fullsized":"2024-09-17T05:25:03.134779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.161122+0000","last_change":"2024-09-17T05:25:03.155466+0000","last_active":"2024-09-17T05:25:03.161122+0000","last_peered":"2024-09-17T05:25:03.161122+0000","last_clean":"2024-09-17T05:25:03.161122+0000","last_became_active":"2024-09-17T05:25:03.153103+0000","last_became_peered":"2024-09-17T05:25:03.153103+0000","last_unstale":"2024-09-17T05:25:03.161122+0000","last_undegraded":"2024-09-17T05:25:03.161122+0000","last_fullsized":"2024-09-17T05:25:03.161122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138642+0000","last_change":"2024-09-17T05:25:03.138642+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138642+0000","last_undegraded":"2024-09-17T05:25:03.138642+0000","last_fullsized":"2024-09-17T05:25:03.138642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159878+0000","last_change":"2024-09-17T05:25:03.156207+0000","last_active":"2024-09-17T05:25:03.159878+0000","last_peered":"2024-09-17T05:25:03.159878+0000","last_clean":"2024-09-17T05:25:03.159878+0000","last_became_active":"2024-09-17T05:25:03.155889+0000","last_became_peered":"2024-09-17T05:25:03.155889+0000","last_unstale":"2024-09-17T05:25:03.159878+0000","last_undegraded":"2024-09-17T05:25:03.159878+0000","last_fullsized":"2024-09-17T05:25:03.159878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133784+0000","last_change":"2024-09-17T05:25:03.133784+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133784+0000","last_undegraded":"2024-09-17T05:25:03.133784+0000","last_fullsized":"2024-09-17T05:25:03.133784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159266+0000","last_change":"2024-09-17T05:25:03.158082+0000","last_active":"2024-09-17T05:25:03.159266+0000","last_peered":"2024-09-17T05:25:03.159266+0000","last_clean":"2024-09-17T05:25:03.159266+0000","last_became_active":"2024-09-17T05:25:03.157792+0000","last_became_peered":"2024-09-17T05:25:03.157792+0000","last_unstale":"2024-09-17T05:25:03.159266+0000","last_undegraded":"2024-09-17T05:25:03.159266+0000","last_fullsized":"2024-09-17T05:25:03.159266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:29.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133095+0000","last_change":"2024-09-17T05:25:03.133095+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133095+0000","last_undegraded":"2024-09-17T05:25:03.133095+0000","last_fullsized":"2024-09-17T05:25:03.133095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135478+0000","last_change":"2024-09-17T05:25:03.135478+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135478+0000","last_undegraded":"2024-09-17T05:25:03.135478+0000","last_fullsized":"2024-09-17T05:25:03.135478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163246+0000","last_change":"2024-09-17T05:25:03.159779+0000","last_active":"2024-09-17T05:25:03.163246+0000","last_peered":"2024-09-17T05:25:03.163246+0000","last_clean":"2024-09-17T05:25:03.163246+0000","last_became_active":"2024-09-17T05:25:03.159430+0000","last_became_peered":"2024-09-17T05:25:03.159430+0000","last_unstale":"2024-09-17T05:25:03.163246+0000","last_undegraded":"2024-09-17T05:25:03.163246+0000","last_fullsized":"2024-09-17T05:25:03.163246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.145527+0000","last_change":"2024-09-17T05:25:03.145527+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.145527+0000","last_undegraded":"2024-09-17T05:25:03.145527+0000","last_fullsized":"2024-09-17T05:25:03.145527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.158949+0000","last_change":"2024-09-17T05:25:03.153878+0000","last_active":"2024-09-17T05:25:03.158949+0000","last_peered":"2024-09-17T05:25:03.158949+0000","last_clean":"2024-09-17T05:25:03.158949+0000","last_became_active":"2024-09-17T05:25:03.153594+0000","last_became_peered":"2024-09-17T05:25:03.153594+0000","last_unstale":"2024-09-17T05:25:03.158949+0000","last_undegraded":"2024-09-17T05:25:03.158949+0000","last_fullsized":"2024-09-17T05:25:03.158949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-17T05:25:02.130353+0000","last_change":"2024-09-17T05:25:02.130353+0000","last_active":"2024-09-17T05:25:02.108690+0000","last_peered":"2024-09-17T05:25:02.108690+0000","last_clean":"2024-09-17T05:25:02.108690+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:02.130353+0000","last_undegraded":"2024-09-17T05:25:02.130353+0000","last_fullsized":"2024-09-17T05:25:02.130353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:25:03.119083+0000","last_change":"2024-09-17T05:25:03.119083+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.119083+0000","last_undegraded":"2024-09-17T05:25:03.119083+0000","last_fullsized":"2024-09-17T05:25:03.119083+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.161258+0000","last_change":"2024-09-17T05:25:03.156536+0000","last_active":"2024-09-17T05:25:03.161258+0000","last_peered":"2024-09-17T05:25:03.161258+0000","last_clean":"2024-09-17T05:25:03.161258+0000","last_became_active":"2024-09-17T05:25:03.155913+0000","last_became_peered":"2024-09-17T05:25:03.155913+0000","last_unstale":"2024-09-17T05:25:03.161258+0000","last_undegraded":"2024-09-17T05:25:03.161258+0000","last_fullsized":"2024-09-17T05:25:03.161258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:08.253 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:08.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.430+0000 7fe717000640 1 -- 172.21.15.122:0/2572004622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7100b6400 msgr2=0x7fe7100ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.430+0000 7fe717000640 1 --2- 172.21.15.122:0/2572004622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7100b6400 0x7fe7100ccdf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe710074ed0 tx=0x7fe70801c6e0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.430+0000 7fe717000640 1 -- 172.21.15.122:0/2572004622 shutdown_connections 2024-09-17T05:25:08.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.430+0000 7fe717000640 1 --2- 172.21.15.122:0/2572004622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7100b6400 0x7fe7100ccdf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.430+0000 7fe717000640 1 --2- 172.21.15.122:0/2572004622 >> v2:172.21.15.173:3300/0 conn(0x7fe7100cebf0 0x7fe7100cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.430+0000 7fe717000640 1 -- 172.21.15.122:0/2572004622 >> 172.21.15.122:0/2572004622 conn(0x7fe7100d1da0 msgr2=0x7fe7100d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.430+0000 7fe717000640 1 -- 172.21.15.122:0/2572004622 shutdown_connections 2024-09-17T05:25:08.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.430+0000 7fe717000640 1 -- 172.21.15.122:0/2572004622 wait complete. 2024-09-17T05:25:08.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 Processor -- start 2024-09-17T05:25:08.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 -- start start 2024-09-17T05:25:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7100b6400 0x7fe7100c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7100cebf0 0x7fe71014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe71015be80 0x7fe7100b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe710169210 con 0x7fe71015be80 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe710169090 con 0x7fe7100cebf0 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe710169390 con 0x7fe7100b6400 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe71015be80 0x7fe7100b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe71015be80 0x7fe7100b67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36746/0 (socket says 172.21.15.122:36746) 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 -- 172.21.15.122:0/3874462920 learned_addr learned my addr 172.21.15.122:0/3874462920 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe70fe00640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7100b6400 0x7fe7100c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe707400640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7100cebf0 0x7fe71014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 -- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7100b6400 msgr2=0x7fe7100c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7100b6400 0x7fe7100c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 -- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7100cebf0 msgr2=0x7fe71014e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7100cebf0 0x7fe71014e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 -- 172.21.15.122:0/3874462920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7100c95e0 con 0x7fe71015be80 2024-09-17T05:25:08.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe715000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe71015be80 0x7fe7100b67e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe708009850 tx=0x7fe7080a6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe70cc00640 1 -- 172.21.15.122:0/3874462920 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7080ac070 con 0x7fe71015be80 2024-09-17T05:25:08.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe70cc00640 1 -- 172.21.15.122:0/3874462920 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe708005430 con 0x7fe71015be80 2024-09-17T05:25:08.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7100c98c0 con 0x7fe71015be80 2024-09-17T05:25:08.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe70cc00640 1 -- 172.21.15.122:0/3874462920 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7080046d0 con 0x7fe71015be80 2024-09-17T05:25:08.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.434+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7100c9e80 con 0x7fe71015be80 2024-09-17T05:25:08.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.438+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7100ce4e0 con 0x7fe71015be80 2024-09-17T05:25:08.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.438+0000 7fe70cc00640 1 -- 172.21.15.122:0/3874462920 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe7080af070 con 0x7fe71015be80 2024-09-17T05:25:08.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.438+0000 7fe70cc00640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6ec073200 0x7fe6ec0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.438+0000 7fe70cc00640 1 -- 172.21.15.122:0/3874462920 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe70812e350 con 0x7fe71015be80 2024-09-17T05:25:08.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.438+0000 7fe70fe00640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6ec073200 0x7fe6ec0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.438+0000 7fe70fe00640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6ec073200 0x7fe6ec0756c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe70000b450 tx=0x7fe70000d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.442+0000 7fe70cc00640 1 -- 172.21.15.122:0/3874462920 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe7080b6070 con 0x7fe71015be80 2024-09-17T05:25:08.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.606+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7100c6930 con 0x7fe6ec073200 2024-09-17T05:25:08.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.610+0000 7fe70cc00640 1 -- 172.21.15.122:0/3874462920 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247635 (secure 0 0 0) 0x7fe7100c6930 con 0x7fe6ec073200 2024-09-17T05:25:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:08.617 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6ec073200 msgr2=0x7fe6ec0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6ec073200 0x7fe6ec0756c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe70000b450 tx=0x7fe70000d040 comp rx=0 tx=0).stop 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe71015be80 msgr2=0x7fe7100b67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe71015be80 0x7fe7100b67e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe708009850 tx=0x7fe7080a6550 comp rx=0 tx=0).stop 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 shutdown_connections 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6ec073200 0x7fe6ec0756c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe71015be80 0x7fe7100b67e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7100cebf0 0x7fe71014e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 --2- 172.21.15.122:0/3874462920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7100b6400 0x7fe7100c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 >> 172.21.15.122:0/3874462920 conn(0x7fe7100d1da0 msgr2=0x7fe710071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 shutdown_connections 2024-09-17T05:25:08.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.614+0000 7fe717000640 1 -- 172.21.15.122:0/3874462920 wait complete. 2024-09-17T05:25:08.636 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T05:25:08.442331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223084,"kb_used_data":2680,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751636,"statfs":{"total":773094113280,"available":772865675264,"internally_reserved":0,"allocated":2744320,"data_stored":1674475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001365"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139696+0000","last_change":"2024-09-17T05:25:03.139696+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139696+0000","last_undegraded":"2024-09-17T05:25:03.139696+0000","last_fullsized":"2024-09-17T05:25:03.139696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324353+0000","last_change":"2024-09-17T05:25:04.173588+0000","last_active":"2024-09-17T05:25:05.324353+0000","last_peered":"2024-09-17T05:25:05.324353+0000","last_clean":"2024-09-17T05:25:05.324353+0000","last_became_active":"2024-09-17T05:25:04.173260+0000","last_became_peered":"2024-09-17T05:25:04.173260+0000","last_unstale":"2024-09-17T05:25:05.324353+0000","last_undegraded":"2024-09-17T05:25:05.324353+0000","last_fullsized":"2024-09-17T05:25:05.324353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:13.477698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324934+0000","last_change":"2024-09-17T05:25:04.174132+0000","last_active":"2024-09-17T05:25:05.324934+0000","last_peered":"2024-09-17T05:25:05.324934+0000","last_clean":"2024-09-17T05:25:05.324934+0000","last_became_active":"2024-09-17T05:25:04.173337+0000","last_became_peered":"2024-09-17T05:25:04.173337+0000","last_unstale":"2024-09-17T05:25:05.324934+0000","last_undegraded":"2024-09-17T05:25:05.324934+0000","last_fullsized":"2024-09-17T05:25:05.324934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:50.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141593+0000","last_change":"2024-09-17T05:25:03.141593+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141593+0000","last_undegraded":"2024-09-17T05:25:03.141593+0000","last_fullsized":"2024-09-17T05:25:03.141593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139805+0000","last_change":"2024-09-17T05:25:03.139805+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139805+0000","last_undegraded":"2024-09-17T05:25:03.139805+0000","last_fullsized":"2024-09-17T05:25:03.139805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139985+0000","last_change":"2024-09-17T05:25:03.139985+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139985+0000","last_undegraded":"2024-09-17T05:25:03.139985+0000","last_fullsized":"2024-09-17T05:25:03.139985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.137568+0000","last_change":"2024-09-17T05:25:03.137568+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.137568+0000","last_undegraded":"2024-09-17T05:25:03.137568+0000","last_fullsized":"2024-09-17T05:25:03.137568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324125+0000","last_change":"2024-09-17T05:25:04.166345+0000","last_active":"2024-09-17T05:25:05.324125+0000","last_peered":"2024-09-17T05:25:05.324125+0000","last_clean":"2024-09-17T05:25:05.324125+0000","last_became_active":"2024-09-17T05:25:04.165795+0000","last_became_peered":"2024-09-17T05:25:04.165795+0000","last_unstale":"2024-09-17T05:25:05.324125+0000","last_undegraded":"2024-09-17T05:25:05.324125+0000","last_fullsized":"2024-09-17T05:25:05.324125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:33.426170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141669+0000","last_change":"2024-09-17T05:25:03.141669+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141669+0000","last_undegraded":"2024-09-17T05:25:03.141669+0000","last_fullsized":"2024-09-17T05:25:03.141669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136500+0000","last_change":"2024-09-17T05:25:03.136500+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136500+0000","last_undegraded":"2024-09-17T05:25:03.136500+0000","last_fullsized":"2024-09-17T05:25:03.136500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139381+0000","last_change":"2024-09-17T05:25:03.139381+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139381+0000","last_undegraded":"2024-09-17T05:25:03.139381+0000","last_fullsized":"2024-09-17T05:25:03.139381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136937+0000","last_change":"2024-09-17T05:25:03.136937+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136937+0000","last_undegraded":"2024-09-17T05:25:03.136937+0000","last_fullsized":"2024-09-17T05:25:03.136937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138718+0000","last_change":"2024-09-17T05:25:03.138718+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138718+0000","last_undegraded":"2024-09-17T05:25:03.138718+0000","last_fullsized":"2024-09-17T05:25:03.138718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141990+0000","last_change":"2024-09-17T05:25:03.141990+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141990+0000","last_undegraded":"2024-09-17T05:25:03.141990+0000","last_fullsized":"2024-09-17T05:25:03.141990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324904+0000","last_change":"2024-09-17T05:25:04.163764+0000","last_active":"2024-09-17T05:25:05.324904+0000","last_peered":"2024-09-17T05:25:05.324904+0000","last_clean":"2024-09-17T05:25:05.324904+0000","last_became_active":"2024-09-17T05:25:04.163197+0000","last_became_peered":"2024-09-17T05:25:04.163197+0000","last_unstale":"2024-09-17T05:25:05.324904+0000","last_undegraded":"2024-09-17T05:25:05.324904+0000","last_fullsized":"2024-09-17T05:25:05.324904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:01:59.441556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139885+0000","last_change":"2024-09-17T05:25:03.139885+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139885+0000","last_undegraded":"2024-09-17T05:25:03.139885+0000","last_fullsized":"2024-09-17T05:25:03.139885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139387+0000","last_change":"2024-09-17T05:25:03.139387+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139387+0000","last_undegraded":"2024-09-17T05:25:03.139387+0000","last_fullsized":"2024-09-17T05:25:03.139387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133893+0000","last_change":"2024-09-17T05:25:03.133893+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133893+0000","last_undegraded":"2024-09-17T05:25:03.133893+0000","last_fullsized":"2024-09-17T05:25:03.133893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138629+0000","last_change":"2024-09-17T05:25:03.138629+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138629+0000","last_undegraded":"2024-09-17T05:25:03.138629+0000","last_fullsized":"2024-09-17T05:25:03.138629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136431+0000","last_change":"2024-09-17T05:25:03.136431+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136431+0000","last_undegraded":"2024-09-17T05:25:03.136431+0000","last_fullsized":"2024-09-17T05:25:03.136431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143548+0000","last_change":"2024-09-17T05:25:03.143548+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143548+0000","last_undegraded":"2024-09-17T05:25:03.143548+0000","last_fullsized":"2024-09-17T05:25:03.143548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143441+0000","last_change":"2024-09-17T05:25:03.143441+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143441+0000","last_undegraded":"2024-09-17T05:25:03.143441+0000","last_fullsized":"2024-09-17T05:25:03.143441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325148+0000","last_change":"2024-09-17T05:25:04.158936+0000","last_active":"2024-09-17T05:25:05.325148+0000","last_peered":"2024-09-17T05:25:05.325148+0000","last_clean":"2024-09-17T05:25:05.325148+0000","last_became_active":"2024-09-17T05:25:04.158697+0000","last_became_peered":"2024-09-17T05:25:04.158697+0000","last_unstale":"2024-09-17T05:25:05.325148+0000","last_undegraded":"2024-09-17T05:25:05.325148+0000","last_fullsized":"2024-09-17T05:25:05.325148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:45:18.794538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138842+0000","last_change":"2024-09-17T05:25:03.138842+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138842+0000","last_undegraded":"2024-09-17T05:25:03.138842+0000","last_fullsized":"2024-09-17T05:25:03.138842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324281+0000","last_change":"2024-09-17T05:25:04.165631+0000","last_active":"2024-09-17T05:25:05.324281+0000","last_peered":"2024-09-17T05:25:05.324281+0000","last_clean":"2024-09-17T05:25:05.324281+0000","last_became_active":"2024-09-17T05:25:04.163086+0000","last_became_peered":"2024-09-17T05:25:04.163086+0000","last_unstale":"2024-09-17T05:25:05.324281+0000","last_undegraded":"2024-09-17T05:25:05.324281+0000","last_fullsized":"2024-09-17T05:25:05.324281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:59.610017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143104+0000","last_change":"2024-09-17T05:25:03.143104+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143104+0000","last_undegraded":"2024-09-17T05:25:03.143104+0000","last_fullsized":"2024-09-17T05:25:03.143104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139789+0000","last_change":"2024-09-17T05:25:03.139789+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139789+0000","last_undegraded":"2024-09-17T05:25:03.139789+0000","last_fullsized":"2024-09-17T05:25:03.139789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141592+0000","last_change":"2024-09-17T05:25:03.141592+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141592+0000","last_undegraded":"2024-09-17T05:25:03.141592+0000","last_fullsized":"2024-09-17T05:25:03.141592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325254+0000","last_change":"2024-09-17T05:25:04.162109+0000","last_active":"2024-09-17T05:25:05.325254+0000","last_peered":"2024-09-17T05:25:05.325254+0000","last_clean":"2024-09-17T05:25:05.325254+0000","last_became_active":"2024-09-17T05:25:04.161807+0000","last_became_peered":"2024-09-17T05:25:04.161807+0000","last_unstale":"2024-09-17T05:25:05.325254+0000","last_undegraded":"2024-09-17T05:25:05.325254+0000","last_fullsized":"2024-09-17T05:25:05.325254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:23.271600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136665+0000","last_change":"2024-09-17T05:25:03.136665+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136665+0000","last_undegraded":"2024-09-17T05:25:03.136665+0000","last_fullsized":"2024-09-17T05:25:03.136665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133784+0000","last_change":"2024-09-17T05:25:03.133784+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133784+0000","last_undegraded":"2024-09-17T05:25:03.133784+0000","last_fullsized":"2024-09-17T05:25:03.133784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141020+0000","last_change":"2024-09-17T05:25:03.141020+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141020+0000","last_undegraded":"2024-09-17T05:25:03.141020+0000","last_fullsized":"2024-09-17T05:25:03.141020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141848+0000","last_change":"2024-09-17T05:25:03.141848+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141848+0000","last_undegraded":"2024-09-17T05:25:03.141848+0000","last_fullsized":"2024-09-17T05:25:03.141848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325321+0000","last_change":"2024-09-17T05:25:03.160754+0000","last_active":"2024-09-17T05:25:05.325321+0000","last_peered":"2024-09-17T05:25:05.325321+0000","last_clean":"2024-09-17T05:25:05.325321+0000","last_became_active":"2024-09-17T05:25:03.160433+0000","last_became_peered":"2024-09-17T05:25:03.160433+0000","last_unstale":"2024-09-17T05:25:05.325321+0000","last_undegraded":"2024-09-17T05:25:05.325321+0000","last_fullsized":"2024-09-17T05:25:05.325321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:59.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.140257+0000","last_change":"2024-09-17T05:25:03.140257+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.140257+0000","last_undegraded":"2024-09-17T05:25:03.140257+0000","last_fullsized":"2024-09-17T05:25:03.140257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.146360+0000","last_change":"2024-09-17T05:25:03.138686+0000","last_active":"2024-09-17T05:25:03.146360+0000","last_peered":"2024-09-17T05:25:03.146360+0000","last_clean":"2024-09-17T05:25:03.146360+0000","last_became_active":"2024-09-17T05:25:03.138321+0000","last_became_peered":"2024-09-17T05:25:03.138321+0000","last_unstale":"2024-09-17T05:25:03.146360+0000","last_undegraded":"2024-09-17T05:25:03.146360+0000","last_fullsized":"2024-09-17T05:25:03.146360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.165802+0000","last_change":"2024-09-17T05:25:03.161161+0000","last_active":"2024-09-17T05:25:03.165802+0000","last_peered":"2024-09-17T05:25:03.165802+0000","last_clean":"2024-09-17T05:25:03.165802+0000","last_became_active":"2024-09-17T05:25:03.160876+0000","last_became_peered":"2024-09-17T05:25:03.160876+0000","last_unstale":"2024-09-17T05:25:03.165802+0000","last_undegraded":"2024-09-17T05:25:03.165802+0000","last_fullsized":"2024-09-17T05:25:03.165802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324408+0000","last_change":"2024-09-17T05:25:04.173583+0000","last_active":"2024-09-17T05:25:05.324408+0000","last_peered":"2024-09-17T05:25:05.324408+0000","last_clean":"2024-09-17T05:25:05.324408+0000","last_became_active":"2024-09-17T05:25:04.173276+0000","last_became_peered":"2024-09-17T05:25:04.173276+0000","last_unstale":"2024-09-17T05:25:05.324408+0000","last_undegraded":"2024-09-17T05:25:05.324408+0000","last_fullsized":"2024-09-17T05:25:05.324408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:38.290022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.158113+0000","last_change":"2024-09-17T05:25:03.156886+0000","last_active":"2024-09-17T05:25:03.158113+0000","last_peered":"2024-09-17T05:25:03.158113+0000","last_clean":"2024-09-17T05:25:03.158113+0000","last_became_active":"2024-09-17T05:25:03.156595+0000","last_became_peered":"2024-09-17T05:25:03.156595+0000","last_unstale":"2024-09-17T05:25:03.158113+0000","last_undegraded":"2024-09-17T05:25:03.158113+0000","last_fullsized":"2024-09-17T05:25:03.158113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:44.043946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139049+0000","last_change":"2024-09-17T05:25:03.139049+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139049+0000","last_undegraded":"2024-09-17T05:25:03.139049+0000","last_fullsized":"2024-09-17T05:25:03.139049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163284+0000","last_change":"2024-09-17T05:25:03.160352+0000","last_active":"2024-09-17T05:25:03.163284+0000","last_peered":"2024-09-17T05:25:03.163284+0000","last_clean":"2024-09-17T05:25:03.163284+0000","last_became_active":"2024-09-17T05:25:03.160030+0000","last_became_peered":"2024-09-17T05:25:03.160030+0000","last_unstale":"2024-09-17T05:25:03.163284+0000","last_undegraded":"2024-09-17T05:25:03.163284+0000","last_fullsized":"2024-09-17T05:25:03.163284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:45.753119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.142678+0000","last_change":"2024-09-17T05:25:03.142678+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.142678+0000","last_undegraded":"2024-09-17T05:25:03.142678+0000","last_fullsized":"2024-09-17T05:25:03.142678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159013+0000","last_change":"2024-09-17T05:25:03.155547+0000","last_active":"2024-09-17T05:25:03.159013+0000","last_peered":"2024-09-17T05:25:03.159013+0000","last_clean":"2024-09-17T05:25:03.159013+0000","last_became_active":"2024-09-17T05:25:03.155270+0000","last_became_peered":"2024-09-17T05:25:03.155270+0000","last_unstale":"2024-09-17T05:25:03.159013+0000","last_undegraded":"2024-09-17T05:25:03.159013+0000","last_fullsized":"2024-09-17T05:25:03.159013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324064+0000","last_change":"2024-09-17T05:25:04.163379+0000","last_active":"2024-09-17T05:25:05.324064+0000","last_peered":"2024-09-17T05:25:05.324064+0000","last_clean":"2024-09-17T05:25:05.324064+0000","last_became_active":"2024-09-17T05:25:04.163066+0000","last_became_peered":"2024-09-17T05:25:04.163066+0000","last_unstale":"2024-09-17T05:25:05.324064+0000","last_undegraded":"2024-09-17T05:25:05.324064+0000","last_fullsized":"2024-09-17T05:25:05.324064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:43.294053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157636+0000","last_change":"2024-09-17T05:25:03.154476+0000","last_active":"2024-09-17T05:25:03.157636+0000","last_peered":"2024-09-17T05:25:03.157636+0000","last_clean":"2024-09-17T05:25:03.157636+0000","last_became_active":"2024-09-17T05:25:03.153306+0000","last_became_peered":"2024-09-17T05:25:03.153306+0000","last_unstale":"2024-09-17T05:25:03.157636+0000","last_undegraded":"2024-09-17T05:25:03.157636+0000","last_fullsized":"2024-09-17T05:25:03.157636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135989+0000","last_change":"2024-09-17T05:25:03.135989+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135989+0000","last_undegraded":"2024-09-17T05:25:03.135989+0000","last_fullsized":"2024-09-17T05:25:03.135989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156191+0000","last_change":"2024-09-17T05:25:03.152426+0000","last_active":"2024-09-17T05:25:03.156191+0000","last_peered":"2024-09-17T05:25:03.156191+0000","last_clean":"2024-09-17T05:25:03.156191+0000","last_became_active":"2024-09-17T05:25:03.152129+0000","last_became_peered":"2024-09-17T05:25:03.152129+0000","last_unstale":"2024-09-17T05:25:03.156191+0000","last_undegraded":"2024-09-17T05:25:03.156191+0000","last_fullsized":"2024-09-17T05:25:03.156191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135542+0000","last_change":"2024-09-17T05:25:03.135542+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135542+0000","last_undegraded":"2024-09-17T05:25:03.135542+0000","last_fullsized":"2024-09-17T05:25:03.135542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.160537+0000","last_change":"2024-09-17T05:25:03.156640+0000","last_active":"2024-09-17T05:25:03.160537+0000","last_peered":"2024-09-17T05:25:03.160537+0000","last_clean":"2024-09-17T05:25:03.160537+0000","last_became_active":"2024-09-17T05:25:03.156338+0000","last_became_peered":"2024-09-17T05:25:03.156338+0000","last_unstale":"2024-09-17T05:25:03.160537+0000","last_undegraded":"2024-09-17T05:25:03.160537+0000","last_fullsized":"2024-09-17T05:25:03.160537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.140939+0000","last_change":"2024-09-17T05:25:03.140939+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.140939+0000","last_undegraded":"2024-09-17T05:25:03.140939+0000","last_fullsized":"2024-09-17T05:25:03.140939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.138842+0000","last_change":"2024-09-17T05:25:03.136217+0000","last_active":"2024-09-17T05:25:03.138842+0000","last_peered":"2024-09-17T05:25:03.138842+0000","last_clean":"2024-09-17T05:25:03.138842+0000","last_became_active":"2024-09-17T05:25:03.135857+0000","last_became_peered":"2024-09-17T05:25:03.135857+0000","last_unstale":"2024-09-17T05:25:03.138842+0000","last_undegraded":"2024-09-17T05:25:03.138842+0000","last_fullsized":"2024-09-17T05:25:03.138842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.132975+0000","last_change":"2024-09-17T05:25:03.132975+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.132975+0000","last_undegraded":"2024-09-17T05:25:03.132975+0000","last_fullsized":"2024-09-17T05:25:03.132975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.153179+0000","last_change":"2024-09-17T05:25:03.147144+0000","last_active":"2024-09-17T05:25:03.153179+0000","last_peered":"2024-09-17T05:25:03.153179+0000","last_clean":"2024-09-17T05:25:03.153179+0000","last_became_active":"2024-09-17T05:25:03.146907+0000","last_became_peered":"2024-09-17T05:25:03.146907+0000","last_unstale":"2024-09-17T05:25:03.153179+0000","last_undegraded":"2024-09-17T05:25:03.153179+0000","last_fullsized":"2024-09-17T05:25:03.153179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143555+0000","last_change":"2024-09-17T05:25:03.143555+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143555+0000","last_undegraded":"2024-09-17T05:25:03.143555+0000","last_fullsized":"2024-09-17T05:25:03.143555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156107+0000","last_change":"2024-09-17T05:25:03.151397+0000","last_active":"2024-09-17T05:25:03.156107+0000","last_peered":"2024-09-17T05:25:03.156107+0000","last_clean":"2024-09-17T05:25:03.156107+0000","last_became_active":"2024-09-17T05:25:03.151076+0000","last_became_peered":"2024-09-17T05:25:03.151076+0000","last_unstale":"2024-09-17T05:25:03.156107+0000","last_undegraded":"2024-09-17T05:25:03.156107+0000","last_fullsized":"2024-09-17T05:25:03.156107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135522+0000","last_change":"2024-09-17T05:25:03.135522+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135522+0000","last_undegraded":"2024-09-17T05:25:03.135522+0000","last_fullsized":"2024-09-17T05:25:03.135522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163289+0000","last_change":"2024-09-17T05:25:03.159880+0000","last_active":"2024-09-17T05:25:03.163289+0000","last_peered":"2024-09-17T05:25:03.163289+0000","last_clean":"2024-09-17T05:25:03.163289+0000","last_became_active":"2024-09-17T05:25:03.159636+0000","last_became_peered":"2024-09-17T05:25:03.159636+0000","last_unstale":"2024-09-17T05:25:03.163289+0000","last_undegraded":"2024-09-17T05:25:03.163289+0000","last_fullsized":"2024-09-17T05:25:03.163289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.142221+0000","last_change":"2024-09-17T05:25:03.142221+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.142221+0000","last_undegraded":"2024-09-17T05:25:03.142221+0000","last_fullsized":"2024-09-17T05:25:03.142221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.165529+0000","last_change":"2024-09-17T05:25:03.161164+0000","last_active":"2024-09-17T05:25:03.165529+0000","last_peered":"2024-09-17T05:25:03.165529+0000","last_clean":"2024-09-17T05:25:03.165529+0000","last_became_active":"2024-09-17T05:25:03.160875+0000","last_became_peered":"2024-09-17T05:25:03.160875+0000","last_unstale":"2024-09-17T05:25:03.165529+0000","last_undegraded":"2024-09-17T05:25:03.165529+0000","last_fullsized":"2024-09-17T05:25:03.165529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.134230+0000","last_change":"2024-09-17T05:25:03.134230+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.134230+0000","last_undegraded":"2024-09-17T05:25:03.134230+0000","last_fullsized":"2024-09-17T05:25:03.134230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323888+0000","last_change":"2024-09-17T05:25:03.160173+0000","last_active":"2024-09-17T05:25:05.323888+0000","last_peered":"2024-09-17T05:25:05.323888+0000","last_clean":"2024-09-17T05:25:05.323888+0000","last_became_active":"2024-09-17T05:25:03.159759+0000","last_became_peered":"2024-09-17T05:25:03.159759+0000","last_unstale":"2024-09-17T05:25:05.323888+0000","last_undegraded":"2024-09-17T05:25:05.323888+0000","last_fullsized":"2024-09-17T05:25:05.323888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:06.043431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133863+0000","last_change":"2024-09-17T05:25:03.133863+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133863+0000","last_undegraded":"2024-09-17T05:25:03.133863+0000","last_fullsized":"2024-09-17T05:25:03.133863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.155834+0000","last_change":"2024-09-17T05:25:03.151849+0000","last_active":"2024-09-17T05:25:03.155834+0000","last_peered":"2024-09-17T05:25:03.155834+0000","last_clean":"2024-09-17T05:25:03.155834+0000","last_became_active":"2024-09-17T05:25:03.151518+0000","last_became_peered":"2024-09-17T05:25:03.151518+0000","last_unstale":"2024-09-17T05:25:03.155834+0000","last_undegraded":"2024-09-17T05:25:03.155834+0000","last_fullsized":"2024-09-17T05:25:03.155834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.131630+0000","last_change":"2024-09-17T05:25:03.131630+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.131630+0000","last_undegraded":"2024-09-17T05:25:03.131630+0000","last_fullsized":"2024-09-17T05:25:03.131630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.145884+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:03.145884+0000","last_peered":"2024-09-17T05:25:03.145884+0000","last_clean":"2024-09-17T05:25:03.145884+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:03.145884+0000","last_undegraded":"2024-09-17T05:25:03.145884+0000","last_fullsized":"2024-09-17T05:25:03.145884+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157330+0000","last_change":"2024-09-17T05:25:03.154924+0000","last_active":"2024-09-17T05:25:03.157330+0000","last_peered":"2024-09-17T05:25:03.157330+0000","last_clean":"2024-09-17T05:25:03.157330+0000","last_became_active":"2024-09-17T05:25:03.154274+0000","last_became_peered":"2024-09-17T05:25:03.154274+0000","last_unstale":"2024-09-17T05:25:03.157330+0000","last_undegraded":"2024-09-17T05:25:03.157330+0000","last_fullsized":"2024-09-17T05:25:03.157330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:40.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139080+0000","last_change":"2024-09-17T05:25:03.139080+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139080+0000","last_undegraded":"2024-09-17T05:25:03.139080+0000","last_fullsized":"2024-09-17T05:25:03.139080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156480+0000","last_change":"2024-09-17T05:25:03.154006+0000","last_active":"2024-09-17T05:25:03.156480+0000","last_peered":"2024-09-17T05:25:03.156480+0000","last_clean":"2024-09-17T05:25:03.156480+0000","last_became_active":"2024-09-17T05:25:03.153578+0000","last_became_peered":"2024-09-17T05:25:03.153578+0000","last_unstale":"2024-09-17T05:25:03.156480+0000","last_undegraded":"2024-09-17T05:25:03.156480+0000","last_fullsized":"2024-09-17T05:25:03.156480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138448+0000","last_change":"2024-09-17T05:25:03.138448+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138448+0000","last_undegraded":"2024-09-17T05:25:03.138448+0000","last_fullsized":"2024-09-17T05:25:03.138448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.160481+0000","last_change":"2024-09-17T05:25:03.154866+0000","last_active":"2024-09-17T05:25:03.160481+0000","last_peered":"2024-09-17T05:25:03.160481+0000","last_clean":"2024-09-17T05:25:03.160481+0000","last_became_active":"2024-09-17T05:25:03.154116+0000","last_became_peered":"2024-09-17T05:25:03.154116+0000","last_unstale":"2024-09-17T05:25:03.160481+0000","last_undegraded":"2024-09-17T05:25:03.160481+0000","last_fullsized":"2024-09-17T05:25:03.160481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324275+0000","last_change":"2024-09-17T05:25:04.162175+0000","last_active":"2024-09-17T05:25:05.324275+0000","last_peered":"2024-09-17T05:25:05.324275+0000","last_clean":"2024-09-17T05:25:05.324275+0000","last_became_active":"2024-09-17T05:25:04.161820+0000","last_became_peered":"2024-09-17T05:25:04.161820+0000","last_unstale":"2024-09-17T05:25:05.324275+0000","last_undegraded":"2024-09-17T05:25:05.324275+0000","last_fullsized":"2024-09-17T05:25:05.324275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:08.628783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152473+0000","last_change":"2024-09-17T05:25:03.144930+0000","last_active":"2024-09-17T05:25:03.152473+0000","last_peered":"2024-09-17T05:25:03.152473+0000","last_clean":"2024-09-17T05:25:03.152473+0000","last_became_active":"2024-09-17T05:25:03.144665+0000","last_became_peered":"2024-09-17T05:25:03.144665+0000","last_unstale":"2024-09-17T05:25:03.152473+0000","last_undegraded":"2024-09-17T05:25:03.152473+0000","last_fullsized":"2024-09-17T05:25:03.152473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135607+0000","last_change":"2024-09-17T05:25:03.135607+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135607+0000","last_undegraded":"2024-09-17T05:25:03.135607+0000","last_fullsized":"2024-09-17T05:25:03.135607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323728+0000","last_change":"2024-09-17T05:25:03.160065+0000","last_active":"2024-09-17T05:25:05.323728+0000","last_peered":"2024-09-17T05:25:05.323728+0000","last_clean":"2024-09-17T05:25:05.323728+0000","last_became_active":"2024-09-17T05:25:03.159684+0000","last_became_peered":"2024-09-17T05:25:03.159684+0000","last_unstale":"2024-09-17T05:25:05.323728+0000","last_undegraded":"2024-09-17T05:25:05.323728+0000","last_fullsized":"2024-09-17T05:25:05.323728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:04.387303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143119+0000","last_change":"2024-09-17T05:25:03.143119+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143119+0000","last_undegraded":"2024-09-17T05:25:03.143119+0000","last_fullsized":"2024-09-17T05:25:03.143119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152911+0000","last_change":"2024-09-17T05:25:03.150695+0000","last_active":"2024-09-17T05:25:03.152911+0000","last_peered":"2024-09-17T05:25:03.152911+0000","last_clean":"2024-09-17T05:25:03.152911+0000","last_became_active":"2024-09-17T05:25:03.150452+0000","last_became_peered":"2024-09-17T05:25:03.150452+0000","last_unstale":"2024-09-17T05:25:03.152911+0000","last_undegraded":"2024-09-17T05:25:03.152911+0000","last_fullsized":"2024-09-17T05:25:03.152911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139456+0000","last_change":"2024-09-17T05:25:03.139456+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139456+0000","last_undegraded":"2024-09-17T05:25:03.139456+0000","last_fullsized":"2024-09-17T05:25:03.139456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157738+0000","last_change":"2024-09-17T05:25:03.153186+0000","last_active":"2024-09-17T05:25:03.157738+0000","last_peered":"2024-09-17T05:25:03.157738+0000","last_clean":"2024-09-17T05:25:03.157738+0000","last_became_active":"2024-09-17T05:25:03.152579+0000","last_became_peered":"2024-09-17T05:25:03.152579+0000","last_unstale":"2024-09-17T05:25:03.157738+0000","last_undegraded":"2024-09-17T05:25:03.157738+0000","last_fullsized":"2024-09-17T05:25:03.157738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143495+0000","last_change":"2024-09-17T05:25:03.143495+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143495+0000","last_undegraded":"2024-09-17T05:25:03.143495+0000","last_fullsized":"2024-09-17T05:25:03.143495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152271+0000","last_change":"2024-09-17T05:25:03.144098+0000","last_active":"2024-09-17T05:25:03.152271+0000","last_peered":"2024-09-17T05:25:03.152271+0000","last_clean":"2024-09-17T05:25:03.152271+0000","last_became_active":"2024-09-17T05:25:03.137827+0000","last_became_peered":"2024-09-17T05:25:03.137827+0000","last_unstale":"2024-09-17T05:25:03.152271+0000","last_undegraded":"2024-09-17T05:25:03.152271+0000","last_fullsized":"2024-09-17T05:25:03.152271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.146805+0000","last_change":"2024-09-17T05:25:03.146805+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.146805+0000","last_undegraded":"2024-09-17T05:25:03.146805+0000","last_fullsized":"2024-09-17T05:25:03.146805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159052+0000","last_change":"2024-09-17T05:25:03.155817+0000","last_active":"2024-09-17T05:25:03.159052+0000","last_peered":"2024-09-17T05:25:03.159052+0000","last_clean":"2024-09-17T05:25:03.159052+0000","last_became_active":"2024-09-17T05:25:03.155300+0000","last_became_peered":"2024-09-17T05:25:03.155300+0000","last_unstale":"2024-09-17T05:25:03.159052+0000","last_undegraded":"2024-09-17T05:25:03.159052+0000","last_fullsized":"2024-09-17T05:25:03.159052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.134779+0000","last_change":"2024-09-17T05:25:03.134779+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.134779+0000","last_undegraded":"2024-09-17T05:25:03.134779+0000","last_fullsized":"2024-09-17T05:25:03.134779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.161122+0000","last_change":"2024-09-17T05:25:03.155466+0000","last_active":"2024-09-17T05:25:03.161122+0000","last_peered":"2024-09-17T05:25:03.161122+0000","last_clean":"2024-09-17T05:25:03.161122+0000","last_became_active":"2024-09-17T05:25:03.153103+0000","last_became_peered":"2024-09-17T05:25:03.153103+0000","last_unstale":"2024-09-17T05:25:03.161122+0000","last_undegraded":"2024-09-17T05:25:03.161122+0000","last_fullsized":"2024-09-17T05:25:03.161122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138642+0000","last_change":"2024-09-17T05:25:03.138642+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138642+0000","last_undegraded":"2024-09-17T05:25:03.138642+0000","last_fullsized":"2024-09-17T05:25:03.138642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159878+0000","last_change":"2024-09-17T05:25:03.156207+0000","last_active":"2024-09-17T05:25:03.159878+0000","last_peered":"2024-09-17T05:25:03.159878+0000","last_clean":"2024-09-17T05:25:03.159878+0000","last_became_active":"2024-09-17T05:25:03.155889+0000","last_became_peered":"2024-09-17T05:25:03.155889+0000","last_unstale":"2024-09-17T05:25:03.159878+0000","last_undegraded":"2024-09-17T05:25:03.159878+0000","last_fullsized":"2024-09-17T05:25:03.159878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133784+0000","last_change":"2024-09-17T05:25:03.133784+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133784+0000","last_undegraded":"2024-09-17T05:25:03.133784+0000","last_fullsized":"2024-09-17T05:25:03.133784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159266+0000","last_change":"2024-09-17T05:25:03.158082+0000","last_active":"2024-09-17T05:25:03.159266+0000","last_peered":"2024-09-17T05:25:03.159266+0000","last_clean":"2024-09-17T05:25:03.159266+0000","last_became_active":"2024-09-17T05:25:03.157792+0000","last_became_peered":"2024-09-17T05:25:03.157792+0000","last_unstale":"2024-09-17T05:25:03.159266+0000","last_undegraded":"2024-09-17T05:25:03.159266+0000","last_fullsized":"2024-09-17T05:25:03.159266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:29.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133095+0000","last_change":"2024-09-17T05:25:03.133095+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133095+0000","last_undegraded":"2024-09-17T05:25:03.133095+0000","last_fullsized":"2024-09-17T05:25:03.133095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135478+0000","last_change":"2024-09-17T05:25:03.135478+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135478+0000","last_undegraded":"2024-09-17T05:25:03.135478+0000","last_fullsized":"2024-09-17T05:25:03.135478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163246+0000","last_change":"2024-09-17T05:25:03.159779+0000","last_active":"2024-09-17T05:25:03.163246+0000","last_peered":"2024-09-17T05:25:03.163246+0000","last_clean":"2024-09-17T05:25:03.163246+0000","last_became_active":"2024-09-17T05:25:03.159430+0000","last_became_peered":"2024-09-17T05:25:03.159430+0000","last_unstale":"2024-09-17T05:25:03.163246+0000","last_undegraded":"2024-09-17T05:25:03.163246+0000","last_fullsized":"2024-09-17T05:25:03.163246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.145527+0000","last_change":"2024-09-17T05:25:03.145527+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.145527+0000","last_undegraded":"2024-09-17T05:25:03.145527+0000","last_fullsized":"2024-09-17T05:25:03.145527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.158949+0000","last_change":"2024-09-17T05:25:03.153878+0000","last_active":"2024-09-17T05:25:03.158949+0000","last_peered":"2024-09-17T05:25:03.158949+0000","last_clean":"2024-09-17T05:25:03.158949+0000","last_became_active":"2024-09-17T05:25:03.153594+0000","last_became_peered":"2024-09-17T05:25:03.153594+0000","last_unstale":"2024-09-17T05:25:03.158949+0000","last_undegraded":"2024-09-17T05:25:03.158949+0000","last_fullsized":"2024-09-17T05:25:03.158949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323879+0000","last_change":"2024-09-17T05:25:04.163341+0000","last_active":"2024-09-17T05:25:05.323879+0000","last_peered":"2024-09-17T05:25:05.323879+0000","last_clean":"2024-09-17T05:25:05.323879+0000","last_became_active":"2024-09-17T05:25:04.162727+0000","last_became_peered":"2024-09-17T05:25:04.162727+0000","last_unstale":"2024-09-17T05:25:05.323879+0000","last_undegraded":"2024-09-17T05:25:05.323879+0000","last_fullsized":"2024-09-17T05:25:05.323879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:04.451009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324619+0000","last_change":"2024-09-17T05:25:03.157287+0000","last_active":"2024-09-17T05:25:05.324619+0000","last_peered":"2024-09-17T05:25:05.324619+0000","last_clean":"2024-09-17T05:25:05.324619+0000","last_became_active":"2024-09-17T05:25:03.154403+0000","last_became_peered":"2024-09-17T05:25:03.154403+0000","last_unstale":"2024-09-17T05:25:05.324619+0000","last_undegraded":"2024-09-17T05:25:05.324619+0000","last_fullsized":"2024-09-17T05:25:05.324619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:21.699418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324558+0000","last_change":"2024-09-17T05:25:04.167303+0000","last_active":"2024-09-17T05:25:05.324558+0000","last_peered":"2024-09-17T05:25:05.324558+0000","last_clean":"2024-09-17T05:25:05.324558+0000","last_became_active":"2024-09-17T05:25:04.166953+0000","last_became_peered":"2024-09-17T05:25:04.166953+0000","last_unstale":"2024-09-17T05:25:05.324558+0000","last_undegraded":"2024-09-17T05:25:05.324558+0000","last_fullsized":"2024-09-17T05:25:05.324558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:02.920364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.161258+0000","last_change":"2024-09-17T05:25:03.156536+0000","last_active":"2024-09-17T05:25:03.161258+0000","last_peered":"2024-09-17T05:25:03.161258+0000","last_clean":"2024-09-17T05:25:03.161258+0000","last_became_active":"2024-09-17T05:25:03.155913+0000","last_became_peered":"2024-09-17T05:25:03.155913+0000","last_unstale":"2024-09-17T05:25:03.161258+0000","last_undegraded":"2024-09-17T05:25:03.161258+0000","last_fullsized":"2024-09-17T05:25:03.161258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:08.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:08.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- 172.21.15.122:0/4090579211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15bd50 msgr2=0x7f55fc0ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 --2- 172.21.15.122:0/4090579211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15bd50 0x7f55fc0ce5e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009900 tx=0x7f55ec01c7c0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- 172.21.15.122:0/4090579211 shutdown_connections 2024-09-17T05:25:08.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 --2- 172.21.15.122:0/4090579211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc15c130 0x7f55fc0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 --2- 172.21.15.122:0/4090579211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15bd50 0x7f55fc0ce5e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- 172.21.15.122:0/4090579211 >> 172.21.15.122:0/4090579211 conn(0x7f55fc0d1ea0 msgr2=0x7f55fc0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:08.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- 172.21.15.122:0/4090579211 shutdown_connections 2024-09-17T05:25:08.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- 172.21.15.122:0/4090579211 wait complete. 2024-09-17T05:25:08.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 Processor -- start 2024-09-17T05:25:08.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- start start 2024-09-17T05:25:08.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc0cf4e0 0x7f55fc0c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc15bd50 0x7f55fc1661e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15c130 0x7f55fc0c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc168e50 con 0x7f55fc15c130 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc168cd0 con 0x7f55fc0cf4e0 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f5601400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc168fd0 con 0x7f55fc15bd50 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f55fb200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15c130 0x7f55fc0c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.818+0000 7f55fb200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15c130 0x7f55fc0c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36772/0 (socket says 172.21.15.122:36772) 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55fb200640 1 -- 172.21.15.122:0/291725038 learned_addr learned my addr 172.21.15.122:0/291725038 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55f1e00640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc15bd50 0x7f55fc1661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55fa800640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc0cf4e0 0x7f55fc0c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55fb200640 1 -- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc15bd50 msgr2=0x7f55fc1661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55fb200640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc15bd50 0x7f55fc1661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55fb200640 1 -- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc0cf4e0 msgr2=0x7f55fc0c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:08.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55fb200640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc0cf4e0 0x7f55fc0c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:08.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55fb200640 1 -- 172.21.15.122:0/291725038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc071c10 con 0x7f55fc15c130 2024-09-17T05:25:08.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55fb200640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15c130 0x7f55fc0c72a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55f400c730 tx=0x7f55f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55f3400640 1 -- 172.21.15.122:0/291725038 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4003db0 con 0x7f55fc15c130 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55f3400640 1 -- 172.21.15.122:0/291725038 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f400ce40 con 0x7f55fc15c130 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc0cb410 con 0x7f55fc15c130 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55f3400640 1 -- 172.21.15.122:0/291725038 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f40052d0 con 0x7f55fc15c130 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc06fa00 con 0x7f55fc15c130 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55fc0c5dd0 con 0x7f55fc15c130 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.822+0000 7f55f3400640 1 -- 172.21.15.122:0/291725038 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f55f4005470 con 0x7f55fc15c130 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.826+0000 7f55f3400640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d8073200 0x7f55d80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.826+0000 7f55f3400640 1 -- 172.21.15.122:0/291725038 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f55f4093fe0 con 0x7f55fc15c130 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.826+0000 7f55fa800640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d8073200 0x7f55d80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.826+0000 7f55fa800640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d8073200 0x7f55d80756c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0af000 tx=0x7f55ec01b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:08.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.826+0000 7f55f3400640 1 -- 172.21.15.122:0/291725038 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f55f405e980 con 0x7f55fc15c130 2024-09-17T05:25:08.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.994+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55fc070d80 con 0x7f55d8073200 2024-09-17T05:25:09.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:08.998+0000 7f55f3400640 1 -- 172.21.15.122:0/291725038 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247635 (secure 0 0 0) 0x7f55fc070d80 con 0x7f55d8073200 2024-09-17T05:25:09.004 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:09.006 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d8073200 msgr2=0x7f55d80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d8073200 0x7f55d80756c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0af000 tx=0x7f55ec01b000 comp rx=0 tx=0).stop 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15c130 msgr2=0x7f55fc0c72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15c130 0x7f55fc0c72a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55f400c730 tx=0x7f55f400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 shutdown_connections 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d8073200 0x7f55d80756c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc15c130 0x7f55fc0c72a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc15bd50 0x7f55fc1661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 --2- 172.21.15.122:0/291725038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc0cf4e0 0x7f55fc0c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 >> 172.21.15.122:0/291725038 conn(0x7f55fc0d1ea0 msgr2=0x7f55fc0d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 shutdown_connections 2024-09-17T05:25:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.002+0000 7f5601400640 1 -- 172.21.15.122:0/291725038 wait complete. 2024-09-17T05:25:09.024 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T05:25:08.442331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223084,"kb_used_data":2680,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751636,"statfs":{"total":773094113280,"available":772865675264,"internally_reserved":0,"allocated":2744320,"data_stored":1674475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001365"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139696+0000","last_change":"2024-09-17T05:25:03.139696+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139696+0000","last_undegraded":"2024-09-17T05:25:03.139696+0000","last_fullsized":"2024-09-17T05:25:03.139696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324353+0000","last_change":"2024-09-17T05:25:04.173588+0000","last_active":"2024-09-17T05:25:05.324353+0000","last_peered":"2024-09-17T05:25:05.324353+0000","last_clean":"2024-09-17T05:25:05.324353+0000","last_became_active":"2024-09-17T05:25:04.173260+0000","last_became_peered":"2024-09-17T05:25:04.173260+0000","last_unstale":"2024-09-17T05:25:05.324353+0000","last_undegraded":"2024-09-17T05:25:05.324353+0000","last_fullsized":"2024-09-17T05:25:05.324353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:13.477698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324934+0000","last_change":"2024-09-17T05:25:04.174132+0000","last_active":"2024-09-17T05:25:05.324934+0000","last_peered":"2024-09-17T05:25:05.324934+0000","last_clean":"2024-09-17T05:25:05.324934+0000","last_became_active":"2024-09-17T05:25:04.173337+0000","last_became_peered":"2024-09-17T05:25:04.173337+0000","last_unstale":"2024-09-17T05:25:05.324934+0000","last_undegraded":"2024-09-17T05:25:05.324934+0000","last_fullsized":"2024-09-17T05:25:05.324934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:50.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141593+0000","last_change":"2024-09-17T05:25:03.141593+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141593+0000","last_undegraded":"2024-09-17T05:25:03.141593+0000","last_fullsized":"2024-09-17T05:25:03.141593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139805+0000","last_change":"2024-09-17T05:25:03.139805+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139805+0000","last_undegraded":"2024-09-17T05:25:03.139805+0000","last_fullsized":"2024-09-17T05:25:03.139805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139985+0000","last_change":"2024-09-17T05:25:03.139985+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139985+0000","last_undegraded":"2024-09-17T05:25:03.139985+0000","last_fullsized":"2024-09-17T05:25:03.139985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.137568+0000","last_change":"2024-09-17T05:25:03.137568+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.137568+0000","last_undegraded":"2024-09-17T05:25:03.137568+0000","last_fullsized":"2024-09-17T05:25:03.137568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324125+0000","last_change":"2024-09-17T05:25:04.166345+0000","last_active":"2024-09-17T05:25:05.324125+0000","last_peered":"2024-09-17T05:25:05.324125+0000","last_clean":"2024-09-17T05:25:05.324125+0000","last_became_active":"2024-09-17T05:25:04.165795+0000","last_became_peered":"2024-09-17T05:25:04.165795+0000","last_unstale":"2024-09-17T05:25:05.324125+0000","last_undegraded":"2024-09-17T05:25:05.324125+0000","last_fullsized":"2024-09-17T05:25:05.324125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:33.426170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141669+0000","last_change":"2024-09-17T05:25:03.141669+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141669+0000","last_undegraded":"2024-09-17T05:25:03.141669+0000","last_fullsized":"2024-09-17T05:25:03.141669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136500+0000","last_change":"2024-09-17T05:25:03.136500+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136500+0000","last_undegraded":"2024-09-17T05:25:03.136500+0000","last_fullsized":"2024-09-17T05:25:03.136500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139381+0000","last_change":"2024-09-17T05:25:03.139381+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139381+0000","last_undegraded":"2024-09-17T05:25:03.139381+0000","last_fullsized":"2024-09-17T05:25:03.139381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136937+0000","last_change":"2024-09-17T05:25:03.136937+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136937+0000","last_undegraded":"2024-09-17T05:25:03.136937+0000","last_fullsized":"2024-09-17T05:25:03.136937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138718+0000","last_change":"2024-09-17T05:25:03.138718+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138718+0000","last_undegraded":"2024-09-17T05:25:03.138718+0000","last_fullsized":"2024-09-17T05:25:03.138718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141990+0000","last_change":"2024-09-17T05:25:03.141990+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141990+0000","last_undegraded":"2024-09-17T05:25:03.141990+0000","last_fullsized":"2024-09-17T05:25:03.141990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324904+0000","last_change":"2024-09-17T05:25:04.163764+0000","last_active":"2024-09-17T05:25:05.324904+0000","last_peered":"2024-09-17T05:25:05.324904+0000","last_clean":"2024-09-17T05:25:05.324904+0000","last_became_active":"2024-09-17T05:25:04.163197+0000","last_became_peered":"2024-09-17T05:25:04.163197+0000","last_unstale":"2024-09-17T05:25:05.324904+0000","last_undegraded":"2024-09-17T05:25:05.324904+0000","last_fullsized":"2024-09-17T05:25:05.324904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:01:59.441556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139885+0000","last_change":"2024-09-17T05:25:03.139885+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139885+0000","last_undegraded":"2024-09-17T05:25:03.139885+0000","last_fullsized":"2024-09-17T05:25:03.139885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139387+0000","last_change":"2024-09-17T05:25:03.139387+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139387+0000","last_undegraded":"2024-09-17T05:25:03.139387+0000","last_fullsized":"2024-09-17T05:25:03.139387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133893+0000","last_change":"2024-09-17T05:25:03.133893+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133893+0000","last_undegraded":"2024-09-17T05:25:03.133893+0000","last_fullsized":"2024-09-17T05:25:03.133893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138629+0000","last_change":"2024-09-17T05:25:03.138629+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138629+0000","last_undegraded":"2024-09-17T05:25:03.138629+0000","last_fullsized":"2024-09-17T05:25:03.138629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136431+0000","last_change":"2024-09-17T05:25:03.136431+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136431+0000","last_undegraded":"2024-09-17T05:25:03.136431+0000","last_fullsized":"2024-09-17T05:25:03.136431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143548+0000","last_change":"2024-09-17T05:25:03.143548+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143548+0000","last_undegraded":"2024-09-17T05:25:03.143548+0000","last_fullsized":"2024-09-17T05:25:03.143548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143441+0000","last_change":"2024-09-17T05:25:03.143441+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143441+0000","last_undegraded":"2024-09-17T05:25:03.143441+0000","last_fullsized":"2024-09-17T05:25:03.143441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325148+0000","last_change":"2024-09-17T05:25:04.158936+0000","last_active":"2024-09-17T05:25:05.325148+0000","last_peered":"2024-09-17T05:25:05.325148+0000","last_clean":"2024-09-17T05:25:05.325148+0000","last_became_active":"2024-09-17T05:25:04.158697+0000","last_became_peered":"2024-09-17T05:25:04.158697+0000","last_unstale":"2024-09-17T05:25:05.325148+0000","last_undegraded":"2024-09-17T05:25:05.325148+0000","last_fullsized":"2024-09-17T05:25:05.325148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:45:18.794538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138842+0000","last_change":"2024-09-17T05:25:03.138842+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138842+0000","last_undegraded":"2024-09-17T05:25:03.138842+0000","last_fullsized":"2024-09-17T05:25:03.138842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324281+0000","last_change":"2024-09-17T05:25:04.165631+0000","last_active":"2024-09-17T05:25:05.324281+0000","last_peered":"2024-09-17T05:25:05.324281+0000","last_clean":"2024-09-17T05:25:05.324281+0000","last_became_active":"2024-09-17T05:25:04.163086+0000","last_became_peered":"2024-09-17T05:25:04.163086+0000","last_unstale":"2024-09-17T05:25:05.324281+0000","last_undegraded":"2024-09-17T05:25:05.324281+0000","last_fullsized":"2024-09-17T05:25:05.324281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:59.610017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143104+0000","last_change":"2024-09-17T05:25:03.143104+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143104+0000","last_undegraded":"2024-09-17T05:25:03.143104+0000","last_fullsized":"2024-09-17T05:25:03.143104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139789+0000","last_change":"2024-09-17T05:25:03.139789+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139789+0000","last_undegraded":"2024-09-17T05:25:03.139789+0000","last_fullsized":"2024-09-17T05:25:03.139789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141592+0000","last_change":"2024-09-17T05:25:03.141592+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141592+0000","last_undegraded":"2024-09-17T05:25:03.141592+0000","last_fullsized":"2024-09-17T05:25:03.141592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325254+0000","last_change":"2024-09-17T05:25:04.162109+0000","last_active":"2024-09-17T05:25:05.325254+0000","last_peered":"2024-09-17T05:25:05.325254+0000","last_clean":"2024-09-17T05:25:05.325254+0000","last_became_active":"2024-09-17T05:25:04.161807+0000","last_became_peered":"2024-09-17T05:25:04.161807+0000","last_unstale":"2024-09-17T05:25:05.325254+0000","last_undegraded":"2024-09-17T05:25:05.325254+0000","last_fullsized":"2024-09-17T05:25:05.325254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:23.271600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136665+0000","last_change":"2024-09-17T05:25:03.136665+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136665+0000","last_undegraded":"2024-09-17T05:25:03.136665+0000","last_fullsized":"2024-09-17T05:25:03.136665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133784+0000","last_change":"2024-09-17T05:25:03.133784+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133784+0000","last_undegraded":"2024-09-17T05:25:03.133784+0000","last_fullsized":"2024-09-17T05:25:03.133784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141020+0000","last_change":"2024-09-17T05:25:03.141020+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141020+0000","last_undegraded":"2024-09-17T05:25:03.141020+0000","last_fullsized":"2024-09-17T05:25:03.141020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141848+0000","last_change":"2024-09-17T05:25:03.141848+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141848+0000","last_undegraded":"2024-09-17T05:25:03.141848+0000","last_fullsized":"2024-09-17T05:25:03.141848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325321+0000","last_change":"2024-09-17T05:25:03.160754+0000","last_active":"2024-09-17T05:25:05.325321+0000","last_peered":"2024-09-17T05:25:05.325321+0000","last_clean":"2024-09-17T05:25:05.325321+0000","last_became_active":"2024-09-17T05:25:03.160433+0000","last_became_peered":"2024-09-17T05:25:03.160433+0000","last_unstale":"2024-09-17T05:25:05.325321+0000","last_undegraded":"2024-09-17T05:25:05.325321+0000","last_fullsized":"2024-09-17T05:25:05.325321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:59.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.140257+0000","last_change":"2024-09-17T05:25:03.140257+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.140257+0000","last_undegraded":"2024-09-17T05:25:03.140257+0000","last_fullsized":"2024-09-17T05:25:03.140257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.146360+0000","last_change":"2024-09-17T05:25:03.138686+0000","last_active":"2024-09-17T05:25:03.146360+0000","last_peered":"2024-09-17T05:25:03.146360+0000","last_clean":"2024-09-17T05:25:03.146360+0000","last_became_active":"2024-09-17T05:25:03.138321+0000","last_became_peered":"2024-09-17T05:25:03.138321+0000","last_unstale":"2024-09-17T05:25:03.146360+0000","last_undegraded":"2024-09-17T05:25:03.146360+0000","last_fullsized":"2024-09-17T05:25:03.146360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.165802+0000","last_change":"2024-09-17T05:25:03.161161+0000","last_active":"2024-09-17T05:25:03.165802+0000","last_peered":"2024-09-17T05:25:03.165802+0000","last_clean":"2024-09-17T05:25:03.165802+0000","last_became_active":"2024-09-17T05:25:03.160876+0000","last_became_peered":"2024-09-17T05:25:03.160876+0000","last_unstale":"2024-09-17T05:25:03.165802+0000","last_undegraded":"2024-09-17T05:25:03.165802+0000","last_fullsized":"2024-09-17T05:25:03.165802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324408+0000","last_change":"2024-09-17T05:25:04.173583+0000","last_active":"2024-09-17T05:25:05.324408+0000","last_peered":"2024-09-17T05:25:05.324408+0000","last_clean":"2024-09-17T05:25:05.324408+0000","last_became_active":"2024-09-17T05:25:04.173276+0000","last_became_peered":"2024-09-17T05:25:04.173276+0000","last_unstale":"2024-09-17T05:25:05.324408+0000","last_undegraded":"2024-09-17T05:25:05.324408+0000","last_fullsized":"2024-09-17T05:25:05.324408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:38.290022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.158113+0000","last_change":"2024-09-17T05:25:03.156886+0000","last_active":"2024-09-17T05:25:03.158113+0000","last_peered":"2024-09-17T05:25:03.158113+0000","last_clean":"2024-09-17T05:25:03.158113+0000","last_became_active":"2024-09-17T05:25:03.156595+0000","last_became_peered":"2024-09-17T05:25:03.156595+0000","last_unstale":"2024-09-17T05:25:03.158113+0000","last_undegraded":"2024-09-17T05:25:03.158113+0000","last_fullsized":"2024-09-17T05:25:03.158113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:44.043946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139049+0000","last_change":"2024-09-17T05:25:03.139049+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139049+0000","last_undegraded":"2024-09-17T05:25:03.139049+0000","last_fullsized":"2024-09-17T05:25:03.139049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163284+0000","last_change":"2024-09-17T05:25:03.160352+0000","last_active":"2024-09-17T05:25:03.163284+0000","last_peered":"2024-09-17T05:25:03.163284+0000","last_clean":"2024-09-17T05:25:03.163284+0000","last_became_active":"2024-09-17T05:25:03.160030+0000","last_became_peered":"2024-09-17T05:25:03.160030+0000","last_unstale":"2024-09-17T05:25:03.163284+0000","last_undegraded":"2024-09-17T05:25:03.163284+0000","last_fullsized":"2024-09-17T05:25:03.163284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:45.753119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.142678+0000","last_change":"2024-09-17T05:25:03.142678+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.142678+0000","last_undegraded":"2024-09-17T05:25:03.142678+0000","last_fullsized":"2024-09-17T05:25:03.142678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159013+0000","last_change":"2024-09-17T05:25:03.155547+0000","last_active":"2024-09-17T05:25:03.159013+0000","last_peered":"2024-09-17T05:25:03.159013+0000","last_clean":"2024-09-17T05:25:03.159013+0000","last_became_active":"2024-09-17T05:25:03.155270+0000","last_became_peered":"2024-09-17T05:25:03.155270+0000","last_unstale":"2024-09-17T05:25:03.159013+0000","last_undegraded":"2024-09-17T05:25:03.159013+0000","last_fullsized":"2024-09-17T05:25:03.159013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324064+0000","last_change":"2024-09-17T05:25:04.163379+0000","last_active":"2024-09-17T05:25:05.324064+0000","last_peered":"2024-09-17T05:25:05.324064+0000","last_clean":"2024-09-17T05:25:05.324064+0000","last_became_active":"2024-09-17T05:25:04.163066+0000","last_became_peered":"2024-09-17T05:25:04.163066+0000","last_unstale":"2024-09-17T05:25:05.324064+0000","last_undegraded":"2024-09-17T05:25:05.324064+0000","last_fullsized":"2024-09-17T05:25:05.324064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:43.294053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157636+0000","last_change":"2024-09-17T05:25:03.154476+0000","last_active":"2024-09-17T05:25:03.157636+0000","last_peered":"2024-09-17T05:25:03.157636+0000","last_clean":"2024-09-17T05:25:03.157636+0000","last_became_active":"2024-09-17T05:25:03.153306+0000","last_became_peered":"2024-09-17T05:25:03.153306+0000","last_unstale":"2024-09-17T05:25:03.157636+0000","last_undegraded":"2024-09-17T05:25:03.157636+0000","last_fullsized":"2024-09-17T05:25:03.157636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135989+0000","last_change":"2024-09-17T05:25:03.135989+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135989+0000","last_undegraded":"2024-09-17T05:25:03.135989+0000","last_fullsized":"2024-09-17T05:25:03.135989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156191+0000","last_change":"2024-09-17T05:25:03.152426+0000","last_active":"2024-09-17T05:25:03.156191+0000","last_peered":"2024-09-17T05:25:03.156191+0000","last_clean":"2024-09-17T05:25:03.156191+0000","last_became_active":"2024-09-17T05:25:03.152129+0000","last_became_peered":"2024-09-17T05:25:03.152129+0000","last_unstale":"2024-09-17T05:25:03.156191+0000","last_undegraded":"2024-09-17T05:25:03.156191+0000","last_fullsized":"2024-09-17T05:25:03.156191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135542+0000","last_change":"2024-09-17T05:25:03.135542+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135542+0000","last_undegraded":"2024-09-17T05:25:03.135542+0000","last_fullsized":"2024-09-17T05:25:03.135542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.160537+0000","last_change":"2024-09-17T05:25:03.156640+0000","last_active":"2024-09-17T05:25:03.160537+0000","last_peered":"2024-09-17T05:25:03.160537+0000","last_clean":"2024-09-17T05:25:03.160537+0000","last_became_active":"2024-09-17T05:25:03.156338+0000","last_became_peered":"2024-09-17T05:25:03.156338+0000","last_unstale":"2024-09-17T05:25:03.160537+0000","last_undegraded":"2024-09-17T05:25:03.160537+0000","last_fullsized":"2024-09-17T05:25:03.160537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.140939+0000","last_change":"2024-09-17T05:25:03.140939+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.140939+0000","last_undegraded":"2024-09-17T05:25:03.140939+0000","last_fullsized":"2024-09-17T05:25:03.140939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.138842+0000","last_change":"2024-09-17T05:25:03.136217+0000","last_active":"2024-09-17T05:25:03.138842+0000","last_peered":"2024-09-17T05:25:03.138842+0000","last_clean":"2024-09-17T05:25:03.138842+0000","last_became_active":"2024-09-17T05:25:03.135857+0000","last_became_peered":"2024-09-17T05:25:03.135857+0000","last_unstale":"2024-09-17T05:25:03.138842+0000","last_undegraded":"2024-09-17T05:25:03.138842+0000","last_fullsized":"2024-09-17T05:25:03.138842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.132975+0000","last_change":"2024-09-17T05:25:03.132975+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.132975+0000","last_undegraded":"2024-09-17T05:25:03.132975+0000","last_fullsized":"2024-09-17T05:25:03.132975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.153179+0000","last_change":"2024-09-17T05:25:03.147144+0000","last_active":"2024-09-17T05:25:03.153179+0000","last_peered":"2024-09-17T05:25:03.153179+0000","last_clean":"2024-09-17T05:25:03.153179+0000","last_became_active":"2024-09-17T05:25:03.146907+0000","last_became_peered":"2024-09-17T05:25:03.146907+0000","last_unstale":"2024-09-17T05:25:03.153179+0000","last_undegraded":"2024-09-17T05:25:03.153179+0000","last_fullsized":"2024-09-17T05:25:03.153179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143555+0000","last_change":"2024-09-17T05:25:03.143555+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143555+0000","last_undegraded":"2024-09-17T05:25:03.143555+0000","last_fullsized":"2024-09-17T05:25:03.143555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156107+0000","last_change":"2024-09-17T05:25:03.151397+0000","last_active":"2024-09-17T05:25:03.156107+0000","last_peered":"2024-09-17T05:25:03.156107+0000","last_clean":"2024-09-17T05:25:03.156107+0000","last_became_active":"2024-09-17T05:25:03.151076+0000","last_became_peered":"2024-09-17T05:25:03.151076+0000","last_unstale":"2024-09-17T05:25:03.156107+0000","last_undegraded":"2024-09-17T05:25:03.156107+0000","last_fullsized":"2024-09-17T05:25:03.156107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135522+0000","last_change":"2024-09-17T05:25:03.135522+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135522+0000","last_undegraded":"2024-09-17T05:25:03.135522+0000","last_fullsized":"2024-09-17T05:25:03.135522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163289+0000","last_change":"2024-09-17T05:25:03.159880+0000","last_active":"2024-09-17T05:25:03.163289+0000","last_peered":"2024-09-17T05:25:03.163289+0000","last_clean":"2024-09-17T05:25:03.163289+0000","last_became_active":"2024-09-17T05:25:03.159636+0000","last_became_peered":"2024-09-17T05:25:03.159636+0000","last_unstale":"2024-09-17T05:25:03.163289+0000","last_undegraded":"2024-09-17T05:25:03.163289+0000","last_fullsized":"2024-09-17T05:25:03.163289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.142221+0000","last_change":"2024-09-17T05:25:03.142221+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.142221+0000","last_undegraded":"2024-09-17T05:25:03.142221+0000","last_fullsized":"2024-09-17T05:25:03.142221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.165529+0000","last_change":"2024-09-17T05:25:03.161164+0000","last_active":"2024-09-17T05:25:03.165529+0000","last_peered":"2024-09-17T05:25:03.165529+0000","last_clean":"2024-09-17T05:25:03.165529+0000","last_became_active":"2024-09-17T05:25:03.160875+0000","last_became_peered":"2024-09-17T05:25:03.160875+0000","last_unstale":"2024-09-17T05:25:03.165529+0000","last_undegraded":"2024-09-17T05:25:03.165529+0000","last_fullsized":"2024-09-17T05:25:03.165529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.134230+0000","last_change":"2024-09-17T05:25:03.134230+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.134230+0000","last_undegraded":"2024-09-17T05:25:03.134230+0000","last_fullsized":"2024-09-17T05:25:03.134230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323888+0000","last_change":"2024-09-17T05:25:03.160173+0000","last_active":"2024-09-17T05:25:05.323888+0000","last_peered":"2024-09-17T05:25:05.323888+0000","last_clean":"2024-09-17T05:25:05.323888+0000","last_became_active":"2024-09-17T05:25:03.159759+0000","last_became_peered":"2024-09-17T05:25:03.159759+0000","last_unstale":"2024-09-17T05:25:05.323888+0000","last_undegraded":"2024-09-17T05:25:05.323888+0000","last_fullsized":"2024-09-17T05:25:05.323888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:06.043431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133863+0000","last_change":"2024-09-17T05:25:03.133863+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133863+0000","last_undegraded":"2024-09-17T05:25:03.133863+0000","last_fullsized":"2024-09-17T05:25:03.133863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.155834+0000","last_change":"2024-09-17T05:25:03.151849+0000","last_active":"2024-09-17T05:25:03.155834+0000","last_peered":"2024-09-17T05:25:03.155834+0000","last_clean":"2024-09-17T05:25:03.155834+0000","last_became_active":"2024-09-17T05:25:03.151518+0000","last_became_peered":"2024-09-17T05:25:03.151518+0000","last_unstale":"2024-09-17T05:25:03.155834+0000","last_undegraded":"2024-09-17T05:25:03.155834+0000","last_fullsized":"2024-09-17T05:25:03.155834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.131630+0000","last_change":"2024-09-17T05:25:03.131630+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.131630+0000","last_undegraded":"2024-09-17T05:25:03.131630+0000","last_fullsized":"2024-09-17T05:25:03.131630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.145884+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:03.145884+0000","last_peered":"2024-09-17T05:25:03.145884+0000","last_clean":"2024-09-17T05:25:03.145884+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:03.145884+0000","last_undegraded":"2024-09-17T05:25:03.145884+0000","last_fullsized":"2024-09-17T05:25:03.145884+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157330+0000","last_change":"2024-09-17T05:25:03.154924+0000","last_active":"2024-09-17T05:25:03.157330+0000","last_peered":"2024-09-17T05:25:03.157330+0000","last_clean":"2024-09-17T05:25:03.157330+0000","last_became_active":"2024-09-17T05:25:03.154274+0000","last_became_peered":"2024-09-17T05:25:03.154274+0000","last_unstale":"2024-09-17T05:25:03.157330+0000","last_undegraded":"2024-09-17T05:25:03.157330+0000","last_fullsized":"2024-09-17T05:25:03.157330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:40.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139080+0000","last_change":"2024-09-17T05:25:03.139080+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139080+0000","last_undegraded":"2024-09-17T05:25:03.139080+0000","last_fullsized":"2024-09-17T05:25:03.139080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156480+0000","last_change":"2024-09-17T05:25:03.154006+0000","last_active":"2024-09-17T05:25:03.156480+0000","last_peered":"2024-09-17T05:25:03.156480+0000","last_clean":"2024-09-17T05:25:03.156480+0000","last_became_active":"2024-09-17T05:25:03.153578+0000","last_became_peered":"2024-09-17T05:25:03.153578+0000","last_unstale":"2024-09-17T05:25:03.156480+0000","last_undegraded":"2024-09-17T05:25:03.156480+0000","last_fullsized":"2024-09-17T05:25:03.156480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138448+0000","last_change":"2024-09-17T05:25:03.138448+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138448+0000","last_undegraded":"2024-09-17T05:25:03.138448+0000","last_fullsized":"2024-09-17T05:25:03.138448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.160481+0000","last_change":"2024-09-17T05:25:03.154866+0000","last_active":"2024-09-17T05:25:03.160481+0000","last_peered":"2024-09-17T05:25:03.160481+0000","last_clean":"2024-09-17T05:25:03.160481+0000","last_became_active":"2024-09-17T05:25:03.154116+0000","last_became_peered":"2024-09-17T05:25:03.154116+0000","last_unstale":"2024-09-17T05:25:03.160481+0000","last_undegraded":"2024-09-17T05:25:03.160481+0000","last_fullsized":"2024-09-17T05:25:03.160481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324275+0000","last_change":"2024-09-17T05:25:04.162175+0000","last_active":"2024-09-17T05:25:05.324275+0000","last_peered":"2024-09-17T05:25:05.324275+0000","last_clean":"2024-09-17T05:25:05.324275+0000","last_became_active":"2024-09-17T05:25:04.161820+0000","last_became_peered":"2024-09-17T05:25:04.161820+0000","last_unstale":"2024-09-17T05:25:05.324275+0000","last_undegraded":"2024-09-17T05:25:05.324275+0000","last_fullsized":"2024-09-17T05:25:05.324275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:08.628783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152473+0000","last_change":"2024-09-17T05:25:03.144930+0000","last_active":"2024-09-17T05:25:03.152473+0000","last_peered":"2024-09-17T05:25:03.152473+0000","last_clean":"2024-09-17T05:25:03.152473+0000","last_became_active":"2024-09-17T05:25:03.144665+0000","last_became_peered":"2024-09-17T05:25:03.144665+0000","last_unstale":"2024-09-17T05:25:03.152473+0000","last_undegraded":"2024-09-17T05:25:03.152473+0000","last_fullsized":"2024-09-17T05:25:03.152473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135607+0000","last_change":"2024-09-17T05:25:03.135607+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135607+0000","last_undegraded":"2024-09-17T05:25:03.135607+0000","last_fullsized":"2024-09-17T05:25:03.135607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323728+0000","last_change":"2024-09-17T05:25:03.160065+0000","last_active":"2024-09-17T05:25:05.323728+0000","last_peered":"2024-09-17T05:25:05.323728+0000","last_clean":"2024-09-17T05:25:05.323728+0000","last_became_active":"2024-09-17T05:25:03.159684+0000","last_became_peered":"2024-09-17T05:25:03.159684+0000","last_unstale":"2024-09-17T05:25:05.323728+0000","last_undegraded":"2024-09-17T05:25:05.323728+0000","last_fullsized":"2024-09-17T05:25:05.323728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:04.387303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143119+0000","last_change":"2024-09-17T05:25:03.143119+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143119+0000","last_undegraded":"2024-09-17T05:25:03.143119+0000","last_fullsized":"2024-09-17T05:25:03.143119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152911+0000","last_change":"2024-09-17T05:25:03.150695+0000","last_active":"2024-09-17T05:25:03.152911+0000","last_peered":"2024-09-17T05:25:03.152911+0000","last_clean":"2024-09-17T05:25:03.152911+0000","last_became_active":"2024-09-17T05:25:03.150452+0000","last_became_peered":"2024-09-17T05:25:03.150452+0000","last_unstale":"2024-09-17T05:25:03.152911+0000","last_undegraded":"2024-09-17T05:25:03.152911+0000","last_fullsized":"2024-09-17T05:25:03.152911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139456+0000","last_change":"2024-09-17T05:25:03.139456+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139456+0000","last_undegraded":"2024-09-17T05:25:03.139456+0000","last_fullsized":"2024-09-17T05:25:03.139456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157738+0000","last_change":"2024-09-17T05:25:03.153186+0000","last_active":"2024-09-17T05:25:03.157738+0000","last_peered":"2024-09-17T05:25:03.157738+0000","last_clean":"2024-09-17T05:25:03.157738+0000","last_became_active":"2024-09-17T05:25:03.152579+0000","last_became_peered":"2024-09-17T05:25:03.152579+0000","last_unstale":"2024-09-17T05:25:03.157738+0000","last_undegraded":"2024-09-17T05:25:03.157738+0000","last_fullsized":"2024-09-17T05:25:03.157738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143495+0000","last_change":"2024-09-17T05:25:03.143495+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143495+0000","last_undegraded":"2024-09-17T05:25:03.143495+0000","last_fullsized":"2024-09-17T05:25:03.143495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152271+0000","last_change":"2024-09-17T05:25:03.144098+0000","last_active":"2024-09-17T05:25:03.152271+0000","last_peered":"2024-09-17T05:25:03.152271+0000","last_clean":"2024-09-17T05:25:03.152271+0000","last_became_active":"2024-09-17T05:25:03.137827+0000","last_became_peered":"2024-09-17T05:25:03.137827+0000","last_unstale":"2024-09-17T05:25:03.152271+0000","last_undegraded":"2024-09-17T05:25:03.152271+0000","last_fullsized":"2024-09-17T05:25:03.152271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.146805+0000","last_change":"2024-09-17T05:25:03.146805+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.146805+0000","last_undegraded":"2024-09-17T05:25:03.146805+0000","last_fullsized":"2024-09-17T05:25:03.146805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159052+0000","last_change":"2024-09-17T05:25:03.155817+0000","last_active":"2024-09-17T05:25:03.159052+0000","last_peered":"2024-09-17T05:25:03.159052+0000","last_clean":"2024-09-17T05:25:03.159052+0000","last_became_active":"2024-09-17T05:25:03.155300+0000","last_became_peered":"2024-09-17T05:25:03.155300+0000","last_unstale":"2024-09-17T05:25:03.159052+0000","last_undegraded":"2024-09-17T05:25:03.159052+0000","last_fullsized":"2024-09-17T05:25:03.159052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.134779+0000","last_change":"2024-09-17T05:25:03.134779+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.134779+0000","last_undegraded":"2024-09-17T05:25:03.134779+0000","last_fullsized":"2024-09-17T05:25:03.134779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.161122+0000","last_change":"2024-09-17T05:25:03.155466+0000","last_active":"2024-09-17T05:25:03.161122+0000","last_peered":"2024-09-17T05:25:03.161122+0000","last_clean":"2024-09-17T05:25:03.161122+0000","last_became_active":"2024-09-17T05:25:03.153103+0000","last_became_peered":"2024-09-17T05:25:03.153103+0000","last_unstale":"2024-09-17T05:25:03.161122+0000","last_undegraded":"2024-09-17T05:25:03.161122+0000","last_fullsized":"2024-09-17T05:25:03.161122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138642+0000","last_change":"2024-09-17T05:25:03.138642+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138642+0000","last_undegraded":"2024-09-17T05:25:03.138642+0000","last_fullsized":"2024-09-17T05:25:03.138642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159878+0000","last_change":"2024-09-17T05:25:03.156207+0000","last_active":"2024-09-17T05:25:03.159878+0000","last_peered":"2024-09-17T05:25:03.159878+0000","last_clean":"2024-09-17T05:25:03.159878+0000","last_became_active":"2024-09-17T05:25:03.155889+0000","last_became_peered":"2024-09-17T05:25:03.155889+0000","last_unstale":"2024-09-17T05:25:03.159878+0000","last_undegraded":"2024-09-17T05:25:03.159878+0000","last_fullsized":"2024-09-17T05:25:03.159878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133784+0000","last_change":"2024-09-17T05:25:03.133784+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133784+0000","last_undegraded":"2024-09-17T05:25:03.133784+0000","last_fullsized":"2024-09-17T05:25:03.133784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159266+0000","last_change":"2024-09-17T05:25:03.158082+0000","last_active":"2024-09-17T05:25:03.159266+0000","last_peered":"2024-09-17T05:25:03.159266+0000","last_clean":"2024-09-17T05:25:03.159266+0000","last_became_active":"2024-09-17T05:25:03.157792+0000","last_became_peered":"2024-09-17T05:25:03.157792+0000","last_unstale":"2024-09-17T05:25:03.159266+0000","last_undegraded":"2024-09-17T05:25:03.159266+0000","last_fullsized":"2024-09-17T05:25:03.159266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:29.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133095+0000","last_change":"2024-09-17T05:25:03.133095+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133095+0000","last_undegraded":"2024-09-17T05:25:03.133095+0000","last_fullsized":"2024-09-17T05:25:03.133095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135478+0000","last_change":"2024-09-17T05:25:03.135478+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135478+0000","last_undegraded":"2024-09-17T05:25:03.135478+0000","last_fullsized":"2024-09-17T05:25:03.135478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163246+0000","last_change":"2024-09-17T05:25:03.159779+0000","last_active":"2024-09-17T05:25:03.163246+0000","last_peered":"2024-09-17T05:25:03.163246+0000","last_clean":"2024-09-17T05:25:03.163246+0000","last_became_active":"2024-09-17T05:25:03.159430+0000","last_became_peered":"2024-09-17T05:25:03.159430+0000","last_unstale":"2024-09-17T05:25:03.163246+0000","last_undegraded":"2024-09-17T05:25:03.163246+0000","last_fullsized":"2024-09-17T05:25:03.163246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.145527+0000","last_change":"2024-09-17T05:25:03.145527+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.145527+0000","last_undegraded":"2024-09-17T05:25:03.145527+0000","last_fullsized":"2024-09-17T05:25:03.145527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.158949+0000","last_change":"2024-09-17T05:25:03.153878+0000","last_active":"2024-09-17T05:25:03.158949+0000","last_peered":"2024-09-17T05:25:03.158949+0000","last_clean":"2024-09-17T05:25:03.158949+0000","last_became_active":"2024-09-17T05:25:03.153594+0000","last_became_peered":"2024-09-17T05:25:03.153594+0000","last_unstale":"2024-09-17T05:25:03.158949+0000","last_undegraded":"2024-09-17T05:25:03.158949+0000","last_fullsized":"2024-09-17T05:25:03.158949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323879+0000","last_change":"2024-09-17T05:25:04.163341+0000","last_active":"2024-09-17T05:25:05.323879+0000","last_peered":"2024-09-17T05:25:05.323879+0000","last_clean":"2024-09-17T05:25:05.323879+0000","last_became_active":"2024-09-17T05:25:04.162727+0000","last_became_peered":"2024-09-17T05:25:04.162727+0000","last_unstale":"2024-09-17T05:25:05.323879+0000","last_undegraded":"2024-09-17T05:25:05.323879+0000","last_fullsized":"2024-09-17T05:25:05.323879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:04.451009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324619+0000","last_change":"2024-09-17T05:25:03.157287+0000","last_active":"2024-09-17T05:25:05.324619+0000","last_peered":"2024-09-17T05:25:05.324619+0000","last_clean":"2024-09-17T05:25:05.324619+0000","last_became_active":"2024-09-17T05:25:03.154403+0000","last_became_peered":"2024-09-17T05:25:03.154403+0000","last_unstale":"2024-09-17T05:25:05.324619+0000","last_undegraded":"2024-09-17T05:25:05.324619+0000","last_fullsized":"2024-09-17T05:25:05.324619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:21.699418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324558+0000","last_change":"2024-09-17T05:25:04.167303+0000","last_active":"2024-09-17T05:25:05.324558+0000","last_peered":"2024-09-17T05:25:05.324558+0000","last_clean":"2024-09-17T05:25:05.324558+0000","last_became_active":"2024-09-17T05:25:04.166953+0000","last_became_peered":"2024-09-17T05:25:04.166953+0000","last_unstale":"2024-09-17T05:25:05.324558+0000","last_undegraded":"2024-09-17T05:25:05.324558+0000","last_fullsized":"2024-09-17T05:25:05.324558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:02.920364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.161258+0000","last_change":"2024-09-17T05:25:03.156536+0000","last_active":"2024-09-17T05:25:03.161258+0000","last_peered":"2024-09-17T05:25:03.161258+0000","last_clean":"2024-09-17T05:25:03.161258+0000","last_became_active":"2024-09-17T05:25:03.155913+0000","last_became_peered":"2024-09-17T05:25:03.155913+0000","last_unstale":"2024-09-17T05:25:03.161258+0000","last_undegraded":"2024-09-17T05:25:03.161258+0000","last_fullsized":"2024-09-17T05:25:03.161258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:09.028 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-17T05:25:09.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139696+0000', 'last_change': '2024-09-17T05:25:03.139696+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139696+0000', 'last_undegraded': '2024-09-17T05:25:03.139696+0000', 'last_fullsized': '2024-09-17T05:25:03.139696+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.028 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-17T05:25:09.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.141593+0000', 'last_change': '2024-09-17T05:25:03.141593+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.141593+0000', 'last_undegraded': '2024-09-17T05:25:03.141593+0000', 'last_fullsized': '2024-09-17T05:25:03.141593+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:25:09.028 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139805+0000', 'last_change': '2024-09-17T05:25:03.139805+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139805+0000', 'last_undegraded': '2024-09-17T05:25:03.139805+0000', 'last_fullsized': '2024-09-17T05:25:03.139805+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139985+0000', 'last_change': '2024-09-17T05:25:03.139985+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139985+0000', 'last_undegraded': '2024-09-17T05:25:03.139985+0000', 'last_fullsized': '2024-09-17T05:25:03.139985+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.137568+0000', 'last_change': '2024-09-17T05:25:03.137568+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.137568+0000', 'last_undegraded': '2024-09-17T05:25:03.137568+0000', 'last_fullsized': '2024-09-17T05:25:03.137568+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.141669+0000', 'last_change': '2024-09-17T05:25:03.141669+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.141669+0000', 'last_undegraded': '2024-09-17T05:25:03.141669+0000', 'last_fullsized': '2024-09-17T05:25:03.141669+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.136500+0000', 'last_change': '2024-09-17T05:25:03.136500+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.136500+0000', 'last_undegraded': '2024-09-17T05:25:03.136500+0000', 'last_fullsized': '2024-09-17T05:25:03.136500+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-17T05:25:09.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139381+0000', 'last_change': '2024-09-17T05:25:03.139381+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139381+0000', 'last_undegraded': '2024-09-17T05:25:03.139381+0000', 'last_fullsized': '2024-09-17T05:25:03.139381+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.030 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-17T05:25:09.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.136937+0000', 'last_change': '2024-09-17T05:25:03.136937+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.136937+0000', 'last_undegraded': '2024-09-17T05:25:03.136937+0000', 'last_fullsized': '2024-09-17T05:25:03.136937+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.030 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-17T05:25:09.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.138718+0000', 'last_change': '2024-09-17T05:25:03.138718+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.138718+0000', 'last_undegraded': '2024-09-17T05:25:03.138718+0000', 'last_fullsized': '2024-09-17T05:25:03.138718+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:09.030 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-17T05:25:09.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.141990+0000', 'last_change': '2024-09-17T05:25:03.141990+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.141990+0000', 'last_undegraded': '2024-09-17T05:25:03.141990+0000', 'last_fullsized': '2024-09-17T05:25:03.141990+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:09.030 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-17T05:25:09.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139885+0000', 'last_change': '2024-09-17T05:25:03.139885+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139885+0000', 'last_undegraded': '2024-09-17T05:25:03.139885+0000', 'last_fullsized': '2024-09-17T05:25:03.139885+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139387+0000', 'last_change': '2024-09-17T05:25:03.139387+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139387+0000', 'last_undegraded': '2024-09-17T05:25:03.139387+0000', 'last_fullsized': '2024-09-17T05:25:03.139387+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.133893+0000', 'last_change': '2024-09-17T05:25:03.133893+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.133893+0000', 'last_undegraded': '2024-09-17T05:25:03.133893+0000', 'last_fullsized': '2024-09-17T05:25:03.133893+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.138629+0000', 'last_change': '2024-09-17T05:25:03.138629+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.138629+0000', 'last_undegraded': '2024-09-17T05:25:03.138629+0000', 'last_fullsized': '2024-09-17T05:25:03.138629+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.136431+0000', 'last_change': '2024-09-17T05:25:03.136431+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.136431+0000', 'last_undegraded': '2024-09-17T05:25:03.136431+0000', 'last_fullsized': '2024-09-17T05:25:03.136431+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.143548+0000', 'last_change': '2024-09-17T05:25:03.143548+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.143548+0000', 'last_undegraded': '2024-09-17T05:25:03.143548+0000', 'last_fullsized': '2024-09-17T05:25:03.143548+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:25:09.031 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.143441+0000', 'last_change': '2024-09-17T05:25:03.143441+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.143441+0000', 'last_undegraded': '2024-09-17T05:25:03.143441+0000', 'last_fullsized': '2024-09-17T05:25:03.143441+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.138842+0000', 'last_change': '2024-09-17T05:25:03.138842+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.138842+0000', 'last_undegraded': '2024-09-17T05:25:03.138842+0000', 'last_fullsized': '2024-09-17T05:25:03.138842+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.143104+0000', 'last_change': '2024-09-17T05:25:03.143104+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.143104+0000', 'last_undegraded': '2024-09-17T05:25:03.143104+0000', 'last_fullsized': '2024-09-17T05:25:03.143104+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139789+0000', 'last_change': '2024-09-17T05:25:03.139789+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139789+0000', 'last_undegraded': '2024-09-17T05:25:03.139789+0000', 'last_fullsized': '2024-09-17T05:25:03.139789+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.141592+0000', 'last_change': '2024-09-17T05:25:03.141592+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.141592+0000', 'last_undegraded': '2024-09-17T05:25:03.141592+0000', 'last_fullsized': '2024-09-17T05:25:03.141592+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-17T05:25:09.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.136665+0000', 'last_change': '2024-09-17T05:25:03.136665+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.136665+0000', 'last_undegraded': '2024-09-17T05:25:03.136665+0000', 'last_fullsized': '2024-09-17T05:25:03.136665+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.133784+0000', 'last_change': '2024-09-17T05:25:03.133784+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.133784+0000', 'last_undegraded': '2024-09-17T05:25:03.133784+0000', 'last_fullsized': '2024-09-17T05:25:03.133784+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.141020+0000', 'last_change': '2024-09-17T05:25:03.141020+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.141020+0000', 'last_undegraded': '2024-09-17T05:25:03.141020+0000', 'last_fullsized': '2024-09-17T05:25:03.141020+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.141848+0000', 'last_change': '2024-09-17T05:25:03.141848+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.141848+0000', 'last_undegraded': '2024-09-17T05:25:03.141848+0000', 'last_fullsized': '2024-09-17T05:25:03.141848+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.140257+0000', 'last_change': '2024-09-17T05:25:03.140257+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.140257+0000', 'last_undegraded': '2024-09-17T05:25:03.140257+0000', 'last_fullsized': '2024-09-17T05:25:03.140257+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139049+0000', 'last_change': '2024-09-17T05:25:03.139049+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139049+0000', 'last_undegraded': '2024-09-17T05:25:03.139049+0000', 'last_fullsized': '2024-09-17T05:25:03.139049+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:25:09.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.142678+0000', 'last_change': '2024-09-17T05:25:03.142678+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.142678+0000', 'last_undegraded': '2024-09-17T05:25:03.142678+0000', 'last_fullsized': '2024-09-17T05:25:03.142678+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.135989+0000', 'last_change': '2024-09-17T05:25:03.135989+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.135989+0000', 'last_undegraded': '2024-09-17T05:25:03.135989+0000', 'last_fullsized': '2024-09-17T05:25:03.135989+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.135542+0000', 'last_change': '2024-09-17T05:25:03.135542+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.135542+0000', 'last_undegraded': '2024-09-17T05:25:03.135542+0000', 'last_fullsized': '2024-09-17T05:25:03.135542+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.140939+0000', 'last_change': '2024-09-17T05:25:03.140939+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.140939+0000', 'last_undegraded': '2024-09-17T05:25:03.140939+0000', 'last_fullsized': '2024-09-17T05:25:03.140939+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.132975+0000', 'last_change': '2024-09-17T05:25:03.132975+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.132975+0000', 'last_undegraded': '2024-09-17T05:25:03.132975+0000', 'last_fullsized': '2024-09-17T05:25:03.132975+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.143555+0000', 'last_change': '2024-09-17T05:25:03.143555+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.143555+0000', 'last_undegraded': '2024-09-17T05:25:03.143555+0000', 'last_fullsized': '2024-09-17T05:25:03.143555+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:25:09.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.135522+0000', 'last_change': '2024-09-17T05:25:03.135522+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.135522+0000', 'last_undegraded': '2024-09-17T05:25:03.135522+0000', 'last_fullsized': '2024-09-17T05:25:03.135522+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.142221+0000', 'last_change': '2024-09-17T05:25:03.142221+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.142221+0000', 'last_undegraded': '2024-09-17T05:25:03.142221+0000', 'last_fullsized': '2024-09-17T05:25:03.142221+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.134230+0000', 'last_change': '2024-09-17T05:25:03.134230+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.134230+0000', 'last_undegraded': '2024-09-17T05:25:03.134230+0000', 'last_fullsized': '2024-09-17T05:25:03.134230+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.133863+0000', 'last_change': '2024-09-17T05:25:03.133863+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.133863+0000', 'last_undegraded': '2024-09-17T05:25:03.133863+0000', 'last_fullsized': '2024-09-17T05:25:03.133863+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-17T05:25:09.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.131630+0000', 'last_change': '2024-09-17T05:25:03.131630+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.131630+0000', 'last_undegraded': '2024-09-17T05:25:03.131630+0000', 'last_fullsized': '2024-09-17T05:25:03.131630+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139080+0000', 'last_change': '2024-09-17T05:25:03.139080+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139080+0000', 'last_undegraded': '2024-09-17T05:25:03.139080+0000', 'last_fullsized': '2024-09-17T05:25:03.139080+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.138448+0000', 'last_change': '2024-09-17T05:25:03.138448+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.138448+0000', 'last_undegraded': '2024-09-17T05:25:03.138448+0000', 'last_fullsized': '2024-09-17T05:25:03.138448+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.135607+0000', 'last_change': '2024-09-17T05:25:03.135607+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.135607+0000', 'last_undegraded': '2024-09-17T05:25:03.135607+0000', 'last_fullsized': '2024-09-17T05:25:03.135607+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.143119+0000', 'last_change': '2024-09-17T05:25:03.143119+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.143119+0000', 'last_undegraded': '2024-09-17T05:25:03.143119+0000', 'last_fullsized': '2024-09-17T05:25:03.143119+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.139456+0000', 'last_change': '2024-09-17T05:25:03.139456+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.139456+0000', 'last_undegraded': '2024-09-17T05:25:03.139456+0000', 'last_fullsized': '2024-09-17T05:25:03.139456+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-17T05:25:09.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.143495+0000', 'last_change': '2024-09-17T05:25:03.143495+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.143495+0000', 'last_undegraded': '2024-09-17T05:25:03.143495+0000', 'last_fullsized': '2024-09-17T05:25:03.143495+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.146805+0000', 'last_change': '2024-09-17T05:25:03.146805+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.146805+0000', 'last_undegraded': '2024-09-17T05:25:03.146805+0000', 'last_fullsized': '2024-09-17T05:25:03.146805+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.134779+0000', 'last_change': '2024-09-17T05:25:03.134779+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.134779+0000', 'last_undegraded': '2024-09-17T05:25:03.134779+0000', 'last_fullsized': '2024-09-17T05:25:03.134779+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.138642+0000', 'last_change': '2024-09-17T05:25:03.138642+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.138642+0000', 'last_undegraded': '2024-09-17T05:25:03.138642+0000', 'last_fullsized': '2024-09-17T05:25:03.138642+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.133784+0000', 'last_change': '2024-09-17T05:25:03.133784+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.133784+0000', 'last_undegraded': '2024-09-17T05:25:03.133784+0000', 'last_fullsized': '2024-09-17T05:25:03.133784+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.133095+0000', 'last_change': '2024-09-17T05:25:03.133095+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.133095+0000', 'last_undegraded': '2024-09-17T05:25:03.133095+0000', 'last_fullsized': '2024-09-17T05:25:03.133095+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-17T05:25:09.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.135478+0000', 'last_change': '2024-09-17T05:25:03.135478+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.135478+0000', 'last_undegraded': '2024-09-17T05:25:03.135478+0000', 'last_fullsized': '2024-09-17T05:25:03.135478+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:25:09.038 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-17T05:25:09.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:03.145527+0000', 'last_change': '2024-09-17T05:25:03.145527+0000', 'last_active': '2024-09-17T05:25:03.119083+0000', 'last_peered': '2024-09-17T05:25:03.119083+0000', 'last_clean': '2024-09-17T05:25:03.119083+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:03.145527+0000', 'last_undegraded': '2024-09-17T05:25:03.145527+0000', 'last_fullsized': '2024-09-17T05:25:03.145527+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:03.119083+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:25:09.038 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:25:09.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 -- 172.21.15.122:0/489755440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84300b2d00 msgr2=0x7f843015fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 --2- 172.21.15.122:0/489755440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84300b2d00 0x7f843015fec0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f841c009900 tx=0x7f841c01c850 comp rx=0 tx=0).stop 2024-09-17T05:25:09.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 -- 172.21.15.122:0/489755440 shutdown_connections 2024-09-17T05:25:09.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 --2- 172.21.15.122:0/489755440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84300b2d00 0x7f843015fec0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 --2- 172.21.15.122:0/489755440 >> v2:172.21.15.173:3300/0 conn(0x7f84300cfe80 0x7f84300d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 -- 172.21.15.122:0/489755440 >> 172.21.15.122:0/489755440 conn(0x7f84300d1e40 msgr2=0x7f84300d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 -- 172.21.15.122:0/489755440 shutdown_connections 2024-09-17T05:25:09.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 -- 172.21.15.122:0/489755440 wait complete. 2024-09-17T05:25:09.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 Processor -- start 2024-09-17T05:25:09.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.210+0000 7f8434a00640 1 -- start start 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84300b2d00 0x7f843014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84300cfe80 0x7f84300b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8430160400 0x7f84301586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84301695a0 con 0x7f84300b2d00 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8430169420 con 0x7f84300cfe80 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8430169720 con 0x7f8430160400 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84300b2d00 0x7f843014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84300b2d00 0x7f843014fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36776/0 (socket says 172.21.15.122:36776) 2024-09-17T05:25:09.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 -- 172.21.15.122:0/1034908973 learned_addr learned my addr 172.21.15.122:0/1034908973 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:09.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842d400640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84300cfe80 0x7f84300b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842e800640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8430160400 0x7f84301586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 -- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8430160400 msgr2=0x7f84301586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8430160400 0x7f84301586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 -- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84300cfe80 msgr2=0x7f84300b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84300cfe80 0x7f84300b7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 -- 172.21.15.122:0/1034908973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84301501b0 con 0x7f84300b2d00 2024-09-17T05:25:09.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f842de00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84300b2d00 0x7f843014fa40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f842000ec30 tx=0x7f842000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8426000640 1 -- 172.21.15.122:0/1034908973 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420010630 con 0x7f84300b2d00 2024-09-17T05:25:09.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8426000640 1 -- 172.21.15.122:0/1034908973 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84200108c0 con 0x7f84300b2d00 2024-09-17T05:25:09.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8430154be0 con 0x7f84300b2d00 2024-09-17T05:25:09.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8426000640 1 -- 172.21.15.122:0/1034908973 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420010cf0 con 0x7f84300b2d00 2024-09-17T05:25:09.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84301551a0 con 0x7f84300b2d00 2024-09-17T05:25:09.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843015fec0 con 0x7f84300b2d00 2024-09-17T05:25:09.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.214+0000 7f8426000640 1 -- 172.21.15.122:0/1034908973 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8420018420 con 0x7f84300b2d00 2024-09-17T05:25:09.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.218+0000 7f8426000640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8410073200 0x7f84100756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.218+0000 7f8426000640 1 -- 172.21.15.122:0/1034908973 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8420094ff0 con 0x7f84300b2d00 2024-09-17T05:25:09.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.218+0000 7f842d400640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8410073200 0x7f84100756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.218+0000 7f842d400640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8410073200 0x7f84100756c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f841c0af000 tx=0x7f841c01b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.222+0000 7f8426000640 1 -- 172.21.15.122:0/1034908973 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f84200600b0 con 0x7f84300b2d00 2024-09-17T05:25:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.446+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8430159db0 con 0x7f84300b2d00 2024-09-17T05:25:09.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.446+0000 7f8426000640 1 -- 172.21.15.122:0/1034908973 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1137 (secure 0 0 0) 0x7f842005f9b0 con 0x7f84300b2d00 2024-09-17T05:25:09.450 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:09.450 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":30,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550693,"num_in_osds":8,"osd_in_since":1726550682,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":52},{"state_name":"active+clean","count":45}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590438,"bytes_used":228438016,"bytes_avail":772865675264,"bytes_total":773094113280,"inactive_pgs_ratio":0.53608244657516479,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4332}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-17T05:24:39.151595+0000","services":{}},"progress_events":{}} 2024-09-17T05:25:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8410073200 msgr2=0x7f84100756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8410073200 0x7f84100756c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f841c0af000 tx=0x7f841c01b090 comp rx=0 tx=0).stop 2024-09-17T05:25:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84300b2d00 msgr2=0x7f843014fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84300b2d00 0x7f843014fa40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f842000ec30 tx=0x7f842000c6a0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 shutdown_connections 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8410073200 0x7f84100756c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8430160400 0x7f84301586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84300cfe80 0x7f84300b7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 --2- 172.21.15.122:0/1034908973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84300b2d00 0x7f843014fa40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 >> 172.21.15.122:0/1034908973 conn(0x7f84300d1e40 msgr2=0x7f84300cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 shutdown_connections 2024-09-17T05:25:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.450+0000 7f8434a00640 1 -- 172.21.15.122:0/1034908973 wait complete. 2024-09-17T05:25:09.472 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:25:09.473 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.610+0000 7f3206200640 1 -- 172.21.15.122:0/2703277745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3200073070 msgr2=0x7f3200073450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.610+0000 7f3206200640 1 --2- 172.21.15.122:0/2703277745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3200073070 0x7f3200073450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31f000a950 tx=0x7f31f001da20 comp rx=0 tx=0).stop 2024-09-17T05:25:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.610+0000 7f3206200640 1 -- 172.21.15.122:0/2703277745 shutdown_connections 2024-09-17T05:25:09.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.610+0000 7f3206200640 1 --2- 172.21.15.122:0/2703277745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3200073070 0x7f3200073450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.610+0000 7f3206200640 1 -- 172.21.15.122:0/2703277745 >> 172.21.15.122:0/2703277745 conn(0x7f32000ccf10 msgr2=0x7f32000cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.610+0000 7f3206200640 1 -- 172.21.15.122:0/2703277745 shutdown_connections 2024-09-17T05:25:09.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.610+0000 7f3206200640 1 -- 172.21.15.122:0/2703277745 wait complete. 2024-09-17T05:25:09.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 Processor -- start 2024-09-17T05:25:09.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 -- start start 2024-09-17T05:25:09.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200073070 0x7f32001fc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32000cacf0 0x7f32000c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32000cbbd0 0x7f32000c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f320006fb40 con 0x7f32000cacf0 2024-09-17T05:25:09.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f320006f9c0 con 0x7f3200073070 2024-09-17T05:25:09.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f32001686c0 con 0x7f32000cbbd0 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32000cacf0 0x7f32000c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32000cacf0 0x7f32000c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36786/0 (socket says 172.21.15.122:36786) 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 -- 172.21.15.122:0/1790520486 learned_addr learned my addr 172.21.15.122:0/1790520486 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31ff400640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200073070 0x7f32001fc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31ffe00640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32000cbbd0 0x7f32000c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 -- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32000cbbd0 msgr2=0x7f32000c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32000cbbd0 0x7f32000c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 -- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200073070 msgr2=0x7f32001fc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200073070 0x7f32001fc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 -- 172.21.15.122:0/1790520486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32001f8730 con 0x7f32000cacf0 2024-09-17T05:25:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31ffe00640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32000cbbd0 0x7f32000c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:09.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f6a00640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32000cacf0 0x7f32000c3fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f31e800c7f0 tx=0x7f31e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f7e00640 1 -- 172.21.15.122:0/1790520486 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8017070 con 0x7f32000cacf0 2024-09-17T05:25:09.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f7e00640 1 -- 172.21.15.122:0/1790520486 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31e8005070 con 0x7f32000cacf0 2024-09-17T05:25:09.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f31f7e00640 1 -- 172.21.15.122:0/1790520486 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e80054c0 con 0x7f32000cacf0 2024-09-17T05:25:09.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32001f8a10 con 0x7f32000cacf0 2024-09-17T05:25:09.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32001f8fd0 con 0x7f32000cacf0 2024-09-17T05:25:09.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.614+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3200073450 con 0x7f32000cacf0 2024-09-17T05:25:09.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.618+0000 7f31f7e00640 1 -- 172.21.15.122:0/1790520486 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31e801a070 con 0x7f32000cacf0 2024-09-17T05:25:09.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.618+0000 7f31f7e00640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31dc073200 0x7f31dc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:09.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.618+0000 7f31f7e00640 1 -- 172.21.15.122:0/1790520486 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f31e80938a0 con 0x7f32000cacf0 2024-09-17T05:25:09.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.618+0000 7f31ff400640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31dc073200 0x7f31dc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.618+0000 7f31ff400640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31dc073200 0x7f31dc0756c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f31f00b0000 tx=0x7f31f001c260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:09.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.622+0000 7f31f7e00640 1 -- 172.21.15.122:0/1790520486 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f31e805e260 con 0x7f32000cacf0 2024-09-17T05:25:09.736 INFO:tasks.ceph.mds.d.smithi173.stderr:2024-09-17T05:25:09.735+0000 7f18e8800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:09.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.790+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32000b1ba0 con 0x7f31dc073200 2024-09-17T05:25:09.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.794+0000 7f31f7e00640 1 -- 172.21.15.122:0/1790520486 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247635 (secure 0 0 0) 0x7f31f00fa770 con 0x7f31dc073200 2024-09-17T05:25:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:09.801 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:09.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31dc073200 msgr2=0x7f31dc0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31dc073200 0x7f31dc0756c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f31f00b0000 tx=0x7f31f001c260 comp rx=0 tx=0).stop 2024-09-17T05:25:09.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32000cacf0 msgr2=0x7f32000c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:09.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32000cacf0 0x7f32000c3fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f31e800c7f0 tx=0x7f31e800ccb0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 shutdown_connections 2024-09-17T05:25:09.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31dc073200 0x7f31dc0756c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32000cbbd0 0x7f32000c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32000cacf0 0x7f32000c3fc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 --2- 172.21.15.122:0/1790520486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200073070 0x7f32001fc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:09.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 >> 172.21.15.122:0/1790520486 conn(0x7f32000ccf10 msgr2=0x7f3200070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:09.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 shutdown_connections 2024-09-17T05:25:09.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:09.798+0000 7f3206200640 1 -- 172.21.15.122:0/1790520486 wait complete. 2024-09-17T05:25:09.818 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T05:25:08.442331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223084,"kb_used_data":2680,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751636,"statfs":{"total":773094113280,"available":772865675264,"internally_reserved":0,"allocated":2744320,"data_stored":1674475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001365"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139696+0000","last_change":"2024-09-17T05:25:03.139696+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139696+0000","last_undegraded":"2024-09-17T05:25:03.139696+0000","last_fullsized":"2024-09-17T05:25:03.139696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324353+0000","last_change":"2024-09-17T05:25:04.173588+0000","last_active":"2024-09-17T05:25:05.324353+0000","last_peered":"2024-09-17T05:25:05.324353+0000","last_clean":"2024-09-17T05:25:05.324353+0000","last_became_active":"2024-09-17T05:25:04.173260+0000","last_became_peered":"2024-09-17T05:25:04.173260+0000","last_unstale":"2024-09-17T05:25:05.324353+0000","last_undegraded":"2024-09-17T05:25:05.324353+0000","last_fullsized":"2024-09-17T05:25:05.324353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:13.477698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324934+0000","last_change":"2024-09-17T05:25:04.174132+0000","last_active":"2024-09-17T05:25:05.324934+0000","last_peered":"2024-09-17T05:25:05.324934+0000","last_clean":"2024-09-17T05:25:05.324934+0000","last_became_active":"2024-09-17T05:25:04.173337+0000","last_became_peered":"2024-09-17T05:25:04.173337+0000","last_unstale":"2024-09-17T05:25:05.324934+0000","last_undegraded":"2024-09-17T05:25:05.324934+0000","last_fullsized":"2024-09-17T05:25:05.324934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:50.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141593+0000","last_change":"2024-09-17T05:25:03.141593+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141593+0000","last_undegraded":"2024-09-17T05:25:03.141593+0000","last_fullsized":"2024-09-17T05:25:03.141593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139805+0000","last_change":"2024-09-17T05:25:03.139805+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139805+0000","last_undegraded":"2024-09-17T05:25:03.139805+0000","last_fullsized":"2024-09-17T05:25:03.139805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139985+0000","last_change":"2024-09-17T05:25:03.139985+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139985+0000","last_undegraded":"2024-09-17T05:25:03.139985+0000","last_fullsized":"2024-09-17T05:25:03.139985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.137568+0000","last_change":"2024-09-17T05:25:03.137568+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.137568+0000","last_undegraded":"2024-09-17T05:25:03.137568+0000","last_fullsized":"2024-09-17T05:25:03.137568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324125+0000","last_change":"2024-09-17T05:25:04.166345+0000","last_active":"2024-09-17T05:25:05.324125+0000","last_peered":"2024-09-17T05:25:05.324125+0000","last_clean":"2024-09-17T05:25:05.324125+0000","last_became_active":"2024-09-17T05:25:04.165795+0000","last_became_peered":"2024-09-17T05:25:04.165795+0000","last_unstale":"2024-09-17T05:25:05.324125+0000","last_undegraded":"2024-09-17T05:25:05.324125+0000","last_fullsized":"2024-09-17T05:25:05.324125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:33.426170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141669+0000","last_change":"2024-09-17T05:25:03.141669+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141669+0000","last_undegraded":"2024-09-17T05:25:03.141669+0000","last_fullsized":"2024-09-17T05:25:03.141669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136500+0000","last_change":"2024-09-17T05:25:03.136500+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136500+0000","last_undegraded":"2024-09-17T05:25:03.136500+0000","last_fullsized":"2024-09-17T05:25:03.136500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139381+0000","last_change":"2024-09-17T05:25:03.139381+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139381+0000","last_undegraded":"2024-09-17T05:25:03.139381+0000","last_fullsized":"2024-09-17T05:25:03.139381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136937+0000","last_change":"2024-09-17T05:25:03.136937+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136937+0000","last_undegraded":"2024-09-17T05:25:03.136937+0000","last_fullsized":"2024-09-17T05:25:03.136937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138718+0000","last_change":"2024-09-17T05:25:03.138718+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138718+0000","last_undegraded":"2024-09-17T05:25:03.138718+0000","last_fullsized":"2024-09-17T05:25:03.138718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141990+0000","last_change":"2024-09-17T05:25:03.141990+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141990+0000","last_undegraded":"2024-09-17T05:25:03.141990+0000","last_fullsized":"2024-09-17T05:25:03.141990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324904+0000","last_change":"2024-09-17T05:25:04.163764+0000","last_active":"2024-09-17T05:25:05.324904+0000","last_peered":"2024-09-17T05:25:05.324904+0000","last_clean":"2024-09-17T05:25:05.324904+0000","last_became_active":"2024-09-17T05:25:04.163197+0000","last_became_peered":"2024-09-17T05:25:04.163197+0000","last_unstale":"2024-09-17T05:25:05.324904+0000","last_undegraded":"2024-09-17T05:25:05.324904+0000","last_fullsized":"2024-09-17T05:25:05.324904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:01:59.441556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139885+0000","last_change":"2024-09-17T05:25:03.139885+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139885+0000","last_undegraded":"2024-09-17T05:25:03.139885+0000","last_fullsized":"2024-09-17T05:25:03.139885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139387+0000","last_change":"2024-09-17T05:25:03.139387+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139387+0000","last_undegraded":"2024-09-17T05:25:03.139387+0000","last_fullsized":"2024-09-17T05:25:03.139387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133893+0000","last_change":"2024-09-17T05:25:03.133893+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133893+0000","last_undegraded":"2024-09-17T05:25:03.133893+0000","last_fullsized":"2024-09-17T05:25:03.133893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138629+0000","last_change":"2024-09-17T05:25:03.138629+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138629+0000","last_undegraded":"2024-09-17T05:25:03.138629+0000","last_fullsized":"2024-09-17T05:25:03.138629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136431+0000","last_change":"2024-09-17T05:25:03.136431+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136431+0000","last_undegraded":"2024-09-17T05:25:03.136431+0000","last_fullsized":"2024-09-17T05:25:03.136431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143548+0000","last_change":"2024-09-17T05:25:03.143548+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143548+0000","last_undegraded":"2024-09-17T05:25:03.143548+0000","last_fullsized":"2024-09-17T05:25:03.143548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143441+0000","last_change":"2024-09-17T05:25:03.143441+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143441+0000","last_undegraded":"2024-09-17T05:25:03.143441+0000","last_fullsized":"2024-09-17T05:25:03.143441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325148+0000","last_change":"2024-09-17T05:25:04.158936+0000","last_active":"2024-09-17T05:25:05.325148+0000","last_peered":"2024-09-17T05:25:05.325148+0000","last_clean":"2024-09-17T05:25:05.325148+0000","last_became_active":"2024-09-17T05:25:04.158697+0000","last_became_peered":"2024-09-17T05:25:04.158697+0000","last_unstale":"2024-09-17T05:25:05.325148+0000","last_undegraded":"2024-09-17T05:25:05.325148+0000","last_fullsized":"2024-09-17T05:25:05.325148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:45:18.794538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138842+0000","last_change":"2024-09-17T05:25:03.138842+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138842+0000","last_undegraded":"2024-09-17T05:25:03.138842+0000","last_fullsized":"2024-09-17T05:25:03.138842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324281+0000","last_change":"2024-09-17T05:25:04.165631+0000","last_active":"2024-09-17T05:25:05.324281+0000","last_peered":"2024-09-17T05:25:05.324281+0000","last_clean":"2024-09-17T05:25:05.324281+0000","last_became_active":"2024-09-17T05:25:04.163086+0000","last_became_peered":"2024-09-17T05:25:04.163086+0000","last_unstale":"2024-09-17T05:25:05.324281+0000","last_undegraded":"2024-09-17T05:25:05.324281+0000","last_fullsized":"2024-09-17T05:25:05.324281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:59.610017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143104+0000","last_change":"2024-09-17T05:25:03.143104+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143104+0000","last_undegraded":"2024-09-17T05:25:03.143104+0000","last_fullsized":"2024-09-17T05:25:03.143104+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139789+0000","last_change":"2024-09-17T05:25:03.139789+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139789+0000","last_undegraded":"2024-09-17T05:25:03.139789+0000","last_fullsized":"2024-09-17T05:25:03.139789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141592+0000","last_change":"2024-09-17T05:25:03.141592+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141592+0000","last_undegraded":"2024-09-17T05:25:03.141592+0000","last_fullsized":"2024-09-17T05:25:03.141592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325254+0000","last_change":"2024-09-17T05:25:04.162109+0000","last_active":"2024-09-17T05:25:05.325254+0000","last_peered":"2024-09-17T05:25:05.325254+0000","last_clean":"2024-09-17T05:25:05.325254+0000","last_became_active":"2024-09-17T05:25:04.161807+0000","last_became_peered":"2024-09-17T05:25:04.161807+0000","last_unstale":"2024-09-17T05:25:05.325254+0000","last_undegraded":"2024-09-17T05:25:05.325254+0000","last_fullsized":"2024-09-17T05:25:05.325254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:23.271600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.136665+0000","last_change":"2024-09-17T05:25:03.136665+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.136665+0000","last_undegraded":"2024-09-17T05:25:03.136665+0000","last_fullsized":"2024-09-17T05:25:03.136665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133784+0000","last_change":"2024-09-17T05:25:03.133784+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133784+0000","last_undegraded":"2024-09-17T05:25:03.133784+0000","last_fullsized":"2024-09-17T05:25:03.133784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141020+0000","last_change":"2024-09-17T05:25:03.141020+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141020+0000","last_undegraded":"2024-09-17T05:25:03.141020+0000","last_fullsized":"2024-09-17T05:25:03.141020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.141848+0000","last_change":"2024-09-17T05:25:03.141848+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.141848+0000","last_undegraded":"2024-09-17T05:25:03.141848+0000","last_fullsized":"2024-09-17T05:25:03.141848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325321+0000","last_change":"2024-09-17T05:25:03.160754+0000","last_active":"2024-09-17T05:25:05.325321+0000","last_peered":"2024-09-17T05:25:05.325321+0000","last_clean":"2024-09-17T05:25:05.325321+0000","last_became_active":"2024-09-17T05:25:03.160433+0000","last_became_peered":"2024-09-17T05:25:03.160433+0000","last_unstale":"2024-09-17T05:25:05.325321+0000","last_undegraded":"2024-09-17T05:25:05.325321+0000","last_fullsized":"2024-09-17T05:25:05.325321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:59.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.140257+0000","last_change":"2024-09-17T05:25:03.140257+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.140257+0000","last_undegraded":"2024-09-17T05:25:03.140257+0000","last_fullsized":"2024-09-17T05:25:03.140257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.146360+0000","last_change":"2024-09-17T05:25:03.138686+0000","last_active":"2024-09-17T05:25:03.146360+0000","last_peered":"2024-09-17T05:25:03.146360+0000","last_clean":"2024-09-17T05:25:03.146360+0000","last_became_active":"2024-09-17T05:25:03.138321+0000","last_became_peered":"2024-09-17T05:25:03.138321+0000","last_unstale":"2024-09-17T05:25:03.146360+0000","last_undegraded":"2024-09-17T05:25:03.146360+0000","last_fullsized":"2024-09-17T05:25:03.146360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.165802+0000","last_change":"2024-09-17T05:25:03.161161+0000","last_active":"2024-09-17T05:25:03.165802+0000","last_peered":"2024-09-17T05:25:03.165802+0000","last_clean":"2024-09-17T05:25:03.165802+0000","last_became_active":"2024-09-17T05:25:03.160876+0000","last_became_peered":"2024-09-17T05:25:03.160876+0000","last_unstale":"2024-09-17T05:25:03.165802+0000","last_undegraded":"2024-09-17T05:25:03.165802+0000","last_fullsized":"2024-09-17T05:25:03.165802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324408+0000","last_change":"2024-09-17T05:25:04.173583+0000","last_active":"2024-09-17T05:25:05.324408+0000","last_peered":"2024-09-17T05:25:05.324408+0000","last_clean":"2024-09-17T05:25:05.324408+0000","last_became_active":"2024-09-17T05:25:04.173276+0000","last_became_peered":"2024-09-17T05:25:04.173276+0000","last_unstale":"2024-09-17T05:25:05.324408+0000","last_undegraded":"2024-09-17T05:25:05.324408+0000","last_fullsized":"2024-09-17T05:25:05.324408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:38.290022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.158113+0000","last_change":"2024-09-17T05:25:03.156886+0000","last_active":"2024-09-17T05:25:03.158113+0000","last_peered":"2024-09-17T05:25:03.158113+0000","last_clean":"2024-09-17T05:25:03.158113+0000","last_became_active":"2024-09-17T05:25:03.156595+0000","last_became_peered":"2024-09-17T05:25:03.156595+0000","last_unstale":"2024-09-17T05:25:03.158113+0000","last_undegraded":"2024-09-17T05:25:03.158113+0000","last_fullsized":"2024-09-17T05:25:03.158113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:44.043946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139049+0000","last_change":"2024-09-17T05:25:03.139049+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139049+0000","last_undegraded":"2024-09-17T05:25:03.139049+0000","last_fullsized":"2024-09-17T05:25:03.139049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163284+0000","last_change":"2024-09-17T05:25:03.160352+0000","last_active":"2024-09-17T05:25:03.163284+0000","last_peered":"2024-09-17T05:25:03.163284+0000","last_clean":"2024-09-17T05:25:03.163284+0000","last_became_active":"2024-09-17T05:25:03.160030+0000","last_became_peered":"2024-09-17T05:25:03.160030+0000","last_unstale":"2024-09-17T05:25:03.163284+0000","last_undegraded":"2024-09-17T05:25:03.163284+0000","last_fullsized":"2024-09-17T05:25:03.163284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:45.753119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.142678+0000","last_change":"2024-09-17T05:25:03.142678+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.142678+0000","last_undegraded":"2024-09-17T05:25:03.142678+0000","last_fullsized":"2024-09-17T05:25:03.142678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159013+0000","last_change":"2024-09-17T05:25:03.155547+0000","last_active":"2024-09-17T05:25:03.159013+0000","last_peered":"2024-09-17T05:25:03.159013+0000","last_clean":"2024-09-17T05:25:03.159013+0000","last_became_active":"2024-09-17T05:25:03.155270+0000","last_became_peered":"2024-09-17T05:25:03.155270+0000","last_unstale":"2024-09-17T05:25:03.159013+0000","last_undegraded":"2024-09-17T05:25:03.159013+0000","last_fullsized":"2024-09-17T05:25:03.159013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324064+0000","last_change":"2024-09-17T05:25:04.163379+0000","last_active":"2024-09-17T05:25:05.324064+0000","last_peered":"2024-09-17T05:25:05.324064+0000","last_clean":"2024-09-17T05:25:05.324064+0000","last_became_active":"2024-09-17T05:25:04.163066+0000","last_became_peered":"2024-09-17T05:25:04.163066+0000","last_unstale":"2024-09-17T05:25:05.324064+0000","last_undegraded":"2024-09-17T05:25:05.324064+0000","last_fullsized":"2024-09-17T05:25:05.324064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:43.294053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157636+0000","last_change":"2024-09-17T05:25:03.154476+0000","last_active":"2024-09-17T05:25:03.157636+0000","last_peered":"2024-09-17T05:25:03.157636+0000","last_clean":"2024-09-17T05:25:03.157636+0000","last_became_active":"2024-09-17T05:25:03.153306+0000","last_became_peered":"2024-09-17T05:25:03.153306+0000","last_unstale":"2024-09-17T05:25:03.157636+0000","last_undegraded":"2024-09-17T05:25:03.157636+0000","last_fullsized":"2024-09-17T05:25:03.157636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135989+0000","last_change":"2024-09-17T05:25:03.135989+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135989+0000","last_undegraded":"2024-09-17T05:25:03.135989+0000","last_fullsized":"2024-09-17T05:25:03.135989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156191+0000","last_change":"2024-09-17T05:25:03.152426+0000","last_active":"2024-09-17T05:25:03.156191+0000","last_peered":"2024-09-17T05:25:03.156191+0000","last_clean":"2024-09-17T05:25:03.156191+0000","last_became_active":"2024-09-17T05:25:03.152129+0000","last_became_peered":"2024-09-17T05:25:03.152129+0000","last_unstale":"2024-09-17T05:25:03.156191+0000","last_undegraded":"2024-09-17T05:25:03.156191+0000","last_fullsized":"2024-09-17T05:25:03.156191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135542+0000","last_change":"2024-09-17T05:25:03.135542+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135542+0000","last_undegraded":"2024-09-17T05:25:03.135542+0000","last_fullsized":"2024-09-17T05:25:03.135542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.160537+0000","last_change":"2024-09-17T05:25:03.156640+0000","last_active":"2024-09-17T05:25:03.160537+0000","last_peered":"2024-09-17T05:25:03.160537+0000","last_clean":"2024-09-17T05:25:03.160537+0000","last_became_active":"2024-09-17T05:25:03.156338+0000","last_became_peered":"2024-09-17T05:25:03.156338+0000","last_unstale":"2024-09-17T05:25:03.160537+0000","last_undegraded":"2024-09-17T05:25:03.160537+0000","last_fullsized":"2024-09-17T05:25:03.160537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.140939+0000","last_change":"2024-09-17T05:25:03.140939+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.140939+0000","last_undegraded":"2024-09-17T05:25:03.140939+0000","last_fullsized":"2024-09-17T05:25:03.140939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.138842+0000","last_change":"2024-09-17T05:25:03.136217+0000","last_active":"2024-09-17T05:25:03.138842+0000","last_peered":"2024-09-17T05:25:03.138842+0000","last_clean":"2024-09-17T05:25:03.138842+0000","last_became_active":"2024-09-17T05:25:03.135857+0000","last_became_peered":"2024-09-17T05:25:03.135857+0000","last_unstale":"2024-09-17T05:25:03.138842+0000","last_undegraded":"2024-09-17T05:25:03.138842+0000","last_fullsized":"2024-09-17T05:25:03.138842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.132975+0000","last_change":"2024-09-17T05:25:03.132975+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.132975+0000","last_undegraded":"2024-09-17T05:25:03.132975+0000","last_fullsized":"2024-09-17T05:25:03.132975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.153179+0000","last_change":"2024-09-17T05:25:03.147144+0000","last_active":"2024-09-17T05:25:03.153179+0000","last_peered":"2024-09-17T05:25:03.153179+0000","last_clean":"2024-09-17T05:25:03.153179+0000","last_became_active":"2024-09-17T05:25:03.146907+0000","last_became_peered":"2024-09-17T05:25:03.146907+0000","last_unstale":"2024-09-17T05:25:03.153179+0000","last_undegraded":"2024-09-17T05:25:03.153179+0000","last_fullsized":"2024-09-17T05:25:03.153179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143555+0000","last_change":"2024-09-17T05:25:03.143555+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143555+0000","last_undegraded":"2024-09-17T05:25:03.143555+0000","last_fullsized":"2024-09-17T05:25:03.143555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156107+0000","last_change":"2024-09-17T05:25:03.151397+0000","last_active":"2024-09-17T05:25:03.156107+0000","last_peered":"2024-09-17T05:25:03.156107+0000","last_clean":"2024-09-17T05:25:03.156107+0000","last_became_active":"2024-09-17T05:25:03.151076+0000","last_became_peered":"2024-09-17T05:25:03.151076+0000","last_unstale":"2024-09-17T05:25:03.156107+0000","last_undegraded":"2024-09-17T05:25:03.156107+0000","last_fullsized":"2024-09-17T05:25:03.156107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135522+0000","last_change":"2024-09-17T05:25:03.135522+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135522+0000","last_undegraded":"2024-09-17T05:25:03.135522+0000","last_fullsized":"2024-09-17T05:25:03.135522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163289+0000","last_change":"2024-09-17T05:25:03.159880+0000","last_active":"2024-09-17T05:25:03.163289+0000","last_peered":"2024-09-17T05:25:03.163289+0000","last_clean":"2024-09-17T05:25:03.163289+0000","last_became_active":"2024-09-17T05:25:03.159636+0000","last_became_peered":"2024-09-17T05:25:03.159636+0000","last_unstale":"2024-09-17T05:25:03.163289+0000","last_undegraded":"2024-09-17T05:25:03.163289+0000","last_fullsized":"2024-09-17T05:25:03.163289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.142221+0000","last_change":"2024-09-17T05:25:03.142221+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.142221+0000","last_undegraded":"2024-09-17T05:25:03.142221+0000","last_fullsized":"2024-09-17T05:25:03.142221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.165529+0000","last_change":"2024-09-17T05:25:03.161164+0000","last_active":"2024-09-17T05:25:03.165529+0000","last_peered":"2024-09-17T05:25:03.165529+0000","last_clean":"2024-09-17T05:25:03.165529+0000","last_became_active":"2024-09-17T05:25:03.160875+0000","last_became_peered":"2024-09-17T05:25:03.160875+0000","last_unstale":"2024-09-17T05:25:03.165529+0000","last_undegraded":"2024-09-17T05:25:03.165529+0000","last_fullsized":"2024-09-17T05:25:03.165529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.134230+0000","last_change":"2024-09-17T05:25:03.134230+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.134230+0000","last_undegraded":"2024-09-17T05:25:03.134230+0000","last_fullsized":"2024-09-17T05:25:03.134230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323888+0000","last_change":"2024-09-17T05:25:03.160173+0000","last_active":"2024-09-17T05:25:05.323888+0000","last_peered":"2024-09-17T05:25:05.323888+0000","last_clean":"2024-09-17T05:25:05.323888+0000","last_became_active":"2024-09-17T05:25:03.159759+0000","last_became_peered":"2024-09-17T05:25:03.159759+0000","last_unstale":"2024-09-17T05:25:05.323888+0000","last_undegraded":"2024-09-17T05:25:05.323888+0000","last_fullsized":"2024-09-17T05:25:05.323888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:06.043431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133863+0000","last_change":"2024-09-17T05:25:03.133863+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133863+0000","last_undegraded":"2024-09-17T05:25:03.133863+0000","last_fullsized":"2024-09-17T05:25:03.133863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.155834+0000","last_change":"2024-09-17T05:25:03.151849+0000","last_active":"2024-09-17T05:25:03.155834+0000","last_peered":"2024-09-17T05:25:03.155834+0000","last_clean":"2024-09-17T05:25:03.155834+0000","last_became_active":"2024-09-17T05:25:03.151518+0000","last_became_peered":"2024-09-17T05:25:03.151518+0000","last_unstale":"2024-09-17T05:25:03.155834+0000","last_undegraded":"2024-09-17T05:25:03.155834+0000","last_fullsized":"2024-09-17T05:25:03.155834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.874644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.131630+0000","last_change":"2024-09-17T05:25:03.131630+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.131630+0000","last_undegraded":"2024-09-17T05:25:03.131630+0000","last_fullsized":"2024-09-17T05:25:03.131630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":107,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.145884+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:03.145884+0000","last_peered":"2024-09-17T05:25:03.145884+0000","last_clean":"2024-09-17T05:25:03.145884+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:03.145884+0000","last_undegraded":"2024-09-17T05:25:03.145884+0000","last_fullsized":"2024-09-17T05:25:03.145884+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157330+0000","last_change":"2024-09-17T05:25:03.154924+0000","last_active":"2024-09-17T05:25:03.157330+0000","last_peered":"2024-09-17T05:25:03.157330+0000","last_clean":"2024-09-17T05:25:03.157330+0000","last_became_active":"2024-09-17T05:25:03.154274+0000","last_became_peered":"2024-09-17T05:25:03.154274+0000","last_unstale":"2024-09-17T05:25:03.157330+0000","last_undegraded":"2024-09-17T05:25:03.157330+0000","last_fullsized":"2024-09-17T05:25:03.157330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:40.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139080+0000","last_change":"2024-09-17T05:25:03.139080+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139080+0000","last_undegraded":"2024-09-17T05:25:03.139080+0000","last_fullsized":"2024-09-17T05:25:03.139080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.156480+0000","last_change":"2024-09-17T05:25:03.154006+0000","last_active":"2024-09-17T05:25:03.156480+0000","last_peered":"2024-09-17T05:25:03.156480+0000","last_clean":"2024-09-17T05:25:03.156480+0000","last_became_active":"2024-09-17T05:25:03.153578+0000","last_became_peered":"2024-09-17T05:25:03.153578+0000","last_unstale":"2024-09-17T05:25:03.156480+0000","last_undegraded":"2024-09-17T05:25:03.156480+0000","last_fullsized":"2024-09-17T05:25:03.156480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138448+0000","last_change":"2024-09-17T05:25:03.138448+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138448+0000","last_undegraded":"2024-09-17T05:25:03.138448+0000","last_fullsized":"2024-09-17T05:25:03.138448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.160481+0000","last_change":"2024-09-17T05:25:03.154866+0000","last_active":"2024-09-17T05:25:03.160481+0000","last_peered":"2024-09-17T05:25:03.160481+0000","last_clean":"2024-09-17T05:25:03.160481+0000","last_became_active":"2024-09-17T05:25:03.154116+0000","last_became_peered":"2024-09-17T05:25:03.154116+0000","last_unstale":"2024-09-17T05:25:03.160481+0000","last_undegraded":"2024-09-17T05:25:03.160481+0000","last_fullsized":"2024-09-17T05:25:03.160481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324275+0000","last_change":"2024-09-17T05:25:04.162175+0000","last_active":"2024-09-17T05:25:05.324275+0000","last_peered":"2024-09-17T05:25:05.324275+0000","last_clean":"2024-09-17T05:25:05.324275+0000","last_became_active":"2024-09-17T05:25:04.161820+0000","last_became_peered":"2024-09-17T05:25:04.161820+0000","last_unstale":"2024-09-17T05:25:05.324275+0000","last_undegraded":"2024-09-17T05:25:05.324275+0000","last_fullsized":"2024-09-17T05:25:05.324275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:08.628783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152473+0000","last_change":"2024-09-17T05:25:03.144930+0000","last_active":"2024-09-17T05:25:03.152473+0000","last_peered":"2024-09-17T05:25:03.152473+0000","last_clean":"2024-09-17T05:25:03.152473+0000","last_became_active":"2024-09-17T05:25:03.144665+0000","last_became_peered":"2024-09-17T05:25:03.144665+0000","last_unstale":"2024-09-17T05:25:03.152473+0000","last_undegraded":"2024-09-17T05:25:03.152473+0000","last_fullsized":"2024-09-17T05:25:03.152473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135607+0000","last_change":"2024-09-17T05:25:03.135607+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135607+0000","last_undegraded":"2024-09-17T05:25:03.135607+0000","last_fullsized":"2024-09-17T05:25:03.135607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323728+0000","last_change":"2024-09-17T05:25:03.160065+0000","last_active":"2024-09-17T05:25:05.323728+0000","last_peered":"2024-09-17T05:25:05.323728+0000","last_clean":"2024-09-17T05:25:05.323728+0000","last_became_active":"2024-09-17T05:25:03.159684+0000","last_became_peered":"2024-09-17T05:25:03.159684+0000","last_unstale":"2024-09-17T05:25:05.323728+0000","last_undegraded":"2024-09-17T05:25:05.323728+0000","last_fullsized":"2024-09-17T05:25:05.323728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:04.387303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143119+0000","last_change":"2024-09-17T05:25:03.143119+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143119+0000","last_undegraded":"2024-09-17T05:25:03.143119+0000","last_fullsized":"2024-09-17T05:25:03.143119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152911+0000","last_change":"2024-09-17T05:25:03.150695+0000","last_active":"2024-09-17T05:25:03.152911+0000","last_peered":"2024-09-17T05:25:03.152911+0000","last_clean":"2024-09-17T05:25:03.152911+0000","last_became_active":"2024-09-17T05:25:03.150452+0000","last_became_peered":"2024-09-17T05:25:03.150452+0000","last_unstale":"2024-09-17T05:25:03.152911+0000","last_undegraded":"2024-09-17T05:25:03.152911+0000","last_fullsized":"2024-09-17T05:25:03.152911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.139456+0000","last_change":"2024-09-17T05:25:03.139456+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.139456+0000","last_undegraded":"2024-09-17T05:25:03.139456+0000","last_fullsized":"2024-09-17T05:25:03.139456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.157738+0000","last_change":"2024-09-17T05:25:03.153186+0000","last_active":"2024-09-17T05:25:03.157738+0000","last_peered":"2024-09-17T05:25:03.157738+0000","last_clean":"2024-09-17T05:25:03.157738+0000","last_became_active":"2024-09-17T05:25:03.152579+0000","last_became_peered":"2024-09-17T05:25:03.152579+0000","last_unstale":"2024-09-17T05:25:03.157738+0000","last_undegraded":"2024-09-17T05:25:03.157738+0000","last_fullsized":"2024-09-17T05:25:03.157738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.143495+0000","last_change":"2024-09-17T05:25:03.143495+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.143495+0000","last_undegraded":"2024-09-17T05:25:03.143495+0000","last_fullsized":"2024-09-17T05:25:03.143495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.152271+0000","last_change":"2024-09-17T05:25:03.144098+0000","last_active":"2024-09-17T05:25:03.152271+0000","last_peered":"2024-09-17T05:25:03.152271+0000","last_clean":"2024-09-17T05:25:03.152271+0000","last_became_active":"2024-09-17T05:25:03.137827+0000","last_became_peered":"2024-09-17T05:25:03.137827+0000","last_unstale":"2024-09-17T05:25:03.152271+0000","last_undegraded":"2024-09-17T05:25:03.152271+0000","last_fullsized":"2024-09-17T05:25:03.152271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.146805+0000","last_change":"2024-09-17T05:25:03.146805+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.146805+0000","last_undegraded":"2024-09-17T05:25:03.146805+0000","last_fullsized":"2024-09-17T05:25:03.146805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159052+0000","last_change":"2024-09-17T05:25:03.155817+0000","last_active":"2024-09-17T05:25:03.159052+0000","last_peered":"2024-09-17T05:25:03.159052+0000","last_clean":"2024-09-17T05:25:03.159052+0000","last_became_active":"2024-09-17T05:25:03.155300+0000","last_became_peered":"2024-09-17T05:25:03.155300+0000","last_unstale":"2024-09-17T05:25:03.159052+0000","last_undegraded":"2024-09-17T05:25:03.159052+0000","last_fullsized":"2024-09-17T05:25:03.159052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.134779+0000","last_change":"2024-09-17T05:25:03.134779+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.134779+0000","last_undegraded":"2024-09-17T05:25:03.134779+0000","last_fullsized":"2024-09-17T05:25:03.134779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.161122+0000","last_change":"2024-09-17T05:25:03.155466+0000","last_active":"2024-09-17T05:25:03.161122+0000","last_peered":"2024-09-17T05:25:03.161122+0000","last_clean":"2024-09-17T05:25:03.161122+0000","last_became_active":"2024-09-17T05:25:03.153103+0000","last_became_peered":"2024-09-17T05:25:03.153103+0000","last_unstale":"2024-09-17T05:25:03.161122+0000","last_undegraded":"2024-09-17T05:25:03.161122+0000","last_fullsized":"2024-09-17T05:25:03.161122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:31.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.138642+0000","last_change":"2024-09-17T05:25:03.138642+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.138642+0000","last_undegraded":"2024-09-17T05:25:03.138642+0000","last_fullsized":"2024-09-17T05:25:03.138642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159878+0000","last_change":"2024-09-17T05:25:03.156207+0000","last_active":"2024-09-17T05:25:03.159878+0000","last_peered":"2024-09-17T05:25:03.159878+0000","last_clean":"2024-09-17T05:25:03.159878+0000","last_became_active":"2024-09-17T05:25:03.155889+0000","last_became_peered":"2024-09-17T05:25:03.155889+0000","last_unstale":"2024-09-17T05:25:03.159878+0000","last_undegraded":"2024-09-17T05:25:03.159878+0000","last_fullsized":"2024-09-17T05:25:03.159878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133784+0000","last_change":"2024-09-17T05:25:03.133784+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133784+0000","last_undegraded":"2024-09-17T05:25:03.133784+0000","last_fullsized":"2024-09-17T05:25:03.133784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.159266+0000","last_change":"2024-09-17T05:25:03.158082+0000","last_active":"2024-09-17T05:25:03.159266+0000","last_peered":"2024-09-17T05:25:03.159266+0000","last_clean":"2024-09-17T05:25:03.159266+0000","last_became_active":"2024-09-17T05:25:03.157792+0000","last_became_peered":"2024-09-17T05:25:03.157792+0000","last_unstale":"2024-09-17T05:25:03.159266+0000","last_undegraded":"2024-09-17T05:25:03.159266+0000","last_fullsized":"2024-09-17T05:25:03.159266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:29.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.133095+0000","last_change":"2024-09-17T05:25:03.133095+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.133095+0000","last_undegraded":"2024-09-17T05:25:03.133095+0000","last_fullsized":"2024-09-17T05:25:03.133095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.135478+0000","last_change":"2024-09-17T05:25:03.135478+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.135478+0000","last_undegraded":"2024-09-17T05:25:03.135478+0000","last_fullsized":"2024-09-17T05:25:03.135478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.163246+0000","last_change":"2024-09-17T05:25:03.159779+0000","last_active":"2024-09-17T05:25:03.163246+0000","last_peered":"2024-09-17T05:25:03.163246+0000","last_clean":"2024-09-17T05:25:03.163246+0000","last_became_active":"2024-09-17T05:25:03.159430+0000","last_became_peered":"2024-09-17T05:25:03.159430+0000","last_unstale":"2024-09-17T05:25:03.163246+0000","last_undegraded":"2024-09-17T05:25:03.163246+0000","last_fullsized":"2024-09-17T05:25:03.163246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T05:25:03.145527+0000","last_change":"2024-09-17T05:25:03.145527+0000","last_active":"2024-09-17T05:25:03.119083+0000","last_peered":"2024-09-17T05:25:03.119083+0000","last_clean":"2024-09-17T05:25:03.119083+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:03.145527+0000","last_undegraded":"2024-09-17T05:25:03.145527+0000","last_fullsized":"2024-09-17T05:25:03.145527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.158949+0000","last_change":"2024-09-17T05:25:03.153878+0000","last_active":"2024-09-17T05:25:03.158949+0000","last_peered":"2024-09-17T05:25:03.158949+0000","last_clean":"2024-09-17T05:25:03.158949+0000","last_became_active":"2024-09-17T05:25:03.153594+0000","last_became_peered":"2024-09-17T05:25:03.153594+0000","last_unstale":"2024-09-17T05:25:03.158949+0000","last_undegraded":"2024-09-17T05:25:03.158949+0000","last_fullsized":"2024-09-17T05:25:03.158949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323879+0000","last_change":"2024-09-17T05:25:04.163341+0000","last_active":"2024-09-17T05:25:05.323879+0000","last_peered":"2024-09-17T05:25:05.323879+0000","last_clean":"2024-09-17T05:25:05.323879+0000","last_became_active":"2024-09-17T05:25:04.162727+0000","last_became_peered":"2024-09-17T05:25:04.162727+0000","last_unstale":"2024-09-17T05:25:05.323879+0000","last_undegraded":"2024-09-17T05:25:05.323879+0000","last_fullsized":"2024-09-17T05:25:05.323879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:04.451009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324619+0000","last_change":"2024-09-17T05:25:03.157287+0000","last_active":"2024-09-17T05:25:05.324619+0000","last_peered":"2024-09-17T05:25:05.324619+0000","last_clean":"2024-09-17T05:25:05.324619+0000","last_became_active":"2024-09-17T05:25:03.154403+0000","last_became_peered":"2024-09-17T05:25:03.154403+0000","last_unstale":"2024-09-17T05:25:05.324619+0000","last_undegraded":"2024-09-17T05:25:05.324619+0000","last_fullsized":"2024-09-17T05:25:05.324619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:21.699418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324558+0000","last_change":"2024-09-17T05:25:04.167303+0000","last_active":"2024-09-17T05:25:05.324558+0000","last_peered":"2024-09-17T05:25:05.324558+0000","last_clean":"2024-09-17T05:25:05.324558+0000","last_became_active":"2024-09-17T05:25:04.166953+0000","last_became_peered":"2024-09-17T05:25:04.166953+0000","last_unstale":"2024-09-17T05:25:05.324558+0000","last_undegraded":"2024-09-17T05:25:05.324558+0000","last_fullsized":"2024-09-17T05:25:05.324558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:02.920364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T05:25:03.161258+0000","last_change":"2024-09-17T05:25:03.156536+0000","last_active":"2024-09-17T05:25:03.161258+0000","last_peered":"2024-09-17T05:25:03.161258+0000","last_clean":"2024-09-17T05:25:03.161258+0000","last_became_active":"2024-09-17T05:25:03.155913+0000","last_became_peered":"2024-09-17T05:25:03.155913+0000","last_unstale":"2024-09-17T05:25:03.161258+0000","last_undegraded":"2024-09-17T05:25:03.161258+0000","last_fullsized":"2024-09-17T05:25:03.161258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":196608,"data_stored":63209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":647302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:12.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:12.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.970+0000 7f7816200640 1 -- 172.21.15.122:0/3756102116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4003ab0 msgr2=0x7f77f4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.970+0000 7f7816200640 1 --2- 172.21.15.122:0/3756102116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4003ab0 0x7f77f4003ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7800009b60 tx=0x7f780001d270 comp rx=0 tx=0).stop 2024-09-17T05:25:12.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.970+0000 7f7816200640 1 -- 172.21.15.122:0/3756102116 shutdown_connections 2024-09-17T05:25:12.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.970+0000 7f7816200640 1 --2- 172.21.15.122:0/3756102116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77f4003ab0 0x7f77f4003ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.970+0000 7f7816200640 1 -- 172.21.15.122:0/3756102116 >> 172.21.15.122:0/3756102116 conn(0x7f78100d1dc0 msgr2=0x7f78100d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:12.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.970+0000 7f7816200640 1 -- 172.21.15.122:0/3756102116 shutdown_connections 2024-09-17T05:25:12.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.970+0000 7f7816200640 1 -- 172.21.15.122:0/3756102116 wait complete. 2024-09-17T05:25:12.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 Processor -- start 2024-09-17T05:25:12.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 -- start start 2024-09-17T05:25:12.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77f4003ab0 0x7f78100cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f781014cdb0 0x7f78100c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78100c9440 0x7f78101fae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78100cd080 con 0x7f781014cdb0 2024-09-17T05:25:12.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f78100ccf00 con 0x7f78100c9440 2024-09-17T05:25:12.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f78100cd200 con 0x7f77f4003ab0 2024-09-17T05:25:12.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f781014cdb0 0x7f78100c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f781014cdb0 0x7f78100c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42672/0 (socket says 172.21.15.122:42672) 2024-09-17T05:25:12.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 -- 172.21.15.122:0/3539599008 learned_addr learned my addr 172.21.15.122:0/3539599008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:12.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780f400640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77f4003ab0 0x7f78100cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780fe00640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78100c9440 0x7f78101fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 -- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77f4003ab0 msgr2=0x7f78100cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77f4003ab0 0x7f78100cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 -- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78100c9440 msgr2=0x7f78101fae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78100c9440 0x7f78101fae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 -- 172.21.15.122:0/3539599008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78100cafc0 con 0x7f781014cdb0 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f780ea00640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f781014cdb0 0x7f78100c8f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004da0 tx=0x7f77fc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7807400640 1 -- 172.21.15.122:0/3539599008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc011660 con 0x7f781014cdb0 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7807400640 1 -- 172.21.15.122:0/3539599008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77fc016070 con 0x7f781014cdb0 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78100cb2a0 con 0x7f781014cdb0 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7807400640 1 -- 172.21.15.122:0/3539599008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc011ae0 con 0x7f781014cdb0 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.974+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78100cb860 con 0x7f781014cdb0 2024-09-17T05:25:12.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.978+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78100cd760 con 0x7f781014cdb0 2024-09-17T05:25:12.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.978+0000 7f7807400640 1 -- 172.21.15.122:0/3539599008 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77fc011c80 con 0x7f781014cdb0 2024-09-17T05:25:12.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.978+0000 7f7807400640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f77f0073230 0x7f77f00756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:12.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.978+0000 7f7807400640 1 -- 172.21.15.122:0/3539599008 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f77fc0947a0 con 0x7f781014cdb0 2024-09-17T05:25:12.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.978+0000 7f780f400640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f77f0073230 0x7f77f00756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:12.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.978+0000 7f780f400640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f77f0073230 0x7f77f00756f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78000ae000 tx=0x7f780001a010 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:12.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:12.982+0000 7f7807400640 1 -- 172.21.15.122:0/3539599008 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f77fc05f840 con 0x7f781014cdb0 2024-09-17T05:25:13.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.146+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78100762f0 con 0x7f77f0073230 2024-09-17T05:25:13.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.150+0000 7f7807400640 1 -- 172.21.15.122:0/3539599008 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7f78100762f0 con 0x7f77f0073230 2024-09-17T05:25:13.157 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:13.159 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:13.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f77f0073230 msgr2=0x7f77f00756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f77f0073230 0x7f77f00756f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78000ae000 tx=0x7f780001a010 comp rx=0 tx=0).stop 2024-09-17T05:25:13.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f781014cdb0 msgr2=0x7f78100c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f781014cdb0 0x7f78100c8f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004da0 tx=0x7f77fc001640 comp rx=0 tx=0).stop 2024-09-17T05:25:13.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 shutdown_connections 2024-09-17T05:25:13.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f77f0073230 0x7f77f00756f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78100c9440 0x7f78101fae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f781014cdb0 0x7f78100c8f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 --2- 172.21.15.122:0/3539599008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77f4003ab0 0x7f78100cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 >> 172.21.15.122:0/3539599008 conn(0x7f78100d1dc0 msgr2=0x7f7810071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:13.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 shutdown_connections 2024-09-17T05:25:13.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.154+0000 7f7816200640 1 -- 172.21.15.122:0/3539599008 wait complete. 2024-09-17T05:25:13.177 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-17T05:25:12.443498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223364,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751356,"statfs":{"total":773094113280,"available":772865388544,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.310380"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320847+0000","last_change":"2024-09-17T05:25:04.154692+0000","last_active":"2024-09-17T05:25:05.320847+0000","last_peered":"2024-09-17T05:25:05.320847+0000","last_clean":"2024-09-17T05:25:05.320847+0000","last_became_active":"2024-09-17T05:25:04.148232+0000","last_became_peered":"2024-09-17T05:25:04.148232+0000","last_unstale":"2024-09-17T05:25:05.320847+0000","last_undegraded":"2024-09-17T05:25:05.320847+0000","last_fullsized":"2024-09-17T05:25:05.320847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:30.690541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324353+0000","last_change":"2024-09-17T05:25:04.173588+0000","last_active":"2024-09-17T05:25:05.324353+0000","last_peered":"2024-09-17T05:25:05.324353+0000","last_clean":"2024-09-17T05:25:05.324353+0000","last_became_active":"2024-09-17T05:25:04.173260+0000","last_became_peered":"2024-09-17T05:25:04.173260+0000","last_unstale":"2024-09-17T05:25:05.324353+0000","last_undegraded":"2024-09-17T05:25:05.324353+0000","last_fullsized":"2024-09-17T05:25:05.324353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:13.477698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324934+0000","last_change":"2024-09-17T05:25:04.174132+0000","last_active":"2024-09-17T05:25:05.324934+0000","last_peered":"2024-09-17T05:25:05.324934+0000","last_clean":"2024-09-17T05:25:05.324934+0000","last_became_active":"2024-09-17T05:25:04.173337+0000","last_became_peered":"2024-09-17T05:25:04.173337+0000","last_unstale":"2024-09-17T05:25:05.324934+0000","last_undegraded":"2024-09-17T05:25:05.324934+0000","last_fullsized":"2024-09-17T05:25:05.324934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:50.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320851+0000","last_change":"2024-09-17T05:25:04.166451+0000","last_active":"2024-09-17T05:25:05.320851+0000","last_peered":"2024-09-17T05:25:05.320851+0000","last_clean":"2024-09-17T05:25:05.320851+0000","last_became_active":"2024-09-17T05:25:04.166193+0000","last_became_peered":"2024-09-17T05:25:04.166193+0000","last_unstale":"2024-09-17T05:25:05.320851+0000","last_undegraded":"2024-09-17T05:25:05.320851+0000","last_fullsized":"2024-09-17T05:25:05.320851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320896+0000","last_change":"2024-09-17T05:25:04.166447+0000","last_active":"2024-09-17T05:25:05.320896+0000","last_peered":"2024-09-17T05:25:05.320896+0000","last_clean":"2024-09-17T05:25:05.320896+0000","last_became_active":"2024-09-17T05:25:04.166240+0000","last_became_peered":"2024-09-17T05:25:04.166240+0000","last_unstale":"2024-09-17T05:25:05.320896+0000","last_undegraded":"2024-09-17T05:25:05.320896+0000","last_fullsized":"2024-09-17T05:25:05.320896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:20.054457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320554+0000","last_change":"2024-09-17T05:25:04.166670+0000","last_active":"2024-09-17T05:25:05.320554+0000","last_peered":"2024-09-17T05:25:05.320554+0000","last_clean":"2024-09-17T05:25:05.320554+0000","last_became_active":"2024-09-17T05:25:04.166212+0000","last_became_peered":"2024-09-17T05:25:04.166212+0000","last_unstale":"2024-09-17T05:25:05.320554+0000","last_undegraded":"2024-09-17T05:25:05.320554+0000","last_fullsized":"2024-09-17T05:25:05.320554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320762+0000","last_change":"2024-09-17T05:25:04.171397+0000","last_active":"2024-09-17T05:25:05.320762+0000","last_peered":"2024-09-17T05:25:05.320762+0000","last_clean":"2024-09-17T05:25:05.320762+0000","last_became_active":"2024-09-17T05:25:04.171157+0000","last_became_peered":"2024-09-17T05:25:04.171157+0000","last_unstale":"2024-09-17T05:25:05.320762+0000","last_undegraded":"2024-09-17T05:25:05.320762+0000","last_fullsized":"2024-09-17T05:25:05.320762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:49.992366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324125+0000","last_change":"2024-09-17T05:25:04.166345+0000","last_active":"2024-09-17T05:25:05.324125+0000","last_peered":"2024-09-17T05:25:05.324125+0000","last_clean":"2024-09-17T05:25:05.324125+0000","last_became_active":"2024-09-17T05:25:04.165795+0000","last_became_peered":"2024-09-17T05:25:04.165795+0000","last_unstale":"2024-09-17T05:25:05.324125+0000","last_undegraded":"2024-09-17T05:25:05.324125+0000","last_fullsized":"2024-09-17T05:25:05.324125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:33.426170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153496+0000","last_change":"2024-09-17T05:25:04.155768+0000","last_active":"2024-09-17T05:25:05.153496+0000","last_peered":"2024-09-17T05:25:05.153496+0000","last_clean":"2024-09-17T05:25:05.153496+0000","last_became_active":"2024-09-17T05:25:04.155277+0000","last_became_peered":"2024-09-17T05:25:04.155277+0000","last_unstale":"2024-09-17T05:25:05.153496+0000","last_undegraded":"2024-09-17T05:25:05.153496+0000","last_fullsized":"2024-09-17T05:25:05.153496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:44.070039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321217+0000","last_change":"2024-09-17T05:25:04.144665+0000","last_active":"2024-09-17T05:25:05.321217+0000","last_peered":"2024-09-17T05:25:05.321217+0000","last_clean":"2024-09-17T05:25:05.321217+0000","last_became_active":"2024-09-17T05:25:04.144382+0000","last_became_peered":"2024-09-17T05:25:04.144382+0000","last_unstale":"2024-09-17T05:25:05.321217+0000","last_undegraded":"2024-09-17T05:25:05.321217+0000","last_fullsized":"2024-09-17T05:25:05.321217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:09.785934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321369+0000","last_change":"2024-09-17T05:25:04.153782+0000","last_active":"2024-09-17T05:25:05.321369+0000","last_peered":"2024-09-17T05:25:05.321369+0000","last_clean":"2024-09-17T05:25:05.321369+0000","last_became_active":"2024-09-17T05:25:04.148194+0000","last_became_peered":"2024-09-17T05:25:04.148194+0000","last_unstale":"2024-09-17T05:25:05.321369+0000","last_undegraded":"2024-09-17T05:25:05.321369+0000","last_fullsized":"2024-09-17T05:25:05.321369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:44.683027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321040+0000","last_change":"2024-09-17T05:25:04.158630+0000","last_active":"2024-09-17T05:25:05.321040+0000","last_peered":"2024-09-17T05:25:05.321040+0000","last_clean":"2024-09-17T05:25:05.321040+0000","last_became_active":"2024-09-17T05:25:04.158382+0000","last_became_peered":"2024-09-17T05:25:04.158382+0000","last_unstale":"2024-09-17T05:25:05.321040+0000","last_undegraded":"2024-09-17T05:25:05.321040+0000","last_fullsized":"2024-09-17T05:25:05.321040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:27.039745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324868+0000","last_change":"2024-09-17T05:25:04.171927+0000","last_active":"2024-09-17T05:25:05.324868+0000","last_peered":"2024-09-17T05:25:05.324868+0000","last_clean":"2024-09-17T05:25:05.324868+0000","last_became_active":"2024-09-17T05:25:04.171634+0000","last_became_peered":"2024-09-17T05:25:04.171634+0000","last_unstale":"2024-09-17T05:25:05.324868+0000","last_undegraded":"2024-09-17T05:25:05.324868+0000","last_fullsized":"2024-09-17T05:25:05.324868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:09.532282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323783+0000","last_change":"2024-09-17T05:25:04.171235+0000","last_active":"2024-09-17T05:25:05.323783+0000","last_peered":"2024-09-17T05:25:05.323783+0000","last_clean":"2024-09-17T05:25:05.323783+0000","last_became_active":"2024-09-17T05:25:04.170717+0000","last_became_peered":"2024-09-17T05:25:04.170717+0000","last_unstale":"2024-09-17T05:25:05.323783+0000","last_undegraded":"2024-09-17T05:25:05.323783+0000","last_fullsized":"2024-09-17T05:25:05.323783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:32.411584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324904+0000","last_change":"2024-09-17T05:25:04.163764+0000","last_active":"2024-09-17T05:25:05.324904+0000","last_peered":"2024-09-17T05:25:05.324904+0000","last_clean":"2024-09-17T05:25:05.324904+0000","last_became_active":"2024-09-17T05:25:04.163197+0000","last_became_peered":"2024-09-17T05:25:04.163197+0000","last_unstale":"2024-09-17T05:25:05.324904+0000","last_undegraded":"2024-09-17T05:25:05.324904+0000","last_fullsized":"2024-09-17T05:25:05.324904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:01:59.441556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150239+0000","last_change":"2024-09-17T05:25:04.147588+0000","last_active":"2024-09-17T05:25:05.150239+0000","last_peered":"2024-09-17T05:25:05.150239+0000","last_clean":"2024-09-17T05:25:05.150239+0000","last_became_active":"2024-09-17T05:25:04.147316+0000","last_became_peered":"2024-09-17T05:25:04.147316+0000","last_unstale":"2024-09-17T05:25:05.150239+0000","last_undegraded":"2024-09-17T05:25:05.150239+0000","last_fullsized":"2024-09-17T05:25:05.150239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:44.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151006+0000","last_change":"2024-09-17T05:25:04.149293+0000","last_active":"2024-09-17T05:25:05.151006+0000","last_peered":"2024-09-17T05:25:05.151006+0000","last_clean":"2024-09-17T05:25:05.151006+0000","last_became_active":"2024-09-17T05:25:04.149079+0000","last_became_peered":"2024-09-17T05:25:04.149079+0000","last_unstale":"2024-09-17T05:25:05.151006+0000","last_undegraded":"2024-09-17T05:25:05.151006+0000","last_fullsized":"2024-09-17T05:25:05.151006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:13.903114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153437+0000","last_change":"2024-09-17T05:25:04.152808+0000","last_active":"2024-09-17T05:25:05.153437+0000","last_peered":"2024-09-17T05:25:05.153437+0000","last_clean":"2024-09-17T05:25:05.153437+0000","last_became_active":"2024-09-17T05:25:04.152501+0000","last_became_peered":"2024-09-17T05:25:04.152501+0000","last_unstale":"2024-09-17T05:25:05.153437+0000","last_undegraded":"2024-09-17T05:25:05.153437+0000","last_fullsized":"2024-09-17T05:25:05.153437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150922+0000","last_change":"2024-09-17T05:25:04.167124+0000","last_active":"2024-09-17T05:25:05.150922+0000","last_peered":"2024-09-17T05:25:05.150922+0000","last_clean":"2024-09-17T05:25:05.150922+0000","last_became_active":"2024-09-17T05:25:04.166853+0000","last_became_peered":"2024-09-17T05:25:04.166853+0000","last_unstale":"2024-09-17T05:25:05.150922+0000","last_undegraded":"2024-09-17T05:25:05.150922+0000","last_fullsized":"2024-09-17T05:25:05.150922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:05.283255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321253+0000","last_change":"2024-09-17T05:25:04.160482+0000","last_active":"2024-09-17T05:25:05.321253+0000","last_peered":"2024-09-17T05:25:05.321253+0000","last_clean":"2024-09-17T05:25:05.321253+0000","last_became_active":"2024-09-17T05:25:04.156878+0000","last_became_peered":"2024-09-17T05:25:04.156878+0000","last_unstale":"2024-09-17T05:25:05.321253+0000","last_undegraded":"2024-09-17T05:25:05.321253+0000","last_fullsized":"2024-09-17T05:25:05.321253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:44.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323874+0000","last_change":"2024-09-17T05:25:04.169764+0000","last_active":"2024-09-17T05:25:05.323874+0000","last_peered":"2024-09-17T05:25:05.323874+0000","last_clean":"2024-09-17T05:25:05.323874+0000","last_became_active":"2024-09-17T05:25:04.167730+0000","last_became_peered":"2024-09-17T05:25:04.167730+0000","last_unstale":"2024-09-17T05:25:05.323874+0000","last_undegraded":"2024-09-17T05:25:05.323874+0000","last_fullsized":"2024-09-17T05:25:05.323874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:13.526132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152880+0000","last_change":"2024-09-17T05:25:04.163296+0000","last_active":"2024-09-17T05:25:05.152880+0000","last_peered":"2024-09-17T05:25:05.152880+0000","last_clean":"2024-09-17T05:25:05.152880+0000","last_became_active":"2024-09-17T05:25:04.162996+0000","last_became_peered":"2024-09-17T05:25:04.162996+0000","last_unstale":"2024-09-17T05:25:05.152880+0000","last_undegraded":"2024-09-17T05:25:05.152880+0000","last_fullsized":"2024-09-17T05:25:05.152880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:44.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325148+0000","last_change":"2024-09-17T05:25:04.158936+0000","last_active":"2024-09-17T05:25:05.325148+0000","last_peered":"2024-09-17T05:25:05.325148+0000","last_clean":"2024-09-17T05:25:05.325148+0000","last_became_active":"2024-09-17T05:25:04.158697+0000","last_became_peered":"2024-09-17T05:25:04.158697+0000","last_unstale":"2024-09-17T05:25:05.325148+0000","last_undegraded":"2024-09-17T05:25:05.325148+0000","last_fullsized":"2024-09-17T05:25:05.325148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:45:18.794538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150244+0000","last_change":"2024-09-17T05:25:04.160086+0000","last_active":"2024-09-17T05:25:05.150244+0000","last_peered":"2024-09-17T05:25:05.150244+0000","last_clean":"2024-09-17T05:25:05.150244+0000","last_became_active":"2024-09-17T05:25:04.159826+0000","last_became_peered":"2024-09-17T05:25:04.159826+0000","last_unstale":"2024-09-17T05:25:05.150244+0000","last_undegraded":"2024-09-17T05:25:05.150244+0000","last_fullsized":"2024-09-17T05:25:05.150244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:46.502824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324281+0000","last_change":"2024-09-17T05:25:04.165631+0000","last_active":"2024-09-17T05:25:05.324281+0000","last_peered":"2024-09-17T05:25:05.324281+0000","last_clean":"2024-09-17T05:25:05.324281+0000","last_became_active":"2024-09-17T05:25:04.163086+0000","last_became_peered":"2024-09-17T05:25:04.163086+0000","last_unstale":"2024-09-17T05:25:05.324281+0000","last_undegraded":"2024-09-17T05:25:05.324281+0000","last_fullsized":"2024-09-17T05:25:05.324281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:59.610017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321281+0000","last_change":"2024-09-17T05:25:04.152176+0000","last_active":"2024-09-17T05:25:05.321281+0000","last_peered":"2024-09-17T05:25:05.321281+0000","last_clean":"2024-09-17T05:25:05.321281+0000","last_became_active":"2024-09-17T05:25:04.151035+0000","last_became_peered":"2024-09-17T05:25:04.151035+0000","last_unstale":"2024-09-17T05:25:05.321281+0000","last_undegraded":"2024-09-17T05:25:05.321281+0000","last_fullsized":"2024-09-17T05:25:05.321281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:06.019281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325437+0000","last_change":"2024-09-17T05:25:04.172127+0000","last_active":"2024-09-17T05:25:05.325437+0000","last_peered":"2024-09-17T05:25:05.325437+0000","last_clean":"2024-09-17T05:25:05.325437+0000","last_became_active":"2024-09-17T05:25:04.159538+0000","last_became_peered":"2024-09-17T05:25:04.159538+0000","last_unstale":"2024-09-17T05:25:05.325437+0000","last_undegraded":"2024-09-17T05:25:05.325437+0000","last_fullsized":"2024-09-17T05:25:05.325437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:46.839572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324955+0000","last_change":"2024-09-17T05:25:04.176025+0000","last_active":"2024-09-17T05:25:05.324955+0000","last_peered":"2024-09-17T05:25:05.324955+0000","last_clean":"2024-09-17T05:25:05.324955+0000","last_became_active":"2024-09-17T05:25:04.174826+0000","last_became_peered":"2024-09-17T05:25:04.174826+0000","last_unstale":"2024-09-17T05:25:05.324955+0000","last_undegraded":"2024-09-17T05:25:05.324955+0000","last_fullsized":"2024-09-17T05:25:05.324955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:37.465014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325254+0000","last_change":"2024-09-17T05:25:04.162109+0000","last_active":"2024-09-17T05:25:05.325254+0000","last_peered":"2024-09-17T05:25:05.325254+0000","last_clean":"2024-09-17T05:25:05.325254+0000","last_became_active":"2024-09-17T05:25:04.161807+0000","last_became_peered":"2024-09-17T05:25:04.161807+0000","last_unstale":"2024-09-17T05:25:05.325254+0000","last_undegraded":"2024-09-17T05:25:05.325254+0000","last_fullsized":"2024-09-17T05:25:05.325254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:23.271600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150335+0000","last_change":"2024-09-17T05:25:04.160575+0000","last_active":"2024-09-17T05:25:05.150335+0000","last_peered":"2024-09-17T05:25:05.150335+0000","last_clean":"2024-09-17T05:25:05.150335+0000","last_became_active":"2024-09-17T05:25:04.160104+0000","last_became_peered":"2024-09-17T05:25:04.160104+0000","last_unstale":"2024-09-17T05:25:05.150335+0000","last_undegraded":"2024-09-17T05:25:05.150335+0000","last_fullsized":"2024-09-17T05:25:05.150335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:22.186817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153243+0000","last_change":"2024-09-17T05:25:04.156777+0000","last_active":"2024-09-17T05:25:05.153243+0000","last_peered":"2024-09-17T05:25:05.153243+0000","last_clean":"2024-09-17T05:25:05.153243+0000","last_became_active":"2024-09-17T05:25:04.156013+0000","last_became_peered":"2024-09-17T05:25:04.156013+0000","last_unstale":"2024-09-17T05:25:05.153243+0000","last_undegraded":"2024-09-17T05:25:05.153243+0000","last_fullsized":"2024-09-17T05:25:05.153243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320365+0000","last_change":"2024-09-17T05:25:04.170636+0000","last_active":"2024-09-17T05:25:05.320365+0000","last_peered":"2024-09-17T05:25:05.320365+0000","last_clean":"2024-09-17T05:25:05.320365+0000","last_became_active":"2024-09-17T05:25:04.170325+0000","last_became_peered":"2024-09-17T05:25:04.170325+0000","last_unstale":"2024-09-17T05:25:05.320365+0000","last_undegraded":"2024-09-17T05:25:05.320365+0000","last_fullsized":"2024-09-17T05:25:05.320365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:09.604647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152897+0000","last_change":"2024-09-17T05:25:04.162069+0000","last_active":"2024-09-17T05:25:05.152897+0000","last_peered":"2024-09-17T05:25:05.152897+0000","last_clean":"2024-09-17T05:25:05.152897+0000","last_became_active":"2024-09-17T05:25:04.161443+0000","last_became_peered":"2024-09-17T05:25:04.161443+0000","last_unstale":"2024-09-17T05:25:05.152897+0000","last_undegraded":"2024-09-17T05:25:05.152897+0000","last_fullsized":"2024-09-17T05:25:05.152897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325321+0000","last_change":"2024-09-17T05:25:03.160754+0000","last_active":"2024-09-17T05:25:05.325321+0000","last_peered":"2024-09-17T05:25:05.325321+0000","last_clean":"2024-09-17T05:25:05.325321+0000","last_became_active":"2024-09-17T05:25:03.160433+0000","last_became_peered":"2024-09-17T05:25:03.160433+0000","last_unstale":"2024-09-17T05:25:05.325321+0000","last_undegraded":"2024-09-17T05:25:05.325321+0000","last_fullsized":"2024-09-17T05:25:05.325321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:59.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324073+0000","last_change":"2024-09-17T05:25:04.174431+0000","last_active":"2024-09-17T05:25:05.324073+0000","last_peered":"2024-09-17T05:25:05.324073+0000","last_clean":"2024-09-17T05:25:05.324073+0000","last_became_active":"2024-09-17T05:25:04.174139+0000","last_became_peered":"2024-09-17T05:25:04.174139+0000","last_unstale":"2024-09-17T05:25:05.324073+0000","last_undegraded":"2024-09-17T05:25:05.324073+0000","last_fullsized":"2024-09-17T05:25:05.324073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:40.319110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150517+0000","last_change":"2024-09-17T05:25:03.138686+0000","last_active":"2024-09-17T05:25:05.150517+0000","last_peered":"2024-09-17T05:25:05.150517+0000","last_clean":"2024-09-17T05:25:05.150517+0000","last_became_active":"2024-09-17T05:25:03.138321+0000","last_became_peered":"2024-09-17T05:25:03.138321+0000","last_unstale":"2024-09-17T05:25:05.150517+0000","last_undegraded":"2024-09-17T05:25:05.150517+0000","last_fullsized":"2024-09-17T05:25:05.150517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:03.903924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323451+0000","last_change":"2024-09-17T05:25:03.161161+0000","last_active":"2024-09-17T05:25:05.323451+0000","last_peered":"2024-09-17T05:25:05.323451+0000","last_clean":"2024-09-17T05:25:05.323451+0000","last_became_active":"2024-09-17T05:25:03.160876+0000","last_became_peered":"2024-09-17T05:25:03.160876+0000","last_unstale":"2024-09-17T05:25:05.323451+0000","last_undegraded":"2024-09-17T05:25:05.323451+0000","last_fullsized":"2024-09-17T05:25:05.323451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324408+0000","last_change":"2024-09-17T05:25:04.173583+0000","last_active":"2024-09-17T05:25:05.324408+0000","last_peered":"2024-09-17T05:25:05.324408+0000","last_clean":"2024-09-17T05:25:05.324408+0000","last_became_active":"2024-09-17T05:25:04.173276+0000","last_became_peered":"2024-09-17T05:25:04.173276+0000","last_unstale":"2024-09-17T05:25:05.324408+0000","last_undegraded":"2024-09-17T05:25:05.324408+0000","last_fullsized":"2024-09-17T05:25:05.324408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:38.290022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150430+0000","last_change":"2024-09-17T05:25:03.156886+0000","last_active":"2024-09-17T05:25:05.150430+0000","last_peered":"2024-09-17T05:25:05.150430+0000","last_clean":"2024-09-17T05:25:05.150430+0000","last_became_active":"2024-09-17T05:25:03.156595+0000","last_became_peered":"2024-09-17T05:25:03.156595+0000","last_unstale":"2024-09-17T05:25:05.150430+0000","last_undegraded":"2024-09-17T05:25:05.150430+0000","last_fullsized":"2024-09-17T05:25:05.150430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:57.284513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323745+0000","last_change":"2024-09-17T05:25:04.167294+0000","last_active":"2024-09-17T05:25:05.323745+0000","last_peered":"2024-09-17T05:25:05.323745+0000","last_clean":"2024-09-17T05:25:05.323745+0000","last_became_active":"2024-09-17T05:25:04.165115+0000","last_became_peered":"2024-09-17T05:25:04.165115+0000","last_unstale":"2024-09-17T05:25:05.323745+0000","last_undegraded":"2024-09-17T05:25:05.323745+0000","last_fullsized":"2024-09-17T05:25:05.323745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:53.176691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153460+0000","last_change":"2024-09-17T05:25:03.160352+0000","last_active":"2024-09-17T05:25:05.153460+0000","last_peered":"2024-09-17T05:25:05.153460+0000","last_clean":"2024-09-17T05:25:05.153460+0000","last_became_active":"2024-09-17T05:25:03.160030+0000","last_became_peered":"2024-09-17T05:25:03.160030+0000","last_unstale":"2024-09-17T05:25:05.153460+0000","last_undegraded":"2024-09-17T05:25:05.153460+0000","last_fullsized":"2024-09-17T05:25:05.153460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321483+0000","last_change":"2024-09-17T05:25:04.146118+0000","last_active":"2024-09-17T05:25:05.321483+0000","last_peered":"2024-09-17T05:25:05.321483+0000","last_clean":"2024-09-17T05:25:05.321483+0000","last_became_active":"2024-09-17T05:25:04.145700+0000","last_became_peered":"2024-09-17T05:25:04.145700+0000","last_unstale":"2024-09-17T05:25:05.321483+0000","last_undegraded":"2024-09-17T05:25:05.321483+0000","last_fullsized":"2024-09-17T05:25:05.321483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:44.070039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321398+0000","last_change":"2024-09-17T05:25:03.155547+0000","last_active":"2024-09-17T05:25:05.321398+0000","last_peered":"2024-09-17T05:25:05.321398+0000","last_clean":"2024-09-17T05:25:05.321398+0000","last_became_active":"2024-09-17T05:25:03.155270+0000","last_became_peered":"2024-09-17T05:25:03.155270+0000","last_unstale":"2024-09-17T05:25:05.321398+0000","last_undegraded":"2024-09-17T05:25:05.321398+0000","last_fullsized":"2024-09-17T05:25:05.321398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:13.898988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324064+0000","last_change":"2024-09-17T05:25:04.163379+0000","last_active":"2024-09-17T05:25:05.324064+0000","last_peered":"2024-09-17T05:25:05.324064+0000","last_clean":"2024-09-17T05:25:05.324064+0000","last_became_active":"2024-09-17T05:25:04.163066+0000","last_became_peered":"2024-09-17T05:25:04.163066+0000","last_unstale":"2024-09-17T05:25:05.324064+0000","last_undegraded":"2024-09-17T05:25:05.324064+0000","last_fullsized":"2024-09-17T05:25:05.324064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:43.294053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153478+0000","last_change":"2024-09-17T05:25:03.154476+0000","last_active":"2024-09-17T05:25:05.153478+0000","last_peered":"2024-09-17T05:25:05.153478+0000","last_clean":"2024-09-17T05:25:05.153478+0000","last_became_active":"2024-09-17T05:25:03.153306+0000","last_became_peered":"2024-09-17T05:25:03.153306+0000","last_unstale":"2024-09-17T05:25:05.153478+0000","last_undegraded":"2024-09-17T05:25:05.153478+0000","last_fullsized":"2024-09-17T05:25:05.153478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:10.315224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150647+0000","last_change":"2024-09-17T05:25:04.145007+0000","last_active":"2024-09-17T05:25:05.150647+0000","last_peered":"2024-09-17T05:25:05.150647+0000","last_clean":"2024-09-17T05:25:05.150647+0000","last_became_active":"2024-09-17T05:25:04.144652+0000","last_became_peered":"2024-09-17T05:25:04.144652+0000","last_unstale":"2024-09-17T05:25:05.150647+0000","last_undegraded":"2024-09-17T05:25:05.150647+0000","last_fullsized":"2024-09-17T05:25:05.150647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:36.360867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320994+0000","last_change":"2024-09-17T05:25:03.152426+0000","last_active":"2024-09-17T05:25:05.320994+0000","last_peered":"2024-09-17T05:25:05.320994+0000","last_clean":"2024-09-17T05:25:05.320994+0000","last_became_active":"2024-09-17T05:25:03.152129+0000","last_became_peered":"2024-09-17T05:25:03.152129+0000","last_unstale":"2024-09-17T05:25:05.320994+0000","last_undegraded":"2024-09-17T05:25:05.320994+0000","last_fullsized":"2024-09-17T05:25:05.320994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321562+0000","last_change":"2024-09-17T05:25:04.170311+0000","last_active":"2024-09-17T05:25:05.321562+0000","last_peered":"2024-09-17T05:25:05.321562+0000","last_clean":"2024-09-17T05:25:05.321562+0000","last_became_active":"2024-09-17T05:25:04.169997+0000","last_became_peered":"2024-09-17T05:25:04.169997+0000","last_unstale":"2024-09-17T05:25:05.321562+0000","last_undegraded":"2024-09-17T05:25:05.321562+0000","last_fullsized":"2024-09-17T05:25:05.321562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:41.040086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153410+0000","last_change":"2024-09-17T05:25:03.156640+0000","last_active":"2024-09-17T05:25:05.153410+0000","last_peered":"2024-09-17T05:25:05.153410+0000","last_clean":"2024-09-17T05:25:05.153410+0000","last_became_active":"2024-09-17T05:25:03.156338+0000","last_became_peered":"2024-09-17T05:25:03.156338+0000","last_unstale":"2024-09-17T05:25:05.153410+0000","last_undegraded":"2024-09-17T05:25:05.153410+0000","last_fullsized":"2024-09-17T05:25:05.153410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:55.588248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324723+0000","last_change":"2024-09-17T05:25:04.171110+0000","last_active":"2024-09-17T05:25:05.324723+0000","last_peered":"2024-09-17T05:25:05.324723+0000","last_clean":"2024-09-17T05:25:05.324723+0000","last_became_active":"2024-09-17T05:25:04.162548+0000","last_became_peered":"2024-09-17T05:25:04.162548+0000","last_unstale":"2024-09-17T05:25:05.324723+0000","last_undegraded":"2024-09-17T05:25:05.324723+0000","last_fullsized":"2024-09-17T05:25:05.324723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:39.835973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151001+0000","last_change":"2024-09-17T05:25:03.136217+0000","last_active":"2024-09-17T05:25:05.151001+0000","last_peered":"2024-09-17T05:25:05.151001+0000","last_clean":"2024-09-17T05:25:05.151001+0000","last_became_active":"2024-09-17T05:25:03.135857+0000","last_became_peered":"2024-09-17T05:25:03.135857+0000","last_unstale":"2024-09-17T05:25:05.151001+0000","last_undegraded":"2024-09-17T05:25:05.151001+0000","last_fullsized":"2024-09-17T05:25:05.151001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:08.521888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150591+0000","last_change":"2024-09-17T05:25:04.158299+0000","last_active":"2024-09-17T05:25:05.150591+0000","last_peered":"2024-09-17T05:25:05.150591+0000","last_clean":"2024-09-17T05:25:05.150591+0000","last_became_active":"2024-09-17T05:25:04.157981+0000","last_became_peered":"2024-09-17T05:25:04.157981+0000","last_unstale":"2024-09-17T05:25:05.150591+0000","last_undegraded":"2024-09-17T05:25:05.150591+0000","last_fullsized":"2024-09-17T05:25:05.150591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:01.960601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320929+0000","last_change":"2024-09-17T05:25:03.147144+0000","last_active":"2024-09-17T05:25:05.320929+0000","last_peered":"2024-09-17T05:25:05.320929+0000","last_clean":"2024-09-17T05:25:05.320929+0000","last_became_active":"2024-09-17T05:25:03.146907+0000","last_became_peered":"2024-09-17T05:25:03.146907+0000","last_unstale":"2024-09-17T05:25:05.320929+0000","last_undegraded":"2024-09-17T05:25:05.320929+0000","last_fullsized":"2024-09-17T05:25:05.320929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:57.284513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152791+0000","last_change":"2024-09-17T05:25:04.152327+0000","last_active":"2024-09-17T05:25:05.152791+0000","last_peered":"2024-09-17T05:25:05.152791+0000","last_clean":"2024-09-17T05:25:05.152791+0000","last_became_active":"2024-09-17T05:25:04.151767+0000","last_became_peered":"2024-09-17T05:25:04.151767+0000","last_unstale":"2024-09-17T05:25:05.152791+0000","last_undegraded":"2024-09-17T05:25:05.152791+0000","last_fullsized":"2024-09-17T05:25:05.152791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323317+0000","last_change":"2024-09-17T05:25:03.151397+0000","last_active":"2024-09-17T05:25:05.323317+0000","last_peered":"2024-09-17T05:25:05.323317+0000","last_clean":"2024-09-17T05:25:05.323317+0000","last_became_active":"2024-09-17T05:25:03.151076+0000","last_became_peered":"2024-09-17T05:25:03.151076+0000","last_unstale":"2024-09-17T05:25:05.323317+0000","last_undegraded":"2024-09-17T05:25:05.323317+0000","last_fullsized":"2024-09-17T05:25:05.323317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:44.043946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150267+0000","last_change":"2024-09-17T05:25:04.146297+0000","last_active":"2024-09-17T05:25:05.150267+0000","last_peered":"2024-09-17T05:25:05.150267+0000","last_clean":"2024-09-17T05:25:05.150267+0000","last_became_active":"2024-09-17T05:25:04.145987+0000","last_became_peered":"2024-09-17T05:25:04.145987+0000","last_unstale":"2024-09-17T05:25:05.150267+0000","last_undegraded":"2024-09-17T05:25:05.150267+0000","last_fullsized":"2024-09-17T05:25:05.150267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:44.683027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321309+0000","last_change":"2024-09-17T05:25:03.159880+0000","last_active":"2024-09-17T05:25:05.321309+0000","last_peered":"2024-09-17T05:25:05.321309+0000","last_clean":"2024-09-17T05:25:05.321309+0000","last_became_active":"2024-09-17T05:25:03.159636+0000","last_became_peered":"2024-09-17T05:25:03.159636+0000","last_unstale":"2024-09-17T05:25:05.321309+0000","last_undegraded":"2024-09-17T05:25:05.321309+0000","last_fullsized":"2024-09-17T05:25:05.321309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:04.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321681+0000","last_change":"2024-09-17T05:25:04.159287+0000","last_active":"2024-09-17T05:25:05.321681+0000","last_peered":"2024-09-17T05:25:05.321681+0000","last_clean":"2024-09-17T05:25:05.321681+0000","last_became_active":"2024-09-17T05:25:04.158769+0000","last_became_peered":"2024-09-17T05:25:04.158769+0000","last_unstale":"2024-09-17T05:25:05.321681+0000","last_undegraded":"2024-09-17T05:25:05.321681+0000","last_fullsized":"2024-09-17T05:25:05.321681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:27.763662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325478+0000","last_change":"2024-09-17T05:25:03.161164+0000","last_active":"2024-09-17T05:25:05.325478+0000","last_peered":"2024-09-17T05:25:05.325478+0000","last_clean":"2024-09-17T05:25:05.325478+0000","last_became_active":"2024-09-17T05:25:03.160875+0000","last_became_peered":"2024-09-17T05:25:03.160875+0000","last_unstale":"2024-09-17T05:25:05.325478+0000","last_undegraded":"2024-09-17T05:25:05.325478+0000","last_fullsized":"2024-09-17T05:25:05.325478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321087+0000","last_change":"2024-09-17T05:25:04.159000+0000","last_active":"2024-09-17T05:25:05.321087+0000","last_peered":"2024-09-17T05:25:05.321087+0000","last_clean":"2024-09-17T05:25:05.321087+0000","last_became_active":"2024-09-17T05:25:04.158806+0000","last_became_peered":"2024-09-17T05:25:04.158806+0000","last_unstale":"2024-09-17T05:25:05.321087+0000","last_undegraded":"2024-09-17T05:25:05.321087+0000","last_fullsized":"2024-09-17T05:25:05.321087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:50.632241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323888+0000","last_change":"2024-09-17T05:25:03.160173+0000","last_active":"2024-09-17T05:25:05.323888+0000","last_peered":"2024-09-17T05:25:05.323888+0000","last_clean":"2024-09-17T05:25:05.323888+0000","last_became_active":"2024-09-17T05:25:03.159759+0000","last_became_peered":"2024-09-17T05:25:03.159759+0000","last_unstale":"2024-09-17T05:25:05.323888+0000","last_undegraded":"2024-09-17T05:25:05.323888+0000","last_fullsized":"2024-09-17T05:25:05.323888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:06.043431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321445+0000","last_change":"2024-09-17T05:25:04.156812+0000","last_active":"2024-09-17T05:25:05.321445+0000","last_peered":"2024-09-17T05:25:05.321445+0000","last_clean":"2024-09-17T05:25:05.321445+0000","last_became_active":"2024-09-17T05:25:04.147686+0000","last_became_peered":"2024-09-17T05:25:04.147686+0000","last_unstale":"2024-09-17T05:25:05.321445+0000","last_undegraded":"2024-09-17T05:25:05.321445+0000","last_fullsized":"2024-09-17T05:25:05.321445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:03.043749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325125+0000","last_change":"2024-09-17T05:25:03.151849+0000","last_active":"2024-09-17T05:25:05.325125+0000","last_peered":"2024-09-17T05:25:05.325125+0000","last_clean":"2024-09-17T05:25:05.325125+0000","last_became_active":"2024-09-17T05:25:03.151518+0000","last_became_peered":"2024-09-17T05:25:03.151518+0000","last_unstale":"2024-09-17T05:25:05.325125+0000","last_undegraded":"2024-09-17T05:25:05.325125+0000","last_fullsized":"2024-09-17T05:25:05.325125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150742+0000","last_change":"2024-09-17T05:25:04.150792+0000","last_active":"2024-09-17T05:25:05.150742+0000","last_peered":"2024-09-17T05:25:05.150742+0000","last_clean":"2024-09-17T05:25:05.150742+0000","last_became_active":"2024-09-17T05:25:04.150523+0000","last_became_peered":"2024-09-17T05:25:04.150523+0000","last_unstale":"2024-09-17T05:25:05.150742+0000","last_undegraded":"2024-09-17T05:25:05.150742+0000","last_fullsized":"2024-09-17T05:25:05.150742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:09.954167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152794+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:05.152794+0000","last_peered":"2024-09-17T05:25:05.152794+0000","last_clean":"2024-09-17T05:25:05.152794+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:05.152794+0000","last_undegraded":"2024-09-17T05:25:05.152794+0000","last_fullsized":"2024-09-17T05:25:05.152794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150663+0000","last_change":"2024-09-17T05:25:03.154924+0000","last_active":"2024-09-17T05:25:05.150663+0000","last_peered":"2024-09-17T05:25:05.150663+0000","last_clean":"2024-09-17T05:25:05.150663+0000","last_became_active":"2024-09-17T05:25:03.154274+0000","last_became_peered":"2024-09-17T05:25:03.154274+0000","last_unstale":"2024-09-17T05:25:05.150663+0000","last_undegraded":"2024-09-17T05:25:05.150663+0000","last_fullsized":"2024-09-17T05:25:05.150663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:38.825579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323572+0000","last_change":"2024-09-17T05:25:04.169405+0000","last_active":"2024-09-17T05:25:05.323572+0000","last_peered":"2024-09-17T05:25:05.323572+0000","last_clean":"2024-09-17T05:25:05.323572+0000","last_became_active":"2024-09-17T05:25:04.167552+0000","last_became_peered":"2024-09-17T05:25:04.167552+0000","last_unstale":"2024-09-17T05:25:05.323572+0000","last_undegraded":"2024-09-17T05:25:05.323572+0000","last_fullsized":"2024-09-17T05:25:05.323572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:30.468665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150438+0000","last_change":"2024-09-17T05:25:03.154006+0000","last_active":"2024-09-17T05:25:05.150438+0000","last_peered":"2024-09-17T05:25:05.150438+0000","last_clean":"2024-09-17T05:25:05.150438+0000","last_became_active":"2024-09-17T05:25:03.153578+0000","last_became_peered":"2024-09-17T05:25:03.153578+0000","last_unstale":"2024-09-17T05:25:05.150438+0000","last_undegraded":"2024-09-17T05:25:05.150438+0000","last_fullsized":"2024-09-17T05:25:05.150438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:13.898988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324301+0000","last_change":"2024-09-17T05:25:04.160915+0000","last_active":"2024-09-17T05:25:05.324301+0000","last_peered":"2024-09-17T05:25:05.324301+0000","last_clean":"2024-09-17T05:25:05.324301+0000","last_became_active":"2024-09-17T05:25:04.159519+0000","last_became_peered":"2024-09-17T05:25:04.159519+0000","last_unstale":"2024-09-17T05:25:05.324301+0000","last_undegraded":"2024-09-17T05:25:05.324301+0000","last_fullsized":"2024-09-17T05:25:05.324301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:08.226516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153072+0000","last_change":"2024-09-17T05:25:03.154866+0000","last_active":"2024-09-17T05:25:05.153072+0000","last_peered":"2024-09-17T05:25:05.153072+0000","last_clean":"2024-09-17T05:25:05.153072+0000","last_became_active":"2024-09-17T05:25:03.154116+0000","last_became_peered":"2024-09-17T05:25:03.154116+0000","last_unstale":"2024-09-17T05:25:05.153072+0000","last_undegraded":"2024-09-17T05:25:05.153072+0000","last_fullsized":"2024-09-17T05:25:05.153072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324275+0000","last_change":"2024-09-17T05:25:04.162175+0000","last_active":"2024-09-17T05:25:05.324275+0000","last_peered":"2024-09-17T05:25:05.324275+0000","last_clean":"2024-09-17T05:25:05.324275+0000","last_became_active":"2024-09-17T05:25:04.161820+0000","last_became_peered":"2024-09-17T05:25:04.161820+0000","last_unstale":"2024-09-17T05:25:05.324275+0000","last_undegraded":"2024-09-17T05:25:05.324275+0000","last_fullsized":"2024-09-17T05:25:05.324275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:08.628783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151000+0000","last_change":"2024-09-17T05:25:03.144930+0000","last_active":"2024-09-17T05:25:05.151000+0000","last_peered":"2024-09-17T05:25:05.151000+0000","last_clean":"2024-09-17T05:25:05.151000+0000","last_became_active":"2024-09-17T05:25:03.144665+0000","last_became_peered":"2024-09-17T05:25:03.144665+0000","last_unstale":"2024-09-17T05:25:05.151000+0000","last_undegraded":"2024-09-17T05:25:05.151000+0000","last_fullsized":"2024-09-17T05:25:05.151000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:31.401190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321109+0000","last_change":"2024-09-17T05:25:04.155323+0000","last_active":"2024-09-17T05:25:05.321109+0000","last_peered":"2024-09-17T05:25:05.321109+0000","last_clean":"2024-09-17T05:25:05.321109+0000","last_became_active":"2024-09-17T05:25:04.148816+0000","last_became_peered":"2024-09-17T05:25:04.148816+0000","last_unstale":"2024-09-17T05:25:05.321109+0000","last_undegraded":"2024-09-17T05:25:05.321109+0000","last_fullsized":"2024-09-17T05:25:05.321109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:34.744953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323728+0000","last_change":"2024-09-17T05:25:03.160065+0000","last_active":"2024-09-17T05:25:05.323728+0000","last_peered":"2024-09-17T05:25:05.323728+0000","last_clean":"2024-09-17T05:25:05.323728+0000","last_became_active":"2024-09-17T05:25:03.159684+0000","last_became_peered":"2024-09-17T05:25:03.159684+0000","last_unstale":"2024-09-17T05:25:05.323728+0000","last_undegraded":"2024-09-17T05:25:05.323728+0000","last_fullsized":"2024-09-17T05:25:05.323728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:04.387303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320608+0000","last_change":"2024-09-17T05:25:04.171447+0000","last_active":"2024-09-17T05:25:05.320608+0000","last_peered":"2024-09-17T05:25:05.320608+0000","last_clean":"2024-09-17T05:25:05.320608+0000","last_became_active":"2024-09-17T05:25:04.171049+0000","last_became_peered":"2024-09-17T05:25:04.171049+0000","last_unstale":"2024-09-17T05:25:05.320608+0000","last_undegraded":"2024-09-17T05:25:05.320608+0000","last_fullsized":"2024-09-17T05:25:05.320608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:44.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150897+0000","last_change":"2024-09-17T05:25:03.150695+0000","last_active":"2024-09-17T05:25:05.150897+0000","last_peered":"2024-09-17T05:25:05.150897+0000","last_clean":"2024-09-17T05:25:05.150897+0000","last_became_active":"2024-09-17T05:25:03.150452+0000","last_became_peered":"2024-09-17T05:25:03.150452+0000","last_unstale":"2024-09-17T05:25:05.150897+0000","last_undegraded":"2024-09-17T05:25:05.150897+0000","last_fullsized":"2024-09-17T05:25:05.150897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:04.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325109+0000","last_change":"2024-09-17T05:25:04.173584+0000","last_active":"2024-09-17T05:25:05.325109+0000","last_peered":"2024-09-17T05:25:05.325109+0000","last_clean":"2024-09-17T05:25:05.325109+0000","last_became_active":"2024-09-17T05:25:04.173144+0000","last_became_peered":"2024-09-17T05:25:04.173144+0000","last_unstale":"2024-09-17T05:25:05.325109+0000","last_undegraded":"2024-09-17T05:25:05.325109+0000","last_fullsized":"2024-09-17T05:25:05.325109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:22.262151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325559+0000","last_change":"2024-09-17T05:25:03.153186+0000","last_active":"2024-09-17T05:25:05.325559+0000","last_peered":"2024-09-17T05:25:05.325559+0000","last_clean":"2024-09-17T05:25:05.325559+0000","last_became_active":"2024-09-17T05:25:03.152579+0000","last_became_peered":"2024-09-17T05:25:03.152579+0000","last_unstale":"2024-09-17T05:25:05.325559+0000","last_undegraded":"2024-09-17T05:25:05.325559+0000","last_fullsized":"2024-09-17T05:25:05.325559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:40.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325278+0000","last_change":"2024-09-17T05:25:04.178286+0000","last_active":"2024-09-17T05:25:05.325278+0000","last_peered":"2024-09-17T05:25:05.325278+0000","last_clean":"2024-09-17T05:25:05.325278+0000","last_became_active":"2024-09-17T05:25:04.177979+0000","last_became_peered":"2024-09-17T05:25:04.177979+0000","last_unstale":"2024-09-17T05:25:05.325278+0000","last_undegraded":"2024-09-17T05:25:05.325278+0000","last_fullsized":"2024-09-17T05:25:05.325278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:24.204998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151091+0000","last_change":"2024-09-17T05:25:03.144098+0000","last_active":"2024-09-17T05:25:05.151091+0000","last_peered":"2024-09-17T05:25:05.151091+0000","last_clean":"2024-09-17T05:25:05.151091+0000","last_became_active":"2024-09-17T05:25:03.137827+0000","last_became_peered":"2024-09-17T05:25:03.137827+0000","last_unstale":"2024-09-17T05:25:05.151091+0000","last_undegraded":"2024-09-17T05:25:05.151091+0000","last_fullsized":"2024-09-17T05:25:05.151091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:36.454620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325378+0000","last_change":"2024-09-17T05:25:04.169501+0000","last_active":"2024-09-17T05:25:05.325378+0000","last_peered":"2024-09-17T05:25:05.325378+0000","last_clean":"2024-09-17T05:25:05.325378+0000","last_became_active":"2024-09-17T05:25:04.168926+0000","last_became_peered":"2024-09-17T05:25:04.168926+0000","last_unstale":"2024-09-17T05:25:05.325378+0000","last_undegraded":"2024-09-17T05:25:05.325378+0000","last_fullsized":"2024-09-17T05:25:05.325378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:51.977366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320442+0000","last_change":"2024-09-17T05:25:03.155817+0000","last_active":"2024-09-17T05:25:05.320442+0000","last_peered":"2024-09-17T05:25:05.320442+0000","last_clean":"2024-09-17T05:25:05.320442+0000","last_became_active":"2024-09-17T05:25:03.155300+0000","last_became_peered":"2024-09-17T05:25:03.155300+0000","last_unstale":"2024-09-17T05:25:05.320442+0000","last_undegraded":"2024-09-17T05:25:05.320442+0000","last_fullsized":"2024-09-17T05:25:05.320442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:17.915256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150325+0000","last_change":"2024-09-17T05:25:04.167116+0000","last_active":"2024-09-17T05:25:05.150325+0000","last_peered":"2024-09-17T05:25:05.150325+0000","last_clean":"2024-09-17T05:25:05.150325+0000","last_became_active":"2024-09-17T05:25:04.166832+0000","last_became_peered":"2024-09-17T05:25:04.166832+0000","last_unstale":"2024-09-17T05:25:05.150325+0000","last_undegraded":"2024-09-17T05:25:05.150325+0000","last_fullsized":"2024-09-17T05:25:05.150325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:41.040086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323594+0000","last_change":"2024-09-17T05:25:03.155466+0000","last_active":"2024-09-17T05:25:05.323594+0000","last_peered":"2024-09-17T05:25:05.323594+0000","last_clean":"2024-09-17T05:25:05.323594+0000","last_became_active":"2024-09-17T05:25:03.153103+0000","last_became_peered":"2024-09-17T05:25:03.153103+0000","last_unstale":"2024-09-17T05:25:05.323594+0000","last_undegraded":"2024-09-17T05:25:05.323594+0000","last_fullsized":"2024-09-17T05:25:05.323594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:29.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323335+0000","last_change":"2024-09-17T05:25:04.166048+0000","last_active":"2024-09-17T05:25:05.323335+0000","last_peered":"2024-09-17T05:25:05.323335+0000","last_clean":"2024-09-17T05:25:05.323335+0000","last_became_active":"2024-09-17T05:25:04.165647+0000","last_became_peered":"2024-09-17T05:25:04.165647+0000","last_unstale":"2024-09-17T05:25:05.323335+0000","last_undegraded":"2024-09-17T05:25:05.323335+0000","last_fullsized":"2024-09-17T05:25:05.323335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:22.190410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324888+0000","last_change":"2024-09-17T05:25:03.156207+0000","last_active":"2024-09-17T05:25:05.324888+0000","last_peered":"2024-09-17T05:25:05.324888+0000","last_clean":"2024-09-17T05:25:05.324888+0000","last_became_active":"2024-09-17T05:25:03.155889+0000","last_became_peered":"2024-09-17T05:25:03.155889+0000","last_unstale":"2024-09-17T05:25:05.324888+0000","last_undegraded":"2024-09-17T05:25:05.324888+0000","last_fullsized":"2024-09-17T05:25:05.324888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153253+0000","last_change":"2024-09-17T05:25:04.163791+0000","last_active":"2024-09-17T05:25:05.153253+0000","last_peered":"2024-09-17T05:25:05.153253+0000","last_clean":"2024-09-17T05:25:05.153253+0000","last_became_active":"2024-09-17T05:25:04.163047+0000","last_became_peered":"2024-09-17T05:25:04.163047+0000","last_unstale":"2024-09-17T05:25:05.153253+0000","last_undegraded":"2024-09-17T05:25:05.153253+0000","last_fullsized":"2024-09-17T05:25:05.153253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:05.977620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150564+0000","last_change":"2024-09-17T05:25:03.158082+0000","last_active":"2024-09-17T05:25:05.150564+0000","last_peered":"2024-09-17T05:25:05.150564+0000","last_clean":"2024-09-17T05:25:05.150564+0000","last_became_active":"2024-09-17T05:25:03.157792+0000","last_became_peered":"2024-09-17T05:25:03.157792+0000","last_unstale":"2024-09-17T05:25:05.150564+0000","last_undegraded":"2024-09-17T05:25:05.150564+0000","last_fullsized":"2024-09-17T05:25:05.150564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:07.216122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150416+0000","last_change":"2024-09-17T05:25:04.153904+0000","last_active":"2024-09-17T05:25:05.150416+0000","last_peered":"2024-09-17T05:25:05.150416+0000","last_clean":"2024-09-17T05:25:05.150416+0000","last_became_active":"2024-09-17T05:25:04.153582+0000","last_became_peered":"2024-09-17T05:25:04.153582+0000","last_unstale":"2024-09-17T05:25:05.150416+0000","last_undegraded":"2024-09-17T05:25:05.150416+0000","last_fullsized":"2024-09-17T05:25:05.150416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:49.536379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150685+0000","last_change":"2024-09-17T05:25:04.150254+0000","last_active":"2024-09-17T05:25:05.150685+0000","last_peered":"2024-09-17T05:25:05.150685+0000","last_clean":"2024-09-17T05:25:05.150685+0000","last_became_active":"2024-09-17T05:25:04.149481+0000","last_became_peered":"2024-09-17T05:25:04.149481+0000","last_unstale":"2024-09-17T05:25:05.150685+0000","last_undegraded":"2024-09-17T05:25:05.150685+0000","last_fullsized":"2024-09-17T05:25:05.150685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:15.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321580+0000","last_change":"2024-09-17T05:25:03.159779+0000","last_active":"2024-09-17T05:25:05.321580+0000","last_peered":"2024-09-17T05:25:05.321580+0000","last_clean":"2024-09-17T05:25:05.321580+0000","last_became_active":"2024-09-17T05:25:03.159430+0000","last_became_peered":"2024-09-17T05:25:03.159430+0000","last_unstale":"2024-09-17T05:25:05.321580+0000","last_undegraded":"2024-09-17T05:25:05.321580+0000","last_fullsized":"2024-09-17T05:25:05.321580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:33.326424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153233+0000","last_change":"2024-09-17T05:25:04.173233+0000","last_active":"2024-09-17T05:25:05.153233+0000","last_peered":"2024-09-17T05:25:05.153233+0000","last_clean":"2024-09-17T05:25:05.153233+0000","last_became_active":"2024-09-17T05:25:04.172918+0000","last_became_peered":"2024-09-17T05:25:04.172918+0000","last_unstale":"2024-09-17T05:25:05.153233+0000","last_undegraded":"2024-09-17T05:25:05.153233+0000","last_fullsized":"2024-09-17T05:25:05.153233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:06.019281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321565+0000","last_change":"2024-09-17T05:25:03.153878+0000","last_active":"2024-09-17T05:25:05.321565+0000","last_peered":"2024-09-17T05:25:05.321565+0000","last_clean":"2024-09-17T05:25:05.321565+0000","last_became_active":"2024-09-17T05:25:03.153594+0000","last_became_peered":"2024-09-17T05:25:03.153594+0000","last_unstale":"2024-09-17T05:25:05.321565+0000","last_undegraded":"2024-09-17T05:25:05.321565+0000","last_fullsized":"2024-09-17T05:25:05.321565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:16:31.831690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323879+0000","last_change":"2024-09-17T05:25:04.163341+0000","last_active":"2024-09-17T05:25:05.323879+0000","last_peered":"2024-09-17T05:25:05.323879+0000","last_clean":"2024-09-17T05:25:05.323879+0000","last_became_active":"2024-09-17T05:25:04.162727+0000","last_became_peered":"2024-09-17T05:25:04.162727+0000","last_unstale":"2024-09-17T05:25:05.323879+0000","last_undegraded":"2024-09-17T05:25:05.323879+0000","last_fullsized":"2024-09-17T05:25:05.323879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:04.451009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324619+0000","last_change":"2024-09-17T05:25:03.157287+0000","last_active":"2024-09-17T05:25:05.324619+0000","last_peered":"2024-09-17T05:25:05.324619+0000","last_clean":"2024-09-17T05:25:05.324619+0000","last_became_active":"2024-09-17T05:25:03.154403+0000","last_became_peered":"2024-09-17T05:25:03.154403+0000","last_unstale":"2024-09-17T05:25:05.324619+0000","last_undegraded":"2024-09-17T05:25:05.324619+0000","last_fullsized":"2024-09-17T05:25:05.324619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:21.699418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324558+0000","last_change":"2024-09-17T05:25:04.167303+0000","last_active":"2024-09-17T05:25:05.324558+0000","last_peered":"2024-09-17T05:25:05.324558+0000","last_clean":"2024-09-17T05:25:05.324558+0000","last_became_active":"2024-09-17T05:25:04.166953+0000","last_became_peered":"2024-09-17T05:25:04.166953+0000","last_unstale":"2024-09-17T05:25:05.324558+0000","last_undegraded":"2024-09-17T05:25:05.324558+0000","last_fullsized":"2024-09-17T05:25:05.324558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:02.920364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325412+0000","last_change":"2024-09-17T05:25:03.156536+0000","last_active":"2024-09-17T05:25:05.325412+0000","last_peered":"2024-09-17T05:25:05.325412+0000","last_clean":"2024-09-17T05:25:05.325412+0000","last_became_active":"2024-09-17T05:25:03.155913+0000","last_became_peered":"2024-09-17T05:25:03.155913+0000","last_unstale":"2024-09-17T05:25:05.325412+0000","last_undegraded":"2024-09-17T05:25:05.325412+0000","last_fullsized":"2024-09-17T05:25:05.325412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:08.594253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:13.182 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:25:13.182 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T05:25:13.182 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T05:25:13.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 -- 172.21.15.122:0/443645975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0b49e0 msgr2=0x7fbe4c06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 --2- 172.21.15.122:0/443645975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0b49e0 0x7fbe4c06f7b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009c80 tx=0x7fbe4401cd10 comp rx=0 tx=0).stop 2024-09-17T05:25:13.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 -- 172.21.15.122:0/443645975 shutdown_connections 2024-09-17T05:25:13.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 --2- 172.21.15.122:0/443645975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0b49e0 0x7fbe4c06f7b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 --2- 172.21.15.122:0/443645975 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe4c15dae0 0x7fbe4c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 -- 172.21.15.122:0/443645975 >> 172.21.15.122:0/443645975 conn(0x7fbe4c0b18f0 msgr2=0x7fbe4c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:13.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 -- 172.21.15.122:0/443645975 shutdown_connections 2024-09-17T05:25:13.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 -- 172.21.15.122:0/443645975 wait complete. 2024-09-17T05:25:13.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 Processor -- start 2024-09-17T05:25:13.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.358+0000 7fbe53400640 1 -- start start 2024-09-17T05:25:13.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe4c0b49e0 0x7fbe4c14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:13.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0cd100 0x7fbe4c14c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:13.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe4c15dae0 0x7fbe4c1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:13.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c1674b0 con 0x7fbe4c0cd100 2024-09-17T05:25:13.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4c167330 con 0x7fbe4c0b49e0 2024-09-17T05:25:13.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbe4c167630 con 0x7fbe4c15dae0 2024-09-17T05:25:13.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0cd100 0x7fbe4c14c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:13.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0cd100 0x7fbe4c14c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42696/0 (socket says 172.21.15.122:42696) 2024-09-17T05:25:13.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 -- 172.21.15.122:0/2777071893 learned_addr learned my addr 172.21.15.122:0/2777071893 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:13.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe50a00640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe4c0b49e0 0x7fbe4c14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:13.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe51400640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe4c15dae0 0x7fbe4c1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:13.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 -- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe4c15dae0 msgr2=0x7fbe4c1581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe4c15dae0 0x7fbe4c1581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 -- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe4c0b49e0 msgr2=0x7fbe4c14cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe4c0b49e0 0x7fbe4c14cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 -- 172.21.15.122:0/2777071893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4c156b40 con 0x7fbe4c0cd100 2024-09-17T05:25:13.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe51400640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe4c15dae0 0x7fbe4c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:13.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe43e00640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0cd100 0x7fbe4c14c030 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c480 tx=0x7fbe3400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:13.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe49600640 1 -- 172.21.15.122:0/2777071893 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34017070 con 0x7fbe4c0cd100 2024-09-17T05:25:13.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe49600640 1 -- 172.21.15.122:0/2777071893 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe34003600 con 0x7fbe4c0cd100 2024-09-17T05:25:13.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe49600640 1 -- 172.21.15.122:0/2777071893 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34003a50 con 0x7fbe4c0cd100 2024-09-17T05:25:13.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4c154c30 con 0x7fbe4c0cd100 2024-09-17T05:25:13.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe4c1551f0 con 0x7fbe4c0cd100 2024-09-17T05:25:13.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.362+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe4c069410 con 0x7fbe4c0cd100 2024-09-17T05:25:13.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.366+0000 7fbe49600640 1 -- 172.21.15.122:0/2777071893 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe3401a070 con 0x7fbe4c0cd100 2024-09-17T05:25:13.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.366+0000 7fbe49600640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe28073200 0x7fbe280756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:13.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.366+0000 7fbe49600640 1 -- 172.21.15.122:0/2777071893 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbe34093550 con 0x7fbe4c0cd100 2024-09-17T05:25:13.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.366+0000 7fbe50a00640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe28073200 0x7fbe280756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:13.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.366+0000 7fbe50a00640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe28073200 0x7fbe280756c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c006fd0 tx=0x7fbe3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:13.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.370+0000 7fbe49600640 1 -- 172.21.15.122:0/2777071893 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbe3405e6b0 con 0x7fbe4c0cd100 2024-09-17T05:25:13.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fbe4c0acd40 con 0x7fbe4c0cd100 2024-09-17T05:25:13.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe49600640 1 -- 172.21.15.122:0/2777071893 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7fbe3405dfb0 con 0x7fbe4c0cd100 2024-09-17T05:25:13.541 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:13.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe28073200 msgr2=0x7fbe280756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe28073200 0x7fbe280756c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c006fd0 tx=0x7fbe3c008040 comp rx=0 tx=0).stop 2024-09-17T05:25:13.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0cd100 msgr2=0x7fbe4c14c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0cd100 0x7fbe4c14c030 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c480 tx=0x7fbe3400c940 comp rx=0 tx=0).stop 2024-09-17T05:25:13.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 shutdown_connections 2024-09-17T05:25:13.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe28073200 0x7fbe280756c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe4c15dae0 0x7fbe4c1581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe4c0cd100 0x7fbe4c14c030 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 --2- 172.21.15.122:0/2777071893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe4c0b49e0 0x7fbe4c14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 >> 172.21.15.122:0/2777071893 conn(0x7fbe4c0b18f0 msgr2=0x7fbe4c09cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:13.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 shutdown_connections 2024-09-17T05:25:13.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.538+0000 7fbe53400640 1 -- 172.21.15.122:0/2777071893 wait complete. 2024-09-17T05:25:13.562 INFO:teuthology.orchestra.run.smithi122.stdout:[{"version":27,"timestamp":"2024-09-17T05:24:53.235909+0000","name":"","changes":[{"name":"osd.3/osd_mclock_max_capacity_iops_ssd","new_value":"3894.303595"}]}] 2024-09-17T05:25:13.562 INFO:tasks.ceph_manager:config epoch is 27 2024-09-17T05:25:13.562 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T05:25:13.562 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T05:25:13.562 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T05:25:13.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.698+0000 7fda01600640 1 -- 172.21.15.122:0/1810842162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc0737c0 msgr2=0x7fd9fc15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.698+0000 7fda01600640 1 --2- 172.21.15.122:0/1810842162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc0737c0 0x7fd9fc15c1c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009930 tx=0x7fd9e401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.698+0000 7fda01600640 1 -- 172.21.15.122:0/1810842162 shutdown_connections 2024-09-17T05:25:13.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.698+0000 7fda01600640 1 --2- 172.21.15.122:0/1810842162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9fc071030 0x7fd9fc0714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.698+0000 7fda01600640 1 --2- 172.21.15.122:0/1810842162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc0737c0 0x7fd9fc15c1c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.698+0000 7fda01600640 1 -- 172.21.15.122:0/1810842162 >> 172.21.15.122:0/1810842162 conn(0x7fd9fc0d0c90 msgr2=0x7fd9fc0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:13.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.698+0000 7fda01600640 1 -- 172.21.15.122:0/1810842162 shutdown_connections 2024-09-17T05:25:13.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 -- 172.21.15.122:0/1810842162 wait complete. 2024-09-17T05:25:13.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 Processor -- start 2024-09-17T05:25:13.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 -- start start 2024-09-17T05:25:13.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc06cc20 0x7fd9fc1f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:13.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9fc071030 0x7fd9fc1fb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:13.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9fc0737c0 0x7fd9fc1fbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:13.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc168f90 con 0x7fd9fc06cc20 2024-09-17T05:25:13.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc168e10 con 0x7fd9fc071030 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd9fc169110 con 0x7fd9fc0737c0 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc06cc20 0x7fd9fc1f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc06cc20 0x7fd9fc1f7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42716/0 (socket says 172.21.15.122:42716) 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 -- 172.21.15.122:0/3616117750 learned_addr learned my addr 172.21.15.122:0/3616117750 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9fb400640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9fc0737c0 0x7fd9fc1fbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9f2000640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9fc071030 0x7fd9fc1fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 -- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9fc0737c0 msgr2=0x7fd9fc1fbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9fc0737c0 0x7fd9fc1fbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 -- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9fc071030 msgr2=0x7fd9fc1fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9fc071030 0x7fd9fc1fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 -- 172.21.15.122:0/3616117750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc1f7ec0 con 0x7fd9fc06cc20 2024-09-17T05:25:13.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9faa00640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc06cc20 0x7fd9fc1f7980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c870 tx=0x7fd9ec00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:13.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9f3400640 1 -- 172.21.15.122:0/3616117750 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec003b80 con 0x7fd9fc06cc20 2024-09-17T05:25:13.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9f3400640 1 -- 172.21.15.122:0/3616117750 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9ec003d20 con 0x7fd9fc06cc20 2024-09-17T05:25:13.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc0b5c60 con 0x7fd9fc06cc20 2024-09-17T05:25:13.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fd9f3400640 1 -- 172.21.15.122:0/3616117750 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00aab0 con 0x7fd9fc06cc20 2024-09-17T05:25:13.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.702+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc0b6220 con 0x7fd9fc06cc20 2024-09-17T05:25:13.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.706+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9fc0714a0 con 0x7fd9fc06cc20 2024-09-17T05:25:13.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.706+0000 7fd9f3400640 1 -- 172.21.15.122:0/3616117750 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd9ec00ac50 con 0x7fd9fc06cc20 2024-09-17T05:25:13.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.706+0000 7fd9f3400640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:13.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.706+0000 7fd9f3400640 1 -- 172.21.15.122:0/3616117750 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd9ec094150 con 0x7fd9fc06cc20 2024-09-17T05:25:13.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.706+0000 7fd9f2000640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:13.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.706+0000 7fd9f2000640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40af000 tx=0x7fd9e401b0f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:13.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.710+0000 7fd9f3400640 1 -- 172.21.15.122:0/3616117750 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd9ec05f170 con 0x7fd9fc06cc20 2024-09-17T05:25:13.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.918+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd9fc0b6c10 con 0x7fd9fc06cc20 2024-09-17T05:25:13.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.922+0000 7fd9f3400640 1 -- 172.21.15.122:0/3616117750 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186463 (secure 0 0 0) 0x7fd9ec05ea70 con 0x7fd9fc06cc20 2024-09-17T05:25:13.928 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:13.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 msgr2=0x7fd9d80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40af000 tx=0x7fd9e401b0f0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc06cc20 msgr2=0x7fd9fc1f7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc06cc20 0x7fd9fc1f7980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c870 tx=0x7fd9ec00cd30 comp rx=0 tx=0).stop 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 shutdown_connections 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9fc0737c0 0x7fd9fc1fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9fc071030 0x7fd9fc1fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 --2- 172.21.15.122:0/3616117750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9fc06cc20 0x7fd9fc1f7980 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 >> 172.21.15.122:0/3616117750 conn(0x7fd9fc0d0c90 msgr2=0x7fd9fc0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 shutdown_connections 2024-09-17T05:25:13.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:13.926+0000 7fda01600640 1 -- 172.21.15.122:0/3616117750 wait complete. 2024-09-17T05:25:13.949 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3859572675},{"type":"v1","addr":"172.21.15.173:6801","nonce":3859572675}]},"active_addr":"172.21.15.173:6801/3859572675","active_change":"2024-09-17T05:24:44.429532+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4117,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":678819843}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":2530543153}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":1935822029}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":3095745794}]}]} 2024-09-17T05:25:13.953 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T05:25:13.953 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T05:25:13.954 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:14.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- 172.21.15.122:0/2058037015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 msgr2=0x7fe074003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 --2- 172.21.15.122:0/2058037015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 0x7fe074003f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe080006f90 tx=0x7fe08001cee0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe094c00640 1 -- 172.21.15.122:0/2058037015 reap_dead start 2024-09-17T05:25:14.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- 172.21.15.122:0/2058037015 shutdown_connections 2024-09-17T05:25:14.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- 172.21.15.122:0/2058037015 >> 172.21.15.122:0/2058037015 conn(0x7fe0900b2240 msgr2=0x7fe0900b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- 172.21.15.122:0/2058037015 shutdown_connections 2024-09-17T05:25:14.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- 172.21.15.122:0/2058037015 wait complete. 2024-09-17T05:25:14.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 Processor -- start 2024-09-17T05:25:14.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- start start 2024-09-17T05:25:14.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 0x7fe0901fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe090158670 0x7fe0900d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0900ad9d0 0x7fe090155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe09016a170 con 0x7fe074003ae0 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe090169ff0 con 0x7fe090158670 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.094+0000 7fe096c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe09016a2f0 con 0x7fe0900ad9d0 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 0x7fe0901fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 0x7fe0901fc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42728/0 (socket says 172.21.15.122:42728) 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 -- 172.21.15.122:0/3815935946 learned_addr learned my addr 172.21.15.122:0/3815935946 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe087e00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe090158670 0x7fe0900d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe094c00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0900ad9d0 0x7fe090155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 -- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0900ad9d0 msgr2=0x7fe090155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0900ad9d0 0x7fe090155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 -- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe090158670 msgr2=0x7fe0900d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe090158670 0x7fe0900d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 -- 172.21.15.122:0/3815935946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09006a300 con 0x7fe074003ae0 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe094c00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0900ad9d0 0x7fe090155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08fe00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 0x7fe0901fc0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe08001b5e0 tx=0x7fe080006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08cc00640 1 -- 172.21.15.122:0/3815935946 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800a5bf0 con 0x7fe074003ae0 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08cc00640 1 -- 172.21.15.122:0/3815935946 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe080001440 con 0x7fe074003ae0 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08cc00640 1 -- 172.21.15.122:0/3815935946 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800060d0 con 0x7fe074003ae0 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0901fc600 con 0x7fe074003ae0 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0900afa30 con 0x7fe074003ae0 2024-09-17T05:25:14.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0901623e0 con 0x7fe074003ae0 2024-09-17T05:25:14.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.098+0000 7fe08cc00640 1 -- 172.21.15.122:0/3815935946 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe080006270 con 0x7fe074003ae0 2024-09-17T05:25:14.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.102+0000 7fe08cc00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe06c073230 0x7fe06c0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.102+0000 7fe08cc00640 1 -- 172.21.15.122:0/3815935946 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe08012dba0 con 0x7fe074003ae0 2024-09-17T05:25:14.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.102+0000 7fe087e00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe06c073230 0x7fe06c0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.102+0000 7fe087e00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe06c073230 0x7fe06c0756f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe090158a70 tx=0x7fe078009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.106+0000 7fe08cc00640 1 -- 172.21.15.122:0/3815935946 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe080016c90 con 0x7fe074003ae0 2024-09-17T05:25:14.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.274+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe09006cc20 con 0x7fe074003ae0 2024-09-17T05:25:14.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.274+0000 7fe08cc00640 1 -- 172.21.15.122:0/3815935946 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7fe0800ff070 con 0x7fe074003ae0 2024-09-17T05:25:14.278 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:14.278 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":22,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:05.140074+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:14.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe06c073230 msgr2=0x7fe06c0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe06c073230 0x7fe06c0756f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe090158a70 tx=0x7fe078009040 comp rx=0 tx=0).stop 2024-09-17T05:25:14.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 msgr2=0x7fe0901fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 0x7fe0901fc0c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe08001b5e0 tx=0x7fe080006c50 comp rx=0 tx=0).stop 2024-09-17T05:25:14.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 shutdown_connections 2024-09-17T05:25:14.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe06c073230 0x7fe06c0756f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0900ad9d0 0x7fe090155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe090158670 0x7fe0900d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 --2- 172.21.15.122:0/3815935946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe074003ae0 0x7fe0901fc0c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 >> 172.21.15.122:0/3815935946 conn(0x7fe0900b2240 msgr2=0x7fe0900b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 shutdown_connections 2024-09-17T05:25:14.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.278+0000 7fe096c00640 1 -- 172.21.15.122:0/3815935946 wait complete. 2024-09-17T05:25:14.299 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T05:25:14.300 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:14.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 -- 172.21.15.122:0/2992309181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 msgr2=0x7f9d640acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 --2- 172.21.15.122:0/2992309181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 0x7f9d640acd40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009900 tx=0x7f9d4c01c850 comp rx=0 tx=0).stop 2024-09-17T05:25:14.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 -- 172.21.15.122:0/2992309181 shutdown_connections 2024-09-17T05:25:14.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 --2- 172.21.15.122:0/2992309181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 0x7f9d640acd40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 -- 172.21.15.122:0/2992309181 >> 172.21.15.122:0/2992309181 conn(0x7f9d640b18f0 msgr2=0x7f9d640b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 -- 172.21.15.122:0/2992309181 shutdown_connections 2024-09-17T05:25:14.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 -- 172.21.15.122:0/2992309181 wait complete. 2024-09-17T05:25:14.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 Processor -- start 2024-09-17T05:25:14.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 -- start start 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.438+0000 7f9d6b000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 0x7f9d640c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d6b000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d640cd100 0x7f9d6406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d6b000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6415db50 0x7f9d6413d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d6b000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641688f0 con 0x7f9d640b4a50 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d6b000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64168770 con 0x7f9d640cd100 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d6b000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9d64168a70 con 0x7f9d6415db50 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 0x7f9d640c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 0x7f9d640c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42746/0 (socket says 172.21.15.122:42746) 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 -- 172.21.15.122:0/2707134599 learned_addr learned my addr 172.21.15.122:0/2707134599 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d5b400640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d640cd100 0x7f9d6406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d69000640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6415db50 0x7f9d6413d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 -- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6415db50 msgr2=0x7f9d6413d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6415db50 0x7f9d6413d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 -- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d640cd100 msgr2=0x7f9d6406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d640cd100 0x7f9d6406ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 -- 172.21.15.122:0/2707134599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d641f7de0 con 0x7f9d640b4a50 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d63e00640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 0x7f9d640c9020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400c3c0 tx=0x7f9d5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d60c00640 1 -- 172.21.15.122:0/2707134599 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54017070 con 0x7f9d640b4a50 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d60c00640 1 -- 172.21.15.122:0/2707134599 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d540053c0 con 0x7f9d640b4a50 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d641f8370 con 0x7f9d640b4a50 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d60c00640 1 -- 172.21.15.122:0/2707134599 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54003860 con 0x7f9d640b4a50 2024-09-17T05:25:14.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d641f8930 con 0x7f9d640b4a50 2024-09-17T05:25:14.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d64069410 con 0x7f9d640b4a50 2024-09-17T05:25:14.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.442+0000 7f9d60c00640 1 -- 172.21.15.122:0/2707134599 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9d5401a070 con 0x7f9d640b4a50 2024-09-17T05:25:14.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.446+0000 7f9d60c00640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40073200 0x7f9d400756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.446+0000 7f9d60c00640 1 -- 172.21.15.122:0/2707134599 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9d540934e0 con 0x7f9d640b4a50 2024-09-17T05:25:14.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.446+0000 7f9d5b400640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40073200 0x7f9d400756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.446+0000 7f9d5b400640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40073200 0x7f9d400756c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0af000 tx=0x7f9d4c01b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.450+0000 7f9d60c00640 1 -- 172.21.15.122:0/2707134599 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9d5405dea0 con 0x7f9d640b4a50 2024-09-17T05:25:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.622+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d640acd40 con 0x7f9d640b4a50 2024-09-17T05:25:14.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.622+0000 7f9d60c00640 1 -- 172.21.15.122:0/2707134599 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7f9d540628d0 con 0x7f9d640b4a50 2024-09-17T05:25:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:14.627 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":22,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:05.140074+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40073200 msgr2=0x7f9d400756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40073200 0x7f9d400756c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0af000 tx=0x7f9d4c01b090 comp rx=0 tx=0).stop 2024-09-17T05:25:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 msgr2=0x7f9d640c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 0x7f9d640c9020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400c3c0 tx=0x7f9d5400c880 comp rx=0 tx=0).stop 2024-09-17T05:25:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 shutdown_connections 2024-09-17T05:25:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40073200 0x7f9d400756c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6415db50 0x7f9d6413d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d640cd100 0x7f9d6406ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 --2- 172.21.15.122:0/2707134599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d640b4a50 0x7f9d640c9020 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 >> 172.21.15.122:0/2707134599 conn(0x7f9d640b18f0 msgr2=0x7f9d640b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 shutdown_connections 2024-09-17T05:25:14.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.626+0000 7f9d6b000640 1 -- 172.21.15.122:0/2707134599 wait complete. 2024-09-17T05:25:14.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T05:25:14.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T05:25:14.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T05:25:14.650 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T05:25:14.650 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T05:25:14.650 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T05:25:14.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T05:25:14.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T05:25:14.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.894+0000 7fb1ea800640 1 -- 172.21.15.122:0/1134196247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1cc003a10 msgr2=0x7fb1cc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.894+0000 7fb1ea800640 1 --2- 172.21.15.122:0/1134196247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1cc003a10 0x7fb1cc003e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009b90 tx=0x7fb1d801d2e0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.894+0000 7fb1e9e00640 1 -- 172.21.15.122:0/1134196247 reap_dead start 2024-09-17T05:25:14.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.894+0000 7fb1ea800640 1 -- 172.21.15.122:0/1134196247 shutdown_connections 2024-09-17T05:25:14.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.894+0000 7fb1ea800640 1 -- 172.21.15.122:0/1134196247 >> 172.21.15.122:0/1134196247 conn(0x7fb1e40b1600 msgr2=0x7fb1e40b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.894+0000 7fb1ea800640 1 -- 172.21.15.122:0/1134196247 shutdown_connections 2024-09-17T05:25:14.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.894+0000 7fb1ea800640 1 -- 172.21.15.122:0/1134196247 wait complete. 2024-09-17T05:25:14.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.898+0000 7fb1ea800640 1 Processor -- start 2024-09-17T05:25:14.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.898+0000 7fb1ea800640 1 -- start start 2024-09-17T05:25:14.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1ea800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4145fb0 0x7fb1e40c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1ea800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e40cc1a0 0x7fb1e40cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1ea800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40c7040 0x7fb1e40cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1ea800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4162110 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1ea800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e4161f90 con 0x7fb1e4145fb0 2024-09-17T05:25:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1ea800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e4073830 con 0x7fb1e40c7040 2024-09-17T05:25:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e40cc1a0 0x7fb1e40cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e40cc1a0 0x7fb1e40cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42780/0 (socket says 172.21.15.122:42780) 2024-09-17T05:25:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 -- 172.21.15.122:0/1867812451 learned_addr learned my addr 172.21.15.122:0/1867812451 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e9e00640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40c7040 0x7fb1e40cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 -- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40c7040 msgr2=0x7fb1e40cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40c7040 0x7fb1e40cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 -- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4145fb0 msgr2=0x7fb1e40c6b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:25:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e4145fb0 0x7fb1e40c6b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 -- 172.21.15.122:0/1867812451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e40cc6f0 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fb1e8a00640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e40cc1a0 0x7fb1e40cb670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b420 tx=0x7fb1d000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fceada00640 1 -- 172.21.15.122:0/343210021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8073f30 msgr2=0x7fcea8070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fceada00640 1 --2- 172.21.15.122:0/343210021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8073f30 0x7fcea8070ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009bf0 tx=0x7fcea001cb50 comp rx=0 tx=0).stop 2024-09-17T05:25:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fceada00640 1 -- 172.21.15.122:0/343210021 shutdown_connections 2024-09-17T05:25:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fceada00640 1 --2- 172.21.15.122:0/343210021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8073f30 0x7fcea8070ff0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fceada00640 1 -- 172.21.15.122:0/343210021 >> 172.21.15.122:0/343210021 conn(0x7fcea806dd30 msgr2=0x7fcea806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.902+0000 7fceada00640 1 -- 172.21.15.122:0/343210021 shutdown_connections 2024-09-17T05:25:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.906+0000 7fceada00640 1 -- 172.21.15.122:0/343210021 wait complete. 2024-09-17T05:25:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.906+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0011830 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.906+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e40cc9d0 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.906+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e415a480 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.906+0000 7fceada00640 1 Processor -- start 2024-09-17T05:25:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.906+0000 7fceada00640 1 -- start start 2024-09-17T05:25:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.906+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1e4161840 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.906+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e4161840 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fc987e00640 1 -- 172.21.15.122:0/2193639846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9800deb30 msgr2=0x7fc9800342e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fceada00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8070ba0 0x7fcea814c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fc987e00640 1 --2- 172.21.15.122:0/2193639846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9800deb30 0x7fc9800342e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b3e0 tx=0x7fc97801d0c0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fc987e00640 1 -- 172.21.15.122:0/2193639846 shutdown_connections 2024-09-17T05:25:14.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fceada00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea8073f30 0x7fcea8154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fceada00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcea81518b0 0x7fcea81fa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fceada00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcea8168a40 con 0x7fcea81518b0 2024-09-17T05:25:14.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fceada00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcea81688c0 con 0x7fcea8073f30 2024-09-17T05:25:14.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fceada00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcea8168bc0 con 0x7fcea8070ba0 2024-09-17T05:25:14.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fc987e00640 1 --2- 172.21.15.122:0/2193639846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc980035b50 0x7fc980039710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8070ba0 0x7fcea814c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8070ba0 0x7fcea814c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33256/0 (socket says 172.21.15.122:33256) 2024-09-17T05:25:14.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 -- 172.21.15.122:0/1594858372 learned_addr learned my addr 172.21.15.122:0/1594858372 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7400640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea8073f30 0x7fcea8154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fc987e00640 1 --2- 172.21.15.122:0/2193639846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9800deb30 0x7fc9800342e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fc987e00640 1 -- 172.21.15.122:0/2193639846 >> 172.21.15.122:0/2193639846 conn(0x7fc98001dbd0 msgr2=0x7fc9800549c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fc987e00640 1 -- 172.21.15.122:0/2193639846 shutdown_connections 2024-09-17T05:25:14.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fc987e00640 1 -- 172.21.15.122:0/2193639846 wait complete. 2024-09-17T05:25:14.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0011cc0 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 -- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea8073f30 msgr2=0x7fcea8154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea8073f30 0x7fcea8154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 -- 172.21.15.122:0/1594858372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcea81518b0 msgr2=0x7fcea81fa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcea81518b0 0x7fcea81fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fbf67e00640 1 -- 172.21.15.122:0/3128474018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf4c005220 msgr2=0x7fbf4c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fbf67e00640 1 --2- 172.21.15.122:0/3128474018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf4c005220 0x7fbf4c00b9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbf680cf4e0 tx=0x7fbf6001c940 comp rx=0 tx=0).stop 2024-09-17T05:25:14.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fbf67400640 1 -- 172.21.15.122:0/3128474018 reap_dead start 2024-09-17T05:25:14.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 -- 172.21.15.122:0/1594858372 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea81f7e40 con 0x7fcea8070ba0 2024-09-17T05:25:14.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fbf67e00640 1 -- 172.21.15.122:0/3128474018 shutdown_connections 2024-09-17T05:25:14.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fbf67e00640 1 -- 172.21.15.122:0/3128474018 >> 172.21.15.122:0/3128474018 conn(0x7fbf680b1600 msgr2=0x7fbf680b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fbf67e00640 1 -- 172.21.15.122:0/3128474018 shutdown_connections 2024-09-17T05:25:14.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fbf67e00640 1 -- 172.21.15.122:0/3128474018 wait complete. 2024-09-17T05:25:14.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7400640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcea8073f30 0x7fcea8154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:14.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.910+0000 7fcea7e00640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8070ba0 0x7fcea814c7f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c460 tx=0x7fce9800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98017070 con 0x7fcea8070ba0 2024-09-17T05:25:14.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce980053e0 con 0x7fcea8070ba0 2024-09-17T05:25:14.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98003860 con 0x7fcea8070ba0 2024-09-17T05:25:14.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb1d001b460 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc987e00640 1 Processor -- start 2024-09-17T05:25:14.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc987e00640 1 -- start start 2024-09-17T05:25:14.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea8153350 con 0x7fcea8070ba0 2024-09-17T05:25:14.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea81f80e0 con 0x7fcea8070ba0 2024-09-17T05:25:14.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e1600640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb1c4073200 0x7fb1c40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc233200640 1 -- 172.21.15.122:0/3135465633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc22c159cc0 msgr2=0x7fc22c072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc233200640 1 --2- 172.21.15.122:0/3135465633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc22c159cc0 0x7fc22c072680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc220009840 tx=0x7fc22001d7f0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fbf67e00640 1 Processor -- start 2024-09-17T05:25:14.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb1d0094810 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc233200640 1 -- 172.21.15.122:0/3135465633 shutdown_connections 2024-09-17T05:25:14.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc233200640 1 --2- 172.21.15.122:0/3135465633 >> v2:172.21.15.122:3300/0 conn(0x7fc22c0cd600 0x7fc22c0cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc233200640 1 --2- 172.21.15.122:0/3135465633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc22c159cc0 0x7fc22c072680 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc233200640 1 -- 172.21.15.122:0/3135465633 >> 172.21.15.122:0/3135465633 conn(0x7fc22c0cfa30 msgr2=0x7fc22c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e9400640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb1c4073200 0x7fb1c40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e1600640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7fb1c407b400 0x7fb1c407d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 --> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1c407ded0 con 0x7fb1c407b400 2024-09-17T05:25:14.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb1d0094ca0 con 0x7fb1e40cc1a0 2024-09-17T05:25:14.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc233200640 1 -- 172.21.15.122:0/3135465633 shutdown_connections 2024-09-17T05:25:14.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc233200640 1 -- 172.21.15.122:0/3135465633 wait complete. 2024-09-17T05:25:14.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fbf67e00640 1 -- start start 2024-09-17T05:25:14.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc987e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc980035b50 0x7fc980020750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e9e00640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7fb1c407b400 0x7fb1c407d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fb1e9400640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb1c4073200 0x7fb1c40756c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80ae000 tx=0x7fb1d801a080 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc987e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9800deb30 0x7fc9800cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc987e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9800def10 0x7fc9800cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc986000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9800deb30 0x7fc9800cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcea814cd30 con 0x7fcea8070ba0 2024-09-17T05:25:14.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9800ebd80 con 0x7fc980035b50 2024-09-17T05:25:14.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fb1e9e00640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7fb1c407b400 0x7fb1c407d820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9800078e0 con 0x7fc9800deb30 2024-09-17T05:25:14.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc9800ebf00 con 0x7fc9800def10 2024-09-17T05:25:14.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc986000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9800deb30 0x7fc9800cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43452/0 (socket says 172.21.15.122:43452) 2024-09-17T05:25:14.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc986000640 1 -- 172.21.15.122:0/606907315 learned_addr learned my addr 172.21.15.122:0/606907315 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987400640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9800def10 0x7fc9800cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf4c005220 0x7fbf680c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987400640 1 -- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9800deb30 msgr2=0x7fc9800cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987400640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9800deb30 0x7fc9800cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987400640 1 -- 172.21.15.122:0/606907315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc980035b50 msgr2=0x7fc980020750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987400640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc980035b50 0x7fc980020750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987400640 1 -- 172.21.15.122:0/606907315 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98017d660 con 0x7fc9800def10 2024-09-17T05:25:14.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fce9801a070 con 0x7fcea8070ba0 2024-09-17T05:25:14.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc987400640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9800def10 0x7fc9800cf180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc98806b0f0 tx=0x7fc98806a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf68158bb0 0x7fbf68200910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6815b8c0 0x7fbf6815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf680cdc10 con 0x7fbf4c005220 2024-09-17T05:25:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf680cda90 con 0x7fbf68158bb0 2024-09-17T05:25:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbf680cdd90 con 0x7fbf6815b8c0 2024-09-17T05:25:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf4c005220 0x7fbf680c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf4c005220 0x7fbf680c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42834/0 (socket says 172.21.15.122:42834) 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 -- 172.21.15.122:0/177949544 learned_addr learned my addr 172.21.15.122:0/177949544 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67400640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6815b8c0 0x7fbf6815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 -- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6815b8c0 msgr2=0x7fbf6815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6815b8c0 0x7fbf6815bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 -- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf68158bb0 msgr2=0x7fbf68200910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf68158bb0 0x7fbf68200910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 -- 172.21.15.122:0/177949544 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf68073810 con 0x7fbf4c005220 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67400640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6815b8c0 0x7fbf6815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:14.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc441800640 1 -- 172.21.15.122:0/1401555824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc43c0d2060 msgr2=0x7fc43c0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fce9fe00640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fce88073200 0x7fce880756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fce98093490 con 0x7fcea8070ba0 2024-09-17T05:25:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fce9fe00640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7fce8807b3a0 0x7fce8807d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 <== osd.1 v2:172.21.15.122:6824/2252207329 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1c407ded0 con 0x7fb1c407b400 2024-09-17T05:25:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf66a00640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf4c005220 0x7fbf680c9820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800e9c0 tx=0x7fbf5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc441800640 1 --2- 172.21.15.122:0/1401555824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc43c0d2060 0x7fc43c0cf660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009840 tx=0x7fc42c01c820 comp rx=0 tx=0).stop 2024-09-17T05:25:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc441800640 1 -- 172.21.15.122:0/1401555824 shutdown_connections 2024-09-17T05:25:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc441800640 1 --2- 172.21.15.122:0/1401555824 >> v2:172.21.15.122:3300/0 conn(0x7fc43c0b7ab0 0x7fc43c0b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc441800640 1 --2- 172.21.15.122:0/1401555824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc43c0d2060 0x7fc43c0cf660 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.914+0000 7fc441800640 1 -- 172.21.15.122:0/1401555824 >> 172.21.15.122:0/1401555824 conn(0x7fc43c0d3fe0 msgr2=0x7fc43c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc441800640 1 -- 172.21.15.122:0/1401555824 shutdown_connections 2024-09-17T05:25:14.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc441800640 1 -- 172.21.15.122:0/1401555824 wait complete. 2024-09-17T05:25:14.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fcea7400640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fce88073200 0x7fce880756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 --> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce9800bf30 con 0x7fce8807b3a0 2024-09-17T05:25:14.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fce9809a050 con 0x7fcea8070ba0 2024-09-17T05:25:14.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fcead000640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7fce8807b3a0 0x7fce8807d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988090070 con 0x7fc9800def10 2024-09-17T05:25:14.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9880746e0 con 0x7fc9800def10 2024-09-17T05:25:14.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988074b70 con 0x7fc9800def10 2024-09-17T05:25:14.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5800cc00 con 0x7fbf4c005220 2024-09-17T05:25:14.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf5800ce90 con 0x7fbf4c005220 2024-09-17T05:25:14.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58005260 con 0x7fbf4c005220 2024-09-17T05:25:14.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf680c9d60 con 0x7fbf4c005220 2024-09-17T05:25:14.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf68166d90 con 0x7fbf4c005220 2024-09-17T05:25:14.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf40001040 con 0x7fbf4c005220 2024-09-17T05:25:14.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.918+0000 7fc233200640 1 Processor -- start 2024-09-17T05:25:14.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc441800640 1 Processor -- start 2024-09-17T05:25:14.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98017d940 con 0x7fc9800def10 2024-09-17T05:25:14.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc98017df00 con 0x7fc9800def10 2024-09-17T05:25:14.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fcead000640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7fce8807b3a0 0x7fce8807d7c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc441800640 1 -- start start 2024-09-17T05:25:14.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fcea7400640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fce88073200 0x7fce880756c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcea81502b0 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 <== osd.0 v2:172.21.15.122:6808/448781951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce9800bf30 con 0x7fce8807b3a0 2024-09-17T05:25:14.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc98001e860 con 0x7fc9800def10 2024-09-17T05:25:14.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc988093070 con 0x7fc9800def10 2024-09-17T05:25:14.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf58005400 con 0x7fbf4c005220 2024-09-17T05:25:14.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fd087e00640 1 -- 172.21.15.122:0/3413194072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd080070420 msgr2=0x7fd0800cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fd087e00640 1 --2- 172.21.15.122:0/3413194072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd080070420 0x7fd0800cebf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd074009840 tx=0x7fd07401c700 comp rx=0 tx=0).stop 2024-09-17T05:25:14.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fd087e00640 1 -- 172.21.15.122:0/3413194072 shutdown_connections 2024-09-17T05:25:14.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fd087e00640 1 --2- 172.21.15.122:0/3413194072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd080070420 0x7fd0800cebf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fd087e00640 1 -- 172.21.15.122:0/3413194072 >> 172.21.15.122:0/3413194072 conn(0x7fd0800b18f0 msgr2=0x7fd0800b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc233200640 1 -- start start 2024-09-17T05:25:14.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc233200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc22c0cd600 0x7fc22c1fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc233200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc22c0cf570 0x7fc22c1fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc233200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc22c159cc0 0x7fc22c0c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc233200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c06f610 con 0x7fc22c0cd600 2024-09-17T05:25:14.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc233200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c06f490 con 0x7fc22c0cf570 2024-09-17T05:25:14.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc233200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c06f790 con 0x7fc22c159cc0 2024-09-17T05:25:14.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc231400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc22c0cf570 0x7fc22c1fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc231400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc22c0cf570 0x7fc22c1fc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43474/0 (socket says 172.21.15.122:43474) 2024-09-17T05:25:14.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc232800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc22c159cc0 0x7fc22c0c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc231400640 1 -- 172.21.15.122:0/3837886025 learned_addr learned my addr 172.21.15.122:0/3837886025 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc441800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43c0b7ab0 0x7fc43c200670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc441800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc43c0d2060 0x7fc43c1fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc441800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc43c0d2440 0x7fc43c0d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc441800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c16b610 con 0x7fc43c0d2440 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc441800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c16b490 con 0x7fc43c0b7ab0 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc441800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c16b790 con 0x7fc43c0d2060 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc43b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc43c0d2440 0x7fc43c0d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc43b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc43c0d2440 0x7fc43c0d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42836/0 (socket says 172.21.15.122:42836) 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.922+0000 7fc43b400640 1 -- 172.21.15.122:0/3079949724 learned_addr learned my addr 172.21.15.122:0/3079949724 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc43b400640 1 -- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc43c0d2060 msgr2=0x7fc43c1fbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc43b400640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc43c0d2060 0x7fc43c1fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc43b400640 1 -- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43c0b7ab0 msgr2=0x7fc43c200670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc43b400640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc43c0b7ab0 0x7fc43c200670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc43b400640 1 -- 172.21.15.122:0/3079949724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c200bb0 con 0x7fc43c0d2440 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc43b400640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc43c0d2440 0x7fc43c0d09f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc43400b9a0 tx=0x7fc43400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434019070 con 0x7fc43c0d2440 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc43400ccb0 con 0x7fc43c0d2440 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340146a0 con 0x7fc43c0d2440 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c15f930 con 0x7fc43c0d2440 2024-09-17T05:25:14.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.926+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c15fef0 con 0x7fc43c0d2440 2024-09-17T05:25:14.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf5ea00640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf48073230 0x7fbf480756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbf58095050 con 0x7fbf4c005220 2024-09-17T05:25:14.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fd087e00640 1 -- 172.21.15.122:0/3413194072 shutdown_connections 2024-09-17T05:25:14.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc43c15f6e0 con 0x7fc43c0d2440 2024-09-17T05:25:14.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf66000640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf48073230 0x7fbf480756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf5ea00640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7fbf4807b3d0 0x7fbf4807d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 --> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf4807dea0 con 0x7fbf4807b3d0 2024-09-17T05:25:14.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc231e00640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc22c0cd600 0x7fc22c1fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbf5805c060 con 0x7fbf4c005220 2024-09-17T05:25:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf67400640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7fbf4807b3d0 0x7fbf4807d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc97ea00640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc968073200 0x7fc9680756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf66000640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf48073230 0x7fbf480756f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf680c4440 tx=0x7fbf54001160 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc98810f4d0 con 0x7fc9800def10 2024-09-17T05:25:14.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc986a00640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc968073200 0x7fc9680756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc97ea00640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7fc96807b420 0x7fc96807d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf67400640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7fbf4807b3d0 0x7fbf4807d7f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fd087e00640 1 -- 172.21.15.122:0/3413194072 wait complete. 2024-09-17T05:25:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 --> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc98806fe50 con 0x7fc96807b420 2024-09-17T05:25:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc98810f8c0 con 0x7fc9800def10 2024-09-17T05:25:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fd087e00640 1 Processor -- start 2024-09-17T05:25:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fd087e00640 1 -- start start 2024-09-17T05:25:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 <== osd.3 v2:172.21.15.122:6800/3418036108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf4807dea0 con 0x7fbf4807b3d0 2024-09-17T05:25:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.930+0000 7fc986000640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7fc96807b420 0x7fc96807d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc231e00640 1 -- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc22c159cc0 msgr2=0x7fc22c0c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc231e00640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc22c159cc0 0x7fc22c0c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc231e00640 1 -- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc22c0cf570 msgr2=0x7fc22c1fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc231e00640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc22c0cf570 0x7fc22c1fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc231e00640 1 -- 172.21.15.122:0/3837886025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c1fccc0 con 0x7fc22c0cd600 2024-09-17T05:25:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc232800640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc22c159cc0 0x7fc22c0c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc986a00640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc968073200 0x7fc9680756c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9780ae000 tx=0x7fc978019e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc986000640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7fc96807b420 0x7fc96807d840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd087e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd080070420 0x7fd08013ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc43401c070 con 0x7fc43c0d2440 2024-09-17T05:25:14.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd087e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd080144830 0x7fd08013ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc231e00640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc22c0cd600 0x7fc22c1fbcd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc2200ad000 tx=0x7fc2200a6ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc433400640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc418073200 0x7fc4180756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd087e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd08013d270 0x7fd0801f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc434093fe0 con 0x7fc43c0d2440 2024-09-17T05:25:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc433400640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7fc41807b420 0x7fc41807d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 --> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc434012e50 con 0x7fc41807b420 2024-09-17T05:25:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc4340965e0 con 0x7fc43c0d2440 2024-09-17T05:25:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc43aa00640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc418073200 0x7fc4180756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd084a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd080144830 0x7fd08013ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd085e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd08013d270 0x7fd0801f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd084a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd080144830 0x7fd08013ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43510/0 (socket says 172.21.15.122:43510) 2024-09-17T05:25:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd085e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd08013d270 0x7fd0801f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42882/0 (socket says 172.21.15.122:42882) 2024-09-17T05:25:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd085400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd080070420 0x7fd08013ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd085400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd080070420 0x7fd08013ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33316/0 (socket says 172.21.15.122:33316) 2024-09-17T05:25:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd084a00640 1 -- 172.21.15.122:0/951109482 learned_addr learned my addr 172.21.15.122:0/951109482 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800cde70 con 0x7fd08013d270 2024-09-17T05:25:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200b3040 con 0x7fc22c0cd600 2024-09-17T05:25:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 <== osd.2 v2:172.21.15.122:6816/1463445108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc98806fe50 con 0x7fc96807b420 2024-09-17T05:25:14.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800cdcf0 con 0x7fd080144830 2024-09-17T05:25:14.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2200043e0 con 0x7fc22c0cd600 2024-09-17T05:25:14.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200052d0 con 0x7fc22c0cd600 2024-09-17T05:25:14.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd0800cdff0 con 0x7fd080070420 2024-09-17T05:25:14.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc43aa00640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc418073200 0x7fc4180756c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0af000 tx=0x7fc42c01b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7f8049400640 1 -- 172.21.15.122:0/752848203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 msgr2=0x7f802c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7f8049400640 1 --2- 172.21.15.122:0/752848203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 0x7f802c00b9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f804406bfd0 tx=0x7f8044098c40 comp rx=0 tx=0).stop 2024-09-17T05:25:14.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c0c9900 con 0x7fc22c0cd600 2024-09-17T05:25:14.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7f8049400640 1 -- 172.21.15.122:0/752848203 shutdown_connections 2024-09-17T05:25:14.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7f8049400640 1 --2- 172.21.15.122:0/752848203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 0x7f802c00b9d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7f8049400640 1 -- 172.21.15.122:0/752848203 >> 172.21.15.122:0/752848203 conn(0x7f803c01eaa0 msgr2=0x7f803c055830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd085e00640 1 -- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd080070420 msgr2=0x7fd08013ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fd085e00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd080070420 0x7fd08013ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd085e00640 1 -- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd080144830 msgr2=0x7fd08013ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd085e00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd080144830 0x7fd08013ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd085e00640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd080138bf0 con 0x7fd08013d270 2024-09-17T05:25:14.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.934+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c0c8090 con 0x7fc22c0cd600 2024-09-17T05:25:14.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7f8049400640 1 -- 172.21.15.122:0/752848203 shutdown_connections 2024-09-17T05:25:14.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7f8049400640 1 -- 172.21.15.122:0/752848203 wait complete. 2024-09-17T05:25:14.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd085e00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd08013d270 0x7fd0801f72d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0780021b0 tx=0x7fd07800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd084a00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd080144830 0x7fd08013ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:14.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07801e070 con 0x7fd08013d270 2024-09-17T05:25:14.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd07800a930 con 0x7fd08013d270 2024-09-17T05:25:14.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07800ada0 con 0x7fd08013d270 2024-09-17T05:25:14.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0801f7fa0 con 0x7fd08013d270 2024-09-17T05:25:14.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd080139170 con 0x7fd08013d270 2024-09-17T05:25:14.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.938+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd08013e610 con 0x7fd08013d270 2024-09-17T05:25:14.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7fc432000640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7fc41807b420 0x7fc41807d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8049400640 1 Processor -- start 2024-09-17T05:25:14.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8049400640 1 -- start start 2024-09-17T05:25:14.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8049400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 0x7f803c0e1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8049400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803c0f2fc0 0x7f803c0deb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7fc432000640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7fc41807b420 0x7fc41807d840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8049400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f803c00bf80 0x7f803c00c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc22c0cdab0 con 0x7fc22c0cd600 2024-09-17T05:25:14.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8043e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 0x7f803c0e1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8043e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 0x7f803c0e1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43516/0 (socket says 172.21.15.122:43516) 2024-09-17T05:25:14.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8043400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803c0f2fc0 0x7f803c0deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8043400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803c0f2fc0 0x7f803c0deb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33326/0 (socket says 172.21.15.122:33326) 2024-09-17T05:25:14.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f804aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f803c00bf80 0x7f803c00c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f804aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f803c00bf80 0x7f803c00c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42884/0 (socket says 172.21.15.122:42884) 2024-09-17T05:25:14.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8043e00640 1 -- 172.21.15.122:0/3035022042 learned_addr learned my addr 172.21.15.122:0/3035022042 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:14.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0e6960 con 0x7f803c00bf80 2024-09-17T05:25:14.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0e67e0 con 0x7f802c0051d0 2024-09-17T05:25:14.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.942+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f803c0e6ae0 con 0x7f803c0f2fc0 2024-09-17T05:25:14.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 <== osd.6 v2:172.21.15.173:6826/2596490949 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc434012e50 con 0x7fc41807b420 2024-09-17T05:25:14.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd0780223f0 con 0x7fd08013d270 2024-09-17T05:25:14.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8043e00640 1 -- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803c0f2fc0 msgr2=0x7f803c0deb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8043e00640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803c0f2fc0 0x7f803c0deb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8043e00640 1 -- 172.21.15.122:0/3035022042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f803c00bf80 msgr2=0x7f803c00c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8043e00640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f803c00bf80 0x7f803c00c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8043e00640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c0e4dd0 con 0x7f802c0051d0 2024-09-17T05:25:14.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f804aa00640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f803c00bf80 0x7f803c00c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:14.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fd07d600640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd060073230 0x7fd0600756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd0780940d0 con 0x7fd08013d270 2024-09-17T05:25:14.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8043e00640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 0x7f803c0e1920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f803800b3c0 tx=0x7f803800b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fd07d600640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7fd06007b430 0x7fd06007d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd07801bec0 con 0x7fd06007b430 2024-09-17T05:25:14.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd078094480 con 0x7fd08013d270 2024-09-17T05:25:14.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038011830 con 0x7f802c0051d0 2024-09-17T05:25:14.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fd085400640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd060073230 0x7fd0600756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8038016070 con 0x7f802c0051d0 2024-09-17T05:25:14.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038011d60 con 0x7f802c0051d0 2024-09-17T05:25:14.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c0e50b0 con 0x7f802c0051d0 2024-09-17T05:25:14.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fd085400640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd060073230 0x7fd0600756f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd0740af000 tx=0x7fd07400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c0e4aa0 con 0x7f802c0051d0 2024-09-17T05:25:14.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 --> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1e40cb430 con 0x7fb1c407b400 2024-09-17T05:25:14.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fd084a00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7fd06007b430 0x7fd06007d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.946+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc220005470 con 0x7fc22c0cd600 2024-09-17T05:25:14.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc226000640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc210073200 0x7fc2100756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc22012d900 con 0x7fc22c0cd600 2024-09-17T05:25:14.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc226000640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc21007b420 0x7fc21007d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 --> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc220007e80 con 0x7fc21007b420 2024-09-17T05:25:14.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc2201322a0 con 0x7fc22c0cd600 2024-09-17T05:25:14.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f803801b460 con 0x7f802c0051d0 2024-09-17T05:25:14.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fd084a00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7fd06007b430 0x7fd06007d850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc232800640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc21007b420 0x7fc21007d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc231400640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc210073200 0x7fc2100756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f803c03b740 con 0x7f802c0051d0 2024-09-17T05:25:14.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc232800640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc21007b420 0x7fc21007d840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fc231400640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc210073200 0x7fc2100756c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc21c001ec0 tx=0x7fc21c001b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fb1e1600640 1 -- 172.21.15.122:0/1867812451 <== osd.1 v2:172.21.15.122:6824/2252207329 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb1e40cb430 con 0x7fb1c407b400 2024-09-17T05:25:14.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.950+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 <== osd.4 v2:172.21.15.173:6810/1969238728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd07801bec0 con 0x7fd06007b430 2024-09-17T05:25:14.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 <== osd.5 v2:172.21.15.173:6802/2545281431 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc220007e80 con 0x7fc21007b420 2024-09-17T05:25:14.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f8037e00640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8024073230 0x7f80240756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8038093d70 con 0x7f802c0051d0 2024-09-17T05:25:14.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f8043400640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8024073230 0x7f80240756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7fb1c407b400 msgr2=0x7fb1c407d820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7fb1ea800640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7fb1c407b400 0x7fb1c407d820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb1c4073200 msgr2=0x7fb1c40756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7fb1ea800640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb1c4073200 0x7fb1c40756c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80ae000 tx=0x7fb1d801a080 comp rx=0 tx=0).stop 2024-09-17T05:25:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e40cc1a0 msgr2=0x7fb1e40cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7fb1ea800640 1 --2- 172.21.15.122:0/1867812451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1e40cc1a0 0x7fb1e40cb670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b420 tx=0x7fb1d000b8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f8037e00640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f802407b3d0 0x7f802407d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f802407dea0 con 0x7f802407b3d0 2024-09-17T05:25:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8038096390 con 0x7f802c0051d0 2024-09-17T05:25:14.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7fb1e9e00640 1 -- 172.21.15.122:0/1867812451 reap_dead start 2024-09-17T05:25:14.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f804aa00640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f802407b3d0 0x7f802407d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:14.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f8043400640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8024073230 0x7f80240756f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f803c0e4070 tx=0x7f8030005f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f804aa00640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f802407b3d0 0x7f802407d7f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:14.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.954+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 <== osd.7 v2:172.21.15.173:6818/1087122755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f802407dea0 con 0x7f802407b3d0 2024-09-17T05:25:14.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 shutdown_connections 2024-09-17T05:25:14.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 >> 172.21.15.122:0/1867812451 conn(0x7fb1e40b1600 msgr2=0x7fb1e4075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 --> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcea8070ff0 con 0x7fce8807b3a0 2024-09-17T05:25:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 shutdown_connections 2024-09-17T05:25:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fb1ea800640 1 -- 172.21.15.122:0/1867812451 wait complete. 2024-09-17T05:25:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fce9fe00640 1 -- 172.21.15.122:0/1594858372 <== osd.0 v2:172.21.15.122:6808/448781951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcea8070ff0 con 0x7fce8807b3a0 2024-09-17T05:25:14.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7fce8807b3a0 msgr2=0x7fce8807d7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7fce8807b3a0 0x7fce8807d7c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fce88073200 msgr2=0x7fce880756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fce88073200 0x7fce880756c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcea81502b0 tx=0x7fce94009040 comp rx=0 tx=0).stop 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8070ba0 msgr2=0x7fcea814c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 --2- 172.21.15.122:0/1594858372 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcea8070ba0 0x7fcea814c7f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c460 tx=0x7fce9800c920 comp rx=0 tx=0).stop 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fcead000640 1 -- 172.21.15.122:0/1594858372 reap_dead start 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 shutdown_connections 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 >> 172.21.15.122:0/1594858372 conn(0x7fcea806dd30 msgr2=0x7fcea815d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 shutdown_connections 2024-09-17T05:25:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.958+0000 7fceada00640 1 -- 172.21.15.122:0/1594858372 wait complete. 2024-09-17T05:25:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.962+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 --> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf40002de0 con 0x7fbf4807b3d0 2024-09-17T05:25:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.962+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 --> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc980039710 con 0x7fc96807b420 2024-09-17T05:25:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc97ea00640 1 -- 172.21.15.122:0/606907315 <== osd.2 v2:172.21.15.122:6816/1463445108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc980039710 con 0x7fc96807b420 2024-09-17T05:25:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7fc96807b420 msgr2=0x7fc96807d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7fc96807b420 0x7fc96807d840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc968073200 msgr2=0x7fc9680756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc968073200 0x7fc9680756c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9780ae000 tx=0x7fc978019e60 comp rx=0 tx=0).stop 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9800def10 msgr2=0x7fc9800cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 --2- 172.21.15.122:0/606907315 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9800def10 0x7fc9800cf180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc98806b0f0 tx=0x7fc98806a110 comp rx=0 tx=0).stop 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987400640 1 -- 172.21.15.122:0/606907315 reap_dead start 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 shutdown_connections 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 >> 172.21.15.122:0/606907315 conn(0x7fc98001dbd0 msgr2=0x7fc9800334d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 shutdown_connections 2024-09-17T05:25:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf5ea00640 1 -- 172.21.15.122:0/177949544 <== osd.3 v2:172.21.15.122:6800/3418036108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbf40002de0 con 0x7fbf4807b3d0 2024-09-17T05:25:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7fbf4807b3d0 msgr2=0x7fbf4807d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7fbf4807b3d0 0x7fbf4807d7f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf48073230 msgr2=0x7fbf480756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf48073230 0x7fbf480756f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf680c4440 tx=0x7fbf54001160 comp rx=0 tx=0).stop 2024-09-17T05:25:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf4c005220 msgr2=0x7fbf680c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 --2- 172.21.15.122:0/177949544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf4c005220 0x7fbf680c9820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800e9c0 tx=0x7fbf5800ee80 comp rx=0 tx=0).stop 2024-09-17T05:25:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67400640 1 -- 172.21.15.122:0/177949544 reap_dead start 2024-09-17T05:25:14.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 shutdown_connections 2024-09-17T05:25:14.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 >> 172.21.15.122:0/177949544 conn(0x7fbf680b1600 msgr2=0x7fbf68166040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 shutdown_connections 2024-09-17T05:25:14.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.966+0000 7fbf67e00640 1 -- 172.21.15.122:0/177949544 wait complete. 2024-09-17T05:25:14.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.970+0000 7fc987e00640 1 -- 172.21.15.122:0/606907315 wait complete. 2024-09-17T05:25:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.978+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 --> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd0800cf720 con 0x7fd06007b430 2024-09-17T05:25:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd07d600640 1 -- 172.21.15.122:0/951109482 <== osd.4 v2:172.21.15.173:6810/1969238728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd0800cf720 con 0x7fd06007b430 2024-09-17T05:25:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 --> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f803c03ab80 con 0x7f802407b3d0 2024-09-17T05:25:14.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 --> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc22c06ea60 con 0x7fc21007b420 2024-09-17T05:25:14.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 --> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc43c0b7ee0 con 0x7fc41807b420 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7fd06007b430 msgr2=0x7fd06007d850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd087e00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7fd06007b430 0x7fd06007d850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd060073230 msgr2=0x7fd0600756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd087e00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd060073230 0x7fd0600756f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd0740af000 tx=0x7fd07400b040 comp rx=0 tx=0).stop 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd08013d270 msgr2=0x7fd0801f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd087e00640 1 --2- 172.21.15.122:0/951109482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd08013d270 0x7fd0801f72d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd0780021b0 tx=0x7fd07800cb30 comp rx=0 tx=0).stop 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd085e00640 1 -- 172.21.15.122:0/951109482 reap_dead start 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 shutdown_connections 2024-09-17T05:25:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 >> 172.21.15.122:0/951109482 conn(0x7fd0800b18f0 msgr2=0x7fd08013f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc433400640 1 -- 172.21.15.122:0/3079949724 <== osd.6 v2:172.21.15.173:6826/2596490949 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc43c0b7ee0 con 0x7fc41807b420 2024-09-17T05:25:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8037e00640 1 -- 172.21.15.122:0/3035022042 <== osd.7 v2:172.21.15.173:6818/1087122755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f803c03ab80 con 0x7f802407b3d0 2024-09-17T05:25:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc226000640 1 -- 172.21.15.122:0/3837886025 <== osd.5 v2:172.21.15.173:6802/2545281431 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc22c06ea60 con 0x7fc21007b420 2024-09-17T05:25:14.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc21007b420 msgr2=0x7fc21007d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc233200640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc21007b420 0x7fc21007d840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7fc41807b420 msgr2=0x7fc41807d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7fc41807b420 0x7fc41807d840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f802407b3d0 msgr2=0x7f802407d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f802407b3d0 0x7f802407d7f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8024073230 msgr2=0x7f80240756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8024073230 0x7f80240756f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f803c0e4070 tx=0x7f8030005f30 comp rx=0 tx=0).stop 2024-09-17T05:25:14.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc418073200 msgr2=0x7fc4180756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 msgr2=0x7f803c0e1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 --2- 172.21.15.122:0/3035022042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f802c0051d0 0x7f803c0e1920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f803800b3c0 tx=0x7f803800b880 comp rx=0 tx=0).stop 2024-09-17T05:25:14.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc418073200 0x7fc4180756c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0af000 tx=0x7fc42c01b060 comp rx=0 tx=0).stop 2024-09-17T05:25:14.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc43c0d2440 msgr2=0x7fc43c0d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 --2- 172.21.15.122:0/3079949724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc43c0d2440 0x7fc43c0d09f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc43400b9a0 tx=0x7fc43400be60 comp rx=0 tx=0).stop 2024-09-17T05:25:14.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f804aa00640 1 -- 172.21.15.122:0/3035022042 reap_dead start 2024-09-17T05:25:14.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 shutdown_connections 2024-09-17T05:25:14.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 >> 172.21.15.122:0/3035022042 conn(0x7f803c01eaa0 msgr2=0x7f803c033470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc210073200 msgr2=0x7fc2100756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc43b400640 1 -- 172.21.15.122:0/3079949724 reap_dead start 2024-09-17T05:25:14.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 shutdown_connections 2024-09-17T05:25:14.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 shutdown_connections 2024-09-17T05:25:14.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 >> 172.21.15.122:0/3079949724 conn(0x7fc43c0d3fe0 msgr2=0x7fc43c06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:14.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.982+0000 7f8049400640 1 -- 172.21.15.122:0/3035022042 wait complete. 2024-09-17T05:25:14.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc233200640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc210073200 0x7fc2100756c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc21c001ec0 tx=0x7fc21c001b20 comp rx=0 tx=0).stop 2024-09-17T05:25:14.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 shutdown_connections 2024-09-17T05:25:14.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fd087e00640 1 -- 172.21.15.122:0/951109482 wait complete. 2024-09-17T05:25:14.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 shutdown_connections 2024-09-17T05:25:14.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc22c0cd600 msgr2=0x7fc22c1fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:14.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc233200640 1 --2- 172.21.15.122:0/3837886025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc22c0cd600 0x7fc22c1fbcd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc2200ad000 tx=0x7fc2200a6ee0 comp rx=0 tx=0).stop 2024-09-17T05:25:14.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc441800640 1 -- 172.21.15.122:0/3079949724 wait complete. 2024-09-17T05:25:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc232800640 1 -- 172.21.15.122:0/3837886025 reap_dead start 2024-09-17T05:25:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 shutdown_connections 2024-09-17T05:25:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 >> 172.21.15.122:0/3837886025 conn(0x7fc22c0cfa30 msgr2=0x7fc22c158550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 shutdown_connections 2024-09-17T05:25:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:14.986+0000 7fc233200640 1 -- 172.21.15.122:0/3837886025 wait complete. 2024-09-17T05:25:15.001 INFO:teuthology.orchestra.run.smithi122.stdout:64424509448 2024-09-17T05:25:15.002 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T05:25:15.003 INFO:teuthology.orchestra.run.smithi122.stdout:64424509446 2024-09-17T05:25:15.004 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T05:25:15.010 INFO:teuthology.orchestra.run.smithi122.stdout:64424509446 2024-09-17T05:25:15.011 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T05:25:15.014 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:15.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T05:25:15.020 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:15.021 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T05:25:15.023 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:15.024 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T05:25:15.024 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:15.025 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T05:25:15.036 INFO:teuthology.orchestra.run.smithi122.stdout:60129542152 2024-09-17T05:25:15.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T05:25:15.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.258+0000 7f9f83e00640 1 -- 172.21.15.122:0/972627500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f840cdb40 msgr2=0x7f9f8415e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.258+0000 7f9f83e00640 1 --2- 172.21.15.122:0/972627500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f840cdb40 0x7f9f8415e5e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009c80 tx=0x7f9f7c01cd70 comp rx=0 tx=0).stop 2024-09-17T05:25:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.258+0000 7f9f83e00640 1 -- 172.21.15.122:0/972627500 shutdown_connections 2024-09-17T05:25:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.258+0000 7f9f83e00640 1 --2- 172.21.15.122:0/972627500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f840cdb40 0x7f9f8415e5e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.258+0000 7f9f83e00640 1 --2- 172.21.15.122:0/972627500 >> v2:172.21.15.122:3300/0 conn(0x7f9f84070090 0x7f9f840cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.258+0000 7f9f83e00640 1 -- 172.21.15.122:0/972627500 >> 172.21.15.122:0/972627500 conn(0x7f9f840cfa30 msgr2=0x7f9f840cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.258+0000 7f9f83e00640 1 -- 172.21.15.122:0/972627500 shutdown_connections 2024-09-17T05:25:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.258+0000 7f9f83e00640 1 -- 172.21.15.122:0/972627500 wait complete. 2024-09-17T05:25:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.262+0000 7f9f83e00640 1 Processor -- start 2024-09-17T05:25:15.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.262+0000 7f9f83e00640 1 -- start start 2024-09-17T05:25:15.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.262+0000 7f3991200640 1 -- 172.21.15.122:0/2214759162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c06ece0 msgr2=0x7f398c0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.262+0000 7f3991200640 1 --2- 172.21.15.122:0/2214759162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c06ece0 0x7f398c0b3990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f397c009900 tx=0x7f397c01c880 comp rx=0 tx=0).stop 2024-09-17T05:25:15.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.262+0000 7f3991200640 1 -- 172.21.15.122:0/2214759162 shutdown_connections 2024-09-17T05:25:15.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.262+0000 7f3991200640 1 --2- 172.21.15.122:0/2214759162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c06ece0 0x7f398c0b3990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.262+0000 7f3991200640 1 --2- 172.21.15.122:0/2214759162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c161520 0x7f398c06e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.262+0000 7f3991200640 1 -- 172.21.15.122:0/2214759162 >> 172.21.15.122:0/2214759162 conn(0x7f398c0b18f0 msgr2=0x7f398c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f8406fcb0 0x7f9f84143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f84070090 0x7f9f84145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f840cdb40 0x7f9f84146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8406edd0 con 0x7f9f840cdb40 2024-09-17T05:25:15.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8406ec50 con 0x7f9f84070090 2024-09-17T05:25:15.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8406ef50 con 0x7f9f8406fcb0 2024-09-17T05:25:15.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f84070090 0x7f9f84145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f84070090 0x7f9f84145c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43532/0 (socket says 172.21.15.122:43532) 2024-09-17T05:25:15.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 -- 172.21.15.122:0/880705577 learned_addr learned my addr 172.21.15.122:0/880705577 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:15.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83400640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f840cdb40 0x7f9f84146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82a00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f8406fcb0 0x7f9f84143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f2023e00640 1 -- 172.21.15.122:0/1618647653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b5870 msgr2=0x7f2024165670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f2023e00640 1 --2- 172.21.15.122:0/1618647653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b5870 0x7f2024165670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2014009840 tx=0x7f201401c860 comp rx=0 tx=0).stop 2024-09-17T05:25:15.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f3991200640 1 -- 172.21.15.122:0/2214759162 shutdown_connections 2024-09-17T05:25:15.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 -- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f8406fcb0 msgr2=0x7f9f84143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f3991200640 1 -- 172.21.15.122:0/2214759162 wait complete. 2024-09-17T05:25:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f8406fcb0 0x7f9f84143950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 -- 172.21.15.122:0/880705577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f840cdb40 msgr2=0x7f9f84146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f840cdb40 0x7f9f84146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 -- 172.21.15.122:0/880705577 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f840b4d40 con 0x7f9f84070090 2024-09-17T05:25:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82a00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f8406fcb0 0x7f9f84143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83400640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f840cdb40 0x7f9f84146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f82000640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f84070090 0x7f9f84145c50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000d980 tx=0x7f9f7000de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f2023e00640 1 -- 172.21.15.122:0/1618647653 shutdown_connections 2024-09-17T05:25:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f2023e00640 1 --2- 172.21.15.122:0/1618647653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b5870 0x7f2024165670 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f2023e00640 1 --2- 172.21.15.122:0/1618647653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20240b4d80 0x7f20240b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f2023e00640 1 -- 172.21.15.122:0/1618647653 >> 172.21.15.122:0/1618647653 conn(0x7f202409aa10 msgr2=0x7f202409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f2023e00640 1 -- 172.21.15.122:0/1618647653 shutdown_connections 2024-09-17T05:25:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f7aa00640 1 -- 172.21.15.122:0/880705577 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70017070 con 0x7f9f84070090 2024-09-17T05:25:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f7aa00640 1 -- 172.21.15.122:0/880705577 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f70009d20 con 0x7f9f84070090 2024-09-17T05:25:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f7aa00640 1 -- 172.21.15.122:0/880705577 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70004280 con 0x7f9f84070090 2024-09-17T05:25:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f841440a0 con 0x7f9f84070090 2024-09-17T05:25:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3991200640 1 Processor -- start 2024-09-17T05:25:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3991200640 1 -- start start 2024-09-17T05:25:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f84144660 con 0x7f9f84070090 2024-09-17T05:25:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.266+0000 7f2023e00640 1 -- 172.21.15.122:0/1618647653 wait complete. 2024-09-17T05:25:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2023e00640 1 Processor -- start 2024-09-17T05:25:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2023e00640 1 -- start start 2024-09-17T05:25:15.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3991200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c06ece0 0x7f398c151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3991200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c0b3ed0 0x7f398c14b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3991200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c161520 0x7f398c14aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3991200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f398c16b520 con 0x7f398c161520 2024-09-17T05:25:15.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3991200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f398c16b3a0 con 0x7f398c06ece0 2024-09-17T05:25:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3991200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f398c16b6a0 con 0x7f398c0b3ed0 2024-09-17T05:25:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3981400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c0b3ed0 0x7f398c14b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3981400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c0b3ed0 0x7f398c14b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33400/0 (socket says 172.21.15.122:33400) 2024-09-17T05:25:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3981400640 1 -- 172.21.15.122:0/1832516436 learned_addr learned my addr 172.21.15.122:0/1832516436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2023e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b0ce0 0x7f2024207000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2023e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20240b4d80 0x7f202409d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2023e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20240b5870 0x7f202415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3989e00640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c06ece0 0x7f398c151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f398a800640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c161520 0x7f398c14aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3981400640 1 -- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c06ece0 msgr2=0x7f398c151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3981400640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c06ece0 0x7f398c151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f3981400640 1 -- 172.21.15.122:0/1832516436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c161520 msgr2=0x7f398c14aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2022a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b0ce0 0x7f2024207000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2023e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2024168ae0 con 0x7f20240b4d80 2024-09-17T05:25:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2023e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2024168960 con 0x7f20240b0ce0 2024-09-17T05:25:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2022a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b0ce0 0x7f2024207000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43572/0 (socket says 172.21.15.122:43572) 2024-09-17T05:25:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2023e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2024168c60 con 0x7f20240b5870 2024-09-17T05:25:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f2022a00640 1 -- 172.21.15.122:0/67043390 learned_addr learned my addr 172.21.15.122:0/67043390 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:15.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f9f7aa00640 1 -- 172.21.15.122:0/880705577 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9f70003600 con 0x7f9f84070090 2024-09-17T05:25:15.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.270+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f840cd600 con 0x7f9f84070090 2024-09-17T05:25:15.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f2023400640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20240b5870 0x7f202415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f2022a00640 1 -- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20240b5870 msgr2=0x7f202415a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f2022a00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20240b5870 0x7f202415a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3981400640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c161520 0x7f398c14aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3981400640 1 -- 172.21.15.122:0/1832516436 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c151a40 con 0x7f398c0b3ed0 2024-09-17T05:25:15.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f2022a00640 1 -- 172.21.15.122:0/67043390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20240b4d80 msgr2=0x7f202409d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f2022a00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20240b4d80 0x7f202409d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f2022a00640 1 -- 172.21.15.122:0/67043390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2024071ec0 con 0x7f20240b0ce0 2024-09-17T05:25:15.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f2022a00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b0ce0 0x7f2024207000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f201c00cce0 tx=0x7f201c00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3981400640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c0b3ed0 0x7f398c14b5e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f397c009230 tx=0x7f397c005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3982a00640 1 -- 172.21.15.122:0/1832516436 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0b2040 con 0x7f398c0b3ed0 2024-09-17T05:25:15.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3982a00640 1 -- 172.21.15.122:0/1832516436 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f397c004240 con 0x7f398c0b3ed0 2024-09-17T05:25:15.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3982a00640 1 -- 172.21.15.122:0/1832516436 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c005030 con 0x7f398c0b3ed0 2024-09-17T05:25:15.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398c151760 con 0x7f398c0b3ed0 2024-09-17T05:25:15.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f398c0cd630 con 0x7f398c0b3ed0 2024-09-17T05:25:15.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f398a800640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c161520 0x7f398c14aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:15.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f9f7aa00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f64073230 0x7f9f640756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f9f7aa00640 1 -- 172.21.15.122:0/880705577 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9f70093c10 con 0x7f9f84070090 2024-09-17T05:25:15.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398c0b3990 con 0x7f398c0b3ed0 2024-09-17T05:25:15.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f9f82a00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f64073230 0x7f9f640756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.274+0000 7f9f82a00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f64073230 0x7f9f640756f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9f841455d0 tx=0x7f9f74009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f3982a00640 1 -- 172.21.15.122:0/1832516436 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f397c0051d0 con 0x7f398c0b3ed0 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 -- 172.21.15.122:0/885540674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666015bd80 msgr2=0x7f66600ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 --2- 172.21.15.122:0/885540674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666015bd80 0x7f66600ccbc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f664c009840 tx=0x7f664c01c7d0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 -- 172.21.15.122:0/885540674 shutdown_connections 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 --2- 172.21.15.122:0/885540674 >> v2:172.21.15.173:3300/0 conn(0x7f666015e870 0x7f666015ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 --2- 172.21.15.122:0/885540674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6660073820 0x7f6660073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 --2- 172.21.15.122:0/885540674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666015bd80 0x7f66600ccbc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 -- 172.21.15.122:0/885540674 >> 172.21.15.122:0/885540674 conn(0x7f66600d0cd0 msgr2=0x7f66600d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 -- 172.21.15.122:0/885540674 shutdown_connections 2024-09-17T05:25:15.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f6664c00640 1 -- 172.21.15.122:0/885540674 wait complete. 2024-09-17T05:25:15.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f3982a00640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39680732d0 0x7f3968075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f3982a00640 1 -- 172.21.15.122:0/1832516436 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f397c12e270 con 0x7f398c0b3ed0 2024-09-17T05:25:15.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f3989e00640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39680732d0 0x7f3968075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.278+0000 7f3989e00640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39680732d0 0x7f3968075790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f398c14f7c0 tx=0x7f39740091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f6664c00640 1 Processor -- start 2024-09-17T05:25:15.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f6664c00640 1 -- start start 2024-09-17T05:25:15.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f6664c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6660073820 0x7f66600c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f201b400640 1 -- 172.21.15.122:0/67043390 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c004b30 con 0x7f20240b0ce0 2024-09-17T05:25:15.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f201b400640 1 -- 172.21.15.122:0/67043390 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f201c004dc0 con 0x7f20240b0ce0 2024-09-17T05:25:15.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f201b400640 1 -- 172.21.15.122:0/67043390 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c005280 con 0x7f20240b0ce0 2024-09-17T05:25:15.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f6664c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f666015bd80 0x7f66600b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f6664c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f666015e870 0x7f66601570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f6664c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f666006d5b0 con 0x7f6660073820 2024-09-17T05:25:15.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f6664c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f666006d430 con 0x7f666015bd80 2024-09-17T05:25:15.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f6664c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f666006d730 con 0x7f666015e870 2024-09-17T05:25:15.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6660073820 0x7f66600c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6660073820 0x7f66600c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42972/0 (socket says 172.21.15.122:42972) 2024-09-17T05:25:15.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 -- 172.21.15.122:0/1682266195 learned_addr learned my addr 172.21.15.122:0/1682266195 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:15.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202409d9c0 con 0x7f20240b0ce0 2024-09-17T05:25:15.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f9f7aa00640 1 -- 172.21.15.122:0/880705577 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9f840b4d40 con 0x7f9f84070090 2024-09-17T05:25:15.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2024207b10 con 0x7f20240b0ce0 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665d400640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f666015bd80 0x7f66600b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665e800640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f666015e870 0x7f66601570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 -- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f666015e870 msgr2=0x7f66601570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f666015e870 0x7f66601570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 -- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f666015bd80 msgr2=0x7f66600b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f666015bd80 0x7f66600b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 -- 172.21.15.122:0/1682266195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6660156030 con 0x7f6660073820 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665e800640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f666015e870 0x7f66601570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f665de00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6660073820 0x7f66600c7290 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f664c01a400 tx=0x7f664c0a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.282+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20240b11a0 con 0x7f20240b0ce0 2024-09-17T05:25:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f6656000640 1 -- 172.21.15.122:0/1682266195 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c0ba020 con 0x7f6660073820 2024-09-17T05:25:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f6656000640 1 -- 172.21.15.122:0/1682266195 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f664c0043e0 con 0x7f6660073820 2024-09-17T05:25:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f6656000640 1 -- 172.21.15.122:0/1682266195 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c0b2040 con 0x7f6660073820 2024-09-17T05:25:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6660156310 con 0x7f6660073820 2024-09-17T05:25:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f201b400640 1 -- 172.21.15.122:0/67043390 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f201c005420 con 0x7f20240b0ce0 2024-09-17T05:25:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f3982a00640 1 -- 172.21.15.122:0/1832516436 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f397c0164d0 con 0x7f398c0b3ed0 2024-09-17T05:25:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601568d0 con 0x7f6660073820 2024-09-17T05:25:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7fa07ce00640 1 -- 172.21.15.122:0/658566120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa058003a30 msgr2=0x7fa058003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7fa07ce00640 1 --2- 172.21.15.122:0/658566120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa058003a30 0x7fa058003e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa060001cb0 tx=0x7fa06001ca20 comp rx=0 tx=0).stop 2024-09-17T05:25:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f1a97600640 1 -- 172.21.15.122:0/2455326450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a90074500 msgr2=0x7f1a90166340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f1a97600640 1 --2- 172.21.15.122:0/2455326450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a90074500 0x7f1a90166340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009bf0 tx=0x7f1a8801f3d0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f1a97600640 1 -- 172.21.15.122:0/2455326450 shutdown_connections 2024-09-17T05:25:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f1a97600640 1 --2- 172.21.15.122:0/2455326450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a90074500 0x7f1a90166340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f1a97600640 1 -- 172.21.15.122:0/2455326450 >> 172.21.15.122:0/2455326450 conn(0x7f1a900cfff0 msgr2=0x7f1a900d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7fa07ce00640 1 -- 172.21.15.122:0/658566120 shutdown_connections 2024-09-17T05:25:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7fa07ce00640 1 --2- 172.21.15.122:0/658566120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa058003a30 0x7fa058003e70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7fa07ce00640 1 -- 172.21.15.122:0/658566120 >> 172.21.15.122:0/658566120 conn(0x7fa0780d1e30 msgr2=0x7fa0780d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7ff00ca00640 1 -- 172.21.15.122:0/235020869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0080b3160 msgr2=0x7ff008165c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7ff00ca00640 1 --2- 172.21.15.122:0/235020869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0080b3160 0x7ff008165c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feff8009900 tx=0x7feff801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7fa07ce00640 1 -- 172.21.15.122:0/658566120 shutdown_connections 2024-09-17T05:25:15.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66600ccbc0 con 0x7f6660073820 2024-09-17T05:25:15.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7ff00ca00640 1 -- 172.21.15.122:0/235020869 shutdown_connections 2024-09-17T05:25:15.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7ff00ca00640 1 --2- 172.21.15.122:0/235020869 >> v2:172.21.15.173:3300/0 conn(0x7ff0080747f0 0x7ff008162ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7ff00ca00640 1 --2- 172.21.15.122:0/235020869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0080b3160 0x7ff008165c80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7ff00ca00640 1 -- 172.21.15.122:0/235020869 >> 172.21.15.122:0/235020869 conn(0x7ff0080b58c0 msgr2=0x7ff0080b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f1a97600640 1 -- 172.21.15.122:0/2455326450 shutdown_connections 2024-09-17T05:25:15.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7f1a97600640 1 -- 172.21.15.122:0/2455326450 wait complete. 2024-09-17T05:25:15.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.286+0000 7fa07ce00640 1 -- 172.21.15.122:0/658566120 wait complete. 2024-09-17T05:25:15.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7ff00ca00640 1 -- 172.21.15.122:0/235020869 shutdown_connections 2024-09-17T05:25:15.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7ff00ca00640 1 -- 172.21.15.122:0/235020869 wait complete. 2024-09-17T05:25:15.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7f6656000640 1 -- 172.21.15.122:0/1682266195 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f664c005320 con 0x7f6660073820 2024-09-17T05:25:15.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7fa07ce00640 1 Processor -- start 2024-09-17T05:25:15.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7f1a97600640 1 Processor -- start 2024-09-17T05:25:15.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7f201b400640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2004073250 0x7f2004075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7f201b400640 1 -- 172.21.15.122:0/67043390 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f201c094cd0 con 0x7f20240b0ce0 2024-09-17T05:25:15.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7f2022000640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2004073250 0x7f2004075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7f1a97600640 1 -- start start 2024-09-17T05:25:15.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f2022000640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2004073250 0x7f2004075710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20100046c0 tx=0x7f2010007910 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f201b400640 1 -- 172.21.15.122:0/67043390 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f202409d9c0 con 0x7f20240b0ce0 2024-09-17T05:25:15.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.290+0000 7f6656000640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66400732d0 0x7f6640075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f6656000640 1 -- 172.21.15.122:0/1682266195 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f664c12dcb0 con 0x7f6660073820 2024-09-17T05:25:15.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f665d400640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66400732d0 0x7f6640075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a97600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a90074500 0x7f1a902019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7ff00ca00640 1 Processor -- start 2024-09-17T05:25:15.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a97600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a900cb550 0x7f1a90201f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a97600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a900cb930 0x7f1a90204cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a97600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a900adf40 con 0x7f1a90074500 2024-09-17T05:25:15.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a97600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a900cada0 con 0x7f1a900cb930 2024-09-17T05:25:15.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a97600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1a900ae0c0 con 0x7f1a900cb550 2024-09-17T05:25:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7ff00ca00640 1 -- start start 2024-09-17T05:25:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a94c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a90074500 0x7f1a902019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a94c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a90074500 0x7f1a902019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42984/0 (socket says 172.21.15.122:42984) 2024-09-17T05:25:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a94c00640 1 -- 172.21.15.122:0/2508921875 learned_addr learned my addr 172.21.15.122:0/2508921875 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8fe00640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a900cb550 0x7f1a90201f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7fa07ce00640 1 -- start start 2024-09-17T05:25:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7fa07ce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa058003a30 0x7fa0780b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7fa07ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0780c8030 0x7fa078152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7fa07ce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0780cdb10 0x7fa0780cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7fa07ce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780b0230 con 0x7fa0780c8030 2024-09-17T05:25:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7fa07ce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780b00b0 con 0x7fa058003a30 2024-09-17T05:25:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7fa07ce00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa0780b03b0 con 0x7fa0780cdb10 2024-09-17T05:25:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a95600640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a900cb930 0x7f1a90204cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8fe00640 1 -- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a900cb930 msgr2=0x7f1a90204cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8fe00640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a900cb930 0x7f1a90204cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8fe00640 1 -- 172.21.15.122:0/2508921875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a90074500 msgr2=0x7f1a902019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8fe00640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a90074500 0x7f1a902019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8fe00640 1 -- 172.21.15.122:0/2508921875 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9009def0 con 0x7f1a900cb550 2024-09-17T05:25:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa075e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa058003a30 0x7fa0780b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f665d400640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66400732d0 0x7f6640075790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f66600cae50 tx=0x7f665000f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7ff00ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0080747f0 0x7ff00815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7ff00ca00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0080b3160 0x7ff008161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7ff00ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff008163400 0x7ff0080b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7ff00ca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff00816f3b0 con 0x7ff0080747f0 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7ff00ca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff00816f230 con 0x7ff008163400 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7ff00ca00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff00816f530 con 0x7ff0080b3160 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8fe00640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a900cb550 0x7f1a90201f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c3c0 tx=0x7f1a7c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8cc00640 1 -- 172.21.15.122:0/2508921875 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c017070 con 0x7f1a900cb550 2024-09-17T05:25:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8cc00640 1 -- 172.21.15.122:0/2508921875 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a7c0053c0 con 0x7f1a900cb550 2024-09-17T05:25:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9009d930 con 0x7f1a900cb550 2024-09-17T05:25:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a8cc00640 1 -- 172.21.15.122:0/2508921875 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c003860 con 0x7f1a900cb550 2024-09-17T05:25:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.294+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a900c81b0 con 0x7f1a900cb550 2024-09-17T05:25:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa075e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa058003a30 0x7fa0780b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43610/0 (socket says 172.21.15.122:43610) 2024-09-17T05:25:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa075e00640 1 -- 172.21.15.122:0/2099539740 learned_addr learned my addr 172.21.15.122:0/2099539740 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa076800640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0780cdb10 0x7fa0780cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff008163400 0x7ff0080b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa076800640 1 -- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa058003a30 msgr2=0x7fa0780b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa076800640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa058003a30 0x7fa0780b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa076800640 1 -- 172.21.15.122:0/2099539740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0780c8030 msgr2=0x7fa078152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa076800640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0780c8030 0x7fa078152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa076800640 1 -- 172.21.15.122:0/2099539740 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa078152890 con 0x7fa0780cdb10 2024-09-17T05:25:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa075e00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa058003a30 0x7fa0780b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff008163400 0x7ff0080b94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43618/0 (socket says 172.21.15.122:43618) 2024-09-17T05:25:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 -- 172.21.15.122:0/1332368671 learned_addr learned my addr 172.21.15.122:0/1332368671 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007400640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0080747f0 0x7ff00815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff006a00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0080b3160 0x7ff008161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fa076800640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0780cdb10 0x7fa0780cdf90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0780740f0 tx=0x7fa06800ea20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9006eb80 con 0x7f1a900cb550 2024-09-17T05:25:15.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7f1a8cc00640 1 -- 172.21.15.122:0/2508921875 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1a7c01a070 con 0x7f1a900cb550 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 -- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0080b3160 msgr2=0x7ff008161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0080b3160 0x7ff008161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 -- 172.21.15.122:0/1332368671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0080747f0 msgr2=0x7ff00815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0080747f0 0x7ff00815dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 -- 172.21.15.122:0/1332368671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff008160490 con 0x7ff008163400 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7ff007e00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff008163400 0x7ff0080b94c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff00000b970 tx=0x7ff00000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fefff400640 1 -- 172.21.15.122:0/1332368671 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000019070 con 0x7ff008163400 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.298+0000 7fefff400640 1 -- 172.21.15.122:0/1332368671 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff00000cbd0 con 0x7ff008163400 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.302+0000 7fefff400640 1 -- 172.21.15.122:0/1332368671 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0000146a0 con 0x7ff008163400 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.302+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff008160770 con 0x7ff008163400 2024-09-17T05:25:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.302+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff008160d30 con 0x7ff008163400 2024-09-17T05:25:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.302+0000 7f1a8cc00640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a70073250 0x7f1a70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.302+0000 7f1a8cc00640 1 -- 172.21.15.122:0/2508921875 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1a7c0934d0 con 0x7f1a900cb550 2024-09-17T05:25:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.302+0000 7f1a94c00640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a70073250 0x7f1a70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.302+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff008162ec0 con 0x7ff008163400 2024-09-17T05:25:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.302+0000 7fefff400640 1 -- 172.21.15.122:0/1332368671 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff00001c070 con 0x7ff008163400 2024-09-17T05:25:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7fa06e000640 1 -- 172.21.15.122:0/2099539740 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068010950 con 0x7fa0780cdb10 2024-09-17T05:25:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7fa06e000640 1 -- 172.21.15.122:0/2099539740 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa068016070 con 0x7fa0780cdb10 2024-09-17T05:25:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7fa06e000640 1 -- 172.21.15.122:0/2099539740 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068011460 con 0x7fa0780cdb10 2024-09-17T05:25:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7f6656000640 1 -- 172.21.15.122:0/1682266195 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f664c016410 con 0x7f6660073820 2024-09-17T05:25:15.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7f1a94c00640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a70073250 0x7f1a70075710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a840019d0 tx=0x7f1a840014e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7fefff400640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefe8073250 0x7fefe8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7fefff400640 1 -- 172.21.15.122:0/1332368671 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff000095db0 con 0x7ff008163400 2024-09-17T05:25:15.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7ff007400640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefe8073250 0x7fefe8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0780b7280 con 0x7fa0780cdb10 2024-09-17T05:25:15.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7f1a8cc00640 1 -- 172.21.15.122:0/2508921875 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1a7c05dfd0 con 0x7f1a900cb550 2024-09-17T05:25:15.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0780b7840 con 0x7fa0780cdb10 2024-09-17T05:25:15.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7ff007400640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefe8073250 0x7fefe8075710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feff80af000 tx=0x7feff801b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.306+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0780b0910 con 0x7fa0780cdb10 2024-09-17T05:25:15.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.310+0000 7fefff400640 1 -- 172.21.15.122:0/1332368671 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff00005e610 con 0x7ff008163400 2024-09-17T05:25:15.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.310+0000 7fa06e000640 1 -- 172.21.15.122:0/2099539740 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa068011600 con 0x7fa0780cdb10 2024-09-17T05:25:15.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.310+0000 7fa06e000640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0540731f0 0x7fa0540756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.310+0000 7fa06e000640 1 -- 172.21.15.122:0/2099539740 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa0680945c0 con 0x7fa0780cdb10 2024-09-17T05:25:15.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.314+0000 7fa075e00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0540731f0 0x7fa0540756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.314+0000 7fa075e00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0540731f0 0x7fa0540756b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa06001b2a0 tx=0x7fa0600a7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.314+0000 7fa06e000640 1 -- 172.21.15.122:0/2099539740 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa06805f650 con 0x7fa0780cdb10 2024-09-17T05:25:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 -- 172.21.15.122:0/3197754572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 msgr2=0x7f4940073190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 --2- 172.21.15.122:0/3197754572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 0x7f4940073190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49400aef00 tx=0x7f492c01c8d0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 -- 172.21.15.122:0/3197754572 shutdown_connections 2024-09-17T05:25:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 --2- 172.21.15.122:0/3197754572 >> v2:172.21.15.173:3300/0 conn(0x7f494006ef70 0x7f4940073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 --2- 172.21.15.122:0/3197754572 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400cd600 0x7f49400736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 --2- 172.21.15.122:0/3197754572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 0x7f4940073190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 -- 172.21.15.122:0/3197754572 >> 172.21.15.122:0/3197754572 conn(0x7f49400b1600 msgr2=0x7f49400b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 -- 172.21.15.122:0/3197754572 shutdown_connections 2024-09-17T05:25:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.338+0000 7f4946400640 1 -- 172.21.15.122:0/3197754572 wait complete. 2024-09-17T05:25:15.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.342+0000 7f4946400640 1 Processor -- start 2024-09-17T05:25:15.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.342+0000 7f4946400640 1 -- start start 2024-09-17T05:25:15.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.342+0000 7f4946400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f494006ef70 0x7f49400c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.342+0000 7f4946400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 0x7f494016e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.342+0000 7f4945000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f494006ef70 0x7f49400c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f493fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 0x7f494016e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f493fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 0x7f494016e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43026/0 (socket says 172.21.15.122:43026) 2024-09-17T05:25:15.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4945000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f494006ef70 0x7f49400c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43646/0 (socket says 172.21.15.122:43646) 2024-09-17T05:25:15.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f493fe00640 1 -- 172.21.15.122:0/783921812 learned_addr learned my addr 172.21.15.122:0/783921812 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:15.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4946400640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400cd600 0x7f494020aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4946400640 1 -- 172.21.15.122:0/783921812 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4940177e30 con 0x7f49400ca080 2024-09-17T05:25:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4946400640 1 -- 172.21.15.122:0/783921812 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4940177cb0 con 0x7f494006ef70 2024-09-17T05:25:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4945a00640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400cd600 0x7f494020aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4946400640 1 -- 172.21.15.122:0/783921812 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4940177fb0 con 0x7f49400cd600 2024-09-17T05:25:15.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4945000640 1 -- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400cd600 msgr2=0x7f494020aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4945000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400cd600 0x7f494020aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4945000640 1 -- 172.21.15.122:0/783921812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 msgr2=0x7f494016e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4945000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 0x7f494016e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4945000640 1 -- 172.21.15.122:0/783921812 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49400c9db0 con 0x7f494006ef70 2024-09-17T05:25:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f4945000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f494006ef70 0x7f49400c6e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f492c0ac000 tx=0x7f492c0037e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f493cc00640 1 -- 172.21.15.122:0/783921812 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c0bd070 con 0x7f494006ef70 2024-09-17T05:25:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f493cc00640 1 -- 172.21.15.122:0/783921812 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f492c003d80 con 0x7f494006ef70 2024-09-17T05:25:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.346+0000 7f493cc00640 1 -- 172.21.15.122:0/783921812 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c005280 con 0x7f494006ef70 2024-09-17T05:25:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.350+0000 7f4946400640 1 -- 172.21.15.122:0/783921812 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494016e670 con 0x7f494006ef70 2024-09-17T05:25:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.350+0000 7f4946400640 1 -- 172.21.15.122:0/783921812 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401681c0 con 0x7f494006ef70 2024-09-17T05:25:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.354+0000 7f493cc00640 1 -- 172.21.15.122:0/783921812 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f492c005420 con 0x7f494006ef70 2024-09-17T05:25:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.354+0000 7f4936000640 1 -- 172.21.15.122:0/783921812 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49180052a0 con 0x7f494006ef70 2024-09-17T05:25:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.354+0000 7f493cc00640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49200732e0 0x7f49200757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.354+0000 7f493cc00640 1 -- 172.21.15.122:0/783921812 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f492c12e8e0 con 0x7f494006ef70 2024-09-17T05:25:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.354+0000 7f493fe00640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49200732e0 0x7f49200757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.354+0000 7f493fe00640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49200732e0 0x7f49200757a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49400c64b0 tx=0x7f49300090e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.358+0000 7f493cc00640 1 -- 172.21.15.122:0/783921812 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f492c0164f0 con 0x7f494006ef70 2024-09-17T05:25:15.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.630+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9f84074520 con 0x7f9f84070090 2024-09-17T05:25:15.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.630+0000 7f9f7aa00640 1 -- 172.21.15.122:0/880705577 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9f7005e5d0 con 0x7f9f84070090 2024-09-17T05:25:15.634 INFO:teuthology.orchestra.run.smithi122.stdout:64424509445 2024-09-17T05:25:15.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f64073230 msgr2=0x7f9f640756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f64073230 0x7f9f640756f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9f841455d0 tx=0x7f9f74009040 comp rx=0 tx=0).stop 2024-09-17T05:25:15.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f84070090 msgr2=0x7f9f84145c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f84070090 0x7f9f84145c50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000d980 tx=0x7f9f7000de40 comp rx=0 tx=0).stop 2024-09-17T05:25:15.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 shutdown_connections 2024-09-17T05:25:15.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f64073230 0x7f9f640756f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f840cdb40 0x7f9f84146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f84070090 0x7f9f84145c50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 --2- 172.21.15.122:0/880705577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f8406fcb0 0x7f9f84143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 >> 172.21.15.122:0/880705577 conn(0x7f9f840cfa30 msgr2=0x7f9f840730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 shutdown_connections 2024-09-17T05:25:15.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.634+0000 7f9f83e00640 1 -- 172.21.15.122:0/880705577 wait complete. 2024-09-17T05:25:15.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.650+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f398c0b2750 con 0x7f398c0b3ed0 2024-09-17T05:25:15.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.654+0000 7f3982a00640 1 -- 172.21.15.122:0/1832516436 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f397c0f9390 con 0x7f398c0b3ed0 2024-09-17T05:25:15.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.654+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f20240ce0c0 con 0x7f20240b0ce0 2024-09-17T05:25:15.659 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:15.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.654+0000 7f201b400640 1 -- 172.21.15.122:0/67043390 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f201c014030 con 0x7f20240b0ce0 2024-09-17T05:25:15.660 INFO:teuthology.orchestra.run.smithi122.stdout:64424509445 2024-09-17T05:25:15.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39680732d0 msgr2=0x7f3968075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39680732d0 0x7f3968075790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f398c14f7c0 tx=0x7f39740091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c0b3ed0 msgr2=0x7f398c14b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c0b3ed0 0x7f398c14b5e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f397c009230 tx=0x7f397c005e70 comp rx=0 tx=0).stop 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 shutdown_connections 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39680732d0 0x7f3968075790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c161520 0x7f398c14aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c0b3ed0 0x7f398c14b5e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 --2- 172.21.15.122:0/1832516436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c06ece0 0x7f398c151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 >> 172.21.15.122:0/1832516436 conn(0x7f398c0b18f0 msgr2=0x7f398c157770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 shutdown_connections 2024-09-17T05:25:15.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f3991200640 1 -- 172.21.15.122:0/1832516436 wait complete. 2024-09-17T05:25:15.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2004073250 msgr2=0x7f2004075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f2023e00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2004073250 0x7f2004075710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20100046c0 tx=0x7f2010007910 comp rx=0 tx=0).stop 2024-09-17T05:25:15.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b0ce0 msgr2=0x7f2024207000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.658+0000 7f2023e00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b0ce0 0x7f2024207000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f201c00cce0 tx=0x7f201c00a620 comp rx=0 tx=0).stop 2024-09-17T05:25:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.662+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 shutdown_connections 2024-09-17T05:25:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.662+0000 7f2023e00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2004073250 0x7f2004075710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.662+0000 7f2023e00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20240b5870 0x7f202415a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.662+0000 7f2023e00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20240b4d80 0x7f202409d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.662+0000 7f2023e00640 1 --2- 172.21.15.122:0/67043390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20240b0ce0 0x7f2024207000 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.662+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 >> 172.21.15.122:0/67043390 conn(0x7f202409aa10 msgr2=0x7f202409adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.662+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 shutdown_connections 2024-09-17T05:25:15.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.662+0000 7f2023e00640 1 -- 172.21.15.122:0/67043390 wait complete. 2024-09-17T05:25:15.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-09-17T05:25:15.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.670+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1a900b5f70 con 0x7f1a900cb550 2024-09-17T05:25:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.670+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff00809d0c0 con 0x7ff008163400 2024-09-17T05:25:15.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.670+0000 7fefff400640 1 -- 172.21.15.122:0/1332368671 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff000063040 con 0x7ff008163400 2024-09-17T05:25:15.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.670+0000 7f1a8cc00640 1 -- 172.21.15.122:0/2508921875 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1a7c062a00 con 0x7f1a900cb550 2024-09-17T05:25:15.673 INFO:teuthology.orchestra.run.smithi122.stdout:64424509446 2024-09-17T05:25:15.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.670+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6660071b60 con 0x7f6660073820 2024-09-17T05:25:15.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.670+0000 7f6656000640 1 -- 172.21.15.122:0/1682266195 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f664c016410 con 0x7f6660073820 2024-09-17T05:25:15.674 INFO:teuthology.orchestra.run.smithi122.stdout:64424509446 2024-09-17T05:25:15.674 INFO:teuthology.orchestra.run.smithi122.stdout:64424509446 2024-09-17T05:25:15.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefe8073250 msgr2=0x7fefe8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa0780d0a20 con 0x7fa0780cdb10 2024-09-17T05:25:15.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefe8073250 0x7fefe8075710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feff80af000 tx=0x7feff801b120 comp rx=0 tx=0).stop 2024-09-17T05:25:15.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff008163400 msgr2=0x7ff0080b94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a70073250 msgr2=0x7f1a70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a70073250 0x7f1a70075710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a840019d0 tx=0x7f1a840014e0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a900cb550 msgr2=0x7f1a90201f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a900cb550 0x7f1a90201f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c3c0 tx=0x7f1a7c00c880 comp rx=0 tx=0).stop 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 shutdown_connections 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a70073250 0x7f1a70075710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a900cb930 0x7f1a90204cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a900cb550 0x7f1a90201f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 --2- 172.21.15.122:0/2508921875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a90074500 0x7f1a902019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 >> 172.21.15.122:0/2508921875 conn(0x7f1a900cfff0 msgr2=0x7f1a9006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff008163400 0x7ff0080b94c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff00000b970 tx=0x7ff00000be30 comp rx=0 tx=0).stop 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 shutdown_connections 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefe8073250 0x7fefe8075710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff008163400 0x7ff0080b94c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0080b3160 0x7ff008161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 --2- 172.21.15.122:0/1332368671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0080747f0 0x7ff00815dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 >> 172.21.15.122:0/1332368671 conn(0x7ff0080b58c0 msgr2=0x7ff0080b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 shutdown_connections 2024-09-17T05:25:15.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7ff00ca00640 1 -- 172.21.15.122:0/1332368671 wait complete. 2024-09-17T05:25:15.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 shutdown_connections 2024-09-17T05:25:15.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f1a97600640 1 -- 172.21.15.122:0/2508921875 wait complete. 2024-09-17T05:25:15.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66400732d0 msgr2=0x7f6640075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66400732d0 0x7f6640075790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f66600cae50 tx=0x7f665000f040 comp rx=0 tx=0).stop 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6660073820 msgr2=0x7f66600c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6660073820 0x7f66600c7290 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f664c01a400 tx=0x7f664c0a5600 comp rx=0 tx=0).stop 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 shutdown_connections 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66400732d0 0x7f6640075790 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f666015e870 0x7f66601570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f666015bd80 0x7f66600b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 --2- 172.21.15.122:0/1682266195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6660073820 0x7f66600c7290 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 >> 172.21.15.122:0/1682266195 conn(0x7f66600d0cd0 msgr2=0x7f66600cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7fa06e000640 1 -- 172.21.15.122:0/2099539740 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa06805ef50 con 0x7fa0780cdb10 2024-09-17T05:25:15.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 shutdown_connections 2024-09-17T05:25:15.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.674+0000 7f6664c00640 1 -- 172.21.15.122:0/1682266195 wait complete. 2024-09-17T05:25:15.682 INFO:teuthology.orchestra.run.smithi122.stdout:64424509446 2024-09-17T05:25:15.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0540731f0 msgr2=0x7fa0540756b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0540731f0 0x7fa0540756b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa06001b2a0 tx=0x7fa0600a7040 comp rx=0 tx=0).stop 2024-09-17T05:25:15.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0780cdb10 msgr2=0x7fa0780cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0780cdb10 0x7fa0780cdf90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0780740f0 tx=0x7fa06800ea20 comp rx=0 tx=0).stop 2024-09-17T05:25:15.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 shutdown_connections 2024-09-17T05:25:15.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0540731f0 0x7fa0540756b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0780cdb10 0x7fa0780cdf90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0780c8030 0x7fa078152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 --2- 172.21.15.122:0/2099539740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa058003a30 0x7fa0780b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 >> 172.21.15.122:0/2099539740 conn(0x7fa0780d1e30 msgr2=0x7fa07815fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 shutdown_connections 2024-09-17T05:25:15.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.682+0000 7fa07ce00640 1 -- 172.21.15.122:0/2099539740 wait complete. 2024-09-17T05:25:15.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.1 2024-09-17T05:25:15.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.702+0000 7f4936000640 1 -- 172.21.15.122:0/783921812 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4918005590 con 0x7f494006ef70 2024-09-17T05:25:15.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2024-09-17T05:25:15.705 INFO:teuthology.orchestra.run.smithi122.stdout:60129542151 2024-09-17T05:25:15.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.702+0000 7f493cc00640 1 -- 172.21.15.122:0/783921812 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f492c0f9320 con 0x7f494006ef70 2024-09-17T05:25:15.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 -- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49200732e0 msgr2=0x7f49200757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49200732e0 0x7f49200757a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49400c64b0 tx=0x7f49300090e0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 -- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f494006ef70 msgr2=0x7f49400c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:15.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f494006ef70 0x7f49400c6e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f492c0ac000 tx=0x7f492c0037e0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 -- 172.21.15.122:0/783921812 shutdown_connections 2024-09-17T05:25:15.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49200732e0 0x7f49200757a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400cd600 0x7f494020aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49400ca080 0x7f494016e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 --2- 172.21.15.122:0/783921812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f494006ef70 0x7f49400c6e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:15.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 -- 172.21.15.122:0/783921812 >> 172.21.15.122:0/783921812 conn(0x7f49400b1600 msgr2=0x7f49400afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:15.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 -- 172.21.15.122:0/783921812 shutdown_connections 2024-09-17T05:25:15.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:15.706+0000 7f4936000640 1 -- 172.21.15.122:0/783921812 wait complete. 2024-09-17T05:25:15.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-09-17T05:25:15.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-17T05:25:15.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-17T05:25:15.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-09-17T05:25:15.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.5 2024-09-17T05:25:16.670 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T05:25:16.700 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T05:25:16.706 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T05:25:16.714 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T05:25:16.716 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T05:25:16.717 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T05:25:16.723 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T05:25:16.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T05:25:16.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.854+0000 7fd452000640 1 -- 172.21.15.122:0/2015896043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 msgr2=0x7fd4300040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.854+0000 7fd44a000640 1 -- 172.21.15.122:0/2015896043 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b1070 con 0x7fd430003c80 2024-09-17T05:25:16.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.854+0000 7fd452000640 1 --2- 172.21.15.122:0/2015896043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 0x7fd4300040a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009b60 tx=0x7fd43c01d1c0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.854+0000 7fd452000640 1 -- 172.21.15.122:0/2015896043 shutdown_connections 2024-09-17T05:25:16.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.854+0000 7fd452000640 1 --2- 172.21.15.122:0/2015896043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 0x7fd4300040a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.854+0000 7fd452000640 1 -- 172.21.15.122:0/2015896043 >> 172.21.15.122:0/2015896043 conn(0x7fd44c06cc20 msgr2=0x7fd44c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:16.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.854+0000 7fd452000640 1 -- 172.21.15.122:0/2015896043 shutdown_connections 2024-09-17T05:25:16.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.858+0000 7fd452000640 1 -- 172.21.15.122:0/2015896043 wait complete. 2024-09-17T05:25:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.858+0000 7fd452000640 1 Processor -- start 2024-09-17T05:25:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd452000640 1 -- start start 2024-09-17T05:25:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd452000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 0x7fd44c1f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd452000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c157310 0x7fd44c0affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd452000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd44c152900 0x7fd44c14f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd452000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c0b2f70 con 0x7fd44c157310 2024-09-17T05:25:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd44c152900 0x7fd44c14f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd44c152900 0x7fd44c14f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33492/0 (socket says 172.21.15.122:33492) 2024-09-17T05:25:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44be00640 1 -- 172.21.15.122:0/3851448146 learned_addr learned my addr 172.21.15.122:0/3851448146 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c0b30f0 con 0x7fd430003c80 2024-09-17T05:25:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c0b3270 con 0x7fd44c152900 2024-09-17T05:25:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44b400640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 0x7fd44c1f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44aa00640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c157310 0x7fd44c0affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44be00640 1 -- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 msgr2=0x7fd44c1f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44be00640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 0x7fd44c1f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44be00640 1 -- 172.21.15.122:0/3851448146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c157310 msgr2=0x7fd44c0affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44be00640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c157310 0x7fd44c0affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.862+0000 7fd44be00640 1 -- 172.21.15.122:0/3851448146 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c157810 con 0x7fd44c152900 2024-09-17T05:25:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd44b400640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 0x7fd44c1f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd44aa00640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c157310 0x7fd44c0affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd44be00640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd44c152900 0x7fd44c14f280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b3e50 tx=0x7fd438006430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd443400640 1 -- 172.21.15.122:0/3851448146 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438013440 con 0x7fd44c152900 2024-09-17T05:25:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd443400640 1 -- 172.21.15.122:0/3851448146 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd43801a070 con 0x7fd44c152900 2024-09-17T05:25:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd443400640 1 -- 172.21.15.122:0/3851448146 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4380138c0 con 0x7fd44c152900 2024-09-17T05:25:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c151e20 con 0x7fd44c152900 2024-09-17T05:25:16.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c1523e0 con 0x7fd44c152900 2024-09-17T05:25:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.866+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4240052a0 con 0x7fd44c152900 2024-09-17T05:25:16.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.870+0000 7fd443400640 1 -- 172.21.15.122:0/3851448146 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd438013a60 con 0x7fd44c152900 2024-09-17T05:25:16.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.874+0000 7fd443400640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd42c073230 0x7fd42c0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.874+0000 7fd443400640 1 -- 172.21.15.122:0/3851448146 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd4380970e0 con 0x7fd44c152900 2024-09-17T05:25:16.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.874+0000 7fd44b400640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd42c073230 0x7fd42c0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.874+0000 7fd443400640 1 -- 172.21.15.122:0/3851448146 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd44c15b2c0 con 0x7fd44c152900 2024-09-17T05:25:16.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.874+0000 7fd44b400640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd42c073230 0x7fd42c0756f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0ae000 tx=0x7fd43c019fb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2318985726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a580b6290 msgr2=0x7f5a580740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f5a5ca00640 1 --2- 172.21.15.122:0/2318985726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a580b6290 0x7f5a580740c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009840 tx=0x7f5a4801c760 comp rx=0 tx=0).stop 2024-09-17T05:25:16.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2318985726 shutdown_connections 2024-09-17T05:25:16.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f5a5ca00640 1 --2- 172.21.15.122:0/2318985726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a580b6290 0x7f5a580740c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2318985726 >> 172.21.15.122:0/2318985726 conn(0x7f5a580d0cd0 msgr2=0x7f5a580d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:16.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2318985726 shutdown_connections 2024-09-17T05:25:16.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2318985726 wait complete. 2024-09-17T05:25:16.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f3fd2e00640 1 -- 172.21.15.122:0/2142069916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 msgr2=0x7f3fc4057190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/2142069916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 0x7f3fc4057190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009900 tx=0x7f3fb801c7d0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f3fd2e00640 1 -- 172.21.15.122:0/2142069916 shutdown_connections 2024-09-17T05:25:16.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/2142069916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 0x7f3fc4057190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/2142069916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fc40e6920 0x7f3fc403bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f3fd2e00640 1 -- 172.21.15.122:0/2142069916 >> 172.21.15.122:0/2142069916 conn(0x7f3fc4055360 msgr2=0x7f3fc4055770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:16.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f3fd2e00640 1 -- 172.21.15.122:0/2142069916 shutdown_connections 2024-09-17T05:25:16.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.942+0000 7f3fd2e00640 1 -- 172.21.15.122:0/2142069916 wait complete. 2024-09-17T05:25:16.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a5ca00640 1 Processor -- start 2024-09-17T05:25:16.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f3fd2e00640 1 Processor -- start 2024-09-17T05:25:16.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f3fd2e00640 1 -- start start 2024-09-17T05:25:16.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a5ca00640 1 -- start start 2024-09-17T05:25:16.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a5ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a58071f00 0x7f5a5815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a5ca00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a580b6670 0x7f5a580ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a5ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a58155b00 0x7f5a58155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a5ca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58169ca0 con 0x7f5a58155b00 2024-09-17T05:25:16.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a5ca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58169b20 con 0x7f5a58071f00 2024-09-17T05:25:16.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a5ca00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5a58169e20 con 0x7f5a580b6670 2024-09-17T05:25:16.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a55e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a58071f00 0x7f5a5815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a55e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a58071f00 0x7f5a5815ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43700/0 (socket says 172.21.15.122:43700) 2024-09-17T05:25:16.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.946+0000 7f5a55e00640 1 -- 172.21.15.122:0/2322371564 learned_addr learned my addr 172.21.15.122:0/2322371564 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:16.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a55400640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a580b6670 0x7f5a580ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd2e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc40576d0 0x7f3fc40d6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd2e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fc40e6920 0x7f3fc4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd2e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 0x7f3fc403ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a55e00640 1 -- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a580b6670 msgr2=0x7f5a580ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a55e00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a580b6670 0x7f5a580ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a55e00640 1 -- 172.21.15.122:0/2322371564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a58155b00 msgr2=0x7f5a58155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a55e00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a58155b00 0x7f5a58155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a55e00640 1 -- 172.21.15.122:0/2322371564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a58156700 con 0x7f5a58071f00 2024-09-17T05:25:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fcbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc40576d0 0x7f3fc40d6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fcbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc40576d0 0x7f3fc40d6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43094/0 (socket says 172.21.15.122:43094) 2024-09-17T05:25:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fcbe00640 1 -- 172.21.15.122:0/3242347761 learned_addr learned my addr 172.21.15.122:0/3242347761 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc40de6d0 con 0x7f3fc40576d0 2024-09-17T05:25:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc40de550 con 0x7f3fc40e6d00 2024-09-17T05:25:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc40de850 con 0x7f3fc40e6920 2024-09-17T05:25:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd0e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 0x7f3fc403ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fcb400640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fc40e6920 0x7f3fc4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a55e00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a58071f00 0x7f5a5815ff00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5a4801a390 tx=0x7f5a48004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f89f7200640 1 -- 172.21.15.122:0/1657757211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 msgr2=0x7f89dc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f89f7200640 1 --2- 172.21.15.122:0/1657757211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 0x7f89dc003de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89e40016e0 tx=0x7f89e401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f89f6800640 1 -- 172.21.15.122:0/1657757211 reap_dead start 2024-09-17T05:25:16.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd0e00640 1 -- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fc40e6920 msgr2=0x7f3fc4038d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd0e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fc40e6920 0x7f3fc4038d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd0e00640 1 -- 172.21.15.122:0/3242347761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc40576d0 msgr2=0x7f3fc40d6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd0e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc40576d0 0x7f3fc40d6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd0e00640 1 -- 172.21.15.122:0/3242347761 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc40e4650 con 0x7f3fc40e6d00 2024-09-17T05:25:16.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a4e000640 1 -- 172.21.15.122:0/2322371564 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a480b2070 con 0x7f5a58071f00 2024-09-17T05:25:16.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f89f7200640 1 -- 172.21.15.122:0/1657757211 shutdown_connections 2024-09-17T05:25:16.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f89f7200640 1 -- 172.21.15.122:0/1657757211 >> 172.21.15.122:0/1657757211 conn(0x7f89f0054570 msgr2=0x7f89f0054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:16.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fd0e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 0x7f3fc403ab00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc06cb90 tx=0x7f3fcc06a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f89f7200640 1 -- 172.21.15.122:0/1657757211 shutdown_connections 2024-09-17T05:25:16.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f3fbfe00640 1 -- 172.21.15.122:0/3242347761 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc090070 con 0x7f3fc40e6d00 2024-09-17T05:25:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a4e000640 1 -- 172.21.15.122:0/2322371564 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a48004320 con 0x7f5a58071f00 2024-09-17T05:25:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a58159580 con 0x7f5a58071f00 2024-09-17T05:25:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f5a4e000640 1 -- 172.21.15.122:0/2322371564 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a480b2070 con 0x7f5a58071f00 2024-09-17T05:25:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.954+0000 7f3fbfe00640 1 -- 172.21.15.122:0/3242347761 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fcc072920 con 0x7f3fc40e6d00 2024-09-17T05:25:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.954+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc40e4930 con 0x7f3fc40e6d00 2024-09-17T05:25:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.954+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc403a300 con 0x7f3fc40e6d00 2024-09-17T05:25:16.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.954+0000 7f3fbfe00640 1 -- 172.21.15.122:0/3242347761 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc072d50 con 0x7f3fc40e6d00 2024-09-17T05:25:16.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.950+0000 7f89f7200640 1 -- 172.21.15.122:0/1657757211 wait complete. 2024-09-17T05:25:16.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.954+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a58159b00 con 0x7f5a58071f00 2024-09-17T05:25:16.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.954+0000 7f89f7200640 1 Processor -- start 2024-09-17T05:25:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.954+0000 7f3fbfe00640 1 -- 172.21.15.122:0/3242347761 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3fcc093070 con 0x7f3fc40e6d00 2024-09-17T05:25:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.954+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa80052a0 con 0x7f3fc40e6d00 2024-09-17T05:25:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f3fbfe00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3fac073200 0x7f3fac0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f3fbfe00640 1 -- 172.21.15.122:0/3242347761 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3fcc10f180 con 0x7f3fc40e6d00 2024-09-17T05:25:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f7200640 1 -- start start 2024-09-17T05:25:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f5a4e000640 1 -- 172.21.15.122:0/2322371564 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a480bd030 con 0x7f5a58071f00 2024-09-17T05:25:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a580740c0 con 0x7f5a58071f00 2024-09-17T05:25:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f5a4e000640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5a38073400 0x7f5a380758c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f5a4e000640 1 -- 172.21.15.122:0/2322371564 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5a480a5620 con 0x7f5a58071f00 2024-09-17T05:25:16.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f5a55400640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5a38073400 0x7f5a380758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f7200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 0x7f89f0177b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89f004eb10 0x7f89f004ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f7200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f002b270 0x7f89f002b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f7200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89f001d0b0 con 0x7f89f002b270 2024-09-17T05:25:16.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89f001cf30 con 0x7f89f004eb10 2024-09-17T05:25:16.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f7200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f89f001d230 con 0x7f89dc0039c0 2024-09-17T05:25:16.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f3fcbe00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3fac073200 0x7f3fac0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f5e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 0x7f89f0177b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f002b270 0x7f89f002b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89f004eb10 0x7f89f004ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f5e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 0x7f89f0177b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33542/0 (socket says 172.21.15.122:33542) 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.958+0000 7f89f5e00640 1 -- 172.21.15.122:0/2460653733 learned_addr learned my addr 172.21.15.122:0/2460653733 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f5e00640 1 -- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89f004eb10 msgr2=0x7f89f004ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f5e00640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89f004eb10 0x7f89f004ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f5e00640 1 -- 172.21.15.122:0/2460653733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f002b270 msgr2=0x7f89f002b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f5e00640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f002b270 0x7f89f002b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f5e00640 1 -- 172.21.15.122:0/2460653733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f004f610 con 0x7f89dc0039c0 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f6800640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f002b270 0x7f89f002b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:16.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f5e00640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 0x7f89f0177b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89e401ccb0 tx=0x7f89e4003bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f5a55400640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5a38073400 0x7f5a380758c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000f70 tx=0x7f5a44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f3fcbe00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3fac073200 0x7f3fac0756c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0005fd0 tx=0x7f3fc000a4b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89ee000640 1 -- 172.21.15.122:0/2460653733 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40ab030 con 0x7f89dc0039c0 2024-09-17T05:25:16.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f004f8f0 con 0x7f89dc0039c0 2024-09-17T05:25:16.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89ee000640 1 -- 172.21.15.122:0/2460653733 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89e40ac040 con 0x7f89dc0039c0 2024-09-17T05:25:16.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89ee000640 1 -- 172.21.15.122:0/2460653733 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40a6510 con 0x7f89dc0039c0 2024-09-17T05:25:16.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f004feb0 con 0x7f89dc0039c0 2024-09-17T05:25:16.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.962+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f001c820 con 0x7f89dc0039c0 2024-09-17T05:25:16.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.966+0000 7f3fbfe00640 1 -- 172.21.15.122:0/3242347761 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3fcc0d9c00 con 0x7f3fc40e6d00 2024-09-17T05:25:16.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.966+0000 7f89ee000640 1 -- 172.21.15.122:0/2460653733 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89e40b9020 con 0x7f89dc0039c0 2024-09-17T05:25:16.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.966+0000 7f89ee000640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89d8073200 0x7f89d80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:16.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.966+0000 7f89ee000640 1 -- 172.21.15.122:0/2460653733 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f89e412d9e0 con 0x7f89dc0039c0 2024-09-17T05:25:16.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.966+0000 7f89f5400640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89d8073200 0x7f89d80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:16.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.970+0000 7f89f5400640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89d8073200 0x7f89d80756c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89e8005d10 tx=0x7f89e800b310 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:16.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.974+0000 7f5a4e000640 1 -- 172.21.15.122:0/2322371564 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5a48016410 con 0x7f5a58071f00 2024-09-17T05:25:16.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:16.974+0000 7f89ee000640 1 -- 172.21.15.122:0/2460653733 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f89e4016530 con 0x7f89dc0039c0 2024-09-17T05:25:17.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.010+0000 7f07f1000640 1 -- 172.21.15.122:0/2548884604 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07e4037e20 msgr2=0x7f07e4057f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.010+0000 7f07f1000640 1 --2- 172.21.15.122:0/2548884604 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07e4037e20 0x7f07e4057f70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009870 tx=0x7f07d801c870 comp rx=0 tx=0).stop 2024-09-17T05:25:17.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.010+0000 7f07f1000640 1 -- 172.21.15.122:0/2548884604 shutdown_connections 2024-09-17T05:25:17.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.010+0000 7f07f1000640 1 --2- 172.21.15.122:0/2548884604 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07e4037e20 0x7f07e4057f70 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009870 tx=0x7f07d801c870 comp rx=0 tx=0).stop 2024-09-17T05:25:17.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.010+0000 7f07f1000640 1 -- 172.21.15.122:0/2548884604 >> 172.21.15.122:0/2548884604 conn(0x7f07e403a3e0 msgr2=0x7f07e403a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.010+0000 7f07f1000640 1 -- 172.21.15.122:0/2548884604 shutdown_connections 2024-09-17T05:25:17.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.010+0000 7f07f1000640 1 -- 172.21.15.122:0/2548884604 wait complete. 2024-09-17T05:25:17.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.014+0000 7f07f1000640 1 Processor -- start 2024-09-17T05:25:17.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.014+0000 7f07f1000640 1 -- start start 2024-09-17T05:25:17.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.014+0000 7f07f1000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07e40e6e60 0x7f07e40d6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07f1000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e404de70 0x7f07e404e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07e40e6e60 0x7f07e40d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07e40e6e60 0x7f07e40d6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43718/0 (socket says 172.21.15.122:43718) 2024-09-17T05:25:17.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a7be00640 1 -- 172.21.15.122:0/697727510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006ff0 msgr2=0x7f2a740de1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e404de70 0x7f07e404e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e404de70 0x7f07e404e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43122/0 (socket says 172.21.15.122:43122) 2024-09-17T05:25:17.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a79600640 1 -- 172.21.15.122:0/697727510 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c127bc0 con 0x7f2a74006ff0 2024-09-17T05:25:17.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a7be00640 1 --2- 172.21.15.122:0/697727510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006ff0 0x7f2a740de1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0524e0 tx=0x7f2a7c098470 comp rx=0 tx=0).stop 2024-09-17T05:25:17.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 -- 172.21.15.122:0/2382203279 learned_addr learned my addr 172.21.15.122:0/2382203279 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:17.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07f1000640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07e40e26a0 0x7f07e404e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40f4d30 con 0x7f07e404de70 2024-09-17T05:25:17.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a7be00640 1 -- 172.21.15.122:0/697727510 shutdown_connections 2024-09-17T05:25:17.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a7be00640 1 --2- 172.21.15.122:0/697727510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006ff0 0x7f2a740de1c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a7be00640 1 --2- 172.21.15.122:0/697727510 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a74006120 0x7f2a740e30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40f4bb0 con 0x7f07e40e6e60 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f07e40f4eb0 con 0x7f07e40e26a0 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07ebe00640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07e40e26a0 0x7f07e404e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 -- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07e40e26a0 msgr2=0x7f07e404e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07e40e26a0 0x7f07e404e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 -- 172.21.15.122:0/2382203279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e404de70 msgr2=0x7f07e404e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e404de70 0x7f07e404e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 -- 172.21.15.122:0/2382203279 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e40e3690 con 0x7f07e40e6e60 2024-09-17T05:25:17.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07eb400640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07e40e6e60 0x7f07e40d6680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c450 tx=0x7f07e000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a7be00640 1 -- 172.21.15.122:0/697727510 >> 172.21.15.122:0/697727510 conn(0x7f2a7401d780 msgr2=0x7f2a74054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a7be00640 1 -- 172.21.15.122:0/697727510 shutdown_connections 2024-09-17T05:25:17.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07df400640 1 -- 172.21.15.122:0/2382203279 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0016020 con 0x7f07e40e6e60 2024-09-17T05:25:17.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e40e3910 con 0x7f07e40e6e60 2024-09-17T05:25:17.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e40de400 con 0x7f07e40e6e60 2024-09-17T05:25:17.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07df400640 1 -- 172.21.15.122:0/2382203279 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07e0012830 con 0x7f07e40e6e60 2024-09-17T05:25:17.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f07df400640 1 -- 172.21.15.122:0/2382203279 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0012c60 con 0x7f07e40e6e60 2024-09-17T05:25:17.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.022+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c80052a0 con 0x7f07e40e6e60 2024-09-17T05:25:17.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.018+0000 7f2a7be00640 1 -- 172.21.15.122:0/697727510 wait complete. 2024-09-17T05:25:17.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.022+0000 7f2a7be00640 1 Processor -- start 2024-09-17T05:25:17.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.022+0000 7f2a7be00640 1 -- start start 2024-09-17T05:25:17.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006120 0x7f2a740339c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a74006ff0 0x7f2a7400ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006120 0x7f2a740339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a74053ef0 0x7f2a7417ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.022+0000 7f07df400640 1 -- 172.21.15.122:0/2382203279 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f07e0019020 con 0x7f07e40e6e60 2024-09-17T05:25:17.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f07df400640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f07cc073200 0x7f07cc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f07df400640 1 -- 172.21.15.122:0/2382203279 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f07e0093bc0 con 0x7f07e40e6e60 2024-09-17T05:25:17.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f07eaa00640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f07cc073200 0x7f07cc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7a000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a74006ff0 0x7f2a7400ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7a000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a74006ff0 0x7f2a7400ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:43724/0 (socket says 172.21.15.122:43724) 2024-09-17T05:25:17.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006120 0x7f2a740339c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43140/0 (socket says 172.21.15.122:43140) 2024-09-17T05:25:17.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7a000640 1 -- 172.21.15.122:0/948833542 learned_addr learned my addr 172.21.15.122:0/948833542 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:17.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7be00640 1 -- 172.21.15.122:0/948833542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74054e00 con 0x7f2a74006120 2024-09-17T05:25:17.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7be00640 1 -- 172.21.15.122:0/948833542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74054c80 con 0x7f2a74006ff0 2024-09-17T05:25:17.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7be00640 1 -- 172.21.15.122:0/948833542 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2a740de700 con 0x7f2a74053ef0 2024-09-17T05:25:17.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7b400640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a74053ef0 0x7f2a7417ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7a000640 1 -- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a74053ef0 msgr2=0x7f2a7417ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7a000640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a74053ef0 0x7f2a7417ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7a000640 1 -- 172.21.15.122:0/948833542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006120 msgr2=0x7f2a740339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7a000640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006120 0x7f2a740339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.026+0000 7f2a7a000640 1 -- 172.21.15.122:0/948833542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a740489e0 con 0x7f2a74006ff0 2024-09-17T05:25:17.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7f2a7a000640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a74006ff0 0x7f2a7400ac70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c970 tx=0x7f2a6800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7f07eaa00640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f07cc073200 0x7f07cc0756c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f07d80af000 tx=0x7f07d801b0b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7f2a72a00640 1 -- 172.21.15.122:0/948833542 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68007c70 con 0x7f2a74006ff0 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7f2a72a00640 1 -- 172.21.15.122:0/948833542 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a6800f450 con 0x7f2a74006ff0 2024-09-17T05:25:17.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7f07df400640 1 -- 172.21.15.122:0/2382203279 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f07e0020030 con 0x7f07e40e6e60 2024-09-17T05:25:17.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7f2a72a00640 1 -- 172.21.15.122:0/948833542 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800f8a0 con 0x7f2a74006ff0 2024-09-17T05:25:17.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7f2a7be00640 1 -- 172.21.15.122:0/948833542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a74048ca0 con 0x7f2a74006ff0 2024-09-17T05:25:17.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7f2a7be00640 1 -- 172.21.15.122:0/948833542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a74049260 con 0x7f2a74006ff0 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7fe75c800640 1 -- 172.21.15.122:0/400696977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe75415e090 msgr2=0x7fe7540d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7fe75c800640 1 --2- 172.21.15.122:0/400696977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe75415e090 0x7fe7540d1b80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe744009840 tx=0x7fe74401c760 comp rx=0 tx=0).stop 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7fe75c800640 1 -- 172.21.15.122:0/400696977 shutdown_connections 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.030+0000 7fe75c800640 1 --2- 172.21.15.122:0/400696977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe75415e090 0x7fe7540d1b80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.034+0000 7fe75c800640 1 -- 172.21.15.122:0/400696977 >> 172.21.15.122:0/400696977 conn(0x7fe7540d3fb0 msgr2=0x7fe7540d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.034+0000 7fe75c800640 1 -- 172.21.15.122:0/400696977 shutdown_connections 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.034+0000 7fe75c800640 1 -- 172.21.15.122:0/400696977 wait complete. 2024-09-17T05:25:17.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.034+0000 7f2a67e00640 1 -- 172.21.15.122:0/948833542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a74049710 con 0x7f2a74006ff0 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.034+0000 7f2a72a00640 1 -- 172.21.15.122:0/948833542 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2a6800fa40 con 0x7f2a74006ff0 2024-09-17T05:25:17.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.034+0000 7fe75c800640 1 Processor -- start 2024-09-17T05:25:17.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.034+0000 7fe75c800640 1 -- start start 2024-09-17T05:25:17.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7f2a72a00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a5c073200 0x7f2a5c0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7f2a72a00640 1 -- 172.21.15.122:0/948833542 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2a680946c0 con 0x7f2a74006ff0 2024-09-17T05:25:17.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe75c800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7540cfc50 0x7fe7540c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe75c800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75415e090 0x7fe7540c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe75c800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe754162320 0x7fe7540b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe75c800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe75416c680 con 0x7fe7540cfc50 2024-09-17T05:25:17.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe75c800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe75416c500 con 0x7fe754162320 2024-09-17T05:25:17.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75415e090 0x7fe7540c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe75c800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe75416c800 con 0x7fe75415e090 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75415e090 0x7fe7540c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33594/0 (socket says 172.21.15.122:33594) 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 -- 172.21.15.122:0/3930295170 learned_addr learned my addr 172.21.15.122:0/3930295170 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe75a800640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe754162320 0x7fe7540b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759e00640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7540cfc50 0x7fe7540c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 -- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe754162320 msgr2=0x7fe7540b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe754162320 0x7fe7540b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 -- 172.21.15.122:0/3930295170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7540cfc50 msgr2=0x7fe7540c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7540cfc50 0x7fe7540c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 -- 172.21.15.122:0/3930295170 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7540c81c0 con 0x7fe75415e090 2024-09-17T05:25:17.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7f2a7aa00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a5c073200 0x7f2a5c0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe759400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75415e090 0x7fe7540c7370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b4d0 tx=0x7fe74800b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.038+0000 7fe74e000640 1 -- 172.21.15.122:0/3930295170 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748014070 con 0x7fe75415e090 2024-09-17T05:25:17.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.042+0000 7fe74e000640 1 -- 172.21.15.122:0/3930295170 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe748010980 con 0x7fe75415e090 2024-09-17T05:25:17.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.042+0000 7f2a7aa00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a5c073200 0x7f2a5c0756c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0036b0 tx=0x7f2a6c00c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.042+0000 7f2a72a00640 1 -- 172.21.15.122:0/948833542 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2a68014030 con 0x7f2a74006ff0 2024-09-17T05:25:17.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.042+0000 7fe74e000640 1 -- 172.21.15.122:0/3930295170 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748010dd0 con 0x7fe75415e090 2024-09-17T05:25:17.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.042+0000 7fe75c800640 1 -- 172.21.15.122:0/3930295170 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7540c84a0 con 0x7fe75415e090 2024-09-17T05:25:17.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.042+0000 7fe75c800640 1 -- 172.21.15.122:0/3930295170 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe754155480 con 0x7fe75415e090 2024-09-17T05:25:17.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.046+0000 7fe74e000640 1 -- 172.21.15.122:0/3930295170 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe748017070 con 0x7fe75415e090 2024-09-17T05:25:17.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.046+0000 7fe743400640 1 -- 172.21.15.122:0/3930295170 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72c0052a0 con 0x7fe75415e090 2024-09-17T05:25:17.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.046+0000 7fe74e000640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7380732d0 0x7fe738075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.050+0000 7fe74e000640 1 -- 172.21.15.122:0/3930295170 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe748093f50 con 0x7fe75415e090 2024-09-17T05:25:17.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.050+0000 7fe759e00640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7380732d0 0x7fe738075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.050+0000 7fe759e00640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7380732d0 0x7fe738075790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7440af000 tx=0x7fe74401afa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.050+0000 7fe74e000640 1 -- 172.21.15.122:0/3930295170 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe74805e990 con 0x7fe75415e090 2024-09-17T05:25:17.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.082+0000 7f69f7200640 1 -- 172.21.15.122:0/322319362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d8003c90 msgr2=0x7f69d80040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.082+0000 7f69f7200640 1 --2- 172.21.15.122:0/322319362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d8003c90 0x7f69d80040d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009940 tx=0x7f69e801c6f0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.086+0000 7f69f7200640 1 -- 172.21.15.122:0/322319362 shutdown_connections 2024-09-17T05:25:17.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.086+0000 7f69f7200640 1 --2- 172.21.15.122:0/322319362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d8003c90 0x7f69d80040d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.086+0000 7f69f7200640 1 -- 172.21.15.122:0/322319362 >> 172.21.15.122:0/322319362 conn(0x7f69f80b1600 msgr2=0x7f69f80b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.086+0000 7f69f7200640 1 -- 172.21.15.122:0/322319362 shutdown_connections 2024-09-17T05:25:17.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.086+0000 7f69f7200640 1 -- 172.21.15.122:0/322319362 wait complete. 2024-09-17T05:25:17.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.086+0000 7f69f7200640 1 Processor -- start 2024-09-17T05:25:17.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.086+0000 7f69f7200640 1 -- start start 2024-09-17T05:25:17.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f7200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d8003c90 0x7f69f81559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f7200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f8155f30 0x7f69f8157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f8158430 0x7f69f8151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f8155f30 0x7f69f8157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f7200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69f80ce590 con 0x7f69f8155f30 2024-09-17T05:25:17.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f8155f30 0x7f69f8157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43164/0 (socket says 172.21.15.122:43164) 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 -- 172.21.15.122:0/3903382951 learned_addr learned my addr 172.21.15.122:0/3903382951 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f5e00640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d8003c90 0x7f69f81559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69f80ce410 con 0x7f69f8158430 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f69f80ce710 con 0x7f69d8003c90 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f6800640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f8158430 0x7f69f8151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 -- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d8003c90 msgr2=0x7f69f81559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d8003c90 0x7f69f81559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 -- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f8158430 msgr2=0x7f69f8151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f8158430 0x7f69f8151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 -- 172.21.15.122:0/3903382951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f81f92d0 con 0x7f69f8155f30 2024-09-17T05:25:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69f6800640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f8158430 0x7f69f8151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:17.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.090+0000 7f69ed400640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f8155f30 0x7f69f8157ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f69e0003fb0 tx=0x7f69e00064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.094+0000 7f69eea00640 1 -- 172.21.15.122:0/3903382951 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0026070 con 0x7f69f8155f30 2024-09-17T05:25:17.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.094+0000 7f69eea00640 1 -- 172.21.15.122:0/3903382951 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69e000fa00 con 0x7f69f8155f30 2024-09-17T05:25:17.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.094+0000 7f69eea00640 1 -- 172.21.15.122:0/3903382951 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e000fe50 con 0x7f69f8155f30 2024-09-17T05:25:17.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.094+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f81f8510 con 0x7f69f8155f30 2024-09-17T05:25:17.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.094+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81f8ad0 con 0x7f69f8155f30 2024-09-17T05:25:17.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.094+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f80cec70 con 0x7f69f8155f30 2024-09-17T05:25:17.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.098+0000 7f69eea00640 1 -- 172.21.15.122:0/3903382951 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f69e0036440 con 0x7f69f8155f30 2024-09-17T05:25:17.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.098+0000 7f69eea00640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69d4073280 0x7f69d4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.098+0000 7f69eea00640 1 -- 172.21.15.122:0/3903382951 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f69e00a64b0 con 0x7f69f8155f30 2024-09-17T05:25:17.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.106+0000 7f69f5e00640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69d4073280 0x7f69d4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.106+0000 7f69f5e00640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69d4073280 0x7f69d4075740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f69e80ae000 tx=0x7f69e8019490 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.110+0000 7f69eea00640 1 -- 172.21.15.122:0/3903382951 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f69e0070ef0 con 0x7f69f8155f30 2024-09-17T05:25:17.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.250+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd424005860 con 0x7fd44c152900 2024-09-17T05:25:17.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.250+0000 7fd443400640 1 -- 172.21.15.122:0/3851448146 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd438062170 con 0x7fd44c152900 2024-09-17T05:25:17.257 INFO:teuthology.orchestra.run.smithi122.stdout:64424509446 2024-09-17T05:25:17.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd42c073230 msgr2=0x7fd42c0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd42c073230 0x7fd42c0756f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0ae000 tx=0x7fd43c019fb0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd44c152900 msgr2=0x7fd44c14f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd44c152900 0x7fd44c14f280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b3e50 tx=0x7fd438006430 comp rx=0 tx=0).stop 2024-09-17T05:25:17.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 shutdown_connections 2024-09-17T05:25:17.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd42c073230 0x7fd42c0756f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd44c152900 0x7fd44c14f280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd44c157310 0x7fd44c0affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 --2- 172.21.15.122:0/3851448146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd430003c80 0x7fd44c1f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.254+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 >> 172.21.15.122:0/3851448146 conn(0x7fd44c06cc20 msgr2=0x7fd44c159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.258+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 shutdown_connections 2024-09-17T05:25:17.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.258+0000 7fd452000640 1 -- 172.21.15.122:0/3851448146 wait complete. 2024-09-17T05:25:17.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-09-17T05:25:17.290 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:17.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.326+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5a5806d080 con 0x7f5a58071f00 2024-09-17T05:25:17.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.326+0000 7f5a4e000640 1 -- 172.21.15.122:0/2322371564 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5a480f8be0 con 0x7f5a58071f00 2024-09-17T05:25:17.329 INFO:teuthology.orchestra.run.smithi122.stdout:64424509446 2024-09-17T05:25:17.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5a38073400 msgr2=0x7f5a380758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5a38073400 0x7f5a380758c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000f70 tx=0x7f5a44009040 comp rx=0 tx=0).stop 2024-09-17T05:25:17.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a58071f00 msgr2=0x7f5a5815ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a58071f00 0x7f5a5815ff00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5a4801a390 tx=0x7f5a48004040 comp rx=0 tx=0).stop 2024-09-17T05:25:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 shutdown_connections 2024-09-17T05:25:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5a38073400 0x7f5a380758c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a58155b00 0x7f5a58155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a580b6670 0x7f5a580ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 --2- 172.21.15.122:0/2322371564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a58071f00 0x7f5a5815ff00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 >> 172.21.15.122:0/2322371564 conn(0x7f5a580d0cd0 msgr2=0x7f5a5815c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 shutdown_connections 2024-09-17T05:25:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.330+0000 7f5a5ca00640 1 -- 172.21.15.122:0/2322371564 wait complete. 2024-09-17T05:25:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.338+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f89f004b350 con 0x7f89dc0039c0 2024-09-17T05:25:17.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.338+0000 7f89ee000640 1 -- 172.21.15.122:0/2460653733 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f89e40f8420 con 0x7f89dc0039c0 2024-09-17T05:25:17.343 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89d8073200 msgr2=0x7f89d80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89d8073200 0x7f89d80756c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89e8005d10 tx=0x7f89e800b310 comp rx=0 tx=0).stop 2024-09-17T05:25:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 msgr2=0x7f89f0177b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 0x7f89f0177b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89e401ccb0 tx=0x7f89e4003bc0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 shutdown_connections 2024-09-17T05:25:17.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89d8073200 0x7f89d80756c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f002b270 0x7f89f002b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89f004eb10 0x7f89f004ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 --2- 172.21.15.122:0/2460653733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89dc0039c0 0x7f89f0177b00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 >> 172.21.15.122:0/2460653733 conn(0x7f89f0054570 msgr2=0x7f89f00de1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 shutdown_connections 2024-09-17T05:25:17.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.346+0000 7f89f7200640 1 -- 172.21.15.122:0/2460653733 wait complete. 2024-09-17T05:25:17.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.354+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3fa8005590 con 0x7f3fc40e6d00 2024-09-17T05:25:17.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-09-17T05:25:17.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.354+0000 7f3fbfe00640 1 -- 172.21.15.122:0/3242347761 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3fcc0de630 con 0x7f3fc40e6d00 2024-09-17T05:25:17.359 INFO:teuthology.orchestra.run.smithi122.stdout:64424509448 2024-09-17T05:25:17.359 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3fac073200 msgr2=0x7f3fac0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3fac073200 0x7f3fac0756c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0005fd0 tx=0x7f3fc000a4b0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 msgr2=0x7f3fc403ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 0x7f3fc403ab00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc06cb90 tx=0x7f3fcc06a300 comp rx=0 tx=0).stop 2024-09-17T05:25:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 shutdown_connections 2024-09-17T05:25:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3fac073200 0x7f3fac0756c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fc40e6d00 0x7f3fc403ab00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fc40e6920 0x7f3fc4038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 --2- 172.21.15.122:0/3242347761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc40576d0 0x7f3fc40d6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 >> 172.21.15.122:0/3242347761 conn(0x7f3fc4055360 msgr2=0x7f3fc40dcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 shutdown_connections 2024-09-17T05:25:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.358+0000 7f3fd2e00640 1 -- 172.21.15.122:0/3242347761 wait complete. 2024-09-17T05:25:17.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.374+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f07c8005590 con 0x7f07e40e6e60 2024-09-17T05:25:17.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.374+0000 7f07df400640 1 -- 172.21.15.122:0/2382203279 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f07e40e3690 con 0x7f07e40e6e60 2024-09-17T05:25:17.379 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f07cc073200 msgr2=0x7f07cc0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f07cc073200 0x7f07cc0756c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f07d80af000 tx=0x7f07d801b0b0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07e40e6e60 msgr2=0x7f07e40d6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07e40e6e60 0x7f07e40d6680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c450 tx=0x7f07e000c910 comp rx=0 tx=0).stop 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 shutdown_connections 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f07cc073200 0x7f07cc0756c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07e40e26a0 0x7f07e404e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e404de70 0x7f07e404e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 --2- 172.21.15.122:0/2382203279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07e40e6e60 0x7f07e40d6680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 >> 172.21.15.122:0/2382203279 conn(0x7f07e403a3e0 msgr2=0x7f07e40e5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 shutdown_connections 2024-09-17T05:25:17.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.378+0000 7f07f1000640 1 -- 172.21.15.122:0/2382203279 wait complete. 2024-09-17T05:25:17.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-09-17T05:25:17.384 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:17.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-09-17T05:25:17.390 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:17.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.394+0000 7f2a67e00640 1 -- 172.21.15.122:0/948833542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2a7401dc80 con 0x7f2a74006ff0 2024-09-17T05:25:17.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.394+0000 7f2a72a00640 1 -- 172.21.15.122:0/948833542 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2a6805f080 con 0x7f2a74006ff0 2024-09-17T05:25:17.397 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 -- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a5c073200 msgr2=0x7f2a5c0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a5c073200 0x7f2a5c0756c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0036b0 tx=0x7f2a6c00c040 comp rx=0 tx=0).stop 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 -- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a74006ff0 msgr2=0x7f2a7400ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a74006ff0 0x7f2a7400ac70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c970 tx=0x7f2a6800ce30 comp rx=0 tx=0).stop 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 -- 172.21.15.122:0/948833542 shutdown_connections 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a5c073200 0x7f2a5c0756c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a74053ef0 0x7f2a7417ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a74006ff0 0x7f2a7400ac70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 --2- 172.21.15.122:0/948833542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a74006120 0x7f2a740339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 -- 172.21.15.122:0/948833542 >> 172.21.15.122:0/948833542 conn(0x7f2a7401d780 msgr2=0x7f2a740d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 -- 172.21.15.122:0/948833542 shutdown_connections 2024-09-17T05:25:17.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.398+0000 7f2a67e00640 1 -- 172.21.15.122:0/948833542 wait complete. 2024-09-17T05:25:17.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 -- 172.21.15.122:0/3930295170 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe72c005590 con 0x7fe75415e090 2024-09-17T05:25:17.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe74e000640 1 -- 172.21.15.122:0/3930295170 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe7480633c0 con 0x7fe75415e090 2024-09-17T05:25:17.404 INFO:teuthology.orchestra.run.smithi122.stdout:64424509447 2024-09-17T05:25:17.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-09-17T05:25:17.405 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:17.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 -- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7380732d0 msgr2=0x7fe738075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7380732d0 0x7fe738075790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe7440af000 tx=0x7fe74401afa0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 -- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75415e090 msgr2=0x7fe7540c7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75415e090 0x7fe7540c7370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b4d0 tx=0x7fe74800b990 comp rx=0 tx=0).stop 2024-09-17T05:25:17.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 -- 172.21.15.122:0/3930295170 shutdown_connections 2024-09-17T05:25:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7380732d0 0x7fe738075790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe754162320 0x7fe7540b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75415e090 0x7fe7540c7370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 --2- 172.21.15.122:0/3930295170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7540cfc50 0x7fe7540c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 -- 172.21.15.122:0/3930295170 >> 172.21.15.122:0/3930295170 conn(0x7fe7540d3fb0 msgr2=0x7fe75406e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 -- 172.21.15.122:0/3930295170 shutdown_connections 2024-09-17T05:25:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.402+0000 7fe743400640 1 -- 172.21.15.122:0/3930295170 wait complete. 2024-09-17T05:25:17.423 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-17T05:25:17.423 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:17.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-17T05:25:17.428 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:17.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f69f8073c60 con 0x7f69f8155f30 2024-09-17T05:25:17.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69eea00640 1 -- 172.21.15.122:0/3903382951 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f69f8073c60 con 0x7f69f8155f30 2024-09-17T05:25:17.440 INFO:teuthology.orchestra.run.smithi122.stdout:60129542152 2024-09-17T05:25:17.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69d4073280 msgr2=0x7f69d4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69d4073280 0x7f69d4075740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f69e80ae000 tx=0x7f69e8019490 comp rx=0 tx=0).stop 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f8155f30 msgr2=0x7f69f8157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f8155f30 0x7f69f8157ef0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f69e0003fb0 tx=0x7f69e00064f0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 shutdown_connections 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69d4073280 0x7f69d4075740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f8158430 0x7f69f8151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69f8155f30 0x7f69f8157ef0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 --2- 172.21.15.122:0/3903382951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d8003c90 0x7f69f81559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 >> 172.21.15.122:0/3903382951 conn(0x7f69f80b1600 msgr2=0x7f69f8072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 shutdown_connections 2024-09-17T05:25:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.438+0000 7f69f7200640 1 -- 172.21.15.122:0/3903382951 wait complete. 2024-09-17T05:25:17.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.5 2024-09-17T05:25:17.461 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:17.461 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:25:17.461 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:17.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.638+0000 7f983c600640 1 -- 172.21.15.122:0/3725847301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 msgr2=0x7f9834165950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.638+0000 7f983c600640 1 --2- 172.21.15.122:0/3725847301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 0x7f9834165950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f982c009900 tx=0x7f982c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.638+0000 7f983c600640 1 -- 172.21.15.122:0/3725847301 shutdown_connections 2024-09-17T05:25:17.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.638+0000 7f983c600640 1 --2- 172.21.15.122:0/3725847301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 0x7f9834165950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.638+0000 7f983c600640 1 -- 172.21.15.122:0/3725847301 >> 172.21.15.122:0/3725847301 conn(0x7f98340b18f0 msgr2=0x7f98340b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.638+0000 7f983c600640 1 -- 172.21.15.122:0/3725847301 shutdown_connections 2024-09-17T05:25:17.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.638+0000 7f983c600640 1 -- 172.21.15.122:0/3725847301 wait complete. 2024-09-17T05:25:17.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 Processor -- start 2024-09-17T05:25:17.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 -- start start 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 0x7f983414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98340b4bd0 0x7f98340acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f983415b580 0x7f98341fb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9834168ba0 con 0x7f98340b3300 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9834168a20 con 0x7f983415b580 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9834168d20 con 0x7f98340b4bd0 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 0x7f983414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 0x7f983414aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43180/0 (socket says 172.21.15.122:43180) 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 -- 172.21.15.122:0/2789964922 learned_addr learned my addr 172.21.15.122:0/2789964922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983a600640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f983415b580 0x7f98341fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839200640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98340b4bd0 0x7f98340acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 -- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98340b4bd0 msgr2=0x7f98340acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98340b4bd0 0x7f98340acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 -- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f983415b580 msgr2=0x7f98341fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f983415b580 0x7f98341fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 -- 172.21.15.122:0/2789964922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983414b840 con 0x7f98340b3300 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839200640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98340b4bd0 0x7f98340acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9839c00640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 0x7f983414aed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f982800ba50 tx=0x7f982800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9831600640 1 -- 172.21.15.122:0/2789964922 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828019070 con 0x7f98340b3300 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9831600640 1 -- 172.21.15.122:0/2789964922 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f982800cd10 con 0x7f98340b3300 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9834072910 con 0x7f98340b3300 2024-09-17T05:25:17.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f9831600640 1 -- 172.21.15.122:0/2789964922 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98280146a0 con 0x7f98340b3300 2024-09-17T05:25:17.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.642+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983414bb20 con 0x7f98340b3300 2024-09-17T05:25:17.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.646+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98340b2bf0 con 0x7f98340b3300 2024-09-17T05:25:17.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.646+0000 7f9831600640 1 -- 172.21.15.122:0/2789964922 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f982801c070 con 0x7f98340b3300 2024-09-17T05:25:17.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.646+0000 7f9831600640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9814073200 0x7f98140756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:17.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.646+0000 7f9831600640 1 -- 172.21.15.122:0/2789964922 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9828093ff0 con 0x7f98340b3300 2024-09-17T05:25:17.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.646+0000 7f9839200640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9814073200 0x7f98140756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:17.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.650+0000 7f9839200640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9814073200 0x7f98140756c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98340b09c0 tx=0x7f9820009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:17.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.650+0000 7f9831600640 1 -- 172.21.15.122:0/2789964922 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f982805e940 con 0x7f98340b3300 2024-09-17T05:25:17.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.814+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98340b1380 con 0x7f9814073200 2024-09-17T05:25:17.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.818+0000 7f9831600640 1 -- 172.21.15.122:0/2789964922 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f98340b1380 con 0x7f9814073200 2024-09-17T05:25:17.824 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:17.825 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:17.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9814073200 msgr2=0x7f98140756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9814073200 0x7f98140756c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98340b09c0 tx=0x7f9820009040 comp rx=0 tx=0).stop 2024-09-17T05:25:17.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 msgr2=0x7f983414aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:17.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 0x7f983414aed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f982800ba50 tx=0x7f982800bf10 comp rx=0 tx=0).stop 2024-09-17T05:25:17.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 shutdown_connections 2024-09-17T05:25:17.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9814073200 0x7f98140756c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f983415b580 0x7f98341fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98340b4bd0 0x7f98340acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 --2- 172.21.15.122:0/2789964922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98340b3300 0x7f983414aed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:17.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 >> 172.21.15.122:0/2789964922 conn(0x7f98340b18f0 msgr2=0x7f983415b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:17.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 shutdown_connections 2024-09-17T05:25:17.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:17.822+0000 7f983c600640 1 -- 172.21.15.122:0/2789964922 wait complete. 2024-09-17T05:25:17.845 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-17T05:25:16.444700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223364,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751356,"statfs":{"total":773094113280,"available":772865388544,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.294153"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320847+0000","last_change":"2024-09-17T05:25:04.154692+0000","last_active":"2024-09-17T05:25:05.320847+0000","last_peered":"2024-09-17T05:25:05.320847+0000","last_clean":"2024-09-17T05:25:05.320847+0000","last_became_active":"2024-09-17T05:25:04.148232+0000","last_became_peered":"2024-09-17T05:25:04.148232+0000","last_unstale":"2024-09-17T05:25:05.320847+0000","last_undegraded":"2024-09-17T05:25:05.320847+0000","last_fullsized":"2024-09-17T05:25:05.320847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:30.690541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324353+0000","last_change":"2024-09-17T05:25:04.173588+0000","last_active":"2024-09-17T05:25:05.324353+0000","last_peered":"2024-09-17T05:25:05.324353+0000","last_clean":"2024-09-17T05:25:05.324353+0000","last_became_active":"2024-09-17T05:25:04.173260+0000","last_became_peered":"2024-09-17T05:25:04.173260+0000","last_unstale":"2024-09-17T05:25:05.324353+0000","last_undegraded":"2024-09-17T05:25:05.324353+0000","last_fullsized":"2024-09-17T05:25:05.324353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:13.477698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324934+0000","last_change":"2024-09-17T05:25:04.174132+0000","last_active":"2024-09-17T05:25:05.324934+0000","last_peered":"2024-09-17T05:25:05.324934+0000","last_clean":"2024-09-17T05:25:05.324934+0000","last_became_active":"2024-09-17T05:25:04.173337+0000","last_became_peered":"2024-09-17T05:25:04.173337+0000","last_unstale":"2024-09-17T05:25:05.324934+0000","last_undegraded":"2024-09-17T05:25:05.324934+0000","last_fullsized":"2024-09-17T05:25:05.324934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:50.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320851+0000","last_change":"2024-09-17T05:25:04.166451+0000","last_active":"2024-09-17T05:25:05.320851+0000","last_peered":"2024-09-17T05:25:05.320851+0000","last_clean":"2024-09-17T05:25:05.320851+0000","last_became_active":"2024-09-17T05:25:04.166193+0000","last_became_peered":"2024-09-17T05:25:04.166193+0000","last_unstale":"2024-09-17T05:25:05.320851+0000","last_undegraded":"2024-09-17T05:25:05.320851+0000","last_fullsized":"2024-09-17T05:25:05.320851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320896+0000","last_change":"2024-09-17T05:25:04.166447+0000","last_active":"2024-09-17T05:25:05.320896+0000","last_peered":"2024-09-17T05:25:05.320896+0000","last_clean":"2024-09-17T05:25:05.320896+0000","last_became_active":"2024-09-17T05:25:04.166240+0000","last_became_peered":"2024-09-17T05:25:04.166240+0000","last_unstale":"2024-09-17T05:25:05.320896+0000","last_undegraded":"2024-09-17T05:25:05.320896+0000","last_fullsized":"2024-09-17T05:25:05.320896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:20.054457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320554+0000","last_change":"2024-09-17T05:25:04.166670+0000","last_active":"2024-09-17T05:25:05.320554+0000","last_peered":"2024-09-17T05:25:05.320554+0000","last_clean":"2024-09-17T05:25:05.320554+0000","last_became_active":"2024-09-17T05:25:04.166212+0000","last_became_peered":"2024-09-17T05:25:04.166212+0000","last_unstale":"2024-09-17T05:25:05.320554+0000","last_undegraded":"2024-09-17T05:25:05.320554+0000","last_fullsized":"2024-09-17T05:25:05.320554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320762+0000","last_change":"2024-09-17T05:25:04.171397+0000","last_active":"2024-09-17T05:25:05.320762+0000","last_peered":"2024-09-17T05:25:05.320762+0000","last_clean":"2024-09-17T05:25:05.320762+0000","last_became_active":"2024-09-17T05:25:04.171157+0000","last_became_peered":"2024-09-17T05:25:04.171157+0000","last_unstale":"2024-09-17T05:25:05.320762+0000","last_undegraded":"2024-09-17T05:25:05.320762+0000","last_fullsized":"2024-09-17T05:25:05.320762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:49.992366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324125+0000","last_change":"2024-09-17T05:25:04.166345+0000","last_active":"2024-09-17T05:25:05.324125+0000","last_peered":"2024-09-17T05:25:05.324125+0000","last_clean":"2024-09-17T05:25:05.324125+0000","last_became_active":"2024-09-17T05:25:04.165795+0000","last_became_peered":"2024-09-17T05:25:04.165795+0000","last_unstale":"2024-09-17T05:25:05.324125+0000","last_undegraded":"2024-09-17T05:25:05.324125+0000","last_fullsized":"2024-09-17T05:25:05.324125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:33.426170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153496+0000","last_change":"2024-09-17T05:25:04.155768+0000","last_active":"2024-09-17T05:25:05.153496+0000","last_peered":"2024-09-17T05:25:05.153496+0000","last_clean":"2024-09-17T05:25:05.153496+0000","last_became_active":"2024-09-17T05:25:04.155277+0000","last_became_peered":"2024-09-17T05:25:04.155277+0000","last_unstale":"2024-09-17T05:25:05.153496+0000","last_undegraded":"2024-09-17T05:25:05.153496+0000","last_fullsized":"2024-09-17T05:25:05.153496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:44.070039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321217+0000","last_change":"2024-09-17T05:25:04.144665+0000","last_active":"2024-09-17T05:25:05.321217+0000","last_peered":"2024-09-17T05:25:05.321217+0000","last_clean":"2024-09-17T05:25:05.321217+0000","last_became_active":"2024-09-17T05:25:04.144382+0000","last_became_peered":"2024-09-17T05:25:04.144382+0000","last_unstale":"2024-09-17T05:25:05.321217+0000","last_undegraded":"2024-09-17T05:25:05.321217+0000","last_fullsized":"2024-09-17T05:25:05.321217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:09.785934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321369+0000","last_change":"2024-09-17T05:25:04.153782+0000","last_active":"2024-09-17T05:25:05.321369+0000","last_peered":"2024-09-17T05:25:05.321369+0000","last_clean":"2024-09-17T05:25:05.321369+0000","last_became_active":"2024-09-17T05:25:04.148194+0000","last_became_peered":"2024-09-17T05:25:04.148194+0000","last_unstale":"2024-09-17T05:25:05.321369+0000","last_undegraded":"2024-09-17T05:25:05.321369+0000","last_fullsized":"2024-09-17T05:25:05.321369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:44.683027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321040+0000","last_change":"2024-09-17T05:25:04.158630+0000","last_active":"2024-09-17T05:25:05.321040+0000","last_peered":"2024-09-17T05:25:05.321040+0000","last_clean":"2024-09-17T05:25:05.321040+0000","last_became_active":"2024-09-17T05:25:04.158382+0000","last_became_peered":"2024-09-17T05:25:04.158382+0000","last_unstale":"2024-09-17T05:25:05.321040+0000","last_undegraded":"2024-09-17T05:25:05.321040+0000","last_fullsized":"2024-09-17T05:25:05.321040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:27.039745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324868+0000","last_change":"2024-09-17T05:25:04.171927+0000","last_active":"2024-09-17T05:25:05.324868+0000","last_peered":"2024-09-17T05:25:05.324868+0000","last_clean":"2024-09-17T05:25:05.324868+0000","last_became_active":"2024-09-17T05:25:04.171634+0000","last_became_peered":"2024-09-17T05:25:04.171634+0000","last_unstale":"2024-09-17T05:25:05.324868+0000","last_undegraded":"2024-09-17T05:25:05.324868+0000","last_fullsized":"2024-09-17T05:25:05.324868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:09.532282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323783+0000","last_change":"2024-09-17T05:25:04.171235+0000","last_active":"2024-09-17T05:25:05.323783+0000","last_peered":"2024-09-17T05:25:05.323783+0000","last_clean":"2024-09-17T05:25:05.323783+0000","last_became_active":"2024-09-17T05:25:04.170717+0000","last_became_peered":"2024-09-17T05:25:04.170717+0000","last_unstale":"2024-09-17T05:25:05.323783+0000","last_undegraded":"2024-09-17T05:25:05.323783+0000","last_fullsized":"2024-09-17T05:25:05.323783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:32.411584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324904+0000","last_change":"2024-09-17T05:25:04.163764+0000","last_active":"2024-09-17T05:25:05.324904+0000","last_peered":"2024-09-17T05:25:05.324904+0000","last_clean":"2024-09-17T05:25:05.324904+0000","last_became_active":"2024-09-17T05:25:04.163197+0000","last_became_peered":"2024-09-17T05:25:04.163197+0000","last_unstale":"2024-09-17T05:25:05.324904+0000","last_undegraded":"2024-09-17T05:25:05.324904+0000","last_fullsized":"2024-09-17T05:25:05.324904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:01:59.441556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150239+0000","last_change":"2024-09-17T05:25:04.147588+0000","last_active":"2024-09-17T05:25:05.150239+0000","last_peered":"2024-09-17T05:25:05.150239+0000","last_clean":"2024-09-17T05:25:05.150239+0000","last_became_active":"2024-09-17T05:25:04.147316+0000","last_became_peered":"2024-09-17T05:25:04.147316+0000","last_unstale":"2024-09-17T05:25:05.150239+0000","last_undegraded":"2024-09-17T05:25:05.150239+0000","last_fullsized":"2024-09-17T05:25:05.150239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:44.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151006+0000","last_change":"2024-09-17T05:25:04.149293+0000","last_active":"2024-09-17T05:25:05.151006+0000","last_peered":"2024-09-17T05:25:05.151006+0000","last_clean":"2024-09-17T05:25:05.151006+0000","last_became_active":"2024-09-17T05:25:04.149079+0000","last_became_peered":"2024-09-17T05:25:04.149079+0000","last_unstale":"2024-09-17T05:25:05.151006+0000","last_undegraded":"2024-09-17T05:25:05.151006+0000","last_fullsized":"2024-09-17T05:25:05.151006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:13.903114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153437+0000","last_change":"2024-09-17T05:25:04.152808+0000","last_active":"2024-09-17T05:25:05.153437+0000","last_peered":"2024-09-17T05:25:05.153437+0000","last_clean":"2024-09-17T05:25:05.153437+0000","last_became_active":"2024-09-17T05:25:04.152501+0000","last_became_peered":"2024-09-17T05:25:04.152501+0000","last_unstale":"2024-09-17T05:25:05.153437+0000","last_undegraded":"2024-09-17T05:25:05.153437+0000","last_fullsized":"2024-09-17T05:25:05.153437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150922+0000","last_change":"2024-09-17T05:25:04.167124+0000","last_active":"2024-09-17T05:25:05.150922+0000","last_peered":"2024-09-17T05:25:05.150922+0000","last_clean":"2024-09-17T05:25:05.150922+0000","last_became_active":"2024-09-17T05:25:04.166853+0000","last_became_peered":"2024-09-17T05:25:04.166853+0000","last_unstale":"2024-09-17T05:25:05.150922+0000","last_undegraded":"2024-09-17T05:25:05.150922+0000","last_fullsized":"2024-09-17T05:25:05.150922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:05.283255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321253+0000","last_change":"2024-09-17T05:25:04.160482+0000","last_active":"2024-09-17T05:25:05.321253+0000","last_peered":"2024-09-17T05:25:05.321253+0000","last_clean":"2024-09-17T05:25:05.321253+0000","last_became_active":"2024-09-17T05:25:04.156878+0000","last_became_peered":"2024-09-17T05:25:04.156878+0000","last_unstale":"2024-09-17T05:25:05.321253+0000","last_undegraded":"2024-09-17T05:25:05.321253+0000","last_fullsized":"2024-09-17T05:25:05.321253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:44.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323874+0000","last_change":"2024-09-17T05:25:04.169764+0000","last_active":"2024-09-17T05:25:05.323874+0000","last_peered":"2024-09-17T05:25:05.323874+0000","last_clean":"2024-09-17T05:25:05.323874+0000","last_became_active":"2024-09-17T05:25:04.167730+0000","last_became_peered":"2024-09-17T05:25:04.167730+0000","last_unstale":"2024-09-17T05:25:05.323874+0000","last_undegraded":"2024-09-17T05:25:05.323874+0000","last_fullsized":"2024-09-17T05:25:05.323874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:13.526132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152880+0000","last_change":"2024-09-17T05:25:04.163296+0000","last_active":"2024-09-17T05:25:05.152880+0000","last_peered":"2024-09-17T05:25:05.152880+0000","last_clean":"2024-09-17T05:25:05.152880+0000","last_became_active":"2024-09-17T05:25:04.162996+0000","last_became_peered":"2024-09-17T05:25:04.162996+0000","last_unstale":"2024-09-17T05:25:05.152880+0000","last_undegraded":"2024-09-17T05:25:05.152880+0000","last_fullsized":"2024-09-17T05:25:05.152880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:44.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325148+0000","last_change":"2024-09-17T05:25:04.158936+0000","last_active":"2024-09-17T05:25:05.325148+0000","last_peered":"2024-09-17T05:25:05.325148+0000","last_clean":"2024-09-17T05:25:05.325148+0000","last_became_active":"2024-09-17T05:25:04.158697+0000","last_became_peered":"2024-09-17T05:25:04.158697+0000","last_unstale":"2024-09-17T05:25:05.325148+0000","last_undegraded":"2024-09-17T05:25:05.325148+0000","last_fullsized":"2024-09-17T05:25:05.325148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:45:18.794538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150244+0000","last_change":"2024-09-17T05:25:04.160086+0000","last_active":"2024-09-17T05:25:05.150244+0000","last_peered":"2024-09-17T05:25:05.150244+0000","last_clean":"2024-09-17T05:25:05.150244+0000","last_became_active":"2024-09-17T05:25:04.159826+0000","last_became_peered":"2024-09-17T05:25:04.159826+0000","last_unstale":"2024-09-17T05:25:05.150244+0000","last_undegraded":"2024-09-17T05:25:05.150244+0000","last_fullsized":"2024-09-17T05:25:05.150244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:46.502824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324281+0000","last_change":"2024-09-17T05:25:04.165631+0000","last_active":"2024-09-17T05:25:05.324281+0000","last_peered":"2024-09-17T05:25:05.324281+0000","last_clean":"2024-09-17T05:25:05.324281+0000","last_became_active":"2024-09-17T05:25:04.163086+0000","last_became_peered":"2024-09-17T05:25:04.163086+0000","last_unstale":"2024-09-17T05:25:05.324281+0000","last_undegraded":"2024-09-17T05:25:05.324281+0000","last_fullsized":"2024-09-17T05:25:05.324281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:59.610017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321281+0000","last_change":"2024-09-17T05:25:04.152176+0000","last_active":"2024-09-17T05:25:05.321281+0000","last_peered":"2024-09-17T05:25:05.321281+0000","last_clean":"2024-09-17T05:25:05.321281+0000","last_became_active":"2024-09-17T05:25:04.151035+0000","last_became_peered":"2024-09-17T05:25:04.151035+0000","last_unstale":"2024-09-17T05:25:05.321281+0000","last_undegraded":"2024-09-17T05:25:05.321281+0000","last_fullsized":"2024-09-17T05:25:05.321281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:06.019281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325437+0000","last_change":"2024-09-17T05:25:04.172127+0000","last_active":"2024-09-17T05:25:05.325437+0000","last_peered":"2024-09-17T05:25:05.325437+0000","last_clean":"2024-09-17T05:25:05.325437+0000","last_became_active":"2024-09-17T05:25:04.159538+0000","last_became_peered":"2024-09-17T05:25:04.159538+0000","last_unstale":"2024-09-17T05:25:05.325437+0000","last_undegraded":"2024-09-17T05:25:05.325437+0000","last_fullsized":"2024-09-17T05:25:05.325437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:46.839572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324955+0000","last_change":"2024-09-17T05:25:04.176025+0000","last_active":"2024-09-17T05:25:05.324955+0000","last_peered":"2024-09-17T05:25:05.324955+0000","last_clean":"2024-09-17T05:25:05.324955+0000","last_became_active":"2024-09-17T05:25:04.174826+0000","last_became_peered":"2024-09-17T05:25:04.174826+0000","last_unstale":"2024-09-17T05:25:05.324955+0000","last_undegraded":"2024-09-17T05:25:05.324955+0000","last_fullsized":"2024-09-17T05:25:05.324955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:37.465014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325254+0000","last_change":"2024-09-17T05:25:04.162109+0000","last_active":"2024-09-17T05:25:05.325254+0000","last_peered":"2024-09-17T05:25:05.325254+0000","last_clean":"2024-09-17T05:25:05.325254+0000","last_became_active":"2024-09-17T05:25:04.161807+0000","last_became_peered":"2024-09-17T05:25:04.161807+0000","last_unstale":"2024-09-17T05:25:05.325254+0000","last_undegraded":"2024-09-17T05:25:05.325254+0000","last_fullsized":"2024-09-17T05:25:05.325254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:23.271600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150335+0000","last_change":"2024-09-17T05:25:04.160575+0000","last_active":"2024-09-17T05:25:05.150335+0000","last_peered":"2024-09-17T05:25:05.150335+0000","last_clean":"2024-09-17T05:25:05.150335+0000","last_became_active":"2024-09-17T05:25:04.160104+0000","last_became_peered":"2024-09-17T05:25:04.160104+0000","last_unstale":"2024-09-17T05:25:05.150335+0000","last_undegraded":"2024-09-17T05:25:05.150335+0000","last_fullsized":"2024-09-17T05:25:05.150335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:22.186817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153243+0000","last_change":"2024-09-17T05:25:04.156777+0000","last_active":"2024-09-17T05:25:05.153243+0000","last_peered":"2024-09-17T05:25:05.153243+0000","last_clean":"2024-09-17T05:25:05.153243+0000","last_became_active":"2024-09-17T05:25:04.156013+0000","last_became_peered":"2024-09-17T05:25:04.156013+0000","last_unstale":"2024-09-17T05:25:05.153243+0000","last_undegraded":"2024-09-17T05:25:05.153243+0000","last_fullsized":"2024-09-17T05:25:05.153243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320365+0000","last_change":"2024-09-17T05:25:04.170636+0000","last_active":"2024-09-17T05:25:05.320365+0000","last_peered":"2024-09-17T05:25:05.320365+0000","last_clean":"2024-09-17T05:25:05.320365+0000","last_became_active":"2024-09-17T05:25:04.170325+0000","last_became_peered":"2024-09-17T05:25:04.170325+0000","last_unstale":"2024-09-17T05:25:05.320365+0000","last_undegraded":"2024-09-17T05:25:05.320365+0000","last_fullsized":"2024-09-17T05:25:05.320365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:09.604647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152897+0000","last_change":"2024-09-17T05:25:04.162069+0000","last_active":"2024-09-17T05:25:05.152897+0000","last_peered":"2024-09-17T05:25:05.152897+0000","last_clean":"2024-09-17T05:25:05.152897+0000","last_became_active":"2024-09-17T05:25:04.161443+0000","last_became_peered":"2024-09-17T05:25:04.161443+0000","last_unstale":"2024-09-17T05:25:05.152897+0000","last_undegraded":"2024-09-17T05:25:05.152897+0000","last_fullsized":"2024-09-17T05:25:05.152897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325321+0000","last_change":"2024-09-17T05:25:03.160754+0000","last_active":"2024-09-17T05:25:05.325321+0000","last_peered":"2024-09-17T05:25:05.325321+0000","last_clean":"2024-09-17T05:25:05.325321+0000","last_became_active":"2024-09-17T05:25:03.160433+0000","last_became_peered":"2024-09-17T05:25:03.160433+0000","last_unstale":"2024-09-17T05:25:05.325321+0000","last_undegraded":"2024-09-17T05:25:05.325321+0000","last_fullsized":"2024-09-17T05:25:05.325321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:59.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324073+0000","last_change":"2024-09-17T05:25:04.174431+0000","last_active":"2024-09-17T05:25:05.324073+0000","last_peered":"2024-09-17T05:25:05.324073+0000","last_clean":"2024-09-17T05:25:05.324073+0000","last_became_active":"2024-09-17T05:25:04.174139+0000","last_became_peered":"2024-09-17T05:25:04.174139+0000","last_unstale":"2024-09-17T05:25:05.324073+0000","last_undegraded":"2024-09-17T05:25:05.324073+0000","last_fullsized":"2024-09-17T05:25:05.324073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:40.319110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150517+0000","last_change":"2024-09-17T05:25:03.138686+0000","last_active":"2024-09-17T05:25:05.150517+0000","last_peered":"2024-09-17T05:25:05.150517+0000","last_clean":"2024-09-17T05:25:05.150517+0000","last_became_active":"2024-09-17T05:25:03.138321+0000","last_became_peered":"2024-09-17T05:25:03.138321+0000","last_unstale":"2024-09-17T05:25:05.150517+0000","last_undegraded":"2024-09-17T05:25:05.150517+0000","last_fullsized":"2024-09-17T05:25:05.150517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:03.903924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323451+0000","last_change":"2024-09-17T05:25:03.161161+0000","last_active":"2024-09-17T05:25:05.323451+0000","last_peered":"2024-09-17T05:25:05.323451+0000","last_clean":"2024-09-17T05:25:05.323451+0000","last_became_active":"2024-09-17T05:25:03.160876+0000","last_became_peered":"2024-09-17T05:25:03.160876+0000","last_unstale":"2024-09-17T05:25:05.323451+0000","last_undegraded":"2024-09-17T05:25:05.323451+0000","last_fullsized":"2024-09-17T05:25:05.323451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324408+0000","last_change":"2024-09-17T05:25:04.173583+0000","last_active":"2024-09-17T05:25:05.324408+0000","last_peered":"2024-09-17T05:25:05.324408+0000","last_clean":"2024-09-17T05:25:05.324408+0000","last_became_active":"2024-09-17T05:25:04.173276+0000","last_became_peered":"2024-09-17T05:25:04.173276+0000","last_unstale":"2024-09-17T05:25:05.324408+0000","last_undegraded":"2024-09-17T05:25:05.324408+0000","last_fullsized":"2024-09-17T05:25:05.324408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:38.290022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150430+0000","last_change":"2024-09-17T05:25:03.156886+0000","last_active":"2024-09-17T05:25:05.150430+0000","last_peered":"2024-09-17T05:25:05.150430+0000","last_clean":"2024-09-17T05:25:05.150430+0000","last_became_active":"2024-09-17T05:25:03.156595+0000","last_became_peered":"2024-09-17T05:25:03.156595+0000","last_unstale":"2024-09-17T05:25:05.150430+0000","last_undegraded":"2024-09-17T05:25:05.150430+0000","last_fullsized":"2024-09-17T05:25:05.150430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:57.284513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323745+0000","last_change":"2024-09-17T05:25:04.167294+0000","last_active":"2024-09-17T05:25:05.323745+0000","last_peered":"2024-09-17T05:25:05.323745+0000","last_clean":"2024-09-17T05:25:05.323745+0000","last_became_active":"2024-09-17T05:25:04.165115+0000","last_became_peered":"2024-09-17T05:25:04.165115+0000","last_unstale":"2024-09-17T05:25:05.323745+0000","last_undegraded":"2024-09-17T05:25:05.323745+0000","last_fullsized":"2024-09-17T05:25:05.323745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:53.176691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153460+0000","last_change":"2024-09-17T05:25:03.160352+0000","last_active":"2024-09-17T05:25:05.153460+0000","last_peered":"2024-09-17T05:25:05.153460+0000","last_clean":"2024-09-17T05:25:05.153460+0000","last_became_active":"2024-09-17T05:25:03.160030+0000","last_became_peered":"2024-09-17T05:25:03.160030+0000","last_unstale":"2024-09-17T05:25:05.153460+0000","last_undegraded":"2024-09-17T05:25:05.153460+0000","last_fullsized":"2024-09-17T05:25:05.153460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321483+0000","last_change":"2024-09-17T05:25:04.146118+0000","last_active":"2024-09-17T05:25:05.321483+0000","last_peered":"2024-09-17T05:25:05.321483+0000","last_clean":"2024-09-17T05:25:05.321483+0000","last_became_active":"2024-09-17T05:25:04.145700+0000","last_became_peered":"2024-09-17T05:25:04.145700+0000","last_unstale":"2024-09-17T05:25:05.321483+0000","last_undegraded":"2024-09-17T05:25:05.321483+0000","last_fullsized":"2024-09-17T05:25:05.321483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:44.070039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321398+0000","last_change":"2024-09-17T05:25:03.155547+0000","last_active":"2024-09-17T05:25:05.321398+0000","last_peered":"2024-09-17T05:25:05.321398+0000","last_clean":"2024-09-17T05:25:05.321398+0000","last_became_active":"2024-09-17T05:25:03.155270+0000","last_became_peered":"2024-09-17T05:25:03.155270+0000","last_unstale":"2024-09-17T05:25:05.321398+0000","last_undegraded":"2024-09-17T05:25:05.321398+0000","last_fullsized":"2024-09-17T05:25:05.321398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:13.898988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324064+0000","last_change":"2024-09-17T05:25:04.163379+0000","last_active":"2024-09-17T05:25:05.324064+0000","last_peered":"2024-09-17T05:25:05.324064+0000","last_clean":"2024-09-17T05:25:05.324064+0000","last_became_active":"2024-09-17T05:25:04.163066+0000","last_became_peered":"2024-09-17T05:25:04.163066+0000","last_unstale":"2024-09-17T05:25:05.324064+0000","last_undegraded":"2024-09-17T05:25:05.324064+0000","last_fullsized":"2024-09-17T05:25:05.324064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:43.294053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153478+0000","last_change":"2024-09-17T05:25:03.154476+0000","last_active":"2024-09-17T05:25:05.153478+0000","last_peered":"2024-09-17T05:25:05.153478+0000","last_clean":"2024-09-17T05:25:05.153478+0000","last_became_active":"2024-09-17T05:25:03.153306+0000","last_became_peered":"2024-09-17T05:25:03.153306+0000","last_unstale":"2024-09-17T05:25:05.153478+0000","last_undegraded":"2024-09-17T05:25:05.153478+0000","last_fullsized":"2024-09-17T05:25:05.153478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:10.315224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150647+0000","last_change":"2024-09-17T05:25:04.145007+0000","last_active":"2024-09-17T05:25:05.150647+0000","last_peered":"2024-09-17T05:25:05.150647+0000","last_clean":"2024-09-17T05:25:05.150647+0000","last_became_active":"2024-09-17T05:25:04.144652+0000","last_became_peered":"2024-09-17T05:25:04.144652+0000","last_unstale":"2024-09-17T05:25:05.150647+0000","last_undegraded":"2024-09-17T05:25:05.150647+0000","last_fullsized":"2024-09-17T05:25:05.150647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:36.360867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320994+0000","last_change":"2024-09-17T05:25:03.152426+0000","last_active":"2024-09-17T05:25:05.320994+0000","last_peered":"2024-09-17T05:25:05.320994+0000","last_clean":"2024-09-17T05:25:05.320994+0000","last_became_active":"2024-09-17T05:25:03.152129+0000","last_became_peered":"2024-09-17T05:25:03.152129+0000","last_unstale":"2024-09-17T05:25:05.320994+0000","last_undegraded":"2024-09-17T05:25:05.320994+0000","last_fullsized":"2024-09-17T05:25:05.320994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321562+0000","last_change":"2024-09-17T05:25:04.170311+0000","last_active":"2024-09-17T05:25:05.321562+0000","last_peered":"2024-09-17T05:25:05.321562+0000","last_clean":"2024-09-17T05:25:05.321562+0000","last_became_active":"2024-09-17T05:25:04.169997+0000","last_became_peered":"2024-09-17T05:25:04.169997+0000","last_unstale":"2024-09-17T05:25:05.321562+0000","last_undegraded":"2024-09-17T05:25:05.321562+0000","last_fullsized":"2024-09-17T05:25:05.321562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:41.040086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153410+0000","last_change":"2024-09-17T05:25:03.156640+0000","last_active":"2024-09-17T05:25:05.153410+0000","last_peered":"2024-09-17T05:25:05.153410+0000","last_clean":"2024-09-17T05:25:05.153410+0000","last_became_active":"2024-09-17T05:25:03.156338+0000","last_became_peered":"2024-09-17T05:25:03.156338+0000","last_unstale":"2024-09-17T05:25:05.153410+0000","last_undegraded":"2024-09-17T05:25:05.153410+0000","last_fullsized":"2024-09-17T05:25:05.153410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:55.588248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324723+0000","last_change":"2024-09-17T05:25:04.171110+0000","last_active":"2024-09-17T05:25:05.324723+0000","last_peered":"2024-09-17T05:25:05.324723+0000","last_clean":"2024-09-17T05:25:05.324723+0000","last_became_active":"2024-09-17T05:25:04.162548+0000","last_became_peered":"2024-09-17T05:25:04.162548+0000","last_unstale":"2024-09-17T05:25:05.324723+0000","last_undegraded":"2024-09-17T05:25:05.324723+0000","last_fullsized":"2024-09-17T05:25:05.324723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:39.835973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151001+0000","last_change":"2024-09-17T05:25:03.136217+0000","last_active":"2024-09-17T05:25:05.151001+0000","last_peered":"2024-09-17T05:25:05.151001+0000","last_clean":"2024-09-17T05:25:05.151001+0000","last_became_active":"2024-09-17T05:25:03.135857+0000","last_became_peered":"2024-09-17T05:25:03.135857+0000","last_unstale":"2024-09-17T05:25:05.151001+0000","last_undegraded":"2024-09-17T05:25:05.151001+0000","last_fullsized":"2024-09-17T05:25:05.151001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:08.521888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150591+0000","last_change":"2024-09-17T05:25:04.158299+0000","last_active":"2024-09-17T05:25:05.150591+0000","last_peered":"2024-09-17T05:25:05.150591+0000","last_clean":"2024-09-17T05:25:05.150591+0000","last_became_active":"2024-09-17T05:25:04.157981+0000","last_became_peered":"2024-09-17T05:25:04.157981+0000","last_unstale":"2024-09-17T05:25:05.150591+0000","last_undegraded":"2024-09-17T05:25:05.150591+0000","last_fullsized":"2024-09-17T05:25:05.150591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:01.960601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320929+0000","last_change":"2024-09-17T05:25:03.147144+0000","last_active":"2024-09-17T05:25:05.320929+0000","last_peered":"2024-09-17T05:25:05.320929+0000","last_clean":"2024-09-17T05:25:05.320929+0000","last_became_active":"2024-09-17T05:25:03.146907+0000","last_became_peered":"2024-09-17T05:25:03.146907+0000","last_unstale":"2024-09-17T05:25:05.320929+0000","last_undegraded":"2024-09-17T05:25:05.320929+0000","last_fullsized":"2024-09-17T05:25:05.320929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:57.284513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152791+0000","last_change":"2024-09-17T05:25:04.152327+0000","last_active":"2024-09-17T05:25:05.152791+0000","last_peered":"2024-09-17T05:25:05.152791+0000","last_clean":"2024-09-17T05:25:05.152791+0000","last_became_active":"2024-09-17T05:25:04.151767+0000","last_became_peered":"2024-09-17T05:25:04.151767+0000","last_unstale":"2024-09-17T05:25:05.152791+0000","last_undegraded":"2024-09-17T05:25:05.152791+0000","last_fullsized":"2024-09-17T05:25:05.152791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323317+0000","last_change":"2024-09-17T05:25:03.151397+0000","last_active":"2024-09-17T05:25:05.323317+0000","last_peered":"2024-09-17T05:25:05.323317+0000","last_clean":"2024-09-17T05:25:05.323317+0000","last_became_active":"2024-09-17T05:25:03.151076+0000","last_became_peered":"2024-09-17T05:25:03.151076+0000","last_unstale":"2024-09-17T05:25:05.323317+0000","last_undegraded":"2024-09-17T05:25:05.323317+0000","last_fullsized":"2024-09-17T05:25:05.323317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:44.043946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150267+0000","last_change":"2024-09-17T05:25:04.146297+0000","last_active":"2024-09-17T05:25:05.150267+0000","last_peered":"2024-09-17T05:25:05.150267+0000","last_clean":"2024-09-17T05:25:05.150267+0000","last_became_active":"2024-09-17T05:25:04.145987+0000","last_became_peered":"2024-09-17T05:25:04.145987+0000","last_unstale":"2024-09-17T05:25:05.150267+0000","last_undegraded":"2024-09-17T05:25:05.150267+0000","last_fullsized":"2024-09-17T05:25:05.150267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:44.683027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321309+0000","last_change":"2024-09-17T05:25:03.159880+0000","last_active":"2024-09-17T05:25:05.321309+0000","last_peered":"2024-09-17T05:25:05.321309+0000","last_clean":"2024-09-17T05:25:05.321309+0000","last_became_active":"2024-09-17T05:25:03.159636+0000","last_became_peered":"2024-09-17T05:25:03.159636+0000","last_unstale":"2024-09-17T05:25:05.321309+0000","last_undegraded":"2024-09-17T05:25:05.321309+0000","last_fullsized":"2024-09-17T05:25:05.321309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:04.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321681+0000","last_change":"2024-09-17T05:25:04.159287+0000","last_active":"2024-09-17T05:25:05.321681+0000","last_peered":"2024-09-17T05:25:05.321681+0000","last_clean":"2024-09-17T05:25:05.321681+0000","last_became_active":"2024-09-17T05:25:04.158769+0000","last_became_peered":"2024-09-17T05:25:04.158769+0000","last_unstale":"2024-09-17T05:25:05.321681+0000","last_undegraded":"2024-09-17T05:25:05.321681+0000","last_fullsized":"2024-09-17T05:25:05.321681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:27.763662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325478+0000","last_change":"2024-09-17T05:25:03.161164+0000","last_active":"2024-09-17T05:25:05.325478+0000","last_peered":"2024-09-17T05:25:05.325478+0000","last_clean":"2024-09-17T05:25:05.325478+0000","last_became_active":"2024-09-17T05:25:03.160875+0000","last_became_peered":"2024-09-17T05:25:03.160875+0000","last_unstale":"2024-09-17T05:25:05.325478+0000","last_undegraded":"2024-09-17T05:25:05.325478+0000","last_fullsized":"2024-09-17T05:25:05.325478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321087+0000","last_change":"2024-09-17T05:25:04.159000+0000","last_active":"2024-09-17T05:25:05.321087+0000","last_peered":"2024-09-17T05:25:05.321087+0000","last_clean":"2024-09-17T05:25:05.321087+0000","last_became_active":"2024-09-17T05:25:04.158806+0000","last_became_peered":"2024-09-17T05:25:04.158806+0000","last_unstale":"2024-09-17T05:25:05.321087+0000","last_undegraded":"2024-09-17T05:25:05.321087+0000","last_fullsized":"2024-09-17T05:25:05.321087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:50.632241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323888+0000","last_change":"2024-09-17T05:25:03.160173+0000","last_active":"2024-09-17T05:25:05.323888+0000","last_peered":"2024-09-17T05:25:05.323888+0000","last_clean":"2024-09-17T05:25:05.323888+0000","last_became_active":"2024-09-17T05:25:03.159759+0000","last_became_peered":"2024-09-17T05:25:03.159759+0000","last_unstale":"2024-09-17T05:25:05.323888+0000","last_undegraded":"2024-09-17T05:25:05.323888+0000","last_fullsized":"2024-09-17T05:25:05.323888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:06.043431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321445+0000","last_change":"2024-09-17T05:25:04.156812+0000","last_active":"2024-09-17T05:25:05.321445+0000","last_peered":"2024-09-17T05:25:05.321445+0000","last_clean":"2024-09-17T05:25:05.321445+0000","last_became_active":"2024-09-17T05:25:04.147686+0000","last_became_peered":"2024-09-17T05:25:04.147686+0000","last_unstale":"2024-09-17T05:25:05.321445+0000","last_undegraded":"2024-09-17T05:25:05.321445+0000","last_fullsized":"2024-09-17T05:25:05.321445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:03.043749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325125+0000","last_change":"2024-09-17T05:25:03.151849+0000","last_active":"2024-09-17T05:25:05.325125+0000","last_peered":"2024-09-17T05:25:05.325125+0000","last_clean":"2024-09-17T05:25:05.325125+0000","last_became_active":"2024-09-17T05:25:03.151518+0000","last_became_peered":"2024-09-17T05:25:03.151518+0000","last_unstale":"2024-09-17T05:25:05.325125+0000","last_undegraded":"2024-09-17T05:25:05.325125+0000","last_fullsized":"2024-09-17T05:25:05.325125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150742+0000","last_change":"2024-09-17T05:25:04.150792+0000","last_active":"2024-09-17T05:25:05.150742+0000","last_peered":"2024-09-17T05:25:05.150742+0000","last_clean":"2024-09-17T05:25:05.150742+0000","last_became_active":"2024-09-17T05:25:04.150523+0000","last_became_peered":"2024-09-17T05:25:04.150523+0000","last_unstale":"2024-09-17T05:25:05.150742+0000","last_undegraded":"2024-09-17T05:25:05.150742+0000","last_fullsized":"2024-09-17T05:25:05.150742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:09.954167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152794+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:05.152794+0000","last_peered":"2024-09-17T05:25:05.152794+0000","last_clean":"2024-09-17T05:25:05.152794+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:05.152794+0000","last_undegraded":"2024-09-17T05:25:05.152794+0000","last_fullsized":"2024-09-17T05:25:05.152794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150663+0000","last_change":"2024-09-17T05:25:03.154924+0000","last_active":"2024-09-17T05:25:05.150663+0000","last_peered":"2024-09-17T05:25:05.150663+0000","last_clean":"2024-09-17T05:25:05.150663+0000","last_became_active":"2024-09-17T05:25:03.154274+0000","last_became_peered":"2024-09-17T05:25:03.154274+0000","last_unstale":"2024-09-17T05:25:05.150663+0000","last_undegraded":"2024-09-17T05:25:05.150663+0000","last_fullsized":"2024-09-17T05:25:05.150663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:38.825579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323572+0000","last_change":"2024-09-17T05:25:04.169405+0000","last_active":"2024-09-17T05:25:05.323572+0000","last_peered":"2024-09-17T05:25:05.323572+0000","last_clean":"2024-09-17T05:25:05.323572+0000","last_became_active":"2024-09-17T05:25:04.167552+0000","last_became_peered":"2024-09-17T05:25:04.167552+0000","last_unstale":"2024-09-17T05:25:05.323572+0000","last_undegraded":"2024-09-17T05:25:05.323572+0000","last_fullsized":"2024-09-17T05:25:05.323572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:30.468665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150438+0000","last_change":"2024-09-17T05:25:03.154006+0000","last_active":"2024-09-17T05:25:05.150438+0000","last_peered":"2024-09-17T05:25:05.150438+0000","last_clean":"2024-09-17T05:25:05.150438+0000","last_became_active":"2024-09-17T05:25:03.153578+0000","last_became_peered":"2024-09-17T05:25:03.153578+0000","last_unstale":"2024-09-17T05:25:05.150438+0000","last_undegraded":"2024-09-17T05:25:05.150438+0000","last_fullsized":"2024-09-17T05:25:05.150438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:13.898988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324301+0000","last_change":"2024-09-17T05:25:04.160915+0000","last_active":"2024-09-17T05:25:05.324301+0000","last_peered":"2024-09-17T05:25:05.324301+0000","last_clean":"2024-09-17T05:25:05.324301+0000","last_became_active":"2024-09-17T05:25:04.159519+0000","last_became_peered":"2024-09-17T05:25:04.159519+0000","last_unstale":"2024-09-17T05:25:05.324301+0000","last_undegraded":"2024-09-17T05:25:05.324301+0000","last_fullsized":"2024-09-17T05:25:05.324301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:08.226516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153072+0000","last_change":"2024-09-17T05:25:03.154866+0000","last_active":"2024-09-17T05:25:05.153072+0000","last_peered":"2024-09-17T05:25:05.153072+0000","last_clean":"2024-09-17T05:25:05.153072+0000","last_became_active":"2024-09-17T05:25:03.154116+0000","last_became_peered":"2024-09-17T05:25:03.154116+0000","last_unstale":"2024-09-17T05:25:05.153072+0000","last_undegraded":"2024-09-17T05:25:05.153072+0000","last_fullsized":"2024-09-17T05:25:05.153072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324275+0000","last_change":"2024-09-17T05:25:04.162175+0000","last_active":"2024-09-17T05:25:05.324275+0000","last_peered":"2024-09-17T05:25:05.324275+0000","last_clean":"2024-09-17T05:25:05.324275+0000","last_became_active":"2024-09-17T05:25:04.161820+0000","last_became_peered":"2024-09-17T05:25:04.161820+0000","last_unstale":"2024-09-17T05:25:05.324275+0000","last_undegraded":"2024-09-17T05:25:05.324275+0000","last_fullsized":"2024-09-17T05:25:05.324275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:08.628783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151000+0000","last_change":"2024-09-17T05:25:03.144930+0000","last_active":"2024-09-17T05:25:05.151000+0000","last_peered":"2024-09-17T05:25:05.151000+0000","last_clean":"2024-09-17T05:25:05.151000+0000","last_became_active":"2024-09-17T05:25:03.144665+0000","last_became_peered":"2024-09-17T05:25:03.144665+0000","last_unstale":"2024-09-17T05:25:05.151000+0000","last_undegraded":"2024-09-17T05:25:05.151000+0000","last_fullsized":"2024-09-17T05:25:05.151000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:31.401190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321109+0000","last_change":"2024-09-17T05:25:04.155323+0000","last_active":"2024-09-17T05:25:05.321109+0000","last_peered":"2024-09-17T05:25:05.321109+0000","last_clean":"2024-09-17T05:25:05.321109+0000","last_became_active":"2024-09-17T05:25:04.148816+0000","last_became_peered":"2024-09-17T05:25:04.148816+0000","last_unstale":"2024-09-17T05:25:05.321109+0000","last_undegraded":"2024-09-17T05:25:05.321109+0000","last_fullsized":"2024-09-17T05:25:05.321109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:34.744953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323728+0000","last_change":"2024-09-17T05:25:03.160065+0000","last_active":"2024-09-17T05:25:05.323728+0000","last_peered":"2024-09-17T05:25:05.323728+0000","last_clean":"2024-09-17T05:25:05.323728+0000","last_became_active":"2024-09-17T05:25:03.159684+0000","last_became_peered":"2024-09-17T05:25:03.159684+0000","last_unstale":"2024-09-17T05:25:05.323728+0000","last_undegraded":"2024-09-17T05:25:05.323728+0000","last_fullsized":"2024-09-17T05:25:05.323728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:04.387303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320608+0000","last_change":"2024-09-17T05:25:04.171447+0000","last_active":"2024-09-17T05:25:05.320608+0000","last_peered":"2024-09-17T05:25:05.320608+0000","last_clean":"2024-09-17T05:25:05.320608+0000","last_became_active":"2024-09-17T05:25:04.171049+0000","last_became_peered":"2024-09-17T05:25:04.171049+0000","last_unstale":"2024-09-17T05:25:05.320608+0000","last_undegraded":"2024-09-17T05:25:05.320608+0000","last_fullsized":"2024-09-17T05:25:05.320608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:44.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150897+0000","last_change":"2024-09-17T05:25:03.150695+0000","last_active":"2024-09-17T05:25:05.150897+0000","last_peered":"2024-09-17T05:25:05.150897+0000","last_clean":"2024-09-17T05:25:05.150897+0000","last_became_active":"2024-09-17T05:25:03.150452+0000","last_became_peered":"2024-09-17T05:25:03.150452+0000","last_unstale":"2024-09-17T05:25:05.150897+0000","last_undegraded":"2024-09-17T05:25:05.150897+0000","last_fullsized":"2024-09-17T05:25:05.150897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:04.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325109+0000","last_change":"2024-09-17T05:25:04.173584+0000","last_active":"2024-09-17T05:25:05.325109+0000","last_peered":"2024-09-17T05:25:05.325109+0000","last_clean":"2024-09-17T05:25:05.325109+0000","last_became_active":"2024-09-17T05:25:04.173144+0000","last_became_peered":"2024-09-17T05:25:04.173144+0000","last_unstale":"2024-09-17T05:25:05.325109+0000","last_undegraded":"2024-09-17T05:25:05.325109+0000","last_fullsized":"2024-09-17T05:25:05.325109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:22.262151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325559+0000","last_change":"2024-09-17T05:25:03.153186+0000","last_active":"2024-09-17T05:25:05.325559+0000","last_peered":"2024-09-17T05:25:05.325559+0000","last_clean":"2024-09-17T05:25:05.325559+0000","last_became_active":"2024-09-17T05:25:03.152579+0000","last_became_peered":"2024-09-17T05:25:03.152579+0000","last_unstale":"2024-09-17T05:25:05.325559+0000","last_undegraded":"2024-09-17T05:25:05.325559+0000","last_fullsized":"2024-09-17T05:25:05.325559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:40.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325278+0000","last_change":"2024-09-17T05:25:04.178286+0000","last_active":"2024-09-17T05:25:05.325278+0000","last_peered":"2024-09-17T05:25:05.325278+0000","last_clean":"2024-09-17T05:25:05.325278+0000","last_became_active":"2024-09-17T05:25:04.177979+0000","last_became_peered":"2024-09-17T05:25:04.177979+0000","last_unstale":"2024-09-17T05:25:05.325278+0000","last_undegraded":"2024-09-17T05:25:05.325278+0000","last_fullsized":"2024-09-17T05:25:05.325278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:24.204998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151091+0000","last_change":"2024-09-17T05:25:03.144098+0000","last_active":"2024-09-17T05:25:05.151091+0000","last_peered":"2024-09-17T05:25:05.151091+0000","last_clean":"2024-09-17T05:25:05.151091+0000","last_became_active":"2024-09-17T05:25:03.137827+0000","last_became_peered":"2024-09-17T05:25:03.137827+0000","last_unstale":"2024-09-17T05:25:05.151091+0000","last_undegraded":"2024-09-17T05:25:05.151091+0000","last_fullsized":"2024-09-17T05:25:05.151091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:36.454620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325378+0000","last_change":"2024-09-17T05:25:04.169501+0000","last_active":"2024-09-17T05:25:05.325378+0000","last_peered":"2024-09-17T05:25:05.325378+0000","last_clean":"2024-09-17T05:25:05.325378+0000","last_became_active":"2024-09-17T05:25:04.168926+0000","last_became_peered":"2024-09-17T05:25:04.168926+0000","last_unstale":"2024-09-17T05:25:05.325378+0000","last_undegraded":"2024-09-17T05:25:05.325378+0000","last_fullsized":"2024-09-17T05:25:05.325378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:51.977366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320442+0000","last_change":"2024-09-17T05:25:03.155817+0000","last_active":"2024-09-17T05:25:05.320442+0000","last_peered":"2024-09-17T05:25:05.320442+0000","last_clean":"2024-09-17T05:25:05.320442+0000","last_became_active":"2024-09-17T05:25:03.155300+0000","last_became_peered":"2024-09-17T05:25:03.155300+0000","last_unstale":"2024-09-17T05:25:05.320442+0000","last_undegraded":"2024-09-17T05:25:05.320442+0000","last_fullsized":"2024-09-17T05:25:05.320442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:17.915256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150325+0000","last_change":"2024-09-17T05:25:04.167116+0000","last_active":"2024-09-17T05:25:05.150325+0000","last_peered":"2024-09-17T05:25:05.150325+0000","last_clean":"2024-09-17T05:25:05.150325+0000","last_became_active":"2024-09-17T05:25:04.166832+0000","last_became_peered":"2024-09-17T05:25:04.166832+0000","last_unstale":"2024-09-17T05:25:05.150325+0000","last_undegraded":"2024-09-17T05:25:05.150325+0000","last_fullsized":"2024-09-17T05:25:05.150325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:41.040086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323594+0000","last_change":"2024-09-17T05:25:03.155466+0000","last_active":"2024-09-17T05:25:05.323594+0000","last_peered":"2024-09-17T05:25:05.323594+0000","last_clean":"2024-09-17T05:25:05.323594+0000","last_became_active":"2024-09-17T05:25:03.153103+0000","last_became_peered":"2024-09-17T05:25:03.153103+0000","last_unstale":"2024-09-17T05:25:05.323594+0000","last_undegraded":"2024-09-17T05:25:05.323594+0000","last_fullsized":"2024-09-17T05:25:05.323594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:29.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323335+0000","last_change":"2024-09-17T05:25:04.166048+0000","last_active":"2024-09-17T05:25:05.323335+0000","last_peered":"2024-09-17T05:25:05.323335+0000","last_clean":"2024-09-17T05:25:05.323335+0000","last_became_active":"2024-09-17T05:25:04.165647+0000","last_became_peered":"2024-09-17T05:25:04.165647+0000","last_unstale":"2024-09-17T05:25:05.323335+0000","last_undegraded":"2024-09-17T05:25:05.323335+0000","last_fullsized":"2024-09-17T05:25:05.323335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:22.190410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324888+0000","last_change":"2024-09-17T05:25:03.156207+0000","last_active":"2024-09-17T05:25:05.324888+0000","last_peered":"2024-09-17T05:25:05.324888+0000","last_clean":"2024-09-17T05:25:05.324888+0000","last_became_active":"2024-09-17T05:25:03.155889+0000","last_became_peered":"2024-09-17T05:25:03.155889+0000","last_unstale":"2024-09-17T05:25:05.324888+0000","last_undegraded":"2024-09-17T05:25:05.324888+0000","last_fullsized":"2024-09-17T05:25:05.324888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153253+0000","last_change":"2024-09-17T05:25:04.163791+0000","last_active":"2024-09-17T05:25:05.153253+0000","last_peered":"2024-09-17T05:25:05.153253+0000","last_clean":"2024-09-17T05:25:05.153253+0000","last_became_active":"2024-09-17T05:25:04.163047+0000","last_became_peered":"2024-09-17T05:25:04.163047+0000","last_unstale":"2024-09-17T05:25:05.153253+0000","last_undegraded":"2024-09-17T05:25:05.153253+0000","last_fullsized":"2024-09-17T05:25:05.153253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:05.977620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150564+0000","last_change":"2024-09-17T05:25:03.158082+0000","last_active":"2024-09-17T05:25:05.150564+0000","last_peered":"2024-09-17T05:25:05.150564+0000","last_clean":"2024-09-17T05:25:05.150564+0000","last_became_active":"2024-09-17T05:25:03.157792+0000","last_became_peered":"2024-09-17T05:25:03.157792+0000","last_unstale":"2024-09-17T05:25:05.150564+0000","last_undegraded":"2024-09-17T05:25:05.150564+0000","last_fullsized":"2024-09-17T05:25:05.150564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:07.216122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150416+0000","last_change":"2024-09-17T05:25:04.153904+0000","last_active":"2024-09-17T05:25:05.150416+0000","last_peered":"2024-09-17T05:25:05.150416+0000","last_clean":"2024-09-17T05:25:05.150416+0000","last_became_active":"2024-09-17T05:25:04.153582+0000","last_became_peered":"2024-09-17T05:25:04.153582+0000","last_unstale":"2024-09-17T05:25:05.150416+0000","last_undegraded":"2024-09-17T05:25:05.150416+0000","last_fullsized":"2024-09-17T05:25:05.150416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:49.536379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150685+0000","last_change":"2024-09-17T05:25:04.150254+0000","last_active":"2024-09-17T05:25:05.150685+0000","last_peered":"2024-09-17T05:25:05.150685+0000","last_clean":"2024-09-17T05:25:05.150685+0000","last_became_active":"2024-09-17T05:25:04.149481+0000","last_became_peered":"2024-09-17T05:25:04.149481+0000","last_unstale":"2024-09-17T05:25:05.150685+0000","last_undegraded":"2024-09-17T05:25:05.150685+0000","last_fullsized":"2024-09-17T05:25:05.150685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:15.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321580+0000","last_change":"2024-09-17T05:25:03.159779+0000","last_active":"2024-09-17T05:25:05.321580+0000","last_peered":"2024-09-17T05:25:05.321580+0000","last_clean":"2024-09-17T05:25:05.321580+0000","last_became_active":"2024-09-17T05:25:03.159430+0000","last_became_peered":"2024-09-17T05:25:03.159430+0000","last_unstale":"2024-09-17T05:25:05.321580+0000","last_undegraded":"2024-09-17T05:25:05.321580+0000","last_fullsized":"2024-09-17T05:25:05.321580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:33.326424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153233+0000","last_change":"2024-09-17T05:25:04.173233+0000","last_active":"2024-09-17T05:25:05.153233+0000","last_peered":"2024-09-17T05:25:05.153233+0000","last_clean":"2024-09-17T05:25:05.153233+0000","last_became_active":"2024-09-17T05:25:04.172918+0000","last_became_peered":"2024-09-17T05:25:04.172918+0000","last_unstale":"2024-09-17T05:25:05.153233+0000","last_undegraded":"2024-09-17T05:25:05.153233+0000","last_fullsized":"2024-09-17T05:25:05.153233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:06.019281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321565+0000","last_change":"2024-09-17T05:25:03.153878+0000","last_active":"2024-09-17T05:25:05.321565+0000","last_peered":"2024-09-17T05:25:05.321565+0000","last_clean":"2024-09-17T05:25:05.321565+0000","last_became_active":"2024-09-17T05:25:03.153594+0000","last_became_peered":"2024-09-17T05:25:03.153594+0000","last_unstale":"2024-09-17T05:25:05.321565+0000","last_undegraded":"2024-09-17T05:25:05.321565+0000","last_fullsized":"2024-09-17T05:25:05.321565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:16:31.831690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323879+0000","last_change":"2024-09-17T05:25:04.163341+0000","last_active":"2024-09-17T05:25:05.323879+0000","last_peered":"2024-09-17T05:25:05.323879+0000","last_clean":"2024-09-17T05:25:05.323879+0000","last_became_active":"2024-09-17T05:25:04.162727+0000","last_became_peered":"2024-09-17T05:25:04.162727+0000","last_unstale":"2024-09-17T05:25:05.323879+0000","last_undegraded":"2024-09-17T05:25:05.323879+0000","last_fullsized":"2024-09-17T05:25:05.323879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:04.451009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324619+0000","last_change":"2024-09-17T05:25:03.157287+0000","last_active":"2024-09-17T05:25:05.324619+0000","last_peered":"2024-09-17T05:25:05.324619+0000","last_clean":"2024-09-17T05:25:05.324619+0000","last_became_active":"2024-09-17T05:25:03.154403+0000","last_became_peered":"2024-09-17T05:25:03.154403+0000","last_unstale":"2024-09-17T05:25:05.324619+0000","last_undegraded":"2024-09-17T05:25:05.324619+0000","last_fullsized":"2024-09-17T05:25:05.324619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:21.699418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324558+0000","last_change":"2024-09-17T05:25:04.167303+0000","last_active":"2024-09-17T05:25:05.324558+0000","last_peered":"2024-09-17T05:25:05.324558+0000","last_clean":"2024-09-17T05:25:05.324558+0000","last_became_active":"2024-09-17T05:25:04.166953+0000","last_became_peered":"2024-09-17T05:25:04.166953+0000","last_unstale":"2024-09-17T05:25:05.324558+0000","last_undegraded":"2024-09-17T05:25:05.324558+0000","last_fullsized":"2024-09-17T05:25:05.324558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:02.920364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325412+0000","last_change":"2024-09-17T05:25:03.156536+0000","last_active":"2024-09-17T05:25:05.325412+0000","last_peered":"2024-09-17T05:25:05.325412+0000","last_clean":"2024-09-17T05:25:05.325412+0000","last_became_active":"2024-09-17T05:25:03.155913+0000","last_became_peered":"2024-09-17T05:25:03.155913+0000","last_unstale":"2024-09-17T05:25:05.325412+0000","last_undegraded":"2024-09-17T05:25:05.325412+0000","last_fullsized":"2024-09-17T05:25:05.325412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:08.594253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:17.848 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:18.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.022+0000 7fae44e00640 1 -- 172.21.15.122:0/595216404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae400cf300 msgr2=0x7fae4006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.022+0000 7fae44e00640 1 --2- 172.21.15.122:0/595216404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae400cf300 0x7fae4006ecb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fae400708e0 tx=0x7fae2c01c7d0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.022+0000 7fae44e00640 1 -- 172.21.15.122:0/595216404 shutdown_connections 2024-09-17T05:25:18.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.022+0000 7fae44e00640 1 --2- 172.21.15.122:0/595216404 >> v2:172.21.15.173:3300/0 conn(0x7fae4006e810 0x7fae4006f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.022+0000 7fae44e00640 1 --2- 172.21.15.122:0/595216404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae400cf300 0x7fae4006ecb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.022+0000 7fae44e00640 1 -- 172.21.15.122:0/595216404 >> 172.21.15.122:0/595216404 conn(0x7fae400b18f0 msgr2=0x7fae400b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.022+0000 7fae44e00640 1 -- 172.21.15.122:0/595216404 shutdown_connections 2024-09-17T05:25:18.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.022+0000 7fae44e00640 1 -- 172.21.15.122:0/595216404 wait complete. 2024-09-17T05:25:18.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 Processor -- start 2024-09-17T05:25:18.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 -- start start 2024-09-17T05:25:18.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4006e810 0x7fae401424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400cf300 0x7fae4009bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015ade0 0x7fae400c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae401686c0 con 0x7fae4015ade0 2024-09-17T05:25:18.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fae40168840 con 0x7fae400cf300 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fae401689c0 con 0x7fae4006e810 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015ade0 0x7fae400c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015ade0 0x7fae400c4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43200/0 (socket says 172.21.15.122:43200) 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 -- 172.21.15.122:0/2023278467 learned_addr learned my addr 172.21.15.122:0/2023278467 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3d400640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400cf300 0x7fae4009bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 -- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4006e810 msgr2=0x7fae401424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3de00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4006e810 0x7fae401424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4006e810 0x7fae401424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 -- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400cf300 msgr2=0x7fae4009bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400cf300 0x7fae4009bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 -- 172.21.15.122:0/2023278467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4009d340 con 0x7fae4015ade0 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3d400640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400cf300 0x7fae4009bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3de00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4006e810 0x7fae401424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae3e800640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015ade0 0x7fae400c4b60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fae3800eb20 tx=0x7fae3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae36000640 1 -- 172.21.15.122:0/2023278467 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38004b30 con 0x7fae4015ade0 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae36000640 1 -- 172.21.15.122:0/2023278467 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae38004cf0 con 0x7fae4015ade0 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4009d890 con 0x7fae4015ade0 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae36000640 1 -- 172.21.15.122:0/2023278467 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38005280 con 0x7fae4015ade0 2024-09-17T05:25:18.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.026+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4009b530 con 0x7fae4015ade0 2024-09-17T05:25:18.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.030+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae40069410 con 0x7fae4015ade0 2024-09-17T05:25:18.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.030+0000 7fae36000640 1 -- 172.21.15.122:0/2023278467 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fae38005420 con 0x7fae4015ade0 2024-09-17T05:25:18.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.030+0000 7fae36000640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.030+0000 7fae36000640 1 -- 172.21.15.122:0/2023278467 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fae38094ff0 con 0x7fae4015ade0 2024-09-17T05:25:18.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.030+0000 7fae3de00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.034+0000 7fae3de00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae30001230 tx=0x7fae30009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.034+0000 7fae36000640 1 -- 172.21.15.122:0/2023278467 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fae38060090 con 0x7fae4015ade0 2024-09-17T05:25:18.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.198+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae4006aba0 con 0x7fae20073200 2024-09-17T05:25:18.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.202+0000 7fae36000640 1 -- 172.21.15.122:0/2023278467 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7fae4006aba0 con 0x7fae20073200 2024-09-17T05:25:18.209 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:18.211 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 msgr2=0x7fae200756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae30001230 tx=0x7fae30009040 comp rx=0 tx=0).stop 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015ade0 msgr2=0x7fae400c4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015ade0 0x7fae400c4b60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fae3800eb20 tx=0x7fae3800c2d0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 shutdown_connections 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015ade0 0x7fae400c4b60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400cf300 0x7fae4009bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 --2- 172.21.15.122:0/2023278467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4006e810 0x7fae401424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 >> 172.21.15.122:0/2023278467 conn(0x7fae400b18f0 msgr2=0x7fae400b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 shutdown_connections 2024-09-17T05:25:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.206+0000 7fae44e00640 1 -- 172.21.15.122:0/2023278467 wait complete. 2024-09-17T05:25:18.230 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-17T05:25:16.444700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223364,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751356,"statfs":{"total":773094113280,"available":772865388544,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.294153"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320847+0000","last_change":"2024-09-17T05:25:04.154692+0000","last_active":"2024-09-17T05:25:05.320847+0000","last_peered":"2024-09-17T05:25:05.320847+0000","last_clean":"2024-09-17T05:25:05.320847+0000","last_became_active":"2024-09-17T05:25:04.148232+0000","last_became_peered":"2024-09-17T05:25:04.148232+0000","last_unstale":"2024-09-17T05:25:05.320847+0000","last_undegraded":"2024-09-17T05:25:05.320847+0000","last_fullsized":"2024-09-17T05:25:05.320847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:30.690541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324353+0000","last_change":"2024-09-17T05:25:04.173588+0000","last_active":"2024-09-17T05:25:05.324353+0000","last_peered":"2024-09-17T05:25:05.324353+0000","last_clean":"2024-09-17T05:25:05.324353+0000","last_became_active":"2024-09-17T05:25:04.173260+0000","last_became_peered":"2024-09-17T05:25:04.173260+0000","last_unstale":"2024-09-17T05:25:05.324353+0000","last_undegraded":"2024-09-17T05:25:05.324353+0000","last_fullsized":"2024-09-17T05:25:05.324353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:13.477698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324934+0000","last_change":"2024-09-17T05:25:04.174132+0000","last_active":"2024-09-17T05:25:05.324934+0000","last_peered":"2024-09-17T05:25:05.324934+0000","last_clean":"2024-09-17T05:25:05.324934+0000","last_became_active":"2024-09-17T05:25:04.173337+0000","last_became_peered":"2024-09-17T05:25:04.173337+0000","last_unstale":"2024-09-17T05:25:05.324934+0000","last_undegraded":"2024-09-17T05:25:05.324934+0000","last_fullsized":"2024-09-17T05:25:05.324934+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:50.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320851+0000","last_change":"2024-09-17T05:25:04.166451+0000","last_active":"2024-09-17T05:25:05.320851+0000","last_peered":"2024-09-17T05:25:05.320851+0000","last_clean":"2024-09-17T05:25:05.320851+0000","last_became_active":"2024-09-17T05:25:04.166193+0000","last_became_peered":"2024-09-17T05:25:04.166193+0000","last_unstale":"2024-09-17T05:25:05.320851+0000","last_undegraded":"2024-09-17T05:25:05.320851+0000","last_fullsized":"2024-09-17T05:25:05.320851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320896+0000","last_change":"2024-09-17T05:25:04.166447+0000","last_active":"2024-09-17T05:25:05.320896+0000","last_peered":"2024-09-17T05:25:05.320896+0000","last_clean":"2024-09-17T05:25:05.320896+0000","last_became_active":"2024-09-17T05:25:04.166240+0000","last_became_peered":"2024-09-17T05:25:04.166240+0000","last_unstale":"2024-09-17T05:25:05.320896+0000","last_undegraded":"2024-09-17T05:25:05.320896+0000","last_fullsized":"2024-09-17T05:25:05.320896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:20.054457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320554+0000","last_change":"2024-09-17T05:25:04.166670+0000","last_active":"2024-09-17T05:25:05.320554+0000","last_peered":"2024-09-17T05:25:05.320554+0000","last_clean":"2024-09-17T05:25:05.320554+0000","last_became_active":"2024-09-17T05:25:04.166212+0000","last_became_peered":"2024-09-17T05:25:04.166212+0000","last_unstale":"2024-09-17T05:25:05.320554+0000","last_undegraded":"2024-09-17T05:25:05.320554+0000","last_fullsized":"2024-09-17T05:25:05.320554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320762+0000","last_change":"2024-09-17T05:25:04.171397+0000","last_active":"2024-09-17T05:25:05.320762+0000","last_peered":"2024-09-17T05:25:05.320762+0000","last_clean":"2024-09-17T05:25:05.320762+0000","last_became_active":"2024-09-17T05:25:04.171157+0000","last_became_peered":"2024-09-17T05:25:04.171157+0000","last_unstale":"2024-09-17T05:25:05.320762+0000","last_undegraded":"2024-09-17T05:25:05.320762+0000","last_fullsized":"2024-09-17T05:25:05.320762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:49.992366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324125+0000","last_change":"2024-09-17T05:25:04.166345+0000","last_active":"2024-09-17T05:25:05.324125+0000","last_peered":"2024-09-17T05:25:05.324125+0000","last_clean":"2024-09-17T05:25:05.324125+0000","last_became_active":"2024-09-17T05:25:04.165795+0000","last_became_peered":"2024-09-17T05:25:04.165795+0000","last_unstale":"2024-09-17T05:25:05.324125+0000","last_undegraded":"2024-09-17T05:25:05.324125+0000","last_fullsized":"2024-09-17T05:25:05.324125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:33.426170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153496+0000","last_change":"2024-09-17T05:25:04.155768+0000","last_active":"2024-09-17T05:25:05.153496+0000","last_peered":"2024-09-17T05:25:05.153496+0000","last_clean":"2024-09-17T05:25:05.153496+0000","last_became_active":"2024-09-17T05:25:04.155277+0000","last_became_peered":"2024-09-17T05:25:04.155277+0000","last_unstale":"2024-09-17T05:25:05.153496+0000","last_undegraded":"2024-09-17T05:25:05.153496+0000","last_fullsized":"2024-09-17T05:25:05.153496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:44.070039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321217+0000","last_change":"2024-09-17T05:25:04.144665+0000","last_active":"2024-09-17T05:25:05.321217+0000","last_peered":"2024-09-17T05:25:05.321217+0000","last_clean":"2024-09-17T05:25:05.321217+0000","last_became_active":"2024-09-17T05:25:04.144382+0000","last_became_peered":"2024-09-17T05:25:04.144382+0000","last_unstale":"2024-09-17T05:25:05.321217+0000","last_undegraded":"2024-09-17T05:25:05.321217+0000","last_fullsized":"2024-09-17T05:25:05.321217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:09.785934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321369+0000","last_change":"2024-09-17T05:25:04.153782+0000","last_active":"2024-09-17T05:25:05.321369+0000","last_peered":"2024-09-17T05:25:05.321369+0000","last_clean":"2024-09-17T05:25:05.321369+0000","last_became_active":"2024-09-17T05:25:04.148194+0000","last_became_peered":"2024-09-17T05:25:04.148194+0000","last_unstale":"2024-09-17T05:25:05.321369+0000","last_undegraded":"2024-09-17T05:25:05.321369+0000","last_fullsized":"2024-09-17T05:25:05.321369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:44.683027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321040+0000","last_change":"2024-09-17T05:25:04.158630+0000","last_active":"2024-09-17T05:25:05.321040+0000","last_peered":"2024-09-17T05:25:05.321040+0000","last_clean":"2024-09-17T05:25:05.321040+0000","last_became_active":"2024-09-17T05:25:04.158382+0000","last_became_peered":"2024-09-17T05:25:04.158382+0000","last_unstale":"2024-09-17T05:25:05.321040+0000","last_undegraded":"2024-09-17T05:25:05.321040+0000","last_fullsized":"2024-09-17T05:25:05.321040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:27.039745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324868+0000","last_change":"2024-09-17T05:25:04.171927+0000","last_active":"2024-09-17T05:25:05.324868+0000","last_peered":"2024-09-17T05:25:05.324868+0000","last_clean":"2024-09-17T05:25:05.324868+0000","last_became_active":"2024-09-17T05:25:04.171634+0000","last_became_peered":"2024-09-17T05:25:04.171634+0000","last_unstale":"2024-09-17T05:25:05.324868+0000","last_undegraded":"2024-09-17T05:25:05.324868+0000","last_fullsized":"2024-09-17T05:25:05.324868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:09.532282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323783+0000","last_change":"2024-09-17T05:25:04.171235+0000","last_active":"2024-09-17T05:25:05.323783+0000","last_peered":"2024-09-17T05:25:05.323783+0000","last_clean":"2024-09-17T05:25:05.323783+0000","last_became_active":"2024-09-17T05:25:04.170717+0000","last_became_peered":"2024-09-17T05:25:04.170717+0000","last_unstale":"2024-09-17T05:25:05.323783+0000","last_undegraded":"2024-09-17T05:25:05.323783+0000","last_fullsized":"2024-09-17T05:25:05.323783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:32.411584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324904+0000","last_change":"2024-09-17T05:25:04.163764+0000","last_active":"2024-09-17T05:25:05.324904+0000","last_peered":"2024-09-17T05:25:05.324904+0000","last_clean":"2024-09-17T05:25:05.324904+0000","last_became_active":"2024-09-17T05:25:04.163197+0000","last_became_peered":"2024-09-17T05:25:04.163197+0000","last_unstale":"2024-09-17T05:25:05.324904+0000","last_undegraded":"2024-09-17T05:25:05.324904+0000","last_fullsized":"2024-09-17T05:25:05.324904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:01:59.441556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150239+0000","last_change":"2024-09-17T05:25:04.147588+0000","last_active":"2024-09-17T05:25:05.150239+0000","last_peered":"2024-09-17T05:25:05.150239+0000","last_clean":"2024-09-17T05:25:05.150239+0000","last_became_active":"2024-09-17T05:25:04.147316+0000","last_became_peered":"2024-09-17T05:25:04.147316+0000","last_unstale":"2024-09-17T05:25:05.150239+0000","last_undegraded":"2024-09-17T05:25:05.150239+0000","last_fullsized":"2024-09-17T05:25:05.150239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:44.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151006+0000","last_change":"2024-09-17T05:25:04.149293+0000","last_active":"2024-09-17T05:25:05.151006+0000","last_peered":"2024-09-17T05:25:05.151006+0000","last_clean":"2024-09-17T05:25:05.151006+0000","last_became_active":"2024-09-17T05:25:04.149079+0000","last_became_peered":"2024-09-17T05:25:04.149079+0000","last_unstale":"2024-09-17T05:25:05.151006+0000","last_undegraded":"2024-09-17T05:25:05.151006+0000","last_fullsized":"2024-09-17T05:25:05.151006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:13.903114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153437+0000","last_change":"2024-09-17T05:25:04.152808+0000","last_active":"2024-09-17T05:25:05.153437+0000","last_peered":"2024-09-17T05:25:05.153437+0000","last_clean":"2024-09-17T05:25:05.153437+0000","last_became_active":"2024-09-17T05:25:04.152501+0000","last_became_peered":"2024-09-17T05:25:04.152501+0000","last_unstale":"2024-09-17T05:25:05.153437+0000","last_undegraded":"2024-09-17T05:25:05.153437+0000","last_fullsized":"2024-09-17T05:25:05.153437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150922+0000","last_change":"2024-09-17T05:25:04.167124+0000","last_active":"2024-09-17T05:25:05.150922+0000","last_peered":"2024-09-17T05:25:05.150922+0000","last_clean":"2024-09-17T05:25:05.150922+0000","last_became_active":"2024-09-17T05:25:04.166853+0000","last_became_peered":"2024-09-17T05:25:04.166853+0000","last_unstale":"2024-09-17T05:25:05.150922+0000","last_undegraded":"2024-09-17T05:25:05.150922+0000","last_fullsized":"2024-09-17T05:25:05.150922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:05.283255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321253+0000","last_change":"2024-09-17T05:25:04.160482+0000","last_active":"2024-09-17T05:25:05.321253+0000","last_peered":"2024-09-17T05:25:05.321253+0000","last_clean":"2024-09-17T05:25:05.321253+0000","last_became_active":"2024-09-17T05:25:04.156878+0000","last_became_peered":"2024-09-17T05:25:04.156878+0000","last_unstale":"2024-09-17T05:25:05.321253+0000","last_undegraded":"2024-09-17T05:25:05.321253+0000","last_fullsized":"2024-09-17T05:25:05.321253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:44.569300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323874+0000","last_change":"2024-09-17T05:25:04.169764+0000","last_active":"2024-09-17T05:25:05.323874+0000","last_peered":"2024-09-17T05:25:05.323874+0000","last_clean":"2024-09-17T05:25:05.323874+0000","last_became_active":"2024-09-17T05:25:04.167730+0000","last_became_peered":"2024-09-17T05:25:04.167730+0000","last_unstale":"2024-09-17T05:25:05.323874+0000","last_undegraded":"2024-09-17T05:25:05.323874+0000","last_fullsized":"2024-09-17T05:25:05.323874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:13.526132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152880+0000","last_change":"2024-09-17T05:25:04.163296+0000","last_active":"2024-09-17T05:25:05.152880+0000","last_peered":"2024-09-17T05:25:05.152880+0000","last_clean":"2024-09-17T05:25:05.152880+0000","last_became_active":"2024-09-17T05:25:04.162996+0000","last_became_peered":"2024-09-17T05:25:04.162996+0000","last_unstale":"2024-09-17T05:25:05.152880+0000","last_undegraded":"2024-09-17T05:25:05.152880+0000","last_fullsized":"2024-09-17T05:25:05.152880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:44.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325148+0000","last_change":"2024-09-17T05:25:04.158936+0000","last_active":"2024-09-17T05:25:05.325148+0000","last_peered":"2024-09-17T05:25:05.325148+0000","last_clean":"2024-09-17T05:25:05.325148+0000","last_became_active":"2024-09-17T05:25:04.158697+0000","last_became_peered":"2024-09-17T05:25:04.158697+0000","last_unstale":"2024-09-17T05:25:05.325148+0000","last_undegraded":"2024-09-17T05:25:05.325148+0000","last_fullsized":"2024-09-17T05:25:05.325148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:45:18.794538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150244+0000","last_change":"2024-09-17T05:25:04.160086+0000","last_active":"2024-09-17T05:25:05.150244+0000","last_peered":"2024-09-17T05:25:05.150244+0000","last_clean":"2024-09-17T05:25:05.150244+0000","last_became_active":"2024-09-17T05:25:04.159826+0000","last_became_peered":"2024-09-17T05:25:04.159826+0000","last_unstale":"2024-09-17T05:25:05.150244+0000","last_undegraded":"2024-09-17T05:25:05.150244+0000","last_fullsized":"2024-09-17T05:25:05.150244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:46.502824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324281+0000","last_change":"2024-09-17T05:25:04.165631+0000","last_active":"2024-09-17T05:25:05.324281+0000","last_peered":"2024-09-17T05:25:05.324281+0000","last_clean":"2024-09-17T05:25:05.324281+0000","last_became_active":"2024-09-17T05:25:04.163086+0000","last_became_peered":"2024-09-17T05:25:04.163086+0000","last_unstale":"2024-09-17T05:25:05.324281+0000","last_undegraded":"2024-09-17T05:25:05.324281+0000","last_fullsized":"2024-09-17T05:25:05.324281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:59.610017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321281+0000","last_change":"2024-09-17T05:25:04.152176+0000","last_active":"2024-09-17T05:25:05.321281+0000","last_peered":"2024-09-17T05:25:05.321281+0000","last_clean":"2024-09-17T05:25:05.321281+0000","last_became_active":"2024-09-17T05:25:04.151035+0000","last_became_peered":"2024-09-17T05:25:04.151035+0000","last_unstale":"2024-09-17T05:25:05.321281+0000","last_undegraded":"2024-09-17T05:25:05.321281+0000","last_fullsized":"2024-09-17T05:25:05.321281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:06.019281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325437+0000","last_change":"2024-09-17T05:25:04.172127+0000","last_active":"2024-09-17T05:25:05.325437+0000","last_peered":"2024-09-17T05:25:05.325437+0000","last_clean":"2024-09-17T05:25:05.325437+0000","last_became_active":"2024-09-17T05:25:04.159538+0000","last_became_peered":"2024-09-17T05:25:04.159538+0000","last_unstale":"2024-09-17T05:25:05.325437+0000","last_undegraded":"2024-09-17T05:25:05.325437+0000","last_fullsized":"2024-09-17T05:25:05.325437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:04:46.839572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324955+0000","last_change":"2024-09-17T05:25:04.176025+0000","last_active":"2024-09-17T05:25:05.324955+0000","last_peered":"2024-09-17T05:25:05.324955+0000","last_clean":"2024-09-17T05:25:05.324955+0000","last_became_active":"2024-09-17T05:25:04.174826+0000","last_became_peered":"2024-09-17T05:25:04.174826+0000","last_unstale":"2024-09-17T05:25:05.324955+0000","last_undegraded":"2024-09-17T05:25:05.324955+0000","last_fullsized":"2024-09-17T05:25:05.324955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:37.465014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325254+0000","last_change":"2024-09-17T05:25:04.162109+0000","last_active":"2024-09-17T05:25:05.325254+0000","last_peered":"2024-09-17T05:25:05.325254+0000","last_clean":"2024-09-17T05:25:05.325254+0000","last_became_active":"2024-09-17T05:25:04.161807+0000","last_became_peered":"2024-09-17T05:25:04.161807+0000","last_unstale":"2024-09-17T05:25:05.325254+0000","last_undegraded":"2024-09-17T05:25:05.325254+0000","last_fullsized":"2024-09-17T05:25:05.325254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:23.271600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150335+0000","last_change":"2024-09-17T05:25:04.160575+0000","last_active":"2024-09-17T05:25:05.150335+0000","last_peered":"2024-09-17T05:25:05.150335+0000","last_clean":"2024-09-17T05:25:05.150335+0000","last_became_active":"2024-09-17T05:25:04.160104+0000","last_became_peered":"2024-09-17T05:25:04.160104+0000","last_unstale":"2024-09-17T05:25:05.150335+0000","last_undegraded":"2024-09-17T05:25:05.150335+0000","last_fullsized":"2024-09-17T05:25:05.150335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:22.186817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153243+0000","last_change":"2024-09-17T05:25:04.156777+0000","last_active":"2024-09-17T05:25:05.153243+0000","last_peered":"2024-09-17T05:25:05.153243+0000","last_clean":"2024-09-17T05:25:05.153243+0000","last_became_active":"2024-09-17T05:25:04.156013+0000","last_became_peered":"2024-09-17T05:25:04.156013+0000","last_unstale":"2024-09-17T05:25:05.153243+0000","last_undegraded":"2024-09-17T05:25:05.153243+0000","last_fullsized":"2024-09-17T05:25:05.153243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320365+0000","last_change":"2024-09-17T05:25:04.170636+0000","last_active":"2024-09-17T05:25:05.320365+0000","last_peered":"2024-09-17T05:25:05.320365+0000","last_clean":"2024-09-17T05:25:05.320365+0000","last_became_active":"2024-09-17T05:25:04.170325+0000","last_became_peered":"2024-09-17T05:25:04.170325+0000","last_unstale":"2024-09-17T05:25:05.320365+0000","last_undegraded":"2024-09-17T05:25:05.320365+0000","last_fullsized":"2024-09-17T05:25:05.320365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:09.604647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152897+0000","last_change":"2024-09-17T05:25:04.162069+0000","last_active":"2024-09-17T05:25:05.152897+0000","last_peered":"2024-09-17T05:25:05.152897+0000","last_clean":"2024-09-17T05:25:05.152897+0000","last_became_active":"2024-09-17T05:25:04.161443+0000","last_became_peered":"2024-09-17T05:25:04.161443+0000","last_unstale":"2024-09-17T05:25:05.152897+0000","last_undegraded":"2024-09-17T05:25:05.152897+0000","last_fullsized":"2024-09-17T05:25:05.152897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:20.202246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325321+0000","last_change":"2024-09-17T05:25:03.160754+0000","last_active":"2024-09-17T05:25:05.325321+0000","last_peered":"2024-09-17T05:25:05.325321+0000","last_clean":"2024-09-17T05:25:05.325321+0000","last_became_active":"2024-09-17T05:25:03.160433+0000","last_became_peered":"2024-09-17T05:25:03.160433+0000","last_unstale":"2024-09-17T05:25:05.325321+0000","last_undegraded":"2024-09-17T05:25:05.325321+0000","last_fullsized":"2024-09-17T05:25:05.325321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:59.347982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324073+0000","last_change":"2024-09-17T05:25:04.174431+0000","last_active":"2024-09-17T05:25:05.324073+0000","last_peered":"2024-09-17T05:25:05.324073+0000","last_clean":"2024-09-17T05:25:05.324073+0000","last_became_active":"2024-09-17T05:25:04.174139+0000","last_became_peered":"2024-09-17T05:25:04.174139+0000","last_unstale":"2024-09-17T05:25:05.324073+0000","last_undegraded":"2024-09-17T05:25:05.324073+0000","last_fullsized":"2024-09-17T05:25:05.324073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:40.319110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150517+0000","last_change":"2024-09-17T05:25:03.138686+0000","last_active":"2024-09-17T05:25:05.150517+0000","last_peered":"2024-09-17T05:25:05.150517+0000","last_clean":"2024-09-17T05:25:05.150517+0000","last_became_active":"2024-09-17T05:25:03.138321+0000","last_became_peered":"2024-09-17T05:25:03.138321+0000","last_unstale":"2024-09-17T05:25:05.150517+0000","last_undegraded":"2024-09-17T05:25:05.150517+0000","last_fullsized":"2024-09-17T05:25:05.150517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:03.903924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323451+0000","last_change":"2024-09-17T05:25:03.161161+0000","last_active":"2024-09-17T05:25:05.323451+0000","last_peered":"2024-09-17T05:25:05.323451+0000","last_clean":"2024-09-17T05:25:05.323451+0000","last_became_active":"2024-09-17T05:25:03.160876+0000","last_became_peered":"2024-09-17T05:25:03.160876+0000","last_unstale":"2024-09-17T05:25:05.323451+0000","last_undegraded":"2024-09-17T05:25:05.323451+0000","last_fullsized":"2024-09-17T05:25:05.323451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:04.710792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324408+0000","last_change":"2024-09-17T05:25:04.173583+0000","last_active":"2024-09-17T05:25:05.324408+0000","last_peered":"2024-09-17T05:25:05.324408+0000","last_clean":"2024-09-17T05:25:05.324408+0000","last_became_active":"2024-09-17T05:25:04.173276+0000","last_became_peered":"2024-09-17T05:25:04.173276+0000","last_unstale":"2024-09-17T05:25:05.324408+0000","last_undegraded":"2024-09-17T05:25:05.324408+0000","last_fullsized":"2024-09-17T05:25:05.324408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:38.290022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150430+0000","last_change":"2024-09-17T05:25:03.156886+0000","last_active":"2024-09-17T05:25:05.150430+0000","last_peered":"2024-09-17T05:25:05.150430+0000","last_clean":"2024-09-17T05:25:05.150430+0000","last_became_active":"2024-09-17T05:25:03.156595+0000","last_became_peered":"2024-09-17T05:25:03.156595+0000","last_unstale":"2024-09-17T05:25:05.150430+0000","last_undegraded":"2024-09-17T05:25:05.150430+0000","last_fullsized":"2024-09-17T05:25:05.150430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:57.284513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323745+0000","last_change":"2024-09-17T05:25:04.167294+0000","last_active":"2024-09-17T05:25:05.323745+0000","last_peered":"2024-09-17T05:25:05.323745+0000","last_clean":"2024-09-17T05:25:05.323745+0000","last_became_active":"2024-09-17T05:25:04.165115+0000","last_became_peered":"2024-09-17T05:25:04.165115+0000","last_unstale":"2024-09-17T05:25:05.323745+0000","last_undegraded":"2024-09-17T05:25:05.323745+0000","last_fullsized":"2024-09-17T05:25:05.323745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:53.176691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153460+0000","last_change":"2024-09-17T05:25:03.160352+0000","last_active":"2024-09-17T05:25:05.153460+0000","last_peered":"2024-09-17T05:25:05.153460+0000","last_clean":"2024-09-17T05:25:05.153460+0000","last_became_active":"2024-09-17T05:25:03.160030+0000","last_became_peered":"2024-09-17T05:25:03.160030+0000","last_unstale":"2024-09-17T05:25:05.153460+0000","last_undegraded":"2024-09-17T05:25:05.153460+0000","last_fullsized":"2024-09-17T05:25:05.153460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321483+0000","last_change":"2024-09-17T05:25:04.146118+0000","last_active":"2024-09-17T05:25:05.321483+0000","last_peered":"2024-09-17T05:25:05.321483+0000","last_clean":"2024-09-17T05:25:05.321483+0000","last_became_active":"2024-09-17T05:25:04.145700+0000","last_became_peered":"2024-09-17T05:25:04.145700+0000","last_unstale":"2024-09-17T05:25:05.321483+0000","last_undegraded":"2024-09-17T05:25:05.321483+0000","last_fullsized":"2024-09-17T05:25:05.321483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:44.070039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321398+0000","last_change":"2024-09-17T05:25:03.155547+0000","last_active":"2024-09-17T05:25:05.321398+0000","last_peered":"2024-09-17T05:25:05.321398+0000","last_clean":"2024-09-17T05:25:05.321398+0000","last_became_active":"2024-09-17T05:25:03.155270+0000","last_became_peered":"2024-09-17T05:25:03.155270+0000","last_unstale":"2024-09-17T05:25:05.321398+0000","last_undegraded":"2024-09-17T05:25:05.321398+0000","last_fullsized":"2024-09-17T05:25:05.321398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:13.898988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324064+0000","last_change":"2024-09-17T05:25:04.163379+0000","last_active":"2024-09-17T05:25:05.324064+0000","last_peered":"2024-09-17T05:25:05.324064+0000","last_clean":"2024-09-17T05:25:05.324064+0000","last_became_active":"2024-09-17T05:25:04.163066+0000","last_became_peered":"2024-09-17T05:25:04.163066+0000","last_unstale":"2024-09-17T05:25:05.324064+0000","last_undegraded":"2024-09-17T05:25:05.324064+0000","last_fullsized":"2024-09-17T05:25:05.324064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:43.294053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153478+0000","last_change":"2024-09-17T05:25:03.154476+0000","last_active":"2024-09-17T05:25:05.153478+0000","last_peered":"2024-09-17T05:25:05.153478+0000","last_clean":"2024-09-17T05:25:05.153478+0000","last_became_active":"2024-09-17T05:25:03.153306+0000","last_became_peered":"2024-09-17T05:25:03.153306+0000","last_unstale":"2024-09-17T05:25:05.153478+0000","last_undegraded":"2024-09-17T05:25:05.153478+0000","last_fullsized":"2024-09-17T05:25:05.153478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:10.315224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150647+0000","last_change":"2024-09-17T05:25:04.145007+0000","last_active":"2024-09-17T05:25:05.150647+0000","last_peered":"2024-09-17T05:25:05.150647+0000","last_clean":"2024-09-17T05:25:05.150647+0000","last_became_active":"2024-09-17T05:25:04.144652+0000","last_became_peered":"2024-09-17T05:25:04.144652+0000","last_unstale":"2024-09-17T05:25:05.150647+0000","last_undegraded":"2024-09-17T05:25:05.150647+0000","last_fullsized":"2024-09-17T05:25:05.150647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:36.360867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320994+0000","last_change":"2024-09-17T05:25:03.152426+0000","last_active":"2024-09-17T05:25:05.320994+0000","last_peered":"2024-09-17T05:25:05.320994+0000","last_clean":"2024-09-17T05:25:05.320994+0000","last_became_active":"2024-09-17T05:25:03.152129+0000","last_became_peered":"2024-09-17T05:25:03.152129+0000","last_unstale":"2024-09-17T05:25:05.320994+0000","last_undegraded":"2024-09-17T05:25:05.320994+0000","last_fullsized":"2024-09-17T05:25:05.320994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:43.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321562+0000","last_change":"2024-09-17T05:25:04.170311+0000","last_active":"2024-09-17T05:25:05.321562+0000","last_peered":"2024-09-17T05:25:05.321562+0000","last_clean":"2024-09-17T05:25:05.321562+0000","last_became_active":"2024-09-17T05:25:04.169997+0000","last_became_peered":"2024-09-17T05:25:04.169997+0000","last_unstale":"2024-09-17T05:25:05.321562+0000","last_undegraded":"2024-09-17T05:25:05.321562+0000","last_fullsized":"2024-09-17T05:25:05.321562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:41.040086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153410+0000","last_change":"2024-09-17T05:25:03.156640+0000","last_active":"2024-09-17T05:25:05.153410+0000","last_peered":"2024-09-17T05:25:05.153410+0000","last_clean":"2024-09-17T05:25:05.153410+0000","last_became_active":"2024-09-17T05:25:03.156338+0000","last_became_peered":"2024-09-17T05:25:03.156338+0000","last_unstale":"2024-09-17T05:25:05.153410+0000","last_undegraded":"2024-09-17T05:25:05.153410+0000","last_fullsized":"2024-09-17T05:25:05.153410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:55.588248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324723+0000","last_change":"2024-09-17T05:25:04.171110+0000","last_active":"2024-09-17T05:25:05.324723+0000","last_peered":"2024-09-17T05:25:05.324723+0000","last_clean":"2024-09-17T05:25:05.324723+0000","last_became_active":"2024-09-17T05:25:04.162548+0000","last_became_peered":"2024-09-17T05:25:04.162548+0000","last_unstale":"2024-09-17T05:25:05.324723+0000","last_undegraded":"2024-09-17T05:25:05.324723+0000","last_fullsized":"2024-09-17T05:25:05.324723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:39.835973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151001+0000","last_change":"2024-09-17T05:25:03.136217+0000","last_active":"2024-09-17T05:25:05.151001+0000","last_peered":"2024-09-17T05:25:05.151001+0000","last_clean":"2024-09-17T05:25:05.151001+0000","last_became_active":"2024-09-17T05:25:03.135857+0000","last_became_peered":"2024-09-17T05:25:03.135857+0000","last_unstale":"2024-09-17T05:25:05.151001+0000","last_undegraded":"2024-09-17T05:25:05.151001+0000","last_fullsized":"2024-09-17T05:25:05.151001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:08.521888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150591+0000","last_change":"2024-09-17T05:25:04.158299+0000","last_active":"2024-09-17T05:25:05.150591+0000","last_peered":"2024-09-17T05:25:05.150591+0000","last_clean":"2024-09-17T05:25:05.150591+0000","last_became_active":"2024-09-17T05:25:04.157981+0000","last_became_peered":"2024-09-17T05:25:04.157981+0000","last_unstale":"2024-09-17T05:25:05.150591+0000","last_undegraded":"2024-09-17T05:25:05.150591+0000","last_fullsized":"2024-09-17T05:25:05.150591+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:01.960601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320929+0000","last_change":"2024-09-17T05:25:03.147144+0000","last_active":"2024-09-17T05:25:05.320929+0000","last_peered":"2024-09-17T05:25:05.320929+0000","last_clean":"2024-09-17T05:25:05.320929+0000","last_became_active":"2024-09-17T05:25:03.146907+0000","last_became_peered":"2024-09-17T05:25:03.146907+0000","last_unstale":"2024-09-17T05:25:05.320929+0000","last_undegraded":"2024-09-17T05:25:05.320929+0000","last_fullsized":"2024-09-17T05:25:05.320929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:57.284513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152791+0000","last_change":"2024-09-17T05:25:04.152327+0000","last_active":"2024-09-17T05:25:05.152791+0000","last_peered":"2024-09-17T05:25:05.152791+0000","last_clean":"2024-09-17T05:25:05.152791+0000","last_became_active":"2024-09-17T05:25:04.151767+0000","last_became_peered":"2024-09-17T05:25:04.151767+0000","last_unstale":"2024-09-17T05:25:05.152791+0000","last_undegraded":"2024-09-17T05:25:05.152791+0000","last_fullsized":"2024-09-17T05:25:05.152791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:31.196174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323317+0000","last_change":"2024-09-17T05:25:03.151397+0000","last_active":"2024-09-17T05:25:05.323317+0000","last_peered":"2024-09-17T05:25:05.323317+0000","last_clean":"2024-09-17T05:25:05.323317+0000","last_became_active":"2024-09-17T05:25:03.151076+0000","last_became_peered":"2024-09-17T05:25:03.151076+0000","last_unstale":"2024-09-17T05:25:05.323317+0000","last_undegraded":"2024-09-17T05:25:05.323317+0000","last_fullsized":"2024-09-17T05:25:05.323317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:44.043946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150267+0000","last_change":"2024-09-17T05:25:04.146297+0000","last_active":"2024-09-17T05:25:05.150267+0000","last_peered":"2024-09-17T05:25:05.150267+0000","last_clean":"2024-09-17T05:25:05.150267+0000","last_became_active":"2024-09-17T05:25:04.145987+0000","last_became_peered":"2024-09-17T05:25:04.145987+0000","last_unstale":"2024-09-17T05:25:05.150267+0000","last_undegraded":"2024-09-17T05:25:05.150267+0000","last_fullsized":"2024-09-17T05:25:05.150267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:44.683027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321309+0000","last_change":"2024-09-17T05:25:03.159880+0000","last_active":"2024-09-17T05:25:05.321309+0000","last_peered":"2024-09-17T05:25:05.321309+0000","last_clean":"2024-09-17T05:25:05.321309+0000","last_became_active":"2024-09-17T05:25:03.159636+0000","last_became_peered":"2024-09-17T05:25:03.159636+0000","last_unstale":"2024-09-17T05:25:05.321309+0000","last_undegraded":"2024-09-17T05:25:05.321309+0000","last_fullsized":"2024-09-17T05:25:05.321309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:04.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321681+0000","last_change":"2024-09-17T05:25:04.159287+0000","last_active":"2024-09-17T05:25:05.321681+0000","last_peered":"2024-09-17T05:25:05.321681+0000","last_clean":"2024-09-17T05:25:05.321681+0000","last_became_active":"2024-09-17T05:25:04.158769+0000","last_became_peered":"2024-09-17T05:25:04.158769+0000","last_unstale":"2024-09-17T05:25:05.321681+0000","last_undegraded":"2024-09-17T05:25:05.321681+0000","last_fullsized":"2024-09-17T05:25:05.321681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:27.763662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325478+0000","last_change":"2024-09-17T05:25:03.161164+0000","last_active":"2024-09-17T05:25:05.325478+0000","last_peered":"2024-09-17T05:25:05.325478+0000","last_clean":"2024-09-17T05:25:05.325478+0000","last_became_active":"2024-09-17T05:25:03.160875+0000","last_became_peered":"2024-09-17T05:25:03.160875+0000","last_unstale":"2024-09-17T05:25:05.325478+0000","last_undegraded":"2024-09-17T05:25:05.325478+0000","last_fullsized":"2024-09-17T05:25:05.325478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321087+0000","last_change":"2024-09-17T05:25:04.159000+0000","last_active":"2024-09-17T05:25:05.321087+0000","last_peered":"2024-09-17T05:25:05.321087+0000","last_clean":"2024-09-17T05:25:05.321087+0000","last_became_active":"2024-09-17T05:25:04.158806+0000","last_became_peered":"2024-09-17T05:25:04.158806+0000","last_unstale":"2024-09-17T05:25:05.321087+0000","last_undegraded":"2024-09-17T05:25:05.321087+0000","last_fullsized":"2024-09-17T05:25:05.321087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:50.632241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323888+0000","last_change":"2024-09-17T05:25:03.160173+0000","last_active":"2024-09-17T05:25:05.323888+0000","last_peered":"2024-09-17T05:25:05.323888+0000","last_clean":"2024-09-17T05:25:05.323888+0000","last_became_active":"2024-09-17T05:25:03.159759+0000","last_became_peered":"2024-09-17T05:25:03.159759+0000","last_unstale":"2024-09-17T05:25:05.323888+0000","last_undegraded":"2024-09-17T05:25:05.323888+0000","last_fullsized":"2024-09-17T05:25:05.323888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:06.043431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321445+0000","last_change":"2024-09-17T05:25:04.156812+0000","last_active":"2024-09-17T05:25:05.321445+0000","last_peered":"2024-09-17T05:25:05.321445+0000","last_clean":"2024-09-17T05:25:05.321445+0000","last_became_active":"2024-09-17T05:25:04.147686+0000","last_became_peered":"2024-09-17T05:25:04.147686+0000","last_unstale":"2024-09-17T05:25:05.321445+0000","last_undegraded":"2024-09-17T05:25:05.321445+0000","last_fullsized":"2024-09-17T05:25:05.321445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:03.043749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325125+0000","last_change":"2024-09-17T05:25:03.151849+0000","last_active":"2024-09-17T05:25:05.325125+0000","last_peered":"2024-09-17T05:25:05.325125+0000","last_clean":"2024-09-17T05:25:05.325125+0000","last_became_active":"2024-09-17T05:25:03.151518+0000","last_became_peered":"2024-09-17T05:25:03.151518+0000","last_unstale":"2024-09-17T05:25:05.325125+0000","last_undegraded":"2024-09-17T05:25:05.325125+0000","last_fullsized":"2024-09-17T05:25:05.325125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150742+0000","last_change":"2024-09-17T05:25:04.150792+0000","last_active":"2024-09-17T05:25:05.150742+0000","last_peered":"2024-09-17T05:25:05.150742+0000","last_clean":"2024-09-17T05:25:05.150742+0000","last_became_active":"2024-09-17T05:25:04.150523+0000","last_became_peered":"2024-09-17T05:25:04.150523+0000","last_unstale":"2024-09-17T05:25:05.150742+0000","last_undegraded":"2024-09-17T05:25:05.150742+0000","last_fullsized":"2024-09-17T05:25:05.150742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:09.954167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.152794+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:05.152794+0000","last_peered":"2024-09-17T05:25:05.152794+0000","last_clean":"2024-09-17T05:25:05.152794+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:05.152794+0000","last_undegraded":"2024-09-17T05:25:05.152794+0000","last_fullsized":"2024-09-17T05:25:05.152794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150663+0000","last_change":"2024-09-17T05:25:03.154924+0000","last_active":"2024-09-17T05:25:05.150663+0000","last_peered":"2024-09-17T05:25:05.150663+0000","last_clean":"2024-09-17T05:25:05.150663+0000","last_became_active":"2024-09-17T05:25:03.154274+0000","last_became_peered":"2024-09-17T05:25:03.154274+0000","last_unstale":"2024-09-17T05:25:05.150663+0000","last_undegraded":"2024-09-17T05:25:05.150663+0000","last_fullsized":"2024-09-17T05:25:05.150663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:38.825579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323572+0000","last_change":"2024-09-17T05:25:04.169405+0000","last_active":"2024-09-17T05:25:05.323572+0000","last_peered":"2024-09-17T05:25:05.323572+0000","last_clean":"2024-09-17T05:25:05.323572+0000","last_became_active":"2024-09-17T05:25:04.167552+0000","last_became_peered":"2024-09-17T05:25:04.167552+0000","last_unstale":"2024-09-17T05:25:05.323572+0000","last_undegraded":"2024-09-17T05:25:05.323572+0000","last_fullsized":"2024-09-17T05:25:05.323572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:30.468665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150438+0000","last_change":"2024-09-17T05:25:03.154006+0000","last_active":"2024-09-17T05:25:05.150438+0000","last_peered":"2024-09-17T05:25:05.150438+0000","last_clean":"2024-09-17T05:25:05.150438+0000","last_became_active":"2024-09-17T05:25:03.153578+0000","last_became_peered":"2024-09-17T05:25:03.153578+0000","last_unstale":"2024-09-17T05:25:05.150438+0000","last_undegraded":"2024-09-17T05:25:05.150438+0000","last_fullsized":"2024-09-17T05:25:05.150438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:13.898988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324301+0000","last_change":"2024-09-17T05:25:04.160915+0000","last_active":"2024-09-17T05:25:05.324301+0000","last_peered":"2024-09-17T05:25:05.324301+0000","last_clean":"2024-09-17T05:25:05.324301+0000","last_became_active":"2024-09-17T05:25:04.159519+0000","last_became_peered":"2024-09-17T05:25:04.159519+0000","last_unstale":"2024-09-17T05:25:05.324301+0000","last_undegraded":"2024-09-17T05:25:05.324301+0000","last_fullsized":"2024-09-17T05:25:05.324301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:08.226516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153072+0000","last_change":"2024-09-17T05:25:03.154866+0000","last_active":"2024-09-17T05:25:05.153072+0000","last_peered":"2024-09-17T05:25:05.153072+0000","last_clean":"2024-09-17T05:25:05.153072+0000","last_became_active":"2024-09-17T05:25:03.154116+0000","last_became_peered":"2024-09-17T05:25:03.154116+0000","last_unstale":"2024-09-17T05:25:05.153072+0000","last_undegraded":"2024-09-17T05:25:05.153072+0000","last_fullsized":"2024-09-17T05:25:05.153072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:55.771551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324275+0000","last_change":"2024-09-17T05:25:04.162175+0000","last_active":"2024-09-17T05:25:05.324275+0000","last_peered":"2024-09-17T05:25:05.324275+0000","last_clean":"2024-09-17T05:25:05.324275+0000","last_became_active":"2024-09-17T05:25:04.161820+0000","last_became_peered":"2024-09-17T05:25:04.161820+0000","last_unstale":"2024-09-17T05:25:05.324275+0000","last_undegraded":"2024-09-17T05:25:05.324275+0000","last_fullsized":"2024-09-17T05:25:05.324275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:08.628783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151000+0000","last_change":"2024-09-17T05:25:03.144930+0000","last_active":"2024-09-17T05:25:05.151000+0000","last_peered":"2024-09-17T05:25:05.151000+0000","last_clean":"2024-09-17T05:25:05.151000+0000","last_became_active":"2024-09-17T05:25:03.144665+0000","last_became_peered":"2024-09-17T05:25:03.144665+0000","last_unstale":"2024-09-17T05:25:05.151000+0000","last_undegraded":"2024-09-17T05:25:05.151000+0000","last_fullsized":"2024-09-17T05:25:05.151000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:31.401190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321109+0000","last_change":"2024-09-17T05:25:04.155323+0000","last_active":"2024-09-17T05:25:05.321109+0000","last_peered":"2024-09-17T05:25:05.321109+0000","last_clean":"2024-09-17T05:25:05.321109+0000","last_became_active":"2024-09-17T05:25:04.148816+0000","last_became_peered":"2024-09-17T05:25:04.148816+0000","last_unstale":"2024-09-17T05:25:05.321109+0000","last_undegraded":"2024-09-17T05:25:05.321109+0000","last_fullsized":"2024-09-17T05:25:05.321109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:34.744953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323728+0000","last_change":"2024-09-17T05:25:03.160065+0000","last_active":"2024-09-17T05:25:05.323728+0000","last_peered":"2024-09-17T05:25:05.323728+0000","last_clean":"2024-09-17T05:25:05.323728+0000","last_became_active":"2024-09-17T05:25:03.159684+0000","last_became_peered":"2024-09-17T05:25:03.159684+0000","last_unstale":"2024-09-17T05:25:05.323728+0000","last_undegraded":"2024-09-17T05:25:05.323728+0000","last_fullsized":"2024-09-17T05:25:05.323728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:04.387303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320608+0000","last_change":"2024-09-17T05:25:04.171447+0000","last_active":"2024-09-17T05:25:05.320608+0000","last_peered":"2024-09-17T05:25:05.320608+0000","last_clean":"2024-09-17T05:25:05.320608+0000","last_became_active":"2024-09-17T05:25:04.171049+0000","last_became_peered":"2024-09-17T05:25:04.171049+0000","last_unstale":"2024-09-17T05:25:05.320608+0000","last_undegraded":"2024-09-17T05:25:05.320608+0000","last_fullsized":"2024-09-17T05:25:05.320608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:44.990639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150897+0000","last_change":"2024-09-17T05:25:03.150695+0000","last_active":"2024-09-17T05:25:05.150897+0000","last_peered":"2024-09-17T05:25:05.150897+0000","last_clean":"2024-09-17T05:25:05.150897+0000","last_became_active":"2024-09-17T05:25:03.150452+0000","last_became_peered":"2024-09-17T05:25:03.150452+0000","last_unstale":"2024-09-17T05:25:05.150897+0000","last_undegraded":"2024-09-17T05:25:05.150897+0000","last_fullsized":"2024-09-17T05:25:05.150897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:04.967226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325109+0000","last_change":"2024-09-17T05:25:04.173584+0000","last_active":"2024-09-17T05:25:05.325109+0000","last_peered":"2024-09-17T05:25:05.325109+0000","last_clean":"2024-09-17T05:25:05.325109+0000","last_became_active":"2024-09-17T05:25:04.173144+0000","last_became_peered":"2024-09-17T05:25:04.173144+0000","last_unstale":"2024-09-17T05:25:05.325109+0000","last_undegraded":"2024-09-17T05:25:05.325109+0000","last_fullsized":"2024-09-17T05:25:05.325109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:22.262151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325559+0000","last_change":"2024-09-17T05:25:03.153186+0000","last_active":"2024-09-17T05:25:05.325559+0000","last_peered":"2024-09-17T05:25:05.325559+0000","last_clean":"2024-09-17T05:25:05.325559+0000","last_became_active":"2024-09-17T05:25:03.152579+0000","last_became_peered":"2024-09-17T05:25:03.152579+0000","last_unstale":"2024-09-17T05:25:05.325559+0000","last_undegraded":"2024-09-17T05:25:05.325559+0000","last_fullsized":"2024-09-17T05:25:05.325559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:40.038477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325278+0000","last_change":"2024-09-17T05:25:04.178286+0000","last_active":"2024-09-17T05:25:05.325278+0000","last_peered":"2024-09-17T05:25:05.325278+0000","last_clean":"2024-09-17T05:25:05.325278+0000","last_became_active":"2024-09-17T05:25:04.177979+0000","last_became_peered":"2024-09-17T05:25:04.177979+0000","last_unstale":"2024-09-17T05:25:05.325278+0000","last_undegraded":"2024-09-17T05:25:05.325278+0000","last_fullsized":"2024-09-17T05:25:05.325278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:24.204998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.151091+0000","last_change":"2024-09-17T05:25:03.144098+0000","last_active":"2024-09-17T05:25:05.151091+0000","last_peered":"2024-09-17T05:25:05.151091+0000","last_clean":"2024-09-17T05:25:05.151091+0000","last_became_active":"2024-09-17T05:25:03.137827+0000","last_became_peered":"2024-09-17T05:25:03.137827+0000","last_unstale":"2024-09-17T05:25:05.151091+0000","last_undegraded":"2024-09-17T05:25:05.151091+0000","last_fullsized":"2024-09-17T05:25:05.151091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:36.454620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325378+0000","last_change":"2024-09-17T05:25:04.169501+0000","last_active":"2024-09-17T05:25:05.325378+0000","last_peered":"2024-09-17T05:25:05.325378+0000","last_clean":"2024-09-17T05:25:05.325378+0000","last_became_active":"2024-09-17T05:25:04.168926+0000","last_became_peered":"2024-09-17T05:25:04.168926+0000","last_unstale":"2024-09-17T05:25:05.325378+0000","last_undegraded":"2024-09-17T05:25:05.325378+0000","last_fullsized":"2024-09-17T05:25:05.325378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:51.977366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.320442+0000","last_change":"2024-09-17T05:25:03.155817+0000","last_active":"2024-09-17T05:25:05.320442+0000","last_peered":"2024-09-17T05:25:05.320442+0000","last_clean":"2024-09-17T05:25:05.320442+0000","last_became_active":"2024-09-17T05:25:03.155300+0000","last_became_peered":"2024-09-17T05:25:03.155300+0000","last_unstale":"2024-09-17T05:25:05.320442+0000","last_undegraded":"2024-09-17T05:25:05.320442+0000","last_fullsized":"2024-09-17T05:25:05.320442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:17.915256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150325+0000","last_change":"2024-09-17T05:25:04.167116+0000","last_active":"2024-09-17T05:25:05.150325+0000","last_peered":"2024-09-17T05:25:05.150325+0000","last_clean":"2024-09-17T05:25:05.150325+0000","last_became_active":"2024-09-17T05:25:04.166832+0000","last_became_peered":"2024-09-17T05:25:04.166832+0000","last_unstale":"2024-09-17T05:25:05.150325+0000","last_undegraded":"2024-09-17T05:25:05.150325+0000","last_fullsized":"2024-09-17T05:25:05.150325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:41.040086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323594+0000","last_change":"2024-09-17T05:25:03.155466+0000","last_active":"2024-09-17T05:25:05.323594+0000","last_peered":"2024-09-17T05:25:05.323594+0000","last_clean":"2024-09-17T05:25:05.323594+0000","last_became_active":"2024-09-17T05:25:03.153103+0000","last_became_peered":"2024-09-17T05:25:03.153103+0000","last_unstale":"2024-09-17T05:25:05.323594+0000","last_undegraded":"2024-09-17T05:25:05.323594+0000","last_fullsized":"2024-09-17T05:25:05.323594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:29.910645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323335+0000","last_change":"2024-09-17T05:25:04.166048+0000","last_active":"2024-09-17T05:25:05.323335+0000","last_peered":"2024-09-17T05:25:05.323335+0000","last_clean":"2024-09-17T05:25:05.323335+0000","last_became_active":"2024-09-17T05:25:04.165647+0000","last_became_peered":"2024-09-17T05:25:04.165647+0000","last_unstale":"2024-09-17T05:25:05.323335+0000","last_undegraded":"2024-09-17T05:25:05.323335+0000","last_fullsized":"2024-09-17T05:25:05.323335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:22.190410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324888+0000","last_change":"2024-09-17T05:25:03.156207+0000","last_active":"2024-09-17T05:25:05.324888+0000","last_peered":"2024-09-17T05:25:05.324888+0000","last_clean":"2024-09-17T05:25:05.324888+0000","last_became_active":"2024-09-17T05:25:03.155889+0000","last_became_peered":"2024-09-17T05:25:03.155889+0000","last_unstale":"2024-09-17T05:25:05.324888+0000","last_undegraded":"2024-09-17T05:25:05.324888+0000","last_fullsized":"2024-09-17T05:25:05.324888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:39.555234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153253+0000","last_change":"2024-09-17T05:25:04.163791+0000","last_active":"2024-09-17T05:25:05.153253+0000","last_peered":"2024-09-17T05:25:05.153253+0000","last_clean":"2024-09-17T05:25:05.153253+0000","last_became_active":"2024-09-17T05:25:04.163047+0000","last_became_peered":"2024-09-17T05:25:04.163047+0000","last_unstale":"2024-09-17T05:25:05.153253+0000","last_undegraded":"2024-09-17T05:25:05.153253+0000","last_fullsized":"2024-09-17T05:25:05.153253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:05.977620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150564+0000","last_change":"2024-09-17T05:25:03.158082+0000","last_active":"2024-09-17T05:25:05.150564+0000","last_peered":"2024-09-17T05:25:05.150564+0000","last_clean":"2024-09-17T05:25:05.150564+0000","last_became_active":"2024-09-17T05:25:03.157792+0000","last_became_peered":"2024-09-17T05:25:03.157792+0000","last_unstale":"2024-09-17T05:25:05.150564+0000","last_undegraded":"2024-09-17T05:25:05.150564+0000","last_fullsized":"2024-09-17T05:25:05.150564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:07.216122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150416+0000","last_change":"2024-09-17T05:25:04.153904+0000","last_active":"2024-09-17T05:25:05.150416+0000","last_peered":"2024-09-17T05:25:05.150416+0000","last_clean":"2024-09-17T05:25:05.150416+0000","last_became_active":"2024-09-17T05:25:04.153582+0000","last_became_peered":"2024-09-17T05:25:04.153582+0000","last_unstale":"2024-09-17T05:25:05.150416+0000","last_undegraded":"2024-09-17T05:25:05.150416+0000","last_fullsized":"2024-09-17T05:25:05.150416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:49.536379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.150685+0000","last_change":"2024-09-17T05:25:04.150254+0000","last_active":"2024-09-17T05:25:05.150685+0000","last_peered":"2024-09-17T05:25:05.150685+0000","last_clean":"2024-09-17T05:25:05.150685+0000","last_became_active":"2024-09-17T05:25:04.149481+0000","last_became_peered":"2024-09-17T05:25:04.149481+0000","last_unstale":"2024-09-17T05:25:05.150685+0000","last_undegraded":"2024-09-17T05:25:05.150685+0000","last_fullsized":"2024-09-17T05:25:05.150685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:15.901642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321580+0000","last_change":"2024-09-17T05:25:03.159779+0000","last_active":"2024-09-17T05:25:05.321580+0000","last_peered":"2024-09-17T05:25:05.321580+0000","last_clean":"2024-09-17T05:25:05.321580+0000","last_became_active":"2024-09-17T05:25:03.159430+0000","last_became_peered":"2024-09-17T05:25:03.159430+0000","last_unstale":"2024-09-17T05:25:05.321580+0000","last_undegraded":"2024-09-17T05:25:05.321580+0000","last_fullsized":"2024-09-17T05:25:05.321580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:33.326424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.153233+0000","last_change":"2024-09-17T05:25:04.173233+0000","last_active":"2024-09-17T05:25:05.153233+0000","last_peered":"2024-09-17T05:25:05.153233+0000","last_clean":"2024-09-17T05:25:05.153233+0000","last_became_active":"2024-09-17T05:25:04.172918+0000","last_became_peered":"2024-09-17T05:25:04.172918+0000","last_unstale":"2024-09-17T05:25:05.153233+0000","last_undegraded":"2024-09-17T05:25:05.153233+0000","last_fullsized":"2024-09-17T05:25:05.153233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:06.019281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.321565+0000","last_change":"2024-09-17T05:25:03.153878+0000","last_active":"2024-09-17T05:25:05.321565+0000","last_peered":"2024-09-17T05:25:05.321565+0000","last_clean":"2024-09-17T05:25:05.321565+0000","last_became_active":"2024-09-17T05:25:03.153594+0000","last_became_peered":"2024-09-17T05:25:03.153594+0000","last_unstale":"2024-09-17T05:25:05.321565+0000","last_undegraded":"2024-09-17T05:25:05.321565+0000","last_fullsized":"2024-09-17T05:25:05.321565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:16:31.831690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.323879+0000","last_change":"2024-09-17T05:25:04.163341+0000","last_active":"2024-09-17T05:25:05.323879+0000","last_peered":"2024-09-17T05:25:05.323879+0000","last_clean":"2024-09-17T05:25:05.323879+0000","last_became_active":"2024-09-17T05:25:04.162727+0000","last_became_peered":"2024-09-17T05:25:04.162727+0000","last_unstale":"2024-09-17T05:25:05.323879+0000","last_undegraded":"2024-09-17T05:25:05.323879+0000","last_fullsized":"2024-09-17T05:25:05.323879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:04.451009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324619+0000","last_change":"2024-09-17T05:25:03.157287+0000","last_active":"2024-09-17T05:25:05.324619+0000","last_peered":"2024-09-17T05:25:05.324619+0000","last_clean":"2024-09-17T05:25:05.324619+0000","last_became_active":"2024-09-17T05:25:03.154403+0000","last_became_peered":"2024-09-17T05:25:03.154403+0000","last_unstale":"2024-09-17T05:25:05.324619+0000","last_undegraded":"2024-09-17T05:25:05.324619+0000","last_fullsized":"2024-09-17T05:25:05.324619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:21.699418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.324558+0000","last_change":"2024-09-17T05:25:04.167303+0000","last_active":"2024-09-17T05:25:05.324558+0000","last_peered":"2024-09-17T05:25:05.324558+0000","last_clean":"2024-09-17T05:25:05.324558+0000","last_became_active":"2024-09-17T05:25:04.166953+0000","last_became_peered":"2024-09-17T05:25:04.166953+0000","last_unstale":"2024-09-17T05:25:05.324558+0000","last_undegraded":"2024-09-17T05:25:05.324558+0000","last_fullsized":"2024-09-17T05:25:05.324558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:03.119083+0000","last_clean_scrub_stamp":"2024-09-17T05:25:03.119083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:02.920364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T05:25:05.325412+0000","last_change":"2024-09-17T05:25:03.156536+0000","last_active":"2024-09-17T05:25:05.325412+0000","last_peered":"2024-09-17T05:25:05.325412+0000","last_clean":"2024-09-17T05:25:05.325412+0000","last_became_active":"2024-09-17T05:25:03.155913+0000","last_became_peered":"2024-09-17T05:25:03.155913+0000","last_unstale":"2024-09-17T05:25:05.325412+0000","last_undegraded":"2024-09-17T05:25:05.325412+0000","last_fullsized":"2024-09-17T05:25:05.325412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:02.108690+0000","last_clean_scrub_stamp":"2024-09-17T05:25:02.108690+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:08.594253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:18.234 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:25:18.234 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T05:25:18.234 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T05:25:18.234 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:25:18.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- 172.21.15.122:0/1548029291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efca40b2db0 msgr2=0x7efca415ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 --2- 172.21.15.122:0/1548029291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efca40b2db0 0x7efca415ba10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009840 tx=0x7efc8c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- 172.21.15.122:0/1548029291 shutdown_connections 2024-09-17T05:25:18.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 --2- 172.21.15.122:0/1548029291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efca4072f00 0x7efca4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 --2- 172.21.15.122:0/1548029291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efca40b2db0 0x7efca415ba10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- 172.21.15.122:0/1548029291 >> 172.21.15.122:0/1548029291 conn(0x7efca406cdf0 msgr2=0x7efca406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- 172.21.15.122:0/1548029291 shutdown_connections 2024-09-17T05:25:18.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- 172.21.15.122:0/1548029291 wait complete. 2024-09-17T05:25:18.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 Processor -- start 2024-09-17T05:25:18.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- start start 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca406fdc0 0x7efca414b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efca4072f00 0x7efca406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efca40b2db0 0x7efca406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efca4167f70 con 0x7efca406fdc0 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efca4167df0 con 0x7efca4072f00 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca9e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efca41680f0 con 0x7efca40b2db0 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca406fdc0 0x7efca414b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca406fdc0 0x7efca414b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43216/0 (socket says 172.21.15.122:43216) 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca2a00640 1 -- 172.21.15.122:0/1575511087 learned_addr learned my addr 172.21.15.122:0/1575511087 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:18.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efc9a000640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efca4072f00 0x7efca406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.410+0000 7efca3400640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efca40b2db0 0x7efca406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca2a00640 1 -- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efca40b2db0 msgr2=0x7efca406f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca2a00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efca40b2db0 0x7efca406f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca2a00640 1 -- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efca4072f00 msgr2=0x7efca406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca2a00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efca4072f00 0x7efca406ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca2a00640 1 -- 172.21.15.122:0/1575511087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca414bc10 con 0x7efca406fdc0 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca2a00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca406fdc0 0x7efca414b6d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c7f0 tx=0x7efc9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efc9be00640 1 -- 172.21.15.122:0/1575511087 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94017070 con 0x7efca406fdc0 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efc9be00640 1 -- 172.21.15.122:0/1575511087 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc94005070 con 0x7efca406fdc0 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca414fc80 con 0x7efca406fdc0 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efc9be00640 1 -- 172.21.15.122:0/1575511087 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc940054c0 con 0x7efca406fdc0 2024-09-17T05:25:18.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca4159f60 con 0x7efca406fdc0 2024-09-17T05:25:18.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca4073390 con 0x7efca406fdc0 2024-09-17T05:25:18.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efc9be00640 1 -- 172.21.15.122:0/1575511087 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc9401a070 con 0x7efca406fdc0 2024-09-17T05:25:18.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efc9be00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc80073200 0x7efc800756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.414+0000 7efc9be00640 1 -- 172.21.15.122:0/1575511087 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efc940938f0 con 0x7efca406fdc0 2024-09-17T05:25:18.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.418+0000 7efc9a000640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc80073200 0x7efc800756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.418+0000 7efc9a000640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc80073200 0x7efc800756c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0af000 tx=0x7efc8c01b030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.418+0000 7efc9be00640 1 -- 172.21.15.122:0/1575511087 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efca414bc10 con 0x7efca406fdc0 2024-09-17T05:25:18.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efca40b4170 con 0x7efca406fdc0 2024-09-17T05:25:18.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efc9be00640 1 -- 172.21.15.122:0/1575511087 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7efc9405e2b0 con 0x7efca406fdc0 2024-09-17T05:25:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:18.625 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc80073200 msgr2=0x7efc800756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc80073200 0x7efc800756c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0af000 tx=0x7efc8c01b030 comp rx=0 tx=0).stop 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca406fdc0 msgr2=0x7efca414b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca406fdc0 0x7efca414b6d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c7f0 tx=0x7efc9400ccb0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 shutdown_connections 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc80073200 0x7efc800756c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efca40b2db0 0x7efca406f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efca4072f00 0x7efca406ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 --2- 172.21.15.122:0/1575511087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efca406fdc0 0x7efca414b6d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 >> 172.21.15.122:0/1575511087 conn(0x7efca406cdf0 msgr2=0x7efca4151980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 shutdown_connections 2024-09-17T05:25:18.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.622+0000 7efca9e00640 1 -- 172.21.15.122:0/1575511087 wait complete. 2024-09-17T05:25:18.645 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T05:25:18.645 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:18.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- 172.21.15.122:0/2941637174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 msgr2=0x7f2c340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 --2- 172.21.15.122:0/2941637174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 0x7f2c340051d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009ee0 tx=0x7f2c4401d5f0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c52600640 1 -- 172.21.15.122:0/2941637174 reap_dead start 2024-09-17T05:25:18.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- 172.21.15.122:0/2941637174 shutdown_connections 2024-09-17T05:25:18.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- 172.21.15.122:0/2941637174 >> 172.21.15.122:0/2941637174 conn(0x7f2c4c0d1dc0 msgr2=0x7f2c4c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- 172.21.15.122:0/2941637174 shutdown_connections 2024-09-17T05:25:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- 172.21.15.122:0/2941637174 wait complete. 2024-09-17T05:25:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 Processor -- start 2024-09-17T05:25:18.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- start start 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 0x7f2c4c15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c4c1e91d0 0x7f2c4c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c4c0ccf10 0x7f2c4c1e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0cebb0 con 0x7f2c34004d50 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c15c490 con 0x7f2c4c0ccf10 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c54600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c0ced30 con 0x7f2c4c1e91d0 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 0x7f2c4c15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 0x7f2c4c15ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43246/0 (socket says 172.21.15.122:43246) 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51c00640 1 -- 172.21.15.122:0/659443238 learned_addr learned my addr 172.21.15.122:0/659443238 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:18.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51200640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c4c1e91d0 0x7f2c4c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c52600640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c4c0ccf10 0x7f2c4c1e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51c00640 1 -- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c4c1e91d0 msgr2=0x7f2c4c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51c00640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c4c1e91d0 0x7f2c4c0cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51c00640 1 -- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c4c0ccf10 msgr2=0x7f2c4c1e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51c00640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c4c0ccf10 0x7f2c4c1e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.786+0000 7f2c51c00640 1 -- 172.21.15.122:0/659443238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c1e5980 con 0x7f2c34004d50 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c51c00640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 0x7f2c4c15ac60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c40011ae0 tx=0x7f2c40009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c49600640 1 -- 172.21.15.122:0/659443238 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000fcd0 con 0x7f2c34004d50 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c49600640 1 -- 172.21.15.122:0/659443238 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c4000fe70 con 0x7f2c34004d50 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c0b2280 con 0x7f2c34004d50 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c49600640 1 -- 172.21.15.122:0/659443238 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40004260 con 0x7f2c34004d50 2024-09-17T05:25:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c0b1cc0 con 0x7f2c34004d50 2024-09-17T05:25:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4c0ce670 con 0x7f2c34004d50 2024-09-17T05:25:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c49600640 1 -- 172.21.15.122:0/659443238 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2c40003600 con 0x7f2c34004d50 2024-09-17T05:25:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c49600640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c2c073230 0x7f2c2c0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:18.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c49600640 1 -- 172.21.15.122:0/659443238 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2c40094d10 con 0x7f2c34004d50 2024-09-17T05:25:18.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.790+0000 7f2c51200640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c2c073230 0x7f2c2c0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:18.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.794+0000 7f2c51200640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c2c073230 0x7f2c2c0756f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c38000c10 tx=0x7f2c380014c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:18.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.794+0000 7f2c49600640 1 -- 172.21.15.122:0/659443238 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2c4005fda0 con 0x7f2c34004d50 2024-09-17T05:25:19.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c4c0714a0 con 0x7f2c34004d50 2024-09-17T05:25:19.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c49600640 1 -- 172.21.15.122:0/659443238 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f2c4005f6a0 con 0x7f2c34004d50 2024-09-17T05:25:19.001 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:19.001 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:03.536393+0000","modified":"2024-09-17T05:25:06.553594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4332":{"gid":4332,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4332,"qdb_cluster":[4332]},"id":1}]} 2024-09-17T05:25:19.001 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c2c073230 msgr2=0x7f2c2c0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c2c073230 0x7f2c2c0756f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c38000c10 tx=0x7f2c380014c0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 msgr2=0x7f2c4c15ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 0x7f2c4c15ac60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c40011ae0 tx=0x7f2c40009f20 comp rx=0 tx=0).stop 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 shutdown_connections 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c2c073230 0x7f2c2c0756f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c4c0ccf10 0x7f2c4c1e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c4c1e91d0 0x7f2c4c0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 --2- 172.21.15.122:0/659443238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c34004d50 0x7f2c4c15ac60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 >> 172.21.15.122:0/659443238 conn(0x7f2c4c0d1dc0 msgr2=0x7f2c4c071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:19.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 shutdown_connections 2024-09-17T05:25:19.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:18.998+0000 7f2c54600640 1 -- 172.21.15.122:0/659443238 wait complete. 2024-09-17T05:25:19.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:19.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.162+0000 7fb607200640 1 -- 172.21.15.122:0/3247573583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb608152a50 msgr2=0x7fb60815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.162+0000 7fb607200640 1 --2- 172.21.15.122:0/3247573583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb608152a50 0x7fb60815ccb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb6080ce3b0 tx=0x7fb5f801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.162+0000 7fb607200640 1 -- 172.21.15.122:0/3247573583 shutdown_connections 2024-09-17T05:25:19.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.162+0000 7fb607200640 1 --2- 172.21.15.122:0/3247573583 >> v2:172.21.15.173:3300/0 conn(0x7fb60815fd50 0x7fb60806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.162+0000 7fb607200640 1 --2- 172.21.15.122:0/3247573583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb608152a50 0x7fb60815ccb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.162+0000 7fb607200640 1 -- 172.21.15.122:0/3247573583 >> 172.21.15.122:0/3247573583 conn(0x7fb60806dd30 msgr2=0x7fb60806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:19.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.162+0000 7fb607200640 1 -- 172.21.15.122:0/3247573583 shutdown_connections 2024-09-17T05:25:19.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.162+0000 7fb607200640 1 -- 172.21.15.122:0/3247573583 wait complete. 2024-09-17T05:25:19.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 Processor -- start 2024-09-17T05:25:19.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 -- start start 2024-09-17T05:25:19.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60806cc20 0x7fb608148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb608152a50 0x7fb6081540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb60815fd50 0x7fb60806ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6081689e0 con 0x7fb60806cc20 2024-09-17T05:25:19.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6080b4880 con 0x7fb60815fd50 2024-09-17T05:25:19.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb608168b60 con 0x7fb608152a50 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60806cc20 0x7fb608148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60806cc20 0x7fb608148090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43276/0 (socket says 172.21.15.122:43276) 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 -- 172.21.15.122:0/3151495855 learned_addr learned my addr 172.21.15.122:0/3151495855 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb606800640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb60815fd50 0x7fb60806ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605400640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb608152a50 0x7fb6081540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 -- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb608152a50 msgr2=0x7fb6081540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb608152a50 0x7fb6081540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 -- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb60815fd50 msgr2=0x7fb60806ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb60815fd50 0x7fb60806ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 -- 172.21.15.122:0/3151495855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60814a380 con 0x7fb60806cc20 2024-09-17T05:25:19.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb605e00640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60806cc20 0x7fb608148090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80194d0 tx=0x7fb5f80a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:19.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb5fe000640 1 -- 172.21.15.122:0/3151495855 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80b2040 con 0x7fb60806cc20 2024-09-17T05:25:19.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb5fe000640 1 -- 172.21.15.122:0/3151495855 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5f80044a0 con 0x7fb60806cc20 2024-09-17T05:25:19.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb5fe000640 1 -- 172.21.15.122:0/3151495855 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8005290 con 0x7fb60806cc20 2024-09-17T05:25:19.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60814a660 con 0x7fb60806cc20 2024-09-17T05:25:19.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60814ac20 con 0x7fb60806cc20 2024-09-17T05:25:19.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.166+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb608148fe0 con 0x7fb60806cc20 2024-09-17T05:25:19.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.170+0000 7fb5fe000640 1 -- 172.21.15.122:0/3151495855 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb5f8005430 con 0x7fb60806cc20 2024-09-17T05:25:19.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.170+0000 7fb5fe000640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5e8073200 0x7fb5e80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.170+0000 7fb5fe000640 1 -- 172.21.15.122:0/3151495855 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb5f812e1d0 con 0x7fb60806cc20 2024-09-17T05:25:19.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.170+0000 7fb605400640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5e8073200 0x7fb5e80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.170+0000 7fb605400640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5e8073200 0x7fb5e80756c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb6080ce480 tx=0x7fb5f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:19.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.174+0000 7fb5fe000640 1 -- 172.21.15.122:0/3151495855 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb5f80c4060 con 0x7fb60806cc20 2024-09-17T05:25:19.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb60815ccb0 con 0x7fb60806cc20 2024-09-17T05:25:19.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb5fe000640 1 -- 172.21.15.122:0/3151495855 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7fb5f80164b0 con 0x7fb60806cc20 2024-09-17T05:25:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:19.381 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:03.536393+0000","modified":"2024-09-17T05:25:06.553594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4332":{"gid":4332,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4332,"qdb_cluster":[4332]},"id":1}]} 2024-09-17T05:25:19.381 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5e8073200 msgr2=0x7fb5e80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5e8073200 0x7fb5e80756c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb6080ce480 tx=0x7fb5f4009040 comp rx=0 tx=0).stop 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60806cc20 msgr2=0x7fb608148090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60806cc20 0x7fb608148090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80194d0 tx=0x7fb5f80a57b0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 shutdown_connections 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5e8073200 0x7fb5e80756c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb60815fd50 0x7fb60806ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb608152a50 0x7fb6081540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 --2- 172.21.15.122:0/3151495855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60806cc20 0x7fb608148090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 >> 172.21.15.122:0/3151495855 conn(0x7fb60806dd30 msgr2=0x7fb6080b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:19.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 shutdown_connections 2024-09-17T05:25:19.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.378+0000 7fb607200640 1 -- 172.21.15.122:0/3151495855 wait complete. 2024-09-17T05:25:19.401 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:19.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.538+0000 7fe2b4800640 1 -- 172.21.15.122:0/2209368576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 msgr2=0x7fe294003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.538+0000 7fe2b4800640 1 --2- 172.21.15.122:0/2209368576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 0x7fe294003e80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000c260 tx=0x7fe2a001cf10 comp rx=0 tx=0).stop 2024-09-17T05:25:19.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.538+0000 7fe2b4800640 1 -- 172.21.15.122:0/2209368576 shutdown_connections 2024-09-17T05:25:19.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.538+0000 7fe2b4800640 1 --2- 172.21.15.122:0/2209368576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 0x7fe294003e80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.538+0000 7fe2b4800640 1 -- 172.21.15.122:0/2209368576 >> 172.21.15.122:0/2209368576 conn(0x7fe2ac0ccf10 msgr2=0x7fe2ac0cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:19.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.538+0000 7fe2b4800640 1 -- 172.21.15.122:0/2209368576 shutdown_connections 2024-09-17T05:25:19.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.538+0000 7fe2b4800640 1 -- 172.21.15.122:0/2209368576 wait complete. 2024-09-17T05:25:19.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 Processor -- start 2024-09-17T05:25:19.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 -- start start 2024-09-17T05:25:19.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 0x7fe2ac15e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ac15cc20 0x7fe2ac154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ac0b5640 0x7fe2ac0b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0ada60 con 0x7fe294003a60 2024-09-17T05:25:19.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0ad8e0 con 0x7fe2ac15cc20 2024-09-17T05:25:19.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0adbe0 con 0x7fe2ac0b5640 2024-09-17T05:25:19.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 0x7fe2ac15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 0x7fe2ac15e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43292/0 (socket says 172.21.15.122:43292) 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 -- 172.21.15.122:0/3112231647 learned_addr learned my addr 172.21.15.122:0/3112231647 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b2800640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ac0b5640 0x7fe2ac0b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1400640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ac15cc20 0x7fe2ac154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 -- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ac0b5640 msgr2=0x7fe2ac0b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ac0b5640 0x7fe2ac0b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 -- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ac15cc20 msgr2=0x7fe2ac154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ac15cc20 0x7fe2ac154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 -- 172.21.15.122:0/3112231647 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac154ea0 con 0x7fe294003a60 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1400640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ac15cc20 0x7fe2ac154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b1e00640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 0x7fe2ac15e800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0000c10 tx=0x7fe2a0006c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2a6000640 1 -- 172.21.15.122:0/3112231647 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00060d0 con 0x7fe294003a60 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2a6000640 1 -- 172.21.15.122:0/3112231647 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a0003600 con 0x7fe294003a60 2024-09-17T05:25:19.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2a6000640 1 -- 172.21.15.122:0/3112231647 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00039d0 con 0x7fe294003a60 2024-09-17T05:25:19.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac1fb890 con 0x7fe294003a60 2024-09-17T05:25:19.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.542+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac1fbe50 con 0x7fe294003a60 2024-09-17T05:25:19.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.546+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ac161470 con 0x7fe294003a60 2024-09-17T05:25:19.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.546+0000 7fe2a6000640 1 -- 172.21.15.122:0/3112231647 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe2a0003b70 con 0x7fe294003a60 2024-09-17T05:25:19.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.546+0000 7fe2a6000640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe290073230 0x7fe2900756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.546+0000 7fe2a6000640 1 -- 172.21.15.122:0/3112231647 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe2a012e2e0 con 0x7fe294003a60 2024-09-17T05:25:19.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.546+0000 7fe2b1400640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe290073230 0x7fe2900756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.546+0000 7fe2b1400640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe290073230 0x7fe2900756f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0b1dc0 tx=0x7fe29c006b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:19.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.550+0000 7fe2a6000640 1 -- 172.21.15.122:0/3112231647 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe2a00c0030 con 0x7fe294003a60 2024-09-17T05:25:19.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2ac0cae30 con 0x7fe294003a60 2024-09-17T05:25:19.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2a6000640 1 -- 172.21.15.122:0/3112231647 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7fe2a0016cf0 con 0x7fe294003a60 2024-09-17T05:25:19.725 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:19.725 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":22,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:05.140074+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:19.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe290073230 msgr2=0x7fe2900756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe290073230 0x7fe2900756f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0b1dc0 tx=0x7fe29c006b50 comp rx=0 tx=0).stop 2024-09-17T05:25:19.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 msgr2=0x7fe2ac15e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 0x7fe2ac15e800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0000c10 tx=0x7fe2a0006c80 comp rx=0 tx=0).stop 2024-09-17T05:25:19.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 shutdown_connections 2024-09-17T05:25:19.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe290073230 0x7fe2900756f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ac0b5640 0x7fe2ac0b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ac15cc20 0x7fe2ac154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 --2- 172.21.15.122:0/3112231647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe294003a60 0x7fe2ac15e800 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 >> 172.21.15.122:0/3112231647 conn(0x7fe2ac0ccf10 msgr2=0x7fe2ac0706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:19.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 shutdown_connections 2024-09-17T05:25:19.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.722+0000 7fe2b4800640 1 -- 172.21.15.122:0/3112231647 wait complete. 2024-09-17T05:25:19.746 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:19.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.882+0000 7f0a7da00640 1 -- 172.21.15.122:0/956018075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 msgr2=0x7f0a780ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.882+0000 7f0a7da00640 1 --2- 172.21.15.122:0/956018075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 0x7f0a780ce670 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0a78073d10 tx=0x7f0a6801caf0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.882+0000 7f0a7da00640 1 -- 172.21.15.122:0/956018075 shutdown_connections 2024-09-17T05:25:19.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.882+0000 7f0a7da00640 1 --2- 172.21.15.122:0/956018075 >> v2:172.21.15.173:3300/0 conn(0x7f0a780cdb30 0x7f0a780cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.882+0000 7f0a7da00640 1 --2- 172.21.15.122:0/956018075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 0x7f0a780ce670 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.882+0000 7f0a7da00640 1 -- 172.21.15.122:0/956018075 >> 172.21.15.122:0/956018075 conn(0x7f0a780d1dc0 msgr2=0x7f0a780d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:19.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.882+0000 7f0a7da00640 1 -- 172.21.15.122:0/956018075 shutdown_connections 2024-09-17T05:25:19.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.882+0000 7f0a7da00640 1 -- 172.21.15.122:0/956018075 wait complete. 2024-09-17T05:25:19.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 Processor -- start 2024-09-17T05:25:19.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 -- start start 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a780cdb30 0x7f0a7815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a780d28e0 0x7f0a780c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 0x7f0a7815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78168fc0 con 0x7f0a7815bea0 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78168e40 con 0x7f0a780cdb30 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0a78169140 con 0x7f0a780d28e0 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 0x7f0a7815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 0x7f0a7815a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43312/0 (socket says 172.21.15.122:43312) 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 -- 172.21.15.122:0/2208895720 learned_addr learned my addr 172.21.15.122:0/2208895720 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a76000640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a780d28e0 0x7f0a780c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a76a00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a780cdb30 0x7f0a7815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 -- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a780d28e0 msgr2=0x7f0a780c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a780d28e0 0x7f0a780c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 -- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a780cdb30 msgr2=0x7f0a7815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a780cdb30 0x7f0a7815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 -- 172.21.15.122:0/2208895720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7815aa30 con 0x7f0a7815bea0 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a76000640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a780d28e0 0x7f0a780c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a77400640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 0x7f0a7815a4f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000c3a0 tx=0x7f0a7000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a6ea00640 1 -- 172.21.15.122:0/2208895720 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70017070 con 0x7f0a7815bea0 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a6ea00640 1 -- 172.21.15.122:0/2208895720 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a70003600 con 0x7f0a7815bea0 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a6ea00640 1 -- 172.21.15.122:0/2208895720 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a70003a50 con 0x7f0a7815bea0 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a780c5630 con 0x7f0a7815bea0 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a780c8020 con 0x7f0a7815bea0 2024-09-17T05:25:19.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.886+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a780c7e10 con 0x7f0a7815bea0 2024-09-17T05:25:19.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.890+0000 7f0a6ea00640 1 -- 172.21.15.122:0/2208895720 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0a7001a070 con 0x7f0a7815bea0 2024-09-17T05:25:19.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.890+0000 7f0a6ea00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a58073200 0x7f0a580756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:19.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.890+0000 7f0a6ea00640 1 -- 172.21.15.122:0/2208895720 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0a70093400 con 0x7f0a7815bea0 2024-09-17T05:25:19.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.890+0000 7f0a76a00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a58073200 0x7f0a580756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:19.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.890+0000 7f0a76a00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a58073200 0x7f0a580756c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0a680af000 tx=0x7f0a6801b330 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:19.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:19.894+0000 7f0a6ea00640 1 -- 172.21.15.122:0/2208895720 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0a7005de50 con 0x7f0a7815bea0 2024-09-17T05:25:20.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.094+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a780ccc80 con 0x7f0a7815bea0 2024-09-17T05:25:20.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.094+0000 7f0a6ea00640 1 -- 172.21.15.122:0/2208895720 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f0a70062880 con 0x7f0a7815bea0 2024-09-17T05:25:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:20.099 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:03.536393+0000","modified":"2024-09-17T05:25:06.553594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4332":{"gid":4332,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4332,"qdb_cluster":[4332]},"id":1}]} 2024-09-17T05:25:20.099 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-17T05:25:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a58073200 msgr2=0x7f0a580756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a58073200 0x7f0a580756c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0a680af000 tx=0x7f0a6801b330 comp rx=0 tx=0).stop 2024-09-17T05:25:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 msgr2=0x7f0a7815a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 0x7f0a7815a4f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000c3a0 tx=0x7f0a7000c860 comp rx=0 tx=0).stop 2024-09-17T05:25:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 shutdown_connections 2024-09-17T05:25:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a58073200 0x7f0a580756c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a7815bea0 0x7f0a7815a4f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a780d28e0 0x7f0a780c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 --2- 172.21.15.122:0/2208895720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a780cdb30 0x7f0a7815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 >> 172.21.15.122:0/2208895720 conn(0x7f0a780d1dc0 msgr2=0x7f0a780b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 shutdown_connections 2024-09-17T05:25:20.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.098+0000 7f0a7da00640 1 -- 172.21.15.122:0/2208895720 wait complete. 2024-09-17T05:25:20.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:03.536393+0000', 'modified': '2024-09-17T05:25:06.553594+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4332}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4332': {'gid': 4332, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6837/2742876385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6835', 'nonce': 2742876385}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 2742876385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4332, 'qdb_cluster': [4332]} 2024-09-17T05:25:20.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:25:20.522 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:25:20.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 -- 172.21.15.122:0/2641452141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf00cd8c0 msgr2=0x7fbbf00cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 --2- 172.21.15.122:0/2641452141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf00cd8c0 0x7fbbf00cea20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0074290 tx=0x7fbbe801cbc0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 -- 172.21.15.122:0/2641452141 shutdown_connections 2024-09-17T05:25:20.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 --2- 172.21.15.122:0/2641452141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf00cd8c0 0x7fbbf00cea20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 --2- 172.21.15.122:0/2641452141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbf015be80 0x7fbbf00ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 -- 172.21.15.122:0/2641452141 >> 172.21.15.122:0/2641452141 conn(0x7fbbf00d1da0 msgr2=0x7fbbf00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 -- 172.21.15.122:0/2641452141 shutdown_connections 2024-09-17T05:25:20.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 -- 172.21.15.122:0/2641452141 wait complete. 2024-09-17T05:25:20.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 Processor -- start 2024-09-17T05:25:20.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.662+0000 7fbbf8400640 1 -- start start 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf8400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf00cd8c0 0x7fbbf01f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf8400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbf00d28c0 0x7fbbf015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf8400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf015be80 0x7fbbf015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf8400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0168de0 con 0x7fbbf015be80 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf8400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0168c60 con 0x7fbbf00cd8c0 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf8400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf0168f60 con 0x7fbbf00d28c0 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf015be80 0x7fbbf015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf015be80 0x7fbbf015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43326/0 (socket says 172.21.15.122:43326) 2024-09-17T05:25:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 -- 172.21.15.122:0/1088800240 learned_addr learned my addr 172.21.15.122:0/1088800240 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf00cd8c0 0x7fbbf01f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf5000640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbf00d28c0 0x7fbbf015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 -- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbf00d28c0 msgr2=0x7fbbf015ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbf00d28c0 0x7fbbf015ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 -- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf00cd8c0 msgr2=0x7fbbf01f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf00cd8c0 0x7fbbf01f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 -- 172.21.15.122:0/1088800240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf01f8690 con 0x7fbbf015be80 2024-09-17T05:25:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf6400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf015be80 0x7fbbf015e680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbe801d460 tx=0x7fbbe80a5970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbed600640 1 -- 172.21.15.122:0/1088800240 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80bb070 con 0x7fbbf015be80 2024-09-17T05:25:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbed600640 1 -- 172.21.15.122:0/1088800240 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe8004d40 con 0x7fbbf015be80 2024-09-17T05:25:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbed600640 1 -- 172.21.15.122:0/1088800240 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80b2040 con 0x7fbbf015be80 2024-09-17T05:25:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf8400640 1 -- 172.21.15.122:0/1088800240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf015d1f0 con 0x7fbbf015be80 2024-09-17T05:25:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.666+0000 7fbbf8400640 1 -- 172.21.15.122:0/1088800240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf015f340 con 0x7fbbf015be80 2024-09-17T05:25:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbed600640 1 -- 172.21.15.122:0/1088800240 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbbe8004320 con 0x7fbbf015be80 2024-09-17T05:25:20.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbed600640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbd0073200 0x7fbbd00756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbed600640 1 -- 172.21.15.122:0/1088800240 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbbe812e080 con 0x7fbbf015be80 2024-09-17T05:25:20.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbfb89b000 1 Processor -- start 2024-09-17T05:25:20.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbfb89b000 1 -- start start 2024-09-17T05:25:20.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbd0073200 0x7fbbd00756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbd0073200 0x7fbbd00756c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40058d0 tx=0x7fbbe4005f70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:20.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c379db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c32d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c32e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca4c24e3f0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ca4c18bb20 con 0x55ca4c3799d0 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55ca4c17af50 con 0x55ca4c32de10 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c32d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c32d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43328/0 (socket says 172.21.15.122:43328) 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 -- 172.21.15.122:0/3750826409 learned_addr learned my addr 172.21.15.122:0/3750826409 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c379db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf6400640 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c32e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 -- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 msgr2=0x55ca4c32e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c32e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 -- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 msgr2=0x55ca4c379db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c379db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 -- 172.21.15.122:0/3750826409 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4c142590 con 0x55ca4c37a2f0 2024-09-17T05:25:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf6400640 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c32e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbf5000640 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c32d8d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc004ff0 tx=0x7fbbdc0054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99fc147e8188268 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbe2000640 1 -- 172.21.15.122:0/3750826409 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc017020 con 0x55ca4c37a2f0 2024-09-17T05:25:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbe2000640 1 -- 172.21.15.122:0/3750826409 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbdc003c90 con 0x55ca4c37a2f0 2024-09-17T05:25:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.670+0000 7fbbe2000640 1 -- 172.21.15.122:0/3750826409 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc00fab0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3750826409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 msgr2=0x55ca4c32d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c32d8d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc004ff0 tx=0x7fbbdc0054b0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3750826409 shutdown_connections 2024-09-17T05:25:20.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c32e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c32d8d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3750826409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c379db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3750826409 >> 172.21.15.122:0/3750826409 conn(0x55ca4c217130 msgr2=0x55ca4c35ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3750826409 shutdown_connections 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3750826409 wait complete. 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- start start 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 Processor -- start 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c35a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c35ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c35b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca4c1155d0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ca4c17ed60 con 0x55ca4c3799d0 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55ca4c121500 con 0x55ca4c32de10 2024-09-17T05:25:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 10 client.0 _ll_register_callbacks cb 0x55ca4c125810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c35b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c35b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43330/0 (socket says 172.21.15.122:43330) 2024-09-17T05:25:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 -- 172.21.15.122:0/1727582273 learned_addr learned my addr 172.21.15.122:0/1727582273 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf5000640 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c35ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c35a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 -- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 msgr2=0x55ca4c35a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c35a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 -- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 msgr2=0x55ca4c35ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c35ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 -- 172.21.15.122:0/1727582273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4c119170 con 0x55ca4c37a2f0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbf6400640 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c35b3b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80f8a60 tx=0x7fbbe80f8960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb253c2a8690e71f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbd6a00640 1 -- 172.21.15.122:0/1727582273 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80f8ad0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbd6a00640 1 -- 172.21.15.122:0/1727582273 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbbe8101f20 con 0x55ca4c37a2f0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- 172.21.15.122:0/1727582273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca4c1130f0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbd6a00640 1 -- 172.21.15.122:0/1727582273 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fbbe80fd500 con 0x55ca4c37a2f0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 10 client.4656 fetch_fsmap learned FSMap version 8 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.674+0000 7fbbfb89b000 1 -- 172.21.15.122:0/1727582273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ca4c1ffef0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbd6a00640 1 -- 172.21.15.122:0/1727582273 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ca4c1130f0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbd6a00640 1 -- 172.21.15.122:0/1727582273 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fbbe80b5030 con 0x55ca4c37a2f0 2024-09-17T05:25:20.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T05:25:20.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 10 client.4656 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:25:20.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 20 client.4656 populate_metadata read hostname 'smithi122' 2024-09-17T05:25:20.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] conn(0x55ca4c35c0f0 0x55ca4c38bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 4 client.4656 mds_command: new command op to 4332 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:25:20.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 -- 172.21.15.122:0/1727582273 --> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4c18da70 con 0x55ca4c35c0f0 2024-09-17T05:25:20.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] conn(0x55ca4c35c0f0 0x55ca4c38bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbd6a00640 10 client.4656 ms_handle_connect on v2:172.21.15.173:6835/2742876385 2024-09-17T05:25:20.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] conn(0x55ca4c35c0f0 0x55ca4c38bc90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96e886109bfca06 server_cookie=9398057d74c331b7 in_seq=0 out_seq=0 2024-09-17T05:25:20.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbd6a00640 1 -- 172.21.15.122:0/1727582273 <== mds.0 v2:172.21.15.173:6835/2742876385 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca4c18da70 con 0x55ca4c35c0f0 2024-09-17T05:25:20.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbd6a00640 10 client.4656 handle_command_reply: tid=0 2024-09-17T05:25:20.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 client.4656 shutdown 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 -- 172.21.15.122:0/1727582273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 msgr2=0x55ca4c35b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c35b3b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80f8a60 tx=0x7fbbe80f8960 comp rx=0 tx=0).stop 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 -- 172.21.15.122:0/1727582273 shutdown_connections 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] conn(0x55ca4c35c0f0 0x55ca4c38bc90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c35b3b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c35ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/1727582273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c35a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 -- 172.21.15.122:0/1727582273 >> 172.21.15.122:0/1727582273 conn(0x55ca4c217130 msgr2=0x55ca4c32cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbd6a00640 0 client.4656 ms_handle_reset on v2:172.21.15.173:6835/2742876385 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 -- 172.21.15.122:0/1727582273 shutdown_connections 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 1 -- 172.21.15.122:0/1727582273 wait complete. 2024-09-17T05:25:20.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.678+0000 7fbbfb89b000 20 client.4656 trim_cache size 0 max 16384 2024-09-17T05:25:20.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 Processor -- start 2024-09-17T05:25:20.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- start start 2024-09-17T05:25:20.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c3ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c3ec730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c3ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0203fd0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf02030c0 con 0x55ca4c3799d0 2024-09-17T05:25:20.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf01fb030 con 0x55ca4c32de10 2024-09-17T05:25:20.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c3ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c3ecc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43336/0 (socket says 172.21.15.122:43336) 2024-09-17T05:25:20.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 -- 172.21.15.122:0/3119305113 learned_addr learned my addr 172.21.15.122:0/3119305113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:20.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 -- 172.21.15.122:0/3119305113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 msgr2=0x55ca4c3ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 --2- 172.21.15.122:0/3119305113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c3ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 -- 172.21.15.122:0/3119305113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 msgr2=0x55ca4c3ec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 --2- 172.21.15.122:0/3119305113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c3ec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 -- 172.21.15.122:0/3119305113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4c1d6d90 con 0x55ca4c37a2f0 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbf6400640 1 --2- 172.21.15.122:0/3119305113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c3ecc70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbbe801d0a0 tx=0x7fbbe8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a12f02b1a9ecef server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbe1600640 1 -- 172.21.15.122:0/3119305113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80f4970 con 0x55ca4c37a2f0 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbe1600640 1 -- 172.21.15.122:0/3119305113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe80fd1c0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbe1600640 1 -- 172.21.15.122:0/3119305113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80f8240 con 0x55ca4c37a2f0 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3119305113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 msgr2=0x55ca4c3ecc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3119305113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c3ecc70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbbe801d0a0 tx=0x7fbbe8009bd0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3119305113 shutdown_connections 2024-09-17T05:25:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3119305113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c37a2f0 0x55ca4c3ecc70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3119305113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c3ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3119305113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c32de10 0x55ca4c3ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3119305113 >> 172.21.15.122:0/3119305113 conn(0x55ca4c217130 msgr2=0x55ca4c3781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3119305113 shutdown_connections 2024-09-17T05:25:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.698+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3119305113 wait complete. 2024-09-17T05:25:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 -- start start 2024-09-17T05:25:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 Processor -- start 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c32de10 0x55ca4c3f0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c3f0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c37a2f0 0x55ca4c3f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca4c146ec0 con 0x55ca4c32de10 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01fb680 con 0x55ca4c3799d0 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55ca4c1155d0 con 0x55ca4c37a2f0 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 10 client.0 _ll_register_callbacks cb 0x7fbbdc007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c32de10 0x55ca4c3f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c32de10 0x55ca4c3f0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43340/0 (socket says 172.21.15.122:43340) 2024-09-17T05:25:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 -- 172.21.15.122:0/3947557547 learned_addr learned my addr 172.21.15.122:0/3947557547 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5000640 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c3f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf6400640 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c37a2f0 0x55ca4c3f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 -- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c37a2f0 msgr2=0x55ca4c3f0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c37a2f0 0x55ca4c3f0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 -- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 msgr2=0x55ca4c3f0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c3f0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 -- 172.21.15.122:0/3947557547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4c141a90 con 0x55ca4c32de10 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf6400640 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c37a2f0 0x55ca4c3f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5a00640 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c32de10 0x55ca4c3f0020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40106e0 tx=0x7fbbe40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d551d1f08987a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbd6a00640 1 -- 172.21.15.122:0/3947557547 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4033020 con 0x55ca4c32de10 2024-09-17T05:25:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbd6a00640 1 -- 172.21.15.122:0/3947557547 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbbe402f630 con 0x55ca4c32de10 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3947557547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe80fd1c0 con 0x55ca4c32de10 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbd6a00640 1 -- 172.21.15.122:0/3947557547 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fbbe403d050 con 0x55ca4c32de10 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 10 client.4662 fetch_fsmap learned FSMap version 8 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3947557547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ca4c120050 con 0x55ca4c32de10 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbd6a00640 1 -- 172.21.15.122:0/3947557547 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe80fd1c0 con 0x55ca4c32de10 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbd6a00640 1 -- 172.21.15.122:0/3947557547 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fbbe4040020 con 0x55ca4c32de10 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 10 client.4662 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:25:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 20 client.4662 populate_metadata read hostname 'smithi122' 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] conn(0x55ca4c38f350 0x55ca4c3f9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 4 client.4662 mds_command: new command op to 4332 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3947557547 --> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] -- command(tid 0: {"prefix": "status"}) -- 0x55ca4c189520 con 0x55ca4c38f350 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.702+0000 7fbbf5000640 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] conn(0x55ca4c38f350 0x55ca4c3f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf5000640 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] conn(0x55ca4c38f350 0x55ca4c3f9280 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd69e6b671b1f79 server_cookie=be4c251b79029cd6 in_seq=0 out_seq=0 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbd6a00640 10 client.4662 ms_handle_connect on v2:172.21.15.173:6835/2742876385 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbd6a00640 1 -- 172.21.15.122:0/3947557547 <== mds.0 v2:172.21.15.173:6835/2742876385 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55ca4c189520 con 0x55ca4c38f350 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbd6a00640 10 client.4662 handle_command_reply: tid=0 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 client.4662 shutdown 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3947557547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c32de10 msgr2=0x55ca4c3f0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c32de10 0x55ca4c3f0020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40106e0 tx=0x7fbbe40046d0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3947557547 shutdown_connections 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:6835/2742876385,v1:172.21.15.173:6837/2742876385] conn(0x55ca4c38f350 0x55ca4c3f9280 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ca4c37a2f0 0x55ca4c3f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ca4c3799d0 0x55ca4c3f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbd6a00640 0 client.4662 ms_handle_reset on v2:172.21.15.173:6835/2742876385 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 --2- 172.21.15.122:0/3947557547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca4c32de10 0x55ca4c3f0020 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3947557547 >> 172.21.15.122:0/3947557547 conn(0x55ca4c217130 msgr2=0x55ca4c3789f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3947557547 shutdown_connections 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 1 -- 172.21.15.122:0/3947557547 wait complete. 2024-09-17T05:25:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbfb89b000 20 client.4662 trim_cache size 0 max 16384 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4332, 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 16.578838341000001, 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 8, 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 22, 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 22, 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 21.979808695999999 2024-09-17T05:25:20.714 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:25:20.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 -- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbd0073200 msgr2=0x7fbbd00756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbd0073200 0x7fbbd00756c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40058d0 tx=0x7fbbe4005f70 comp rx=0 tx=0).stop 2024-09-17T05:25:20.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 -- 172.21.15.122:0/1088800240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf015be80 msgr2=0x7fbbf015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:20.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf015be80 0x7fbbf015e680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbbe801d460 tx=0x7fbbe80a5970 comp rx=0 tx=0).stop 2024-09-17T05:25:20.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 -- 172.21.15.122:0/1088800240 shutdown_connections 2024-09-17T05:25:20.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbd0073200 0x7fbbd00756c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbf015be80 0x7fbbf015e680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbf00d28c0 0x7fbbf015ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 --2- 172.21.15.122:0/1088800240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbf00cd8c0 0x7fbbf01f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:20.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 -- 172.21.15.122:0/1088800240 >> 172.21.15.122:0/1088800240 conn(0x7fbbf00d1da0 msgr2=0x7fbbf00d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:20.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 -- 172.21.15.122:0/1088800240 shutdown_connections 2024-09-17T05:25:20.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:20.706+0000 7fbbf8400640 1 -- 172.21.15.122:0/1088800240 wait complete. 2024-09-17T05:25:20.728 INFO:teuthology.run_tasks:Running task kclient... 2024-09-17T05:25:20.752 INFO:tasks.kclient:Mounting kernel clients... 2024-09-17T05:25:20.752 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-17T05:25:20.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:20.753 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:25:20.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:20.762 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-17T05:25:20.824 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-09-17T05:25:20.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:25:20.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:20.826 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:25:20.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:20.875 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-17T05:25:20.944 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-09-17T05:25:20.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:25:20.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:20.947 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2024-09-17T05:25:20.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:20.958 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2024-09-17T05:25:21.028 INFO:teuthology.orchestra.run.smithi173.stderr:Cannot find device "ceph-brx" 2024-09-17T05:25:21.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:25:21.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:21.032 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2024-09-17T05:25:21.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:21.081 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2024-09-17T05:25:21.148 INFO:teuthology.orchestra.run.smithi173.stderr:Cannot find device "ceph-brx" 2024-09-17T05:25:21.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:25:21.151 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-17T05:25:21.151 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:25:21.151 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:21.151 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:25:21.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- 172.21.15.122:0/1510228617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 msgr2=0x7f2804004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 --2- 172.21.15.122:0/1510228617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 0x7f2804004c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f280c006fd0 tx=0x7f280c01cf20 comp rx=0 tx=0).stop 2024-09-17T05:25:21.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- 172.21.15.122:0/1510228617 shutdown_connections 2024-09-17T05:25:21.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 --2- 172.21.15.122:0/1510228617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 0x7f2804004c90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- 172.21.15.122:0/1510228617 >> 172.21.15.122:0/1510228617 conn(0x7f28200d1dc0 msgr2=0x7f28200d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:21.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- 172.21.15.122:0/1510228617 shutdown_connections 2024-09-17T05:25:21.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- 172.21.15.122:0/1510228617 wait complete. 2024-09-17T05:25:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 Processor -- start 2024-09-17T05:25:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- start start 2024-09-17T05:25:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 0x7f282015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2820070980 0x7f2820070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f282015eb60 0x7f282015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28200cea00 con 0x7f2804004810 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f28200ce880 con 0x7f282015eb60 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f2826000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f28200ceb80 con 0x7f2820070980 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 0x7f282015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 0x7f282015c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43634/0 (socket says 172.21.15.122:43634) 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281f400640 1 -- 172.21.15.122:0/4157520907 learned_addr learned my addr 172.21.15.122:0/4157520907 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281fe00640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f282015eb60 0x7f282015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281ea00640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2820070980 0x7f2820070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281f400640 1 -- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2820070980 msgr2=0x7f2820070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281f400640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2820070980 0x7f2820070e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281f400640 1 -- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f282015eb60 msgr2=0x7f282015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281f400640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f282015eb60 0x7f282015efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.294+0000 7f281f400640 1 -- 172.21.15.122:0/4157520907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28201e7860 con 0x7f2804004810 2024-09-17T05:25:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f281f400640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 0x7f282015c8a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2810011ab0 tx=0x7f2810011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2817400640 1 -- 172.21.15.122:0/4157520907 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281001e070 con 0x7f2804004810 2024-09-17T05:25:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2817400640 1 -- 172.21.15.122:0/4157520907 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f281000fc50 con 0x7f2804004810 2024-09-17T05:25:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2817400640 1 -- 172.21.15.122:0/4157520907 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810019410 con 0x7f2804004810 2024-09-17T05:25:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28201e7b40 con 0x7f2804004810 2024-09-17T05:25:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28201e8100 con 0x7f2804004810 2024-09-17T05:25:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28200cd340 con 0x7f2804004810 2024-09-17T05:25:21.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2817400640 1 -- 172.21.15.122:0/4157520907 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2810021070 con 0x7f2804004810 2024-09-17T05:25:21.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2817400640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2800073230 0x7f28000756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:21.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.298+0000 7f2817400640 1 -- 172.21.15.122:0/4157520907 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f28100949b0 con 0x7f2804004810 2024-09-17T05:25:21.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.302+0000 7f281ea00640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2800073230 0x7f28000756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.302+0000 7f281ea00640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2800073230 0x7f28000756f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f280c0ac000 tx=0x7f280c019d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.302+0000 7f2817400640 1 -- 172.21.15.122:0/4157520907 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f281005f410 con 0x7f2804004810 2024-09-17T05:25:21.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f282015f520 con 0x7f2804004810 2024-09-17T05:25:21.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2817400640 1 -- 172.21.15.122:0/4157520907 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f2810063e40 con 0x7f2804004810 2024-09-17T05:25:21.508 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:25:21.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2800073230 msgr2=0x7f28000756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2800073230 0x7f28000756f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f280c0ac000 tx=0x7f280c019d60 comp rx=0 tx=0).stop 2024-09-17T05:25:21.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 msgr2=0x7f282015c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 0x7f282015c8a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2810011ab0 tx=0x7f2810011f70 comp rx=0 tx=0).stop 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 shutdown_connections 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2800073230 0x7f28000756f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f282015eb60 0x7f282015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2820070980 0x7f2820070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 --2- 172.21.15.122:0/4157520907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2804004810 0x7f282015c8a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 >> 172.21.15.122:0/4157520907 conn(0x7f28200d1dc0 msgr2=0x7f2820072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 shutdown_connections 2024-09-17T05:25:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21.506+0000 7f2826000640 1 -- 172.21.15.122:0/4157520907 wait complete. 2024-09-17T05:25:21.529 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:25:21.529 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:25:21.529 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:25:21.530 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:25:21.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:21.530 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:25:21.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:25:21.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:25:21.530 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:25:21.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:21.530 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:25:21.539 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:21.539 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:25:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42446sec preferred_lft 42446sec 2024-09-17T05:25:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:25:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:21.541 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:25:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:25:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:21.541 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T05:25:21.541 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:21.541 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:21.542 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add name ceph-brx type bridge 2024-09-17T05:25:21.542 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr flush dev ceph-brx 2024-09-17T05:25:21.542 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set ceph-brx up 2024-09-17T05:25:21.542 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T05:25:21.542 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:21.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:21.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:21.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:21.693 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:25:21.711 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:25:21.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:21.714 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:25:21.780 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:21.781 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:25:21.782 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:21.782 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:21.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:25:21.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:25:21.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:25:21.782 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:21.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:21.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:21.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:21.916 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:25:21.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:21.925 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-09-17T05:25:21.979 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:21.979 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:21.979 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:25:21.979 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-17T05:25:21.980 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:22.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:22.105 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-17T05:25:22.106 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:22.106 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:22.106 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-17T05:25:22.106 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T05:25:22.106 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-17T05:25:22.106 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-17T05:25:22.106 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-17T05:25:22.106 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:22.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:22.271 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:22.272 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:22.272 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:25:22.272 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T05:25:22.272 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:22.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:22.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:22.355 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:22.407 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:25:22.407 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:22.407 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:22.458 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:25:22.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:22.458 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:22.551 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:22.551 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:22.551 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:23.206 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:23.221 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:23.221 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:23.221 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:23.221 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:23.221 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:23.221 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:25:23.221 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:25:23.221 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:25:23.225 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:25:23.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:23.225 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:23.247 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:23.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:23.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:23.366 INFO:teuthology.orchestra.run.smithi122.stdout:client4532 2024-09-17T05:25:23.367 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:25:23.367 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:23.375 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:23.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:23.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:23.488 INFO:teuthology.orchestra.run.smithi122.stdout:client4532 2024-09-17T05:25:23.488 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:23.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:23.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:23.555 INFO:teuthology.orchestra.run.smithi122.stdout:client4532 2024-09-17T05:25:23.556 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51547930-c98d-41c6-b8ce-f14d7623b707.client4532/status') 2024-09-17T05:25:23.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:23.621 INFO:teuthology.orchestra.run.smithi122.stdout:instance: client.4532 (3)172.21.15.122:0/1784022771 2024-09-17T05:25:23.621 INFO:teuthology.orchestra.run.smithi122.stdout:blocklisted: false 2024-09-17T05:25:23.621 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2024-09-17T05:25:23.622 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2024-09-17T05:25:23.622 INFO:teuthology.orchestra.run.smithi122.stderr:71 bytes copied, 5.0066e-05 s, 1.4 MB/s 2024-09-17T05:25:23.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:23.630 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-17T05:25:23.630 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:25:23.630 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T05:25:23.630 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:25:23.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- 172.21.15.122:0/238694405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9dc0052f0 msgr2=0x7fd9dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 --2- 172.21.15.122:0/238694405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9dc0052f0 0x7fd9dc00b9d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009e00 tx=0x7fd9f001d170 comp rx=0 tx=0).stop 2024-09-17T05:25:23.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- 172.21.15.122:0/238694405 shutdown_connections 2024-09-17T05:25:23.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 --2- 172.21.15.122:0/238694405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9dc0052f0 0x7fd9dc00b9d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- 172.21.15.122:0/238694405 >> 172.21.15.122:0/238694405 conn(0x7fd9f80d1dc0 msgr2=0x7fd9f80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:23.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- 172.21.15.122:0/238694405 shutdown_connections 2024-09-17T05:25:23.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- 172.21.15.122:0/238694405 wait complete. 2024-09-17T05:25:23.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 Processor -- start 2024-09-17T05:25:23.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- start start 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9dc0052f0 0x7fd9f806e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9f806f2e0 0x7fd9f81e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f81551b0 0x7fd9f8155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80d2a60 con 0x7fd9f81551b0 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80d28e0 con 0x7fd9dc0052f0 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9ff200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f80d2be0 con 0x7fd9f806f2e0 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.770+0000 7fd9fd200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f81551b0 0x7fd9f8155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9fd200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f81551b0 0x7fd9f8155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43648/0 (socket says 172.21.15.122:43648) 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9fd200640 1 -- 172.21.15.122:0/611960063 learned_addr learned my addr 172.21.15.122:0/611960063 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:23.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9f7e00640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9dc0052f0 0x7fd9f806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9f7400640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9f806f2e0 0x7fd9f81e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9fd200640 1 -- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9f806f2e0 msgr2=0x7fd9f81e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9fd200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9f806f2e0 0x7fd9f81e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9fd200640 1 -- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9dc0052f0 msgr2=0x7fd9f806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9fd200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9dc0052f0 0x7fd9f806e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9fd200640 1 -- 172.21.15.122:0/611960063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f8155fb0 con 0x7fd9f81551b0 2024-09-17T05:25:23.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9fd200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f81551b0 0x7fd9f8155630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd9f001cfa0 tx=0x7fd9f00046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:23.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9efe00640 1 -- 172.21.15.122:0/611960063 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00bd070 con 0x7fd9f81551b0 2024-09-17T05:25:23.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9efe00640 1 -- 172.21.15.122:0/611960063 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f00a5e60 con 0x7fd9f81551b0 2024-09-17T05:25:23.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9efe00640 1 -- 172.21.15.122:0/611960063 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0009a00 con 0x7fd9f81551b0 2024-09-17T05:25:23.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f81f7d20 con 0x7fd9f81551b0 2024-09-17T05:25:23.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81f82e0 con 0x7fd9f81551b0 2024-09-17T05:25:23.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f80ce910 con 0x7fd9f81551b0 2024-09-17T05:25:23.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.774+0000 7fd9efe00640 1 -- 172.21.15.122:0/611960063 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd9f00b5040 con 0x7fd9f81551b0 2024-09-17T05:25:23.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.778+0000 7fd9efe00640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:23.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.778+0000 7fd9efe00640 1 -- 172.21.15.122:0/611960063 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd9f012e610 con 0x7fd9f81551b0 2024-09-17T05:25:23.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.778+0000 7fd9f7e00640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:23.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.778+0000 7fd9f7e00640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8006fd0 tx=0x7fd9e8006b80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:23.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.782+0000 7fd9efe00640 1 -- 172.21.15.122:0/611960063 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd9f0016e80 con 0x7fd9f81551b0 2024-09-17T05:25:23.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.982+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd9f80b7b10 con 0x7fd9f81551b0 2024-09-17T05:25:23.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.982+0000 7fd9efe00640 1 -- 172.21.15.122:0/611960063 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fd9f0009280 con 0x7fd9f81551b0 2024-09-17T05:25:23.986 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:25:23.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 msgr2=0x7fd9d80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8006fd0 tx=0x7fd9e8006b80 comp rx=0 tx=0).stop 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f81551b0 msgr2=0x7fd9f8155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f81551b0 0x7fd9f8155630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd9f001cfa0 tx=0x7fd9f00046a0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 shutdown_connections 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd9d8073200 0x7fd9d80756c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f81551b0 0x7fd9f8155630 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9f806f2e0 0x7fd9f81e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 --2- 172.21.15.122:0/611960063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9dc0052f0 0x7fd9f806e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 >> 172.21.15.122:0/611960063 conn(0x7fd9f80d1dc0 msgr2=0x7fd9f80b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 shutdown_connections 2024-09-17T05:25:23.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:23.986+0000 7fd9ff200640 1 -- 172.21.15.122:0/611960063 wait complete. 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:25:24.008 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-17T05:25:24.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:24.009 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:25:24.017 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:25:24.018 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42443sec preferred_lft 42443sec 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:25:24.019 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:24.020 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:25:24.020 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.020 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-09-17T05:25:24.020 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.020 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:25:24.020 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:25:24.020 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-09-17T05:25:24.020 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:24.022 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:25:24.080 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:25:24.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:24.082 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:24.134 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:24.135 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:25:24.135 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:24.135 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:24.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:25:24.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:25:24.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:25:24.135 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:24.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:24.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:24.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:24.269 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:25:24.279 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:25:24.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:24.279 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-09-17T05:25:24.334 INFO:teuthology.orchestra.run.smithi122.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-17T05:25:24.335 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:24.335 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:24.335 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:25:24.335 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-17T05:25:24.335 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:24.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:24.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:24.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:24.465 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T05:25:24.485 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:24.485 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.486 INFO:teuthology.orchestra.run.smithi122.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:25:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether b2:a7:7b:ac:e2:24 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:25:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:25:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::b0a7:7bff:feac:e224/64 scope link 2024-09-17T05:25:24.486 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:24.488 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T05:25:24.550 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.551 INFO:teuthology.orchestra.run.smithi122.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:25:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether b2:a7:7b:ac:e2:24 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:25:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:25:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::b0a7:7bff:feac:e224/64 scope link 2024-09-17T05:25:24.551 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:24.552 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-17T05:25:24.552 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:24.553 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:24.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-17T05:25:24.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T05:25:24.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-17T05:25:24.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-17T05:25:24.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-17T05:25:24.553 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:24.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:24.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:24.732 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:24.732 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:24.732 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.1 up 2024-09-17T05:25:24.732 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T05:25:24.732 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:24.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:24.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:24.815 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:24.815 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-17T05:25:24.871 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-17T05:25:24.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:24.871 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-17T05:25:24.926 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:25:24.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:24.927 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:25:25.015 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:25:25.015 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:25:25.015 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:25.030 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:25:25.030 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:25:25.030 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:25.030 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:25.030 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:25.041 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:25:25.041 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:25:25.041 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:25.041 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:25.041 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:25.041 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:25:25.041 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:25:25.041 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:25:25.045 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:25:25.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:25.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-17T05:25:25.064 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:25.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:25.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:25.163 INFO:teuthology.orchestra.run.smithi122.stdout:client4720 2024-09-17T05:25:25.163 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:25:25.163 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:25.219 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:25.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:25.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:25.319 INFO:teuthology.orchestra.run.smithi122.stdout:client4720 2024-09-17T05:25:25.319 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:25.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:25.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:25.375 INFO:teuthology.orchestra.run.smithi122.stdout:client4720 2024-09-17T05:25:25.375 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51547930-c98d-41c6-b8ce-f14d7623b707.client4720/status') 2024-09-17T05:25:25.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:25.433 INFO:teuthology.orchestra.run.smithi122.stdout:instance: client.4720 (3)172.21.15.122:0/3480733347 2024-09-17T05:25:25.433 INFO:teuthology.orchestra.run.smithi122.stdout:blocklisted: false 2024-09-17T05:25:25.433 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2024-09-17T05:25:25.434 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2024-09-17T05:25:25.434 INFO:teuthology.orchestra.run.smithi122.stderr:71 bytes copied, 5.0319e-05 s, 1.4 MB/s 2024-09-17T05:25:25.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:25.441 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-17T05:25:25.442 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:25:25.442 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:25:25.442 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:25:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 -- 172.21.15.122:0/3095955002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5340d28e0 msgr2=0x7fe5340cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 --2- 172.21.15.122:0/3095955002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5340d28e0 0x7fe5340cc9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5340745d0 tx=0x7fe52c01cd80 comp rx=0 tx=0).stop 2024-09-17T05:25:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 -- 172.21.15.122:0/3095955002 shutdown_connections 2024-09-17T05:25:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 --2- 172.21.15.122:0/3095955002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5340d28e0 0x7fe5340cc9d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 --2- 172.21.15.122:0/3095955002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5340cdb30 0x7fe5340cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 --2- 172.21.15.122:0/3095955002 >> v2:172.21.15.173:3300/0 conn(0x7fe53415bea0 0x7fe5340ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 -- 172.21.15.122:0/3095955002 >> 172.21.15.122:0/3095955002 conn(0x7fe5340d1dc0 msgr2=0x7fe5340d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:25.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 -- 172.21.15.122:0/3095955002 shutdown_connections 2024-09-17T05:25:25.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.618+0000 7fe53bc00640 1 -- 172.21.15.122:0/3095955002 wait complete. 2024-09-17T05:25:25.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 Processor -- start 2024-09-17T05:25:25.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 -- start start 2024-09-17T05:25:25.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5340cdb30 0x7fe5340b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:25.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5340d28e0 0x7fe534153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:25.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe53415bea0 0x7fe534155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:25.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5340af080 con 0x7fe53415bea0 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe5340aef00 con 0x7fe5340d28e0 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe5340af200 con 0x7fe5340cdb30 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe53415bea0 0x7fe534155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5340cdb30 0x7fe5340b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe53415bea0 0x7fe534155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43666/0 (socket says 172.21.15.122:43666) 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 -- 172.21.15.122:0/973787005 learned_addr learned my addr 172.21.15.122:0/973787005 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe52be00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5340d28e0 0x7fe534153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 -- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5340cdb30 msgr2=0x7fe5340b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5340cdb30 0x7fe5340b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 -- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5340d28e0 msgr2=0x7fe534153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5340d28e0 0x7fe534153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 -- 172.21.15.122:0/973787005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5340b16a0 con 0x7fe53415bea0 2024-09-17T05:25:25.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe539c00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe53415bea0 0x7fe534155330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe52c01b430 tx=0x7fe52c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:25.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe531600640 1 -- 172.21.15.122:0/973787005 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0bb070 con 0x7fe53415bea0 2024-09-17T05:25:25.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe531600640 1 -- 172.21.15.122:0/973787005 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe52c004d40 con 0x7fe53415bea0 2024-09-17T05:25:25.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe531600640 1 -- 172.21.15.122:0/973787005 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0b2040 con 0x7fe53415bea0 2024-09-17T05:25:25.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5340b1980 con 0x7fe53415bea0 2024-09-17T05:25:25.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.622+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5340b2220 con 0x7fe53415bea0 2024-09-17T05:25:25.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.626+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5340cc9d0 con 0x7fe53415bea0 2024-09-17T05:25:25.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.626+0000 7fe531600640 1 -- 172.21.15.122:0/973787005 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe52c0a5ca0 con 0x7fe53415bea0 2024-09-17T05:25:25.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.626+0000 7fe531600640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe510073200 0x7fe5100756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:25.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.626+0000 7fe531600640 1 -- 172.21.15.122:0/973787005 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe52c12e090 con 0x7fe53415bea0 2024-09-17T05:25:25.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.626+0000 7fe539200640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe510073200 0x7fe5100756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:25.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.626+0000 7fe539200640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe510073200 0x7fe5100756c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe534154da0 tx=0x7fe52400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:25.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.630+0000 7fe531600640 1 -- 172.21.15.122:0/973787005 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe52c016970 con 0x7fe53415bea0 2024-09-17T05:25:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.834+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe534158dc0 con 0x7fe53415bea0 2024-09-17T05:25:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.834+0000 7fe531600640 1 -- 172.21.15.122:0/973787005 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe52c0f8a60 con 0x7fe53415bea0 2024-09-17T05:25:25.837 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:25:25.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.834+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe510073200 msgr2=0x7fe5100756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:25.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.834+0000 7fe53bc00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe510073200 0x7fe5100756c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe534154da0 tx=0x7fe52400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.834+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe53415bea0 msgr2=0x7fe534155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.834+0000 7fe53bc00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe53415bea0 0x7fe534155330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe52c01b430 tx=0x7fe52c003f60 comp rx=0 tx=0).stop 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.834+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 shutdown_connections 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.838+0000 7fe53bc00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe510073200 0x7fe5100756c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.838+0000 7fe53bc00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe53415bea0 0x7fe534155330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.838+0000 7fe53bc00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5340d28e0 0x7fe534153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.838+0000 7fe53bc00640 1 --2- 172.21.15.122:0/973787005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5340cdb30 0x7fe5340b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.838+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 >> 172.21.15.122:0/973787005 conn(0x7fe5340d1dc0 msgr2=0x7fe5340b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.838+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 shutdown_connections 2024-09-17T05:25:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:25.838+0000 7fe53bc00640 1 -- 172.21.15.122:0/973787005 wait complete. 2024-09-17T05:25:25.859 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:25:25.859 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:25:25.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2024-09-17T05:25:25.859 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:25:25.859 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:25:25.859 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:25:25.860 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:25:25.860 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:25:25.860 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:25:25.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:25.860 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:25.870 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42441sec preferred_lft 42441sec 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:7d:51:c5:6b brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:25.871 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:25:25.872 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:25.872 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T05:25:25.873 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:25.873 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:25.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link add name ceph-brx type bridge 2024-09-17T05:25:25.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip addr flush dev ceph-brx 2024-09-17T05:25:25.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set ceph-brx up 2024-09-17T05:25:25.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T05:25:25.873 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:25.968 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:26.030 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:26.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:26.037 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:25:26.056 INFO:teuthology.orchestra.run.smithi173.stdout:1 2024-09-17T05:25:26.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:26.059 DEBUG:teuthology.orchestra.run.smithi173:> route 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:26.135 INFO:teuthology.orchestra.run.smithi173.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:26.136 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:25:26.136 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:26.136 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:26.136 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:25:26.136 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:25:26.136 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:25:26.136 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:26.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:26.265 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:26.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:26.272 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2024-09-17T05:25:26.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:26.281 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list-id 2024-09-17T05:25:26.332 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:26.332 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:26.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:25:26.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-17T05:25:26.333 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:26.425 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:26.456 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:26.463 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-17T05:25:26.463 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:26.463 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:26.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-17T05:25:26.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T05:25:26.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-17T05:25:26.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-17T05:25:26.463 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-17T05:25:26.464 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:26.550 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:26.664 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:26.671 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:26.671 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:26.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2024-09-17T05:25:26.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T05:25:26.671 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:26.717 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:26.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:26.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:26.754 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:25:26.763 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:25:26.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:26.763 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:25:26.815 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:25:26.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:26.815 DEBUG:teuthology.orchestra.run.smithi173:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:25:26.907 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:25:26.908 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:25:26.908 INFO:teuthology.orchestra.run.smithi173.stdout:invalid new device string format 2024-09-17T05:25:27.595 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:25:27.595 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:25:27.595 INFO:teuthology.orchestra.run.smithi173.stdout:invalid new device string format 2024-09-17T05:25:27.595 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:27.595 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:27.607 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:25:27.607 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:25:27.607 INFO:teuthology.orchestra.run.smithi173.stdout:invalid new device string format 2024-09-17T05:25:27.608 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:27.608 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:27.608 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:25:27.608 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:25:27.608 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:25:27.611 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:25:27.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:27.611 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:25:27.630 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:27.719 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:27.737 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:27.743 INFO:teuthology.orchestra.run.smithi173.stdout:client4544 2024-09-17T05:25:27.743 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:25:27.744 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:27.796 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:27.885 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:27.888 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:27.894 INFO:teuthology.orchestra.run.smithi173.stdout:client4544 2024-09-17T05:25:27.894 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:27.941 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:27.944 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:27.950 INFO:teuthology.orchestra.run.smithi173.stdout:client4544 2024-09-17T05:25:27.951 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51547930-c98d-41c6-b8ce-f14d7623b707.client4544/status') 2024-09-17T05:25:27.996 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:28.009 INFO:teuthology.orchestra.run.smithi173.stdout:instance: client.4544 (3)192.168.0.1:0/728850314 2024-09-17T05:25:28.009 INFO:teuthology.orchestra.run.smithi173.stdout:blocklisted: false 2024-09-17T05:25:28.009 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2024-09-17T05:25:28.009 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2024-09-17T05:25:28.009 INFO:teuthology.orchestra.run.smithi173.stderr:68 bytes copied, 5.9873e-05 s, 1.1 MB/s 2024-09-17T05:25:28.010 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:28.017 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-17T05:25:28.017 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:25:28.017 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T05:25:28.018 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 -- 172.21.15.122:0/1892547714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 msgr2=0x7f3324074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 --2- 172.21.15.122:0/1892547714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 0x7f3324074340 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3324073b40 tx=0x7f331001c800 comp rx=0 tx=0).stop 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 -- 172.21.15.122:0/1892547714 shutdown_connections 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 --2- 172.21.15.122:0/1892547714 >> v2:172.21.15.173:3300/0 conn(0x7f3324074880 0x7f332415bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 --2- 172.21.15.122:0/1892547714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 0x7f3324074340 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 -- 172.21.15.122:0/1892547714 >> 172.21.15.122:0/1892547714 conn(0x7f3324073490 msgr2=0x7f33240d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 -- 172.21.15.122:0/1892547714 shutdown_connections 2024-09-17T05:25:28.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 -- 172.21.15.122:0/1892547714 wait complete. 2024-09-17T05:25:28.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 Processor -- start 2024-09-17T05:25:28.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.170+0000 7f3323200640 1 -- start start 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3324074880 0x7f33240c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33240cccd0 0x7f332415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 0x7f332415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3324168e60 con 0x7f33241602b0 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3324168ce0 con 0x7f3324074880 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3324168fe0 con 0x7f33240cccd0 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 0x7f332415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 0x7f332415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43678/0 (socket says 172.21.15.122:43678) 2024-09-17T05:25:28.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 -- 172.21.15.122:0/4165318619 learned_addr learned my addr 172.21.15.122:0/4165318619 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3321400640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33240cccd0 0x7f332415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3321e00640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3324074880 0x7f33240c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 -- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33240cccd0 msgr2=0x7f332415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33240cccd0 0x7f332415e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 -- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3324074880 msgr2=0x7f33240c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3324074880 0x7f33240c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 -- 172.21.15.122:0/4165318619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33240b30f0 con 0x7f33241602b0 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3321400640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33240cccd0 0x7f332415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3322800640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 0x7f332415ebe0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f331c00cce0 tx=0x7f331c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f331a000640 1 -- 172.21.15.122:0/4165318619 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c003980 con 0x7f33241602b0 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f331a000640 1 -- 172.21.15.122:0/4165318619 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f331c003c10 con 0x7f33241602b0 2024-09-17T05:25:28.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33240c89c0 con 0x7f33241602b0 2024-09-17T05:25:28.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f331a000640 1 -- 172.21.15.122:0/4165318619 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c004030 con 0x7f33241602b0 2024-09-17T05:25:28.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33240c8f80 con 0x7f33241602b0 2024-09-17T05:25:28.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332415bef0 con 0x7f33241602b0 2024-09-17T05:25:28.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.174+0000 7f331a000640 1 -- 172.21.15.122:0/4165318619 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f331c0041d0 con 0x7f33241602b0 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.178+0000 7f331a000640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3304073200 0x7f33040756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.178+0000 7f331a000640 1 -- 172.21.15.122:0/4165318619 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f331c094950 con 0x7f33241602b0 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.178+0000 7f3321e00640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3304073200 0x7f33040756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:28.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.178+0000 7f3321e00640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3304073200 0x7f33040756c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3314007fd0 tx=0x7f3314007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:28.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.182+0000 7f331a000640 1 -- 172.21.15.122:0/4165318619 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f331c014070 con 0x7f33241602b0 2024-09-17T05:25:28.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.382+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3324068fd0 con 0x7f33241602b0 2024-09-17T05:25:28.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.382+0000 7f331a000640 1 -- 172.21.15.122:0/4165318619 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f331c05f440 con 0x7f33241602b0 2024-09-17T05:25:28.386 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3304073200 msgr2=0x7f33040756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3304073200 0x7f33040756c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3314007fd0 tx=0x7f3314007ea0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 msgr2=0x7f332415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 0x7f332415ebe0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f331c00cce0 tx=0x7f331c00a620 comp rx=0 tx=0).stop 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 shutdown_connections 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3304073200 0x7f33040756c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33241602b0 0x7f332415ebe0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33240cccd0 0x7f332415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 --2- 172.21.15.122:0/4165318619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3324074880 0x7f33240c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 >> 172.21.15.122:0/4165318619 conn(0x7f3324073490 msgr2=0x7f33240af010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 shutdown_connections 2024-09-17T05:25:28.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:28.386+0000 7f3323200640 1 -- 172.21.15.122:0/4165318619 wait complete. 2024-09-17T05:25:28.409 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:25:28.409 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:25:28.409 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2024-09-17T05:25:28.409 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-17T05:25:28.409 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T05:25:28.409 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:25:28.409 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:25:28.410 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:25:28.410 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-17T05:25:28.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:28.410 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2024-09-17T05:25:28.419 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42438sec preferred_lft 42438sec 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:7d:51:c5:6b brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:28.420 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:25:28.421 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:28.422 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:25:28.485 INFO:teuthology.orchestra.run.smithi173.stdout:1 2024-09-17T05:25:28.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:28.488 DEBUG:teuthology.orchestra.run.smithi173:> route 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:28.548 INFO:teuthology.orchestra.run.smithi173.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:28.549 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:25:28.549 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:28.549 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:28.549 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:25:28.549 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:25:28.549 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:25:28.549 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:28.641 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:28.681 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:28.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:28.687 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2024-09-17T05:25:28.696 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:25:28.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:28.697 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list-id 2024-09-17T05:25:28.748 INFO:teuthology.orchestra.run.smithi173.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-17T05:25:28.748 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:28.748 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:28.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:25:28.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-17T05:25:28.749 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:28.837 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:28.867 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:28.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:28.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether f2:bc:8c:97:c7:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::f0bc:8cff:fe97:c7ee/64 scope link 2024-09-17T05:25:28.894 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:28.896 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T05:25:28.963 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:28.963 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether f2:bc:8c:97:c7:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::f0bc:8cff:fe97:c7ee/64 scope link 2024-09-17T05:25:28.964 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:28.965 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-17T05:25:28.966 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:28.966 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:28.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-17T05:25:28.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T05:25:28.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-17T05:25:28.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-17T05:25:28.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-17T05:25:28.966 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:29.054 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:29.138 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:29.144 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:29.144 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2024-09-17T05:25:29.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2024-09-17T05:25:29.145 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T05:25:29.145 DEBUG:teuthology.orchestra.run.smithi173:> ') 2024-09-17T05:25:29.192 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:29.222 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:29.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:29.229 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-17T05:25:29.280 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-17T05:25:29.280 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:29.280 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-17T05:25:29.331 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:25:29.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:29.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:25:29.419 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:25:29.419 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:25:29.419 INFO:teuthology.orchestra.run.smithi173.stdout:invalid new device string format 2024-09-17T05:25:29.434 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:25:29.434 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:25:29.435 INFO:teuthology.orchestra.run.smithi173.stdout:invalid new device string format 2024-09-17T05:25:29.435 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:29.435 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:29.444 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:25:29.444 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:25:29.444 INFO:teuthology.orchestra.run.smithi173.stdout:invalid new device string format 2024-09-17T05:25:29.444 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:29.444 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:29.445 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:25:29.445 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:25:29.445 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:25:29.448 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:25:29.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:29.449 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-17T05:25:29.467 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:29.558 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:29.561 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:29.568 INFO:teuthology.orchestra.run.smithi173.stdout:client4732 2024-09-17T05:25:29.568 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:25:29.568 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:29.577 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:29.665 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:29.668 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:29.674 INFO:teuthology.orchestra.run.smithi173.stdout:client4732 2024-09-17T05:25:29.674 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:25:29.765 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:25:29.768 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:29.775 INFO:teuthology.orchestra.run.smithi173.stdout:client4732 2024-09-17T05:25:29.775 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/51547930-c98d-41c6-b8ce-f14d7623b707.client4732/status') 2024-09-17T05:25:29.821 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:29.833 INFO:teuthology.orchestra.run.smithi173.stdout:instance: client.4732 (3)192.168.0.2:0/3198467616 2024-09-17T05:25:29.833 INFO:teuthology.orchestra.run.smithi173.stdout:blocklisted: false 2024-09-17T05:25:29.833 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2024-09-17T05:25:29.833 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2024-09-17T05:25:29.833 INFO:teuthology.orchestra.run.smithi173.stderr:69 bytes copied, 4.9062e-05 s, 1.4 MB/s 2024-09-17T05:25:29.834 INFO:teuthology.orchestra.run.smithi173.stderr:2024-09-17T05:25:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:29.841 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-17T05:25:29.984 INFO:tasks.cephfs_test_runner:Starting test: test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Initiate cloning where clone subvolume and source subvolume are located 2024-09-17T05:25:29.985 INFO:tasks.cephfs_test_runner:test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:25:29.985 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:30.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.130+0000 7f8bf4600640 1 -- 172.21.15.122:0/2922210133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec0cecf0 msgr2=0x7f8bec0cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.130+0000 7f8bf4600640 1 --2- 172.21.15.122:0/2922210133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec0cecf0 0x7f8bec0cc940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0d0f60 tx=0x7f8bd801c7a0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.130+0000 7f8bf4600640 1 -- 172.21.15.122:0/2922210133 shutdown_connections 2024-09-17T05:25:30.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.130+0000 7f8bf4600640 1 --2- 172.21.15.122:0/2922210133 >> v2:172.21.15.173:3300/0 conn(0x7f8bec15c130 0x7f8bec0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.130+0000 7f8bf4600640 1 --2- 172.21.15.122:0/2922210133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec0cecf0 0x7f8bec0cc940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.130+0000 7f8bf4600640 1 -- 172.21.15.122:0/2922210133 >> 172.21.15.122:0/2922210133 conn(0x7f8bec0d1ea0 msgr2=0x7f8bec0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.130+0000 7f8bf4600640 1 -- 172.21.15.122:0/2922210133 shutdown_connections 2024-09-17T05:25:30.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 -- 172.21.15.122:0/2922210133 wait complete. 2024-09-17T05:25:30.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 Processor -- start 2024-09-17T05:25:30.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 -- start start 2024-09-17T05:25:30.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bec0cecf0 0x7f8bec0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bec15bd50 0x7f8bec1f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec15c130 0x7f8bec0c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec169570 con 0x7f8bec15c130 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec1693f0 con 0x7f8bec0cecf0 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8bec1696f0 con 0x7f8bec15bd50 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec15c130 0x7f8bec0c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf1c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bec0cecf0 0x7f8bec0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec15c130 0x7f8bec0c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43696/0 (socket says 172.21.15.122:43696) 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 -- 172.21.15.122:0/1670352828 learned_addr learned my addr 172.21.15.122:0/1670352828 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf1200640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bec15bd50 0x7f8bec1f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 -- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bec15bd50 msgr2=0x7f8bec1f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bec15bd50 0x7f8bec1f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 -- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bec0cecf0 msgr2=0x7f8bec0ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bec0cecf0 0x7f8bec0ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 -- 172.21.15.122:0/1670352828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec071720 con 0x7f8bec15c130 2024-09-17T05:25:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf2600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec15c130 0x7f8bec0c72a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8be400e940 tx=0x7f8be400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8be9600640 1 -- 172.21.15.122:0/1670352828 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4004d30 con 0x7f8bec15c130 2024-09-17T05:25:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8be9600640 1 -- 172.21.15.122:0/1670352828 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8be400cb20 con 0x7f8bec15c130 2024-09-17T05:25:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8be9600640 1 -- 172.21.15.122:0/1670352828 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4005020 con 0x7f8bec15c130 2024-09-17T05:25:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec153070 con 0x7f8bec15c130 2024-09-17T05:25:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.134+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec153630 con 0x7f8bec15c130 2024-09-17T05:25:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.138+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bec0ce590 con 0x7f8bec15c130 2024-09-17T05:25:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.138+0000 7f8be9600640 1 -- 172.21.15.122:0/1670352828 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8be40051c0 con 0x7f8bec15c130 2024-09-17T05:25:30.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.138+0000 7f8be9600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bcc073200 0x7f8bcc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.138+0000 7f8be9600640 1 -- 172.21.15.122:0/1670352828 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8be4094880 con 0x7f8bec15c130 2024-09-17T05:25:30.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.138+0000 7f8bf1c00640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bcc073200 0x7f8bcc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.138+0000 7f8bf1c00640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bcc073200 0x7f8bcc0756c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8be0007fd0 tx=0x7f8be0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.142+0000 7f8be9600640 1 -- 172.21.15.122:0/1670352828 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8be4016030 con 0x7f8bec15c130 2024-09-17T05:25:30.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.310+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bec076360 con 0x7f8bec15c130 2024-09-17T05:25:30.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.310+0000 7f8be9600640 1 -- 172.21.15.122:0/1670352828 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7f8be405f2d0 con 0x7f8bec15c130 2024-09-17T05:25:30.317 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:30.317 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":22,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:05.140074+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bcc073200 msgr2=0x7f8bcc0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bcc073200 0x7f8bcc0756c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8be0007fd0 tx=0x7f8be0009040 comp rx=0 tx=0).stop 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec15c130 msgr2=0x7f8bec0c72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec15c130 0x7f8bec0c72a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8be400e940 tx=0x7f8be400ee00 comp rx=0 tx=0).stop 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 shutdown_connections 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bcc073200 0x7f8bcc0756c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bec15c130 0x7f8bec0c72a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bec15bd50 0x7f8bec1f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 --2- 172.21.15.122:0/1670352828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bec0cecf0 0x7f8bec0ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 >> 172.21.15.122:0/1670352828 conn(0x7f8bec0d1ea0 msgr2=0x7f8bec0d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 shutdown_connections 2024-09-17T05:25:30.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.314+0000 7f8bf4600640 1 -- 172.21.15.122:0/1670352828 wait complete. 2024-09-17T05:25:30.338 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-17T05:25:02.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-17T05:25:02.482131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:25:30.338 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:25:30.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- 172.21.15.122:0/4046600307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f015bf50 msgr2=0x7f31f00cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 --2- 172.21.15.122:0/4046600307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f015bf50 0x7f31f00cd300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f31f00d2d30 tx=0x7f31e401c800 comp rx=0 tx=0).stop 2024-09-17T05:25:30.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- 172.21.15.122:0/4046600307 shutdown_connections 2024-09-17T05:25:30.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 --2- 172.21.15.122:0/4046600307 >> v2:172.21.15.173:3300/0 conn(0x7f31f00ced70 0x7f31f00b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 --2- 172.21.15.122:0/4046600307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f015bf50 0x7f31f00cd300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- 172.21.15.122:0/4046600307 >> 172.21.15.122:0/4046600307 conn(0x7f31f00d1c50 msgr2=0x7f31f00d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- 172.21.15.122:0/4046600307 shutdown_connections 2024-09-17T05:25:30.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- 172.21.15.122:0/4046600307 wait complete. 2024-09-17T05:25:30.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 Processor -- start 2024-09-17T05:25:30.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- start start 2024-09-17T05:25:30.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31f00ced70 0x7f31f0144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f00d2770 0x7f31f009b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31f015bf50 0x7f31f0147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0168af0 con 0x7f31f00d2770 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0168970 con 0x7f31f015bf50 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f7000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f31f0168c70 con 0x7f31f00ced70 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f00d2770 0x7f31f009b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f00d2770 0x7f31f009b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43714/0 (socket says 172.21.15.122:43714) 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 -- 172.21.15.122:0/3598259645 learned_addr learned my addr 172.21.15.122:0/3598259645 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f6600640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31f015bf50 0x7f31f0147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5c00640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31f00ced70 0x7f31f0144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 -- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31f00ced70 msgr2=0x7f31f0144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31f00ced70 0x7f31f0144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 -- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31f015bf50 msgr2=0x7f31f0147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31f015bf50 0x7f31f0147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 -- 172.21.15.122:0/3598259645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f0145100 con 0x7f31f00d2770 2024-09-17T05:25:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.482+0000 7f31f5200640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f00d2770 0x7f31f009b920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c500 tx=0x7f31dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31ed600640 1 -- 172.21.15.122:0/3598259645 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc017070 con 0x7f31f00d2770 2024-09-17T05:25:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31ed600640 1 -- 172.21.15.122:0/3598259645 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31dc003600 con 0x7f31f00d2770 2024-09-17T05:25:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f009be60 con 0x7f31f00d2770 2024-09-17T05:25:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31ed600640 1 -- 172.21.15.122:0/3598259645 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc003a90 con 0x7f31f00d2770 2024-09-17T05:25:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f00c6e30 con 0x7f31f00d2770 2024-09-17T05:25:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f00ce440 con 0x7f31f00d2770 2024-09-17T05:25:30.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31ed600640 1 -- 172.21.15.122:0/3598259645 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31dc01a070 con 0x7f31f00d2770 2024-09-17T05:25:30.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31ed600640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31d0073230 0x7f31d00756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31ed600640 1 -- 172.21.15.122:0/3598259645 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f31dc0936f0 con 0x7f31f00d2770 2024-09-17T05:25:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.486+0000 7f31f5c00640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31d0073230 0x7f31d00756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.490+0000 7f31f5c00640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31d0073230 0x7f31d00756f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31e40af000 tx=0x7f31e401b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.490+0000 7f31ed600640 1 -- 172.21.15.122:0/3598259645 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f31dc05e850 con 0x7f31f00d2770 2024-09-17T05:25:30.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.658+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31f00b4300 con 0x7f31f00d2770 2024-09-17T05:25:30.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.658+0000 7f31ed600640 1 -- 172.21.15.122:0/3598259645 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f31dc05e150 con 0x7f31f00d2770 2024-09-17T05:25:30.663 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:25:30.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31d0073230 msgr2=0x7f31d00756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31d0073230 0x7f31d00756f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31e40af000 tx=0x7f31e401b040 comp rx=0 tx=0).stop 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f00d2770 msgr2=0x7f31f009b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f00d2770 0x7f31f009b920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c500 tx=0x7f31dc00c9c0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 shutdown_connections 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31d0073230 0x7f31d00756f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31f015bf50 0x7f31f0147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31f00d2770 0x7f31f009b920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 --2- 172.21.15.122:0/3598259645 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31f00ced70 0x7f31f0144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 >> 172.21.15.122:0/3598259645 conn(0x7f31f00d1c50 msgr2=0x7f31f00cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 shutdown_connections 2024-09-17T05:25:30.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.662+0000 7f31f7000640 1 -- 172.21.15.122:0/3598259645 wait complete. 2024-09-17T05:25:30.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-17T05:25:30.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 -- 172.21.15.122:0/1790623665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93780ce7f0 msgr2=0x7f93780cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 --2- 172.21.15.122:0/1790623665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93780ce7f0 0x7f93780cfd30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9370009bc0 tx=0x7f937001ca80 comp rx=0 tx=0).stop 2024-09-17T05:25:30.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 -- 172.21.15.122:0/1790623665 shutdown_connections 2024-09-17T05:25:30.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 --2- 172.21.15.122:0/1790623665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93780ce7f0 0x7f93780cfd30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 -- 172.21.15.122:0/1790623665 >> 172.21.15.122:0/1790623665 conn(0x7f93780d1e30 msgr2=0x7f93780d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:30.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 -- 172.21.15.122:0/1790623665 shutdown_connections 2024-09-17T05:25:30.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 -- 172.21.15.122:0/1790623665 wait complete. 2024-09-17T05:25:30.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 Processor -- start 2024-09-17T05:25:30.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.826+0000 7f937f400640 1 -- start start 2024-09-17T05:25:30.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93780b27a0 0x7f93781f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93780ce7f0 0x7f9378149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937815f640 0x7f937814a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f937816a7b0 con 0x7f937815f640 2024-09-17T05:25:30.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93780b10a0 con 0x7f93780b27a0 2024-09-17T05:25:30.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f937816a930 con 0x7f93780ce7f0 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937815f640 0x7f937814a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93780b27a0 0x7f93781f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937815f640 0x7f937814a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43730/0 (socket says 172.21.15.122:43730) 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 -- 172.21.15.122:0/3889448506 learned_addr learned my addr 172.21.15.122:0/3889448506 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f9377e00640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93780ce7f0 0x7f9378149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 -- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93780ce7f0 msgr2=0x7f9378149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93780ce7f0 0x7f9378149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 -- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93780b27a0 msgr2=0x7f93781f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93780b27a0 0x7f93781f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:30.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 -- 172.21.15.122:0/3889448506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937806deb0 con 0x7f937815f640 2024-09-17T05:25:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937d400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937815f640 0x7f937814a3f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9370009bc0 tx=0x7f9370009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f9374c00640 1 -- 172.21.15.122:0/3889448506 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700ba020 con 0x7f937815f640 2024-09-17T05:25:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f9374c00640 1 -- 172.21.15.122:0/3889448506 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9370004330 con 0x7f937815f640 2024-09-17T05:25:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f9374c00640 1 -- 172.21.15.122:0/3889448506 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700b2040 con 0x7f937815f640 2024-09-17T05:25:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9378167500 con 0x7f937815f640 2024-09-17T05:25:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937806fc70 con 0x7f937815f640 2024-09-17T05:25:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93780cfd30 con 0x7f937815f640 2024-09-17T05:25:30.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.830+0000 7f9374c00640 1 -- 172.21.15.122:0/3889448506 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f93700bd020 con 0x7f937815f640 2024-09-17T05:25:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.834+0000 7f9374c00640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9358073200 0x7f93580756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.834+0000 7f9374c00640 1 -- 172.21.15.122:0/3889448506 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f937012e9d0 con 0x7f937815f640 2024-09-17T05:25:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.834+0000 7f937ca00640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9358073200 0x7f93580756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.834+0000 7f937ca00640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9358073200 0x7f93580756c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f936c0010a0 tx=0x7f936c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:30.838+0000 7f9374c00640 1 -- 172.21.15.122:0/3889448506 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f93700c4030 con 0x7f937815f640 2024-09-17T05:25:31.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f937806e080 con 0x7f937815f640 2024-09-17T05:25:31.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f9374c00640 1 -- 172.21.15.122:0/3889448506 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f9370016790 con 0x7f937815f640 2024-09-17T05:25:31.009 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-09-17T05:25:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9358073200 msgr2=0x7f93580756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9358073200 0x7f93580756c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f936c0010a0 tx=0x7f936c009040 comp rx=0 tx=0).stop 2024-09-17T05:25:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937815f640 msgr2=0x7f937814a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937815f640 0x7f937814a3f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9370009bc0 tx=0x7f9370009280 comp rx=0 tx=0).stop 2024-09-17T05:25:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 shutdown_connections 2024-09-17T05:25:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9358073200 0x7f93580756c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937815f640 0x7f937814a3f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93780ce7f0 0x7f9378149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 --2- 172.21.15.122:0/3889448506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93780b27a0 0x7f93781f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 >> 172.21.15.122:0/3889448506 conn(0x7f93780d1e30 msgr2=0x7f937814f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:31.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 shutdown_connections 2024-09-17T05:25:31.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.006+0000 7f937f400640 1 -- 172.21.15.122:0/3889448506 wait complete. 2024-09-17T05:25:31.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-17T05:25:31.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- 172.21.15.122:0/1563250130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 msgr2=0x7fcd300affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 --2- 172.21.15.122:0/1563250130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 0x7fcd300affc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009c80 tx=0x7fcd2801cda0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- 172.21.15.122:0/1563250130 shutdown_connections 2024-09-17T05:25:31.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 --2- 172.21.15.122:0/1563250130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 0x7fcd300affc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- 172.21.15.122:0/1563250130 >> 172.21.15.122:0/1563250130 conn(0x7fcd300d1dd0 msgr2=0x7fcd300d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:31.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- 172.21.15.122:0/1563250130 shutdown_connections 2024-09-17T05:25:31.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- 172.21.15.122:0/1563250130 wait complete. 2024-09-17T05:25:31.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 Processor -- start 2024-09-17T05:25:31.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- start start 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 0x7fcd301fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd300b1650 0x7fcd30164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd301605d0 0x7fcd30147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3016a6d0 con 0x7fcd300afb10 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3016a550 con 0x7fcd301605d0 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd37e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3016a850 con 0x7fcd300b1650 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 0x7fcd301fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 0x7fcd301fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43740/0 (socket says 172.21.15.122:43740) 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35400640 1 -- 172.21.15.122:0/1753725393 learned_addr learned my addr 172.21.15.122:0/1753725393 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35e00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd301605d0 0x7fcd30147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd34a00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd300b1650 0x7fcd30164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35400640 1 -- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd300b1650 msgr2=0x7fcd30164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35400640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd300b1650 0x7fcd30164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35400640 1 -- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd301605d0 msgr2=0x7fcd30147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35400640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd301605d0 0x7fcd30147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.174+0000 7fcd35400640 1 -- 172.21.15.122:0/1753725393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd300b64c0 con 0x7fcd300afb10 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd35e00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd301605d0 0x7fcd30147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd35400640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 0x7fcd301fc530 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c3f0 tx=0x7fcd2400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd2d600640 1 -- 172.21.15.122:0/1753725393 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24017070 con 0x7fcd300afb10 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd2d600640 1 -- 172.21.15.122:0/1753725393 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd240053c0 con 0x7fcd300afb10 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd2d600640 1 -- 172.21.15.122:0/1753725393 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24003860 con 0x7fcd300afb10 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd301fca70 con 0x7fcd300afb10 2024-09-17T05:25:31.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd300b6730 con 0x7fcd300afb10 2024-09-17T05:25:31.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd300affc0 con 0x7fcd300afb10 2024-09-17T05:25:31.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd2d600640 1 -- 172.21.15.122:0/1753725393 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd2401a070 con 0x7fcd300afb10 2024-09-17T05:25:31.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.178+0000 7fcd2d600640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd10073200 0x7fcd100756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.182+0000 7fcd2d600640 1 -- 172.21.15.122:0/1753725393 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcd24093590 con 0x7fcd300afb10 2024-09-17T05:25:31.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.182+0000 7fcd34a00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd10073200 0x7fcd100756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.182+0000 7fcd34a00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd10073200 0x7fcd100756c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcd3006d1e0 tx=0x7fcd1c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:31.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.186+0000 7fcd2d600640 1 -- 172.21.15.122:0/1753725393 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcd301fca70 con 0x7fcd300afb10 2024-09-17T05:25:31.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.350+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcd300d1080 con 0x7fcd300afb10 2024-09-17T05:25:31.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.350+0000 7fcd2d600640 1 -- 172.21.15.122:0/1753725393 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fcd2405e630 con 0x7fcd300afb10 2024-09-17T05:25:31.354 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-09-17T05:25:31.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd10073200 msgr2=0x7fcd100756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd10073200 0x7fcd100756c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcd3006d1e0 tx=0x7fcd1c009040 comp rx=0 tx=0).stop 2024-09-17T05:25:31.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 msgr2=0x7fcd301fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 0x7fcd301fc530 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c3f0 tx=0x7fcd2400c8b0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 shutdown_connections 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd10073200 0x7fcd100756c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd301605d0 0x7fcd30147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd300b1650 0x7fcd30164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 --2- 172.21.15.122:0/1753725393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd300afb10 0x7fcd301fc530 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 >> 172.21.15.122:0/1753725393 conn(0x7fcd300d1dd0 msgr2=0x7fcd300d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 shutdown_connections 2024-09-17T05:25:31.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.354+0000 7fcd37e00640 1 -- 172.21.15.122:0/1753725393 wait complete. 2024-09-17T05:25:31.376 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-17T05:25:31.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.514+0000 7f9fb9c00640 1 -- 172.21.15.122:0/417638716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 msgr2=0x7f9f98004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.514+0000 7f9fb9c00640 1 --2- 172.21.15.122:0/417638716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 0x7f9f98004c90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001890 tx=0x7f9f9c01f030 comp rx=0 tx=0).stop 2024-09-17T05:25:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.514+0000 7f9fb3400640 1 -- 172.21.15.122:0/417638716 reap_dead start 2024-09-17T05:25:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.514+0000 7f9fb9c00640 1 -- 172.21.15.122:0/417638716 shutdown_connections 2024-09-17T05:25:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.514+0000 7f9fb9c00640 1 -- 172.21.15.122:0/417638716 >> 172.21.15.122:0/417638716 conn(0x7f9fb40d1dc0 msgr2=0x7f9fb40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 -- 172.21.15.122:0/417638716 shutdown_connections 2024-09-17T05:25:31.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 -- 172.21.15.122:0/417638716 wait complete. 2024-09-17T05:25:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 Processor -- start 2024-09-17T05:25:31.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 -- start start 2024-09-17T05:25:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 0x7f9fb4157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c7d60 0x7f9fb40b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40b3c60 0x7f9fb41564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb415c450 con 0x7f9f98004810 2024-09-17T05:25:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb40d2af0 con 0x7f9fb40c7d60 2024-09-17T05:25:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb9c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb40af0c0 con 0x7f9fb40b3c60 2024-09-17T05:25:31.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 0x7f9fb4157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 0x7f9fb4157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49242/0 (socket says 172.21.15.122:49242) 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 -- 172.21.15.122:0/1688509209 learned_addr learned my addr 172.21.15.122:0/1688509209 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9faa000640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c7d60 0x7f9fb40b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb3400640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40b3c60 0x7f9fb41564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 -- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40b3c60 msgr2=0x7f9fb41564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40b3c60 0x7f9fb41564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 -- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c7d60 msgr2=0x7f9fb40b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c7d60 0x7f9fb40b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 -- 172.21.15.122:0/1688509209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb4145e90 con 0x7f9f98004810 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb3400640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40b3c60 0x7f9fb41564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:31.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.518+0000 7f9fb2a00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 0x7f9fb4157ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4001230 tx=0x7f9fa40066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:31.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fab400640 1 -- 172.21.15.122:0/1688509209 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4015070 con 0x7f9f98004810 2024-09-17T05:25:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fab400640 1 -- 172.21.15.122:0/1688509209 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fa4011900 con 0x7f9f98004810 2024-09-17T05:25:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb4146170 con 0x7f9f98004810 2024-09-17T05:25:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fab400640 1 -- 172.21.15.122:0/1688509209 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4011d50 con 0x7f9f98004810 2024-09-17T05:25:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb4146730 con 0x7f9f98004810 2024-09-17T05:25:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb40ce670 con 0x7f9f98004810 2024-09-17T05:25:31.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fab400640 1 -- 172.21.15.122:0/1688509209 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9fa400f430 con 0x7f9f98004810 2024-09-17T05:25:31.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fab400640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f90073200 0x7f9f900756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:31.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9fab400640 1 -- 172.21.15.122:0/1688509209 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9fa4094f50 con 0x7f9f98004810 2024-09-17T05:25:31.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.522+0000 7f9faa000640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f90073200 0x7f9f900756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:31.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.526+0000 7f9faa000640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f90073200 0x7f9f900756c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001860 tx=0x7f9f9c01be70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:31.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.526+0000 7f9fab400640 1 -- 172.21.15.122:0/1688509209 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9fa405f9a0 con 0x7f9f98004810 2024-09-17T05:25:31.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:25:31.597 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:25:31.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:31.754+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f9fb415a7f0 con 0x7f9f98004810 2024-09-17T05:25:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fab400640 1 -- 172.21.15.122:0/1688509209 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v51) ==== 187+0+0 (secure 0 0 0) 0x7f9fa40643d0 con 0x7f9f98004810 2024-09-17T05:25:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f90073200 msgr2=0x7f9f900756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f90073200 0x7f9f900756c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c001860 tx=0x7f9f9c01be70 comp rx=0 tx=0).stop 2024-09-17T05:25:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 msgr2=0x7f9fb4157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 0x7f9fb4157ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4001230 tx=0x7f9fa40066c0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 shutdown_connections 2024-09-17T05:25:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f90073200 0x7f9f900756c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40b3c60 0x7f9fb41564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c7d60 0x7f9fb40b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 --2- 172.21.15.122:0/1688509209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f98004810 0x7f9fb4157ae0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 >> 172.21.15.122:0/1688509209 conn(0x7f9fb40d1dc0 msgr2=0x7f9fb40c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:32.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 shutdown_connections 2024-09-17T05:25:32.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.458+0000 7f9fb9c00640 1 -- 172.21.15.122:0/1688509209 wait complete. 2024-09-17T05:25:32.482 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- 172.21.15.122:0/1982396368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 msgr2=0x7f32900b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 --2- 172.21.15.122:0/1982396368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 0x7f32900b5880 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3290074060 tx=0x7f328801cd80 comp rx=0 tx=0).stop 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- 172.21.15.122:0/1982396368 shutdown_connections 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 --2- 172.21.15.122:0/1982396368 >> v2:172.21.15.173:3300/0 conn(0x7f32900d28c0 0x7f32900cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 --2- 172.21.15.122:0/1982396368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 0x7f32900b5880 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 --2- 172.21.15.122:0/1982396368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f329015be80 0x7f32900ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- 172.21.15.122:0/1982396368 >> 172.21.15.122:0/1982396368 conn(0x7f32900d1da0 msgr2=0x7f32900d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- 172.21.15.122:0/1982396368 shutdown_connections 2024-09-17T05:25:32.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- 172.21.15.122:0/1982396368 wait complete. 2024-09-17T05:25:32.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 Processor -- start 2024-09-17T05:25:32.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- start start 2024-09-17T05:25:32.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 0x7f3290158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32900d28c0 0x7f3290146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f329015be80 0x7f329015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32900cd000 con 0x7f32900b53f0 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f32900cce80 con 0x7f32900d28c0 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3297c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f32900cd180 con 0x7f329015be80 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 0x7f3290158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 0x7f3290158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49266/0 (socket says 172.21.15.122:49266) 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295200640 1 -- 172.21.15.122:0/3804142302 learned_addr learned my addr 172.21.15.122:0/3804142302 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f328fe00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32900d28c0 0x7f3290146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295c00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f329015be80 0x7f329015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295200640 1 -- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f329015be80 msgr2=0x7f329015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295200640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f329015be80 0x7f329015e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295200640 1 -- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32900d28c0 msgr2=0x7f3290146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295200640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32900d28c0 0x7f3290146fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f3295200640 1 -- 172.21.15.122:0/3804142302 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3290147520 con 0x7f32900b53f0 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.622+0000 7f328fe00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32900d28c0 0x7f3290146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f3295200640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 0x7f3290158550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f328400c9f0 tx=0x7f328400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f328cc00640 1 -- 172.21.15.122:0/3804142302 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284007bb0 con 0x7f32900b53f0 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f328cc00640 1 -- 172.21.15.122:0/3804142302 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3284007e40 con 0x7f32900b53f0 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f328cc00640 1 -- 172.21.15.122:0/3804142302 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328400f690 con 0x7f32900b53f0 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3290156790 con 0x7f32900b53f0 2024-09-17T05:25:32.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3290156d50 con 0x7f32900b53f0 2024-09-17T05:25:32.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32900b5880 con 0x7f32900b53f0 2024-09-17T05:25:32.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f328cc00640 1 -- 172.21.15.122:0/3804142302 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f328400f830 con 0x7f32900b53f0 2024-09-17T05:25:32.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f328cc00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3270073200 0x7f32700756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:32.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.626+0000 7f328cc00640 1 -- 172.21.15.122:0/3804142302 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f32840949a0 con 0x7f32900b53f0 2024-09-17T05:25:32.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.630+0000 7f328fe00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3270073200 0x7f32700756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:32.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.630+0000 7f328fe00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3270073200 0x7f32700756c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f327c0058d0 tx=0x7f327c005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:32.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.630+0000 7f328cc00640 1 -- 172.21.15.122:0/3804142302 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3284014030 con 0x7f32900b53f0 2024-09-17T05:25:32.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.802+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f32900cc0b0 con 0x7f32900b53f0 2024-09-17T05:25:32.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.802+0000 7f328cc00640 1 -- 172.21.15.122:0/3804142302 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f328405f9b0 con 0x7f32900b53f0 2024-09-17T05:25:32.806 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:25:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:25:32.808 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:25:32.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3270073200 msgr2=0x7f32700756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3270073200 0x7f32700756c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f327c0058d0 tx=0x7f327c005860 comp rx=0 tx=0).stop 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 msgr2=0x7f3290158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 0x7f3290158550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f328400c9f0 tx=0x7f328400ceb0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 shutdown_connections 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3270073200 0x7f32700756c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f329015be80 0x7f329015e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32900d28c0 0x7f3290146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 --2- 172.21.15.122:0/3804142302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32900b53f0 0x7f3290158550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 >> 172.21.15.122:0/3804142302 conn(0x7f32900d1da0 msgr2=0x7f3290159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 shutdown_connections 2024-09-17T05:25:32.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.806+0000 7f3297c00640 1 -- 172.21.15.122:0/3804142302 wait complete. 2024-09-17T05:25:32.829 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:25:32.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.970+0000 7f36a1600640 1 -- 172.21.15.122:0/3460696857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c0afb10 msgr2=0x7f369c0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.970+0000 7f36a1600640 1 --2- 172.21.15.122:0/3460696857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c0afb10 0x7f369c0affc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3694009c80 tx=0x7f369401cda0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.970+0000 7f36a1600640 1 -- 172.21.15.122:0/3460696857 shutdown_connections 2024-09-17T05:25:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.970+0000 7f36a1600640 1 --2- 172.21.15.122:0/3460696857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c0afb10 0x7f369c0affc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.970+0000 7f36a1600640 1 --2- 172.21.15.122:0/3460696857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f369c1605d0 0x7f369c0d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.970+0000 7f36a1600640 1 -- 172.21.15.122:0/3460696857 >> 172.21.15.122:0/3460696857 conn(0x7f369c0d1dd0 msgr2=0x7f369c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.970+0000 7f36a1600640 1 -- 172.21.15.122:0/3460696857 shutdown_connections 2024-09-17T05:25:32.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.970+0000 7f36a1600640 1 -- 172.21.15.122:0/3460696857 wait complete. 2024-09-17T05:25:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 Processor -- start 2024-09-17T05:25:32.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 -- start start 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f369c0afb10 0x7f369c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f369c0b1650 0x7f369c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c1605d0 0x7f369c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f369c169540 con 0x7f369c1605d0 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f369c0b1430 con 0x7f369c0afb10 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f369c1696c0 con 0x7f369c0b1650 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c1605d0 0x7f369c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c1605d0 0x7f369c0ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49278/0 (socket says 172.21.15.122:49278) 2024-09-17T05:25:32.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f369c0afb10 0x7f369c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 -- 172.21.15.122:0/1257480454 learned_addr learned my addr 172.21.15.122:0/1257480454 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369a000640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f369c0b1650 0x7f369c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 -- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f369c0b1650 msgr2=0x7f369c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f369c0b1650 0x7f369c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 -- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f369c0afb10 msgr2=0x7f369c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f369c0afb10 0x7f369c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 -- 172.21.15.122:0/1257480454 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c1fb3c0 con 0x7f369c1605d0 2024-09-17T05:25:32.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f369b400640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c1605d0 0x7f369c0ce2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f369401b450 tx=0x7f3694009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:32.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f3692a00640 1 -- 172.21.15.122:0/1257480454 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36940bb070 con 0x7f369c1605d0 2024-09-17T05:25:32.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f3692a00640 1 -- 172.21.15.122:0/1257480454 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3694004d10 con 0x7f369c1605d0 2024-09-17T05:25:32.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f3692a00640 1 -- 172.21.15.122:0/1257480454 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36940b2040 con 0x7f369c1605d0 2024-09-17T05:25:32.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c1fb6a0 con 0x7f369c1605d0 2024-09-17T05:25:32.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.974+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c15a890 con 0x7f369c1605d0 2024-09-17T05:25:32.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.978+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369c1fc080 con 0x7f369c1605d0 2024-09-17T05:25:32.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.978+0000 7f3692a00640 1 -- 172.21.15.122:0/1257480454 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36940a5bf0 con 0x7f369c1605d0 2024-09-17T05:25:32.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.978+0000 7f3692a00640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f367c073200 0x7f367c0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:32.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.978+0000 7f3692a00640 1 -- 172.21.15.122:0/1257480454 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f369412de90 con 0x7f369c1605d0 2024-09-17T05:25:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.978+0000 7f369aa00640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f367c073200 0x7f367c0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:32.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.978+0000 7f369aa00640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f367c073200 0x7f367c0756c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f368c00bb50 tx=0x7f368c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:32.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:32.982+0000 7f3692a00640 1 -- 172.21.15.122:0/1257480454 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f369c1fc080 con 0x7f369c1605d0 2024-09-17T05:25:33.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.150+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f369c0affc0 con 0x7f369c1605d0 2024-09-17T05:25:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f3692a00640 1 -- 172.21.15.122:0/1257480454 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f369412f020 con 0x7f369c1605d0 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f367c073200 msgr2=0x7f367c0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f367c073200 0x7f367c0756c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f368c00bb50 tx=0x7f368c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c1605d0 msgr2=0x7f369c0ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c1605d0 0x7f369c0ce2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f369401b450 tx=0x7f3694009390 comp rx=0 tx=0).stop 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 shutdown_connections 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f367c073200 0x7f367c0756c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f369c1605d0 0x7f369c0ce2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f369c0b1650 0x7f369c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 --2- 172.21.15.122:0/1257480454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f369c0afb10 0x7f369c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 >> 172.21.15.122:0/1257480454 conn(0x7f369c0d1dd0 msgr2=0x7f369c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 shutdown_connections 2024-09-17T05:25:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.154+0000 7f36a1600640 1 -- 172.21.15.122:0/1257480454 wait complete. 2024-09-17T05:25:33.181 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:25:33.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:33.269 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:25:33.367 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:25:33.367 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:25:33.367 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:33.376 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:25:33.377 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:33.426 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:25:33.427 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:33.480 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:25:33.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 -- 172.21.15.122:0/1242047652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e6c0cd100 msgr2=0x7f7e6c0b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:33.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 --2- 172.21.15.122:0/1242047652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e6c0cd100 0x7f7e6c0b42d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009900 tx=0x7f7e5401c850 comp rx=0 tx=0).stop 2024-09-17T05:25:33.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 -- 172.21.15.122:0/1242047652 shutdown_connections 2024-09-17T05:25:33.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 --2- 172.21.15.122:0/1242047652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e6c0cd100 0x7f7e6c0b42d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 --2- 172.21.15.122:0/1242047652 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e6c15dae0 0x7f7e6c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 -- 172.21.15.122:0/1242047652 >> 172.21.15.122:0/1242047652 conn(0x7f7e6c0b18f0 msgr2=0x7f7e6c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:33.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 -- 172.21.15.122:0/1242047652 shutdown_connections 2024-09-17T05:25:33.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 -- 172.21.15.122:0/1242047652 wait complete. 2024-09-17T05:25:33.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.662+0000 7f7e72e00640 1 Processor -- start 2024-09-17T05:25:33.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 -- start start 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e6c0b4810 0x7f7e6c14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e6c0cd100 0x7f7e6c1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e6c15dae0 0x7f7e6c14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c1674b0 con 0x7f7e6c0cd100 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c167330 con 0x7f7e6c0b4810 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6c167630 con 0x7f7e6c15dae0 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e6c0cd100 0x7f7e6c1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e6c0cd100 0x7f7e6c1583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49284/0 (socket says 172.21.15.122:49284) 2024-09-17T05:25:33.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 -- 172.21.15.122:0/3779214453 learned_addr learned my addr 172.21.15.122:0/3779214453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:33.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e6be00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e6c0b4810 0x7f7e6c14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:33.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e70e00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e6c15dae0 0x7f7e6c14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:33.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 -- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e6c15dae0 msgr2=0x7f7e6c14fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:33.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e6c15dae0 0x7f7e6c14fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 -- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e6c0b4810 msgr2=0x7f7e6c14cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:33.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e6c0b4810 0x7f7e6c14cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 -- 172.21.15.122:0/3779214453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c14d4d0 con 0x7f7e6c0cd100 2024-09-17T05:25:33.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e63400640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e6c0cd100 0x7f7e6c1583a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e54019460 tx=0x7f7e54005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:33.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e68c00640 1 -- 172.21.15.122:0/3779214453 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e540b2040 con 0x7f7e6c0cd100 2024-09-17T05:25:33.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e68c00640 1 -- 172.21.15.122:0/3779214453 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e54004330 con 0x7f7e6c0cd100 2024-09-17T05:25:33.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e68c00640 1 -- 172.21.15.122:0/3779214453 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54003860 con 0x7f7e6c0cd100 2024-09-17T05:25:33.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c1588e0 con 0x7f7e6c0cd100 2024-09-17T05:25:33.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c14f0b0 con 0x7f7e6c0cd100 2024-09-17T05:25:33.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.666+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6c0b42d0 con 0x7f7e6c0cd100 2024-09-17T05:25:33.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.670+0000 7f7e68c00640 1 -- 172.21.15.122:0/3779214453 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7e54003a00 con 0x7f7e6c0cd100 2024-09-17T05:25:33.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.670+0000 7f7e68c00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7e48073200 0x7f7e480756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:33.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.670+0000 7f7e68c00640 1 -- 172.21.15.122:0/3779214453 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7e5412dcd0 con 0x7f7e6c0cd100 2024-09-17T05:25:33.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.670+0000 7f7e6be00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7e48073200 0x7f7e480756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:33.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.670+0000 7f7e6be00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7e48073200 0x7f7e480756c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c1f91b0 tx=0x7f7e5c010040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:33.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.674+0000 7f7e68c00640 1 -- 172.21.15.122:0/3779214453 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7e540164d0 con 0x7f7e6c0cd100 2024-09-17T05:25:33.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.850+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7e6c0ae200 con 0x7f7e6c0cd100 2024-09-17T05:25:33.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.850+0000 7f7e68c00640 1 -- 172.21.15.122:0/3779214453 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f7e540f86a0 con 0x7f7e6c0cd100 2024-09-17T05:25:33.855 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:25:33.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7e48073200 msgr2=0x7f7e480756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:33.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7e48073200 0x7f7e480756c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c1f91b0 tx=0x7f7e5c010040 comp rx=0 tx=0).stop 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e6c0cd100 msgr2=0x7f7e6c1583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e6c0cd100 0x7f7e6c1583a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e54019460 tx=0x7f7e54005700 comp rx=0 tx=0).stop 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 shutdown_connections 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7e48073200 0x7f7e480756c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e6c15dae0 0x7f7e6c14fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e6c0cd100 0x7f7e6c1583a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 --2- 172.21.15.122:0/3779214453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e6c0b4810 0x7f7e6c14cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 >> 172.21.15.122:0/3779214453 conn(0x7f7e6c0b18f0 msgr2=0x7f7e6c0af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 shutdown_connections 2024-09-17T05:25:33.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:33.854+0000 7f7e72e00640 1 -- 172.21.15.122:0/3779214453 wait complete. 2024-09-17T05:25:33.877 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:25:33.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:33.877 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:25:33.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:25:33.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:33.909 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:25:33.979 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:25:33.979 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:33.989 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:25:33.989 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:34.041 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:25:34.041 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:34.097 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:25:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.234+0000 7fe278200640 1 -- 172.21.15.122:0/2933804443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 msgr2=0x7fe2700ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.234+0000 7fe278200640 1 --2- 172.21.15.122:0/2933804443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 0x7fe2700ae9b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009900 tx=0x7fe25c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.234+0000 7fe278200640 1 -- 172.21.15.122:0/2933804443 shutdown_connections 2024-09-17T05:25:34.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.234+0000 7fe278200640 1 --2- 172.21.15.122:0/2933804443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 0x7fe2700ae9b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.234+0000 7fe278200640 1 -- 172.21.15.122:0/2933804443 >> 172.21.15.122:0/2933804443 conn(0x7fe2700d1dc0 msgr2=0x7fe2700d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:34.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.234+0000 7fe278200640 1 -- 172.21.15.122:0/2933804443 shutdown_connections 2024-09-17T05:25:34.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.234+0000 7fe278200640 1 -- 172.21.15.122:0/2933804443 wait complete. 2024-09-17T05:25:34.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 Processor -- start 2024-09-17T05:25:34.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 -- start start 2024-09-17T05:25:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700cdb30 0x7fe2700c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 0x7fe270149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe27015bea0 0x7fe2700b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe27016a8a0 con 0x7fe2700ced80 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe27016a720 con 0x7fe2700cdb30 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe27016aa20 con 0x7fe27015bea0 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 0x7fe270149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 0x7fe270149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49312/0 (socket says 172.21.15.122:49312) 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 -- 172.21.15.122:0/1786192052 learned_addr learned my addr 172.21.15.122:0/1786192052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe275800640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700cdb30 0x7fe2700c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe276200640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe27015bea0 0x7fe2700b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 -- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe27015bea0 msgr2=0x7fe2700b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe27015bea0 0x7fe2700b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 -- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700cdb30 msgr2=0x7fe2700c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700cdb30 0x7fe2700c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 -- 172.21.15.122:0/1786192052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2701f91d0 con 0x7fe2700ced80 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe276200640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe27015bea0 0x7fe2700b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe274e00640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 0x7fe270149470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009850 tx=0x7fe25c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe26d600640 1 -- 172.21.15.122:0/1786192052 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0b2040 con 0x7fe2700ced80 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe26d600640 1 -- 172.21.15.122:0/1786192052 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe25c005030 con 0x7fe2700ced80 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe26d600640 1 -- 172.21.15.122:0/1786192052 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c005460 con 0x7fe2700ced80 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.238+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2701f9490 con 0x7fe2700ced80 2024-09-17T05:25:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.242+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2701f9a50 con 0x7fe2700ced80 2024-09-17T05:25:34.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.242+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2700ce670 con 0x7fe2700ced80 2024-09-17T05:25:34.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.242+0000 7fe26d600640 1 -- 172.21.15.122:0/1786192052 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe25c0046d0 con 0x7fe2700ced80 2024-09-17T05:25:34.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.242+0000 7fe26d600640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe250073200 0x7fe2500756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:34.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.242+0000 7fe26d600640 1 -- 172.21.15.122:0/1786192052 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe25c12e410 con 0x7fe2700ced80 2024-09-17T05:25:34.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.242+0000 7fe275800640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe250073200 0x7fe2500756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:34.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.242+0000 7fe275800640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe250073200 0x7fe2500756c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe264007fd0 tx=0x7fe264009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:34.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.246+0000 7fe26d600640 1 -- 172.21.15.122:0/1786192052 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe25c0164d0 con 0x7fe2700ced80 2024-09-17T05:25:34.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe2701fa0a0 con 0x7fe2700ced80 2024-09-17T05:25:34.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe26d600640 1 -- 172.21.15.122:0/1786192052 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fe25c0f9420 con 0x7fe2700ced80 2024-09-17T05:25:34.428 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:25:34.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe250073200 msgr2=0x7fe2500756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:34.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe250073200 0x7fe2500756c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe264007fd0 tx=0x7fe264009040 comp rx=0 tx=0).stop 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 msgr2=0x7fe270149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 0x7fe270149470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009850 tx=0x7fe25c0040c0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 shutdown_connections 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe250073200 0x7fe2500756c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe27015bea0 0x7fe2700b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2700ced80 0x7fe270149470 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 --2- 172.21.15.122:0/1786192052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700cdb30 0x7fe2700c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 >> 172.21.15.122:0/1786192052 conn(0x7fe2700d1dc0 msgr2=0x7fe2700d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 shutdown_connections 2024-09-17T05:25:34.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.426+0000 7fe278200640 1 -- 172.21.15.122:0/1786192052 wait complete. 2024-09-17T05:25:34.450 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:25:34.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:34.450 DEBUG:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-17T05:25:34.486 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:25:34.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:34.486 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:25:34.543 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:25:34.543 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:34.553 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:25:34.553 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:34.607 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:25:34.607 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:34.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:25:34.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6888c00640 1 -- 172.21.15.122:0/1646280588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 msgr2=0x7f6868004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6888c00640 1 --2- 172.21.15.122:0/1646280588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 0x7f6868004c90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f686c009b60 tx=0x7f686c01d280 comp rx=0 tx=0).stop 2024-09-17T05:25:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6882800640 1 -- 172.21.15.122:0/1646280588 reap_dead start 2024-09-17T05:25:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6888c00640 1 -- 172.21.15.122:0/1646280588 shutdown_connections 2024-09-17T05:25:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6888c00640 1 -- 172.21.15.122:0/1646280588 >> 172.21.15.122:0/1646280588 conn(0x7f68840d1e30 msgr2=0x7f68840d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6888c00640 1 -- 172.21.15.122:0/1646280588 shutdown_connections 2024-09-17T05:25:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6888c00640 1 -- 172.21.15.122:0/1646280588 wait complete. 2024-09-17T05:25:34.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6888c00640 1 Processor -- start 2024-09-17T05:25:34.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.842+0000 7f6888c00640 1 -- start start 2024-09-17T05:25:34.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 0x7f688406f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:34.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68841513b0 0x7f68840b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:34.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f688415cc40 0x7f688415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:34.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68840b08c0 con 0x7f6868004810 2024-09-17T05:25:34.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68840b2320 con 0x7f688415cc40 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f68840b0a40 con 0x7f68841513b0 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 0x7f688406f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 0x7f688406f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49338/0 (socket says 172.21.15.122:49338) 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 -- 172.21.15.122:0/3356765054 learned_addr learned my addr 172.21.15.122:0/3356765054 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6879400640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68841513b0 0x7f68840b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6882800640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f688415cc40 0x7f688415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 -- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68841513b0 msgr2=0x7f68840b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68841513b0 0x7f68840b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 -- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f688415cc40 msgr2=0x7f688415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f688415cc40 0x7f688415d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 -- 172.21.15.122:0/3356765054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688406e0d0 con 0x7f6868004810 2024-09-17T05:25:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6881e00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 0x7f688406f4b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f687400e590 tx=0x7f687400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:34.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f687aa00640 1 -- 172.21.15.122:0/3356765054 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874015070 con 0x7f6868004810 2024-09-17T05:25:34.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f687aa00640 1 -- 172.21.15.122:0/3356765054 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6874010980 con 0x7f6868004810 2024-09-17T05:25:34.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f687aa00640 1 -- 172.21.15.122:0/3356765054 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874010dd0 con 0x7f6868004810 2024-09-17T05:25:34.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688406e3b0 con 0x7f6868004810 2024-09-17T05:25:34.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688406e970 con 0x7f6868004810 2024-09-17T05:25:34.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.846+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68840cecb0 con 0x7f6868004810 2024-09-17T05:25:34.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.850+0000 7f687aa00640 1 -- 172.21.15.122:0/3356765054 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6874018070 con 0x7f6868004810 2024-09-17T05:25:34.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.850+0000 7f687aa00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6860073200 0x7f68600756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:34.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.850+0000 7f687aa00640 1 -- 172.21.15.122:0/3356765054 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6874094660 con 0x7f6868004810 2024-09-17T05:25:34.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.850+0000 7f6879400640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6860073200 0x7f68600756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:34.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.850+0000 7f6879400640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6860073200 0x7f68600756c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f686c0ae000 tx=0x7f686c01a020 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:34.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:34.854+0000 7f687aa00640 1 -- 172.21.15.122:0/3356765054 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f687405f700 con 0x7f6868004810 2024-09-17T05:25:35.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.034+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f68840ce9d0 con 0x7f6868004810 2024-09-17T05:25:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.034+0000 7f687aa00640 1 -- 172.21.15.122:0/3356765054 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f687405f000 con 0x7f6868004810 2024-09-17T05:25:35.038 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:25:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6860073200 msgr2=0x7f68600756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6860073200 0x7f68600756c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f686c0ae000 tx=0x7f686c01a020 comp rx=0 tx=0).stop 2024-09-17T05:25:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 msgr2=0x7f688406f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 0x7f688406f4b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f687400e590 tx=0x7f687400ea50 comp rx=0 tx=0).stop 2024-09-17T05:25:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 shutdown_connections 2024-09-17T05:25:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6860073200 0x7f68600756c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f688415cc40 0x7f688415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68841513b0 0x7f68840b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 --2- 172.21.15.122:0/3356765054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6868004810 0x7f688406f4b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 >> 172.21.15.122:0/3356765054 conn(0x7f68840d1e30 msgr2=0x7f6884072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 shutdown_connections 2024-09-17T05:25:35.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.038+0000 7f6888c00640 1 -- 172.21.15.122:0/3356765054 wait complete. 2024-09-17T05:25:35.060 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-17T05:25:35.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:35.060 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-17T05:25:35.084 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:25:35.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:35.084 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-17T05:25:35.140 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:25:35.140 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:35.150 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:25:35.150 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:35.204 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:25:35.205 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:25:35.256 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:25:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- 172.21.15.122:0/1077909593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 msgr2=0x7f96bc0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 --2- 172.21.15.122:0/1077909593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 0x7f96bc0aeef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0cc3a0 tx=0x7f96b401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- 172.21.15.122:0/1077909593 shutdown_connections 2024-09-17T05:25:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 --2- 172.21.15.122:0/1077909593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 0x7f96bc0aeef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 --2- 172.21.15.122:0/1077909593 >> v2:172.21.15.173:3300/0 conn(0x7f96bc0ced80 0x7f96bc0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- 172.21.15.122:0/1077909593 >> 172.21.15.122:0/1077909593 conn(0x7f96bc0d1dc0 msgr2=0x7f96bc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- 172.21.15.122:0/1077909593 shutdown_connections 2024-09-17T05:25:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- 172.21.15.122:0/1077909593 wait complete. 2024-09-17T05:25:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 Processor -- start 2024-09-17T05:25:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- start start 2024-09-17T05:25:35.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 0x7f96bc0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96bc0ced80 0x7f96bc150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc15bea0 0x7f96bc0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1692f0 con 0x7f96bc0cdb30 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc169170 con 0x7f96bc15bea0 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c4600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f96bc169470 con 0x7f96bc0ced80 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 0x7f96bc0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 0x7f96bc0b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49358/0 (socket says 172.21.15.122:49358) 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1c00640 1 -- 172.21.15.122:0/954307352 learned_addr learned my addr 172.21.15.122:0/954307352 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c2600640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc15bea0 0x7f96bc0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1200640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96bc0ced80 0x7f96bc150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:35.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1c00640 1 -- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96bc0ced80 msgr2=0x7f96bc150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1c00640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96bc0ced80 0x7f96bc150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1c00640 1 -- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc15bea0 msgr2=0x7f96bc0b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1c00640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc15bea0 0x7f96bc0b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.398+0000 7f96c1c00640 1 -- 172.21.15.122:0/954307352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc0b7210 con 0x7f96bc0cdb30 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96c1c00640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 0x7f96bc0b5140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c760 tx=0x7f96b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96b9600640 1 -- 172.21.15.122:0/954307352 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0003b80 con 0x7f96bc0cdb30 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96b9600640 1 -- 172.21.15.122:0/954307352 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96b0003d40 con 0x7f96bc0cdb30 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96b9600640 1 -- 172.21.15.122:0/954307352 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b000aab0 con 0x7f96bc0cdb30 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc0b74b0 con 0x7f96bc0cdb30 2024-09-17T05:25:35.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc0b7a70 con 0x7f96bc0cdb30 2024-09-17T05:25:35.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96bc0ce670 con 0x7f96bc0cdb30 2024-09-17T05:25:35.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96b9600640 1 -- 172.21.15.122:0/954307352 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f96b000ac50 con 0x7f96bc0cdb30 2024-09-17T05:25:35.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96b9600640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f969c073230 0x7f969c0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:35.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.402+0000 7f96b9600640 1 -- 172.21.15.122:0/954307352 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f96b0093ee0 con 0x7f96bc0cdb30 2024-09-17T05:25:35.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.406+0000 7f96c1200640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f969c073230 0x7f969c0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:35.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.406+0000 7f96c1200640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f969c073230 0x7f969c0756f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96bc1fb120 tx=0x7f96a800b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:35.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.406+0000 7f96b9600640 1 -- 172.21.15.122:0/954307352 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f96b005e930 con 0x7f96bc0cdb30 2024-09-17T05:25:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.586+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f96bc0725b0 con 0x7f96bc0cdb30 2024-09-17T05:25:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.586+0000 7f96b9600640 1 -- 172.21.15.122:0/954307352 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f96b0063360 con 0x7f96bc0cdb30 2024-09-17T05:25:35.591 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:25:35.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f969c073230 msgr2=0x7f969c0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f969c073230 0x7f969c0756f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96bc1fb120 tx=0x7f96a800b390 comp rx=0 tx=0).stop 2024-09-17T05:25:35.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 msgr2=0x7f96bc0b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 0x7f96bc0b5140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c760 tx=0x7f96b000cc20 comp rx=0 tx=0).stop 2024-09-17T05:25:35.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 shutdown_connections 2024-09-17T05:25:35.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f969c073230 0x7f969c0756f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96bc15bea0 0x7f96bc0b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96bc0ced80 0x7f96bc150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 --2- 172.21.15.122:0/954307352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96bc0cdb30 0x7f96bc0b5140 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 >> 172.21.15.122:0/954307352 conn(0x7f96bc0d1dc0 msgr2=0x7f96bc0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:35.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 shutdown_connections 2024-09-17T05:25:35.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.590+0000 7f96c4600640 1 -- 172.21.15.122:0/954307352 wait complete. 2024-09-17T05:25:35.612 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-17T05:25:35.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:35.613 DEBUG:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-17T05:25:35.639 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:25:35.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:35.639 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-17T05:25:35.689 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:25:35.689 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:35.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.826+0000 7f3613600640 1 -- 172.21.15.122:0/1270024960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 msgr2=0x7f360c0b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.826+0000 7f3613600640 1 --2- 172.21.15.122:0/1270024960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 0x7f360c0b14c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009900 tx=0x7f35fc01c820 comp rx=0 tx=0).stop 2024-09-17T05:25:35.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.826+0000 7f3613600640 1 -- 172.21.15.122:0/1270024960 shutdown_connections 2024-09-17T05:25:35.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.826+0000 7f3613600640 1 --2- 172.21.15.122:0/1270024960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f360c0afca0 0x7f360c0b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.826+0000 7f3613600640 1 --2- 172.21.15.122:0/1270024960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 0x7f360c0b14c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.826+0000 7f3613600640 1 -- 172.21.15.122:0/1270024960 >> 172.21.15.122:0/1270024960 conn(0x7f360c0d1e30 msgr2=0x7f360c0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:35.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.826+0000 7f3613600640 1 -- 172.21.15.122:0/1270024960 shutdown_connections 2024-09-17T05:25:35.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.826+0000 7f3613600640 1 -- 172.21.15.122:0/1270024960 wait complete. 2024-09-17T05:25:35.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 Processor -- start 2024-09-17T05:25:35.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 -- start start 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f360c0afca0 0x7f360c1430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f360c0b2e80 0x7f360c159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 0x7f360c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f360c168e00 con 0x7f360c15f490 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f360c168c80 con 0x7f360c0b2e80 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f360c168f80 con 0x7f360c0afca0 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 0x7f360c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 0x7f360c06ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49376/0 (socket says 172.21.15.122:49376) 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 -- 172.21.15.122:0/842226248 learned_addr learned my addr 172.21.15.122:0/842226248 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:35.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 -- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f360c0afca0 msgr2=0x7f360c1430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f360c0afca0 0x7f360c1430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 -- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f360c0b2e80 msgr2=0x7f360c159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f360c0b2e80 0x7f360c159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 -- 172.21.15.122:0/842226248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c150830 con 0x7f360c15f490 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3611600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 0x7f360c06ee70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f360400e9a0 tx=0x7f360400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3608c00640 1 -- 172.21.15.122:0/842226248 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604004d30 con 0x7f360c15f490 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3608c00640 1 -- 172.21.15.122:0/842226248 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f360400cb20 con 0x7f360c15f490 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360c1594a0 con 0x7f360c15f490 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3608c00640 1 -- 172.21.15.122:0/842226248 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604005020 con 0x7f360c15f490 2024-09-17T05:25:35.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.830+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360c159a60 con 0x7f360c15f490 2024-09-17T05:25:35.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.834+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f360c0b0110 con 0x7f360c15f490 2024-09-17T05:25:35.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.834+0000 7f3608c00640 1 -- 172.21.15.122:0/842226248 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36040051c0 con 0x7f360c15f490 2024-09-17T05:25:35.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.834+0000 7f3608c00640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f35ec073230 0x7f35ec0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:35.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.834+0000 7f3608c00640 1 -- 172.21.15.122:0/842226248 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3604094bb0 con 0x7f360c15f490 2024-09-17T05:25:35.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.834+0000 7f3610c00640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f35ec073230 0x7f35ec0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:35.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.834+0000 7f3610c00640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f35ec073230 0x7f35ec0756f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0af000 tx=0x7f35fc0195c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:35.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:35.838+0000 7f3608c00640 1 -- 172.21.15.122:0/842226248 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3604016030 con 0x7f360c15f490 2024-09-17T05:25:36.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.042+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f360c1435e0 con 0x7f360c15f490 2024-09-17T05:25:36.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.042+0000 7f3608c00640 1 -- 172.21.15.122:0/842226248 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f360405fbc0 con 0x7f360c15f490 2024-09-17T05:25:36.047 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:36.047 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:03.536393+0000","modified":"2024-09-17T05:25:06.553594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4332":{"gid":4332,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4332,"qdb_cluster":[4332]},"id":1}]} 2024-09-17T05:25:36.048 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-17T05:25:36.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f35ec073230 msgr2=0x7f35ec0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f35ec073230 0x7f35ec0756f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0af000 tx=0x7f35fc0195c0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 msgr2=0x7f360c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 0x7f360c06ee70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f360400e9a0 tx=0x7f360400ee60 comp rx=0 tx=0).stop 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 shutdown_connections 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f35ec073230 0x7f35ec0756f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c15f490 0x7f360c06ee70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f360c0b2e80 0x7f360c159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 --2- 172.21.15.122:0/842226248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f360c0afca0 0x7f360c1430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 >> 172.21.15.122:0/842226248 conn(0x7f360c0d1e30 msgr2=0x7f360c0d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 shutdown_connections 2024-09-17T05:25:36.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.046+0000 7f3613600640 1 -- 172.21.15.122:0/842226248 wait complete. 2024-09-17T05:25:36.069 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:36.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.210+0000 7f3a79c00640 1 -- 172.21.15.122:0/3543775147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 msgr2=0x7f3a74073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.210+0000 7f3a79c00640 1 --2- 172.21.15.122:0/3543775147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 0x7f3a74073c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3a740748c0 tx=0x7f3a6c01c770 comp rx=0 tx=0).stop 2024-09-17T05:25:36.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.210+0000 7f3a79c00640 1 -- 172.21.15.122:0/3543775147 shutdown_connections 2024-09-17T05:25:36.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.210+0000 7f3a79c00640 1 --2- 172.21.15.122:0/3543775147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 0x7f3a74073c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.210+0000 7f3a79c00640 1 --2- 172.21.15.122:0/3543775147 >> v2:172.21.15.173:3300/0 conn(0x7f3a741518b0 0x7f3a74158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.210+0000 7f3a79c00640 1 -- 172.21.15.122:0/3543775147 >> 172.21.15.122:0/3543775147 conn(0x7f3a7406dd30 msgr2=0x7f3a7406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:36.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.210+0000 7f3a79c00640 1 -- 172.21.15.122:0/3543775147 shutdown_connections 2024-09-17T05:25:36.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.210+0000 7f3a79c00640 1 -- 172.21.15.122:0/3543775147 wait complete. 2024-09-17T05:25:36.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 Processor -- start 2024-09-17T05:25:36.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 -- start start 2024-09-17T05:25:36.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 0x7f3a7414c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a741518b0 0x7f3a740cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a741590a0 0x7f3a741f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a741676a0 con 0x7f3a74073820 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a740b5390 con 0x7f3a741590a0 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3a74167820 con 0x7f3a741518b0 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 0x7f3a7414c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 0x7f3a7414c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49388/0 (socket says 172.21.15.122:49388) 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 -- 172.21.15.122:0/1483575477 learned_addr learned my addr 172.21.15.122:0/1483575477 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79200640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a741590a0 0x7f3a741f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a6b400640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a741518b0 0x7f3a740cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 -- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a741518b0 msgr2=0x7f3a740cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a741518b0 0x7f3a740cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 -- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a741590a0 msgr2=0x7f3a741f79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a741590a0 0x7f3a741f79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 -- 172.21.15.122:0/1483575477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a741520e0 con 0x7f3a74073820 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a6b400640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a741518b0 0x7f3a740cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a73e00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 0x7f3a7414c180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400cc30 tx=0x7f3a6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a70c00640 1 -- 172.21.15.122:0/1483575477 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6400f650 con 0x7f3a74073820 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a70c00640 1 -- 172.21.15.122:0/1483575477 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a6400ae50 con 0x7f3a74073820 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a70c00640 1 -- 172.21.15.122:0/1483575477 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6400fb80 con 0x7f3a74073820 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a741523c0 con 0x7f3a74073820 2024-09-17T05:25:36.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a74152980 con 0x7f3a74073820 2024-09-17T05:25:36.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.214+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a74073c70 con 0x7f3a74073820 2024-09-17T05:25:36.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.218+0000 7f3a70c00640 1 -- 172.21.15.122:0/1483575477 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a64018400 con 0x7f3a74073820 2024-09-17T05:25:36.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.218+0000 7f3a70c00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a50073200 0x7f3a500756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.218+0000 7f3a70c00640 1 -- 172.21.15.122:0/1483575477 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3a64094dd0 con 0x7f3a74073820 2024-09-17T05:25:36.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.218+0000 7f3a6b400640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a50073200 0x7f3a500756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.218+0000 7f3a6b400640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a50073200 0x7f3a500756c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a7406d910 tx=0x7f3a5c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:36.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.222+0000 7f3a70c00640 1 -- 172.21.15.122:0/1483575477 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3a64014030 con 0x7f3a74073820 2024-09-17T05:25:36.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.422+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a74069970 con 0x7f3a74073820 2024-09-17T05:25:36.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a70c00640 1 -- 172.21.15.122:0/1483575477 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f3a6405fdf0 con 0x7f3a74073820 2024-09-17T05:25:36.428 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:36.428 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:03.536393+0000","modified":"2024-09-17T05:25:06.553594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4332":{"gid":4332,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4332,"qdb_cluster":[4332]},"id":1}]} 2024-09-17T05:25:36.428 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-17T05:25:36.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a50073200 msgr2=0x7f3a500756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a50073200 0x7f3a500756c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a7406d910 tx=0x7f3a5c008040 comp rx=0 tx=0).stop 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 msgr2=0x7f3a7414c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 0x7f3a7414c180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400cc30 tx=0x7f3a6400a620 comp rx=0 tx=0).stop 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 shutdown_connections 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a50073200 0x7f3a500756c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a741590a0 0x7f3a741f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a741518b0 0x7f3a740cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 --2- 172.21.15.122:0/1483575477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a74073820 0x7f3a7414c180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 >> 172.21.15.122:0/1483575477 conn(0x7f3a7406dd30 msgr2=0x7f3a74150530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 shutdown_connections 2024-09-17T05:25:36.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.426+0000 7f3a79c00640 1 -- 172.21.15.122:0/1483575477 wait complete. 2024-09-17T05:25:36.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:36.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 -- 172.21.15.122:0/2853251972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bc415db50 msgr2=0x7f3bc40acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/2853251972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bc415db50 0x7f3bc40acd40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009900 tx=0x7f3bb401c850 comp rx=0 tx=0).stop 2024-09-17T05:25:36.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 -- 172.21.15.122:0/2853251972 shutdown_connections 2024-09-17T05:25:36.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/2853251972 >> v2:172.21.15.173:3300/0 conn(0x7f3bc40cd100 0x7f3bc40b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/2853251972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bc415db50 0x7f3bc40acd40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 -- 172.21.15.122:0/2853251972 >> 172.21.15.122:0/2853251972 conn(0x7f3bc40b18f0 msgr2=0x7f3bc40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:36.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 -- 172.21.15.122:0/2853251972 shutdown_connections 2024-09-17T05:25:36.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 -- 172.21.15.122:0/2853251972 wait complete. 2024-09-17T05:25:36.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 Processor -- start 2024-09-17T05:25:36.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.590+0000 7f3bc8a00640 1 -- start start 2024-09-17T05:25:36.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bc40b4880 0x7f3bc415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bc40cd100 0x7f3bc40b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bc415db50 0x7f3bc415c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4165110 con 0x7f3bc415db50 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40cccc0 con 0x7f3bc40cd100 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc4165290 con 0x7f3bc40b4880 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bc415db50 0x7f3bc415c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bc415db50 0x7f3bc415c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49402/0 (socket says 172.21.15.122:49402) 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 -- 172.21.15.122:0/1938800637 learned_addr learned my addr 172.21.15.122:0/1938800637 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc1e00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bc40b4880 0x7f3bc415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bb9400640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bc40cd100 0x7f3bc40b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 -- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bc40b4880 msgr2=0x7f3bc415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bc40b4880 0x7f3bc415a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 -- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bc40cd100 msgr2=0x7f3bc40b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bc40cd100 0x7f3bc40b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 -- 172.21.15.122:0/1938800637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc41f8850 con 0x7f3bc415db50 2024-09-17T05:25:36.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc2800640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bc415db50 0x7f3bc415c810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c770 tx=0x7f3bbc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:36.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bbaa00640 1 -- 172.21.15.122:0/1938800637 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc017070 con 0x7f3bc415db50 2024-09-17T05:25:36.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bbaa00640 1 -- 172.21.15.122:0/1938800637 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bbc005070 con 0x7f3bc415db50 2024-09-17T05:25:36.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc40afba0 con 0x7f3bc415db50 2024-09-17T05:25:36.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bbaa00640 1 -- 172.21.15.122:0/1938800637 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0054c0 con 0x7f3bc415db50 2024-09-17T05:25:36.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc40b0160 con 0x7f3bc415db50 2024-09-17T05:25:36.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.594+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc4069410 con 0x7f3bc415db50 2024-09-17T05:25:36.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.598+0000 7f3bbaa00640 1 -- 172.21.15.122:0/1938800637 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3bbc01a070 con 0x7f3bc415db50 2024-09-17T05:25:36.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.598+0000 7f3bbaa00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3ba0073200 0x7f3ba00756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.598+0000 7f3bbaa00640 1 -- 172.21.15.122:0/1938800637 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3bbc093860 con 0x7f3bc415db50 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.598+0000 7f3bc1e00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3ba0073200 0x7f3ba00756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.598+0000 7f3bc1e00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3ba0073200 0x7f3ba00756c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40af000 tx=0x7f3bb401b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:36.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.602+0000 7f3bbaa00640 1 -- 172.21.15.122:0/1938800637 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3bbc05e220 con 0x7f3bc415db50 2024-09-17T05:25:36.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.770+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bc40b0330 con 0x7f3bc415db50 2024-09-17T05:25:36.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.770+0000 7f3bbaa00640 1 -- 172.21.15.122:0/1938800637 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7f3bbc062c50 con 0x7f3bc415db50 2024-09-17T05:25:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:36.775 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":22,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:05.140074+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:36.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3ba0073200 msgr2=0x7f3ba00756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3ba0073200 0x7f3ba00756c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40af000 tx=0x7f3bb401b090 comp rx=0 tx=0).stop 2024-09-17T05:25:36.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bc415db50 msgr2=0x7f3bc415c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bc415db50 0x7f3bc415c810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c770 tx=0x7f3bbc00cc30 comp rx=0 tx=0).stop 2024-09-17T05:25:36.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 shutdown_connections 2024-09-17T05:25:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3ba0073200 0x7f3ba00756c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3bc415db50 0x7f3bc415c810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bc40cd100 0x7f3bc40b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 --2- 172.21.15.122:0/1938800637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bc40b4880 0x7f3bc415a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 >> 172.21.15.122:0/1938800637 conn(0x7f3bc40b18f0 msgr2=0x7f3bc4158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 shutdown_connections 2024-09-17T05:25:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.774+0000 7f3bc8a00640 1 -- 172.21.15.122:0/1938800637 wait complete. 2024-09-17T05:25:36.796 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:25:36.796 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.934+0000 7f43cb000640 1 -- 172.21.15.122:0/2346288248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40d28e0 msgr2=0x7f43c40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.934+0000 7f43cb000640 1 --2- 172.21.15.122:0/2346288248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40d28e0 0x7f43c40cebb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f43c40745d0 tx=0x7f43bc01caf0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.934+0000 7f43cb000640 1 -- 172.21.15.122:0/2346288248 shutdown_connections 2024-09-17T05:25:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.934+0000 7f43cb000640 1 --2- 172.21.15.122:0/2346288248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40d28e0 0x7f43c40cebb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.934+0000 7f43cb000640 1 --2- 172.21.15.122:0/2346288248 >> v2:172.21.15.173:3300/0 conn(0x7f43c415bea0 0x7f43c40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.934+0000 7f43cb000640 1 -- 172.21.15.122:0/2346288248 >> 172.21.15.122:0/2346288248 conn(0x7f43c40d1dc0 msgr2=0x7f43c40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.934+0000 7f43cb000640 1 -- 172.21.15.122:0/2346288248 shutdown_connections 2024-09-17T05:25:36.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.934+0000 7f43cb000640 1 -- 172.21.15.122:0/2346288248 wait complete. 2024-09-17T05:25:36.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 Processor -- start 2024-09-17T05:25:36.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 -- start start 2024-09-17T05:25:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40cdb30 0x7f43c40c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43c40d28e0 0x7f43c41fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43c415bea0 0x7f43c41fb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4168fc0 con 0x7f43c40cdb30 2024-09-17T05:25:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4168e40 con 0x7f43c40d28e0 2024-09-17T05:25:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f43c4169140 con 0x7f43c415bea0 2024-09-17T05:25:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40cdb30 0x7f43c40c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40cdb30 0x7f43c40c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49410/0 (socket says 172.21.15.122:49410) 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 -- 172.21.15.122:0/2177517321 learned_addr learned my addr 172.21.15.122:0/2177517321 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c9000640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43c415bea0 0x7f43c41fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3400640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43c40d28e0 0x7f43c41fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 -- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43c415bea0 msgr2=0x7f43c41fb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43c415bea0 0x7f43c41fb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 -- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43c40d28e0 msgr2=0x7f43c41fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43c40d28e0 0x7f43c41fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 -- 172.21.15.122:0/2177517321 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c40b69d0 con 0x7f43c40cdb30 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43c3e00640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40cdb30 0x7f43c40c7f10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f43b40058d0 tx=0x7f43b400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43bbe00640 1 -- 172.21.15.122:0/2177517321 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4017070 con 0x7f43c40cdb30 2024-09-17T05:25:36.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43bbe00640 1 -- 172.21.15.122:0/2177517321 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43b4009d20 con 0x7f43c40cdb30 2024-09-17T05:25:36.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43bbe00640 1 -- 172.21.15.122:0/2177517321 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b40106c0 con 0x7f43c40cdb30 2024-09-17T05:25:36.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c40b5c10 con 0x7f43c40cdb30 2024-09-17T05:25:36.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c40b61d0 con 0x7f43c40cdb30 2024-09-17T05:25:36.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.938+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c40cc9d0 con 0x7f43c40cdb30 2024-09-17T05:25:36.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.942+0000 7f43bbe00640 1 -- 172.21.15.122:0/2177517321 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43b401a070 con 0x7f43c40cdb30 2024-09-17T05:25:36.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.942+0000 7f43bbe00640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43a4073200 0x7f43a40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:36.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.942+0000 7f43bbe00640 1 -- 172.21.15.122:0/2177517321 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f43b4094340 con 0x7f43c40cdb30 2024-09-17T05:25:36.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.942+0000 7f43c3400640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43a4073200 0x7f43a40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:36.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.942+0000 7f43c3400640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43a4073200 0x7f43a40756c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43b0000f70 tx=0x7f43b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:36.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:36.946+0000 7f43bbe00640 1 -- 172.21.15.122:0/2177517321 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f43b405f3e0 con 0x7f43c40cdb30 2024-09-17T05:25:37.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.146+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43c40c87d0 con 0x7f43c40cdb30 2024-09-17T05:25:37.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.146+0000 7f43bbe00640 1 -- 172.21.15.122:0/2177517321 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f43b405ece0 con 0x7f43c40cdb30 2024-09-17T05:25:37.151 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:37.151 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:03.536393+0000","modified":"2024-09-17T05:25:06.553594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4332":{"gid":4332,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4332,"qdb_cluster":[4332]},"id":1}]} 2024-09-17T05:25:37.151 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43a4073200 msgr2=0x7f43a40756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43a4073200 0x7f43a40756c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43b0000f70 tx=0x7f43b0009040 comp rx=0 tx=0).stop 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40cdb30 msgr2=0x7f43c40c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40cdb30 0x7f43c40c7f10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f43b40058d0 tx=0x7f43b400dde0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 shutdown_connections 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43a4073200 0x7f43a40756c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43c415bea0 0x7f43c41fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43c40d28e0 0x7f43c41fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 --2- 172.21.15.122:0/2177517321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c40cdb30 0x7f43c40c7f10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 >> 172.21.15.122:0/2177517321 conn(0x7f43c40d1dc0 msgr2=0x7f43c415b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 shutdown_connections 2024-09-17T05:25:37.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.150+0000 7f43cb000640 1 -- 172.21.15.122:0/2177517321 wait complete. 2024-09-17T05:25:37.172 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.310+0000 7f6b39e00640 1 -- 172.21.15.122:0/3324885240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b340cdb30 msgr2=0x7f6b340cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.310+0000 7f6b39e00640 1 --2- 172.21.15.122:0/3324885240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b340cdb30 0x7f6b340cebb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009840 tx=0x7f6b1c01c820 comp rx=0 tx=0).stop 2024-09-17T05:25:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.310+0000 7f6b39e00640 1 -- 172.21.15.122:0/3324885240 shutdown_connections 2024-09-17T05:25:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.310+0000 7f6b39e00640 1 --2- 172.21.15.122:0/3324885240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b340cdb30 0x7f6b340cebb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.310+0000 7f6b39e00640 1 --2- 172.21.15.122:0/3324885240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b3415bea0 0x7f6b340ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.310+0000 7f6b39e00640 1 -- 172.21.15.122:0/3324885240 >> 172.21.15.122:0/3324885240 conn(0x7f6b340d1dc0 msgr2=0x7f6b340d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.310+0000 7f6b39e00640 1 -- 172.21.15.122:0/3324885240 shutdown_connections 2024-09-17T05:25:37.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.310+0000 7f6b39e00640 1 -- 172.21.15.122:0/3324885240 wait complete. 2024-09-17T05:25:37.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 Processor -- start 2024-09-17T05:25:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 -- start start 2024-09-17T05:25:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b340cdb30 0x7f6b3409f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b340d28e0 0x7f6b34156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3415bea0 0x7f6b34158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34168fc0 con 0x7f6b3415bea0 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34168e40 con 0x7f6b340d28e0 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6b34169140 con 0x7f6b340cdb30 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3415bea0 0x7f6b34158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3415bea0 0x7f6b34158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49418/0 (socket says 172.21.15.122:49418) 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 -- 172.21.15.122:0/1021465979 learned_addr learned my addr 172.21.15.122:0/1021465979 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b32a00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b340cdb30 0x7f6b3409f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b2a000640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b340d28e0 0x7f6b34156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 -- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b340cdb30 msgr2=0x7f6b3409f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b340cdb30 0x7f6b3409f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 -- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b340d28e0 msgr2=0x7f6b34156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b340d28e0 0x7f6b34156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 -- 172.21.15.122:0/1021465979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3409fac0 con 0x7f6b3415bea0 2024-09-17T05:25:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b33400640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3415bea0 0x7f6b34158da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00cbc0 tx=0x7f6b2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b2be00640 1 -- 172.21.15.122:0/1021465979 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c016020 con 0x7f6b3415bea0 2024-09-17T05:25:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b2be00640 1 -- 172.21.15.122:0/1021465979 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b2c00ab00 con 0x7f6b3415bea0 2024-09-17T05:25:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b2be00640 1 -- 172.21.15.122:0/1021465979 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c012420 con 0x7f6b3415bea0 2024-09-17T05:25:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b34144b60 con 0x7f6b3415bea0 2024-09-17T05:25:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.314+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b34156350 con 0x7f6b3415bea0 2024-09-17T05:25:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.318+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b340cc9d0 con 0x7f6b3415bea0 2024-09-17T05:25:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.318+0000 7f6b2be00640 1 -- 172.21.15.122:0/1021465979 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b2c019020 con 0x7f6b3415bea0 2024-09-17T05:25:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.318+0000 7f6b2be00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b10073200 0x7f6b100756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.318+0000 7f6b2be00640 1 -- 172.21.15.122:0/1021465979 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6b2c094490 con 0x7f6b3415bea0 2024-09-17T05:25:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.318+0000 7f6b32a00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b10073200 0x7f6b100756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.318+0000 7f6b32a00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b10073200 0x7f6b100756c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009ad0 tx=0x7f6b240091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.322+0000 7f6b2be00640 1 -- 172.21.15.122:0/1021465979 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6b2c020020 con 0x7f6b3415bea0 2024-09-17T05:25:37.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.526+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b340b58c0 con 0x7f6b3415bea0 2024-09-17T05:25:37.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.526+0000 7f6b2be00640 1 -- 172.21.15.122:0/1021465979 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f6b2c05f530 con 0x7f6b3415bea0 2024-09-17T05:25:37.530 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:37.530 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"btime":"2024-09-17T05:25:07:157035+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1804005199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1804005199},{"type":"v1","addr":"172.21.15.173:6836","nonce":1804005199}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/189044874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":189044874},{"type":"v1","addr":"172.21.15.122:6833","nonce":189044874}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4320,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/771117559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":771117559},{"type":"v1","addr":"172.21.15.122:6835","nonce":771117559}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:03.536393+0000","modified":"2024-09-17T05:25:06.553594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4332":{"gid":4332,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2742876385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2742876385},{"type":"v1","addr":"172.21.15.173:6837","nonce":2742876385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4332,"qdb_cluster":[4332]},"id":1}]} 2024-09-17T05:25:37.531 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-09-17T05:25:37.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b10073200 msgr2=0x7f6b100756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b10073200 0x7f6b100756c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009ad0 tx=0x7f6b240091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3415bea0 msgr2=0x7f6b34158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3415bea0 0x7f6b34158da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00cbc0 tx=0x7f6b2c00ccb0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 shutdown_connections 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b10073200 0x7f6b100756c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3415bea0 0x7f6b34158da0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b340d28e0 0x7f6b34156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 --2- 172.21.15.122:0/1021465979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b340cdb30 0x7f6b3409f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 >> 172.21.15.122:0/1021465979 conn(0x7f6b340d1dc0 msgr2=0x7f6b340b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 shutdown_connections 2024-09-17T05:25:37.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.530+0000 7f6b39e00640 1 -- 172.21.15.122:0/1021465979 wait complete. 2024-09-17T05:25:37.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:37.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 -- 172.21.15.122:0/4030422733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b880d28e0 msgr2=0x7f9b880cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/4030422733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b880d28e0 0x7f9b880cebb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9b880745d0 tx=0x7f9b8001ccf0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 -- 172.21.15.122:0/4030422733 shutdown_connections 2024-09-17T05:25:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/4030422733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b880d28e0 0x7f9b880cebb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/4030422733 >> v2:172.21.15.173:3300/0 conn(0x7f9b8815bea0 0x7f9b880ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 -- 172.21.15.122:0/4030422733 >> 172.21.15.122:0/4030422733 conn(0x7f9b880d1dc0 msgr2=0x7f9b880d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 -- 172.21.15.122:0/4030422733 shutdown_connections 2024-09-17T05:25:37.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 -- 172.21.15.122:0/4030422733 wait complete. 2024-09-17T05:25:37.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 Processor -- start 2024-09-17T05:25:37.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.689+0000 7f9b8ee00640 1 -- start start 2024-09-17T05:25:37.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b880cdb30 0x7f9b880c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b880d28e0 0x7f9b88155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8815bea0 0x7f9b881f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88168fc0 con 0x7f9b8815bea0 2024-09-17T05:25:37.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88168e40 con 0x7f9b880d28e0 2024-09-17T05:25:37.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9b88169140 con 0x7f9b880cdb30 2024-09-17T05:25:37.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8815bea0 0x7f9b881f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8815bea0 0x7f9b881f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49430/0 (socket says 172.21.15.122:49430) 2024-09-17T05:25:37.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 -- 172.21.15.122:0/1515712861 learned_addr learned my addr 172.21.15.122:0/1515712861 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:37.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b87400640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b880d28e0 0x7f9b88155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b87e00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b880cdb30 0x7f9b880c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 -- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b880cdb30 msgr2=0x7f9b880c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b880cdb30 0x7f9b880c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 -- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b880d28e0 msgr2=0x7f9b88155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b880d28e0 0x7f9b88155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 -- 172.21.15.122:0/1515712861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b880b3c90 con 0x7f9b8815bea0 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b87400640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b880d28e0 0x7f9b88155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b87e00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b880cdb30 0x7f9b880c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ce00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8815bea0 0x7f9b881f9030 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b80019900 tx=0x7f9b800a5ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b7fe00640 1 -- 172.21.15.122:0/1515712861 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800bb070 con 0x7f9b8815bea0 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b7fe00640 1 -- 172.21.15.122:0/1515712861 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b80004d40 con 0x7f9b8815bea0 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b7fe00640 1 -- 172.21.15.122:0/1515712861 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800b2040 con 0x7f9b8815bea0 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b88150030 con 0x7f9b8815bea0 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b881505f0 con 0x7f9b8815bea0 2024-09-17T05:25:37.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.693+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b880cc9d0 con 0x7f9b8815bea0 2024-09-17T05:25:37.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.697+0000 7f9b7fe00640 1 -- 172.21.15.122:0/1515712861 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9b800be070 con 0x7f9b8815bea0 2024-09-17T05:25:37.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.697+0000 7f9b7fe00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b68073230 0x7f9b680756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:37.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.697+0000 7f9b7fe00640 1 -- 172.21.15.122:0/1515712861 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9b8012e120 con 0x7f9b8815bea0 2024-09-17T05:25:37.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.697+0000 7f9b87e00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b68073230 0x7f9b680756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:37.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.697+0000 7f9b87e00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b68073230 0x7f9b680756f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b78005fd0 tx=0x7f9b78005d00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:37.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.701+0000 7f9b7fe00640 1 -- 172.21.15.122:0/1515712861 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9b80016970 con 0x7f9b8815bea0 2024-09-17T05:25:37.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.869+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b880b3e60 con 0x7f9b8815bea0 2024-09-17T05:25:37.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.869+0000 7f9b7fe00640 1 -- 172.21.15.122:0/1515712861 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7f9b800f91c0 con 0x7f9b8815bea0 2024-09-17T05:25:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:37.874 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":22,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:05.140074+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:02.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:02.482131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:37.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b68073230 msgr2=0x7f9b680756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b68073230 0x7f9b680756f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b78005fd0 tx=0x7f9b78005d00 comp rx=0 tx=0).stop 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8815bea0 msgr2=0x7f9b881f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8815bea0 0x7f9b881f9030 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9b80019900 tx=0x7f9b800a5ad0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 shutdown_connections 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b68073230 0x7f9b680756f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b8815bea0 0x7f9b881f9030 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b880d28e0 0x7f9b88155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 --2- 172.21.15.122:0/1515712861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b880cdb30 0x7f9b880c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 >> 172.21.15.122:0/1515712861 conn(0x7f9b880d1dc0 msgr2=0x7f9b880b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 shutdown_connections 2024-09-17T05:25:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:37.873+0000 7f9b8ee00640 1 -- 172.21.15.122:0/1515712861 wait complete. 2024-09-17T05:25:37.897 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:25:38.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.033+0000 7f88f0a00640 1 -- 172.21.15.122:0/353524476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 msgr2=0x7f88ec06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.033+0000 7f88f0a00640 1 --2- 172.21.15.122:0/353524476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 0x7f88ec06f7b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009840 tx=0x7f88d801c700 comp rx=0 tx=0).stop 2024-09-17T05:25:38.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.033+0000 7f88f0a00640 1 -- 172.21.15.122:0/353524476 shutdown_connections 2024-09-17T05:25:38.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.033+0000 7f88f0a00640 1 --2- 172.21.15.122:0/353524476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88ec0b49e0 0x7f88ec0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.033+0000 7f88f0a00640 1 --2- 172.21.15.122:0/353524476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 0x7f88ec06f7b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.033+0000 7f88f0a00640 1 -- 172.21.15.122:0/353524476 >> 172.21.15.122:0/353524476 conn(0x7f88ec0b18f0 msgr2=0x7f88ec0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:38.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.033+0000 7f88f0a00640 1 -- 172.21.15.122:0/353524476 shutdown_connections 2024-09-17T05:25:38.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.033+0000 7f88f0a00640 1 -- 172.21.15.122:0/353524476 wait complete. 2024-09-17T05:25:38.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 Processor -- start 2024-09-17T05:25:38.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 -- start start 2024-09-17T05:25:38.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88ec0b49e0 0x7f88ec150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 0x7f88ec0aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88ec15dae0 0x7f88ec162470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec0cf0d0 con 0x7f88ec0cd100 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec0cef50 con 0x7f88ec0b49e0 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec0cf250 con 0x7f88ec15dae0 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 0x7f88ec0aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 0x7f88ec0aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49444/0 (socket says 172.21.15.122:49444) 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 -- 172.21.15.122:0/173086434 learned_addr learned my addr 172.21.15.122:0/173086434 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9e00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88ec0b49e0 0x7f88ec150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88ea800640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88ec15dae0 0x7f88ec162470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 -- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88ec15dae0 msgr2=0x7f88ec162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88ec15dae0 0x7f88ec162470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 -- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88ec0b49e0 msgr2=0x7f88ec150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88ec0b49e0 0x7f88ec150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 -- 172.21.15.122:0/173086434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec1520e0 con 0x7f88ec0cd100 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88ea800640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88ec15dae0 0x7f88ec162470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9e00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88ec0b49e0 0x7f88ec150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e9400640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 0x7f88ec0aeef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c10 tx=0x7f88d80a6ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e2000640 1 -- 172.21.15.122:0/173086434 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d80ac070 con 0x7f88ec0cd100 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e2000640 1 -- 172.21.15.122:0/173086434 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88d8004110 con 0x7f88ec0cd100 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1523c0 con 0x7f88ec0cd100 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88e2000640 1 -- 172.21.15.122:0/173086434 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8005030 con 0x7f88ec0cd100 2024-09-17T05:25:38.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.037+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec152980 con 0x7f88ec0cd100 2024-09-17T05:25:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.041+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec069410 con 0x7f88ec0cd100 2024-09-17T05:25:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.041+0000 7f88e2000640 1 -- 172.21.15.122:0/173086434 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f88d80af070 con 0x7f88ec0cd100 2024-09-17T05:25:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.041+0000 7f88e2000640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88cc073200 0x7f88cc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.041+0000 7f88e2000640 1 -- 172.21.15.122:0/173086434 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f88d812dfe0 con 0x7f88ec0cd100 2024-09-17T05:25:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.041+0000 7f88e9e00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88cc073200 0x7f88cc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.045+0000 7f88e9e00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88cc073200 0x7f88cc0756c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f88dc004a40 tx=0x7f88dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.045+0000 7f88e2000640 1 -- 172.21.15.122:0/173086434 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f88d80bf030 con 0x7f88ec0cd100 2024-09-17T05:25:38.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.245+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f88ec15c190 con 0x7f88ec0cd100 2024-09-17T05:25:38.250 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:25:38.245+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:25:38.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.249+0000 7f88e2000640 1 -- 172.21.15.122:0/173086434 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f88d8016410 con 0x7f88ec0cd100 2024-09-17T05:25:38.252 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:25:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88cc073200 msgr2=0x7f88cc0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88cc073200 0x7f88cc0756c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f88dc004a40 tx=0x7f88dc009040 comp rx=0 tx=0).stop 2024-09-17T05:25:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 msgr2=0x7f88ec0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 0x7f88ec0aeef0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c10 tx=0x7f88d80a6ba0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 shutdown_connections 2024-09-17T05:25:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88cc073200 0x7f88cc0756c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88ec15dae0 0x7f88ec162470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88ec0cd100 0x7f88ec0aeef0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 --2- 172.21.15.122:0/173086434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88ec0b49e0 0x7f88ec150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 >> 172.21.15.122:0/173086434 conn(0x7f88ec0b18f0 msgr2=0x7f88ec153640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 shutdown_connections 2024-09-17T05:25:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.253+0000 7f88f0a00640 1 -- 172.21.15.122:0/173086434 wait complete. 2024-09-17T05:25:38.268 INFO:tasks.ceph.mds.d.smithi173.stderr: -6969> 2024-09-17T05:25:09.735+0000 7f18e8800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:38.275 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:25:38.321 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:25:38.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 -- 172.21.15.122:0/2524057566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01240d29b0 msgr2=0x7f01240b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 --2- 172.21.15.122:0/2524057566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01240d29b0 0x7f01240b1cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f010c009900 tx=0x7f010c01f0d0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 -- 172.21.15.122:0/2524057566 shutdown_connections 2024-09-17T05:25:38.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 --2- 172.21.15.122:0/2524057566 >> v2:172.21.15.173:3300/0 conn(0x7f01240d04d0 0x7f01240af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 --2- 172.21.15.122:0/2524057566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01240d29b0 0x7f01240b1cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 -- 172.21.15.122:0/2524057566 >> 172.21.15.122:0/2524057566 conn(0x7f01240d1e90 msgr2=0x7f01240d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:38.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 -- 172.21.15.122:0/2524057566 shutdown_connections 2024-09-17T05:25:38.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 -- 172.21.15.122:0/2524057566 wait complete. 2024-09-17T05:25:38.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 Processor -- start 2024-09-17T05:25:38.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.417+0000 7f0128c00640 1 -- start start 2024-09-17T05:25:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01240d00f0 0x7f01240b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01240d04d0 0x7f012414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01240d29b0 0x7f01241550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01240b0ed0 con 0x7f01240d29b0 2024-09-17T05:25:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f01240b0d50 con 0x7f01240d04d0 2024-09-17T05:25:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f01240af740 con 0x7f01240d00f0 2024-09-17T05:25:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01240d29b0 0x7f01241550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01240d29b0 0x7f01241550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49446/0 (socket says 172.21.15.122:49446) 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 -- 172.21.15.122:0/229789546 learned_addr learned my addr 172.21.15.122:0/229789546 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0119400640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01240d04d0 0x7f012414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0121e00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01240d00f0 0x7f01240b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 -- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01240d00f0 msgr2=0x7f01240b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01240d00f0 0x7f01240b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 -- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01240d04d0 msgr2=0x7f012414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01240d04d0 0x7f012414feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 -- 172.21.15.122:0/229789546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0124155610 con 0x7f01240d29b0 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0119400640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01240d04d0 0x7f012414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0122800640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01240d29b0 0x7f01241550d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e9b0 tx=0x7f011c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f011aa00640 1 -- 172.21.15.122:0/229789546 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c00cd80 con 0x7f01240d29b0 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f011aa00640 1 -- 172.21.15.122:0/229789546 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f011c004020 con 0x7f01240d29b0 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01240c69d0 con 0x7f01240d29b0 2024-09-17T05:25:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f011aa00640 1 -- 172.21.15.122:0/229789546 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c004450 con 0x7f01240d29b0 2024-09-17T05:25:38.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01240c6130 con 0x7f01240d29b0 2024-09-17T05:25:38.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01241e8650 con 0x7f01240d29b0 2024-09-17T05:25:38.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.421+0000 7f011aa00640 1 -- 172.21.15.122:0/229789546 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f011c003600 con 0x7f01240d29b0 2024-09-17T05:25:38.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.425+0000 7f011aa00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100073200 0x7f01000756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:38.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.425+0000 7f011aa00640 1 -- 172.21.15.122:0/229789546 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7f011c094b40 con 0x7f01240d29b0 2024-09-17T05:25:38.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.425+0000 7f0121e00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100073200 0x7f01000756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:38.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.425+0000 7f0121e00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100073200 0x7f01000756c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0114001f30 tx=0x7f0114009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:38.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.429+0000 7f011aa00640 1 -- 172.21.15.122:0/229789546 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f011c05f540 con 0x7f01240d29b0 2024-09-17T05:25:38.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:38.633+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f01240af200 con 0x7f01240d29b0 2024-09-17T05:25:39.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.253+0000 7f011aa00640 1 -- 172.21.15.122:0/229789546 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f011c063f70 con 0x7f01240d29b0 2024-09-17T05:25:39.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100073200 msgr2=0x7f01000756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100073200 0x7f01000756c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0114001f30 tx=0x7f0114009040 comp rx=0 tx=0).stop 2024-09-17T05:25:39.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01240d29b0 msgr2=0x7f01241550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01240d29b0 0x7f01241550d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e9b0 tx=0x7f011c00ee70 comp rx=0 tx=0).stop 2024-09-17T05:25:39.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 shutdown_connections 2024-09-17T05:25:39.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100073200 0x7f01000756c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01240d29b0 0x7f01241550d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01240d04d0 0x7f012414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 --2- 172.21.15.122:0/229789546 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01240d00f0 0x7f01240b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 >> 172.21.15.122:0/229789546 conn(0x7f01240d1e90 msgr2=0x7f012415bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 shutdown_connections 2024-09-17T05:25:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.257+0000 7f0128c00640 1 -- 172.21.15.122:0/229789546 wait complete. 2024-09-17T05:25:39.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:25:39.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.421+0000 7efc30600640 1 -- 172.21.15.122:0/3156898032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc280aef60 msgr2=0x7efc280ae660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.421+0000 7efc30600640 1 --2- 172.21.15.122:0/3156898032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc280aef60 0x7efc280ae660 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc14009840 tx=0x7efc1401c790 comp rx=0 tx=0).stop 2024-09-17T05:25:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.421+0000 7efc30600640 1 -- 172.21.15.122:0/3156898032 shutdown_connections 2024-09-17T05:25:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.421+0000 7efc30600640 1 --2- 172.21.15.122:0/3156898032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc280aef60 0x7efc280ae660 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.421+0000 7efc30600640 1 -- 172.21.15.122:0/3156898032 >> 172.21.15.122:0/3156898032 conn(0x7efc280cfbc0 msgr2=0x7efc280cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.421+0000 7efc30600640 1 -- 172.21.15.122:0/3156898032 shutdown_connections 2024-09-17T05:25:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 -- 172.21.15.122:0/3156898032 wait complete. 2024-09-17T05:25:39.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 Processor -- start 2024-09-17T05:25:39.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 -- start start 2024-09-17T05:25:39.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc280aef60 0x7efc281650d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:39.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc2815e360 0x7efc2809b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:39.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc2815e740 0x7efc2809d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:39.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc28169540 con 0x7efc2815e740 2024-09-17T05:25:39.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc2815f900 con 0x7efc2815e360 2024-09-17T05:25:39.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc30600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efc281696c0 con 0x7efc280aef60 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc2815e740 0x7efc2809d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc2815e360 0x7efc2809b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc2815e740 0x7efc2809d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49452/0 (socket says 172.21.15.122:49452) 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 -- 172.21.15.122:0/3256747352 learned_addr learned my addr 172.21.15.122:0/3256747352 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2dc00640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc280aef60 0x7efc281650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 -- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc280aef60 msgr2=0x7efc281650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc280aef60 0x7efc281650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 -- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc2815e360 msgr2=0x7efc2809b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc2815e360 0x7efc2809b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 -- 172.21.15.122:0/3256747352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc28164030 con 0x7efc2815e740 2024-09-17T05:25:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.425+0000 7efc2e600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc2815e740 0x7efc2809d130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc2000cae0 tx=0x7efc2000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc25600640 1 -- 172.21.15.122:0/3256747352 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20016020 con 0x7efc2815e740 2024-09-17T05:25:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc25600640 1 -- 172.21.15.122:0/3256747352 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc2000ab40 con 0x7efc2815e740 2024-09-17T05:25:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc25600640 1 -- 172.21.15.122:0/3256747352 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2000ce20 con 0x7efc2815e740 2024-09-17T05:25:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc28164310 con 0x7efc2815e740 2024-09-17T05:25:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc28164bb0 con 0x7efc2815e740 2024-09-17T05:25:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc28069410 con 0x7efc2815e740 2024-09-17T05:25:39.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc25600640 1 -- 172.21.15.122:0/3256747352 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc20019020 con 0x7efc2815e740 2024-09-17T05:25:39.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc25600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc08073200 0x7efc080756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:39.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc25600640 1 -- 172.21.15.122:0/3256747352 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5795+0+0 (secure 0 0 0) 0x7efc20094330 con 0x7efc2815e740 2024-09-17T05:25:39.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.429+0000 7efc2dc00640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc08073200 0x7efc080756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:39.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.433+0000 7efc2dc00640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc08073200 0x7efc080756c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc1c007fd0 tx=0x7efc1c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:39.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.433+0000 7efc25600640 1 -- 172.21.15.122:0/3256747352 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efc20020020 con 0x7efc2815e740 2024-09-17T05:25:39.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:39.605+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc28157020 con 0x7efc2815e740 2024-09-17T05:25:40.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.485+0000 7efc25600640 1 -- 172.21.15.122:0/3256747352 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7efc2005ed30 con 0x7efc2815e740 2024-09-17T05:25:40.490 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:25:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc08073200 msgr2=0x7efc080756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc08073200 0x7efc080756c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc1c007fd0 tx=0x7efc1c009040 comp rx=0 tx=0).stop 2024-09-17T05:25:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc2815e740 msgr2=0x7efc2809d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc2815e740 0x7efc2809d130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc2000cae0 tx=0x7efc2000cbd0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 shutdown_connections 2024-09-17T05:25:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc08073200 0x7efc080756c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc2815e740 0x7efc2809d130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc2815e360 0x7efc2809b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 --2- 172.21.15.122:0/3256747352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc280aef60 0x7efc281650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 >> 172.21.15.122:0/3256747352 conn(0x7efc280cfbc0 msgr2=0x7efc280cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 shutdown_connections 2024-09-17T05:25:40.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.489+0000 7efc30600640 1 -- 172.21.15.122:0/3256747352 wait complete. 2024-09-17T05:25:40.519 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:25:40.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.661+0000 7f31c0600640 1 -- 172.21.15.122:0/3789933340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31b80cea60 msgr2=0x7f31b806eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.661+0000 7f31c0600640 1 --2- 172.21.15.122:0/3789933340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31b80cea60 0x7f31b806eb40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009bc0 tx=0x7f31b001cae0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.661+0000 7f31c0600640 1 -- 172.21.15.122:0/3789933340 shutdown_connections 2024-09-17T05:25:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.661+0000 7f31c0600640 1 --2- 172.21.15.122:0/3789933340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31b80cea60 0x7f31b806eb40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.661+0000 7f31c0600640 1 --2- 172.21.15.122:0/3789933340 >> v2:172.21.15.173:3300/0 conn(0x7f31b80734f0 0x7f31b815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.661+0000 7f31c0600640 1 -- 172.21.15.122:0/3789933340 >> 172.21.15.122:0/3789933340 conn(0x7f31b80cfbc0 msgr2=0x7f31b80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 -- 172.21.15.122:0/3789933340 shutdown_connections 2024-09-17T05:25:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 -- 172.21.15.122:0/3789933340 wait complete. 2024-09-17T05:25:40.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 Processor -- start 2024-09-17T05:25:40.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 -- start start 2024-09-17T05:25:40.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31b80734f0 0x7f31b8154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31b80cea60 0x7f31b809d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31b80cf440 0x7f31b81f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31b806faf0 con 0x7f31b80cf440 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f31b806f970 con 0x7f31b80cea60 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f31b806fc70 con 0x7f31b80734f0 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31b80cf440 0x7f31b81f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31b80cf440 0x7f31b81f8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49462/0 (socket says 172.21.15.122:49462) 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 -- 172.21.15.122:0/3008913530 learned_addr learned my addr 172.21.15.122:0/3008913530 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 -- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31b80734f0 msgr2=0x7f31b8154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31b80734f0 0x7f31b8154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 -- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31b80cea60 msgr2=0x7f31b809d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31b80cea60 0x7f31b809d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 -- 172.21.15.122:0/3008913530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b81f7d90 con 0x7f31b80cf440 2024-09-17T05:25:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31be600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31b80cf440 0x7f31b81f8520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f31b001a710 tx=0x7f31b0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31b5600640 1 -- 172.21.15.122:0/3008913530 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00b2040 con 0x7f31b80cf440 2024-09-17T05:25:40.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31b5600640 1 -- 172.21.15.122:0/3008913530 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31b0004d40 con 0x7f31b80cf440 2024-09-17T05:25:40.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b809b470 con 0x7f31b80cf440 2024-09-17T05:25:40.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.665+0000 7f31b5600640 1 -- 172.21.15.122:0/3008913530 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0004210 con 0x7f31b80cf440 2024-09-17T05:25:40.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.669+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b81f8000 con 0x7f31b80cf440 2024-09-17T05:25:40.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.669+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b8069410 con 0x7f31b80cf440 2024-09-17T05:25:40.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.669+0000 7f31b5600640 1 -- 172.21.15.122:0/3008913530 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31b00be060 con 0x7f31b80cf440 2024-09-17T05:25:40.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.669+0000 7f31b5600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3198073200 0x7f31980756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:40.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.669+0000 7f31b5600640 1 -- 172.21.15.122:0/3008913530 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5357+0+0 (secure 0 0 0) 0x7f31b012dde0 con 0x7f31b80cf440 2024-09-17T05:25:40.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.669+0000 7f31bdc00640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3198073200 0x7f31980756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:40.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.673+0000 7f31bdc00640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3198073200 0x7f31980756c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0058d0 tx=0x7f31ac005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:40.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.673+0000 7f31b5600640 1 -- 172.21.15.122:0/3008913530 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f31b0016790 con 0x7f31b80cf440 2024-09-17T05:25:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:40.841+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f31b815a9d0 con 0x7f31b80cf440 2024-09-17T05:25:41.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.493+0000 7f31b5600640 1 -- 172.21.15.122:0/3008913530 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f31b00f8fe0 con 0x7f31b80cf440 2024-09-17T05:25:41.496 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:25:41.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.493+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3198073200 msgr2=0x7f31980756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.493+0000 7f31c0600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3198073200 0x7f31980756c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0058d0 tx=0x7f31ac005860 comp rx=0 tx=0).stop 2024-09-17T05:25:41.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.493+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31b80cf440 msgr2=0x7f31b81f8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.493+0000 7f31c0600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31b80cf440 0x7f31b81f8520 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f31b001a710 tx=0x7f31b0003f60 comp rx=0 tx=0).stop 2024-09-17T05:25:41.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.497+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 shutdown_connections 2024-09-17T05:25:41.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.497+0000 7f31c0600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3198073200 0x7f31980756c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.497+0000 7f31c0600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31b80cf440 0x7f31b81f8520 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.497+0000 7f31c0600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31b80cea60 0x7f31b809d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.497+0000 7f31c0600640 1 --2- 172.21.15.122:0/3008913530 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31b80734f0 0x7f31b8154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.497+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 >> 172.21.15.122:0/3008913530 conn(0x7f31b80cfbc0 msgr2=0x7f31b8151130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.497+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 shutdown_connections 2024-09-17T05:25:41.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.497+0000 7f31c0600640 1 -- 172.21.15.122:0/3008913530 wait complete. 2024-09-17T05:25:41.523 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:25:41.523 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:25:41.523 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:25:41.523 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:41.523 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:25:41.524 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:25:41.524 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:25:41.524 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:41.524 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:25:41.524 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:25:41.524 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:25:41.524 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:41.525 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:25:41.525 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:25:41.525 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:25:41.525 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:25:41.535 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:25:41.536 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:25:41.536 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:25:41.537 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:25:41.538 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:25:41.538 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:25:41.539 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:25:41.540 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:25:41.586 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:25:41.586 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:41.587 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:25:41.587 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:41.587 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:25:41.588 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:41.588 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:25:41.588 DEBUG:teuthology.parallel:result is None 2024-09-17T05:25:41.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:25:41.674 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:25:41.675 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:25:41.675 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:25:41.675 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:25:41.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.733+0000 7f43f3e00640 1 -- 172.21.15.122:0/2962611103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43f40cefb0 msgr2=0x7f43f40cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.733+0000 7f43f3e00640 1 --2- 172.21.15.122:0/2962611103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43f40cefb0 0x7f43f40cc5b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009900 tx=0x7f43dc01c850 comp rx=0 tx=0).stop 2024-09-17T05:25:41.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.733+0000 7f43f3e00640 1 -- 172.21.15.122:0/2962611103 shutdown_connections 2024-09-17T05:25:41.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.733+0000 7f43f3e00640 1 --2- 172.21.15.122:0/2962611103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43f40cefb0 0x7f43f40cc5b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.733+0000 7f43f3e00640 1 --2- 172.21.15.122:0/2962611103 >> v2:172.21.15.173:3300/0 conn(0x7f43f415f000 0x7f43f40d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.733+0000 7f43f3e00640 1 -- 172.21.15.122:0/2962611103 >> 172.21.15.122:0/2962611103 conn(0x7f43f409aa10 msgr2=0x7f43f409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:41.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.733+0000 7f43f3e00640 1 -- 172.21.15.122:0/2962611103 shutdown_connections 2024-09-17T05:25:41.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.733+0000 7f43f3e00640 1 -- 172.21.15.122:0/2962611103 wait complete. 2024-09-17T05:25:41.740 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-17T05:25:41.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 Processor -- start 2024-09-17T05:25:41.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 -- start start 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43f40ccaf0 0x7f43f406f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43f40cefb0 0x7f43f4163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f415f000 0x7f43f41599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43f41692c0 con 0x7f43f40cefb0 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f43f4169140 con 0x7f43f415f000 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f43f4169440 con 0x7f43f40ccaf0 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43f40cefb0 0x7f43f4163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43f40cefb0 0x7f43f4163ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41306/0 (socket says 172.21.15.122:41306) 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 -- 172.21.15.122:0/1571229002 learned_addr learned my addr 172.21.15.122:0/1571229002 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f2a00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43f40ccaf0 0x7f43f406f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3400640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f415f000 0x7f43f41599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 -- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43f40ccaf0 msgr2=0x7f43f406f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43f40ccaf0 0x7f43f406f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 -- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f415f000 msgr2=0x7f43f41599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f415f000 0x7f43f41599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 -- 172.21.15.122:0/1571229002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f40cb120 con 0x7f43f40cefb0 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ea000640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43f40cefb0 0x7f43f4163ad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009850 tx=0x7f43dc005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ebe00640 1 -- 172.21.15.122:0/1571229002 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0b2040 con 0x7f43f40cefb0 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ebe00640 1 -- 172.21.15.122:0/1571229002 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43dc004310 con 0x7f43f40cefb0 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43ebe00640 1 -- 172.21.15.122:0/1571229002 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0052d0 con 0x7f43f40cefb0 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f40cb400 con 0x7f43f40cefb0 2024-09-17T05:25:41.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.737+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f40cb9c0 con 0x7f43f40cefb0 2024-09-17T05:25:41.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.741+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f40cc5b0 con 0x7f43f40cefb0 2024-09-17T05:25:41.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.741+0000 7f43ebe00640 1 -- 172.21.15.122:0/1571229002 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43dc005470 con 0x7f43f40cefb0 2024-09-17T05:25:41.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.741+0000 7f43ebe00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43d0073200 0x7f43d00756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:41.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.741+0000 7f43ebe00640 1 -- 172.21.15.122:0/1571229002 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f43dc12d5b0 con 0x7f43f40cefb0 2024-09-17T05:25:41.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.741+0000 7f43f2a00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43d0073200 0x7f43d00756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:41.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.745+0000 7f43f2a00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43d0073200 0x7f43d00756c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009980 tx=0x7f43e4009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:41.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.745+0000 7f43ebe00640 1 -- 172.21.15.122:0/1571229002 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f43dc0164d0 con 0x7f43f40cefb0 2024-09-17T05:25:41.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:41.933+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f43f4164010 con 0x7f43f40cefb0 2024-09-17T05:25:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43ebe00640 1 -- 172.21.15.122:0/1571229002 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f43dc0f8270 con 0x7f43f40cefb0 2024-09-17T05:25:42.512 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43d0073200 msgr2=0x7f43d00756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43d0073200 0x7f43d00756c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009980 tx=0x7f43e4009500 comp rx=0 tx=0).stop 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43f40cefb0 msgr2=0x7f43f4163ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43f40cefb0 0x7f43f4163ad0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009850 tx=0x7f43dc005de0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 shutdown_connections 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f43d0073200 0x7f43d00756c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f415f000 0x7f43f41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43f40cefb0 0x7f43f4163ad0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 --2- 172.21.15.122:0/1571229002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43f40ccaf0 0x7f43f406f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 >> 172.21.15.122:0/1571229002 conn(0x7f43f409aa10 msgr2=0x7f43f40ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 shutdown_connections 2024-09-17T05:25:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.509+0000 7f43f3e00640 1 -- 172.21.15.122:0/1571229002 wait complete. 2024-09-17T05:25:42.537 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:25:42.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 -- 172.21.15.122:0/3877634764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a180b0d40 msgr2=0x7f5a1806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 --2- 172.21.15.122:0/3877634764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a180b0d40 0x7f5a1806dd30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5a18073c90 tx=0x7f5a0801c800 comp rx=0 tx=0).stop 2024-09-17T05:25:42.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 -- 172.21.15.122:0/3877634764 shutdown_connections 2024-09-17T05:25:42.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 --2- 172.21.15.122:0/3877634764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a180b0d40 0x7f5a1806dd30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 --2- 172.21.15.122:0/3877634764 >> v2:172.21.15.173:3300/0 conn(0x7f5a181615d0 0x7f5a180aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 -- 172.21.15.122:0/3877634764 >> 172.21.15.122:0/3877634764 conn(0x7f5a180d1e30 msgr2=0x7f5a180d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:42.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 -- 172.21.15.122:0/3877634764 shutdown_connections 2024-09-17T05:25:42.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 -- 172.21.15.122:0/3877634764 wait complete. 2024-09-17T05:25:42.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.673+0000 7f5a20800640 1 Processor -- start 2024-09-17T05:25:42.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 -- start start 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a180b0d40 0x7f5a1814e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a181615d0 0x7f5a180b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a181619b0 0x7f5a180b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1816b6c0 con 0x7f5a181619b0 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a180d13e0 con 0x7f5a181615d0 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1816b840 con 0x7f5a180b0d40 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a181619b0 0x7f5a180b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a181619b0 0x7f5a180b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41320/0 (socket says 172.21.15.122:41320) 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 -- 172.21.15.122:0/4166259293 learned_addr learned my addr 172.21.15.122:0/4166259293 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1de00640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a180b0d40 0x7f5a1814e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1d400640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a181615d0 0x7f5a180b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 -- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a180b0d40 msgr2=0x7f5a1814e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a180b0d40 0x7f5a1814e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 -- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a181615d0 msgr2=0x7f5a180b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a181615d0 0x7f5a180b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 -- 172.21.15.122:0/4166259293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1815aef0 con 0x7f5a181619b0 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1de00640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a180b0d40 0x7f5a1814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1d400640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a181615d0 0x7f5a180b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a1e800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a181619b0 0x7f5a180b5630 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400c730 tx=0x7f5a1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a12000640 1 -- 172.21.15.122:0/4166259293 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14003b80 con 0x7f5a181619b0 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a12000640 1 -- 172.21.15.122:0/4166259293 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a14003d40 con 0x7f5a181619b0 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a12000640 1 -- 172.21.15.122:0/4166259293 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a140052d0 con 0x7f5a181619b0 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1815b1d0 con 0x7f5a181619b0 2024-09-17T05:25:42.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1815b790 con 0x7f5a181619b0 2024-09-17T05:25:42.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.677+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1806dd30 con 0x7f5a181619b0 2024-09-17T05:25:42.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.681+0000 7f5a12000640 1 -- 172.21.15.122:0/4166259293 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a14005680 con 0x7f5a181619b0 2024-09-17T05:25:42.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.681+0000 7f5a12000640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc073200 0x7f59fc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:42.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.681+0000 7f5a12000640 1 -- 172.21.15.122:0/4166259293 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5a14093db0 con 0x7f5a181619b0 2024-09-17T05:25:42.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.681+0000 7f5a1de00640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc073200 0x7f59fc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:42.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.681+0000 7f5a1de00640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc073200 0x7f59fc0756c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c005fd0 tx=0x7f5a0c00a5e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:42.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.685+0000 7f5a12000640 1 -- 172.21.15.122:0/4166259293 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5a1405f240 con 0x7f5a181619b0 2024-09-17T05:25:42.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.909+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5a18159cf0 con 0x7f5a181619b0 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a12000640 1 -- 172.21.15.122:0/4166259293 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v7) ==== 80+0+6645 (secure 0 0 0) 0x7f5a1405eb40 con 0x7f5a181619b0 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:25:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:25:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:25:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:25:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:25:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:25:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:25:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:25:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:25:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:25:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:25:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:25:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-mds", 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCXEulmeEP1CBAAugCuKOWtGz6WNNVTHq5DMg==", 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-mds" 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-mgr", 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCXEulm2lT1CBAAaBG823+uV2iTkKfTOLwc+g==", 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-osd", 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCXEulm2WP1CBAAdtwzRN3qrWKhYa+Qq+l/pg==", 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-osd" 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-rbd", 2024-09-17T05:25:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCXEulm63L1CBAAzUnRilM+rjQPVrvNj0OYCQ==", 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCXEulmZIL1CBAAb6JCvG933uW1IifJDQHH9w==", 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-17T05:25:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-rgw", 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCXEulmtJH1CBAA9l4TION05QoMr6MxwB44VA==", 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:25:42.936 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:25:42.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc073200 msgr2=0x7f59fc0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc073200 0x7f59fc0756c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c005fd0 tx=0x7f5a0c00a5e0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a181619b0 msgr2=0x7f5a180b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:42.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a181619b0 0x7f5a180b5630 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400c730 tx=0x7f5a1400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 shutdown_connections 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc073200 0x7f59fc0756c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a181619b0 0x7f5a180b5630 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a181615d0 0x7f5a180b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 --2- 172.21.15.122:0/4166259293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a180b0d40 0x7f5a1814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 >> 172.21.15.122:0/4166259293 conn(0x7f5a180d1e30 msgr2=0x7f5a180759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 shutdown_connections 2024-09-17T05:25:42.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:42.913+0000 7f5a20800640 1 -- 172.21.15.122:0/4166259293 wait complete. 2024-09-17T05:25:42.938 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-17T05:25:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.077+0000 7f558b000640 1 -- 172.21.15.122:0/1192541812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 msgr2=0x7f556800cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.077+0000 7f558b000640 1 --2- 172.21.15.122:0/1192541812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 0x7f556800cae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f557c001dc0 tx=0x7f557c01cb30 comp rx=0 tx=0).stop 2024-09-17T05:25:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.077+0000 7f558b000640 1 -- 172.21.15.122:0/1192541812 shutdown_connections 2024-09-17T05:25:43.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.077+0000 7f558b000640 1 --2- 172.21.15.122:0/1192541812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 0x7f556800cae0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.077+0000 7f558b000640 1 -- 172.21.15.122:0/1192541812 >> 172.21.15.122:0/1192541812 conn(0x7f55840b18f0 msgr2=0x7f55840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.077+0000 7f558b000640 1 -- 172.21.15.122:0/1192541812 shutdown_connections 2024-09-17T05:25:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.077+0000 7f558b000640 1 -- 172.21.15.122:0/1192541812 wait complete. 2024-09-17T05:25:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.077+0000 7f558b000640 1 Processor -- start 2024-09-17T05:25:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 -- start start 2024-09-17T05:25:43.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 0x7f55841577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f558409ca00 0x7f558409d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55840ca950 0x7f5584073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55840ceba0 con 0x7f55680064e0 2024-09-17T05:25:43.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f55840cea20 con 0x7f558409ca00 2024-09-17T05:25:43.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f55840ced20 con 0x7f55840ca950 2024-09-17T05:25:43.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 0x7f55841577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 0x7f55841577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41344/0 (socket says 172.21.15.122:41344) 2024-09-17T05:25:43.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 -- 172.21.15.122:0/3909478469 learned_addr learned my addr 172.21.15.122:0/3909478469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:43.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5589000640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55840ca950 0x7f5584073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f557b400640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f558409ca00 0x7f558409d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 -- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55840ca950 msgr2=0x7f5584073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55840ca950 0x7f5584073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 -- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f558409ca00 msgr2=0x7f558409d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f558409ca00 0x7f558409d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 -- 172.21.15.122:0/3909478469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5584157d10 con 0x7f55680064e0 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5583e00640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 0x7f55841577d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5574006630 tx=0x7f55740039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5580c00640 1 -- 172.21.15.122:0/3909478469 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574015070 con 0x7f55680064e0 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5580c00640 1 -- 172.21.15.122:0/3909478469 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5574011900 con 0x7f55680064e0 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f5580c00640 1 -- 172.21.15.122:0/3909478469 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574011d30 con 0x7f55680064e0 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55840c9b90 con 0x7f55680064e0 2024-09-17T05:25:43.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55840ca150 con 0x7f55680064e0 2024-09-17T05:25:43.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.081+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55840cd610 con 0x7f55680064e0 2024-09-17T05:25:43.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.085+0000 7f5580c00640 1 -- 172.21.15.122:0/3909478469 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5574018070 con 0x7f55680064e0 2024-09-17T05:25:43.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.085+0000 7f5580c00640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5560073230 0x7f55600756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.085+0000 7f5580c00640 1 -- 172.21.15.122:0/3909478469 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f55740947c0 con 0x7f55680064e0 2024-09-17T05:25:43.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.085+0000 7f557b400640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5560073230 0x7f55600756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.085+0000 7f557b400640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5560073230 0x7f55600756f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f55840ae780 tx=0x7f556c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.089+0000 7f5580c00640 1 -- 172.21.15.122:0/3909478469 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f557405fbb0 con 0x7f55680064e0 2024-09-17T05:25:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.317+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f55841599c0 con 0x7f55680064e0 2024-09-17T05:25:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f5580c00640 1 -- 172.21.15.122:0/3909478469 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f557405f4b0 con 0x7f55680064e0 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5560073230 msgr2=0x7f55600756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5560073230 0x7f55600756f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f55840ae780 tx=0x7f556c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 msgr2=0x7f55841577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 0x7f55841577d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5574006630 tx=0x7f55740039d0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 shutdown_connections 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5560073230 0x7f55600756f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55840ca950 0x7f5584073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f558409ca00 0x7f558409d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 --2- 172.21.15.122:0/3909478469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680064e0 0x7f55841577d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 >> 172.21.15.122:0/3909478469 conn(0x7f55840b18f0 msgr2=0x7f55840ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 shutdown_connections 2024-09-17T05:25:43.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.321+0000 7f558b000640 1 -- 172.21.15.122:0/3909478469 wait complete. 2024-09-17T05:25:43.346 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-17T05:25:43.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6879e00640 1 -- 172.21.15.122:0/777875780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6858004110 msgr2=0x7f6858004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6879e00640 1 --2- 172.21.15.122:0/777875780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6858004110 0x7f6858004810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f686000bb10 tx=0x7f686001cda0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6873400640 1 -- 172.21.15.122:0/777875780 reap_dead start 2024-09-17T05:25:43.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6879e00640 1 -- 172.21.15.122:0/777875780 shutdown_connections 2024-09-17T05:25:43.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6879e00640 1 -- 172.21.15.122:0/777875780 >> 172.21.15.122:0/777875780 conn(0x7f68740d1dc0 msgr2=0x7f68740d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6879e00640 1 -- 172.21.15.122:0/777875780 shutdown_connections 2024-09-17T05:25:43.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6879e00640 1 -- 172.21.15.122:0/777875780 wait complete. 2024-09-17T05:25:43.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6879e00640 1 Processor -- start 2024-09-17T05:25:43.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.485+0000 7f6879e00640 1 -- start start 2024-09-17T05:25:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6858004110 0x7f6874158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68740b3aa0 0x7f687415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68740b3f20 0x7f687415e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68740ae9b0 con 0x7f68740b3aa0 2024-09-17T05:25:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68740d2af0 con 0x7f68740b3f20 2024-09-17T05:25:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f68740aeb30 con 0x7f6858004110 2024-09-17T05:25:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68740b3aa0 0x7f687415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6858004110 0x7f6874158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68740b3aa0 0x7f687415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41364/0 (socket says 172.21.15.122:41364) 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6858004110 0x7f6874158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:39352/0 (socket says 172.21.15.122:39352) 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 -- 172.21.15.122:0/1805992638 learned_addr learned my addr 172.21.15.122:0/1805992638 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6873400640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68740b3f20 0x7f687415e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 -- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6858004110 msgr2=0x7f6874158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6858004110 0x7f6874158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 -- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68740b3f20 msgr2=0x7f687415e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68740b3f20 0x7f687415e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 -- 172.21.15.122:0/1805992638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687415c550 con 0x7f68740b3aa0 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6872000640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68740b3aa0 0x7f687415e110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f686000ba60 tx=0x7f68600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f686b400640 1 -- 172.21.15.122:0/1805992638 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600be070 con 0x7f68740b3aa0 2024-09-17T05:25:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f686b400640 1 -- 172.21.15.122:0/1805992638 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68600a5e50 con 0x7f68740b3aa0 2024-09-17T05:25:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687415c830 con 0x7f68740b3aa0 2024-09-17T05:25:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f686b400640 1 -- 172.21.15.122:0/1805992638 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68600ab050 con 0x7f68740b3aa0 2024-09-17T05:25:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687415cdf0 con 0x7f68740b3aa0 2024-09-17T05:25:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68740ce670 con 0x7f68740b3aa0 2024-09-17T05:25:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.489+0000 7f686b400640 1 -- 172.21.15.122:0/1805992638 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f68600c1070 con 0x7f68740b3aa0 2024-09-17T05:25:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.493+0000 7f686b400640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6854073230 0x7f68540756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.493+0000 7f686b400640 1 -- 172.21.15.122:0/1805992638 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f686012df90 con 0x7f68740b3aa0 2024-09-17T05:25:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.493+0000 7f6872a00640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6854073230 0x7f68540756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.493+0000 7f6872a00640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6854073230 0x7f68540756f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68740b37f0 tx=0x7f6864006ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.497+0000 7f686b400640 1 -- 172.21.15.122:0/1805992638 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6860016ab0 con 0x7f68740b3aa0 2024-09-17T05:25:43.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.729+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f68740ce320 con 0x7f68740b3aa0 2024-09-17T05:25:43.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f686b400640 1 -- 172.21.15.122:0/1805992638 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f68600f8c10 con 0x7f68740b3aa0 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6854073230 msgr2=0x7f68540756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6854073230 0x7f68540756f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f68740b37f0 tx=0x7f6864006ab0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68740b3aa0 msgr2=0x7f687415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68740b3aa0 0x7f687415e110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f686000ba60 tx=0x7f68600058a0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 shutdown_connections 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6854073230 0x7f68540756f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68740b3f20 0x7f687415e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68740b3aa0 0x7f687415e110 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 --2- 172.21.15.122:0/1805992638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6858004110 0x7f6874158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 >> 172.21.15.122:0/1805992638 conn(0x7f68740d1dc0 msgr2=0x7f687406ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 shutdown_connections 2024-09-17T05:25:43.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.733+0000 7f6879e00640 1 -- 172.21.15.122:0/1805992638 wait complete. 2024-09-17T05:25:43.757 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-17T05:25:43.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- 172.21.15.122:0/3042212412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640afac0 msgr2=0x7fd46415ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 --2- 172.21.15.122:0/3042212412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640afac0 0x7fd46415ca00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009c80 tx=0x7fd45c01cda0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- 172.21.15.122:0/3042212412 shutdown_connections 2024-09-17T05:25:43.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 --2- 172.21.15.122:0/3042212412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640afac0 0x7fd46415ca00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 --2- 172.21.15.122:0/3042212412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4640d0b30 0x7fd46415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- 172.21.15.122:0/3042212412 >> 172.21.15.122:0/3042212412 conn(0x7fd4640b2240 msgr2=0x7fd4640b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:43.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- 172.21.15.122:0/3042212412 shutdown_connections 2024-09-17T05:25:43.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- 172.21.15.122:0/3042212412 wait complete. 2024-09-17T05:25:43.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 Processor -- start 2024-09-17T05:25:43.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- start start 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4640afac0 0x7fd46414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640d0b30 0x7fd4640aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd464159990 0x7fd46415e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641699f0 con 0x7fd4640d0b30 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd464169870 con 0x7fd464159990 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd468e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd464169b70 con 0x7fd4640afac0 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd459400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640d0b30 0x7fd4640aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd459400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640d0b30 0x7fd4640aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41368/0 (socket says 172.21.15.122:41368) 2024-09-17T05:25:43.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd459400640 1 -- 172.21.15.122:0/427211697 learned_addr learned my addr 172.21.15.122:0/427211697 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:43.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd461e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4640afac0 0x7fd46414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd462800640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd464159990 0x7fd46415e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd459400640 1 -- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4640afac0 msgr2=0x7fd46414ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd459400640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4640afac0 0x7fd46414ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd459400640 1 -- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd464159990 msgr2=0x7fd46415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:43.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd459400640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd464159990 0x7fd46415e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:43.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd459400640 1 -- 172.21.15.122:0/427211697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46415d700 con 0x7fd4640d0b30 2024-09-17T05:25:43.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.897+0000 7fd462800640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd464159990 0x7fd46415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:43.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd459400640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640d0b30 0x7fd4640aec60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00d980 tx=0x7fd44c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd45aa00640 1 -- 172.21.15.122:0/427211697 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c017070 con 0x7fd4640d0b30 2024-09-17T05:25:43.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd45aa00640 1 -- 172.21.15.122:0/427211697 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd44c009d20 con 0x7fd4640d0b30 2024-09-17T05:25:43.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd45aa00640 1 -- 172.21.15.122:0/427211697 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0106c0 con 0x7fd4640d0b30 2024-09-17T05:25:43.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46415e990 con 0x7fd4640d0b30 2024-09-17T05:25:43.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46415d970 con 0x7fd4640d0b30 2024-09-17T05:25:43.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4640d0420 con 0x7fd4640d0b30 2024-09-17T05:25:43.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd45aa00640 1 -- 172.21.15.122:0/427211697 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd44c01a070 con 0x7fd4640d0b30 2024-09-17T05:25:43.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd45aa00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd440073230 0x7fd4400756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:43.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd45aa00640 1 -- 172.21.15.122:0/427211697 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd44c0940f0 con 0x7fd4640d0b30 2024-09-17T05:25:43.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.901+0000 7fd461e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd440073230 0x7fd4400756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:43.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.905+0000 7fd461e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd440073230 0x7fd4400756f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd454007fd0 tx=0x7fd454009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:43.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:43.905+0000 7fd45aa00640 1 -- 172.21.15.122:0/427211697 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd44c05ee00 con 0x7fd4640d0b30 2024-09-17T05:25:44.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.133+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fd464068a80 con 0x7fd4640d0b30 2024-09-17T05:25:44.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd45aa00640 1 -- 172.21.15.122:0/427211697 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fd44c063830 con 0x7fd4640d0b30 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd440073230 msgr2=0x7fd4400756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd440073230 0x7fd4400756f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd454007fd0 tx=0x7fd454009040 comp rx=0 tx=0).stop 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640d0b30 msgr2=0x7fd4640aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640d0b30 0x7fd4640aec60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00d980 tx=0x7fd44c00de40 comp rx=0 tx=0).stop 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 shutdown_connections 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd440073230 0x7fd4400756f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd464159990 0x7fd46415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4640d0b30 0x7fd4640aec60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 --2- 172.21.15.122:0/427211697 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4640afac0 0x7fd46414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 >> 172.21.15.122:0/427211697 conn(0x7fd4640b2240 msgr2=0x7fd464156680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 shutdown_connections 2024-09-17T05:25:44.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.137+0000 7fd468e00640 1 -- 172.21.15.122:0/427211697 wait complete. 2024-09-17T05:25:44.162 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-17T05:25:44.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 -- 172.21.15.122:0/1475580632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f834815bfa0 msgr2=0x7f83480cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 --2- 172.21.15.122:0/1475580632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f834815bfa0 0x7f83480cd000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8330009900 tx=0x7f833001f0d0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 -- 172.21.15.122:0/1475580632 shutdown_connections 2024-09-17T05:25:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 --2- 172.21.15.122:0/1475580632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f834815bfa0 0x7f83480cd000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 -- 172.21.15.122:0/1475580632 >> 172.21.15.122:0/1475580632 conn(0x7f83480b18f0 msgr2=0x7f83480b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 -- 172.21.15.122:0/1475580632 shutdown_connections 2024-09-17T05:25:44.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 -- 172.21.15.122:0/1475580632 wait complete. 2024-09-17T05:25:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 Processor -- start 2024-09-17T05:25:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.301+0000 7f834f800640 1 -- start start 2024-09-17T05:25:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83480cd540 0x7f834809b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834815bfa0 0x7f834809ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f834815c3e0 0x7f834815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f834806f2f0 con 0x7f83480cd540 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f834806f170 con 0x7f834815c3e0 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f834806f470 con 0x7f834815bfa0 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83480cd540 0x7f834809b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83480cd540 0x7f834809b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41400/0 (socket says 172.21.15.122:41400) 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 -- 172.21.15.122:0/191306747 learned_addr learned my addr 172.21.15.122:0/191306747 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834d800640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f834815c3e0 0x7f834815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 -- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834815bfa0 msgr2=0x7f834809ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f833fe00640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834815bfa0 0x7f834809ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834815bfa0 0x7f834809ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 -- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f834815c3e0 msgr2=0x7f834815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f834815c3e0 0x7f834815a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 -- 172.21.15.122:0/191306747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83481f5f70 con 0x7f83480cd540 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834d800640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f834815c3e0 0x7f834815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f833fe00640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834815bfa0 0x7f834809ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834ce00640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83480cd540 0x7f834809b4d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8338001ec0 tx=0x7f833800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f8345600640 1 -- 172.21.15.122:0/191306747 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338017070 con 0x7f83480cd540 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f8345600640 1 -- 172.21.15.122:0/191306747 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8338003600 con 0x7f83480cd540 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f8345600640 1 -- 172.21.15.122:0/191306747 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338003a50 con 0x7f83480cd540 2024-09-17T05:25:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83481f6250 con 0x7f83480cd540 2024-09-17T05:25:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481f6810 con 0x7f83480cd540 2024-09-17T05:25:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8348069410 con 0x7f83480cd540 2024-09-17T05:25:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.305+0000 7f8345600640 1 -- 172.21.15.122:0/191306747 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f833801a070 con 0x7f83480cd540 2024-09-17T05:25:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.309+0000 7f8345600640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8324073200 0x7f83240756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.309+0000 7f8345600640 1 -- 172.21.15.122:0/191306747 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f8338093230 con 0x7f83480cd540 2024-09-17T05:25:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.309+0000 7f833fe00640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8324073200 0x7f83240756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.309+0000 7f833fe00640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8324073200 0x7f83240756c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f83300b2000 tx=0x7f833001d910 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.313+0000 7f8345600640 1 -- 172.21.15.122:0/191306747 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f833805e590 con 0x7f83480cd540 2024-09-17T05:25:44.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.541+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f83480cd000 con 0x7f83480cd540 2024-09-17T05:25:44.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.541+0000 7f8345600640 1 -- 172.21.15.122:0/191306747 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7f833805de90 con 0x7f83480cd540 2024-09-17T05:25:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8324073200 msgr2=0x7f83240756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8324073200 0x7f83240756c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f83300b2000 tx=0x7f833001d910 comp rx=0 tx=0).stop 2024-09-17T05:25:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83480cd540 msgr2=0x7f834809b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83480cd540 0x7f834809b4d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8338001ec0 tx=0x7f833800c890 comp rx=0 tx=0).stop 2024-09-17T05:25:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 shutdown_connections 2024-09-17T05:25:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8324073200 0x7f83240756c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f834815c3e0 0x7f834815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834815bfa0 0x7f834809ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 --2- 172.21.15.122:0/191306747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83480cd540 0x7f834809b4d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 >> 172.21.15.122:0/191306747 conn(0x7f83480b18f0 msgr2=0x7f83480b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 shutdown_connections 2024-09-17T05:25:44.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.545+0000 7f834f800640 1 -- 172.21.15.122:0/191306747 wait complete. 2024-09-17T05:25:44.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-17T05:25:44.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.705+0000 7fae45800640 1 -- 172.21.15.122:0/3669937533 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae400ced80 msgr2=0x7fae400ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.705+0000 7fae45800640 1 --2- 172.21.15.122:0/3669937533 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae400ced80 0x7fae400ae9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009870 tx=0x7fae2c01c860 comp rx=0 tx=0).stop 2024-09-17T05:25:44.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.705+0000 7fae45800640 1 -- 172.21.15.122:0/3669937533 shutdown_connections 2024-09-17T05:25:44.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.705+0000 7fae45800640 1 --2- 172.21.15.122:0/3669937533 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae400ced80 0x7fae400ae9b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.705+0000 7fae45800640 1 -- 172.21.15.122:0/3669937533 >> 172.21.15.122:0/3669937533 conn(0x7fae400d1dc0 msgr2=0x7fae400d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.705+0000 7fae45800640 1 -- 172.21.15.122:0/3669937533 shutdown_connections 2024-09-17T05:25:44.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.705+0000 7fae45800640 1 -- 172.21.15.122:0/3669937533 wait complete. 2024-09-17T05:25:44.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae45800640 1 Processor -- start 2024-09-17T05:25:44.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae45800640 1 -- start start 2024-09-17T05:25:44.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae45800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae400cdb30 0x7fae40153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae45800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400ced80 0x7fae40154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae45800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015bea0 0x7fae401f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae45800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae4016a890 con 0x7fae4015bea0 2024-09-17T05:25:44.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae45800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fae4016a710 con 0x7fae400ced80 2024-09-17T05:25:44.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae45800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fae4016aa10 con 0x7fae400cdb30 2024-09-17T05:25:44.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015bea0 0x7fae401f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015bea0 0x7fae401f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41406/0 (socket says 172.21.15.122:41406) 2024-09-17T05:25:44.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 -- 172.21.15.122:0/1015545653 learned_addr learned my addr 172.21.15.122:0/1015545653 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:44.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3e000640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400ced80 0x7fae40154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3ea00640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae400cdb30 0x7fae40153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 -- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae400cdb30 msgr2=0x7fae40153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae400cdb30 0x7fae40153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 -- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400ced80 msgr2=0x7fae40154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400ced80 0x7fae40154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 -- 172.21.15.122:0/1015545653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae40070980 con 0x7fae4015bea0 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae3f400640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015bea0 0x7fae401f9450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c410 tx=0x7fae3800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae36a00640 1 -- 172.21.15.122:0/1015545653 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38011790 con 0x7fae4015bea0 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae36a00640 1 -- 172.21.15.122:0/1015545653 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae38017070 con 0x7fae4015bea0 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.709+0000 7fae36a00640 1 -- 172.21.15.122:0/1015545653 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38011cc0 con 0x7fae4015bea0 2024-09-17T05:25:44.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.713+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae40154e40 con 0x7fae4015bea0 2024-09-17T05:25:44.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.713+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae40155400 con 0x7fae4015bea0 2024-09-17T05:25:44.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.713+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae400ce670 con 0x7fae4015bea0 2024-09-17T05:25:44.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.713+0000 7fae36a00640 1 -- 172.21.15.122:0/1015545653 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fae3801c460 con 0x7fae4015bea0 2024-09-17T05:25:44.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.713+0000 7fae36a00640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:44.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.713+0000 7fae36a00640 1 -- 172.21.15.122:0/1015545653 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fae38093c10 con 0x7fae4015bea0 2024-09-17T05:25:44.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.713+0000 7fae3ea00640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:44.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.717+0000 7fae3ea00640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae30000f90 tx=0x7fae30009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:44.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.717+0000 7fae36a00640 1 -- 172.21.15.122:0/1015545653 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fae3805e920 con 0x7fae4015bea0 2024-09-17T05:25:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.945+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fae400cbe80 con 0x7fae4015bea0 2024-09-17T05:25:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae36a00640 1 -- 172.21.15.122:0/1015545653 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v12) ==== 97+0+0 (secure 0 0 0) 0x7fae38063350 con 0x7fae4015bea0 2024-09-17T05:25:44.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 msgr2=0x7fae200756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fae30000f90 tx=0x7fae30009040 comp rx=0 tx=0).stop 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015bea0 msgr2=0x7fae401f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015bea0 0x7fae401f9450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c410 tx=0x7fae3800c8d0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 shutdown_connections 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fae20073200 0x7fae200756c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae4015bea0 0x7fae401f9450 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae400ced80 0x7fae40154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 --2- 172.21.15.122:0/1015545653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae400cdb30 0x7fae40153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 >> 172.21.15.122:0/1015545653 conn(0x7fae400d1dc0 msgr2=0x7fae400d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 shutdown_connections 2024-09-17T05:25:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:44.949+0000 7fae45800640 1 -- 172.21.15.122:0/1015545653 wait complete. 2024-09-17T05:25:44.975 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-17T05:25:45.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.113+0000 7fb387600640 1 -- 172.21.15.122:0/576177190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 msgr2=0x7fb364004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.113+0000 7fb387600640 1 --2- 172.21.15.122:0/576177190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 0x7fb364004c90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3680016c0 tx=0x7fb36801c6e0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.113+0000 7fb387600640 1 -- 172.21.15.122:0/576177190 shutdown_connections 2024-09-17T05:25:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.113+0000 7fb387600640 1 --2- 172.21.15.122:0/576177190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 0x7fb364004c90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.113+0000 7fb387600640 1 -- 172.21.15.122:0/576177190 >> 172.21.15.122:0/576177190 conn(0x7fb3800d1e30 msgr2=0x7fb3800d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.113+0000 7fb387600640 1 -- 172.21.15.122:0/576177190 shutdown_connections 2024-09-17T05:25:45.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.113+0000 7fb387600640 1 -- 172.21.15.122:0/576177190 wait complete. 2024-09-17T05:25:45.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 Processor -- start 2024-09-17T05:25:45.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 -- start start 2024-09-17T05:25:45.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 0x7fb3800b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3801584b0 0x7fb38014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3800b38b0 0x7fb3800b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb38006e0c0 con 0x7fb364004810 2024-09-17T05:25:45.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb38006df40 con 0x7fb3800b38b0 2024-09-17T05:25:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb38006e240 con 0x7fb3801584b0 2024-09-17T05:25:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 0x7fb3800b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 0x7fb3800b7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41420/0 (socket says 172.21.15.122:41420) 2024-09-17T05:25:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 -- 172.21.15.122:0/1825860291 learned_addr learned my addr 172.21.15.122:0/1825860291 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb385600640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3800b38b0 0x7fb3800b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb377e00640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3801584b0 0x7fb38014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 -- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3801584b0 msgr2=0x7fb38014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3801584b0 0x7fb38014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 -- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3800b38b0 msgr2=0x7fb3800b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3800b38b0 0x7fb3800b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 -- 172.21.15.122:0/1825860291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3800b5f00 con 0x7fb364004810 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb377e00640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3801584b0 0x7fb38014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb384c00640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 0x7fb3800b7770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb370011ae0 tx=0x7fb370011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb37d600640 1 -- 172.21.15.122:0/1825860291 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000fc70 con 0x7fb364004810 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb37d600640 1 -- 172.21.15.122:0/1825860291 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb37000fe10 con 0x7fb364004810 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb37d600640 1 -- 172.21.15.122:0/1825860291 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370019670 con 0x7fb364004810 2024-09-17T05:25:45.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3800b61e0 con 0x7fb364004810 2024-09-17T05:25:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.117+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3800b67a0 con 0x7fb364004810 2024-09-17T05:25:45.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.121+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3800b2460 con 0x7fb364004810 2024-09-17T05:25:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.121+0000 7fb37d600640 1 -- 172.21.15.122:0/1825860291 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb370019810 con 0x7fb364004810 2024-09-17T05:25:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.121+0000 7fb37d600640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb35c073230 0x7fb35c0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.121+0000 7fb37d600640 1 -- 172.21.15.122:0/1825860291 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fb370094ab0 con 0x7fb364004810 2024-09-17T05:25:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.121+0000 7fb377e00640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb35c073230 0x7fb35c0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.125+0000 7fb377e00640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb35c073230 0x7fb35c0756f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb3640057f0 tx=0x7fb3680a8040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.125+0000 7fb37d600640 1 -- 172.21.15.122:0/1825860291 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb37005fe10 con 0x7fb364004810 2024-09-17T05:25:45.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.353+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fb380160d50 con 0x7fb364004810 2024-09-17T05:25:45.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.357+0000 7fb37d600640 1 -- 172.21.15.122:0/1825860291 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v13) ==== 90+0+0 (secure 0 0 0) 0x7fb37005f710 con 0x7fb364004810 2024-09-17T05:25:45.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.357+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb35c073230 msgr2=0x7fb35c0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.357+0000 7fb387600640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb35c073230 0x7fb35c0756f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb3640057f0 tx=0x7fb3680a8040 comp rx=0 tx=0).stop 2024-09-17T05:25:45.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 msgr2=0x7fb3800b7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 0x7fb3800b7770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb370011ae0 tx=0x7fb370011ee0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 shutdown_connections 2024-09-17T05:25:45.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb35c073230 0x7fb35c0756f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3800b38b0 0x7fb3800b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3801584b0 0x7fb38014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 --2- 172.21.15.122:0/1825860291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb364004810 0x7fb3800b7770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 >> 172.21.15.122:0/1825860291 conn(0x7fb3800d1e30 msgr2=0x7fb3801593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 shutdown_connections 2024-09-17T05:25:45.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.361+0000 7fb387600640 1 -- 172.21.15.122:0/1825860291 wait complete. 2024-09-17T05:25:45.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:25:45.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- 172.21.15.122:0/2791212689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b0ce0 msgr2=0x7f54200b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 --2- 172.21.15.122:0/2791212689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b0ce0 0x7f54200b5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f540c009840 tx=0x7f540c01c820 comp rx=0 tx=0).stop 2024-09-17T05:25:45.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- 172.21.15.122:0/2791212689 shutdown_connections 2024-09-17T05:25:45.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 --2- 172.21.15.122:0/2791212689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54200b11a0 0x7f5420165670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 --2- 172.21.15.122:0/2791212689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b0ce0 0x7f54200b5870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- 172.21.15.122:0/2791212689 >> 172.21.15.122:0/2791212689 conn(0x7f542009aa10 msgr2=0x7f542009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- 172.21.15.122:0/2791212689 shutdown_connections 2024-09-17T05:25:45.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- 172.21.15.122:0/2791212689 wait complete. 2024-09-17T05:25:45.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 Processor -- start 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- start start 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54200b0ce0 0x7f54201517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54200b11a0 0x7f542014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b4d80 0x7f542014df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5420168a70 con 0x7f54200b4d80 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f54201688f0 con 0x7f54200b11a0 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5420168bf0 con 0x7f54200b0ce0 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b4d80 0x7f542014df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b4d80 0x7f542014df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41444/0 (socket says 172.21.15.122:41444) 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425400640 1 -- 172.21.15.122:0/1402474639 learned_addr learned my addr 172.21.15.122:0/1402474639 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5424a00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54200b0ce0 0x7f54201517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f541fe00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54200b11a0 0x7f542014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425400640 1 -- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54200b0ce0 msgr2=0x7f54201517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425400640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54200b0ce0 0x7f54201517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425400640 1 -- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54200b11a0 msgr2=0x7f542014b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425400640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54200b11a0 0x7f542014b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.525+0000 7f5425400640 1 -- 172.21.15.122:0/1402474639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542014b770 con 0x7f54200b4d80 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f5425400640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b4d80 0x7f542014df40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f541800db20 tx=0x7f541800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f541cc00640 1 -- 172.21.15.122:0/1402474639 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418017070 con 0x7f54200b4d80 2024-09-17T05:25:45.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f541cc00640 1 -- 172.21.15.122:0/1402474639 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5418004b30 con 0x7f54200b4d80 2024-09-17T05:25:45.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f541cc00640 1 -- 172.21.15.122:0/1402474639 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418005020 con 0x7f54200b4d80 2024-09-17T05:25:45.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542014b9f0 con 0x7f54200b4d80 2024-09-17T05:25:45.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542014bfb0 con 0x7f54200b4d80 2024-09-17T05:25:45.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54200b5160 con 0x7f54200b4d80 2024-09-17T05:25:45.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f541cc00640 1 -- 172.21.15.122:0/1402474639 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54180051c0 con 0x7f54200b4d80 2024-09-17T05:25:45.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f541cc00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5400073200 0x7f54000756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f541cc00640 1 -- 172.21.15.122:0/1402474639 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5418094390 con 0x7f54200b4d80 2024-09-17T05:25:45.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.529+0000 7f5424a00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5400073200 0x7f54000756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.533+0000 7f5424a00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5400073200 0x7f54000756c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5414001640 tx=0x7f5414009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.533+0000 7f541cc00640 1 -- 172.21.15.122:0/1402474639 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5418020070 con 0x7f54200b4d80 2024-09-17T05:25:45.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.701+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f54200ce0c0 con 0x7f54200b4d80 2024-09-17T05:25:45.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f541cc00640 1 -- 172.21.15.122:0/1402474639 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f542014b770 con 0x7f54200b4d80 2024-09-17T05:25:45.708 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:45.708 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:25:45.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:25:45.708 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:25:45.708 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:25:45.708 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:25:45.708 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:25:45.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5400073200 msgr2=0x7f54000756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5400073200 0x7f54000756c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5414001640 tx=0x7f5414009040 comp rx=0 tx=0).stop 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b4d80 msgr2=0x7f542014df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b4d80 0x7f542014df40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f541800db20 tx=0x7f541800b260 comp rx=0 tx=0).stop 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 shutdown_connections 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5400073200 0x7f54000756c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54200b4d80 0x7f542014df40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54200b11a0 0x7f542014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 --2- 172.21.15.122:0/1402474639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54200b0ce0 0x7f54201517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 >> 172.21.15.122:0/1402474639 conn(0x7f542009aa10 msgr2=0x7f542006f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 shutdown_connections 2024-09-17T05:25:45.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.705+0000 7f5425e00640 1 -- 172.21.15.122:0/1402474639 wait complete. 2024-09-17T05:25:45.728 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:25:45.728 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:25:45.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.869+0000 7fd798400640 1 -- 172.21.15.122:0/2266824047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7900d28e0 msgr2=0x7fd7900d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.869+0000 7fd798400640 1 --2- 172.21.15.122:0/2266824047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7900d28e0 0x7fd7900d0340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009840 tx=0x7fd77c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.869+0000 7fd798400640 1 -- 172.21.15.122:0/2266824047 shutdown_connections 2024-09-17T05:25:45.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.869+0000 7fd798400640 1 --2- 172.21.15.122:0/2266824047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7900d28e0 0x7fd7900d0340 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.869+0000 7fd798400640 1 -- 172.21.15.122:0/2266824047 >> 172.21.15.122:0/2266824047 conn(0x7fd7900d1dc0 msgr2=0x7fd7900d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:45.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.869+0000 7fd798400640 1 -- 172.21.15.122:0/2266824047 shutdown_connections 2024-09-17T05:25:45.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.869+0000 7fd798400640 1 -- 172.21.15.122:0/2266824047 wait complete. 2024-09-17T05:25:45.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.869+0000 7fd798400640 1 Processor -- start 2024-09-17T05:25:45.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 -- start start 2024-09-17T05:25:45.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7900af850 0x7fd7900c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7900d28e0 0x7fd7900b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79015f390 0x7fd790158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd790169eb0 con 0x7fd79015f390 2024-09-17T05:25:45.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd790169d30 con 0x7fd7900d28e0 2024-09-17T05:25:45.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd79016a030 con 0x7fd7900af850 2024-09-17T05:25:45.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79015f390 0x7fd790158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79015f390 0x7fd790158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41460/0 (socket says 172.21.15.122:41460) 2024-09-17T05:25:45.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 -- 172.21.15.122:0/174430656 learned_addr learned my addr 172.21.15.122:0/174430656 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:45.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd795000640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7900d28e0 0x7fd7900b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd795a00640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7900af850 0x7fd7900c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 -- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7900af850 msgr2=0x7fd7900c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7900af850 0x7fd7900c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 -- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7900d28e0 msgr2=0x7fd7900b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:45.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7900d28e0 0x7fd7900b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:45.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 -- 172.21.15.122:0/174430656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd790159e00 con 0x7fd79015f390 2024-09-17T05:25:45.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd796400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79015f390 0x7fd790158a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd78800c3c0 tx=0x7fd78800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd78d600640 1 -- 172.21.15.122:0/174430656 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788011790 con 0x7fd79015f390 2024-09-17T05:25:45.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd78d600640 1 -- 172.21.15.122:0/174430656 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd788017070 con 0x7fd79015f390 2024-09-17T05:25:45.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd78d600640 1 -- 172.21.15.122:0/174430656 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788011cc0 con 0x7fd79015f390 2024-09-17T05:25:45.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd790157330 con 0x7fd79015f390 2024-09-17T05:25:45.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd790159b20 con 0x7fd79015f390 2024-09-17T05:25:45.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.873+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7900afc30 con 0x7fd79015f390 2024-09-17T05:25:45.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.877+0000 7fd78d600640 1 -- 172.21.15.122:0/174430656 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd78801c460 con 0x7fd79015f390 2024-09-17T05:25:45.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.877+0000 7fd78d600640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd770073200 0x7fd7700756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:45.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.877+0000 7fd78d600640 1 -- 172.21.15.122:0/174430656 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd788093b70 con 0x7fd79015f390 2024-09-17T05:25:45.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.877+0000 7fd795a00640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd770073200 0x7fd7700756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:45.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.877+0000 7fd795a00640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd770073200 0x7fd7700756c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7900719d0 tx=0x7fd784001a00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:45.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:45.881+0000 7fd78d600640 1 -- 172.21.15.122:0/174430656 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd78805e910 con 0x7fd79015f390 2024-09-17T05:25:46.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.049+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd7900ce2c0 con 0x7fd79015f390 2024-09-17T05:25:46.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.517+0000 7fd78d600640 1 -- 172.21.15.122:0/174430656 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7fd788063340 con 0x7fd79015f390 2024-09-17T05:25:46.521 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd770073200 msgr2=0x7fd7700756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd770073200 0x7fd7700756c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7900719d0 tx=0x7fd784001a00 comp rx=0 tx=0).stop 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79015f390 msgr2=0x7fd790158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79015f390 0x7fd790158a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd78800c3c0 tx=0x7fd78800c880 comp rx=0 tx=0).stop 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 shutdown_connections 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd770073200 0x7fd7700756c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79015f390 0x7fd790158a00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7900d28e0 0x7fd7900b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 --2- 172.21.15.122:0/174430656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7900af850 0x7fd7900c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 >> 172.21.15.122:0/174430656 conn(0x7fd7900d1dc0 msgr2=0x7fd790072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:46.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 shutdown_connections 2024-09-17T05:25:46.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.521+0000 7fd798400640 1 -- 172.21.15.122:0/174430656 wait complete. 2024-09-17T05:25:46.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:25:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.697+0000 7f7182400640 1 -- 172.21.15.122:0/3297833860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f717c0cbb90 msgr2=0x7f717c0c48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.697+0000 7f7182400640 1 --2- 172.21.15.122:0/3297833860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f717c0cbb90 0x7f717c0c48e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f716800a950 tx=0x7f716801db70 comp rx=0 tx=0).stop 2024-09-17T05:25:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.697+0000 7f7182400640 1 -- 172.21.15.122:0/3297833860 shutdown_connections 2024-09-17T05:25:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.697+0000 7f7182400640 1 --2- 172.21.15.122:0/3297833860 >> v2:172.21.15.173:3300/0 conn(0x7f717c0cb340 0x7f717c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.697+0000 7f7182400640 1 --2- 172.21.15.122:0/3297833860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f717c0cbb90 0x7f717c0c48e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.697+0000 7f7182400640 1 -- 172.21.15.122:0/3297833860 >> 172.21.15.122:0/3297833860 conn(0x7f717c0cc4a0 msgr2=0x7f717c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.697+0000 7f7182400640 1 -- 172.21.15.122:0/3297833860 shutdown_connections 2024-09-17T05:25:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.697+0000 7f7182400640 1 -- 172.21.15.122:0/3297833860 wait complete. 2024-09-17T05:25:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 Processor -- start 2024-09-17T05:25:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 -- start start 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f717c0cb340 0x7f717c1fbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f717c0cbb90 0x7f717c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717c15a290 0x7f717c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f717c15e8e0 con 0x7f717c15a290 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f717c15e760 con 0x7f717c0cbb90 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f717c15ea60 con 0x7f717c0cb340 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717c15a290 0x7f717c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717c15a290 0x7f717c0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41472/0 (socket says 172.21.15.122:41472) 2024-09-17T05:25:46.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 -- 172.21.15.122:0/2188997858 learned_addr learned my addr 172.21.15.122:0/2188997858 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181000640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f717c0cb340 0x7f717c1fbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f717be00640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f717c0cbb90 0x7f717c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 -- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f717c0cb340 msgr2=0x7f717c1fbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f717c0cb340 0x7f717c1fbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 -- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f717c0cbb90 msgr2=0x7f717c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f717c0cbb90 0x7f717c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 -- 172.21.15.122:0/2188997858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c0b3f70 con 0x7f717c15a290 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f717be00640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f717c0cbb90 0x7f717c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7181a00640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717c15a290 0x7f717c0b1230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f717400efd0 tx=0x7f717400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7178c00640 1 -- 172.21.15.122:0/2188997858 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174003b80 con 0x7f717c15a290 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7178c00640 1 -- 172.21.15.122:0/2188997858 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7174003d20 con 0x7f717c15a290 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c0b4250 con 0x7f717c15a290 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7178c00640 1 -- 172.21.15.122:0/2188997858 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174004280 con 0x7f717c15a290 2024-09-17T05:25:46.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c0b4810 con 0x7f717c15a290 2024-09-17T05:25:46.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.701+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c15dc80 con 0x7f717c15a290 2024-09-17T05:25:46.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.705+0000 7f7178c00640 1 -- 172.21.15.122:0/2188997858 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7174004420 con 0x7f717c15a290 2024-09-17T05:25:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.705+0000 7f7178c00640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f715c073200 0x7f715c0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.705+0000 7f7178c00640 1 -- 172.21.15.122:0/2188997858 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f7174094e70 con 0x7f717c15a290 2024-09-17T05:25:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.705+0000 7f7181000640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f715c073200 0x7f715c0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.705+0000 7f7181000640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f715c073200 0x7f715c0756c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f717c0b2d20 tx=0x7f7170009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.709+0000 7f7178c00640 1 -- 172.21.15.122:0/2188997858 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f71740600f0 con 0x7f717c15a290 2024-09-17T05:25:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:46.877+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f717c0cd130 con 0x7f717c15a290 2024-09-17T05:25:47.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.521+0000 7f7178c00640 1 -- 172.21.15.122:0/2188997858 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f717405f9f0 con 0x7f717c15a290 2024-09-17T05:25:47.524 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:25:47.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.521+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f715c073200 msgr2=0x7f715c0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.521+0000 7f7182400640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f715c073200 0x7f715c0756c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f717c0b2d20 tx=0x7f7170009040 comp rx=0 tx=0).stop 2024-09-17T05:25:47.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.521+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717c15a290 msgr2=0x7f717c0b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.521+0000 7f7182400640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717c15a290 0x7f717c0b1230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f717400efd0 tx=0x7f717400c6a0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.525+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 shutdown_connections 2024-09-17T05:25:47.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.525+0000 7f7182400640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f715c073200 0x7f715c0756c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.525+0000 7f7182400640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717c15a290 0x7f717c0b1230 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.525+0000 7f7182400640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f717c0cbb90 0x7f717c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.525+0000 7f7182400640 1 --2- 172.21.15.122:0/2188997858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f717c0cb340 0x7f717c1fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.525+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 >> 172.21.15.122:0/2188997858 conn(0x7f717c0cc4a0 msgr2=0x7f717c0c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:47.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.525+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 shutdown_connections 2024-09-17T05:25:47.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.525+0000 7f7182400640 1 -- 172.21.15.122:0/2188997858 wait complete. 2024-09-17T05:25:47.565 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:25:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.705+0000 7fbc3a200640 1 -- 172.21.15.122:0/3901703685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415bfa0 msgr2=0x7fbc340cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.705+0000 7fbc3a200640 1 --2- 172.21.15.122:0/3901703685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415bfa0 0x7fbc340cd000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbc340708b0 tx=0x7fbc1c01f0b0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.705+0000 7fbc3a200640 1 -- 172.21.15.122:0/3901703685 shutdown_connections 2024-09-17T05:25:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.705+0000 7fbc3a200640 1 --2- 172.21.15.122:0/3901703685 >> v2:172.21.15.173:3300/0 conn(0x7fbc340cd540 0x7fbc3406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.705+0000 7fbc3a200640 1 --2- 172.21.15.122:0/3901703685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415bfa0 0x7fbc340cd000 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.705+0000 7fbc3a200640 1 -- 172.21.15.122:0/3901703685 >> 172.21.15.122:0/3901703685 conn(0x7fbc340b18f0 msgr2=0x7fbc340b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.705+0000 7fbc3a200640 1 -- 172.21.15.122:0/3901703685 shutdown_connections 2024-09-17T05:25:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.705+0000 7fbc3a200640 1 -- 172.21.15.122:0/3901703685 wait complete. 2024-09-17T05:25:47.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 Processor -- start 2024-09-17T05:25:47.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 -- start start 2024-09-17T05:25:47.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc340cd540 0x7fbc34154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc3415bfa0 0x7fbc340ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415c3e0 0x7fbc340b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3406f070 con 0x7fbc3415c3e0 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3406eef0 con 0x7fbc3415bfa0 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3406f1f0 con 0x7fbc340cd540 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415c3e0 0x7fbc340b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415c3e0 0x7fbc340b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41494/0 (socket says 172.21.15.122:41494) 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 -- 172.21.15.122:0/1508209742 learned_addr learned my addr 172.21.15.122:0/1508209742 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc2aa00640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc3415bfa0 0x7fbc340ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:47.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33400640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc340cd540 0x7fbc34154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 -- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc340cd540 msgr2=0x7fbc34154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc340cd540 0x7fbc34154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 -- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc3415bfa0 msgr2=0x7fbc340ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc3415bfa0 0x7fbc340ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 -- 172.21.15.122:0/1508209742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc340aef50 con 0x7fbc3415c3e0 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33400640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc340cd540 0x7fbc34154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc33e00640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415c3e0 0x7fbc340b3d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d6a0 tx=0x7fbc2c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc2be00640 1 -- 172.21.15.122:0/1508209742 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c017070 con 0x7fbc3415c3e0 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc2be00640 1 -- 172.21.15.122:0/1508209742 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc2c005310 con 0x7fbc3415c3e0 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc2be00640 1 -- 172.21.15.122:0/1508209742 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c009820 con 0x7fbc3415c3e0 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc34156270 con 0x7fbc3415c3e0 2024-09-17T05:25:47.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc34156830 con 0x7fbc3415c3e0 2024-09-17T05:25:47.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.709+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc34069410 con 0x7fbc3415c3e0 2024-09-17T05:25:47.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.713+0000 7fbc2be00640 1 -- 172.21.15.122:0/1508209742 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbc2c0099c0 con 0x7fbc3415c3e0 2024-09-17T05:25:47.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.713+0000 7fbc2be00640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc10073200 0x7fbc100756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:47.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.713+0000 7fbc2be00640 1 -- 172.21.15.122:0/1508209742 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7fbc2c094140 con 0x7fbc3415c3e0 2024-09-17T05:25:47.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.713+0000 7fbc33400640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc10073200 0x7fbc100756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:47.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.713+0000 7fbc33400640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc10073200 0x7fbc100756c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbc24001db0 tx=0x7fbc24009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:47.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.717+0000 7fbc2be00640 1 -- 172.21.15.122:0/1508209742 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbc2c05ebf0 con 0x7fbc3415c3e0 2024-09-17T05:25:47.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:47.917+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbc340b16d0 con 0x7fbc3415c3e0 2024-09-17T05:25:48.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-17T05:25:48.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.541+0000 7fbc2be00640 1 -- 172.21.15.122:0/1508209742 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fbc2c063620 con 0x7fbc3415c3e0 2024-09-17T05:25:48.549 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 4 and data pool 5 2024-09-17T05:25:48.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc10073200 msgr2=0x7fbc100756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc10073200 0x7fbc100756c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbc24001db0 tx=0x7fbc24009040 comp rx=0 tx=0).stop 2024-09-17T05:25:48.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415c3e0 msgr2=0x7fbc340b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415c3e0 0x7fbc340b3d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d6a0 tx=0x7fbc2c00db60 comp rx=0 tx=0).stop 2024-09-17T05:25:48.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 shutdown_connections 2024-09-17T05:25:48.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc10073200 0x7fbc100756c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc3415c3e0 0x7fbc340b3d40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc3415bfa0 0x7fbc340ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 --2- 172.21.15.122:0/1508209742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc340cd540 0x7fbc34154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 >> 172.21.15.122:0/1508209742 conn(0x7fbc340b18f0 msgr2=0x7fbc3415e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:48.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 shutdown_connections 2024-09-17T05:25:48.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.553+0000 7fbc3a200640 1 -- 172.21.15.122:0/1508209742 wait complete. 2024-09-17T05:25:48.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:48.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 -- 172.21.15.122:0/2182018284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe27415f000 msgr2=0x7fe2740d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 --2- 172.21.15.122:0/2182018284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe27415f000 0x7fe2740d01b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe268009900 tx=0x7fe26801c7c0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 -- 172.21.15.122:0/2182018284 shutdown_connections 2024-09-17T05:25:48.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 --2- 172.21.15.122:0/2182018284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe27415f000 0x7fe2740d01b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 -- 172.21.15.122:0/2182018284 >> 172.21.15.122:0/2182018284 conn(0x7fe27409aa10 msgr2=0x7fe27409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:48.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 -- 172.21.15.122:0/2182018284 shutdown_connections 2024-09-17T05:25:48.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 -- 172.21.15.122:0/2182018284 wait complete. 2024-09-17T05:25:48.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 Processor -- start 2024-09-17T05:25:48.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.725+0000 7fe27b200640 1 -- start start 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2740b2590 0x7fe27414d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2740cefb0 0x7fe27414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe27415f000 0x7fe274155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe27416a0c0 con 0x7fe2740b2590 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe274169f40 con 0x7fe27415f000 2024-09-17T05:25:48.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe27416a240 con 0x7fe2740cefb0 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2740b2590 0x7fe27414d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2740b2590 0x7fe27414d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41512/0 (socket says 172.21.15.122:41512) 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 -- 172.21.15.122:0/2768283539 learned_addr learned my addr 172.21.15.122:0/2768283539 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27a800640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe27415f000 0x7fe274155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279400640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2740cefb0 0x7fe27414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 -- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2740cefb0 msgr2=0x7fe27414e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2740cefb0 0x7fe27414e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 -- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe27415f000 msgr2=0x7fe274155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe27415f000 0x7fe274155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 -- 172.21.15.122:0/2768283539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27414e590 con 0x7fe2740b2590 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe279e00640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2740b2590 0x7fe27414d9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe26801a3f0 tx=0x7fe2680a65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe26e000640 1 -- 172.21.15.122:0/2768283539 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680ac070 con 0x7fe2740b2590 2024-09-17T05:25:48.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe26e000640 1 -- 172.21.15.122:0/2768283539 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe268004320 con 0x7fe2740b2590 2024-09-17T05:25:48.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2740c6c50 con 0x7fe2740b2590 2024-09-17T05:25:48.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe26e000640 1 -- 172.21.15.122:0/2768283539 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680053a0 con 0x7fe2740b2590 2024-09-17T05:25:48.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2740c7210 con 0x7fe2740b2590 2024-09-17T05:25:48.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2740b0a20 con 0x7fe2740b2590 2024-09-17T05:25:48.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.729+0000 7fe26e000640 1 -- 172.21.15.122:0/2768283539 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe2680af070 con 0x7fe2740b2590 2024-09-17T05:25:48.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.733+0000 7fe26e000640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe258073200 0x7fe2580756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:48.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.733+0000 7fe26e000640 1 -- 172.21.15.122:0/2768283539 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe26812dd50 con 0x7fe2740b2590 2024-09-17T05:25:48.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.733+0000 7fe279400640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe258073200 0x7fe2580756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:48.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.733+0000 7fe279400640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe258073200 0x7fe2580756c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe274156aa0 tx=0x7fe264009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:48.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.737+0000 7fe26e000640 1 -- 172.21.15.122:0/2768283539 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe2680bf020 con 0x7fe2740b2590 2024-09-17T05:25:48.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.901+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2740b0740 con 0x7fe2740b2590 2024-09-17T05:25:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe26e000640 1 -- 172.21.15.122:0/2768283539 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16742 (secure 0 0 0) 0x7fe2680b6070 con 0x7fe2740b2590 2024-09-17T05:25:48.908 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:48.908 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":32,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:47.923673+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:46.061761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:46.889922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe258073200 msgr2=0x7fe2580756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe258073200 0x7fe2580756c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe274156aa0 tx=0x7fe264009040 comp rx=0 tx=0).stop 2024-09-17T05:25:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2740b2590 msgr2=0x7fe27414d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2740b2590 0x7fe27414d9a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe26801a3f0 tx=0x7fe2680a65a0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 shutdown_connections 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe258073200 0x7fe2580756c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe27415f000 0x7fe274155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2740cefb0 0x7fe27414e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 --2- 172.21.15.122:0/2768283539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2740b2590 0x7fe27414d9a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 >> 172.21.15.122:0/2768283539 conn(0x7fe27409aa10 msgr2=0x7fe2740aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 shutdown_connections 2024-09-17T05:25:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:48.905+0000 7fe27b200640 1 -- 172.21.15.122:0/2768283539 wait complete. 2024-09-17T05:25:48.933 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.069+0000 7fb823e00640 1 -- 172.21.15.122:0/2301992406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb81c0cdb30 msgr2=0x7fb81c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.069+0000 7fb823e00640 1 --2- 172.21.15.122:0/2301992406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb81c0cdb30 0x7fb81c0cebb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb808009840 tx=0x7fb80801c820 comp rx=0 tx=0).stop 2024-09-17T05:25:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.069+0000 7fb823e00640 1 -- 172.21.15.122:0/2301992406 shutdown_connections 2024-09-17T05:25:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.069+0000 7fb823e00640 1 --2- 172.21.15.122:0/2301992406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb81c0cdb30 0x7fb81c0cebb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.069+0000 7fb823e00640 1 --2- 172.21.15.122:0/2301992406 >> v2:172.21.15.173:3300/0 conn(0x7fb81c15bea0 0x7fb81c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.069+0000 7fb823e00640 1 -- 172.21.15.122:0/2301992406 >> 172.21.15.122:0/2301992406 conn(0x7fb81c0d1dc0 msgr2=0x7fb81c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 -- 172.21.15.122:0/2301992406 shutdown_connections 2024-09-17T05:25:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 -- 172.21.15.122:0/2301992406 wait complete. 2024-09-17T05:25:49.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 Processor -- start 2024-09-17T05:25:49.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 -- start start 2024-09-17T05:25:49.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb81c0cdb30 0x7fb81c0b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c0d28e0 0x7fb81c0b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb81c15bea0 0x7fb81c0b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c168fc0 con 0x7fb81c15bea0 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c168e40 con 0x7fb81c0d28e0 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb81c169140 con 0x7fb81c0cdb30 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb81c15bea0 0x7fb81c0b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb81c15bea0 0x7fb81c0b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41524/0 (socket says 172.21.15.122:41524) 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 -- 172.21.15.122:0/246829829 learned_addr learned my addr 172.21.15.122:0/246829829 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb820a00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c0d28e0 0x7fb81c0b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821400640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb81c0cdb30 0x7fb81c0b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 -- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb81c0cdb30 msgr2=0x7fb81c0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb81c0cdb30 0x7fb81c0b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 -- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c0d28e0 msgr2=0x7fb81c0b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c0d28e0 0x7fb81c0b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 -- 172.21.15.122:0/246829829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c145c80 con 0x7fb81c15bea0 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb821e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb81c15bea0 0x7fb81c0b2fd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb81400cbc0 tx=0x7fb81400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb819600640 1 -- 172.21.15.122:0/246829829 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814016020 con 0x7fb81c15bea0 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb819600640 1 -- 172.21.15.122:0/246829829 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb81400ab00 con 0x7fb81c15bea0 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.073+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c0b6320 con 0x7fb81c15bea0 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.077+0000 7fb819600640 1 -- 172.21.15.122:0/246829829 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814012420 con 0x7fb81c15bea0 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.077+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c0b68e0 con 0x7fb81c15bea0 2024-09-17T05:25:49.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.077+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81c0cc9d0 con 0x7fb81c15bea0 2024-09-17T05:25:49.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.077+0000 7fb819600640 1 -- 172.21.15.122:0/246829829 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb814019020 con 0x7fb81c15bea0 2024-09-17T05:25:49.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.077+0000 7fb819600640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7fc073200 0x7fb7fc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.077+0000 7fb819600640 1 -- 172.21.15.122:0/246829829 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb814094400 con 0x7fb81c15bea0 2024-09-17T05:25:49.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.077+0000 7fb821400640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7fc073200 0x7fb7fc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.081+0000 7fb821400640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7fc073200 0x7fb7fc0756c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb810009b40 tx=0x7fb8100091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:49.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.081+0000 7fb819600640 1 -- 172.21.15.122:0/246829829 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb814020020 con 0x7fb81c15bea0 2024-09-17T05:25:49.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.253+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb81c0b5220 con 0x7fb81c15bea0 2024-09-17T05:25:49.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.253+0000 7fb819600640 1 -- 172.21.15.122:0/246829829 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16742 (secure 0 0 0) 0x7fb81405edc0 con 0x7fb81c15bea0 2024-09-17T05:25:49.257 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:49.257 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":32,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:47.923673+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:46.061761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:46.889922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:49.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.253+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7fc073200 msgr2=0x7fb7fc0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.253+0000 7fb823e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7fc073200 0x7fb7fc0756c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb810009b40 tx=0x7fb8100091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.253+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb81c15bea0 msgr2=0x7fb81c0b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.253+0000 7fb823e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb81c15bea0 0x7fb81c0b2fd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb81400cbc0 tx=0x7fb81400ccb0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.257+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 shutdown_connections 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.257+0000 7fb823e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7fc073200 0x7fb7fc0756c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.257+0000 7fb823e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb81c15bea0 0x7fb81c0b2fd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.257+0000 7fb823e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb81c0d28e0 0x7fb81c0b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.257+0000 7fb823e00640 1 --2- 172.21.15.122:0/246829829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb81c0cdb30 0x7fb81c0b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.257+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 >> 172.21.15.122:0/246829829 conn(0x7fb81c0d1dc0 msgr2=0x7fb81c15ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.257+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 shutdown_connections 2024-09-17T05:25:49.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.257+0000 7fb823e00640 1 -- 172.21.15.122:0/246829829 wait complete. 2024-09-17T05:25:49.279 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:25:49.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.417+0000 7f1d50c00640 1 -- 172.21.15.122:0/140230407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c15bd50 msgr2=0x7f1d4c0ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.417+0000 7f1d50c00640 1 --2- 172.21.15.122:0/140230407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c15bd50 0x7f1d4c0ce5e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009840 tx=0x7f1d3c01c700 comp rx=0 tx=0).stop 2024-09-17T05:25:49.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.417+0000 7f1d50c00640 1 -- 172.21.15.122:0/140230407 shutdown_connections 2024-09-17T05:25:49.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.417+0000 7f1d50c00640 1 --2- 172.21.15.122:0/140230407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d4c15c130 0x7f1d4c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.417+0000 7f1d50c00640 1 --2- 172.21.15.122:0/140230407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c15bd50 0x7f1d4c0ce5e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.417+0000 7f1d50c00640 1 -- 172.21.15.122:0/140230407 >> 172.21.15.122:0/140230407 conn(0x7f1d4c0d1ea0 msgr2=0x7f1d4c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:49.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.417+0000 7f1d50c00640 1 -- 172.21.15.122:0/140230407 shutdown_connections 2024-09-17T05:25:49.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.417+0000 7f1d50c00640 1 -- 172.21.15.122:0/140230407 wait complete. 2024-09-17T05:25:49.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 Processor -- start 2024-09-17T05:25:49.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 -- start start 2024-09-17T05:25:49.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c0cf4e0 0x7f1d4c14e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d4c15bd50 0x7f1d4c0c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d4c15c130 0x7f1d4c1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c168e50 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c168cd0 con 0x7f1d4c15c130 2024-09-17T05:25:49.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c168fd0 con 0x7f1d4c15bd50 2024-09-17T05:25:49.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c0cf4e0 0x7f1d4c14e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c0cf4e0 0x7f1d4c14e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41546/0 (socket says 172.21.15.122:41546) 2024-09-17T05:25:49.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 -- 172.21.15.122:0/4025969053 learned_addr learned my addr 172.21.15.122:0/4025969053 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:49.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d4a800640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d4c15c130 0x7f1d4c1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d41400640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d4c15bd50 0x7f1d4c0c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 -- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d4c15bd50 msgr2=0x7f1d4c0c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d4c15bd50 0x7f1d4c0c5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 -- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d4c15c130 msgr2=0x7f1d4c1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:49.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d4c15c130 0x7f1d4c1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:49.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 -- 172.21.15.122:0/4025969053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c0c5550 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d41400640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d4c15bd50 0x7f1d4c0c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:49.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d49e00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c0cf4e0 0x7f1d4c14e680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009790 tx=0x7f1d3c0a6770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:49.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d42a00640 1 -- 172.21.15.122:0/4025969053 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0bf020 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d42a00640 1 -- 172.21.15.122:0/4025969053 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d3c005020 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d42a00640 1 -- 172.21.15.122:0/4025969053 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c005430 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c0c92a0 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.421+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c0b79d0 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.425+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c0c5dd0 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.425+0000 7f1d42a00640 1 -- 172.21.15.122:0/4025969053 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1d3c0c2070 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.425+0000 7f1d42a00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d28073200 0x7f1d280756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:49.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.425+0000 7f1d42a00640 1 -- 172.21.15.122:0/4025969053 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1d3c12eb80 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.425+0000 7f1d41400640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d28073200 0x7f1d280756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:49.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.425+0000 7f1d41400640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d28073200 0x7f1d280756c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c072230 tx=0x7f1d3400b5b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:49.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.429+0000 7f1d42a00640 1 -- 172.21.15.122:0/4025969053 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1d3c016410 con 0x7f1d4c0cf4e0 2024-09-17T05:25:49.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:49.633+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1d4c070b60 con 0x7f1d4c0cf4e0 2024-09-17T05:25:50.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d42a00640 1 -- 172.21.15.122:0/4025969053 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f1d3c0f9540 con 0x7f1d4c0cf4e0 2024-09-17T05:25:50.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d28073200 msgr2=0x7f1d280756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d28073200 0x7f1d280756c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c072230 tx=0x7f1d3400b5b0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c0cf4e0 msgr2=0x7f1d4c14e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c0cf4e0 0x7f1d4c14e680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009790 tx=0x7f1d3c0a6770 comp rx=0 tx=0).stop 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 shutdown_connections 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d28073200 0x7f1d280756c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d4c15c130 0x7f1d4c1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d4c15bd50 0x7f1d4c0c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 --2- 172.21.15.122:0/4025969053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d4c0cf4e0 0x7f1d4c14e680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 >> 172.21.15.122:0/4025969053 conn(0x7f1d4c0d1ea0 msgr2=0x7f1d4c15a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 shutdown_connections 2024-09-17T05:25:50.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.561+0000 7f1d50c00640 1 -- 172.21.15.122:0/4025969053 wait complete. 2024-09-17T05:25:50.589 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:25:50.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:25:50.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 -- 172.21.15.122:0/3752028904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 msgr2=0x7fa4b00b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/3752028904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 0x7fa4b00b1320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa498009900 tx=0x7fa49801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 -- 172.21.15.122:0/3752028904 shutdown_connections 2024-09-17T05:25:50.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/3752028904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4b00cdf20 0x7fa4b00b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/3752028904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 0x7fa4b00b1320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 -- 172.21.15.122:0/3752028904 >> 172.21.15.122:0/3752028904 conn(0x7fa4b00d1e00 msgr2=0x7fa4b00d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:50.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 -- 172.21.15.122:0/3752028904 shutdown_connections 2024-09-17T05:25:50.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 -- 172.21.15.122:0/3752028904 wait complete. 2024-09-17T05:25:50.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 Processor -- start 2024-09-17T05:25:50.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.733+0000 7fa4b7a00640 1 -- start start 2024-09-17T05:25:50.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 0x7fa4b0151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4b00cdf20 0x7fa4b0159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4b015f340 0x7fa4b0159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0168f80 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0168e00 con 0x7fa4b00cdf20 2024-09-17T05:25:50.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b0169100 con 0x7fa4b015f340 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 0x7fa4b0151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 0x7fa4b0151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41570/0 (socket says 172.21.15.122:41570) 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 -- 172.21.15.122:0/1033975216 learned_addr learned my addr 172.21.15.122:0/1033975216 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5a00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4b015f340 0x7fa4b0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4a7e00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4b00cdf20 0x7fa4b0159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 -- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4b015f340 msgr2=0x7fa4b0159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4b015f340 0x7fa4b0159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 -- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4b00cdf20 msgr2=0x7fa4b0159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:50.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4b00cdf20 0x7fa4b0159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:50.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 -- 172.21.15.122:0/1033975216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b015a4d0 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b5000640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 0x7fa4b0151150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000c3c0 tx=0x7fa4a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:50.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4ad600640 1 -- 172.21.15.122:0/1033975216 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0017070 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4ad600640 1 -- 172.21.15.122:0/1033975216 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4a00053c0 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4ad600640 1 -- 172.21.15.122:0/1033975216 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0003860 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b00b01b0 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b01640f0 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b00b34a0 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.737+0000 7fa4ad600640 1 -- 172.21.15.122:0/1033975216 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4a001a070 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.741+0000 7fa4ad600640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa48c073200 0x7fa48c0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:50.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.741+0000 7fa4ad600640 1 -- 172.21.15.122:0/1033975216 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa4a0093450 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.741+0000 7fa4a7e00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa48c073200 0x7fa48c0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:50.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.741+0000 7fa4a7e00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa48c073200 0x7fa48c0756c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4980af000 tx=0x7fa49801b0f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:50.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.745+0000 7fa4ad600640 1 -- 172.21.15.122:0/1033975216 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa4b015a4d0 con 0x7fa4b00b3bb0 2024-09-17T05:25:50.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:50.949+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa4b00b3970 con 0x7fa4b00b3bb0 2024-09-17T05:25:51.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.561+0000 7fa4ad600640 1 -- 172.21.15.122:0/1033975216 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fa4b00d2c90 con 0x7fa4b00b3bb0 2024-09-17T05:25:51.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa48c073200 msgr2=0x7fa48c0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa48c073200 0x7fa48c0756c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4980af000 tx=0x7fa49801b0f0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 msgr2=0x7fa4b0151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 0x7fa4b0151150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000c3c0 tx=0x7fa4a000c880 comp rx=0 tx=0).stop 2024-09-17T05:25:51.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 shutdown_connections 2024-09-17T05:25:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa48c073200 0x7fa48c0756c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4b015f340 0x7fa4b0159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4b00cdf20 0x7fa4b0159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 --2- 172.21.15.122:0/1033975216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4b00b3bb0 0x7fa4b0151150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 >> 172.21.15.122:0/1033975216 conn(0x7fa4b00d1e00 msgr2=0x7fa4b006e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 shutdown_connections 2024-09-17T05:25:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.565+0000 7fa4b7a00640 1 -- 172.21.15.122:0/1033975216 wait complete. 2024-09-17T05:25:51.588 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:25:51.588 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:51.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 -- 172.21.15.122:0/1270632636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 msgr2=0x7f27fc0ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 --2- 172.21.15.122:0/1270632636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 0x7f27fc0ccfc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009c80 tx=0x7f27f401cda0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 -- 172.21.15.122:0/1270632636 shutdown_connections 2024-09-17T05:25:51.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 --2- 172.21.15.122:0/1270632636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 0x7f27fc0ccfc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 -- 172.21.15.122:0/1270632636 >> 172.21.15.122:0/1270632636 conn(0x7f27fc0d1da0 msgr2=0x7f27fc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:51.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 -- 172.21.15.122:0/1270632636 shutdown_connections 2024-09-17T05:25:51.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 -- 172.21.15.122:0/1270632636 wait complete. 2024-09-17T05:25:51.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 Processor -- start 2024-09-17T05:25:51.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.729+0000 7f2801e00640 1 -- start start 2024-09-17T05:25:51.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 0x7f27fc0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:51.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27fc0cebf0 0x7f27fc15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:51.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27fc15be80 0x7f27fc158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc16a310 con 0x7f27fc0b6400 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc16a190 con 0x7f27fc15be80 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc16a490 con 0x7f27fc0cebf0 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 0x7f27fc0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 0x7f27fc0cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53714/0 (socket says 172.21.15.122:53714) 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 -- 172.21.15.122:0/1584477574 learned_addr learned my addr 172.21.15.122:0/1584477574 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27fb400640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27fc15be80 0x7f27fc158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27f3e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27fc0cebf0 0x7f27fc15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 -- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27fc0cebf0 msgr2=0x7f27fc15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27fc0cebf0 0x7f27fc15a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 -- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27fc15be80 msgr2=0x7f27fc158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27fc15be80 0x7f27fc158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 -- 172.21.15.122:0/1584477574 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc1556b0 con 0x7f27fc0b6400 2024-09-17T05:25:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27faa00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 0x7f27fc0cbad0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00c3c0 tx=0x7f27ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27f3400640 1 -- 172.21.15.122:0/1584477574 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec017070 con 0x7f27fc0b6400 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27f3400640 1 -- 172.21.15.122:0/1584477574 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27ec0053c0 con 0x7f27fc0b6400 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27f3400640 1 -- 172.21.15.122:0/1584477574 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec003860 con 0x7f27fc0b6400 2024-09-17T05:25:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc0cc010 con 0x7f27fc0b6400 2024-09-17T05:25:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc159820 con 0x7f27fc0b6400 2024-09-17T05:25:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc0ce4e0 con 0x7f27fc0b6400 2024-09-17T05:25:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.733+0000 7f27f3400640 1 -- 172.21.15.122:0/1584477574 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27ec01a070 con 0x7f27fc0b6400 2024-09-17T05:25:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.737+0000 7f27f3400640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27d8073200 0x7f27d80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.737+0000 7f27f3400640 1 -- 172.21.15.122:0/1584477574 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f27ec093590 con 0x7f27fc0b6400 2024-09-17T05:25:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.737+0000 7f27f3e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27d8073200 0x7f27d80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.737+0000 7f27f3e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27d8073200 0x7f27d80756c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0c98d0 tx=0x7f27e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:51.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.741+0000 7f27f3400640 1 -- 172.21.15.122:0/1584477574 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f27fc0d0350 con 0x7f27fc0b6400 2024-09-17T05:25:51.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.941+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27fc06f670 con 0x7f27fc0b6400 2024-09-17T05:25:51.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.941+0000 7f27f3400640 1 -- 172.21.15.122:0/1584477574 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5235 (secure 0 0 0) 0x7f27ec05df50 con 0x7f27fc0b6400 2024-09-17T05:25:51.945 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:51.946 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":16,"btime":"2024-09-17T05:25:51:564038+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2936956280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2936956280},{"type":"v1","addr":"172.21.15.173:6836","nonce":2936956280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2308148662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2308148662},{"type":"v1","addr":"172.21.15.122:6834","nonce":2308148662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3699957348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":3699957348},{"type":"v1","addr":"172.21.15.122:6835","nonce":3699957348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:47.923640+0000","modified":"2024-09-17T05:25:50.952955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2399039177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2399039177},{"type":"v1","addr":"172.21.15.173:6837","nonce":2399039177}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4873,"qdb_cluster":[4873]},"id":2}]} 2024-09-17T05:25:51.946 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 16 2024-09-17T05:25:51.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27d8073200 msgr2=0x7f27d80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27d8073200 0x7f27d80756c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0c98d0 tx=0x7f27e4009040 comp rx=0 tx=0).stop 2024-09-17T05:25:51.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 msgr2=0x7f27fc0cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:51.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 0x7f27fc0cbad0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00c3c0 tx=0x7f27ec00c880 comp rx=0 tx=0).stop 2024-09-17T05:25:51.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 shutdown_connections 2024-09-17T05:25:51.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27d8073200 0x7f27d80756c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27fc15be80 0x7f27fc158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27fc0cebf0 0x7f27fc15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 --2- 172.21.15.122:0/1584477574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27fc0b6400 0x7f27fc0cbad0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:51.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 >> 172.21.15.122:0/1584477574 conn(0x7f27fc0d1da0 msgr2=0x7f27fc0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:51.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 shutdown_connections 2024-09-17T05:25:51.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:51.945+0000 7f2801e00640 1 -- 172.21.15.122:0/1584477574 wait complete. 2024-09-17T05:25:51.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:25:52.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.105+0000 7f6dae200640 1 -- 172.21.15.122:0/1046152354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80cfe00 msgr2=0x7f6da815f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.105+0000 7f6dae200640 1 --2- 172.21.15.122:0/1046152354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80cfe00 0x7f6da815f2d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009840 tx=0x7f6d9401c700 comp rx=0 tx=0).stop 2024-09-17T05:25:52.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.105+0000 7f6dae200640 1 -- 172.21.15.122:0/1046152354 shutdown_connections 2024-09-17T05:25:52.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.105+0000 7f6dae200640 1 --2- 172.21.15.122:0/1046152354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80cfe00 0x7f6da815f2d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.105+0000 7f6dae200640 1 -- 172.21.15.122:0/1046152354 >> 172.21.15.122:0/1046152354 conn(0x7f6da80d0d50 msgr2=0x7f6da80d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:52.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.105+0000 7f6dae200640 1 -- 172.21.15.122:0/1046152354 shutdown_connections 2024-09-17T05:25:52.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.105+0000 7f6dae200640 1 -- 172.21.15.122:0/1046152354 wait complete. 2024-09-17T05:25:52.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 Processor -- start 2024-09-17T05:25:52.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 -- start start 2024-09-17T05:25:52.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6da8073880 0x7f6da815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80affd0 0x7f6da814c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da80cfe00 0x7f6da80b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6da80ccf90 con 0x7f6da80affd0 2024-09-17T05:25:52.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6da80cce10 con 0x7f6da80cfe00 2024-09-17T05:25:52.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6da80cd110 con 0x7f6da8073880 2024-09-17T05:25:52.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80affd0 0x7f6da814c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80affd0 0x7f6da814c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53728/0 (socket says 172.21.15.122:53728) 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 -- 172.21.15.122:0/1570317037 learned_addr learned my addr 172.21.15.122:0/1570317037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da7e00640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da80cfe00 0x7f6da80b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da7400640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6da8073880 0x7f6da815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 -- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6da8073880 msgr2=0x7f6da815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6da8073880 0x7f6da815e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 -- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da80cfe00 msgr2=0x7f6da80b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da80cfe00 0x7f6da80b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 -- 172.21.15.122:0/1570317037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da80b3f50 con 0x7f6da80affd0 2024-09-17T05:25:52.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6da6a00640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80affd0 0x7f6da814c3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009790 tx=0x7f6d94004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6d9f400640 1 -- 172.21.15.122:0/1570317037 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d940bc020 con 0x7f6da80affd0 2024-09-17T05:25:52.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6d9f400640 1 -- 172.21.15.122:0/1570317037 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d94005030 con 0x7f6da80affd0 2024-09-17T05:25:52.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6d9f400640 1 -- 172.21.15.122:0/1570317037 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94005460 con 0x7f6da80affd0 2024-09-17T05:25:52.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da8070680 con 0x7f6da80affd0 2024-09-17T05:25:52.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da80b44d0 con 0x7f6da80affd0 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.109+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da815f2d0 con 0x7f6da80affd0 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.113+0000 7f6d9f400640 1 -- 172.21.15.122:0/1570317037 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d940bf020 con 0x7f6da80affd0 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.113+0000 7f6d9f400640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d88073200 0x7f6d880756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.113+0000 7f6d9f400640 1 -- 172.21.15.122:0/1570317037 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6d9412f750 con 0x7f6da80affd0 2024-09-17T05:25:52.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.113+0000 7f6da7400640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d88073200 0x7f6d880756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.113+0000 7f6da7400640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d88073200 0x7f6d880756c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d98007fd0 tx=0x7f6d98007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.117+0000 7f6d9f400640 1 -- 172.21.15.122:0/1570317037 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6d940c6030 con 0x7f6da80affd0 2024-09-17T05:25:52.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.285+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6da8074b20 con 0x7f6da80affd0 2024-09-17T05:25:52.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.285+0000 7f6d9f400640 1 -- 172.21.15.122:0/1570317037 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16729 (secure 0 0 0) 0x7f6d94016410 con 0x7f6da80affd0 2024-09-17T05:25:52.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:52.291 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":33,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:25:49.545657+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:46.061761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:46.889922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d88073200 msgr2=0x7f6d880756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d88073200 0x7f6d880756c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d98007fd0 tx=0x7f6d98007ea0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80affd0 msgr2=0x7f6da814c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80affd0 0x7f6da814c3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009790 tx=0x7f6d94004040 comp rx=0 tx=0).stop 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 shutdown_connections 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d88073200 0x7f6d880756c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da80cfe00 0x7f6da80b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da80affd0 0x7f6da814c3c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 --2- 172.21.15.122:0/1570317037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6da8073880 0x7f6da815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 >> 172.21.15.122:0/1570317037 conn(0x7f6da80d0d50 msgr2=0x7f6da814f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:52.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 shutdown_connections 2024-09-17T05:25:52.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.289+0000 7f6dae200640 1 -- 172.21.15.122:0/1570317037 wait complete. 2024-09-17T05:25:52.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:25:52.312 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 -- 172.21.15.122:0/4041594480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 msgr2=0x7f43180b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 --2- 172.21.15.122:0/4041594480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 0x7f43180b11a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f430c009900 tx=0x7f430c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 -- 172.21.15.122:0/4041594480 shutdown_connections 2024-09-17T05:25:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 --2- 172.21.15.122:0/4041594480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43180b5870 0x7f4318165670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 --2- 172.21.15.122:0/4041594480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 0x7f43180b11a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 -- 172.21.15.122:0/4041594480 >> 172.21.15.122:0/4041594480 conn(0x7f431809aa10 msgr2=0x7f431809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 -- 172.21.15.122:0/4041594480 shutdown_connections 2024-09-17T05:25:52.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 -- 172.21.15.122:0/4041594480 wait complete. 2024-09-17T05:25:52.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 Processor -- start 2024-09-17T05:25:52.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.449+0000 7f431f000640 1 -- start start 2024-09-17T05:25:52.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43180b0ce0 0x7f4318149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 0x7f4318152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43180b5870 0x7f4318155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4318168ad0 con 0x7f43180b4d80 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4318168950 con 0x7f43180b0ce0 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4318168c50 con 0x7f43180b5870 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 0x7f4318152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 0x7f4318152b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53742/0 (socket says 172.21.15.122:53742) 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 -- 172.21.15.122:0/2094337316 learned_addr learned my addr 172.21.15.122:0/2094337316 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 -- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43180b5870 msgr2=0x7f4318155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43180b5870 0x7f4318155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431dc00640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43180b0ce0 0x7f4318149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 -- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43180b0ce0 msgr2=0x7f4318149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43180b0ce0 0x7f4318149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 -- 172.21.15.122:0/2094337316 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43181f5d20 con 0x7f43180b4d80 2024-09-17T05:25:52.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431dc00640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43180b0ce0 0x7f4318149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:52.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431d200640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 0x7f4318152b60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f430400c450 tx=0x7f430400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f4315600640 1 -- 172.21.15.122:0/2094337316 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304012830 con 0x7f43180b4d80 2024-09-17T05:25:52.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f4315600640 1 -- 172.21.15.122:0/2094337316 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4304017070 con 0x7f43180b4d80 2024-09-17T05:25:52.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f4315600640 1 -- 172.21.15.122:0/2094337316 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304012d80 con 0x7f43180b4d80 2024-09-17T05:25:52.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43181f6000 con 0x7f43180b4d80 2024-09-17T05:25:52.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181f65c0 con 0x7f43180b4d80 2024-09-17T05:25:52.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.453+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431814af50 con 0x7f43180b4d80 2024-09-17T05:25:52.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.457+0000 7f4315600640 1 -- 172.21.15.122:0/2094337316 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f430401c460 con 0x7f43180b4d80 2024-09-17T05:25:52.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.457+0000 7f4315600640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42f8073200 0x7f42f80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.457+0000 7f4315600640 1 -- 172.21.15.122:0/2094337316 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4304093fe0 con 0x7f43180b4d80 2024-09-17T05:25:52.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.457+0000 7f431dc00640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42f8073200 0x7f42f80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.457+0000 7f431dc00640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42f8073200 0x7f42f80756c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f430c0af000 tx=0x7f430c01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.461+0000 7f4315600640 1 -- 172.21.15.122:0/2094337316 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f430405f0e0 con 0x7f43180b4d80 2024-09-17T05:25:52.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.629+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43180ce0c0 con 0x7f42f8073200 2024-09-17T05:25:52.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.633+0000 7f4315600640 1 -- 172.21.15.122:0/2094337316 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251525 (secure 0 0 0) 0x7f43180ce0c0 con 0x7f42f8073200 2024-09-17T05:25:52.640 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42f8073200 msgr2=0x7f42f80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42f8073200 0x7f42f80756c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f430c0af000 tx=0x7f430c01b120 comp rx=0 tx=0).stop 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 msgr2=0x7f4318152b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 0x7f4318152b60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f430400c450 tx=0x7f430400c910 comp rx=0 tx=0).stop 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 shutdown_connections 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42f8073200 0x7f42f80756c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43180b5870 0x7f4318155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43180b4d80 0x7f4318152b60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 --2- 172.21.15.122:0/2094337316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43180b0ce0 0x7f4318149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 >> 172.21.15.122:0/2094337316 conn(0x7f431809aa10 msgr2=0x7f431814f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 shutdown_connections 2024-09-17T05:25:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.637+0000 7f431f000640 1 -- 172.21.15.122:0/2094337316 wait complete. 2024-09-17T05:25:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-17T05:25:52.454124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224260,"kb_used_data":3848,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750460,"statfs":{"total":773094113280,"available":772864471040,"internally_reserved":0,"allocated":3940352,"data_stored":2200987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928592"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565313+0000","last_change":"2024-09-17T05:25:48.563496+0000","last_active":"2024-09-17T05:25:48.565313+0000","last_peered":"2024-09-17T05:25:48.565313+0000","last_clean":"2024-09-17T05:25:48.565313+0000","last_became_active":"2024-09-17T05:25:48.563114+0000","last_became_peered":"2024-09-17T05:25:48.563114+0000","last_unstale":"2024-09-17T05:25:48.565313+0000","last_undegraded":"2024-09-17T05:25:48.565313+0000","last_fullsized":"2024-09-17T05:25:48.565313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540092+0000","last_change":"2024-09-17T05:25:47.540092+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540092+0000","last_undegraded":"2024-09-17T05:25:47.540092+0000","last_fullsized":"2024-09-17T05:25:47.540092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.552034+0000","last_change":"2024-09-17T05:25:48.549461+0000","last_active":"2024-09-17T05:25:48.552034+0000","last_peered":"2024-09-17T05:25:48.552034+0000","last_clean":"2024-09-17T05:25:48.552034+0000","last_became_active":"2024-09-17T05:25:48.549168+0000","last_became_peered":"2024-09-17T05:25:48.549168+0000","last_unstale":"2024-09-17T05:25:48.552034+0000","last_undegraded":"2024-09-17T05:25:48.552034+0000","last_fullsized":"2024-09-17T05:25:48.552034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:15.092436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.574864+0000","last_change":"2024-09-17T05:25:48.565056+0000","last_active":"2024-09-17T05:25:48.574864+0000","last_peered":"2024-09-17T05:25:48.574864+0000","last_clean":"2024-09-17T05:25:48.574864+0000","last_became_active":"2024-09-17T05:25:48.564810+0000","last_became_peered":"2024-09-17T05:25:48.564810+0000","last_unstale":"2024-09-17T05:25:48.574864+0000","last_undegraded":"2024-09-17T05:25:48.574864+0000","last_fullsized":"2024-09-17T05:25:48.574864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.541154+0000","last_change":"2024-09-17T05:25:47.541154+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.541154+0000","last_undegraded":"2024-09-17T05:25:47.541154+0000","last_fullsized":"2024-09-17T05:25:47.541154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568106+0000","last_change":"2024-09-17T05:25:48.563410+0000","last_active":"2024-09-17T05:25:48.568106+0000","last_peered":"2024-09-17T05:25:48.568106+0000","last_clean":"2024-09-17T05:25:48.568106+0000","last_became_active":"2024-09-17T05:25:48.563094+0000","last_became_peered":"2024-09-17T05:25:48.563094+0000","last_unstale":"2024-09-17T05:25:48.568106+0000","last_undegraded":"2024-09-17T05:25:48.568106+0000","last_fullsized":"2024-09-17T05:25:48.568106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:47.445644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561093+0000","last_change":"2024-09-17T05:25:48.555613+0000","last_active":"2024-09-17T05:25:48.561093+0000","last_peered":"2024-09-17T05:25:48.561093+0000","last_clean":"2024-09-17T05:25:48.561093+0000","last_became_active":"2024-09-17T05:25:48.555015+0000","last_became_peered":"2024-09-17T05:25:48.555015+0000","last_unstale":"2024-09-17T05:25:48.561093+0000","last_undegraded":"2024-09-17T05:25:48.561093+0000","last_fullsized":"2024-09-17T05:25:48.561093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:05.497779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.576391+0000","last_change":"2024-09-17T05:25:48.571453+0000","last_active":"2024-09-17T05:25:48.576391+0000","last_peered":"2024-09-17T05:25:48.576391+0000","last_clean":"2024-09-17T05:25:48.576391+0000","last_became_active":"2024-09-17T05:25:48.571191+0000","last_became_peered":"2024-09-17T05:25:48.571191+0000","last_unstale":"2024-09-17T05:25:48.576391+0000","last_undegraded":"2024-09-17T05:25:48.576391+0000","last_fullsized":"2024-09-17T05:25:48.576391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566421+0000","last_change":"2024-09-17T05:25:48.563932+0000","last_active":"2024-09-17T05:25:48.566421+0000","last_peered":"2024-09-17T05:25:48.566421+0000","last_clean":"2024-09-17T05:25:48.566421+0000","last_became_active":"2024-09-17T05:25:48.563651+0000","last_became_peered":"2024-09-17T05:25:48.563651+0000","last_unstale":"2024-09-17T05:25:48.566421+0000","last_undegraded":"2024-09-17T05:25:48.566421+0000","last_fullsized":"2024-09-17T05:25:48.566421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:54.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584841+0000","last_change":"2024-09-17T05:25:48.569820+0000","last_active":"2024-09-17T05:25:48.584841+0000","last_peered":"2024-09-17T05:25:48.584841+0000","last_clean":"2024-09-17T05:25:48.584841+0000","last_became_active":"2024-09-17T05:25:48.569549+0000","last_became_peered":"2024-09-17T05:25:48.569549+0000","last_unstale":"2024-09-17T05:25:48.584841+0000","last_undegraded":"2024-09-17T05:25:48.584841+0000","last_fullsized":"2024-09-17T05:25:48.584841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559866+0000","last_change":"2024-09-17T05:25:48.556416+0000","last_active":"2024-09-17T05:25:48.559866+0000","last_peered":"2024-09-17T05:25:48.559866+0000","last_clean":"2024-09-17T05:25:48.559866+0000","last_became_active":"2024-09-17T05:25:48.556113+0000","last_became_peered":"2024-09-17T05:25:48.556113+0000","last_unstale":"2024-09-17T05:25:48.559866+0000","last_undegraded":"2024-09-17T05:25:48.559866+0000","last_fullsized":"2024-09-17T05:25:48.559866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:01.838706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580005+0000","last_change":"2024-09-17T05:25:48.578672+0000","last_active":"2024-09-17T05:25:48.580005+0000","last_peered":"2024-09-17T05:25:48.580005+0000","last_clean":"2024-09-17T05:25:48.580005+0000","last_became_active":"2024-09-17T05:25:48.577799+0000","last_became_peered":"2024-09-17T05:25:48.577799+0000","last_unstale":"2024-09-17T05:25:48.580005+0000","last_undegraded":"2024-09-17T05:25:48.580005+0000","last_fullsized":"2024-09-17T05:25:48.580005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:28.971195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588071+0000","last_change":"2024-09-17T05:25:48.583732+0000","last_active":"2024-09-17T05:25:48.588071+0000","last_peered":"2024-09-17T05:25:48.588071+0000","last_clean":"2024-09-17T05:25:48.588071+0000","last_became_active":"2024-09-17T05:25:48.583467+0000","last_became_peered":"2024-09-17T05:25:48.583467+0000","last_unstale":"2024-09-17T05:25:48.588071+0000","last_undegraded":"2024-09-17T05:25:48.588071+0000","last_fullsized":"2024-09-17T05:25:48.588071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:40.017087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.581449+0000","last_change":"2024-09-17T05:25:48.568509+0000","last_active":"2024-09-17T05:25:48.581449+0000","last_peered":"2024-09-17T05:25:48.581449+0000","last_clean":"2024-09-17T05:25:48.581449+0000","last_became_active":"2024-09-17T05:25:48.568196+0000","last_became_peered":"2024-09-17T05:25:48.568196+0000","last_unstale":"2024-09-17T05:25:48.581449+0000","last_undegraded":"2024-09-17T05:25:48.581449+0000","last_fullsized":"2024-09-17T05:25:48.581449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564320+0000","last_change":"2024-09-17T05:25:48.557116+0000","last_active":"2024-09-17T05:25:48.564320+0000","last_peered":"2024-09-17T05:25:48.564320+0000","last_clean":"2024-09-17T05:25:48.564320+0000","last_became_active":"2024-09-17T05:25:48.556674+0000","last_became_peered":"2024-09-17T05:25:48.556674+0000","last_unstale":"2024-09-17T05:25:48.564320+0000","last_undegraded":"2024-09-17T05:25:48.564320+0000","last_fullsized":"2024-09-17T05:25:48.564320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563788+0000","last_change":"2024-09-17T05:25:48.555359+0000","last_active":"2024-09-17T05:25:48.563788+0000","last_peered":"2024-09-17T05:25:48.563788+0000","last_clean":"2024-09-17T05:25:48.563788+0000","last_became_active":"2024-09-17T05:25:48.554078+0000","last_became_peered":"2024-09-17T05:25:48.554078+0000","last_unstale":"2024-09-17T05:25:48.563788+0000","last_undegraded":"2024-09-17T05:25:48.563788+0000","last_fullsized":"2024-09-17T05:25:48.563788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559662+0000","last_change":"2024-09-17T05:25:48.555168+0000","last_active":"2024-09-17T05:25:48.559662+0000","last_peered":"2024-09-17T05:25:48.559662+0000","last_clean":"2024-09-17T05:25:48.559662+0000","last_became_active":"2024-09-17T05:25:48.554887+0000","last_became_peered":"2024-09-17T05:25:48.554887+0000","last_unstale":"2024-09-17T05:25:48.559662+0000","last_undegraded":"2024-09-17T05:25:48.559662+0000","last_fullsized":"2024-09-17T05:25:48.559662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:18.816281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585276+0000","last_change":"2024-09-17T05:25:48.580714+0000","last_active":"2024-09-17T05:25:48.585276+0000","last_peered":"2024-09-17T05:25:48.585276+0000","last_clean":"2024-09-17T05:25:48.585276+0000","last_became_active":"2024-09-17T05:25:48.580355+0000","last_became_peered":"2024-09-17T05:25:48.580355+0000","last_unstale":"2024-09-17T05:25:48.585276+0000","last_undegraded":"2024-09-17T05:25:48.585276+0000","last_fullsized":"2024-09-17T05:25:48.585276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:13.837862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.567853+0000","last_change":"2024-09-17T05:25:48.559308+0000","last_active":"2024-09-17T05:25:48.567853+0000","last_peered":"2024-09-17T05:25:48.567853+0000","last_clean":"2024-09-17T05:25:48.567853+0000","last_became_active":"2024-09-17T05:25:48.558922+0000","last_became_peered":"2024-09-17T05:25:48.558922+0000","last_unstale":"2024-09-17T05:25:48.567853+0000","last_undegraded":"2024-09-17T05:25:48.567853+0000","last_fullsized":"2024-09-17T05:25:48.567853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:54.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584951+0000","last_change":"2024-09-17T05:25:48.578440+0000","last_active":"2024-09-17T05:25:48.584951+0000","last_peered":"2024-09-17T05:25:48.584951+0000","last_clean":"2024-09-17T05:25:48.584951+0000","last_became_active":"2024-09-17T05:25:48.578166+0000","last_became_peered":"2024-09-17T05:25:48.578166+0000","last_unstale":"2024-09-17T05:25:48.584951+0000","last_undegraded":"2024-09-17T05:25:48.584951+0000","last_fullsized":"2024-09-17T05:25:48.584951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:00.303537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.578195+0000","last_change":"2024-09-17T05:25:48.576041+0000","last_active":"2024-09-17T05:25:48.578195+0000","last_peered":"2024-09-17T05:25:48.578195+0000","last_clean":"2024-09-17T05:25:48.578195+0000","last_became_active":"2024-09-17T05:25:48.575747+0000","last_became_peered":"2024-09-17T05:25:48.575747+0000","last_unstale":"2024-09-17T05:25:48.578195+0000","last_undegraded":"2024-09-17T05:25:48.578195+0000","last_fullsized":"2024-09-17T05:25:48.578195+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566379+0000","last_change":"2024-09-17T05:25:48.564740+0000","last_active":"2024-09-17T05:25:48.566379+0000","last_peered":"2024-09-17T05:25:48.566379+0000","last_clean":"2024-09-17T05:25:48.566379+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:48.566379+0000","last_undegraded":"2024-09-17T05:25:48.566379+0000","last_fullsized":"2024-09-17T05:25:48.566379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.582889+0000","last_change":"2024-09-17T05:25:48.575949+0000","last_active":"2024-09-17T05:25:48.582889+0000","last_peered":"2024-09-17T05:25:48.582889+0000","last_clean":"2024-09-17T05:25:48.582889+0000","last_became_active":"2024-09-17T05:25:48.573427+0000","last_became_peered":"2024-09-17T05:25:48.573427+0000","last_unstale":"2024-09-17T05:25:48.582889+0000","last_undegraded":"2024-09-17T05:25:48.582889+0000","last_fullsized":"2024-09-17T05:25:48.582889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:20.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.570555+0000","last_change":"2024-09-17T05:25:48.566361+0000","last_active":"2024-09-17T05:25:48.570555+0000","last_peered":"2024-09-17T05:25:48.570555+0000","last_clean":"2024-09-17T05:25:48.570555+0000","last_became_active":"2024-09-17T05:25:48.566054+0000","last_became_peered":"2024-09-17T05:25:48.566054+0000","last_unstale":"2024-09-17T05:25:48.570555+0000","last_undegraded":"2024-09-17T05:25:48.570555+0000","last_fullsized":"2024-09-17T05:25:48.570555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.575678+0000","last_change":"2024-09-17T05:25:48.565723+0000","last_active":"2024-09-17T05:25:48.575678+0000","last_peered":"2024-09-17T05:25:48.575678+0000","last_clean":"2024-09-17T05:25:48.575678+0000","last_became_active":"2024-09-17T05:25:48.565457+0000","last_became_peered":"2024-09-17T05:25:48.565457+0000","last_unstale":"2024-09-17T05:25:48.575678+0000","last_undegraded":"2024-09-17T05:25:48.575678+0000","last_fullsized":"2024-09-17T05:25:48.575678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562166+0000","last_change":"2024-09-17T05:25:48.555370+0000","last_active":"2024-09-17T05:25:48.562166+0000","last_peered":"2024-09-17T05:25:48.562166+0000","last_clean":"2024-09-17T05:25:48.562166+0000","last_became_active":"2024-09-17T05:25:48.554994+0000","last_became_peered":"2024-09-17T05:25:48.554994+0000","last_unstale":"2024-09-17T05:25:48.562166+0000","last_undegraded":"2024-09-17T05:25:48.562166+0000","last_fullsized":"2024-09-17T05:25:48.562166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:11.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.541326+0000","last_change":"2024-09-17T05:25:47.541326+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.541326+0000","last_undegraded":"2024-09-17T05:25:47.541326+0000","last_fullsized":"2024-09-17T05:25:47.541326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.579598+0000","last_change":"2024-09-17T05:25:48.576799+0000","last_active":"2024-09-17T05:25:48.579598+0000","last_peered":"2024-09-17T05:25:48.579598+0000","last_clean":"2024-09-17T05:25:48.579598+0000","last_became_active":"2024-09-17T05:25:48.576460+0000","last_became_peered":"2024-09-17T05:25:48.576460+0000","last_unstale":"2024-09-17T05:25:48.579598+0000","last_undegraded":"2024-09-17T05:25:48.579598+0000","last_fullsized":"2024-09-17T05:25:48.579598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563457+0000","last_change":"2024-09-17T05:25:48.555831+0000","last_active":"2024-09-17T05:25:48.563457+0000","last_peered":"2024-09-17T05:25:48.563457+0000","last_clean":"2024-09-17T05:25:48.563457+0000","last_became_active":"2024-09-17T05:25:48.555546+0000","last_became_peered":"2024-09-17T05:25:48.555546+0000","last_unstale":"2024-09-17T05:25:48.563457+0000","last_undegraded":"2024-09-17T05:25:48.563457+0000","last_fullsized":"2024-09-17T05:25:48.563457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563859+0000","last_change":"2024-09-17T05:25:48.554655+0000","last_active":"2024-09-17T05:25:48.563859+0000","last_peered":"2024-09-17T05:25:48.563859+0000","last_clean":"2024-09-17T05:25:48.563859+0000","last_became_active":"2024-09-17T05:25:48.553919+0000","last_became_peered":"2024-09-17T05:25:48.553919+0000","last_unstale":"2024-09-17T05:25:48.563859+0000","last_undegraded":"2024-09-17T05:25:48.563859+0000","last_fullsized":"2024-09-17T05:25:48.563859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564934+0000","last_change":"2024-09-17T05:25:48.560930+0000","last_active":"2024-09-17T05:25:48.564934+0000","last_peered":"2024-09-17T05:25:48.564934+0000","last_clean":"2024-09-17T05:25:48.564934+0000","last_became_active":"2024-09-17T05:25:48.560700+0000","last_became_peered":"2024-09-17T05:25:48.560700+0000","last_unstale":"2024-09-17T05:25:48.564934+0000","last_undegraded":"2024-09-17T05:25:48.564934+0000","last_fullsized":"2024-09-17T05:25:48.564934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:11.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.551685+0000","last_change":"2024-09-17T05:25:48.549526+0000","last_active":"2024-09-17T05:25:48.551685+0000","last_peered":"2024-09-17T05:25:48.551685+0000","last_clean":"2024-09-17T05:25:48.551685+0000","last_became_active":"2024-09-17T05:25:48.549278+0000","last_became_peered":"2024-09-17T05:25:48.549278+0000","last_unstale":"2024-09-17T05:25:48.551685+0000","last_undegraded":"2024-09-17T05:25:48.551685+0000","last_fullsized":"2024-09-17T05:25:48.551685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:40.352200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583989+0000","last_change":"2024-09-17T05:25:48.579263+0000","last_active":"2024-09-17T05:25:48.583989+0000","last_peered":"2024-09-17T05:25:48.583989+0000","last_clean":"2024-09-17T05:25:48.583989+0000","last_became_active":"2024-09-17T05:25:48.578226+0000","last_became_peered":"2024-09-17T05:25:48.578226+0000","last_unstale":"2024-09-17T05:25:48.583989+0000","last_undegraded":"2024-09-17T05:25:48.583989+0000","last_fullsized":"2024-09-17T05:25:48.583989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:49.685150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580051+0000","last_change":"2024-09-17T05:25:47.542992+0000","last_active":"2024-09-17T05:25:48.580051+0000","last_peered":"2024-09-17T05:25:48.580051+0000","last_clean":"2024-09-17T05:25:48.580051+0000","last_became_active":"2024-09-17T05:25:47.542678+0000","last_became_peered":"2024-09-17T05:25:47.542678+0000","last_unstale":"2024-09-17T05:25:48.580051+0000","last_undegraded":"2024-09-17T05:25:48.580051+0000","last_fullsized":"2024-09-17T05:25:48.580051+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:18.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562177+0000","last_change":"2024-09-17T05:25:48.558850+0000","last_active":"2024-09-17T05:25:48.562177+0000","last_peered":"2024-09-17T05:25:48.562177+0000","last_clean":"2024-09-17T05:25:48.562177+0000","last_became_active":"2024-09-17T05:25:48.558570+0000","last_became_peered":"2024-09-17T05:25:48.558570+0000","last_unstale":"2024-09-17T05:25:48.562177+0000","last_undegraded":"2024-09-17T05:25:48.562177+0000","last_fullsized":"2024-09-17T05:25:48.562177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:27.875015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585957+0000","last_change":"2024-09-17T05:25:47.550543+0000","last_active":"2024-09-17T05:25:48.585957+0000","last_peered":"2024-09-17T05:25:48.585957+0000","last_clean":"2024-09-17T05:25:48.585957+0000","last_became_active":"2024-09-17T05:25:47.550230+0000","last_became_peered":"2024-09-17T05:25:47.550230+0000","last_unstale":"2024-09-17T05:25:48.585957+0000","last_undegraded":"2024-09-17T05:25:48.585957+0000","last_fullsized":"2024-09-17T05:25:48.585957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565532+0000","last_change":"2024-09-17T05:25:48.561559+0000","last_active":"2024-09-17T05:25:48.565532+0000","last_peered":"2024-09-17T05:25:48.565532+0000","last_clean":"2024-09-17T05:25:48.565532+0000","last_became_active":"2024-09-17T05:25:48.561199+0000","last_became_peered":"2024-09-17T05:25:48.561199+0000","last_unstale":"2024-09-17T05:25:48.565532+0000","last_undegraded":"2024-09-17T05:25:48.565532+0000","last_fullsized":"2024-09-17T05:25:48.565532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.605791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.581425+0000","last_change":"2024-09-17T05:25:47.555529+0000","last_active":"2024-09-17T05:25:48.581425+0000","last_peered":"2024-09-17T05:25:48.581425+0000","last_clean":"2024-09-17T05:25:48.581425+0000","last_became_active":"2024-09-17T05:25:47.554844+0000","last_became_peered":"2024-09-17T05:25:47.554844+0000","last_unstale":"2024-09-17T05:25:48.581425+0000","last_undegraded":"2024-09-17T05:25:48.581425+0000","last_fullsized":"2024-09-17T05:25:48.581425+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:18.141436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585939+0000","last_change":"2024-09-17T05:25:48.577744+0000","last_active":"2024-09-17T05:25:48.585939+0000","last_peered":"2024-09-17T05:25:48.585939+0000","last_clean":"2024-09-17T05:25:48.585939+0000","last_became_active":"2024-09-17T05:25:48.577492+0000","last_became_peered":"2024-09-17T05:25:48.577492+0000","last_unstale":"2024-09-17T05:25:48.585939+0000","last_undegraded":"2024-09-17T05:25:48.585939+0000","last_fullsized":"2024-09-17T05:25:48.585939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.166901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588295+0000","last_change":"2024-09-17T05:25:47.557387+0000","last_active":"2024-09-17T05:25:48.588295+0000","last_peered":"2024-09-17T05:25:48.588295+0000","last_clean":"2024-09-17T05:25:48.588295+0000","last_became_active":"2024-09-17T05:25:47.557000+0000","last_became_peered":"2024-09-17T05:25:47.557000+0000","last_unstale":"2024-09-17T05:25:48.588295+0000","last_undegraded":"2024-09-17T05:25:48.588295+0000","last_fullsized":"2024-09-17T05:25:48.588295+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:04.492367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566436+0000","last_change":"2024-09-17T05:25:48.563244+0000","last_active":"2024-09-17T05:25:48.566436+0000","last_peered":"2024-09-17T05:25:48.566436+0000","last_clean":"2024-09-17T05:25:48.566436+0000","last_became_active":"2024-09-17T05:25:48.562910+0000","last_became_peered":"2024-09-17T05:25:48.562910+0000","last_unstale":"2024-09-17T05:25:48.566436+0000","last_undegraded":"2024-09-17T05:25:48.566436+0000","last_fullsized":"2024-09-17T05:25:48.566436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:19.146848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.577584+0000","last_change":"2024-09-17T05:25:47.542868+0000","last_active":"2024-09-17T05:25:48.577584+0000","last_peered":"2024-09-17T05:25:48.577584+0000","last_clean":"2024-09-17T05:25:48.577584+0000","last_became_active":"2024-09-17T05:25:47.542655+0000","last_became_peered":"2024-09-17T05:25:47.542655+0000","last_unstale":"2024-09-17T05:25:48.577584+0000","last_undegraded":"2024-09-17T05:25:48.577584+0000","last_fullsized":"2024-09-17T05:25:48.577584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:02.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565297+0000","last_change":"2024-09-17T05:25:48.561829+0000","last_active":"2024-09-17T05:25:48.565297+0000","last_peered":"2024-09-17T05:25:48.565297+0000","last_clean":"2024-09-17T05:25:48.565297+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:48.565297+0000","last_undegraded":"2024-09-17T05:25:48.565297+0000","last_fullsized":"2024-09-17T05:25:48.565297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.577042+0000","last_change":"2024-09-17T05:25:47.552636+0000","last_active":"2024-09-17T05:25:48.577042+0000","last_peered":"2024-09-17T05:25:48.577042+0000","last_clean":"2024-09-17T05:25:48.577042+0000","last_became_active":"2024-09-17T05:25:47.552361+0000","last_became_peered":"2024-09-17T05:25:47.552361+0000","last_unstale":"2024-09-17T05:25:48.577042+0000","last_undegraded":"2024-09-17T05:25:48.577042+0000","last_fullsized":"2024-09-17T05:25:48.577042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:05.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561279+0000","last_change":"2024-09-17T05:25:48.553264+0000","last_active":"2024-09-17T05:25:48.561279+0000","last_peered":"2024-09-17T05:25:48.561279+0000","last_clean":"2024-09-17T05:25:48.561279+0000","last_became_active":"2024-09-17T05:25:48.552992+0000","last_became_peered":"2024-09-17T05:25:48.552992+0000","last_unstale":"2024-09-17T05:25:48.561279+0000","last_undegraded":"2024-09-17T05:25:48.561279+0000","last_fullsized":"2024-09-17T05:25:48.561279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:11.441640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.544893+0000","last_change":"2024-09-17T05:25:47.544385+0000","last_active":"2024-09-17T05:25:48.544893+0000","last_peered":"2024-09-17T05:25:48.544893+0000","last_clean":"2024-09-17T05:25:48.544893+0000","last_became_active":"2024-09-17T05:25:47.544020+0000","last_became_peered":"2024-09-17T05:25:47.544020+0000","last_unstale":"2024-09-17T05:25:48.544893+0000","last_undegraded":"2024-09-17T05:25:48.544893+0000","last_fullsized":"2024-09-17T05:25:48.544893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:03.450940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562607+0000","last_change":"2024-09-17T05:25:48.555818+0000","last_active":"2024-09-17T05:25:48.562607+0000","last_peered":"2024-09-17T05:25:48.562607+0000","last_clean":"2024-09-17T05:25:48.562607+0000","last_became_active":"2024-09-17T05:25:48.555523+0000","last_became_peered":"2024-09-17T05:25:48.555523+0000","last_unstale":"2024-09-17T05:25:48.562607+0000","last_undegraded":"2024-09-17T05:25:48.562607+0000","last_fullsized":"2024-09-17T05:25:48.562607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.605791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.547937+0000","last_change":"2024-09-17T05:25:47.544235+0000","last_active":"2024-09-17T05:25:48.547937+0000","last_peered":"2024-09-17T05:25:48.547937+0000","last_clean":"2024-09-17T05:25:48.547937+0000","last_became_active":"2024-09-17T05:25:47.543878+0000","last_became_peered":"2024-09-17T05:25:47.543878+0000","last_unstale":"2024-09-17T05:25:48.547937+0000","last_undegraded":"2024-09-17T05:25:48.547937+0000","last_fullsized":"2024-09-17T05:25:48.547937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:29.899308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.582300+0000","last_change":"2024-09-17T05:25:48.578808+0000","last_active":"2024-09-17T05:25:48.582300+0000","last_peered":"2024-09-17T05:25:48.582300+0000","last_clean":"2024-09-17T05:25:48.582300+0000","last_became_active":"2024-09-17T05:25:48.578551+0000","last_became_peered":"2024-09-17T05:25:48.578551+0000","last_unstale":"2024-09-17T05:25:48.582300+0000","last_undegraded":"2024-09-17T05:25:48.582300+0000","last_fullsized":"2024-09-17T05:25:48.582300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:32.665784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.557152+0000","last_change":"2024-09-17T05:25:47.556128+0000","last_active":"2024-09-17T05:25:47.557152+0000","last_peered":"2024-09-17T05:25:47.557152+0000","last_clean":"2024-09-17T05:25:47.557152+0000","last_became_active":"2024-09-17T05:25:47.554436+0000","last_became_peered":"2024-09-17T05:25:47.554436+0000","last_unstale":"2024-09-17T05:25:47.557152+0000","last_undegraded":"2024-09-17T05:25:47.557152+0000","last_fullsized":"2024-09-17T05:25:47.557152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:13.641683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584821+0000","last_change":"2024-09-17T05:25:48.569539+0000","last_active":"2024-09-17T05:25:48.584821+0000","last_peered":"2024-09-17T05:25:48.584821+0000","last_clean":"2024-09-17T05:25:48.584821+0000","last_became_active":"2024-09-17T05:25:48.569292+0000","last_became_peered":"2024-09-17T05:25:48.569292+0000","last_unstale":"2024-09-17T05:25:48.584821+0000","last_undegraded":"2024-09-17T05:25:48.584821+0000","last_fullsized":"2024-09-17T05:25:48.584821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.554572+0000","last_change":"2024-09-17T05:25:47.549653+0000","last_active":"2024-09-17T05:25:47.554572+0000","last_peered":"2024-09-17T05:25:47.554572+0000","last_clean":"2024-09-17T05:25:47.554572+0000","last_became_active":"2024-09-17T05:25:47.548638+0000","last_became_peered":"2024-09-17T05:25:47.548638+0000","last_unstale":"2024-09-17T05:25:47.554572+0000","last_undegraded":"2024-09-17T05:25:47.554572+0000","last_fullsized":"2024-09-17T05:25:47.554572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:08.041081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583143+0000","last_change":"2024-09-17T05:25:48.568840+0000","last_active":"2024-09-17T05:25:48.583143+0000","last_peered":"2024-09-17T05:25:48.583143+0000","last_clean":"2024-09-17T05:25:48.583143+0000","last_became_active":"2024-09-17T05:25:48.568324+0000","last_became_peered":"2024-09-17T05:25:48.568324+0000","last_unstale":"2024-09-17T05:25:48.583143+0000","last_undegraded":"2024-09-17T05:25:48.583143+0000","last_fullsized":"2024-09-17T05:25:48.583143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580146+0000","last_change":"2024-09-17T05:25:47.541605+0000","last_active":"2024-09-17T05:25:48.580146+0000","last_peered":"2024-09-17T05:25:48.580146+0000","last_clean":"2024-09-17T05:25:48.580146+0000","last_became_active":"2024-09-17T05:25:47.541312+0000","last_became_peered":"2024-09-17T05:25:47.541312+0000","last_unstale":"2024-09-17T05:25:48.580146+0000","last_undegraded":"2024-09-17T05:25:48.580146+0000","last_fullsized":"2024-09-17T05:25:48.580146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:27.690980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559889+0000","last_change":"2024-09-17T05:25:48.556785+0000","last_active":"2024-09-17T05:25:48.559889+0000","last_peered":"2024-09-17T05:25:48.559889+0000","last_clean":"2024-09-17T05:25:48.559889+0000","last_became_active":"2024-09-17T05:25:48.556443+0000","last_became_peered":"2024-09-17T05:25:48.556443+0000","last_unstale":"2024-09-17T05:25:48.559889+0000","last_undegraded":"2024-09-17T05:25:48.559889+0000","last_fullsized":"2024-09-17T05:25:48.559889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:47.106037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.590749+0000","last_change":"2024-09-17T05:25:47.555429+0000","last_active":"2024-09-17T05:25:48.590749+0000","last_peered":"2024-09-17T05:25:48.590749+0000","last_clean":"2024-09-17T05:25:48.590749+0000","last_became_active":"2024-09-17T05:25:47.555144+0000","last_became_peered":"2024-09-17T05:25:47.555144+0000","last_unstale":"2024-09-17T05:25:48.590749+0000","last_undegraded":"2024-09-17T05:25:48.590749+0000","last_fullsized":"2024-09-17T05:25:48.590749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:24.436569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584084+0000","last_change":"2024-09-17T05:25:48.579894+0000","last_active":"2024-09-17T05:25:48.584084+0000","last_peered":"2024-09-17T05:25:48.584084+0000","last_clean":"2024-09-17T05:25:48.584084+0000","last_became_active":"2024-09-17T05:25:48.578255+0000","last_became_peered":"2024-09-17T05:25:48.578255+0000","last_unstale":"2024-09-17T05:25:48.584084+0000","last_undegraded":"2024-09-17T05:25:48.584084+0000","last_fullsized":"2024-09-17T05:25:48.584084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:58.305009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.579143+0000","last_change":"2024-09-17T05:25:47.548118+0000","last_active":"2024-09-17T05:25:48.579143+0000","last_peered":"2024-09-17T05:25:48.579143+0000","last_clean":"2024-09-17T05:25:48.579143+0000","last_became_active":"2024-09-17T05:25:47.547871+0000","last_became_peered":"2024-09-17T05:25:47.547871+0000","last_unstale":"2024-09-17T05:25:48.579143+0000","last_undegraded":"2024-09-17T05:25:48.579143+0000","last_fullsized":"2024-09-17T05:25:48.579143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585675+0000","last_change":"2024-09-17T05:25:48.582169+0000","last_active":"2024-09-17T05:25:48.585675+0000","last_peered":"2024-09-17T05:25:48.585675+0000","last_clean":"2024-09-17T05:25:48.585675+0000","last_became_active":"2024-09-17T05:25:48.581834+0000","last_became_peered":"2024-09-17T05:25:48.581834+0000","last_unstale":"2024-09-17T05:25:48.585675+0000","last_undegraded":"2024-09-17T05:25:48.585675+0000","last_fullsized":"2024-09-17T05:25:48.585675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:30.341407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.552340+0000","last_change":"2024-09-17T05:25:47.547173+0000","last_active":"2024-09-17T05:25:48.552340+0000","last_peered":"2024-09-17T05:25:48.552340+0000","last_clean":"2024-09-17T05:25:48.552340+0000","last_became_active":"2024-09-17T05:25:47.546912+0000","last_became_peered":"2024-09-17T05:25:47.546912+0000","last_unstale":"2024-09-17T05:25:48.552340+0000","last_undegraded":"2024-09-17T05:25:48.552340+0000","last_fullsized":"2024-09-17T05:25:48.552340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:26.869603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559321+0000","last_change":"2024-09-17T05:25:48.553296+0000","last_active":"2024-09-17T05:25:48.559321+0000","last_peered":"2024-09-17T05:25:48.559321+0000","last_clean":"2024-09-17T05:25:48.559321+0000","last_became_active":"2024-09-17T05:25:48.553058+0000","last_became_peered":"2024-09-17T05:25:48.553058+0000","last_unstale":"2024-09-17T05:25:48.559321+0000","last_undegraded":"2024-09-17T05:25:48.559321+0000","last_fullsized":"2024-09-17T05:25:48.559321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:39.217642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.549156+0000","last_change":"2024-09-17T05:25:47.553779+0000","last_active":"2024-09-17T05:25:48.549156+0000","last_peered":"2024-09-17T05:25:48.549156+0000","last_clean":"2024-09-17T05:25:48.549156+0000","last_became_active":"2024-09-17T05:25:47.553533+0000","last_became_peered":"2024-09-17T05:25:47.553533+0000","last_unstale":"2024-09-17T05:25:48.549156+0000","last_undegraded":"2024-09-17T05:25:48.549156+0000","last_fullsized":"2024-09-17T05:25:48.549156+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.550987+0000","last_change":"2024-09-17T05:25:48.549363+0000","last_active":"2024-09-17T05:25:48.550987+0000","last_peered":"2024-09-17T05:25:48.550987+0000","last_clean":"2024-09-17T05:25:48.550987+0000","last_became_active":"2024-09-17T05:25:48.549077+0000","last_became_peered":"2024-09-17T05:25:48.549077+0000","last_unstale":"2024-09-17T05:25:48.550987+0000","last_undegraded":"2024-09-17T05:25:48.550987+0000","last_fullsized":"2024-09-17T05:25:48.550987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:38.211479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585251+0000","last_change":"2024-09-17T05:25:47.545375+0000","last_active":"2024-09-17T05:25:48.585251+0000","last_peered":"2024-09-17T05:25:48.585251+0000","last_clean":"2024-09-17T05:25:48.585251+0000","last_became_active":"2024-09-17T05:25:47.545098+0000","last_became_peered":"2024-09-17T05:25:47.545098+0000","last_unstale":"2024-09-17T05:25:48.585251+0000","last_undegraded":"2024-09-17T05:25:48.585251+0000","last_fullsized":"2024-09-17T05:25:48.585251+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:10.436228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561228+0000","last_change":"2024-09-17T05:25:48.557134+0000","last_active":"2024-09-17T05:25:48.561228+0000","last_peered":"2024-09-17T05:25:48.561228+0000","last_clean":"2024-09-17T05:25:48.561228+0000","last_became_active":"2024-09-17T05:25:48.556815+0000","last_became_peered":"2024-09-17T05:25:48.556815+0000","last_unstale":"2024-09-17T05:25:48.561228+0000","last_undegraded":"2024-09-17T05:25:48.561228+0000","last_fullsized":"2024-09-17T05:25:48.561228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:41.791223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.586323+0000","last_change":"2024-09-17T05:25:47.551471+0000","last_active":"2024-09-17T05:25:48.586323+0000","last_peered":"2024-09-17T05:25:48.586323+0000","last_clean":"2024-09-17T05:25:48.586323+0000","last_became_active":"2024-09-17T05:25:47.550374+0000","last_became_peered":"2024-09-17T05:25:47.550374+0000","last_unstale":"2024-09-17T05:25:48.586323+0000","last_undegraded":"2024-09-17T05:25:48.586323+0000","last_fullsized":"2024-09-17T05:25:48.586323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:34.028724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561192+0000","last_change":"2024-09-17T05:25:48.559461+0000","last_active":"2024-09-17T05:25:48.561192+0000","last_peered":"2024-09-17T05:25:48.561192+0000","last_clean":"2024-09-17T05:25:48.561192+0000","last_became_active":"2024-09-17T05:25:48.558513+0000","last_became_peered":"2024-09-17T05:25:48.558513+0000","last_unstale":"2024-09-17T05:25:48.561192+0000","last_undegraded":"2024-09-17T05:25:48.561192+0000","last_fullsized":"2024-09-17T05:25:48.561192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:35.034136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.572667+0000","last_change":"2024-09-17T05:25:47.547687+0000","last_active":"2024-09-17T05:25:48.572667+0000","last_peered":"2024-09-17T05:25:48.572667+0000","last_clean":"2024-09-17T05:25:48.572667+0000","last_became_active":"2024-09-17T05:25:47.547426+0000","last_became_peered":"2024-09-17T05:25:47.547426+0000","last_unstale":"2024-09-17T05:25:48.572667+0000","last_undegraded":"2024-09-17T05:25:48.572667+0000","last_fullsized":"2024-09-17T05:25:48.572667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540500+0000","last_change":"2024-09-17T05:25:47.540500+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540500+0000","last_undegraded":"2024-09-17T05:25:47.540500+0000","last_fullsized":"2024-09-17T05:25:47.540500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580070+0000","last_change":"2024-09-17T05:25:47.546486+0000","last_active":"2024-09-17T05:25:48.580070+0000","last_peered":"2024-09-17T05:25:48.580070+0000","last_clean":"2024-09-17T05:25:48.580070+0000","last_became_active":"2024-09-17T05:25:47.545952+0000","last_became_peered":"2024-09-17T05:25:47.545952+0000","last_unstale":"2024-09-17T05:25:48.580070+0000","last_undegraded":"2024-09-17T05:25:48.580070+0000","last_fullsized":"2024-09-17T05:25:48.580070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:17.918336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568072+0000","last_change":"2024-09-17T05:25:48.563376+0000","last_active":"2024-09-17T05:25:48.568072+0000","last_peered":"2024-09-17T05:25:48.568072+0000","last_clean":"2024-09-17T05:25:48.568072+0000","last_became_active":"2024-09-17T05:25:48.563128+0000","last_became_peered":"2024-09-17T05:25:48.563128+0000","last_unstale":"2024-09-17T05:25:48.568072+0000","last_undegraded":"2024-09-17T05:25:48.568072+0000","last_fullsized":"2024-09-17T05:25:48.568072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:29.084922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.547970+0000","last_change":"2024-09-17T05:25:47.550977+0000","last_active":"2024-09-17T05:25:48.547970+0000","last_peered":"2024-09-17T05:25:48.547970+0000","last_clean":"2024-09-17T05:25:48.547970+0000","last_became_active":"2024-09-17T05:25:47.550677+0000","last_became_peered":"2024-09-17T05:25:47.550677+0000","last_unstale":"2024-09-17T05:25:48.547970+0000","last_undegraded":"2024-09-17T05:25:48.547970+0000","last_fullsized":"2024-09-17T05:25:48.547970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:12.600379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583964+0000","last_change":"2024-09-17T05:25:48.578870+0000","last_active":"2024-09-17T05:25:48.583964+0000","last_peered":"2024-09-17T05:25:48.583964+0000","last_clean":"2024-09-17T05:25:48.583964+0000","last_became_active":"2024-09-17T05:25:48.578503+0000","last_became_peered":"2024-09-17T05:25:48.578503+0000","last_unstale":"2024-09-17T05:25:48.583964+0000","last_undegraded":"2024-09-17T05:25:48.583964+0000","last_fullsized":"2024-09-17T05:25:48.583964+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:54.356062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.556401+0000","last_change":"2024-09-17T05:25:47.553626+0000","last_active":"2024-09-17T05:25:47.556401+0000","last_peered":"2024-09-17T05:25:47.556401+0000","last_clean":"2024-09-17T05:25:47.556401+0000","last_became_active":"2024-09-17T05:25:47.553385+0000","last_became_peered":"2024-09-17T05:25:47.553385+0000","last_unstale":"2024-09-17T05:25:47.556401+0000","last_undegraded":"2024-09-17T05:25:47.556401+0000","last_fullsized":"2024-09-17T05:25:47.556401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:59.695362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540601+0000","last_change":"2024-09-17T05:25:47.540601+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540601+0000","last_undegraded":"2024-09-17T05:25:47.540601+0000","last_fullsized":"2024-09-17T05:25:47.540601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.549201+0000","last_change":"2024-09-17T05:25:47.549913+0000","last_active":"2024-09-17T05:25:48.549201+0000","last_peered":"2024-09-17T05:25:48.549201+0000","last_clean":"2024-09-17T05:25:48.549201+0000","last_became_active":"2024-09-17T05:25:47.549593+0000","last_became_peered":"2024-09-17T05:25:47.549593+0000","last_unstale":"2024-09-17T05:25:48.549201+0000","last_undegraded":"2024-09-17T05:25:48.549201+0000","last_fullsized":"2024-09-17T05:25:48.549201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:08.157058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588111+0000","last_change":"2024-09-17T05:25:48.583799+0000","last_active":"2024-09-17T05:25:48.588111+0000","last_peered":"2024-09-17T05:25:48.588111+0000","last_clean":"2024-09-17T05:25:48.588111+0000","last_became_active":"2024-09-17T05:25:48.583489+0000","last_became_peered":"2024-09-17T05:25:48.583489+0000","last_unstale":"2024-09-17T05:25:48.588111+0000","last_undegraded":"2024-09-17T05:25:48.588111+0000","last_fullsized":"2024-09-17T05:25:48.588111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:39.321687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.540541+0000","last_change":"2024-09-17T05:25:47.542635+0000","last_active":"2024-09-17T05:25:48.540541+0000","last_peered":"2024-09-17T05:25:48.540541+0000","last_clean":"2024-09-17T05:25:48.540541+0000","last_became_active":"2024-09-17T05:25:47.542160+0000","last_became_peered":"2024-09-17T05:25:47.542160+0000","last_unstale":"2024-09-17T05:25:48.540541+0000","last_undegraded":"2024-09-17T05:25:48.540541+0000","last_fullsized":"2024-09-17T05:25:48.540541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:33.388849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559087+0000","last_change":"2024-09-17T05:25:48.552387+0000","last_active":"2024-09-17T05:25:48.559087+0000","last_peered":"2024-09-17T05:25:48.559087+0000","last_clean":"2024-09-17T05:25:48.559087+0000","last_became_active":"2024-09-17T05:25:48.552075+0000","last_became_peered":"2024-09-17T05:25:48.552075+0000","last_unstale":"2024-09-17T05:25:48.559087+0000","last_undegraded":"2024-09-17T05:25:48.559087+0000","last_fullsized":"2024-09-17T05:25:48.559087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.456352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.546124+0000","last_change":"2024-09-17T05:25:47.545902+0000","last_active":"2024-09-17T05:25:48.546124+0000","last_peered":"2024-09-17T05:25:48.546124+0000","last_clean":"2024-09-17T05:25:48.546124+0000","last_became_active":"2024-09-17T05:25:47.545602+0000","last_became_peered":"2024-09-17T05:25:47.545602+0000","last_unstale":"2024-09-17T05:25:48.546124+0000","last_undegraded":"2024-09-17T05:25:48.546124+0000","last_fullsized":"2024-09-17T05:25:48.546124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:34.536424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.570355+0000","last_change":"2024-09-17T05:25:48.566441+0000","last_active":"2024-09-17T05:25:48.570355+0000","last_peered":"2024-09-17T05:25:48.570355+0000","last_clean":"2024-09-17T05:25:48.570355+0000","last_became_active":"2024-09-17T05:25:48.566282+0000","last_became_peered":"2024-09-17T05:25:48.566282+0000","last_unstale":"2024-09-17T05:25:48.570355+0000","last_undegraded":"2024-09-17T05:25:48.570355+0000","last_fullsized":"2024-09-17T05:25:48.570355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:25.441981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585494+0000","last_change":"2024-09-17T05:25:47.549606+0000","last_active":"2024-09-17T05:25:48.585494+0000","last_peered":"2024-09-17T05:25:48.585494+0000","last_clean":"2024-09-17T05:25:48.585494+0000","last_became_active":"2024-09-17T05:25:47.549353+0000","last_became_peered":"2024-09-17T05:25:47.549353+0000","last_unstale":"2024-09-17T05:25:48.585494+0000","last_undegraded":"2024-09-17T05:25:48.585494+0000","last_fullsized":"2024-09-17T05:25:48.585494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:46.440233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562140+0000","last_change":"2024-09-17T05:25:48.560961+0000","last_active":"2024-09-17T05:25:48.562140+0000","last_peered":"2024-09-17T05:25:48.562140+0000","last_clean":"2024-09-17T05:25:48.562140+0000","last_became_active":"2024-09-17T05:25:48.560715+0000","last_became_peered":"2024-09-17T05:25:48.560715+0000","last_unstale":"2024-09-17T05:25:48.562140+0000","last_undegraded":"2024-09-17T05:25:48.562140+0000","last_fullsized":"2024-09-17T05:25:48.562140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:49.943382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.556405+0000","last_change":"2024-09-17T05:25:47.552693+0000","last_active":"2024-09-17T05:25:47.556405+0000","last_peered":"2024-09-17T05:25:47.556405+0000","last_clean":"2024-09-17T05:25:47.556405+0000","last_became_active":"2024-09-17T05:25:47.548298+0000","last_became_peered":"2024-09-17T05:25:47.548298+0000","last_unstale":"2024-09-17T05:25:47.556405+0000","last_undegraded":"2024-09-17T05:25:47.556405+0000","last_fullsized":"2024-09-17T05:25:47.556405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:32.326924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568002+0000","last_change":"2024-09-17T05:25:48.559435+0000","last_active":"2024-09-17T05:25:48.568002+0000","last_peered":"2024-09-17T05:25:48.568002+0000","last_clean":"2024-09-17T05:25:48.568002+0000","last_became_active":"2024-09-17T05:25:48.558731+0000","last_became_peered":"2024-09-17T05:25:48.558731+0000","last_unstale":"2024-09-17T05:25:48.568002+0000","last_undegraded":"2024-09-17T05:25:48.568002+0000","last_fullsized":"2024-09-17T05:25:48.568002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:28.971195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.558040+0000","last_change":"2024-09-17T05:25:47.547920+0000","last_active":"2024-09-17T05:25:48.558040+0000","last_peered":"2024-09-17T05:25:48.558040+0000","last_clean":"2024-09-17T05:25:48.558040+0000","last_became_active":"2024-09-17T05:25:47.547593+0000","last_became_peered":"2024-09-17T05:25:47.547593+0000","last_unstale":"2024-09-17T05:25:48.558040+0000","last_undegraded":"2024-09-17T05:25:48.558040+0000","last_fullsized":"2024-09-17T05:25:48.558040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:12.832450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563888+0000","last_change":"2024-09-17T05:25:48.556873+0000","last_active":"2024-09-17T05:25:48.563888+0000","last_peered":"2024-09-17T05:25:48.563888+0000","last_clean":"2024-09-17T05:25:48.563888+0000","last_became_active":"2024-09-17T05:25:48.556481+0000","last_became_peered":"2024-09-17T05:25:48.556481+0000","last_unstale":"2024-09-17T05:25:48.563888+0000","last_undegraded":"2024-09-17T05:25:48.563888+0000","last_fullsized":"2024-09-17T05:25:48.563888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.560890+0000","last_change":"2024-09-17T05:25:47.552565+0000","last_active":"2024-09-17T05:25:48.560890+0000","last_peered":"2024-09-17T05:25:48.560890+0000","last_clean":"2024-09-17T05:25:48.560890+0000","last_became_active":"2024-09-17T05:25:47.549868+0000","last_became_peered":"2024-09-17T05:25:47.549868+0000","last_unstale":"2024-09-17T05:25:48.560890+0000","last_undegraded":"2024-09-17T05:25:48.560890+0000","last_fullsized":"2024-09-17T05:25:48.560890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:45.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585649+0000","last_change":"2024-09-17T05:25:48.579795+0000","last_active":"2024-09-17T05:25:48.585649+0000","last_peered":"2024-09-17T05:25:48.585649+0000","last_clean":"2024-09-17T05:25:48.585649+0000","last_became_active":"2024-09-17T05:25:48.579382+0000","last_became_peered":"2024-09-17T05:25:48.579382+0000","last_unstale":"2024-09-17T05:25:48.585649+0000","last_undegraded":"2024-09-17T05:25:48.585649+0000","last_fullsized":"2024-09-17T05:25:48.585649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:34.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.560830+0000","last_change":"2024-09-17T05:25:47.548934+0000","last_active":"2024-09-17T05:25:48.560830+0000","last_peered":"2024-09-17T05:25:48.560830+0000","last_clean":"2024-09-17T05:25:48.560830+0000","last_became_active":"2024-09-17T05:25:47.548388+0000","last_became_peered":"2024-09-17T05:25:47.548388+0000","last_unstale":"2024-09-17T05:25:48.560830+0000","last_undegraded":"2024-09-17T05:25:48.560830+0000","last_fullsized":"2024-09-17T05:25:48.560830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:32.932862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.575376+0000","last_change":"2024-09-17T05:25:48.565031+0000","last_active":"2024-09-17T05:25:48.575376+0000","last_peered":"2024-09-17T05:25:48.575376+0000","last_clean":"2024-09-17T05:25:48.575376+0000","last_became_active":"2024-09-17T05:25:48.564697+0000","last_became_peered":"2024-09-17T05:25:48.564697+0000","last_unstale":"2024-09-17T05:25:48.575376+0000","last_undegraded":"2024-09-17T05:25:48.575376+0000","last_fullsized":"2024-09-17T05:25:48.575376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.548534+0000","last_change":"2024-09-17T05:25:47.548297+0000","last_active":"2024-09-17T05:25:48.548534+0000","last_peered":"2024-09-17T05:25:48.548534+0000","last_clean":"2024-09-17T05:25:48.548534+0000","last_became_active":"2024-09-17T05:25:47.548063+0000","last_became_peered":"2024-09-17T05:25:47.548063+0000","last_unstale":"2024-09-17T05:25:48.548534+0000","last_undegraded":"2024-09-17T05:25:48.548534+0000","last_fullsized":"2024-09-17T05:25:48.548534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:34.378981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564574+0000","last_change":"2024-09-17T05:25:48.562700+0000","last_active":"2024-09-17T05:25:48.564574+0000","last_peered":"2024-09-17T05:25:48.564574+0000","last_clean":"2024-09-17T05:25:48.564574+0000","last_became_active":"2024-09-17T05:25:48.562183+0000","last_became_peered":"2024-09-17T05:25:48.562183+0000","last_unstale":"2024-09-17T05:25:48.564574+0000","last_undegraded":"2024-09-17T05:25:48.564574+0000","last_fullsized":"2024-09-17T05:25:48.564574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:13.905356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.550500+0000","last_change":"2024-09-17T05:25:47.551652+0000","last_active":"2024-09-17T05:25:48.550500+0000","last_peered":"2024-09-17T05:25:48.550500+0000","last_clean":"2024-09-17T05:25:48.550500+0000","last_became_active":"2024-09-17T05:25:47.551332+0000","last_became_peered":"2024-09-17T05:25:47.551332+0000","last_unstale":"2024-09-17T05:25:48.550500+0000","last_undegraded":"2024-09-17T05:25:48.550500+0000","last_fullsized":"2024-09-17T05:25:48.550500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:38.212230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565013+0000","last_change":"2024-09-17T05:25:48.559526+0000","last_active":"2024-09-17T05:25:48.565013+0000","last_peered":"2024-09-17T05:25:48.565013+0000","last_clean":"2024-09-17T05:25:48.565013+0000","last_became_active":"2024-09-17T05:25:48.559292+0000","last_became_peered":"2024-09-17T05:25:48.559292+0000","last_unstale":"2024-09-17T05:25:48.565013+0000","last_undegraded":"2024-09-17T05:25:48.565013+0000","last_fullsized":"2024-09-17T05:25:48.565013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.545263+0000","last_change":"2024-09-17T05:25:47.546172+0000","last_active":"2024-09-17T05:25:48.545263+0000","last_peered":"2024-09-17T05:25:48.545263+0000","last_clean":"2024-09-17T05:25:48.545263+0000","last_became_active":"2024-09-17T05:25:47.545644+0000","last_became_peered":"2024-09-17T05:25:47.545644+0000","last_unstale":"2024-09-17T05:25:48.545263+0000","last_undegraded":"2024-09-17T05:25:48.545263+0000","last_fullsized":"2024-09-17T05:25:48.545263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:46.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":130,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.541021+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:48.541021+0000","last_peered":"2024-09-17T05:25:48.541021+0000","last_clean":"2024-09-17T05:25:48.541021+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:48.541021+0000","last_undegraded":"2024-09-17T05:25:48.541021+0000","last_fullsized":"2024-09-17T05:25:48.541021+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":368640,"data_stored":134818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":723622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":710502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:52.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:52.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.837+0000 7f6114c00640 1 -- 172.21.15.122:0/2674023072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 msgr2=0x7f61100ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.837+0000 7f6114c00640 1 --2- 172.21.15.122:0/2674023072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 0x7f61100ce5e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f611006d870 tx=0x7f610001cc60 comp rx=0 tx=0).stop 2024-09-17T05:25:52.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.837+0000 7f6114c00640 1 -- 172.21.15.122:0/2674023072 shutdown_connections 2024-09-17T05:25:52.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.837+0000 7f6114c00640 1 --2- 172.21.15.122:0/2674023072 >> v2:172.21.15.173:3300/0 conn(0x7f611015c130 0x7f61100ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.837+0000 7f6114c00640 1 --2- 172.21.15.122:0/2674023072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 0x7f61100ce5e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.837+0000 7f6114c00640 1 -- 172.21.15.122:0/2674023072 >> 172.21.15.122:0/2674023072 conn(0x7f61100d1ea0 msgr2=0x7f61100d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:52.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.837+0000 7f6114c00640 1 -- 172.21.15.122:0/2674023072 shutdown_connections 2024-09-17T05:25:52.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.837+0000 7f6114c00640 1 -- 172.21.15.122:0/2674023072 wait complete. 2024-09-17T05:25:52.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 Processor -- start 2024-09-17T05:25:52.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 -- start start 2024-09-17T05:25:52.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61100cf4e0 0x7f61100cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 0x7f61100b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f611015c130 0x7f611006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6110168e50 con 0x7f611015bd50 2024-09-17T05:25:52.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6110168cd0 con 0x7f611015c130 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6110168fd0 con 0x7f61100cf4e0 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 0x7f61100b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 0x7f61100b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53760/0 (socket says 172.21.15.122:53760) 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 -- 172.21.15.122:0/734183008 learned_addr learned my addr 172.21.15.122:0/734183008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f610de00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61100cf4e0 0x7f61100cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f610e800640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f611015c130 0x7f611006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 -- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61100cf4e0 msgr2=0x7f61100cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61100cf4e0 0x7f61100cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 -- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f611015c130 msgr2=0x7f611006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f611015c130 0x7f611006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:52.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 -- 172.21.15.122:0/734183008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61100c97f0 con 0x7f611015bd50 2024-09-17T05:25:52.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6105400640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 0x7f61100b6f40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005d10 tx=0x7f60f800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6106a00640 1 -- 172.21.15.122:0/734183008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8017070 con 0x7f611015bd50 2024-09-17T05:25:52.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6106a00640 1 -- 172.21.15.122:0/734183008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60f8009b20 con 0x7f611015bd50 2024-09-17T05:25:52.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6106a00640 1 -- 172.21.15.122:0/734183008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8005020 con 0x7f611015bd50 2024-09-17T05:25:52.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61100c9ad0 con 0x7f611015bd50 2024-09-17T05:25:52.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.841+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61100cbba0 con 0x7f611015bd50 2024-09-17T05:25:52.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.845+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61100c5dd0 con 0x7f611015bd50 2024-09-17T05:25:52.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.845+0000 7f6106a00640 1 -- 172.21.15.122:0/734183008 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f60f80051c0 con 0x7f611015bd50 2024-09-17T05:25:52.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.845+0000 7f6106a00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60ec073200 0x7f60ec0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:52.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.845+0000 7f6106a00640 1 -- 172.21.15.122:0/734183008 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f60f8094250 con 0x7f611015bd50 2024-09-17T05:25:52.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.845+0000 7f610de00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60ec073200 0x7f60ec0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:52.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.849+0000 7f610de00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60ec073200 0x7f60ec0756c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61000af000 tx=0x7f610001b4a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:52.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:52.849+0000 7f6106a00640 1 -- 172.21.15.122:0/734183008 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f60f805eca0 con 0x7f611015bd50 2024-09-17T05:25:53.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.013+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61100cfaf0 con 0x7f60ec073200 2024-09-17T05:25:53.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.017+0000 7f6106a00640 1 -- 172.21.15.122:0/734183008 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251525 (secure 0 0 0) 0x7f61100cfaf0 con 0x7f60ec073200 2024-09-17T05:25:53.027 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:53.028 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60ec073200 msgr2=0x7f60ec0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60ec073200 0x7f60ec0756c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f61000af000 tx=0x7f610001b4a0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 msgr2=0x7f61100b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 0x7f61100b6f40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005d10 tx=0x7f60f800dbb0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 shutdown_connections 2024-09-17T05:25:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60ec073200 0x7f60ec0756c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f611015c130 0x7f611006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f611015bd50 0x7f61100b6f40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 --2- 172.21.15.122:0/734183008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61100cf4e0 0x7f61100cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 >> 172.21.15.122:0/734183008 conn(0x7f61100d1ea0 msgr2=0x7f6110071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 shutdown_connections 2024-09-17T05:25:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.021+0000 7f6114c00640 1 -- 172.21.15.122:0/734183008 wait complete. 2024-09-17T05:25:53.045 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-17T05:25:52.454124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224260,"kb_used_data":3848,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750460,"statfs":{"total":773094113280,"available":772864471040,"internally_reserved":0,"allocated":3940352,"data_stored":2200987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928592"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565313+0000","last_change":"2024-09-17T05:25:48.563496+0000","last_active":"2024-09-17T05:25:48.565313+0000","last_peered":"2024-09-17T05:25:48.565313+0000","last_clean":"2024-09-17T05:25:48.565313+0000","last_became_active":"2024-09-17T05:25:48.563114+0000","last_became_peered":"2024-09-17T05:25:48.563114+0000","last_unstale":"2024-09-17T05:25:48.565313+0000","last_undegraded":"2024-09-17T05:25:48.565313+0000","last_fullsized":"2024-09-17T05:25:48.565313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540092+0000","last_change":"2024-09-17T05:25:47.540092+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540092+0000","last_undegraded":"2024-09-17T05:25:47.540092+0000","last_fullsized":"2024-09-17T05:25:47.540092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.552034+0000","last_change":"2024-09-17T05:25:48.549461+0000","last_active":"2024-09-17T05:25:48.552034+0000","last_peered":"2024-09-17T05:25:48.552034+0000","last_clean":"2024-09-17T05:25:48.552034+0000","last_became_active":"2024-09-17T05:25:48.549168+0000","last_became_peered":"2024-09-17T05:25:48.549168+0000","last_unstale":"2024-09-17T05:25:48.552034+0000","last_undegraded":"2024-09-17T05:25:48.552034+0000","last_fullsized":"2024-09-17T05:25:48.552034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:15.092436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.574864+0000","last_change":"2024-09-17T05:25:48.565056+0000","last_active":"2024-09-17T05:25:48.574864+0000","last_peered":"2024-09-17T05:25:48.574864+0000","last_clean":"2024-09-17T05:25:48.574864+0000","last_became_active":"2024-09-17T05:25:48.564810+0000","last_became_peered":"2024-09-17T05:25:48.564810+0000","last_unstale":"2024-09-17T05:25:48.574864+0000","last_undegraded":"2024-09-17T05:25:48.574864+0000","last_fullsized":"2024-09-17T05:25:48.574864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.541154+0000","last_change":"2024-09-17T05:25:47.541154+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.541154+0000","last_undegraded":"2024-09-17T05:25:47.541154+0000","last_fullsized":"2024-09-17T05:25:47.541154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568106+0000","last_change":"2024-09-17T05:25:48.563410+0000","last_active":"2024-09-17T05:25:48.568106+0000","last_peered":"2024-09-17T05:25:48.568106+0000","last_clean":"2024-09-17T05:25:48.568106+0000","last_became_active":"2024-09-17T05:25:48.563094+0000","last_became_peered":"2024-09-17T05:25:48.563094+0000","last_unstale":"2024-09-17T05:25:48.568106+0000","last_undegraded":"2024-09-17T05:25:48.568106+0000","last_fullsized":"2024-09-17T05:25:48.568106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:47.445644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561093+0000","last_change":"2024-09-17T05:25:48.555613+0000","last_active":"2024-09-17T05:25:48.561093+0000","last_peered":"2024-09-17T05:25:48.561093+0000","last_clean":"2024-09-17T05:25:48.561093+0000","last_became_active":"2024-09-17T05:25:48.555015+0000","last_became_peered":"2024-09-17T05:25:48.555015+0000","last_unstale":"2024-09-17T05:25:48.561093+0000","last_undegraded":"2024-09-17T05:25:48.561093+0000","last_fullsized":"2024-09-17T05:25:48.561093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:05.497779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.576391+0000","last_change":"2024-09-17T05:25:48.571453+0000","last_active":"2024-09-17T05:25:48.576391+0000","last_peered":"2024-09-17T05:25:48.576391+0000","last_clean":"2024-09-17T05:25:48.576391+0000","last_became_active":"2024-09-17T05:25:48.571191+0000","last_became_peered":"2024-09-17T05:25:48.571191+0000","last_unstale":"2024-09-17T05:25:48.576391+0000","last_undegraded":"2024-09-17T05:25:48.576391+0000","last_fullsized":"2024-09-17T05:25:48.576391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566421+0000","last_change":"2024-09-17T05:25:48.563932+0000","last_active":"2024-09-17T05:25:48.566421+0000","last_peered":"2024-09-17T05:25:48.566421+0000","last_clean":"2024-09-17T05:25:48.566421+0000","last_became_active":"2024-09-17T05:25:48.563651+0000","last_became_peered":"2024-09-17T05:25:48.563651+0000","last_unstale":"2024-09-17T05:25:48.566421+0000","last_undegraded":"2024-09-17T05:25:48.566421+0000","last_fullsized":"2024-09-17T05:25:48.566421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:54.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584841+0000","last_change":"2024-09-17T05:25:48.569820+0000","last_active":"2024-09-17T05:25:48.584841+0000","last_peered":"2024-09-17T05:25:48.584841+0000","last_clean":"2024-09-17T05:25:48.584841+0000","last_became_active":"2024-09-17T05:25:48.569549+0000","last_became_peered":"2024-09-17T05:25:48.569549+0000","last_unstale":"2024-09-17T05:25:48.584841+0000","last_undegraded":"2024-09-17T05:25:48.584841+0000","last_fullsized":"2024-09-17T05:25:48.584841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559866+0000","last_change":"2024-09-17T05:25:48.556416+0000","last_active":"2024-09-17T05:25:48.559866+0000","last_peered":"2024-09-17T05:25:48.559866+0000","last_clean":"2024-09-17T05:25:48.559866+0000","last_became_active":"2024-09-17T05:25:48.556113+0000","last_became_peered":"2024-09-17T05:25:48.556113+0000","last_unstale":"2024-09-17T05:25:48.559866+0000","last_undegraded":"2024-09-17T05:25:48.559866+0000","last_fullsized":"2024-09-17T05:25:48.559866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:01.838706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580005+0000","last_change":"2024-09-17T05:25:48.578672+0000","last_active":"2024-09-17T05:25:48.580005+0000","last_peered":"2024-09-17T05:25:48.580005+0000","last_clean":"2024-09-17T05:25:48.580005+0000","last_became_active":"2024-09-17T05:25:48.577799+0000","last_became_peered":"2024-09-17T05:25:48.577799+0000","last_unstale":"2024-09-17T05:25:48.580005+0000","last_undegraded":"2024-09-17T05:25:48.580005+0000","last_fullsized":"2024-09-17T05:25:48.580005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:28.971195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588071+0000","last_change":"2024-09-17T05:25:48.583732+0000","last_active":"2024-09-17T05:25:48.588071+0000","last_peered":"2024-09-17T05:25:48.588071+0000","last_clean":"2024-09-17T05:25:48.588071+0000","last_became_active":"2024-09-17T05:25:48.583467+0000","last_became_peered":"2024-09-17T05:25:48.583467+0000","last_unstale":"2024-09-17T05:25:48.588071+0000","last_undegraded":"2024-09-17T05:25:48.588071+0000","last_fullsized":"2024-09-17T05:25:48.588071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:40.017087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.581449+0000","last_change":"2024-09-17T05:25:48.568509+0000","last_active":"2024-09-17T05:25:48.581449+0000","last_peered":"2024-09-17T05:25:48.581449+0000","last_clean":"2024-09-17T05:25:48.581449+0000","last_became_active":"2024-09-17T05:25:48.568196+0000","last_became_peered":"2024-09-17T05:25:48.568196+0000","last_unstale":"2024-09-17T05:25:48.581449+0000","last_undegraded":"2024-09-17T05:25:48.581449+0000","last_fullsized":"2024-09-17T05:25:48.581449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564320+0000","last_change":"2024-09-17T05:25:48.557116+0000","last_active":"2024-09-17T05:25:48.564320+0000","last_peered":"2024-09-17T05:25:48.564320+0000","last_clean":"2024-09-17T05:25:48.564320+0000","last_became_active":"2024-09-17T05:25:48.556674+0000","last_became_peered":"2024-09-17T05:25:48.556674+0000","last_unstale":"2024-09-17T05:25:48.564320+0000","last_undegraded":"2024-09-17T05:25:48.564320+0000","last_fullsized":"2024-09-17T05:25:48.564320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563788+0000","last_change":"2024-09-17T05:25:48.555359+0000","last_active":"2024-09-17T05:25:48.563788+0000","last_peered":"2024-09-17T05:25:48.563788+0000","last_clean":"2024-09-17T05:25:48.563788+0000","last_became_active":"2024-09-17T05:25:48.554078+0000","last_became_peered":"2024-09-17T05:25:48.554078+0000","last_unstale":"2024-09-17T05:25:48.563788+0000","last_undegraded":"2024-09-17T05:25:48.563788+0000","last_fullsized":"2024-09-17T05:25:48.563788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559662+0000","last_change":"2024-09-17T05:25:48.555168+0000","last_active":"2024-09-17T05:25:48.559662+0000","last_peered":"2024-09-17T05:25:48.559662+0000","last_clean":"2024-09-17T05:25:48.559662+0000","last_became_active":"2024-09-17T05:25:48.554887+0000","last_became_peered":"2024-09-17T05:25:48.554887+0000","last_unstale":"2024-09-17T05:25:48.559662+0000","last_undegraded":"2024-09-17T05:25:48.559662+0000","last_fullsized":"2024-09-17T05:25:48.559662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:18.816281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585276+0000","last_change":"2024-09-17T05:25:48.580714+0000","last_active":"2024-09-17T05:25:48.585276+0000","last_peered":"2024-09-17T05:25:48.585276+0000","last_clean":"2024-09-17T05:25:48.585276+0000","last_became_active":"2024-09-17T05:25:48.580355+0000","last_became_peered":"2024-09-17T05:25:48.580355+0000","last_unstale":"2024-09-17T05:25:48.585276+0000","last_undegraded":"2024-09-17T05:25:48.585276+0000","last_fullsized":"2024-09-17T05:25:48.585276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:13.837862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.567853+0000","last_change":"2024-09-17T05:25:48.559308+0000","last_active":"2024-09-17T05:25:48.567853+0000","last_peered":"2024-09-17T05:25:48.567853+0000","last_clean":"2024-09-17T05:25:48.567853+0000","last_became_active":"2024-09-17T05:25:48.558922+0000","last_became_peered":"2024-09-17T05:25:48.558922+0000","last_unstale":"2024-09-17T05:25:48.567853+0000","last_undegraded":"2024-09-17T05:25:48.567853+0000","last_fullsized":"2024-09-17T05:25:48.567853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:54.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584951+0000","last_change":"2024-09-17T05:25:48.578440+0000","last_active":"2024-09-17T05:25:48.584951+0000","last_peered":"2024-09-17T05:25:48.584951+0000","last_clean":"2024-09-17T05:25:48.584951+0000","last_became_active":"2024-09-17T05:25:48.578166+0000","last_became_peered":"2024-09-17T05:25:48.578166+0000","last_unstale":"2024-09-17T05:25:48.584951+0000","last_undegraded":"2024-09-17T05:25:48.584951+0000","last_fullsized":"2024-09-17T05:25:48.584951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:00.303537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.578195+0000","last_change":"2024-09-17T05:25:48.576041+0000","last_active":"2024-09-17T05:25:48.578195+0000","last_peered":"2024-09-17T05:25:48.578195+0000","last_clean":"2024-09-17T05:25:48.578195+0000","last_became_active":"2024-09-17T05:25:48.575747+0000","last_became_peered":"2024-09-17T05:25:48.575747+0000","last_unstale":"2024-09-17T05:25:48.578195+0000","last_undegraded":"2024-09-17T05:25:48.578195+0000","last_fullsized":"2024-09-17T05:25:48.578195+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566379+0000","last_change":"2024-09-17T05:25:48.564740+0000","last_active":"2024-09-17T05:25:48.566379+0000","last_peered":"2024-09-17T05:25:48.566379+0000","last_clean":"2024-09-17T05:25:48.566379+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:48.566379+0000","last_undegraded":"2024-09-17T05:25:48.566379+0000","last_fullsized":"2024-09-17T05:25:48.566379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.582889+0000","last_change":"2024-09-17T05:25:48.575949+0000","last_active":"2024-09-17T05:25:48.582889+0000","last_peered":"2024-09-17T05:25:48.582889+0000","last_clean":"2024-09-17T05:25:48.582889+0000","last_became_active":"2024-09-17T05:25:48.573427+0000","last_became_peered":"2024-09-17T05:25:48.573427+0000","last_unstale":"2024-09-17T05:25:48.582889+0000","last_undegraded":"2024-09-17T05:25:48.582889+0000","last_fullsized":"2024-09-17T05:25:48.582889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:20.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.570555+0000","last_change":"2024-09-17T05:25:48.566361+0000","last_active":"2024-09-17T05:25:48.570555+0000","last_peered":"2024-09-17T05:25:48.570555+0000","last_clean":"2024-09-17T05:25:48.570555+0000","last_became_active":"2024-09-17T05:25:48.566054+0000","last_became_peered":"2024-09-17T05:25:48.566054+0000","last_unstale":"2024-09-17T05:25:48.570555+0000","last_undegraded":"2024-09-17T05:25:48.570555+0000","last_fullsized":"2024-09-17T05:25:48.570555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.575678+0000","last_change":"2024-09-17T05:25:48.565723+0000","last_active":"2024-09-17T05:25:48.575678+0000","last_peered":"2024-09-17T05:25:48.575678+0000","last_clean":"2024-09-17T05:25:48.575678+0000","last_became_active":"2024-09-17T05:25:48.565457+0000","last_became_peered":"2024-09-17T05:25:48.565457+0000","last_unstale":"2024-09-17T05:25:48.575678+0000","last_undegraded":"2024-09-17T05:25:48.575678+0000","last_fullsized":"2024-09-17T05:25:48.575678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562166+0000","last_change":"2024-09-17T05:25:48.555370+0000","last_active":"2024-09-17T05:25:48.562166+0000","last_peered":"2024-09-17T05:25:48.562166+0000","last_clean":"2024-09-17T05:25:48.562166+0000","last_became_active":"2024-09-17T05:25:48.554994+0000","last_became_peered":"2024-09-17T05:25:48.554994+0000","last_unstale":"2024-09-17T05:25:48.562166+0000","last_undegraded":"2024-09-17T05:25:48.562166+0000","last_fullsized":"2024-09-17T05:25:48.562166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:11.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.541326+0000","last_change":"2024-09-17T05:25:47.541326+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.541326+0000","last_undegraded":"2024-09-17T05:25:47.541326+0000","last_fullsized":"2024-09-17T05:25:47.541326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.579598+0000","last_change":"2024-09-17T05:25:48.576799+0000","last_active":"2024-09-17T05:25:48.579598+0000","last_peered":"2024-09-17T05:25:48.579598+0000","last_clean":"2024-09-17T05:25:48.579598+0000","last_became_active":"2024-09-17T05:25:48.576460+0000","last_became_peered":"2024-09-17T05:25:48.576460+0000","last_unstale":"2024-09-17T05:25:48.579598+0000","last_undegraded":"2024-09-17T05:25:48.579598+0000","last_fullsized":"2024-09-17T05:25:48.579598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563457+0000","last_change":"2024-09-17T05:25:48.555831+0000","last_active":"2024-09-17T05:25:48.563457+0000","last_peered":"2024-09-17T05:25:48.563457+0000","last_clean":"2024-09-17T05:25:48.563457+0000","last_became_active":"2024-09-17T05:25:48.555546+0000","last_became_peered":"2024-09-17T05:25:48.555546+0000","last_unstale":"2024-09-17T05:25:48.563457+0000","last_undegraded":"2024-09-17T05:25:48.563457+0000","last_fullsized":"2024-09-17T05:25:48.563457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563859+0000","last_change":"2024-09-17T05:25:48.554655+0000","last_active":"2024-09-17T05:25:48.563859+0000","last_peered":"2024-09-17T05:25:48.563859+0000","last_clean":"2024-09-17T05:25:48.563859+0000","last_became_active":"2024-09-17T05:25:48.553919+0000","last_became_peered":"2024-09-17T05:25:48.553919+0000","last_unstale":"2024-09-17T05:25:48.563859+0000","last_undegraded":"2024-09-17T05:25:48.563859+0000","last_fullsized":"2024-09-17T05:25:48.563859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564934+0000","last_change":"2024-09-17T05:25:48.560930+0000","last_active":"2024-09-17T05:25:48.564934+0000","last_peered":"2024-09-17T05:25:48.564934+0000","last_clean":"2024-09-17T05:25:48.564934+0000","last_became_active":"2024-09-17T05:25:48.560700+0000","last_became_peered":"2024-09-17T05:25:48.560700+0000","last_unstale":"2024-09-17T05:25:48.564934+0000","last_undegraded":"2024-09-17T05:25:48.564934+0000","last_fullsized":"2024-09-17T05:25:48.564934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:11.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.551685+0000","last_change":"2024-09-17T05:25:48.549526+0000","last_active":"2024-09-17T05:25:48.551685+0000","last_peered":"2024-09-17T05:25:48.551685+0000","last_clean":"2024-09-17T05:25:48.551685+0000","last_became_active":"2024-09-17T05:25:48.549278+0000","last_became_peered":"2024-09-17T05:25:48.549278+0000","last_unstale":"2024-09-17T05:25:48.551685+0000","last_undegraded":"2024-09-17T05:25:48.551685+0000","last_fullsized":"2024-09-17T05:25:48.551685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:40.352200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583989+0000","last_change":"2024-09-17T05:25:48.579263+0000","last_active":"2024-09-17T05:25:48.583989+0000","last_peered":"2024-09-17T05:25:48.583989+0000","last_clean":"2024-09-17T05:25:48.583989+0000","last_became_active":"2024-09-17T05:25:48.578226+0000","last_became_peered":"2024-09-17T05:25:48.578226+0000","last_unstale":"2024-09-17T05:25:48.583989+0000","last_undegraded":"2024-09-17T05:25:48.583989+0000","last_fullsized":"2024-09-17T05:25:48.583989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:49.685150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580051+0000","last_change":"2024-09-17T05:25:47.542992+0000","last_active":"2024-09-17T05:25:48.580051+0000","last_peered":"2024-09-17T05:25:48.580051+0000","last_clean":"2024-09-17T05:25:48.580051+0000","last_became_active":"2024-09-17T05:25:47.542678+0000","last_became_peered":"2024-09-17T05:25:47.542678+0000","last_unstale":"2024-09-17T05:25:48.580051+0000","last_undegraded":"2024-09-17T05:25:48.580051+0000","last_fullsized":"2024-09-17T05:25:48.580051+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:18.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562177+0000","last_change":"2024-09-17T05:25:48.558850+0000","last_active":"2024-09-17T05:25:48.562177+0000","last_peered":"2024-09-17T05:25:48.562177+0000","last_clean":"2024-09-17T05:25:48.562177+0000","last_became_active":"2024-09-17T05:25:48.558570+0000","last_became_peered":"2024-09-17T05:25:48.558570+0000","last_unstale":"2024-09-17T05:25:48.562177+0000","last_undegraded":"2024-09-17T05:25:48.562177+0000","last_fullsized":"2024-09-17T05:25:48.562177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:27.875015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585957+0000","last_change":"2024-09-17T05:25:47.550543+0000","last_active":"2024-09-17T05:25:48.585957+0000","last_peered":"2024-09-17T05:25:48.585957+0000","last_clean":"2024-09-17T05:25:48.585957+0000","last_became_active":"2024-09-17T05:25:47.550230+0000","last_became_peered":"2024-09-17T05:25:47.550230+0000","last_unstale":"2024-09-17T05:25:48.585957+0000","last_undegraded":"2024-09-17T05:25:48.585957+0000","last_fullsized":"2024-09-17T05:25:48.585957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565532+0000","last_change":"2024-09-17T05:25:48.561559+0000","last_active":"2024-09-17T05:25:48.565532+0000","last_peered":"2024-09-17T05:25:48.565532+0000","last_clean":"2024-09-17T05:25:48.565532+0000","last_became_active":"2024-09-17T05:25:48.561199+0000","last_became_peered":"2024-09-17T05:25:48.561199+0000","last_unstale":"2024-09-17T05:25:48.565532+0000","last_undegraded":"2024-09-17T05:25:48.565532+0000","last_fullsized":"2024-09-17T05:25:48.565532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.605791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.581425+0000","last_change":"2024-09-17T05:25:47.555529+0000","last_active":"2024-09-17T05:25:48.581425+0000","last_peered":"2024-09-17T05:25:48.581425+0000","last_clean":"2024-09-17T05:25:48.581425+0000","last_became_active":"2024-09-17T05:25:47.554844+0000","last_became_peered":"2024-09-17T05:25:47.554844+0000","last_unstale":"2024-09-17T05:25:48.581425+0000","last_undegraded":"2024-09-17T05:25:48.581425+0000","last_fullsized":"2024-09-17T05:25:48.581425+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:18.141436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585939+0000","last_change":"2024-09-17T05:25:48.577744+0000","last_active":"2024-09-17T05:25:48.585939+0000","last_peered":"2024-09-17T05:25:48.585939+0000","last_clean":"2024-09-17T05:25:48.585939+0000","last_became_active":"2024-09-17T05:25:48.577492+0000","last_became_peered":"2024-09-17T05:25:48.577492+0000","last_unstale":"2024-09-17T05:25:48.585939+0000","last_undegraded":"2024-09-17T05:25:48.585939+0000","last_fullsized":"2024-09-17T05:25:48.585939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.166901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588295+0000","last_change":"2024-09-17T05:25:47.557387+0000","last_active":"2024-09-17T05:25:48.588295+0000","last_peered":"2024-09-17T05:25:48.588295+0000","last_clean":"2024-09-17T05:25:48.588295+0000","last_became_active":"2024-09-17T05:25:47.557000+0000","last_became_peered":"2024-09-17T05:25:47.557000+0000","last_unstale":"2024-09-17T05:25:48.588295+0000","last_undegraded":"2024-09-17T05:25:48.588295+0000","last_fullsized":"2024-09-17T05:25:48.588295+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:04.492367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566436+0000","last_change":"2024-09-17T05:25:48.563244+0000","last_active":"2024-09-17T05:25:48.566436+0000","last_peered":"2024-09-17T05:25:48.566436+0000","last_clean":"2024-09-17T05:25:48.566436+0000","last_became_active":"2024-09-17T05:25:48.562910+0000","last_became_peered":"2024-09-17T05:25:48.562910+0000","last_unstale":"2024-09-17T05:25:48.566436+0000","last_undegraded":"2024-09-17T05:25:48.566436+0000","last_fullsized":"2024-09-17T05:25:48.566436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:19.146848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.577584+0000","last_change":"2024-09-17T05:25:47.542868+0000","last_active":"2024-09-17T05:25:48.577584+0000","last_peered":"2024-09-17T05:25:48.577584+0000","last_clean":"2024-09-17T05:25:48.577584+0000","last_became_active":"2024-09-17T05:25:47.542655+0000","last_became_peered":"2024-09-17T05:25:47.542655+0000","last_unstale":"2024-09-17T05:25:48.577584+0000","last_undegraded":"2024-09-17T05:25:48.577584+0000","last_fullsized":"2024-09-17T05:25:48.577584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:02.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565297+0000","last_change":"2024-09-17T05:25:48.561829+0000","last_active":"2024-09-17T05:25:48.565297+0000","last_peered":"2024-09-17T05:25:48.565297+0000","last_clean":"2024-09-17T05:25:48.565297+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:48.565297+0000","last_undegraded":"2024-09-17T05:25:48.565297+0000","last_fullsized":"2024-09-17T05:25:48.565297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.577042+0000","last_change":"2024-09-17T05:25:47.552636+0000","last_active":"2024-09-17T05:25:48.577042+0000","last_peered":"2024-09-17T05:25:48.577042+0000","last_clean":"2024-09-17T05:25:48.577042+0000","last_became_active":"2024-09-17T05:25:47.552361+0000","last_became_peered":"2024-09-17T05:25:47.552361+0000","last_unstale":"2024-09-17T05:25:48.577042+0000","last_undegraded":"2024-09-17T05:25:48.577042+0000","last_fullsized":"2024-09-17T05:25:48.577042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:05.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561279+0000","last_change":"2024-09-17T05:25:48.553264+0000","last_active":"2024-09-17T05:25:48.561279+0000","last_peered":"2024-09-17T05:25:48.561279+0000","last_clean":"2024-09-17T05:25:48.561279+0000","last_became_active":"2024-09-17T05:25:48.552992+0000","last_became_peered":"2024-09-17T05:25:48.552992+0000","last_unstale":"2024-09-17T05:25:48.561279+0000","last_undegraded":"2024-09-17T05:25:48.561279+0000","last_fullsized":"2024-09-17T05:25:48.561279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:11.441640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.544893+0000","last_change":"2024-09-17T05:25:47.544385+0000","last_active":"2024-09-17T05:25:48.544893+0000","last_peered":"2024-09-17T05:25:48.544893+0000","last_clean":"2024-09-17T05:25:48.544893+0000","last_became_active":"2024-09-17T05:25:47.544020+0000","last_became_peered":"2024-09-17T05:25:47.544020+0000","last_unstale":"2024-09-17T05:25:48.544893+0000","last_undegraded":"2024-09-17T05:25:48.544893+0000","last_fullsized":"2024-09-17T05:25:48.544893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:03.450940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562607+0000","last_change":"2024-09-17T05:25:48.555818+0000","last_active":"2024-09-17T05:25:48.562607+0000","last_peered":"2024-09-17T05:25:48.562607+0000","last_clean":"2024-09-17T05:25:48.562607+0000","last_became_active":"2024-09-17T05:25:48.555523+0000","last_became_peered":"2024-09-17T05:25:48.555523+0000","last_unstale":"2024-09-17T05:25:48.562607+0000","last_undegraded":"2024-09-17T05:25:48.562607+0000","last_fullsized":"2024-09-17T05:25:48.562607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.605791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.547937+0000","last_change":"2024-09-17T05:25:47.544235+0000","last_active":"2024-09-17T05:25:48.547937+0000","last_peered":"2024-09-17T05:25:48.547937+0000","last_clean":"2024-09-17T05:25:48.547937+0000","last_became_active":"2024-09-17T05:25:47.543878+0000","last_became_peered":"2024-09-17T05:25:47.543878+0000","last_unstale":"2024-09-17T05:25:48.547937+0000","last_undegraded":"2024-09-17T05:25:48.547937+0000","last_fullsized":"2024-09-17T05:25:48.547937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:29.899308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.582300+0000","last_change":"2024-09-17T05:25:48.578808+0000","last_active":"2024-09-17T05:25:48.582300+0000","last_peered":"2024-09-17T05:25:48.582300+0000","last_clean":"2024-09-17T05:25:48.582300+0000","last_became_active":"2024-09-17T05:25:48.578551+0000","last_became_peered":"2024-09-17T05:25:48.578551+0000","last_unstale":"2024-09-17T05:25:48.582300+0000","last_undegraded":"2024-09-17T05:25:48.582300+0000","last_fullsized":"2024-09-17T05:25:48.582300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:32.665784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.557152+0000","last_change":"2024-09-17T05:25:47.556128+0000","last_active":"2024-09-17T05:25:47.557152+0000","last_peered":"2024-09-17T05:25:47.557152+0000","last_clean":"2024-09-17T05:25:47.557152+0000","last_became_active":"2024-09-17T05:25:47.554436+0000","last_became_peered":"2024-09-17T05:25:47.554436+0000","last_unstale":"2024-09-17T05:25:47.557152+0000","last_undegraded":"2024-09-17T05:25:47.557152+0000","last_fullsized":"2024-09-17T05:25:47.557152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:13.641683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584821+0000","last_change":"2024-09-17T05:25:48.569539+0000","last_active":"2024-09-17T05:25:48.584821+0000","last_peered":"2024-09-17T05:25:48.584821+0000","last_clean":"2024-09-17T05:25:48.584821+0000","last_became_active":"2024-09-17T05:25:48.569292+0000","last_became_peered":"2024-09-17T05:25:48.569292+0000","last_unstale":"2024-09-17T05:25:48.584821+0000","last_undegraded":"2024-09-17T05:25:48.584821+0000","last_fullsized":"2024-09-17T05:25:48.584821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.554572+0000","last_change":"2024-09-17T05:25:47.549653+0000","last_active":"2024-09-17T05:25:47.554572+0000","last_peered":"2024-09-17T05:25:47.554572+0000","last_clean":"2024-09-17T05:25:47.554572+0000","last_became_active":"2024-09-17T05:25:47.548638+0000","last_became_peered":"2024-09-17T05:25:47.548638+0000","last_unstale":"2024-09-17T05:25:47.554572+0000","last_undegraded":"2024-09-17T05:25:47.554572+0000","last_fullsized":"2024-09-17T05:25:47.554572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:08.041081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583143+0000","last_change":"2024-09-17T05:25:48.568840+0000","last_active":"2024-09-17T05:25:48.583143+0000","last_peered":"2024-09-17T05:25:48.583143+0000","last_clean":"2024-09-17T05:25:48.583143+0000","last_became_active":"2024-09-17T05:25:48.568324+0000","last_became_peered":"2024-09-17T05:25:48.568324+0000","last_unstale":"2024-09-17T05:25:48.583143+0000","last_undegraded":"2024-09-17T05:25:48.583143+0000","last_fullsized":"2024-09-17T05:25:48.583143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580146+0000","last_change":"2024-09-17T05:25:47.541605+0000","last_active":"2024-09-17T05:25:48.580146+0000","last_peered":"2024-09-17T05:25:48.580146+0000","last_clean":"2024-09-17T05:25:48.580146+0000","last_became_active":"2024-09-17T05:25:47.541312+0000","last_became_peered":"2024-09-17T05:25:47.541312+0000","last_unstale":"2024-09-17T05:25:48.580146+0000","last_undegraded":"2024-09-17T05:25:48.580146+0000","last_fullsized":"2024-09-17T05:25:48.580146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:27.690980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559889+0000","last_change":"2024-09-17T05:25:48.556785+0000","last_active":"2024-09-17T05:25:48.559889+0000","last_peered":"2024-09-17T05:25:48.559889+0000","last_clean":"2024-09-17T05:25:48.559889+0000","last_became_active":"2024-09-17T05:25:48.556443+0000","last_became_peered":"2024-09-17T05:25:48.556443+0000","last_unstale":"2024-09-17T05:25:48.559889+0000","last_undegraded":"2024-09-17T05:25:48.559889+0000","last_fullsized":"2024-09-17T05:25:48.559889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:47.106037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.590749+0000","last_change":"2024-09-17T05:25:47.555429+0000","last_active":"2024-09-17T05:25:48.590749+0000","last_peered":"2024-09-17T05:25:48.590749+0000","last_clean":"2024-09-17T05:25:48.590749+0000","last_became_active":"2024-09-17T05:25:47.555144+0000","last_became_peered":"2024-09-17T05:25:47.555144+0000","last_unstale":"2024-09-17T05:25:48.590749+0000","last_undegraded":"2024-09-17T05:25:48.590749+0000","last_fullsized":"2024-09-17T05:25:48.590749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:24.436569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584084+0000","last_change":"2024-09-17T05:25:48.579894+0000","last_active":"2024-09-17T05:25:48.584084+0000","last_peered":"2024-09-17T05:25:48.584084+0000","last_clean":"2024-09-17T05:25:48.584084+0000","last_became_active":"2024-09-17T05:25:48.578255+0000","last_became_peered":"2024-09-17T05:25:48.578255+0000","last_unstale":"2024-09-17T05:25:48.584084+0000","last_undegraded":"2024-09-17T05:25:48.584084+0000","last_fullsized":"2024-09-17T05:25:48.584084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:58.305009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.579143+0000","last_change":"2024-09-17T05:25:47.548118+0000","last_active":"2024-09-17T05:25:48.579143+0000","last_peered":"2024-09-17T05:25:48.579143+0000","last_clean":"2024-09-17T05:25:48.579143+0000","last_became_active":"2024-09-17T05:25:47.547871+0000","last_became_peered":"2024-09-17T05:25:47.547871+0000","last_unstale":"2024-09-17T05:25:48.579143+0000","last_undegraded":"2024-09-17T05:25:48.579143+0000","last_fullsized":"2024-09-17T05:25:48.579143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585675+0000","last_change":"2024-09-17T05:25:48.582169+0000","last_active":"2024-09-17T05:25:48.585675+0000","last_peered":"2024-09-17T05:25:48.585675+0000","last_clean":"2024-09-17T05:25:48.585675+0000","last_became_active":"2024-09-17T05:25:48.581834+0000","last_became_peered":"2024-09-17T05:25:48.581834+0000","last_unstale":"2024-09-17T05:25:48.585675+0000","last_undegraded":"2024-09-17T05:25:48.585675+0000","last_fullsized":"2024-09-17T05:25:48.585675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:30.341407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.552340+0000","last_change":"2024-09-17T05:25:47.547173+0000","last_active":"2024-09-17T05:25:48.552340+0000","last_peered":"2024-09-17T05:25:48.552340+0000","last_clean":"2024-09-17T05:25:48.552340+0000","last_became_active":"2024-09-17T05:25:47.546912+0000","last_became_peered":"2024-09-17T05:25:47.546912+0000","last_unstale":"2024-09-17T05:25:48.552340+0000","last_undegraded":"2024-09-17T05:25:48.552340+0000","last_fullsized":"2024-09-17T05:25:48.552340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:26.869603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559321+0000","last_change":"2024-09-17T05:25:48.553296+0000","last_active":"2024-09-17T05:25:48.559321+0000","last_peered":"2024-09-17T05:25:48.559321+0000","last_clean":"2024-09-17T05:25:48.559321+0000","last_became_active":"2024-09-17T05:25:48.553058+0000","last_became_peered":"2024-09-17T05:25:48.553058+0000","last_unstale":"2024-09-17T05:25:48.559321+0000","last_undegraded":"2024-09-17T05:25:48.559321+0000","last_fullsized":"2024-09-17T05:25:48.559321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:39.217642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.549156+0000","last_change":"2024-09-17T05:25:47.553779+0000","last_active":"2024-09-17T05:25:48.549156+0000","last_peered":"2024-09-17T05:25:48.549156+0000","last_clean":"2024-09-17T05:25:48.549156+0000","last_became_active":"2024-09-17T05:25:47.553533+0000","last_became_peered":"2024-09-17T05:25:47.553533+0000","last_unstale":"2024-09-17T05:25:48.549156+0000","last_undegraded":"2024-09-17T05:25:48.549156+0000","last_fullsized":"2024-09-17T05:25:48.549156+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.550987+0000","last_change":"2024-09-17T05:25:48.549363+0000","last_active":"2024-09-17T05:25:48.550987+0000","last_peered":"2024-09-17T05:25:48.550987+0000","last_clean":"2024-09-17T05:25:48.550987+0000","last_became_active":"2024-09-17T05:25:48.549077+0000","last_became_peered":"2024-09-17T05:25:48.549077+0000","last_unstale":"2024-09-17T05:25:48.550987+0000","last_undegraded":"2024-09-17T05:25:48.550987+0000","last_fullsized":"2024-09-17T05:25:48.550987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:38.211479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585251+0000","last_change":"2024-09-17T05:25:47.545375+0000","last_active":"2024-09-17T05:25:48.585251+0000","last_peered":"2024-09-17T05:25:48.585251+0000","last_clean":"2024-09-17T05:25:48.585251+0000","last_became_active":"2024-09-17T05:25:47.545098+0000","last_became_peered":"2024-09-17T05:25:47.545098+0000","last_unstale":"2024-09-17T05:25:48.585251+0000","last_undegraded":"2024-09-17T05:25:48.585251+0000","last_fullsized":"2024-09-17T05:25:48.585251+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:10.436228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561228+0000","last_change":"2024-09-17T05:25:48.557134+0000","last_active":"2024-09-17T05:25:48.561228+0000","last_peered":"2024-09-17T05:25:48.561228+0000","last_clean":"2024-09-17T05:25:48.561228+0000","last_became_active":"2024-09-17T05:25:48.556815+0000","last_became_peered":"2024-09-17T05:25:48.556815+0000","last_unstale":"2024-09-17T05:25:48.561228+0000","last_undegraded":"2024-09-17T05:25:48.561228+0000","last_fullsized":"2024-09-17T05:25:48.561228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:41.791223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.586323+0000","last_change":"2024-09-17T05:25:47.551471+0000","last_active":"2024-09-17T05:25:48.586323+0000","last_peered":"2024-09-17T05:25:48.586323+0000","last_clean":"2024-09-17T05:25:48.586323+0000","last_became_active":"2024-09-17T05:25:47.550374+0000","last_became_peered":"2024-09-17T05:25:47.550374+0000","last_unstale":"2024-09-17T05:25:48.586323+0000","last_undegraded":"2024-09-17T05:25:48.586323+0000","last_fullsized":"2024-09-17T05:25:48.586323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:34.028724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561192+0000","last_change":"2024-09-17T05:25:48.559461+0000","last_active":"2024-09-17T05:25:48.561192+0000","last_peered":"2024-09-17T05:25:48.561192+0000","last_clean":"2024-09-17T05:25:48.561192+0000","last_became_active":"2024-09-17T05:25:48.558513+0000","last_became_peered":"2024-09-17T05:25:48.558513+0000","last_unstale":"2024-09-17T05:25:48.561192+0000","last_undegraded":"2024-09-17T05:25:48.561192+0000","last_fullsized":"2024-09-17T05:25:48.561192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:35.034136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.572667+0000","last_change":"2024-09-17T05:25:47.547687+0000","last_active":"2024-09-17T05:25:48.572667+0000","last_peered":"2024-09-17T05:25:48.572667+0000","last_clean":"2024-09-17T05:25:48.572667+0000","last_became_active":"2024-09-17T05:25:47.547426+0000","last_became_peered":"2024-09-17T05:25:47.547426+0000","last_unstale":"2024-09-17T05:25:48.572667+0000","last_undegraded":"2024-09-17T05:25:48.572667+0000","last_fullsized":"2024-09-17T05:25:48.572667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540500+0000","last_change":"2024-09-17T05:25:47.540500+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540500+0000","last_undegraded":"2024-09-17T05:25:47.540500+0000","last_fullsized":"2024-09-17T05:25:47.540500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580070+0000","last_change":"2024-09-17T05:25:47.546486+0000","last_active":"2024-09-17T05:25:48.580070+0000","last_peered":"2024-09-17T05:25:48.580070+0000","last_clean":"2024-09-17T05:25:48.580070+0000","last_became_active":"2024-09-17T05:25:47.545952+0000","last_became_peered":"2024-09-17T05:25:47.545952+0000","last_unstale":"2024-09-17T05:25:48.580070+0000","last_undegraded":"2024-09-17T05:25:48.580070+0000","last_fullsized":"2024-09-17T05:25:48.580070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:17.918336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568072+0000","last_change":"2024-09-17T05:25:48.563376+0000","last_active":"2024-09-17T05:25:48.568072+0000","last_peered":"2024-09-17T05:25:48.568072+0000","last_clean":"2024-09-17T05:25:48.568072+0000","last_became_active":"2024-09-17T05:25:48.563128+0000","last_became_peered":"2024-09-17T05:25:48.563128+0000","last_unstale":"2024-09-17T05:25:48.568072+0000","last_undegraded":"2024-09-17T05:25:48.568072+0000","last_fullsized":"2024-09-17T05:25:48.568072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:29.084922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.547970+0000","last_change":"2024-09-17T05:25:47.550977+0000","last_active":"2024-09-17T05:25:48.547970+0000","last_peered":"2024-09-17T05:25:48.547970+0000","last_clean":"2024-09-17T05:25:48.547970+0000","last_became_active":"2024-09-17T05:25:47.550677+0000","last_became_peered":"2024-09-17T05:25:47.550677+0000","last_unstale":"2024-09-17T05:25:48.547970+0000","last_undegraded":"2024-09-17T05:25:48.547970+0000","last_fullsized":"2024-09-17T05:25:48.547970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:12.600379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583964+0000","last_change":"2024-09-17T05:25:48.578870+0000","last_active":"2024-09-17T05:25:48.583964+0000","last_peered":"2024-09-17T05:25:48.583964+0000","last_clean":"2024-09-17T05:25:48.583964+0000","last_became_active":"2024-09-17T05:25:48.578503+0000","last_became_peered":"2024-09-17T05:25:48.578503+0000","last_unstale":"2024-09-17T05:25:48.583964+0000","last_undegraded":"2024-09-17T05:25:48.583964+0000","last_fullsized":"2024-09-17T05:25:48.583964+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:54.356062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.556401+0000","last_change":"2024-09-17T05:25:47.553626+0000","last_active":"2024-09-17T05:25:47.556401+0000","last_peered":"2024-09-17T05:25:47.556401+0000","last_clean":"2024-09-17T05:25:47.556401+0000","last_became_active":"2024-09-17T05:25:47.553385+0000","last_became_peered":"2024-09-17T05:25:47.553385+0000","last_unstale":"2024-09-17T05:25:47.556401+0000","last_undegraded":"2024-09-17T05:25:47.556401+0000","last_fullsized":"2024-09-17T05:25:47.556401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:59.695362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540601+0000","last_change":"2024-09-17T05:25:47.540601+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540601+0000","last_undegraded":"2024-09-17T05:25:47.540601+0000","last_fullsized":"2024-09-17T05:25:47.540601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.549201+0000","last_change":"2024-09-17T05:25:47.549913+0000","last_active":"2024-09-17T05:25:48.549201+0000","last_peered":"2024-09-17T05:25:48.549201+0000","last_clean":"2024-09-17T05:25:48.549201+0000","last_became_active":"2024-09-17T05:25:47.549593+0000","last_became_peered":"2024-09-17T05:25:47.549593+0000","last_unstale":"2024-09-17T05:25:48.549201+0000","last_undegraded":"2024-09-17T05:25:48.549201+0000","last_fullsized":"2024-09-17T05:25:48.549201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:08.157058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588111+0000","last_change":"2024-09-17T05:25:48.583799+0000","last_active":"2024-09-17T05:25:48.588111+0000","last_peered":"2024-09-17T05:25:48.588111+0000","last_clean":"2024-09-17T05:25:48.588111+0000","last_became_active":"2024-09-17T05:25:48.583489+0000","last_became_peered":"2024-09-17T05:25:48.583489+0000","last_unstale":"2024-09-17T05:25:48.588111+0000","last_undegraded":"2024-09-17T05:25:48.588111+0000","last_fullsized":"2024-09-17T05:25:48.588111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:39.321687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.540541+0000","last_change":"2024-09-17T05:25:47.542635+0000","last_active":"2024-09-17T05:25:48.540541+0000","last_peered":"2024-09-17T05:25:48.540541+0000","last_clean":"2024-09-17T05:25:48.540541+0000","last_became_active":"2024-09-17T05:25:47.542160+0000","last_became_peered":"2024-09-17T05:25:47.542160+0000","last_unstale":"2024-09-17T05:25:48.540541+0000","last_undegraded":"2024-09-17T05:25:48.540541+0000","last_fullsized":"2024-09-17T05:25:48.540541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:33.388849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559087+0000","last_change":"2024-09-17T05:25:48.552387+0000","last_active":"2024-09-17T05:25:48.559087+0000","last_peered":"2024-09-17T05:25:48.559087+0000","last_clean":"2024-09-17T05:25:48.559087+0000","last_became_active":"2024-09-17T05:25:48.552075+0000","last_became_peered":"2024-09-17T05:25:48.552075+0000","last_unstale":"2024-09-17T05:25:48.559087+0000","last_undegraded":"2024-09-17T05:25:48.559087+0000","last_fullsized":"2024-09-17T05:25:48.559087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.456352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.546124+0000","last_change":"2024-09-17T05:25:47.545902+0000","last_active":"2024-09-17T05:25:48.546124+0000","last_peered":"2024-09-17T05:25:48.546124+0000","last_clean":"2024-09-17T05:25:48.546124+0000","last_became_active":"2024-09-17T05:25:47.545602+0000","last_became_peered":"2024-09-17T05:25:47.545602+0000","last_unstale":"2024-09-17T05:25:48.546124+0000","last_undegraded":"2024-09-17T05:25:48.546124+0000","last_fullsized":"2024-09-17T05:25:48.546124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:34.536424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.570355+0000","last_change":"2024-09-17T05:25:48.566441+0000","last_active":"2024-09-17T05:25:48.570355+0000","last_peered":"2024-09-17T05:25:48.570355+0000","last_clean":"2024-09-17T05:25:48.570355+0000","last_became_active":"2024-09-17T05:25:48.566282+0000","last_became_peered":"2024-09-17T05:25:48.566282+0000","last_unstale":"2024-09-17T05:25:48.570355+0000","last_undegraded":"2024-09-17T05:25:48.570355+0000","last_fullsized":"2024-09-17T05:25:48.570355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:25.441981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585494+0000","last_change":"2024-09-17T05:25:47.549606+0000","last_active":"2024-09-17T05:25:48.585494+0000","last_peered":"2024-09-17T05:25:48.585494+0000","last_clean":"2024-09-17T05:25:48.585494+0000","last_became_active":"2024-09-17T05:25:47.549353+0000","last_became_peered":"2024-09-17T05:25:47.549353+0000","last_unstale":"2024-09-17T05:25:48.585494+0000","last_undegraded":"2024-09-17T05:25:48.585494+0000","last_fullsized":"2024-09-17T05:25:48.585494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:46.440233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562140+0000","last_change":"2024-09-17T05:25:48.560961+0000","last_active":"2024-09-17T05:25:48.562140+0000","last_peered":"2024-09-17T05:25:48.562140+0000","last_clean":"2024-09-17T05:25:48.562140+0000","last_became_active":"2024-09-17T05:25:48.560715+0000","last_became_peered":"2024-09-17T05:25:48.560715+0000","last_unstale":"2024-09-17T05:25:48.562140+0000","last_undegraded":"2024-09-17T05:25:48.562140+0000","last_fullsized":"2024-09-17T05:25:48.562140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:49.943382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.556405+0000","last_change":"2024-09-17T05:25:47.552693+0000","last_active":"2024-09-17T05:25:47.556405+0000","last_peered":"2024-09-17T05:25:47.556405+0000","last_clean":"2024-09-17T05:25:47.556405+0000","last_became_active":"2024-09-17T05:25:47.548298+0000","last_became_peered":"2024-09-17T05:25:47.548298+0000","last_unstale":"2024-09-17T05:25:47.556405+0000","last_undegraded":"2024-09-17T05:25:47.556405+0000","last_fullsized":"2024-09-17T05:25:47.556405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:32.326924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568002+0000","last_change":"2024-09-17T05:25:48.559435+0000","last_active":"2024-09-17T05:25:48.568002+0000","last_peered":"2024-09-17T05:25:48.568002+0000","last_clean":"2024-09-17T05:25:48.568002+0000","last_became_active":"2024-09-17T05:25:48.558731+0000","last_became_peered":"2024-09-17T05:25:48.558731+0000","last_unstale":"2024-09-17T05:25:48.568002+0000","last_undegraded":"2024-09-17T05:25:48.568002+0000","last_fullsized":"2024-09-17T05:25:48.568002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:28.971195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.558040+0000","last_change":"2024-09-17T05:25:47.547920+0000","last_active":"2024-09-17T05:25:48.558040+0000","last_peered":"2024-09-17T05:25:48.558040+0000","last_clean":"2024-09-17T05:25:48.558040+0000","last_became_active":"2024-09-17T05:25:47.547593+0000","last_became_peered":"2024-09-17T05:25:47.547593+0000","last_unstale":"2024-09-17T05:25:48.558040+0000","last_undegraded":"2024-09-17T05:25:48.558040+0000","last_fullsized":"2024-09-17T05:25:48.558040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:12.832450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563888+0000","last_change":"2024-09-17T05:25:48.556873+0000","last_active":"2024-09-17T05:25:48.563888+0000","last_peered":"2024-09-17T05:25:48.563888+0000","last_clean":"2024-09-17T05:25:48.563888+0000","last_became_active":"2024-09-17T05:25:48.556481+0000","last_became_peered":"2024-09-17T05:25:48.556481+0000","last_unstale":"2024-09-17T05:25:48.563888+0000","last_undegraded":"2024-09-17T05:25:48.563888+0000","last_fullsized":"2024-09-17T05:25:48.563888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.560890+0000","last_change":"2024-09-17T05:25:47.552565+0000","last_active":"2024-09-17T05:25:48.560890+0000","last_peered":"2024-09-17T05:25:48.560890+0000","last_clean":"2024-09-17T05:25:48.560890+0000","last_became_active":"2024-09-17T05:25:47.549868+0000","last_became_peered":"2024-09-17T05:25:47.549868+0000","last_unstale":"2024-09-17T05:25:48.560890+0000","last_undegraded":"2024-09-17T05:25:48.560890+0000","last_fullsized":"2024-09-17T05:25:48.560890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:45.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585649+0000","last_change":"2024-09-17T05:25:48.579795+0000","last_active":"2024-09-17T05:25:48.585649+0000","last_peered":"2024-09-17T05:25:48.585649+0000","last_clean":"2024-09-17T05:25:48.585649+0000","last_became_active":"2024-09-17T05:25:48.579382+0000","last_became_peered":"2024-09-17T05:25:48.579382+0000","last_unstale":"2024-09-17T05:25:48.585649+0000","last_undegraded":"2024-09-17T05:25:48.585649+0000","last_fullsized":"2024-09-17T05:25:48.585649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:34.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.560830+0000","last_change":"2024-09-17T05:25:47.548934+0000","last_active":"2024-09-17T05:25:48.560830+0000","last_peered":"2024-09-17T05:25:48.560830+0000","last_clean":"2024-09-17T05:25:48.560830+0000","last_became_active":"2024-09-17T05:25:47.548388+0000","last_became_peered":"2024-09-17T05:25:47.548388+0000","last_unstale":"2024-09-17T05:25:48.560830+0000","last_undegraded":"2024-09-17T05:25:48.560830+0000","last_fullsized":"2024-09-17T05:25:48.560830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:32.932862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.575376+0000","last_change":"2024-09-17T05:25:48.565031+0000","last_active":"2024-09-17T05:25:48.575376+0000","last_peered":"2024-09-17T05:25:48.575376+0000","last_clean":"2024-09-17T05:25:48.575376+0000","last_became_active":"2024-09-17T05:25:48.564697+0000","last_became_peered":"2024-09-17T05:25:48.564697+0000","last_unstale":"2024-09-17T05:25:48.575376+0000","last_undegraded":"2024-09-17T05:25:48.575376+0000","last_fullsized":"2024-09-17T05:25:48.575376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.548534+0000","last_change":"2024-09-17T05:25:47.548297+0000","last_active":"2024-09-17T05:25:48.548534+0000","last_peered":"2024-09-17T05:25:48.548534+0000","last_clean":"2024-09-17T05:25:48.548534+0000","last_became_active":"2024-09-17T05:25:47.548063+0000","last_became_peered":"2024-09-17T05:25:47.548063+0000","last_unstale":"2024-09-17T05:25:48.548534+0000","last_undegraded":"2024-09-17T05:25:48.548534+0000","last_fullsized":"2024-09-17T05:25:48.548534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:34.378981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564574+0000","last_change":"2024-09-17T05:25:48.562700+0000","last_active":"2024-09-17T05:25:48.564574+0000","last_peered":"2024-09-17T05:25:48.564574+0000","last_clean":"2024-09-17T05:25:48.564574+0000","last_became_active":"2024-09-17T05:25:48.562183+0000","last_became_peered":"2024-09-17T05:25:48.562183+0000","last_unstale":"2024-09-17T05:25:48.564574+0000","last_undegraded":"2024-09-17T05:25:48.564574+0000","last_fullsized":"2024-09-17T05:25:48.564574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:13.905356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.550500+0000","last_change":"2024-09-17T05:25:47.551652+0000","last_active":"2024-09-17T05:25:48.550500+0000","last_peered":"2024-09-17T05:25:48.550500+0000","last_clean":"2024-09-17T05:25:48.550500+0000","last_became_active":"2024-09-17T05:25:47.551332+0000","last_became_peered":"2024-09-17T05:25:47.551332+0000","last_unstale":"2024-09-17T05:25:48.550500+0000","last_undegraded":"2024-09-17T05:25:48.550500+0000","last_fullsized":"2024-09-17T05:25:48.550500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:38.212230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565013+0000","last_change":"2024-09-17T05:25:48.559526+0000","last_active":"2024-09-17T05:25:48.565013+0000","last_peered":"2024-09-17T05:25:48.565013+0000","last_clean":"2024-09-17T05:25:48.565013+0000","last_became_active":"2024-09-17T05:25:48.559292+0000","last_became_peered":"2024-09-17T05:25:48.559292+0000","last_unstale":"2024-09-17T05:25:48.565013+0000","last_undegraded":"2024-09-17T05:25:48.565013+0000","last_fullsized":"2024-09-17T05:25:48.565013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.545263+0000","last_change":"2024-09-17T05:25:47.546172+0000","last_active":"2024-09-17T05:25:48.545263+0000","last_peered":"2024-09-17T05:25:48.545263+0000","last_clean":"2024-09-17T05:25:48.545263+0000","last_became_active":"2024-09-17T05:25:47.545644+0000","last_became_peered":"2024-09-17T05:25:47.545644+0000","last_unstale":"2024-09-17T05:25:48.545263+0000","last_undegraded":"2024-09-17T05:25:48.545263+0000","last_fullsized":"2024-09-17T05:25:48.545263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:46.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":130,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.541021+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:48.541021+0000","last_peered":"2024-09-17T05:25:48.541021+0000","last_clean":"2024-09-17T05:25:48.541021+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:48.541021+0000","last_undegraded":"2024-09-17T05:25:48.541021+0000","last_fullsized":"2024-09-17T05:25:48.541021+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":368640,"data_stored":134818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":723622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":710502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:53.049 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:53.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- 172.21.15.122:0/3665737701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40d28e0 msgr2=0x7f03a40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 --2- 172.21.15.122:0/3665737701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40d28e0 0x7f03a40cebb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f03a40745d0 tx=0x7f039c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- 172.21.15.122:0/3665737701 shutdown_connections 2024-09-17T05:25:53.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 --2- 172.21.15.122:0/3665737701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40d28e0 0x7f03a40cebb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 --2- 172.21.15.122:0/3665737701 >> v2:172.21.15.173:3300/0 conn(0x7f03a415bea0 0x7f03a40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- 172.21.15.122:0/3665737701 >> 172.21.15.122:0/3665737701 conn(0x7f03a40d1dc0 msgr2=0x7f03a40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- 172.21.15.122:0/3665737701 shutdown_connections 2024-09-17T05:25:53.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- 172.21.15.122:0/3665737701 wait complete. 2024-09-17T05:25:53.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 Processor -- start 2024-09-17T05:25:53.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- start start 2024-09-17T05:25:53.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40cdb30 0x7f03a415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a40d28e0 0x7f03a40b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03a415bea0 0x7f03a406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4168fc0 con 0x7f03a40cdb30 2024-09-17T05:25:53.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4168e40 con 0x7f03a40d28e0 2024-09-17T05:25:53.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a8e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f03a4169140 con 0x7f03a415bea0 2024-09-17T05:25:53.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40cdb30 0x7f03a415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40cdb30 0x7f03a415ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53782/0 (socket says 172.21.15.122:53782) 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a1e00640 1 -- 172.21.15.122:0/719249904 learned_addr learned my addr 172.21.15.122:0/719249904 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f0399400640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a40d28e0 0x7f03a40b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a2800640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03a415bea0 0x7f03a406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a1e00640 1 -- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03a415bea0 msgr2=0x7f03a406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a1e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03a415bea0 0x7f03a406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a1e00640 1 -- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a40d28e0 msgr2=0x7f03a40b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a1e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a40d28e0 0x7f03a40b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.225+0000 7f03a1e00640 1 -- 172.21.15.122:0/719249904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a40ca6d0 con 0x7f03a40cdb30 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f03a1e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40cdb30 0x7f03a415ad00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f039400ca20 tx=0x7f039400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f039aa00640 1 -- 172.21.15.122:0/719249904 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394007c70 con 0x7f03a40cdb30 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f039aa00640 1 -- 172.21.15.122:0/719249904 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f039400f450 con 0x7f03a40cdb30 2024-09-17T05:25:53.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a40ca9b0 con 0x7f03a40cdb30 2024-09-17T05:25:53.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f039aa00640 1 -- 172.21.15.122:0/719249904 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039400f8a0 con 0x7f03a40cdb30 2024-09-17T05:25:53.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a415b240 con 0x7f03a40cdb30 2024-09-17T05:25:53.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a40cc9d0 con 0x7f03a40cdb30 2024-09-17T05:25:53.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f039aa00640 1 -- 172.21.15.122:0/719249904 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f039400fa40 con 0x7f03a40cdb30 2024-09-17T05:25:53.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f039aa00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0380073200 0x7f03800756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f039aa00640 1 -- 172.21.15.122:0/719249904 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0394094a80 con 0x7f03a40cdb30 2024-09-17T05:25:53.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.229+0000 7f0399400640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0380073200 0x7f03800756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.233+0000 7f0399400640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0380073200 0x7f03800756c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03a40b4490 tx=0x7f038c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:53.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.233+0000 7f039aa00640 1 -- 172.21.15.122:0/719249904 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0394096e20 con 0x7f03a40cdb30 2024-09-17T05:25:53.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.401+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03a40b5550 con 0x7f0380073200 2024-09-17T05:25:53.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.405+0000 7f039aa00640 1 -- 172.21.15.122:0/719249904 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251525 (secure 0 0 0) 0x7f03a40b5550 con 0x7f0380073200 2024-09-17T05:25:53.412 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:53.413 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:53.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0380073200 msgr2=0x7f03800756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0380073200 0x7f03800756c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03a40b4490 tx=0x7f038c009040 comp rx=0 tx=0).stop 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40cdb30 msgr2=0x7f03a415ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40cdb30 0x7f03a415ad00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f039400ca20 tx=0x7f039400cee0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 shutdown_connections 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0380073200 0x7f03800756c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03a415bea0 0x7f03a406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a40d28e0 0x7f03a40b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 --2- 172.21.15.122:0/719249904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03a40cdb30 0x7f03a415ad00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 >> 172.21.15.122:0/719249904 conn(0x7f03a40d1dc0 msgr2=0x7f03a40b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 shutdown_connections 2024-09-17T05:25:53.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.409+0000 7f03a8e00640 1 -- 172.21.15.122:0/719249904 wait complete. 2024-09-17T05:25:53.432 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-17T05:25:52.454124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224260,"kb_used_data":3848,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750460,"statfs":{"total":773094113280,"available":772864471040,"internally_reserved":0,"allocated":3940352,"data_stored":2200987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928592"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565313+0000","last_change":"2024-09-17T05:25:48.563496+0000","last_active":"2024-09-17T05:25:48.565313+0000","last_peered":"2024-09-17T05:25:48.565313+0000","last_clean":"2024-09-17T05:25:48.565313+0000","last_became_active":"2024-09-17T05:25:48.563114+0000","last_became_peered":"2024-09-17T05:25:48.563114+0000","last_unstale":"2024-09-17T05:25:48.565313+0000","last_undegraded":"2024-09-17T05:25:48.565313+0000","last_fullsized":"2024-09-17T05:25:48.565313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540092+0000","last_change":"2024-09-17T05:25:47.540092+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540092+0000","last_undegraded":"2024-09-17T05:25:47.540092+0000","last_fullsized":"2024-09-17T05:25:47.540092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.552034+0000","last_change":"2024-09-17T05:25:48.549461+0000","last_active":"2024-09-17T05:25:48.552034+0000","last_peered":"2024-09-17T05:25:48.552034+0000","last_clean":"2024-09-17T05:25:48.552034+0000","last_became_active":"2024-09-17T05:25:48.549168+0000","last_became_peered":"2024-09-17T05:25:48.549168+0000","last_unstale":"2024-09-17T05:25:48.552034+0000","last_undegraded":"2024-09-17T05:25:48.552034+0000","last_fullsized":"2024-09-17T05:25:48.552034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:15.092436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.574864+0000","last_change":"2024-09-17T05:25:48.565056+0000","last_active":"2024-09-17T05:25:48.574864+0000","last_peered":"2024-09-17T05:25:48.574864+0000","last_clean":"2024-09-17T05:25:48.574864+0000","last_became_active":"2024-09-17T05:25:48.564810+0000","last_became_peered":"2024-09-17T05:25:48.564810+0000","last_unstale":"2024-09-17T05:25:48.574864+0000","last_undegraded":"2024-09-17T05:25:48.574864+0000","last_fullsized":"2024-09-17T05:25:48.574864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.541154+0000","last_change":"2024-09-17T05:25:47.541154+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.541154+0000","last_undegraded":"2024-09-17T05:25:47.541154+0000","last_fullsized":"2024-09-17T05:25:47.541154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568106+0000","last_change":"2024-09-17T05:25:48.563410+0000","last_active":"2024-09-17T05:25:48.568106+0000","last_peered":"2024-09-17T05:25:48.568106+0000","last_clean":"2024-09-17T05:25:48.568106+0000","last_became_active":"2024-09-17T05:25:48.563094+0000","last_became_peered":"2024-09-17T05:25:48.563094+0000","last_unstale":"2024-09-17T05:25:48.568106+0000","last_undegraded":"2024-09-17T05:25:48.568106+0000","last_fullsized":"2024-09-17T05:25:48.568106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:47.445644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561093+0000","last_change":"2024-09-17T05:25:48.555613+0000","last_active":"2024-09-17T05:25:48.561093+0000","last_peered":"2024-09-17T05:25:48.561093+0000","last_clean":"2024-09-17T05:25:48.561093+0000","last_became_active":"2024-09-17T05:25:48.555015+0000","last_became_peered":"2024-09-17T05:25:48.555015+0000","last_unstale":"2024-09-17T05:25:48.561093+0000","last_undegraded":"2024-09-17T05:25:48.561093+0000","last_fullsized":"2024-09-17T05:25:48.561093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:05.497779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.576391+0000","last_change":"2024-09-17T05:25:48.571453+0000","last_active":"2024-09-17T05:25:48.576391+0000","last_peered":"2024-09-17T05:25:48.576391+0000","last_clean":"2024-09-17T05:25:48.576391+0000","last_became_active":"2024-09-17T05:25:48.571191+0000","last_became_peered":"2024-09-17T05:25:48.571191+0000","last_unstale":"2024-09-17T05:25:48.576391+0000","last_undegraded":"2024-09-17T05:25:48.576391+0000","last_fullsized":"2024-09-17T05:25:48.576391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566421+0000","last_change":"2024-09-17T05:25:48.563932+0000","last_active":"2024-09-17T05:25:48.566421+0000","last_peered":"2024-09-17T05:25:48.566421+0000","last_clean":"2024-09-17T05:25:48.566421+0000","last_became_active":"2024-09-17T05:25:48.563651+0000","last_became_peered":"2024-09-17T05:25:48.563651+0000","last_unstale":"2024-09-17T05:25:48.566421+0000","last_undegraded":"2024-09-17T05:25:48.566421+0000","last_fullsized":"2024-09-17T05:25:48.566421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:54.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584841+0000","last_change":"2024-09-17T05:25:48.569820+0000","last_active":"2024-09-17T05:25:48.584841+0000","last_peered":"2024-09-17T05:25:48.584841+0000","last_clean":"2024-09-17T05:25:48.584841+0000","last_became_active":"2024-09-17T05:25:48.569549+0000","last_became_peered":"2024-09-17T05:25:48.569549+0000","last_unstale":"2024-09-17T05:25:48.584841+0000","last_undegraded":"2024-09-17T05:25:48.584841+0000","last_fullsized":"2024-09-17T05:25:48.584841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559866+0000","last_change":"2024-09-17T05:25:48.556416+0000","last_active":"2024-09-17T05:25:48.559866+0000","last_peered":"2024-09-17T05:25:48.559866+0000","last_clean":"2024-09-17T05:25:48.559866+0000","last_became_active":"2024-09-17T05:25:48.556113+0000","last_became_peered":"2024-09-17T05:25:48.556113+0000","last_unstale":"2024-09-17T05:25:48.559866+0000","last_undegraded":"2024-09-17T05:25:48.559866+0000","last_fullsized":"2024-09-17T05:25:48.559866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:01.838706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580005+0000","last_change":"2024-09-17T05:25:48.578672+0000","last_active":"2024-09-17T05:25:48.580005+0000","last_peered":"2024-09-17T05:25:48.580005+0000","last_clean":"2024-09-17T05:25:48.580005+0000","last_became_active":"2024-09-17T05:25:48.577799+0000","last_became_peered":"2024-09-17T05:25:48.577799+0000","last_unstale":"2024-09-17T05:25:48.580005+0000","last_undegraded":"2024-09-17T05:25:48.580005+0000","last_fullsized":"2024-09-17T05:25:48.580005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:28.971195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588071+0000","last_change":"2024-09-17T05:25:48.583732+0000","last_active":"2024-09-17T05:25:48.588071+0000","last_peered":"2024-09-17T05:25:48.588071+0000","last_clean":"2024-09-17T05:25:48.588071+0000","last_became_active":"2024-09-17T05:25:48.583467+0000","last_became_peered":"2024-09-17T05:25:48.583467+0000","last_unstale":"2024-09-17T05:25:48.588071+0000","last_undegraded":"2024-09-17T05:25:48.588071+0000","last_fullsized":"2024-09-17T05:25:48.588071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:40.017087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.581449+0000","last_change":"2024-09-17T05:25:48.568509+0000","last_active":"2024-09-17T05:25:48.581449+0000","last_peered":"2024-09-17T05:25:48.581449+0000","last_clean":"2024-09-17T05:25:48.581449+0000","last_became_active":"2024-09-17T05:25:48.568196+0000","last_became_peered":"2024-09-17T05:25:48.568196+0000","last_unstale":"2024-09-17T05:25:48.581449+0000","last_undegraded":"2024-09-17T05:25:48.581449+0000","last_fullsized":"2024-09-17T05:25:48.581449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564320+0000","last_change":"2024-09-17T05:25:48.557116+0000","last_active":"2024-09-17T05:25:48.564320+0000","last_peered":"2024-09-17T05:25:48.564320+0000","last_clean":"2024-09-17T05:25:48.564320+0000","last_became_active":"2024-09-17T05:25:48.556674+0000","last_became_peered":"2024-09-17T05:25:48.556674+0000","last_unstale":"2024-09-17T05:25:48.564320+0000","last_undegraded":"2024-09-17T05:25:48.564320+0000","last_fullsized":"2024-09-17T05:25:48.564320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563788+0000","last_change":"2024-09-17T05:25:48.555359+0000","last_active":"2024-09-17T05:25:48.563788+0000","last_peered":"2024-09-17T05:25:48.563788+0000","last_clean":"2024-09-17T05:25:48.563788+0000","last_became_active":"2024-09-17T05:25:48.554078+0000","last_became_peered":"2024-09-17T05:25:48.554078+0000","last_unstale":"2024-09-17T05:25:48.563788+0000","last_undegraded":"2024-09-17T05:25:48.563788+0000","last_fullsized":"2024-09-17T05:25:48.563788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559662+0000","last_change":"2024-09-17T05:25:48.555168+0000","last_active":"2024-09-17T05:25:48.559662+0000","last_peered":"2024-09-17T05:25:48.559662+0000","last_clean":"2024-09-17T05:25:48.559662+0000","last_became_active":"2024-09-17T05:25:48.554887+0000","last_became_peered":"2024-09-17T05:25:48.554887+0000","last_unstale":"2024-09-17T05:25:48.559662+0000","last_undegraded":"2024-09-17T05:25:48.559662+0000","last_fullsized":"2024-09-17T05:25:48.559662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:18.816281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585276+0000","last_change":"2024-09-17T05:25:48.580714+0000","last_active":"2024-09-17T05:25:48.585276+0000","last_peered":"2024-09-17T05:25:48.585276+0000","last_clean":"2024-09-17T05:25:48.585276+0000","last_became_active":"2024-09-17T05:25:48.580355+0000","last_became_peered":"2024-09-17T05:25:48.580355+0000","last_unstale":"2024-09-17T05:25:48.585276+0000","last_undegraded":"2024-09-17T05:25:48.585276+0000","last_fullsized":"2024-09-17T05:25:48.585276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:13.837862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.567853+0000","last_change":"2024-09-17T05:25:48.559308+0000","last_active":"2024-09-17T05:25:48.567853+0000","last_peered":"2024-09-17T05:25:48.567853+0000","last_clean":"2024-09-17T05:25:48.567853+0000","last_became_active":"2024-09-17T05:25:48.558922+0000","last_became_peered":"2024-09-17T05:25:48.558922+0000","last_unstale":"2024-09-17T05:25:48.567853+0000","last_undegraded":"2024-09-17T05:25:48.567853+0000","last_fullsized":"2024-09-17T05:25:48.567853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:54.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584951+0000","last_change":"2024-09-17T05:25:48.578440+0000","last_active":"2024-09-17T05:25:48.584951+0000","last_peered":"2024-09-17T05:25:48.584951+0000","last_clean":"2024-09-17T05:25:48.584951+0000","last_became_active":"2024-09-17T05:25:48.578166+0000","last_became_peered":"2024-09-17T05:25:48.578166+0000","last_unstale":"2024-09-17T05:25:48.584951+0000","last_undegraded":"2024-09-17T05:25:48.584951+0000","last_fullsized":"2024-09-17T05:25:48.584951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:00.303537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.578195+0000","last_change":"2024-09-17T05:25:48.576041+0000","last_active":"2024-09-17T05:25:48.578195+0000","last_peered":"2024-09-17T05:25:48.578195+0000","last_clean":"2024-09-17T05:25:48.578195+0000","last_became_active":"2024-09-17T05:25:48.575747+0000","last_became_peered":"2024-09-17T05:25:48.575747+0000","last_unstale":"2024-09-17T05:25:48.578195+0000","last_undegraded":"2024-09-17T05:25:48.578195+0000","last_fullsized":"2024-09-17T05:25:48.578195+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566379+0000","last_change":"2024-09-17T05:25:48.564740+0000","last_active":"2024-09-17T05:25:48.566379+0000","last_peered":"2024-09-17T05:25:48.566379+0000","last_clean":"2024-09-17T05:25:48.566379+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:48.566379+0000","last_undegraded":"2024-09-17T05:25:48.566379+0000","last_fullsized":"2024-09-17T05:25:48.566379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.582889+0000","last_change":"2024-09-17T05:25:48.575949+0000","last_active":"2024-09-17T05:25:48.582889+0000","last_peered":"2024-09-17T05:25:48.582889+0000","last_clean":"2024-09-17T05:25:48.582889+0000","last_became_active":"2024-09-17T05:25:48.573427+0000","last_became_peered":"2024-09-17T05:25:48.573427+0000","last_unstale":"2024-09-17T05:25:48.582889+0000","last_undegraded":"2024-09-17T05:25:48.582889+0000","last_fullsized":"2024-09-17T05:25:48.582889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:20.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.570555+0000","last_change":"2024-09-17T05:25:48.566361+0000","last_active":"2024-09-17T05:25:48.570555+0000","last_peered":"2024-09-17T05:25:48.570555+0000","last_clean":"2024-09-17T05:25:48.570555+0000","last_became_active":"2024-09-17T05:25:48.566054+0000","last_became_peered":"2024-09-17T05:25:48.566054+0000","last_unstale":"2024-09-17T05:25:48.570555+0000","last_undegraded":"2024-09-17T05:25:48.570555+0000","last_fullsized":"2024-09-17T05:25:48.570555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.575678+0000","last_change":"2024-09-17T05:25:48.565723+0000","last_active":"2024-09-17T05:25:48.575678+0000","last_peered":"2024-09-17T05:25:48.575678+0000","last_clean":"2024-09-17T05:25:48.575678+0000","last_became_active":"2024-09-17T05:25:48.565457+0000","last_became_peered":"2024-09-17T05:25:48.565457+0000","last_unstale":"2024-09-17T05:25:48.575678+0000","last_undegraded":"2024-09-17T05:25:48.575678+0000","last_fullsized":"2024-09-17T05:25:48.575678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562166+0000","last_change":"2024-09-17T05:25:48.555370+0000","last_active":"2024-09-17T05:25:48.562166+0000","last_peered":"2024-09-17T05:25:48.562166+0000","last_clean":"2024-09-17T05:25:48.562166+0000","last_became_active":"2024-09-17T05:25:48.554994+0000","last_became_peered":"2024-09-17T05:25:48.554994+0000","last_unstale":"2024-09-17T05:25:48.562166+0000","last_undegraded":"2024-09-17T05:25:48.562166+0000","last_fullsized":"2024-09-17T05:25:48.562166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:11.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.541326+0000","last_change":"2024-09-17T05:25:47.541326+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.541326+0000","last_undegraded":"2024-09-17T05:25:47.541326+0000","last_fullsized":"2024-09-17T05:25:47.541326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.579598+0000","last_change":"2024-09-17T05:25:48.576799+0000","last_active":"2024-09-17T05:25:48.579598+0000","last_peered":"2024-09-17T05:25:48.579598+0000","last_clean":"2024-09-17T05:25:48.579598+0000","last_became_active":"2024-09-17T05:25:48.576460+0000","last_became_peered":"2024-09-17T05:25:48.576460+0000","last_unstale":"2024-09-17T05:25:48.579598+0000","last_undegraded":"2024-09-17T05:25:48.579598+0000","last_fullsized":"2024-09-17T05:25:48.579598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563457+0000","last_change":"2024-09-17T05:25:48.555831+0000","last_active":"2024-09-17T05:25:48.563457+0000","last_peered":"2024-09-17T05:25:48.563457+0000","last_clean":"2024-09-17T05:25:48.563457+0000","last_became_active":"2024-09-17T05:25:48.555546+0000","last_became_peered":"2024-09-17T05:25:48.555546+0000","last_unstale":"2024-09-17T05:25:48.563457+0000","last_undegraded":"2024-09-17T05:25:48.563457+0000","last_fullsized":"2024-09-17T05:25:48.563457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563859+0000","last_change":"2024-09-17T05:25:48.554655+0000","last_active":"2024-09-17T05:25:48.563859+0000","last_peered":"2024-09-17T05:25:48.563859+0000","last_clean":"2024-09-17T05:25:48.563859+0000","last_became_active":"2024-09-17T05:25:48.553919+0000","last_became_peered":"2024-09-17T05:25:48.553919+0000","last_unstale":"2024-09-17T05:25:48.563859+0000","last_undegraded":"2024-09-17T05:25:48.563859+0000","last_fullsized":"2024-09-17T05:25:48.563859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564934+0000","last_change":"2024-09-17T05:25:48.560930+0000","last_active":"2024-09-17T05:25:48.564934+0000","last_peered":"2024-09-17T05:25:48.564934+0000","last_clean":"2024-09-17T05:25:48.564934+0000","last_became_active":"2024-09-17T05:25:48.560700+0000","last_became_peered":"2024-09-17T05:25:48.560700+0000","last_unstale":"2024-09-17T05:25:48.564934+0000","last_undegraded":"2024-09-17T05:25:48.564934+0000","last_fullsized":"2024-09-17T05:25:48.564934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:11.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.551685+0000","last_change":"2024-09-17T05:25:48.549526+0000","last_active":"2024-09-17T05:25:48.551685+0000","last_peered":"2024-09-17T05:25:48.551685+0000","last_clean":"2024-09-17T05:25:48.551685+0000","last_became_active":"2024-09-17T05:25:48.549278+0000","last_became_peered":"2024-09-17T05:25:48.549278+0000","last_unstale":"2024-09-17T05:25:48.551685+0000","last_undegraded":"2024-09-17T05:25:48.551685+0000","last_fullsized":"2024-09-17T05:25:48.551685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:40.352200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583989+0000","last_change":"2024-09-17T05:25:48.579263+0000","last_active":"2024-09-17T05:25:48.583989+0000","last_peered":"2024-09-17T05:25:48.583989+0000","last_clean":"2024-09-17T05:25:48.583989+0000","last_became_active":"2024-09-17T05:25:48.578226+0000","last_became_peered":"2024-09-17T05:25:48.578226+0000","last_unstale":"2024-09-17T05:25:48.583989+0000","last_undegraded":"2024-09-17T05:25:48.583989+0000","last_fullsized":"2024-09-17T05:25:48.583989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:49.685150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580051+0000","last_change":"2024-09-17T05:25:47.542992+0000","last_active":"2024-09-17T05:25:48.580051+0000","last_peered":"2024-09-17T05:25:48.580051+0000","last_clean":"2024-09-17T05:25:48.580051+0000","last_became_active":"2024-09-17T05:25:47.542678+0000","last_became_peered":"2024-09-17T05:25:47.542678+0000","last_unstale":"2024-09-17T05:25:48.580051+0000","last_undegraded":"2024-09-17T05:25:48.580051+0000","last_fullsized":"2024-09-17T05:25:48.580051+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:18.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562177+0000","last_change":"2024-09-17T05:25:48.558850+0000","last_active":"2024-09-17T05:25:48.562177+0000","last_peered":"2024-09-17T05:25:48.562177+0000","last_clean":"2024-09-17T05:25:48.562177+0000","last_became_active":"2024-09-17T05:25:48.558570+0000","last_became_peered":"2024-09-17T05:25:48.558570+0000","last_unstale":"2024-09-17T05:25:48.562177+0000","last_undegraded":"2024-09-17T05:25:48.562177+0000","last_fullsized":"2024-09-17T05:25:48.562177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:27.875015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585957+0000","last_change":"2024-09-17T05:25:47.550543+0000","last_active":"2024-09-17T05:25:48.585957+0000","last_peered":"2024-09-17T05:25:48.585957+0000","last_clean":"2024-09-17T05:25:48.585957+0000","last_became_active":"2024-09-17T05:25:47.550230+0000","last_became_peered":"2024-09-17T05:25:47.550230+0000","last_unstale":"2024-09-17T05:25:48.585957+0000","last_undegraded":"2024-09-17T05:25:48.585957+0000","last_fullsized":"2024-09-17T05:25:48.585957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565532+0000","last_change":"2024-09-17T05:25:48.561559+0000","last_active":"2024-09-17T05:25:48.565532+0000","last_peered":"2024-09-17T05:25:48.565532+0000","last_clean":"2024-09-17T05:25:48.565532+0000","last_became_active":"2024-09-17T05:25:48.561199+0000","last_became_peered":"2024-09-17T05:25:48.561199+0000","last_unstale":"2024-09-17T05:25:48.565532+0000","last_undegraded":"2024-09-17T05:25:48.565532+0000","last_fullsized":"2024-09-17T05:25:48.565532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.605791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.581425+0000","last_change":"2024-09-17T05:25:47.555529+0000","last_active":"2024-09-17T05:25:48.581425+0000","last_peered":"2024-09-17T05:25:48.581425+0000","last_clean":"2024-09-17T05:25:48.581425+0000","last_became_active":"2024-09-17T05:25:47.554844+0000","last_became_peered":"2024-09-17T05:25:47.554844+0000","last_unstale":"2024-09-17T05:25:48.581425+0000","last_undegraded":"2024-09-17T05:25:48.581425+0000","last_fullsized":"2024-09-17T05:25:48.581425+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:18.141436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585939+0000","last_change":"2024-09-17T05:25:48.577744+0000","last_active":"2024-09-17T05:25:48.585939+0000","last_peered":"2024-09-17T05:25:48.585939+0000","last_clean":"2024-09-17T05:25:48.585939+0000","last_became_active":"2024-09-17T05:25:48.577492+0000","last_became_peered":"2024-09-17T05:25:48.577492+0000","last_unstale":"2024-09-17T05:25:48.585939+0000","last_undegraded":"2024-09-17T05:25:48.585939+0000","last_fullsized":"2024-09-17T05:25:48.585939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.166901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588295+0000","last_change":"2024-09-17T05:25:47.557387+0000","last_active":"2024-09-17T05:25:48.588295+0000","last_peered":"2024-09-17T05:25:48.588295+0000","last_clean":"2024-09-17T05:25:48.588295+0000","last_became_active":"2024-09-17T05:25:47.557000+0000","last_became_peered":"2024-09-17T05:25:47.557000+0000","last_unstale":"2024-09-17T05:25:48.588295+0000","last_undegraded":"2024-09-17T05:25:48.588295+0000","last_fullsized":"2024-09-17T05:25:48.588295+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:04.492367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566436+0000","last_change":"2024-09-17T05:25:48.563244+0000","last_active":"2024-09-17T05:25:48.566436+0000","last_peered":"2024-09-17T05:25:48.566436+0000","last_clean":"2024-09-17T05:25:48.566436+0000","last_became_active":"2024-09-17T05:25:48.562910+0000","last_became_peered":"2024-09-17T05:25:48.562910+0000","last_unstale":"2024-09-17T05:25:48.566436+0000","last_undegraded":"2024-09-17T05:25:48.566436+0000","last_fullsized":"2024-09-17T05:25:48.566436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:19.146848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.577584+0000","last_change":"2024-09-17T05:25:47.542868+0000","last_active":"2024-09-17T05:25:48.577584+0000","last_peered":"2024-09-17T05:25:48.577584+0000","last_clean":"2024-09-17T05:25:48.577584+0000","last_became_active":"2024-09-17T05:25:47.542655+0000","last_became_peered":"2024-09-17T05:25:47.542655+0000","last_unstale":"2024-09-17T05:25:48.577584+0000","last_undegraded":"2024-09-17T05:25:48.577584+0000","last_fullsized":"2024-09-17T05:25:48.577584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:02.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565297+0000","last_change":"2024-09-17T05:25:48.561829+0000","last_active":"2024-09-17T05:25:48.565297+0000","last_peered":"2024-09-17T05:25:48.565297+0000","last_clean":"2024-09-17T05:25:48.565297+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:48.565297+0000","last_undegraded":"2024-09-17T05:25:48.565297+0000","last_fullsized":"2024-09-17T05:25:48.565297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.577042+0000","last_change":"2024-09-17T05:25:47.552636+0000","last_active":"2024-09-17T05:25:48.577042+0000","last_peered":"2024-09-17T05:25:48.577042+0000","last_clean":"2024-09-17T05:25:48.577042+0000","last_became_active":"2024-09-17T05:25:47.552361+0000","last_became_peered":"2024-09-17T05:25:47.552361+0000","last_unstale":"2024-09-17T05:25:48.577042+0000","last_undegraded":"2024-09-17T05:25:48.577042+0000","last_fullsized":"2024-09-17T05:25:48.577042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:05.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561279+0000","last_change":"2024-09-17T05:25:48.553264+0000","last_active":"2024-09-17T05:25:48.561279+0000","last_peered":"2024-09-17T05:25:48.561279+0000","last_clean":"2024-09-17T05:25:48.561279+0000","last_became_active":"2024-09-17T05:25:48.552992+0000","last_became_peered":"2024-09-17T05:25:48.552992+0000","last_unstale":"2024-09-17T05:25:48.561279+0000","last_undegraded":"2024-09-17T05:25:48.561279+0000","last_fullsized":"2024-09-17T05:25:48.561279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:11.441640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.544893+0000","last_change":"2024-09-17T05:25:47.544385+0000","last_active":"2024-09-17T05:25:48.544893+0000","last_peered":"2024-09-17T05:25:48.544893+0000","last_clean":"2024-09-17T05:25:48.544893+0000","last_became_active":"2024-09-17T05:25:47.544020+0000","last_became_peered":"2024-09-17T05:25:47.544020+0000","last_unstale":"2024-09-17T05:25:48.544893+0000","last_undegraded":"2024-09-17T05:25:48.544893+0000","last_fullsized":"2024-09-17T05:25:48.544893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:03.450940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562607+0000","last_change":"2024-09-17T05:25:48.555818+0000","last_active":"2024-09-17T05:25:48.562607+0000","last_peered":"2024-09-17T05:25:48.562607+0000","last_clean":"2024-09-17T05:25:48.562607+0000","last_became_active":"2024-09-17T05:25:48.555523+0000","last_became_peered":"2024-09-17T05:25:48.555523+0000","last_unstale":"2024-09-17T05:25:48.562607+0000","last_undegraded":"2024-09-17T05:25:48.562607+0000","last_fullsized":"2024-09-17T05:25:48.562607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.605791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.547937+0000","last_change":"2024-09-17T05:25:47.544235+0000","last_active":"2024-09-17T05:25:48.547937+0000","last_peered":"2024-09-17T05:25:48.547937+0000","last_clean":"2024-09-17T05:25:48.547937+0000","last_became_active":"2024-09-17T05:25:47.543878+0000","last_became_peered":"2024-09-17T05:25:47.543878+0000","last_unstale":"2024-09-17T05:25:48.547937+0000","last_undegraded":"2024-09-17T05:25:48.547937+0000","last_fullsized":"2024-09-17T05:25:48.547937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:29.899308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.582300+0000","last_change":"2024-09-17T05:25:48.578808+0000","last_active":"2024-09-17T05:25:48.582300+0000","last_peered":"2024-09-17T05:25:48.582300+0000","last_clean":"2024-09-17T05:25:48.582300+0000","last_became_active":"2024-09-17T05:25:48.578551+0000","last_became_peered":"2024-09-17T05:25:48.578551+0000","last_unstale":"2024-09-17T05:25:48.582300+0000","last_undegraded":"2024-09-17T05:25:48.582300+0000","last_fullsized":"2024-09-17T05:25:48.582300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:32.665784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.557152+0000","last_change":"2024-09-17T05:25:47.556128+0000","last_active":"2024-09-17T05:25:47.557152+0000","last_peered":"2024-09-17T05:25:47.557152+0000","last_clean":"2024-09-17T05:25:47.557152+0000","last_became_active":"2024-09-17T05:25:47.554436+0000","last_became_peered":"2024-09-17T05:25:47.554436+0000","last_unstale":"2024-09-17T05:25:47.557152+0000","last_undegraded":"2024-09-17T05:25:47.557152+0000","last_fullsized":"2024-09-17T05:25:47.557152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:13.641683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584821+0000","last_change":"2024-09-17T05:25:48.569539+0000","last_active":"2024-09-17T05:25:48.584821+0000","last_peered":"2024-09-17T05:25:48.584821+0000","last_clean":"2024-09-17T05:25:48.584821+0000","last_became_active":"2024-09-17T05:25:48.569292+0000","last_became_peered":"2024-09-17T05:25:48.569292+0000","last_unstale":"2024-09-17T05:25:48.584821+0000","last_undegraded":"2024-09-17T05:25:48.584821+0000","last_fullsized":"2024-09-17T05:25:48.584821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.554572+0000","last_change":"2024-09-17T05:25:47.549653+0000","last_active":"2024-09-17T05:25:47.554572+0000","last_peered":"2024-09-17T05:25:47.554572+0000","last_clean":"2024-09-17T05:25:47.554572+0000","last_became_active":"2024-09-17T05:25:47.548638+0000","last_became_peered":"2024-09-17T05:25:47.548638+0000","last_unstale":"2024-09-17T05:25:47.554572+0000","last_undegraded":"2024-09-17T05:25:47.554572+0000","last_fullsized":"2024-09-17T05:25:47.554572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:08.041081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583143+0000","last_change":"2024-09-17T05:25:48.568840+0000","last_active":"2024-09-17T05:25:48.583143+0000","last_peered":"2024-09-17T05:25:48.583143+0000","last_clean":"2024-09-17T05:25:48.583143+0000","last_became_active":"2024-09-17T05:25:48.568324+0000","last_became_peered":"2024-09-17T05:25:48.568324+0000","last_unstale":"2024-09-17T05:25:48.583143+0000","last_undegraded":"2024-09-17T05:25:48.583143+0000","last_fullsized":"2024-09-17T05:25:48.583143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580146+0000","last_change":"2024-09-17T05:25:47.541605+0000","last_active":"2024-09-17T05:25:48.580146+0000","last_peered":"2024-09-17T05:25:48.580146+0000","last_clean":"2024-09-17T05:25:48.580146+0000","last_became_active":"2024-09-17T05:25:47.541312+0000","last_became_peered":"2024-09-17T05:25:47.541312+0000","last_unstale":"2024-09-17T05:25:48.580146+0000","last_undegraded":"2024-09-17T05:25:48.580146+0000","last_fullsized":"2024-09-17T05:25:48.580146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:27.690980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559889+0000","last_change":"2024-09-17T05:25:48.556785+0000","last_active":"2024-09-17T05:25:48.559889+0000","last_peered":"2024-09-17T05:25:48.559889+0000","last_clean":"2024-09-17T05:25:48.559889+0000","last_became_active":"2024-09-17T05:25:48.556443+0000","last_became_peered":"2024-09-17T05:25:48.556443+0000","last_unstale":"2024-09-17T05:25:48.559889+0000","last_undegraded":"2024-09-17T05:25:48.559889+0000","last_fullsized":"2024-09-17T05:25:48.559889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:47.106037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.590749+0000","last_change":"2024-09-17T05:25:47.555429+0000","last_active":"2024-09-17T05:25:48.590749+0000","last_peered":"2024-09-17T05:25:48.590749+0000","last_clean":"2024-09-17T05:25:48.590749+0000","last_became_active":"2024-09-17T05:25:47.555144+0000","last_became_peered":"2024-09-17T05:25:47.555144+0000","last_unstale":"2024-09-17T05:25:48.590749+0000","last_undegraded":"2024-09-17T05:25:48.590749+0000","last_fullsized":"2024-09-17T05:25:48.590749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:24.436569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584084+0000","last_change":"2024-09-17T05:25:48.579894+0000","last_active":"2024-09-17T05:25:48.584084+0000","last_peered":"2024-09-17T05:25:48.584084+0000","last_clean":"2024-09-17T05:25:48.584084+0000","last_became_active":"2024-09-17T05:25:48.578255+0000","last_became_peered":"2024-09-17T05:25:48.578255+0000","last_unstale":"2024-09-17T05:25:48.584084+0000","last_undegraded":"2024-09-17T05:25:48.584084+0000","last_fullsized":"2024-09-17T05:25:48.584084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:58.305009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.579143+0000","last_change":"2024-09-17T05:25:47.548118+0000","last_active":"2024-09-17T05:25:48.579143+0000","last_peered":"2024-09-17T05:25:48.579143+0000","last_clean":"2024-09-17T05:25:48.579143+0000","last_became_active":"2024-09-17T05:25:47.547871+0000","last_became_peered":"2024-09-17T05:25:47.547871+0000","last_unstale":"2024-09-17T05:25:48.579143+0000","last_undegraded":"2024-09-17T05:25:48.579143+0000","last_fullsized":"2024-09-17T05:25:48.579143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585675+0000","last_change":"2024-09-17T05:25:48.582169+0000","last_active":"2024-09-17T05:25:48.585675+0000","last_peered":"2024-09-17T05:25:48.585675+0000","last_clean":"2024-09-17T05:25:48.585675+0000","last_became_active":"2024-09-17T05:25:48.581834+0000","last_became_peered":"2024-09-17T05:25:48.581834+0000","last_unstale":"2024-09-17T05:25:48.585675+0000","last_undegraded":"2024-09-17T05:25:48.585675+0000","last_fullsized":"2024-09-17T05:25:48.585675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:30.341407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.552340+0000","last_change":"2024-09-17T05:25:47.547173+0000","last_active":"2024-09-17T05:25:48.552340+0000","last_peered":"2024-09-17T05:25:48.552340+0000","last_clean":"2024-09-17T05:25:48.552340+0000","last_became_active":"2024-09-17T05:25:47.546912+0000","last_became_peered":"2024-09-17T05:25:47.546912+0000","last_unstale":"2024-09-17T05:25:48.552340+0000","last_undegraded":"2024-09-17T05:25:48.552340+0000","last_fullsized":"2024-09-17T05:25:48.552340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:26.869603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559321+0000","last_change":"2024-09-17T05:25:48.553296+0000","last_active":"2024-09-17T05:25:48.559321+0000","last_peered":"2024-09-17T05:25:48.559321+0000","last_clean":"2024-09-17T05:25:48.559321+0000","last_became_active":"2024-09-17T05:25:48.553058+0000","last_became_peered":"2024-09-17T05:25:48.553058+0000","last_unstale":"2024-09-17T05:25:48.559321+0000","last_undegraded":"2024-09-17T05:25:48.559321+0000","last_fullsized":"2024-09-17T05:25:48.559321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:39.217642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.549156+0000","last_change":"2024-09-17T05:25:47.553779+0000","last_active":"2024-09-17T05:25:48.549156+0000","last_peered":"2024-09-17T05:25:48.549156+0000","last_clean":"2024-09-17T05:25:48.549156+0000","last_became_active":"2024-09-17T05:25:47.553533+0000","last_became_peered":"2024-09-17T05:25:47.553533+0000","last_unstale":"2024-09-17T05:25:48.549156+0000","last_undegraded":"2024-09-17T05:25:48.549156+0000","last_fullsized":"2024-09-17T05:25:48.549156+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.550987+0000","last_change":"2024-09-17T05:25:48.549363+0000","last_active":"2024-09-17T05:25:48.550987+0000","last_peered":"2024-09-17T05:25:48.550987+0000","last_clean":"2024-09-17T05:25:48.550987+0000","last_became_active":"2024-09-17T05:25:48.549077+0000","last_became_peered":"2024-09-17T05:25:48.549077+0000","last_unstale":"2024-09-17T05:25:48.550987+0000","last_undegraded":"2024-09-17T05:25:48.550987+0000","last_fullsized":"2024-09-17T05:25:48.550987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:38.211479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585251+0000","last_change":"2024-09-17T05:25:47.545375+0000","last_active":"2024-09-17T05:25:48.585251+0000","last_peered":"2024-09-17T05:25:48.585251+0000","last_clean":"2024-09-17T05:25:48.585251+0000","last_became_active":"2024-09-17T05:25:47.545098+0000","last_became_peered":"2024-09-17T05:25:47.545098+0000","last_unstale":"2024-09-17T05:25:48.585251+0000","last_undegraded":"2024-09-17T05:25:48.585251+0000","last_fullsized":"2024-09-17T05:25:48.585251+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:10.436228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561228+0000","last_change":"2024-09-17T05:25:48.557134+0000","last_active":"2024-09-17T05:25:48.561228+0000","last_peered":"2024-09-17T05:25:48.561228+0000","last_clean":"2024-09-17T05:25:48.561228+0000","last_became_active":"2024-09-17T05:25:48.556815+0000","last_became_peered":"2024-09-17T05:25:48.556815+0000","last_unstale":"2024-09-17T05:25:48.561228+0000","last_undegraded":"2024-09-17T05:25:48.561228+0000","last_fullsized":"2024-09-17T05:25:48.561228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:41.791223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.586323+0000","last_change":"2024-09-17T05:25:47.551471+0000","last_active":"2024-09-17T05:25:48.586323+0000","last_peered":"2024-09-17T05:25:48.586323+0000","last_clean":"2024-09-17T05:25:48.586323+0000","last_became_active":"2024-09-17T05:25:47.550374+0000","last_became_peered":"2024-09-17T05:25:47.550374+0000","last_unstale":"2024-09-17T05:25:48.586323+0000","last_undegraded":"2024-09-17T05:25:48.586323+0000","last_fullsized":"2024-09-17T05:25:48.586323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:34.028724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561192+0000","last_change":"2024-09-17T05:25:48.559461+0000","last_active":"2024-09-17T05:25:48.561192+0000","last_peered":"2024-09-17T05:25:48.561192+0000","last_clean":"2024-09-17T05:25:48.561192+0000","last_became_active":"2024-09-17T05:25:48.558513+0000","last_became_peered":"2024-09-17T05:25:48.558513+0000","last_unstale":"2024-09-17T05:25:48.561192+0000","last_undegraded":"2024-09-17T05:25:48.561192+0000","last_fullsized":"2024-09-17T05:25:48.561192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:35.034136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.572667+0000","last_change":"2024-09-17T05:25:47.547687+0000","last_active":"2024-09-17T05:25:48.572667+0000","last_peered":"2024-09-17T05:25:48.572667+0000","last_clean":"2024-09-17T05:25:48.572667+0000","last_became_active":"2024-09-17T05:25:47.547426+0000","last_became_peered":"2024-09-17T05:25:47.547426+0000","last_unstale":"2024-09-17T05:25:48.572667+0000","last_undegraded":"2024-09-17T05:25:48.572667+0000","last_fullsized":"2024-09-17T05:25:48.572667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540500+0000","last_change":"2024-09-17T05:25:47.540500+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540500+0000","last_undegraded":"2024-09-17T05:25:47.540500+0000","last_fullsized":"2024-09-17T05:25:47.540500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580070+0000","last_change":"2024-09-17T05:25:47.546486+0000","last_active":"2024-09-17T05:25:48.580070+0000","last_peered":"2024-09-17T05:25:48.580070+0000","last_clean":"2024-09-17T05:25:48.580070+0000","last_became_active":"2024-09-17T05:25:47.545952+0000","last_became_peered":"2024-09-17T05:25:47.545952+0000","last_unstale":"2024-09-17T05:25:48.580070+0000","last_undegraded":"2024-09-17T05:25:48.580070+0000","last_fullsized":"2024-09-17T05:25:48.580070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:17.918336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568072+0000","last_change":"2024-09-17T05:25:48.563376+0000","last_active":"2024-09-17T05:25:48.568072+0000","last_peered":"2024-09-17T05:25:48.568072+0000","last_clean":"2024-09-17T05:25:48.568072+0000","last_became_active":"2024-09-17T05:25:48.563128+0000","last_became_peered":"2024-09-17T05:25:48.563128+0000","last_unstale":"2024-09-17T05:25:48.568072+0000","last_undegraded":"2024-09-17T05:25:48.568072+0000","last_fullsized":"2024-09-17T05:25:48.568072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:29.084922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.547970+0000","last_change":"2024-09-17T05:25:47.550977+0000","last_active":"2024-09-17T05:25:48.547970+0000","last_peered":"2024-09-17T05:25:48.547970+0000","last_clean":"2024-09-17T05:25:48.547970+0000","last_became_active":"2024-09-17T05:25:47.550677+0000","last_became_peered":"2024-09-17T05:25:47.550677+0000","last_unstale":"2024-09-17T05:25:48.547970+0000","last_undegraded":"2024-09-17T05:25:48.547970+0000","last_fullsized":"2024-09-17T05:25:48.547970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:12.600379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583964+0000","last_change":"2024-09-17T05:25:48.578870+0000","last_active":"2024-09-17T05:25:48.583964+0000","last_peered":"2024-09-17T05:25:48.583964+0000","last_clean":"2024-09-17T05:25:48.583964+0000","last_became_active":"2024-09-17T05:25:48.578503+0000","last_became_peered":"2024-09-17T05:25:48.578503+0000","last_unstale":"2024-09-17T05:25:48.583964+0000","last_undegraded":"2024-09-17T05:25:48.583964+0000","last_fullsized":"2024-09-17T05:25:48.583964+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:54.356062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.556401+0000","last_change":"2024-09-17T05:25:47.553626+0000","last_active":"2024-09-17T05:25:47.556401+0000","last_peered":"2024-09-17T05:25:47.556401+0000","last_clean":"2024-09-17T05:25:47.556401+0000","last_became_active":"2024-09-17T05:25:47.553385+0000","last_became_peered":"2024-09-17T05:25:47.553385+0000","last_unstale":"2024-09-17T05:25:47.556401+0000","last_undegraded":"2024-09-17T05:25:47.556401+0000","last_fullsized":"2024-09-17T05:25:47.556401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:59.695362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540601+0000","last_change":"2024-09-17T05:25:47.540601+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540601+0000","last_undegraded":"2024-09-17T05:25:47.540601+0000","last_fullsized":"2024-09-17T05:25:47.540601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.549201+0000","last_change":"2024-09-17T05:25:47.549913+0000","last_active":"2024-09-17T05:25:48.549201+0000","last_peered":"2024-09-17T05:25:48.549201+0000","last_clean":"2024-09-17T05:25:48.549201+0000","last_became_active":"2024-09-17T05:25:47.549593+0000","last_became_peered":"2024-09-17T05:25:47.549593+0000","last_unstale":"2024-09-17T05:25:48.549201+0000","last_undegraded":"2024-09-17T05:25:48.549201+0000","last_fullsized":"2024-09-17T05:25:48.549201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:08.157058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588111+0000","last_change":"2024-09-17T05:25:48.583799+0000","last_active":"2024-09-17T05:25:48.588111+0000","last_peered":"2024-09-17T05:25:48.588111+0000","last_clean":"2024-09-17T05:25:48.588111+0000","last_became_active":"2024-09-17T05:25:48.583489+0000","last_became_peered":"2024-09-17T05:25:48.583489+0000","last_unstale":"2024-09-17T05:25:48.588111+0000","last_undegraded":"2024-09-17T05:25:48.588111+0000","last_fullsized":"2024-09-17T05:25:48.588111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:39.321687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.540541+0000","last_change":"2024-09-17T05:25:47.542635+0000","last_active":"2024-09-17T05:25:48.540541+0000","last_peered":"2024-09-17T05:25:48.540541+0000","last_clean":"2024-09-17T05:25:48.540541+0000","last_became_active":"2024-09-17T05:25:47.542160+0000","last_became_peered":"2024-09-17T05:25:47.542160+0000","last_unstale":"2024-09-17T05:25:48.540541+0000","last_undegraded":"2024-09-17T05:25:48.540541+0000","last_fullsized":"2024-09-17T05:25:48.540541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:33.388849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559087+0000","last_change":"2024-09-17T05:25:48.552387+0000","last_active":"2024-09-17T05:25:48.559087+0000","last_peered":"2024-09-17T05:25:48.559087+0000","last_clean":"2024-09-17T05:25:48.559087+0000","last_became_active":"2024-09-17T05:25:48.552075+0000","last_became_peered":"2024-09-17T05:25:48.552075+0000","last_unstale":"2024-09-17T05:25:48.559087+0000","last_undegraded":"2024-09-17T05:25:48.559087+0000","last_fullsized":"2024-09-17T05:25:48.559087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.456352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.546124+0000","last_change":"2024-09-17T05:25:47.545902+0000","last_active":"2024-09-17T05:25:48.546124+0000","last_peered":"2024-09-17T05:25:48.546124+0000","last_clean":"2024-09-17T05:25:48.546124+0000","last_became_active":"2024-09-17T05:25:47.545602+0000","last_became_peered":"2024-09-17T05:25:47.545602+0000","last_unstale":"2024-09-17T05:25:48.546124+0000","last_undegraded":"2024-09-17T05:25:48.546124+0000","last_fullsized":"2024-09-17T05:25:48.546124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:34.536424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.570355+0000","last_change":"2024-09-17T05:25:48.566441+0000","last_active":"2024-09-17T05:25:48.570355+0000","last_peered":"2024-09-17T05:25:48.570355+0000","last_clean":"2024-09-17T05:25:48.570355+0000","last_became_active":"2024-09-17T05:25:48.566282+0000","last_became_peered":"2024-09-17T05:25:48.566282+0000","last_unstale":"2024-09-17T05:25:48.570355+0000","last_undegraded":"2024-09-17T05:25:48.570355+0000","last_fullsized":"2024-09-17T05:25:48.570355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:25.441981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585494+0000","last_change":"2024-09-17T05:25:47.549606+0000","last_active":"2024-09-17T05:25:48.585494+0000","last_peered":"2024-09-17T05:25:48.585494+0000","last_clean":"2024-09-17T05:25:48.585494+0000","last_became_active":"2024-09-17T05:25:47.549353+0000","last_became_peered":"2024-09-17T05:25:47.549353+0000","last_unstale":"2024-09-17T05:25:48.585494+0000","last_undegraded":"2024-09-17T05:25:48.585494+0000","last_fullsized":"2024-09-17T05:25:48.585494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:46.440233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562140+0000","last_change":"2024-09-17T05:25:48.560961+0000","last_active":"2024-09-17T05:25:48.562140+0000","last_peered":"2024-09-17T05:25:48.562140+0000","last_clean":"2024-09-17T05:25:48.562140+0000","last_became_active":"2024-09-17T05:25:48.560715+0000","last_became_peered":"2024-09-17T05:25:48.560715+0000","last_unstale":"2024-09-17T05:25:48.562140+0000","last_undegraded":"2024-09-17T05:25:48.562140+0000","last_fullsized":"2024-09-17T05:25:48.562140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:49.943382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.556405+0000","last_change":"2024-09-17T05:25:47.552693+0000","last_active":"2024-09-17T05:25:47.556405+0000","last_peered":"2024-09-17T05:25:47.556405+0000","last_clean":"2024-09-17T05:25:47.556405+0000","last_became_active":"2024-09-17T05:25:47.548298+0000","last_became_peered":"2024-09-17T05:25:47.548298+0000","last_unstale":"2024-09-17T05:25:47.556405+0000","last_undegraded":"2024-09-17T05:25:47.556405+0000","last_fullsized":"2024-09-17T05:25:47.556405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:32.326924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568002+0000","last_change":"2024-09-17T05:25:48.559435+0000","last_active":"2024-09-17T05:25:48.568002+0000","last_peered":"2024-09-17T05:25:48.568002+0000","last_clean":"2024-09-17T05:25:48.568002+0000","last_became_active":"2024-09-17T05:25:48.558731+0000","last_became_peered":"2024-09-17T05:25:48.558731+0000","last_unstale":"2024-09-17T05:25:48.568002+0000","last_undegraded":"2024-09-17T05:25:48.568002+0000","last_fullsized":"2024-09-17T05:25:48.568002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:28.971195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.558040+0000","last_change":"2024-09-17T05:25:47.547920+0000","last_active":"2024-09-17T05:25:48.558040+0000","last_peered":"2024-09-17T05:25:48.558040+0000","last_clean":"2024-09-17T05:25:48.558040+0000","last_became_active":"2024-09-17T05:25:47.547593+0000","last_became_peered":"2024-09-17T05:25:47.547593+0000","last_unstale":"2024-09-17T05:25:48.558040+0000","last_undegraded":"2024-09-17T05:25:48.558040+0000","last_fullsized":"2024-09-17T05:25:48.558040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:12.832450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563888+0000","last_change":"2024-09-17T05:25:48.556873+0000","last_active":"2024-09-17T05:25:48.563888+0000","last_peered":"2024-09-17T05:25:48.563888+0000","last_clean":"2024-09-17T05:25:48.563888+0000","last_became_active":"2024-09-17T05:25:48.556481+0000","last_became_peered":"2024-09-17T05:25:48.556481+0000","last_unstale":"2024-09-17T05:25:48.563888+0000","last_undegraded":"2024-09-17T05:25:48.563888+0000","last_fullsized":"2024-09-17T05:25:48.563888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.560890+0000","last_change":"2024-09-17T05:25:47.552565+0000","last_active":"2024-09-17T05:25:48.560890+0000","last_peered":"2024-09-17T05:25:48.560890+0000","last_clean":"2024-09-17T05:25:48.560890+0000","last_became_active":"2024-09-17T05:25:47.549868+0000","last_became_peered":"2024-09-17T05:25:47.549868+0000","last_unstale":"2024-09-17T05:25:48.560890+0000","last_undegraded":"2024-09-17T05:25:48.560890+0000","last_fullsized":"2024-09-17T05:25:48.560890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:45.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585649+0000","last_change":"2024-09-17T05:25:48.579795+0000","last_active":"2024-09-17T05:25:48.585649+0000","last_peered":"2024-09-17T05:25:48.585649+0000","last_clean":"2024-09-17T05:25:48.585649+0000","last_became_active":"2024-09-17T05:25:48.579382+0000","last_became_peered":"2024-09-17T05:25:48.579382+0000","last_unstale":"2024-09-17T05:25:48.585649+0000","last_undegraded":"2024-09-17T05:25:48.585649+0000","last_fullsized":"2024-09-17T05:25:48.585649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:34.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.560830+0000","last_change":"2024-09-17T05:25:47.548934+0000","last_active":"2024-09-17T05:25:48.560830+0000","last_peered":"2024-09-17T05:25:48.560830+0000","last_clean":"2024-09-17T05:25:48.560830+0000","last_became_active":"2024-09-17T05:25:47.548388+0000","last_became_peered":"2024-09-17T05:25:47.548388+0000","last_unstale":"2024-09-17T05:25:48.560830+0000","last_undegraded":"2024-09-17T05:25:48.560830+0000","last_fullsized":"2024-09-17T05:25:48.560830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:32.932862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.575376+0000","last_change":"2024-09-17T05:25:48.565031+0000","last_active":"2024-09-17T05:25:48.575376+0000","last_peered":"2024-09-17T05:25:48.575376+0000","last_clean":"2024-09-17T05:25:48.575376+0000","last_became_active":"2024-09-17T05:25:48.564697+0000","last_became_peered":"2024-09-17T05:25:48.564697+0000","last_unstale":"2024-09-17T05:25:48.575376+0000","last_undegraded":"2024-09-17T05:25:48.575376+0000","last_fullsized":"2024-09-17T05:25:48.575376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.548534+0000","last_change":"2024-09-17T05:25:47.548297+0000","last_active":"2024-09-17T05:25:48.548534+0000","last_peered":"2024-09-17T05:25:48.548534+0000","last_clean":"2024-09-17T05:25:48.548534+0000","last_became_active":"2024-09-17T05:25:47.548063+0000","last_became_peered":"2024-09-17T05:25:47.548063+0000","last_unstale":"2024-09-17T05:25:48.548534+0000","last_undegraded":"2024-09-17T05:25:48.548534+0000","last_fullsized":"2024-09-17T05:25:48.548534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:34.378981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564574+0000","last_change":"2024-09-17T05:25:48.562700+0000","last_active":"2024-09-17T05:25:48.564574+0000","last_peered":"2024-09-17T05:25:48.564574+0000","last_clean":"2024-09-17T05:25:48.564574+0000","last_became_active":"2024-09-17T05:25:48.562183+0000","last_became_peered":"2024-09-17T05:25:48.562183+0000","last_unstale":"2024-09-17T05:25:48.564574+0000","last_undegraded":"2024-09-17T05:25:48.564574+0000","last_fullsized":"2024-09-17T05:25:48.564574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:13.905356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.550500+0000","last_change":"2024-09-17T05:25:47.551652+0000","last_active":"2024-09-17T05:25:48.550500+0000","last_peered":"2024-09-17T05:25:48.550500+0000","last_clean":"2024-09-17T05:25:48.550500+0000","last_became_active":"2024-09-17T05:25:47.551332+0000","last_became_peered":"2024-09-17T05:25:47.551332+0000","last_unstale":"2024-09-17T05:25:48.550500+0000","last_undegraded":"2024-09-17T05:25:48.550500+0000","last_fullsized":"2024-09-17T05:25:48.550500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:38.212230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565013+0000","last_change":"2024-09-17T05:25:48.559526+0000","last_active":"2024-09-17T05:25:48.565013+0000","last_peered":"2024-09-17T05:25:48.565013+0000","last_clean":"2024-09-17T05:25:48.565013+0000","last_became_active":"2024-09-17T05:25:48.559292+0000","last_became_peered":"2024-09-17T05:25:48.559292+0000","last_unstale":"2024-09-17T05:25:48.565013+0000","last_undegraded":"2024-09-17T05:25:48.565013+0000","last_fullsized":"2024-09-17T05:25:48.565013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.545263+0000","last_change":"2024-09-17T05:25:47.546172+0000","last_active":"2024-09-17T05:25:48.545263+0000","last_peered":"2024-09-17T05:25:48.545263+0000","last_clean":"2024-09-17T05:25:48.545263+0000","last_became_active":"2024-09-17T05:25:47.545644+0000","last_became_peered":"2024-09-17T05:25:47.545644+0000","last_unstale":"2024-09-17T05:25:48.545263+0000","last_undegraded":"2024-09-17T05:25:48.545263+0000","last_fullsized":"2024-09-17T05:25:48.545263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:46.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":130,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.541021+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:48.541021+0000","last_peered":"2024-09-17T05:25:48.541021+0000","last_clean":"2024-09-17T05:25:48.541021+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:48.541021+0000","last_undegraded":"2024-09-17T05:25:48.541021+0000","last_fullsized":"2024-09-17T05:25:48.541021+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":368640,"data_stored":134818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":723622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":710502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:53.436 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-09-17T05:25:53.436 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:47.540092+0000', 'last_change': '2024-09-17T05:25:47.540092+0000', 'last_active': '2024-09-17T05:25:47.520978+0000', 'last_peered': '2024-09-17T05:25:47.520978+0000', 'last_clean': '2024-09-17T05:25:47.520978+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:47.540092+0000', 'last_undegraded': '2024-09-17T05:25:47.540092+0000', 'last_fullsized': '2024-09-17T05:25:47.540092+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:25:53.436 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-09-17T05:25:53.436 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:47.541154+0000', 'last_change': '2024-09-17T05:25:47.541154+0000', 'last_active': '2024-09-17T05:25:47.520978+0000', 'last_peered': '2024-09-17T05:25:47.520978+0000', 'last_clean': '2024-09-17T05:25:47.520978+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:47.541154+0000', 'last_undegraded': '2024-09-17T05:25:47.541154+0000', 'last_fullsized': '2024-09-17T05:25:47.541154+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:25:53.436 INFO:tasks.ceph.ceph_manager.ceph:PG 5.25 is not active+clean 2024-09-17T05:25:53.436 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:47.541326+0000', 'last_change': '2024-09-17T05:25:47.541326+0000', 'last_active': '2024-09-17T05:25:47.520978+0000', 'last_peered': '2024-09-17T05:25:47.520978+0000', 'last_clean': '2024-09-17T05:25:47.520978+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:47.541326+0000', 'last_undegraded': '2024-09-17T05:25:47.541326+0000', 'last_fullsized': '2024-09-17T05:25:47.541326+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:25:53.437 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-09-17T05:25:53.437 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:47.540500+0000', 'last_change': '2024-09-17T05:25:47.540500+0000', 'last_active': '2024-09-17T05:25:47.520978+0000', 'last_peered': '2024-09-17T05:25:47.520978+0000', 'last_clean': '2024-09-17T05:25:47.520978+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:47.540500+0000', 'last_undegraded': '2024-09-17T05:25:47.540500+0000', 'last_fullsized': '2024-09-17T05:25:47.540500+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:25:53.437 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-17T05:25:53.437 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:25:47.540601+0000', 'last_change': '2024-09-17T05:25:47.540601+0000', 'last_active': '2024-09-17T05:25:47.520978+0000', 'last_peered': '2024-09-17T05:25:47.520978+0000', 'last_clean': '2024-09-17T05:25:47.520978+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:25:47.540601+0000', 'last_undegraded': '2024-09-17T05:25:47.540601+0000', 'last_fullsized': '2024-09-17T05:25:47.540601+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'last_clean_scrub_stamp': '2024-09-17T05:25:47.520978+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:25:53.437 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:25:53.519 INFO:tasks.ceph.mds.d.smithi173.stderr:2024-09-17T05:25:53.518+0000 7f917c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:25:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- 172.21.15.122:0/24083707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 msgr2=0x7f84f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 --2- 172.21.15.122:0/24083707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 0x7f84f4004810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c390 tx=0x7f84f801ce10 comp rx=0 tx=0).stop 2024-09-17T05:25:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8515200640 1 -- 172.21.15.122:0/24083707 reap_dead start 2024-09-17T05:25:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- 172.21.15.122:0/24083707 shutdown_connections 2024-09-17T05:25:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- 172.21.15.122:0/24083707 >> 172.21.15.122:0/24083707 conn(0x7f85100d1e40 msgr2=0x7f85100d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- 172.21.15.122:0/24083707 shutdown_connections 2024-09-17T05:25:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- 172.21.15.122:0/24083707 wait complete. 2024-09-17T05:25:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 Processor -- start 2024-09-17T05:25:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- start start 2024-09-17T05:25:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 0x7f85101600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8510151370 0x7f8510160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85100c7b30 0x7f85100c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8510168050 con 0x7f84f4004220 2024-09-17T05:25:53.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8510167ed0 con 0x7f85100c7b30 2024-09-17T05:25:53.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.613+0000 7f8517200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f85101681d0 con 0x7f8510151370 2024-09-17T05:25:53.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 0x7f85101600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 0x7f85101600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53806/0 (socket says 172.21.15.122:53806) 2024-09-17T05:25:53.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 -- 172.21.15.122:0/3503259995 learned_addr learned my addr 172.21.15.122:0/3503259995 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:53.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 -- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8510151370 msgr2=0x7f8510160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f8507400640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8510151370 0x7f8510160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f8515200640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85100c7b30 0x7f85100c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8510151370 0x7f8510160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 -- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85100c7b30 msgr2=0x7f85100c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85100c7b30 0x7f85100c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 -- 172.21.15.122:0/3503259995 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85100c84f0 con 0x7f84f4004220 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f8507400640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8510151370 0x7f8510160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f8515200640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85100c7b30 0x7f85100c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850fe00640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 0x7f85101600d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8500006090 tx=0x7f8500006550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850cc00640 1 -- 172.21.15.122:0/3503259995 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500015070 con 0x7f84f4004220 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850cc00640 1 -- 172.21.15.122:0/3503259995 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f850000fb30 con 0x7f84f4004220 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85100b6fd0 con 0x7f84f4004220 2024-09-17T05:25:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850cc00640 1 -- 172.21.15.122:0/3503259995 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500011430 con 0x7f84f4004220 2024-09-17T05:25:53.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85100b7270 con 0x7f84f4004220 2024-09-17T05:25:53.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851015fec0 con 0x7f84f4004220 2024-09-17T05:25:53.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.617+0000 7f850cc00640 1 -- 172.21.15.122:0/3503259995 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f85000115d0 con 0x7f84f4004220 2024-09-17T05:25:53.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.621+0000 7f850cc00640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f84ec073230 0x7f84ec0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:53.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.621+0000 7f850cc00640 1 -- 172.21.15.122:0/3503259995 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f8500095100 con 0x7f84f4004220 2024-09-17T05:25:53.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.621+0000 7f8507400640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f84ec073230 0x7f84ec0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:53.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.621+0000 7f8507400640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f84ec073230 0x7f84ec0756f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84f80ac000 tx=0x7f84f8019c50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:53.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.625+0000 7f850cc00640 1 -- 172.21.15.122:0/3503259995 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f85000601a0 con 0x7f84f4004220 2024-09-17T05:25:53.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.849+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f85100b4660 con 0x7f84f4004220 2024-09-17T05:25:53.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.849+0000 7f850cc00640 1 -- 172.21.15.122:0/3503259995 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2753 (secure 0 0 0) 0x7f850005faa0 con 0x7f84f4004220 2024-09-17T05:25:53.855 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:53.855 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550693,"num_in_osds":8,"osd_in_since":1726550682,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592778,"bytes_used":229642240,"bytes_avail":772864471040,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2418,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":16,"btime":"2024-09-17T05:25:51:564038+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":4873}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-17T05:25:48.453399+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f84ec073230 msgr2=0x7f84ec0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f84ec073230 0x7f84ec0756f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84f80ac000 tx=0x7f84f8019c50 comp rx=0 tx=0).stop 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 msgr2=0x7f85101600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 0x7f85101600d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8500006090 tx=0x7f8500006550 comp rx=0 tx=0).stop 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 shutdown_connections 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f84ec073230 0x7f84ec0756f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85100c7b30 0x7f85100c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8510151370 0x7f8510160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 --2- 172.21.15.122:0/3503259995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84f4004220 0x7f85101600d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 >> 172.21.15.122:0/3503259995 conn(0x7f85100d1e40 msgr2=0x7f85100b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 shutdown_connections 2024-09-17T05:25:53.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:53.853+0000 7f8517200640 1 -- 172.21.15.122:0/3503259995 wait complete. 2024-09-17T05:25:53.892 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:25:53.892 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:54.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.029+0000 7f2cc5e00640 1 -- 172.21.15.122:0/1240700090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca4003a10 msgr2=0x7f2ca4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.029+0000 7f2cc5e00640 1 --2- 172.21.15.122:0/1240700090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca4003a10 0x7f2ca4003e30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009b60 tx=0x7f2cb001d2b0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.029+0000 7f2cbf400640 1 -- 172.21.15.122:0/1240700090 reap_dead start 2024-09-17T05:25:54.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.029+0000 7f2cc5e00640 1 -- 172.21.15.122:0/1240700090 shutdown_connections 2024-09-17T05:25:54.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.029+0000 7f2cc5e00640 1 -- 172.21.15.122:0/1240700090 >> 172.21.15.122:0/1240700090 conn(0x7f2cc00b18f0 msgr2=0x7f2cc00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:54.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.029+0000 7f2cc5e00640 1 -- 172.21.15.122:0/1240700090 shutdown_connections 2024-09-17T05:25:54.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.029+0000 7f2cc5e00640 1 -- 172.21.15.122:0/1240700090 wait complete. 2024-09-17T05:25:54.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 Processor -- start 2024-09-17T05:25:54.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 -- start start 2024-09-17T05:25:54.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca4003a10 0x7f2cc01fb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:54.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cc009b610 0x7f2cc00b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:54.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc014e340 0x7f2cc014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:54.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc00cde70 con 0x7f2cc014e340 2024-09-17T05:25:54.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc00cdcf0 con 0x7f2cc009b610 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc00cdff0 con 0x7f2ca4003a10 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc014e340 0x7f2cc014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc014e340 0x7f2cc014e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53834/0 (socket says 172.21.15.122:53834) 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 -- 172.21.15.122:0/3738008991 learned_addr learned my addr 172.21.15.122:0/3738008991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbea00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca4003a10 0x7f2cc01fb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cb6000640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cc009b610 0x7f2cc00b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 -- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca4003a10 msgr2=0x7f2cc01fb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca4003a10 0x7f2cc01fb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 -- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cc009b610 msgr2=0x7f2cc00b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cc009b610 0x7f2cc00b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 -- 172.21.15.122:0/3738008991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc01fbb10 con 0x7f2cc014e340 2024-09-17T05:25:54.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cb6000640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cc009b610 0x7f2cc00b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:54.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cbf400640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc014e340 0x7f2cc014e7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8002240 tx=0x7f2cb800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:54.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cb7e00640 1 -- 172.21.15.122:0/3738008991 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8015070 con 0x7f2cc014e340 2024-09-17T05:25:54.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cb7e00640 1 -- 172.21.15.122:0/3738008991 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb800ca80 con 0x7f2cc014e340 2024-09-17T05:25:54.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cb7e00640 1 -- 172.21.15.122:0/3738008991 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8011460 con 0x7f2cc014e340 2024-09-17T05:25:54.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01fbdf0 con 0x7f2cc014e340 2024-09-17T05:25:54.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.033+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc00adf80 con 0x7f2cc014e340 2024-09-17T05:25:54.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.037+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc015e900 con 0x7f2cc014e340 2024-09-17T05:25:54.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.037+0000 7f2cb7e00640 1 -- 172.21.15.122:0/3738008991 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2cb8018070 con 0x7f2cc014e340 2024-09-17T05:25:54.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.037+0000 7f2cb7e00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c073200 0x7f2c9c0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:54.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.037+0000 7f2cb7e00640 1 -- 172.21.15.122:0/3738008991 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2cb8094840 con 0x7f2cc014e340 2024-09-17T05:25:54.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.037+0000 7f2cbea00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c073200 0x7f2c9c0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:54.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.037+0000 7f2cbea00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c073200 0x7f2c9c0756c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00ae000 tx=0x7f2cb001a050 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.041+0000 7f2cb7e00640 1 -- 172.21.15.122:0/3738008991 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2cb805f290 con 0x7f2cc014e340 2024-09-17T05:25:54.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.209+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cc00cf720 con 0x7f2c9c073200 2024-09-17T05:25:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.213+0000 7f2cb7e00640 1 -- 172.21.15.122:0/3738008991 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251525 (secure 0 0 0) 0x7f2cc00cf720 con 0x7f2c9c073200 2024-09-17T05:25:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:54.222 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:54.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c073200 msgr2=0x7f2c9c0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c073200 0x7f2c9c0756c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00ae000 tx=0x7f2cb001a050 comp rx=0 tx=0).stop 2024-09-17T05:25:54.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc014e340 msgr2=0x7f2cc014e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc014e340 0x7f2cc014e7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8002240 tx=0x7f2cb800ecf0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 shutdown_connections 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c073200 0x7f2c9c0756c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc014e340 0x7f2cc014e7d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cc009b610 0x7f2cc00b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 --2- 172.21.15.122:0/3738008991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca4003a10 0x7f2cc01fb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 >> 172.21.15.122:0/3738008991 conn(0x7f2cc00b18f0 msgr2=0x7f2cc0160280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 shutdown_connections 2024-09-17T05:25:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:54.217+0000 7f2cc5e00640 1 -- 172.21.15.122:0/3738008991 wait complete. 2024-09-17T05:25:54.240 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-17T05:25:52.454124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224260,"kb_used_data":3848,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750460,"statfs":{"total":773094113280,"available":772864471040,"internally_reserved":0,"allocated":3940352,"data_stored":2200987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928592"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565313+0000","last_change":"2024-09-17T05:25:48.563496+0000","last_active":"2024-09-17T05:25:48.565313+0000","last_peered":"2024-09-17T05:25:48.565313+0000","last_clean":"2024-09-17T05:25:48.565313+0000","last_became_active":"2024-09-17T05:25:48.563114+0000","last_became_peered":"2024-09-17T05:25:48.563114+0000","last_unstale":"2024-09-17T05:25:48.565313+0000","last_undegraded":"2024-09-17T05:25:48.565313+0000","last_fullsized":"2024-09-17T05:25:48.565313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540092+0000","last_change":"2024-09-17T05:25:47.540092+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540092+0000","last_undegraded":"2024-09-17T05:25:47.540092+0000","last_fullsized":"2024-09-17T05:25:47.540092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.552034+0000","last_change":"2024-09-17T05:25:48.549461+0000","last_active":"2024-09-17T05:25:48.552034+0000","last_peered":"2024-09-17T05:25:48.552034+0000","last_clean":"2024-09-17T05:25:48.552034+0000","last_became_active":"2024-09-17T05:25:48.549168+0000","last_became_peered":"2024-09-17T05:25:48.549168+0000","last_unstale":"2024-09-17T05:25:48.552034+0000","last_undegraded":"2024-09-17T05:25:48.552034+0000","last_fullsized":"2024-09-17T05:25:48.552034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:15.092436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.574864+0000","last_change":"2024-09-17T05:25:48.565056+0000","last_active":"2024-09-17T05:25:48.574864+0000","last_peered":"2024-09-17T05:25:48.574864+0000","last_clean":"2024-09-17T05:25:48.574864+0000","last_became_active":"2024-09-17T05:25:48.564810+0000","last_became_peered":"2024-09-17T05:25:48.564810+0000","last_unstale":"2024-09-17T05:25:48.574864+0000","last_undegraded":"2024-09-17T05:25:48.574864+0000","last_fullsized":"2024-09-17T05:25:48.574864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.541154+0000","last_change":"2024-09-17T05:25:47.541154+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.541154+0000","last_undegraded":"2024-09-17T05:25:47.541154+0000","last_fullsized":"2024-09-17T05:25:47.541154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568106+0000","last_change":"2024-09-17T05:25:48.563410+0000","last_active":"2024-09-17T05:25:48.568106+0000","last_peered":"2024-09-17T05:25:48.568106+0000","last_clean":"2024-09-17T05:25:48.568106+0000","last_became_active":"2024-09-17T05:25:48.563094+0000","last_became_peered":"2024-09-17T05:25:48.563094+0000","last_unstale":"2024-09-17T05:25:48.568106+0000","last_undegraded":"2024-09-17T05:25:48.568106+0000","last_fullsized":"2024-09-17T05:25:48.568106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:47.445644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561093+0000","last_change":"2024-09-17T05:25:48.555613+0000","last_active":"2024-09-17T05:25:48.561093+0000","last_peered":"2024-09-17T05:25:48.561093+0000","last_clean":"2024-09-17T05:25:48.561093+0000","last_became_active":"2024-09-17T05:25:48.555015+0000","last_became_peered":"2024-09-17T05:25:48.555015+0000","last_unstale":"2024-09-17T05:25:48.561093+0000","last_undegraded":"2024-09-17T05:25:48.561093+0000","last_fullsized":"2024-09-17T05:25:48.561093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:05.497779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.576391+0000","last_change":"2024-09-17T05:25:48.571453+0000","last_active":"2024-09-17T05:25:48.576391+0000","last_peered":"2024-09-17T05:25:48.576391+0000","last_clean":"2024-09-17T05:25:48.576391+0000","last_became_active":"2024-09-17T05:25:48.571191+0000","last_became_peered":"2024-09-17T05:25:48.571191+0000","last_unstale":"2024-09-17T05:25:48.576391+0000","last_undegraded":"2024-09-17T05:25:48.576391+0000","last_fullsized":"2024-09-17T05:25:48.576391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566421+0000","last_change":"2024-09-17T05:25:48.563932+0000","last_active":"2024-09-17T05:25:48.566421+0000","last_peered":"2024-09-17T05:25:48.566421+0000","last_clean":"2024-09-17T05:25:48.566421+0000","last_became_active":"2024-09-17T05:25:48.563651+0000","last_became_peered":"2024-09-17T05:25:48.563651+0000","last_unstale":"2024-09-17T05:25:48.566421+0000","last_undegraded":"2024-09-17T05:25:48.566421+0000","last_fullsized":"2024-09-17T05:25:48.566421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:54.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584841+0000","last_change":"2024-09-17T05:25:48.569820+0000","last_active":"2024-09-17T05:25:48.584841+0000","last_peered":"2024-09-17T05:25:48.584841+0000","last_clean":"2024-09-17T05:25:48.584841+0000","last_became_active":"2024-09-17T05:25:48.569549+0000","last_became_peered":"2024-09-17T05:25:48.569549+0000","last_unstale":"2024-09-17T05:25:48.584841+0000","last_undegraded":"2024-09-17T05:25:48.584841+0000","last_fullsized":"2024-09-17T05:25:48.584841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559866+0000","last_change":"2024-09-17T05:25:48.556416+0000","last_active":"2024-09-17T05:25:48.559866+0000","last_peered":"2024-09-17T05:25:48.559866+0000","last_clean":"2024-09-17T05:25:48.559866+0000","last_became_active":"2024-09-17T05:25:48.556113+0000","last_became_peered":"2024-09-17T05:25:48.556113+0000","last_unstale":"2024-09-17T05:25:48.559866+0000","last_undegraded":"2024-09-17T05:25:48.559866+0000","last_fullsized":"2024-09-17T05:25:48.559866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:01.838706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580005+0000","last_change":"2024-09-17T05:25:48.578672+0000","last_active":"2024-09-17T05:25:48.580005+0000","last_peered":"2024-09-17T05:25:48.580005+0000","last_clean":"2024-09-17T05:25:48.580005+0000","last_became_active":"2024-09-17T05:25:48.577799+0000","last_became_peered":"2024-09-17T05:25:48.577799+0000","last_unstale":"2024-09-17T05:25:48.580005+0000","last_undegraded":"2024-09-17T05:25:48.580005+0000","last_fullsized":"2024-09-17T05:25:48.580005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:28.971195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588071+0000","last_change":"2024-09-17T05:25:48.583732+0000","last_active":"2024-09-17T05:25:48.588071+0000","last_peered":"2024-09-17T05:25:48.588071+0000","last_clean":"2024-09-17T05:25:48.588071+0000","last_became_active":"2024-09-17T05:25:48.583467+0000","last_became_peered":"2024-09-17T05:25:48.583467+0000","last_unstale":"2024-09-17T05:25:48.588071+0000","last_undegraded":"2024-09-17T05:25:48.588071+0000","last_fullsized":"2024-09-17T05:25:48.588071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:40.017087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.581449+0000","last_change":"2024-09-17T05:25:48.568509+0000","last_active":"2024-09-17T05:25:48.581449+0000","last_peered":"2024-09-17T05:25:48.581449+0000","last_clean":"2024-09-17T05:25:48.581449+0000","last_became_active":"2024-09-17T05:25:48.568196+0000","last_became_peered":"2024-09-17T05:25:48.568196+0000","last_unstale":"2024-09-17T05:25:48.581449+0000","last_undegraded":"2024-09-17T05:25:48.581449+0000","last_fullsized":"2024-09-17T05:25:48.581449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564320+0000","last_change":"2024-09-17T05:25:48.557116+0000","last_active":"2024-09-17T05:25:48.564320+0000","last_peered":"2024-09-17T05:25:48.564320+0000","last_clean":"2024-09-17T05:25:48.564320+0000","last_became_active":"2024-09-17T05:25:48.556674+0000","last_became_peered":"2024-09-17T05:25:48.556674+0000","last_unstale":"2024-09-17T05:25:48.564320+0000","last_undegraded":"2024-09-17T05:25:48.564320+0000","last_fullsized":"2024-09-17T05:25:48.564320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563788+0000","last_change":"2024-09-17T05:25:48.555359+0000","last_active":"2024-09-17T05:25:48.563788+0000","last_peered":"2024-09-17T05:25:48.563788+0000","last_clean":"2024-09-17T05:25:48.563788+0000","last_became_active":"2024-09-17T05:25:48.554078+0000","last_became_peered":"2024-09-17T05:25:48.554078+0000","last_unstale":"2024-09-17T05:25:48.563788+0000","last_undegraded":"2024-09-17T05:25:48.563788+0000","last_fullsized":"2024-09-17T05:25:48.563788+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:11.642957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559662+0000","last_change":"2024-09-17T05:25:48.555168+0000","last_active":"2024-09-17T05:25:48.559662+0000","last_peered":"2024-09-17T05:25:48.559662+0000","last_clean":"2024-09-17T05:25:48.559662+0000","last_became_active":"2024-09-17T05:25:48.554887+0000","last_became_peered":"2024-09-17T05:25:48.554887+0000","last_unstale":"2024-09-17T05:25:48.559662+0000","last_undegraded":"2024-09-17T05:25:48.559662+0000","last_fullsized":"2024-09-17T05:25:48.559662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:18.816281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585276+0000","last_change":"2024-09-17T05:25:48.580714+0000","last_active":"2024-09-17T05:25:48.585276+0000","last_peered":"2024-09-17T05:25:48.585276+0000","last_clean":"2024-09-17T05:25:48.585276+0000","last_became_active":"2024-09-17T05:25:48.580355+0000","last_became_peered":"2024-09-17T05:25:48.580355+0000","last_unstale":"2024-09-17T05:25:48.585276+0000","last_undegraded":"2024-09-17T05:25:48.585276+0000","last_fullsized":"2024-09-17T05:25:48.585276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:13.837862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.567853+0000","last_change":"2024-09-17T05:25:48.559308+0000","last_active":"2024-09-17T05:25:48.567853+0000","last_peered":"2024-09-17T05:25:48.567853+0000","last_clean":"2024-09-17T05:25:48.567853+0000","last_became_active":"2024-09-17T05:25:48.558922+0000","last_became_peered":"2024-09-17T05:25:48.558922+0000","last_unstale":"2024-09-17T05:25:48.567853+0000","last_undegraded":"2024-09-17T05:25:48.567853+0000","last_fullsized":"2024-09-17T05:25:48.567853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:54.187829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584951+0000","last_change":"2024-09-17T05:25:48.578440+0000","last_active":"2024-09-17T05:25:48.584951+0000","last_peered":"2024-09-17T05:25:48.584951+0000","last_clean":"2024-09-17T05:25:48.584951+0000","last_became_active":"2024-09-17T05:25:48.578166+0000","last_became_peered":"2024-09-17T05:25:48.578166+0000","last_unstale":"2024-09-17T05:25:48.584951+0000","last_undegraded":"2024-09-17T05:25:48.584951+0000","last_fullsized":"2024-09-17T05:25:48.584951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:00.303537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.578195+0000","last_change":"2024-09-17T05:25:48.576041+0000","last_active":"2024-09-17T05:25:48.578195+0000","last_peered":"2024-09-17T05:25:48.578195+0000","last_clean":"2024-09-17T05:25:48.578195+0000","last_became_active":"2024-09-17T05:25:48.575747+0000","last_became_peered":"2024-09-17T05:25:48.575747+0000","last_unstale":"2024-09-17T05:25:48.578195+0000","last_undegraded":"2024-09-17T05:25:48.578195+0000","last_fullsized":"2024-09-17T05:25:48.578195+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566379+0000","last_change":"2024-09-17T05:25:48.564740+0000","last_active":"2024-09-17T05:25:48.566379+0000","last_peered":"2024-09-17T05:25:48.566379+0000","last_clean":"2024-09-17T05:25:48.566379+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:48.566379+0000","last_undegraded":"2024-09-17T05:25:48.566379+0000","last_fullsized":"2024-09-17T05:25:48.566379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.582889+0000","last_change":"2024-09-17T05:25:48.575949+0000","last_active":"2024-09-17T05:25:48.582889+0000","last_peered":"2024-09-17T05:25:48.582889+0000","last_clean":"2024-09-17T05:25:48.582889+0000","last_became_active":"2024-09-17T05:25:48.573427+0000","last_became_peered":"2024-09-17T05:25:48.573427+0000","last_unstale":"2024-09-17T05:25:48.582889+0000","last_undegraded":"2024-09-17T05:25:48.582889+0000","last_fullsized":"2024-09-17T05:25:48.582889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:20.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.570555+0000","last_change":"2024-09-17T05:25:48.566361+0000","last_active":"2024-09-17T05:25:48.570555+0000","last_peered":"2024-09-17T05:25:48.570555+0000","last_clean":"2024-09-17T05:25:48.570555+0000","last_became_active":"2024-09-17T05:25:48.566054+0000","last_became_peered":"2024-09-17T05:25:48.566054+0000","last_unstale":"2024-09-17T05:25:48.570555+0000","last_undegraded":"2024-09-17T05:25:48.570555+0000","last_fullsized":"2024-09-17T05:25:48.570555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.575678+0000","last_change":"2024-09-17T05:25:48.565723+0000","last_active":"2024-09-17T05:25:48.575678+0000","last_peered":"2024-09-17T05:25:48.575678+0000","last_clean":"2024-09-17T05:25:48.575678+0000","last_became_active":"2024-09-17T05:25:48.565457+0000","last_became_peered":"2024-09-17T05:25:48.565457+0000","last_unstale":"2024-09-17T05:25:48.575678+0000","last_undegraded":"2024-09-17T05:25:48.575678+0000","last_fullsized":"2024-09-17T05:25:48.575678+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562166+0000","last_change":"2024-09-17T05:25:48.555370+0000","last_active":"2024-09-17T05:25:48.562166+0000","last_peered":"2024-09-17T05:25:48.562166+0000","last_clean":"2024-09-17T05:25:48.562166+0000","last_became_active":"2024-09-17T05:25:48.554994+0000","last_became_peered":"2024-09-17T05:25:48.554994+0000","last_unstale":"2024-09-17T05:25:48.562166+0000","last_undegraded":"2024-09-17T05:25:48.562166+0000","last_fullsized":"2024-09-17T05:25:48.562166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:11.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.541326+0000","last_change":"2024-09-17T05:25:47.541326+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.541326+0000","last_undegraded":"2024-09-17T05:25:47.541326+0000","last_fullsized":"2024-09-17T05:25:47.541326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.579598+0000","last_change":"2024-09-17T05:25:48.576799+0000","last_active":"2024-09-17T05:25:48.579598+0000","last_peered":"2024-09-17T05:25:48.579598+0000","last_clean":"2024-09-17T05:25:48.579598+0000","last_became_active":"2024-09-17T05:25:48.576460+0000","last_became_peered":"2024-09-17T05:25:48.576460+0000","last_unstale":"2024-09-17T05:25:48.579598+0000","last_undegraded":"2024-09-17T05:25:48.579598+0000","last_fullsized":"2024-09-17T05:25:48.579598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563457+0000","last_change":"2024-09-17T05:25:48.555831+0000","last_active":"2024-09-17T05:25:48.563457+0000","last_peered":"2024-09-17T05:25:48.563457+0000","last_clean":"2024-09-17T05:25:48.563457+0000","last_became_active":"2024-09-17T05:25:48.555546+0000","last_became_peered":"2024-09-17T05:25:48.555546+0000","last_unstale":"2024-09-17T05:25:48.563457+0000","last_undegraded":"2024-09-17T05:25:48.563457+0000","last_fullsized":"2024-09-17T05:25:48.563457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:45.988538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563859+0000","last_change":"2024-09-17T05:25:48.554655+0000","last_active":"2024-09-17T05:25:48.563859+0000","last_peered":"2024-09-17T05:25:48.563859+0000","last_clean":"2024-09-17T05:25:48.563859+0000","last_became_active":"2024-09-17T05:25:48.553919+0000","last_became_peered":"2024-09-17T05:25:48.553919+0000","last_unstale":"2024-09-17T05:25:48.563859+0000","last_undegraded":"2024-09-17T05:25:48.563859+0000","last_fullsized":"2024-09-17T05:25:48.563859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564934+0000","last_change":"2024-09-17T05:25:48.560930+0000","last_active":"2024-09-17T05:25:48.564934+0000","last_peered":"2024-09-17T05:25:48.564934+0000","last_clean":"2024-09-17T05:25:48.564934+0000","last_became_active":"2024-09-17T05:25:48.560700+0000","last_became_peered":"2024-09-17T05:25:48.560700+0000","last_unstale":"2024-09-17T05:25:48.564934+0000","last_undegraded":"2024-09-17T05:25:48.564934+0000","last_fullsized":"2024-09-17T05:25:48.564934+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:11.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.551685+0000","last_change":"2024-09-17T05:25:48.549526+0000","last_active":"2024-09-17T05:25:48.551685+0000","last_peered":"2024-09-17T05:25:48.551685+0000","last_clean":"2024-09-17T05:25:48.551685+0000","last_became_active":"2024-09-17T05:25:48.549278+0000","last_became_peered":"2024-09-17T05:25:48.549278+0000","last_unstale":"2024-09-17T05:25:48.551685+0000","last_undegraded":"2024-09-17T05:25:48.551685+0000","last_fullsized":"2024-09-17T05:25:48.551685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:32:40.352200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583989+0000","last_change":"2024-09-17T05:25:48.579263+0000","last_active":"2024-09-17T05:25:48.583989+0000","last_peered":"2024-09-17T05:25:48.583989+0000","last_clean":"2024-09-17T05:25:48.583989+0000","last_became_active":"2024-09-17T05:25:48.578226+0000","last_became_peered":"2024-09-17T05:25:48.578226+0000","last_unstale":"2024-09-17T05:25:48.583989+0000","last_undegraded":"2024-09-17T05:25:48.583989+0000","last_fullsized":"2024-09-17T05:25:48.583989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:49.685150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580051+0000","last_change":"2024-09-17T05:25:47.542992+0000","last_active":"2024-09-17T05:25:48.580051+0000","last_peered":"2024-09-17T05:25:48.580051+0000","last_clean":"2024-09-17T05:25:48.580051+0000","last_became_active":"2024-09-17T05:25:47.542678+0000","last_became_peered":"2024-09-17T05:25:47.542678+0000","last_unstale":"2024-09-17T05:25:48.580051+0000","last_undegraded":"2024-09-17T05:25:48.580051+0000","last_fullsized":"2024-09-17T05:25:48.580051+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:18.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562177+0000","last_change":"2024-09-17T05:25:48.558850+0000","last_active":"2024-09-17T05:25:48.562177+0000","last_peered":"2024-09-17T05:25:48.562177+0000","last_clean":"2024-09-17T05:25:48.562177+0000","last_became_active":"2024-09-17T05:25:48.558570+0000","last_became_peered":"2024-09-17T05:25:48.558570+0000","last_unstale":"2024-09-17T05:25:48.562177+0000","last_undegraded":"2024-09-17T05:25:48.562177+0000","last_fullsized":"2024-09-17T05:25:48.562177+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:27.875015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585957+0000","last_change":"2024-09-17T05:25:47.550543+0000","last_active":"2024-09-17T05:25:48.585957+0000","last_peered":"2024-09-17T05:25:48.585957+0000","last_clean":"2024-09-17T05:25:48.585957+0000","last_became_active":"2024-09-17T05:25:47.550230+0000","last_became_peered":"2024-09-17T05:25:47.550230+0000","last_unstale":"2024-09-17T05:25:48.585957+0000","last_undegraded":"2024-09-17T05:25:48.585957+0000","last_fullsized":"2024-09-17T05:25:48.585957+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565532+0000","last_change":"2024-09-17T05:25:48.561559+0000","last_active":"2024-09-17T05:25:48.565532+0000","last_peered":"2024-09-17T05:25:48.565532+0000","last_clean":"2024-09-17T05:25:48.565532+0000","last_became_active":"2024-09-17T05:25:48.561199+0000","last_became_peered":"2024-09-17T05:25:48.561199+0000","last_unstale":"2024-09-17T05:25:48.565532+0000","last_undegraded":"2024-09-17T05:25:48.565532+0000","last_fullsized":"2024-09-17T05:25:48.565532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.605791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.581425+0000","last_change":"2024-09-17T05:25:47.555529+0000","last_active":"2024-09-17T05:25:48.581425+0000","last_peered":"2024-09-17T05:25:48.581425+0000","last_clean":"2024-09-17T05:25:48.581425+0000","last_became_active":"2024-09-17T05:25:47.554844+0000","last_became_peered":"2024-09-17T05:25:47.554844+0000","last_unstale":"2024-09-17T05:25:48.581425+0000","last_undegraded":"2024-09-17T05:25:48.581425+0000","last_fullsized":"2024-09-17T05:25:48.581425+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:18.141436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585939+0000","last_change":"2024-09-17T05:25:48.577744+0000","last_active":"2024-09-17T05:25:48.585939+0000","last_peered":"2024-09-17T05:25:48.585939+0000","last_clean":"2024-09-17T05:25:48.585939+0000","last_became_active":"2024-09-17T05:25:48.577492+0000","last_became_peered":"2024-09-17T05:25:48.577492+0000","last_unstale":"2024-09-17T05:25:48.585939+0000","last_undegraded":"2024-09-17T05:25:48.585939+0000","last_fullsized":"2024-09-17T05:25:48.585939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.166901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588295+0000","last_change":"2024-09-17T05:25:47.557387+0000","last_active":"2024-09-17T05:25:48.588295+0000","last_peered":"2024-09-17T05:25:48.588295+0000","last_clean":"2024-09-17T05:25:48.588295+0000","last_became_active":"2024-09-17T05:25:47.557000+0000","last_became_peered":"2024-09-17T05:25:47.557000+0000","last_unstale":"2024-09-17T05:25:48.588295+0000","last_undegraded":"2024-09-17T05:25:48.588295+0000","last_fullsized":"2024-09-17T05:25:48.588295+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:04.492367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.566436+0000","last_change":"2024-09-17T05:25:48.563244+0000","last_active":"2024-09-17T05:25:48.566436+0000","last_peered":"2024-09-17T05:25:48.566436+0000","last_clean":"2024-09-17T05:25:48.566436+0000","last_became_active":"2024-09-17T05:25:48.562910+0000","last_became_peered":"2024-09-17T05:25:48.562910+0000","last_unstale":"2024-09-17T05:25:48.566436+0000","last_undegraded":"2024-09-17T05:25:48.566436+0000","last_fullsized":"2024-09-17T05:25:48.566436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:19.146848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.577584+0000","last_change":"2024-09-17T05:25:47.542868+0000","last_active":"2024-09-17T05:25:48.577584+0000","last_peered":"2024-09-17T05:25:48.577584+0000","last_clean":"2024-09-17T05:25:48.577584+0000","last_became_active":"2024-09-17T05:25:47.542655+0000","last_became_peered":"2024-09-17T05:25:47.542655+0000","last_unstale":"2024-09-17T05:25:48.577584+0000","last_undegraded":"2024-09-17T05:25:48.577584+0000","last_fullsized":"2024-09-17T05:25:48.577584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:02.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565297+0000","last_change":"2024-09-17T05:25:48.561829+0000","last_active":"2024-09-17T05:25:48.565297+0000","last_peered":"2024-09-17T05:25:48.565297+0000","last_clean":"2024-09-17T05:25:48.565297+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:48.565297+0000","last_undegraded":"2024-09-17T05:25:48.565297+0000","last_fullsized":"2024-09-17T05:25:48.565297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.577042+0000","last_change":"2024-09-17T05:25:47.552636+0000","last_active":"2024-09-17T05:25:48.577042+0000","last_peered":"2024-09-17T05:25:48.577042+0000","last_clean":"2024-09-17T05:25:48.577042+0000","last_became_active":"2024-09-17T05:25:47.552361+0000","last_became_peered":"2024-09-17T05:25:47.552361+0000","last_unstale":"2024-09-17T05:25:48.577042+0000","last_undegraded":"2024-09-17T05:25:48.577042+0000","last_fullsized":"2024-09-17T05:25:48.577042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:05.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561279+0000","last_change":"2024-09-17T05:25:48.553264+0000","last_active":"2024-09-17T05:25:48.561279+0000","last_peered":"2024-09-17T05:25:48.561279+0000","last_clean":"2024-09-17T05:25:48.561279+0000","last_became_active":"2024-09-17T05:25:48.552992+0000","last_became_peered":"2024-09-17T05:25:48.552992+0000","last_unstale":"2024-09-17T05:25:48.561279+0000","last_undegraded":"2024-09-17T05:25:48.561279+0000","last_fullsized":"2024-09-17T05:25:48.561279+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:11.441640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.544893+0000","last_change":"2024-09-17T05:25:47.544385+0000","last_active":"2024-09-17T05:25:48.544893+0000","last_peered":"2024-09-17T05:25:48.544893+0000","last_clean":"2024-09-17T05:25:48.544893+0000","last_became_active":"2024-09-17T05:25:47.544020+0000","last_became_peered":"2024-09-17T05:25:47.544020+0000","last_unstale":"2024-09-17T05:25:48.544893+0000","last_undegraded":"2024-09-17T05:25:48.544893+0000","last_fullsized":"2024-09-17T05:25:48.544893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:03.450940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562607+0000","last_change":"2024-09-17T05:25:48.555818+0000","last_active":"2024-09-17T05:25:48.562607+0000","last_peered":"2024-09-17T05:25:48.562607+0000","last_clean":"2024-09-17T05:25:48.562607+0000","last_became_active":"2024-09-17T05:25:48.555523+0000","last_became_peered":"2024-09-17T05:25:48.555523+0000","last_unstale":"2024-09-17T05:25:48.562607+0000","last_undegraded":"2024-09-17T05:25:48.562607+0000","last_fullsized":"2024-09-17T05:25:48.562607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:13.605791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.547937+0000","last_change":"2024-09-17T05:25:47.544235+0000","last_active":"2024-09-17T05:25:48.547937+0000","last_peered":"2024-09-17T05:25:48.547937+0000","last_clean":"2024-09-17T05:25:48.547937+0000","last_became_active":"2024-09-17T05:25:47.543878+0000","last_became_peered":"2024-09-17T05:25:47.543878+0000","last_unstale":"2024-09-17T05:25:48.547937+0000","last_undegraded":"2024-09-17T05:25:48.547937+0000","last_fullsized":"2024-09-17T05:25:48.547937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:29.899308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.582300+0000","last_change":"2024-09-17T05:25:48.578808+0000","last_active":"2024-09-17T05:25:48.582300+0000","last_peered":"2024-09-17T05:25:48.582300+0000","last_clean":"2024-09-17T05:25:48.582300+0000","last_became_active":"2024-09-17T05:25:48.578551+0000","last_became_peered":"2024-09-17T05:25:48.578551+0000","last_unstale":"2024-09-17T05:25:48.582300+0000","last_undegraded":"2024-09-17T05:25:48.582300+0000","last_fullsized":"2024-09-17T05:25:48.582300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:32.665784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.557152+0000","last_change":"2024-09-17T05:25:47.556128+0000","last_active":"2024-09-17T05:25:47.557152+0000","last_peered":"2024-09-17T05:25:47.557152+0000","last_clean":"2024-09-17T05:25:47.557152+0000","last_became_active":"2024-09-17T05:25:47.554436+0000","last_became_peered":"2024-09-17T05:25:47.554436+0000","last_unstale":"2024-09-17T05:25:47.557152+0000","last_undegraded":"2024-09-17T05:25:47.557152+0000","last_fullsized":"2024-09-17T05:25:47.557152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:13.641683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584821+0000","last_change":"2024-09-17T05:25:48.569539+0000","last_active":"2024-09-17T05:25:48.584821+0000","last_peered":"2024-09-17T05:25:48.584821+0000","last_clean":"2024-09-17T05:25:48.584821+0000","last_became_active":"2024-09-17T05:25:48.569292+0000","last_became_peered":"2024-09-17T05:25:48.569292+0000","last_unstale":"2024-09-17T05:25:48.584821+0000","last_undegraded":"2024-09-17T05:25:48.584821+0000","last_fullsized":"2024-09-17T05:25:48.584821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.554572+0000","last_change":"2024-09-17T05:25:47.549653+0000","last_active":"2024-09-17T05:25:47.554572+0000","last_peered":"2024-09-17T05:25:47.554572+0000","last_clean":"2024-09-17T05:25:47.554572+0000","last_became_active":"2024-09-17T05:25:47.548638+0000","last_became_peered":"2024-09-17T05:25:47.548638+0000","last_unstale":"2024-09-17T05:25:47.554572+0000","last_undegraded":"2024-09-17T05:25:47.554572+0000","last_fullsized":"2024-09-17T05:25:47.554572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:08.041081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583143+0000","last_change":"2024-09-17T05:25:48.568840+0000","last_active":"2024-09-17T05:25:48.583143+0000","last_peered":"2024-09-17T05:25:48.583143+0000","last_clean":"2024-09-17T05:25:48.583143+0000","last_became_active":"2024-09-17T05:25:48.568324+0000","last_became_peered":"2024-09-17T05:25:48.568324+0000","last_unstale":"2024-09-17T05:25:48.583143+0000","last_undegraded":"2024-09-17T05:25:48.583143+0000","last_fullsized":"2024-09-17T05:25:48.583143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:29.940183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580146+0000","last_change":"2024-09-17T05:25:47.541605+0000","last_active":"2024-09-17T05:25:48.580146+0000","last_peered":"2024-09-17T05:25:48.580146+0000","last_clean":"2024-09-17T05:25:48.580146+0000","last_became_active":"2024-09-17T05:25:47.541312+0000","last_became_peered":"2024-09-17T05:25:47.541312+0000","last_unstale":"2024-09-17T05:25:48.580146+0000","last_undegraded":"2024-09-17T05:25:48.580146+0000","last_fullsized":"2024-09-17T05:25:48.580146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:27.690980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559889+0000","last_change":"2024-09-17T05:25:48.556785+0000","last_active":"2024-09-17T05:25:48.559889+0000","last_peered":"2024-09-17T05:25:48.559889+0000","last_clean":"2024-09-17T05:25:48.559889+0000","last_became_active":"2024-09-17T05:25:48.556443+0000","last_became_peered":"2024-09-17T05:25:48.556443+0000","last_unstale":"2024-09-17T05:25:48.559889+0000","last_undegraded":"2024-09-17T05:25:48.559889+0000","last_fullsized":"2024-09-17T05:25:48.559889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:47.106037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.590749+0000","last_change":"2024-09-17T05:25:47.555429+0000","last_active":"2024-09-17T05:25:48.590749+0000","last_peered":"2024-09-17T05:25:48.590749+0000","last_clean":"2024-09-17T05:25:48.590749+0000","last_became_active":"2024-09-17T05:25:47.555144+0000","last_became_peered":"2024-09-17T05:25:47.555144+0000","last_unstale":"2024-09-17T05:25:48.590749+0000","last_undegraded":"2024-09-17T05:25:48.590749+0000","last_fullsized":"2024-09-17T05:25:48.590749+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:24.436569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.584084+0000","last_change":"2024-09-17T05:25:48.579894+0000","last_active":"2024-09-17T05:25:48.584084+0000","last_peered":"2024-09-17T05:25:48.584084+0000","last_clean":"2024-09-17T05:25:48.584084+0000","last_became_active":"2024-09-17T05:25:48.578255+0000","last_became_peered":"2024-09-17T05:25:48.578255+0000","last_unstale":"2024-09-17T05:25:48.584084+0000","last_undegraded":"2024-09-17T05:25:48.584084+0000","last_fullsized":"2024-09-17T05:25:48.584084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:58.305009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.579143+0000","last_change":"2024-09-17T05:25:47.548118+0000","last_active":"2024-09-17T05:25:48.579143+0000","last_peered":"2024-09-17T05:25:48.579143+0000","last_clean":"2024-09-17T05:25:48.579143+0000","last_became_active":"2024-09-17T05:25:47.547871+0000","last_became_peered":"2024-09-17T05:25:47.547871+0000","last_unstale":"2024-09-17T05:25:48.579143+0000","last_undegraded":"2024-09-17T05:25:48.579143+0000","last_fullsized":"2024-09-17T05:25:48.579143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585675+0000","last_change":"2024-09-17T05:25:48.582169+0000","last_active":"2024-09-17T05:25:48.585675+0000","last_peered":"2024-09-17T05:25:48.585675+0000","last_clean":"2024-09-17T05:25:48.585675+0000","last_became_active":"2024-09-17T05:25:48.581834+0000","last_became_peered":"2024-09-17T05:25:48.581834+0000","last_unstale":"2024-09-17T05:25:48.585675+0000","last_undegraded":"2024-09-17T05:25:48.585675+0000","last_fullsized":"2024-09-17T05:25:48.585675+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:30.341407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.552340+0000","last_change":"2024-09-17T05:25:47.547173+0000","last_active":"2024-09-17T05:25:48.552340+0000","last_peered":"2024-09-17T05:25:48.552340+0000","last_clean":"2024-09-17T05:25:48.552340+0000","last_became_active":"2024-09-17T05:25:47.546912+0000","last_became_peered":"2024-09-17T05:25:47.546912+0000","last_unstale":"2024-09-17T05:25:48.552340+0000","last_undegraded":"2024-09-17T05:25:48.552340+0000","last_fullsized":"2024-09-17T05:25:48.552340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:26.869603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559321+0000","last_change":"2024-09-17T05:25:48.553296+0000","last_active":"2024-09-17T05:25:48.559321+0000","last_peered":"2024-09-17T05:25:48.559321+0000","last_clean":"2024-09-17T05:25:48.559321+0000","last_became_active":"2024-09-17T05:25:48.553058+0000","last_became_peered":"2024-09-17T05:25:48.553058+0000","last_unstale":"2024-09-17T05:25:48.559321+0000","last_undegraded":"2024-09-17T05:25:48.559321+0000","last_fullsized":"2024-09-17T05:25:48.559321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:39.217642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.549156+0000","last_change":"2024-09-17T05:25:47.553779+0000","last_active":"2024-09-17T05:25:48.549156+0000","last_peered":"2024-09-17T05:25:48.549156+0000","last_clean":"2024-09-17T05:25:48.549156+0000","last_became_active":"2024-09-17T05:25:47.553533+0000","last_became_peered":"2024-09-17T05:25:47.553533+0000","last_unstale":"2024-09-17T05:25:48.549156+0000","last_undegraded":"2024-09-17T05:25:48.549156+0000","last_fullsized":"2024-09-17T05:25:48.549156+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.550987+0000","last_change":"2024-09-17T05:25:48.549363+0000","last_active":"2024-09-17T05:25:48.550987+0000","last_peered":"2024-09-17T05:25:48.550987+0000","last_clean":"2024-09-17T05:25:48.550987+0000","last_became_active":"2024-09-17T05:25:48.549077+0000","last_became_peered":"2024-09-17T05:25:48.549077+0000","last_unstale":"2024-09-17T05:25:48.550987+0000","last_undegraded":"2024-09-17T05:25:48.550987+0000","last_fullsized":"2024-09-17T05:25:48.550987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:38.211479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585251+0000","last_change":"2024-09-17T05:25:47.545375+0000","last_active":"2024-09-17T05:25:48.585251+0000","last_peered":"2024-09-17T05:25:48.585251+0000","last_clean":"2024-09-17T05:25:48.585251+0000","last_became_active":"2024-09-17T05:25:47.545098+0000","last_became_peered":"2024-09-17T05:25:47.545098+0000","last_unstale":"2024-09-17T05:25:48.585251+0000","last_undegraded":"2024-09-17T05:25:48.585251+0000","last_fullsized":"2024-09-17T05:25:48.585251+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:10.436228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561228+0000","last_change":"2024-09-17T05:25:48.557134+0000","last_active":"2024-09-17T05:25:48.561228+0000","last_peered":"2024-09-17T05:25:48.561228+0000","last_clean":"2024-09-17T05:25:48.561228+0000","last_became_active":"2024-09-17T05:25:48.556815+0000","last_became_peered":"2024-09-17T05:25:48.556815+0000","last_unstale":"2024-09-17T05:25:48.561228+0000","last_undegraded":"2024-09-17T05:25:48.561228+0000","last_fullsized":"2024-09-17T05:25:48.561228+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:41.791223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.586323+0000","last_change":"2024-09-17T05:25:47.551471+0000","last_active":"2024-09-17T05:25:48.586323+0000","last_peered":"2024-09-17T05:25:48.586323+0000","last_clean":"2024-09-17T05:25:48.586323+0000","last_became_active":"2024-09-17T05:25:47.550374+0000","last_became_peered":"2024-09-17T05:25:47.550374+0000","last_unstale":"2024-09-17T05:25:48.586323+0000","last_undegraded":"2024-09-17T05:25:48.586323+0000","last_fullsized":"2024-09-17T05:25:48.586323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:34.028724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.561192+0000","last_change":"2024-09-17T05:25:48.559461+0000","last_active":"2024-09-17T05:25:48.561192+0000","last_peered":"2024-09-17T05:25:48.561192+0000","last_clean":"2024-09-17T05:25:48.561192+0000","last_became_active":"2024-09-17T05:25:48.558513+0000","last_became_peered":"2024-09-17T05:25:48.558513+0000","last_unstale":"2024-09-17T05:25:48.561192+0000","last_undegraded":"2024-09-17T05:25:48.561192+0000","last_fullsized":"2024-09-17T05:25:48.561192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:35.034136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.572667+0000","last_change":"2024-09-17T05:25:47.547687+0000","last_active":"2024-09-17T05:25:48.572667+0000","last_peered":"2024-09-17T05:25:48.572667+0000","last_clean":"2024-09-17T05:25:48.572667+0000","last_became_active":"2024-09-17T05:25:47.547426+0000","last_became_peered":"2024-09-17T05:25:47.547426+0000","last_unstale":"2024-09-17T05:25:48.572667+0000","last_undegraded":"2024-09-17T05:25:48.572667+0000","last_fullsized":"2024-09-17T05:25:48.572667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540500+0000","last_change":"2024-09-17T05:25:47.540500+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540500+0000","last_undegraded":"2024-09-17T05:25:47.540500+0000","last_fullsized":"2024-09-17T05:25:47.540500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.580070+0000","last_change":"2024-09-17T05:25:47.546486+0000","last_active":"2024-09-17T05:25:48.580070+0000","last_peered":"2024-09-17T05:25:48.580070+0000","last_clean":"2024-09-17T05:25:48.580070+0000","last_became_active":"2024-09-17T05:25:47.545952+0000","last_became_peered":"2024-09-17T05:25:47.545952+0000","last_unstale":"2024-09-17T05:25:48.580070+0000","last_undegraded":"2024-09-17T05:25:48.580070+0000","last_fullsized":"2024-09-17T05:25:48.580070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:17.918336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568072+0000","last_change":"2024-09-17T05:25:48.563376+0000","last_active":"2024-09-17T05:25:48.568072+0000","last_peered":"2024-09-17T05:25:48.568072+0000","last_clean":"2024-09-17T05:25:48.568072+0000","last_became_active":"2024-09-17T05:25:48.563128+0000","last_became_peered":"2024-09-17T05:25:48.563128+0000","last_unstale":"2024-09-17T05:25:48.568072+0000","last_undegraded":"2024-09-17T05:25:48.568072+0000","last_fullsized":"2024-09-17T05:25:48.568072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:29.084922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.547970+0000","last_change":"2024-09-17T05:25:47.550977+0000","last_active":"2024-09-17T05:25:48.547970+0000","last_peered":"2024-09-17T05:25:48.547970+0000","last_clean":"2024-09-17T05:25:48.547970+0000","last_became_active":"2024-09-17T05:25:47.550677+0000","last_became_peered":"2024-09-17T05:25:47.550677+0000","last_unstale":"2024-09-17T05:25:48.547970+0000","last_undegraded":"2024-09-17T05:25:48.547970+0000","last_fullsized":"2024-09-17T05:25:48.547970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:12.600379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.583964+0000","last_change":"2024-09-17T05:25:48.578870+0000","last_active":"2024-09-17T05:25:48.583964+0000","last_peered":"2024-09-17T05:25:48.583964+0000","last_clean":"2024-09-17T05:25:48.583964+0000","last_became_active":"2024-09-17T05:25:48.578503+0000","last_became_peered":"2024-09-17T05:25:48.578503+0000","last_unstale":"2024-09-17T05:25:48.583964+0000","last_undegraded":"2024-09-17T05:25:48.583964+0000","last_fullsized":"2024-09-17T05:25:48.583964+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:54.356062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.556401+0000","last_change":"2024-09-17T05:25:47.553626+0000","last_active":"2024-09-17T05:25:47.556401+0000","last_peered":"2024-09-17T05:25:47.556401+0000","last_clean":"2024-09-17T05:25:47.556401+0000","last_became_active":"2024-09-17T05:25:47.553385+0000","last_became_peered":"2024-09-17T05:25:47.553385+0000","last_unstale":"2024-09-17T05:25:47.556401+0000","last_undegraded":"2024-09-17T05:25:47.556401+0000","last_fullsized":"2024-09-17T05:25:47.556401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:59.695362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T05:25:47.540601+0000","last_change":"2024-09-17T05:25:47.540601+0000","last_active":"2024-09-17T05:25:47.520978+0000","last_peered":"2024-09-17T05:25:47.520978+0000","last_clean":"2024-09-17T05:25:47.520978+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:25:47.540601+0000","last_undegraded":"2024-09-17T05:25:47.540601+0000","last_fullsized":"2024-09-17T05:25:47.540601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.549201+0000","last_change":"2024-09-17T05:25:47.549913+0000","last_active":"2024-09-17T05:25:48.549201+0000","last_peered":"2024-09-17T05:25:48.549201+0000","last_clean":"2024-09-17T05:25:48.549201+0000","last_became_active":"2024-09-17T05:25:47.549593+0000","last_became_peered":"2024-09-17T05:25:47.549593+0000","last_unstale":"2024-09-17T05:25:48.549201+0000","last_undegraded":"2024-09-17T05:25:48.549201+0000","last_fullsized":"2024-09-17T05:25:48.549201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:08.157058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.588111+0000","last_change":"2024-09-17T05:25:48.583799+0000","last_active":"2024-09-17T05:25:48.588111+0000","last_peered":"2024-09-17T05:25:48.588111+0000","last_clean":"2024-09-17T05:25:48.588111+0000","last_became_active":"2024-09-17T05:25:48.583489+0000","last_became_peered":"2024-09-17T05:25:48.583489+0000","last_unstale":"2024-09-17T05:25:48.588111+0000","last_undegraded":"2024-09-17T05:25:48.588111+0000","last_fullsized":"2024-09-17T05:25:48.588111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:39.321687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.540541+0000","last_change":"2024-09-17T05:25:47.542635+0000","last_active":"2024-09-17T05:25:48.540541+0000","last_peered":"2024-09-17T05:25:48.540541+0000","last_clean":"2024-09-17T05:25:48.540541+0000","last_became_active":"2024-09-17T05:25:47.542160+0000","last_became_peered":"2024-09-17T05:25:47.542160+0000","last_unstale":"2024-09-17T05:25:48.540541+0000","last_undegraded":"2024-09-17T05:25:48.540541+0000","last_fullsized":"2024-09-17T05:25:48.540541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:33.388849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.559087+0000","last_change":"2024-09-17T05:25:48.552387+0000","last_active":"2024-09-17T05:25:48.559087+0000","last_peered":"2024-09-17T05:25:48.559087+0000","last_clean":"2024-09-17T05:25:48.559087+0000","last_became_active":"2024-09-17T05:25:48.552075+0000","last_became_peered":"2024-09-17T05:25:48.552075+0000","last_unstale":"2024-09-17T05:25:48.559087+0000","last_undegraded":"2024-09-17T05:25:48.559087+0000","last_fullsized":"2024-09-17T05:25:48.559087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:04.456352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.546124+0000","last_change":"2024-09-17T05:25:47.545902+0000","last_active":"2024-09-17T05:25:48.546124+0000","last_peered":"2024-09-17T05:25:48.546124+0000","last_clean":"2024-09-17T05:25:48.546124+0000","last_became_active":"2024-09-17T05:25:47.545602+0000","last_became_peered":"2024-09-17T05:25:47.545602+0000","last_unstale":"2024-09-17T05:25:48.546124+0000","last_undegraded":"2024-09-17T05:25:48.546124+0000","last_fullsized":"2024-09-17T05:25:48.546124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:34.536424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.570355+0000","last_change":"2024-09-17T05:25:48.566441+0000","last_active":"2024-09-17T05:25:48.570355+0000","last_peered":"2024-09-17T05:25:48.570355+0000","last_clean":"2024-09-17T05:25:48.570355+0000","last_became_active":"2024-09-17T05:25:48.566282+0000","last_became_peered":"2024-09-17T05:25:48.566282+0000","last_unstale":"2024-09-17T05:25:48.570355+0000","last_undegraded":"2024-09-17T05:25:48.570355+0000","last_fullsized":"2024-09-17T05:25:48.570355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:25.441981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585494+0000","last_change":"2024-09-17T05:25:47.549606+0000","last_active":"2024-09-17T05:25:48.585494+0000","last_peered":"2024-09-17T05:25:48.585494+0000","last_clean":"2024-09-17T05:25:48.585494+0000","last_became_active":"2024-09-17T05:25:47.549353+0000","last_became_peered":"2024-09-17T05:25:47.549353+0000","last_unstale":"2024-09-17T05:25:48.585494+0000","last_undegraded":"2024-09-17T05:25:48.585494+0000","last_fullsized":"2024-09-17T05:25:48.585494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:46.440233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.562140+0000","last_change":"2024-09-17T05:25:48.560961+0000","last_active":"2024-09-17T05:25:48.562140+0000","last_peered":"2024-09-17T05:25:48.562140+0000","last_clean":"2024-09-17T05:25:48.562140+0000","last_became_active":"2024-09-17T05:25:48.560715+0000","last_became_peered":"2024-09-17T05:25:48.560715+0000","last_unstale":"2024-09-17T05:25:48.562140+0000","last_undegraded":"2024-09-17T05:25:48.562140+0000","last_fullsized":"2024-09-17T05:25:48.562140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:49.943382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-17T05:25:47.556405+0000","last_change":"2024-09-17T05:25:47.552693+0000","last_active":"2024-09-17T05:25:47.556405+0000","last_peered":"2024-09-17T05:25:47.556405+0000","last_clean":"2024-09-17T05:25:47.556405+0000","last_became_active":"2024-09-17T05:25:47.548298+0000","last_became_peered":"2024-09-17T05:25:47.548298+0000","last_unstale":"2024-09-17T05:25:47.556405+0000","last_undegraded":"2024-09-17T05:25:47.556405+0000","last_fullsized":"2024-09-17T05:25:47.556405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:32.326924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.568002+0000","last_change":"2024-09-17T05:25:48.559435+0000","last_active":"2024-09-17T05:25:48.568002+0000","last_peered":"2024-09-17T05:25:48.568002+0000","last_clean":"2024-09-17T05:25:48.568002+0000","last_became_active":"2024-09-17T05:25:48.558731+0000","last_became_peered":"2024-09-17T05:25:48.558731+0000","last_unstale":"2024-09-17T05:25:48.568002+0000","last_undegraded":"2024-09-17T05:25:48.568002+0000","last_fullsized":"2024-09-17T05:25:48.568002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:28.971195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.558040+0000","last_change":"2024-09-17T05:25:47.547920+0000","last_active":"2024-09-17T05:25:48.558040+0000","last_peered":"2024-09-17T05:25:48.558040+0000","last_clean":"2024-09-17T05:25:48.558040+0000","last_became_active":"2024-09-17T05:25:47.547593+0000","last_became_peered":"2024-09-17T05:25:47.547593+0000","last_unstale":"2024-09-17T05:25:48.558040+0000","last_undegraded":"2024-09-17T05:25:48.558040+0000","last_fullsized":"2024-09-17T05:25:48.558040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:12.832450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.563888+0000","last_change":"2024-09-17T05:25:48.556873+0000","last_active":"2024-09-17T05:25:48.563888+0000","last_peered":"2024-09-17T05:25:48.563888+0000","last_clean":"2024-09-17T05:25:48.563888+0000","last_became_active":"2024-09-17T05:25:48.556481+0000","last_became_peered":"2024-09-17T05:25:48.556481+0000","last_unstale":"2024-09-17T05:25:48.563888+0000","last_undegraded":"2024-09-17T05:25:48.563888+0000","last_fullsized":"2024-09-17T05:25:48.563888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:51:44.073981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.560890+0000","last_change":"2024-09-17T05:25:47.552565+0000","last_active":"2024-09-17T05:25:48.560890+0000","last_peered":"2024-09-17T05:25:48.560890+0000","last_clean":"2024-09-17T05:25:48.560890+0000","last_became_active":"2024-09-17T05:25:47.549868+0000","last_became_peered":"2024-09-17T05:25:47.549868+0000","last_unstale":"2024-09-17T05:25:48.560890+0000","last_undegraded":"2024-09-17T05:25:48.560890+0000","last_fullsized":"2024-09-17T05:25:48.560890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:45.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.585649+0000","last_change":"2024-09-17T05:25:48.579795+0000","last_active":"2024-09-17T05:25:48.585649+0000","last_peered":"2024-09-17T05:25:48.585649+0000","last_clean":"2024-09-17T05:25:48.585649+0000","last_became_active":"2024-09-17T05:25:48.579382+0000","last_became_peered":"2024-09-17T05:25:48.579382+0000","last_unstale":"2024-09-17T05:25:48.585649+0000","last_undegraded":"2024-09-17T05:25:48.585649+0000","last_fullsized":"2024-09-17T05:25:48.585649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:34.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.560830+0000","last_change":"2024-09-17T05:25:47.548934+0000","last_active":"2024-09-17T05:25:48.560830+0000","last_peered":"2024-09-17T05:25:48.560830+0000","last_clean":"2024-09-17T05:25:48.560830+0000","last_became_active":"2024-09-17T05:25:47.548388+0000","last_became_peered":"2024-09-17T05:25:47.548388+0000","last_unstale":"2024-09-17T05:25:48.560830+0000","last_undegraded":"2024-09-17T05:25:48.560830+0000","last_fullsized":"2024-09-17T05:25:48.560830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:32.932862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.575376+0000","last_change":"2024-09-17T05:25:48.565031+0000","last_active":"2024-09-17T05:25:48.575376+0000","last_peered":"2024-09-17T05:25:48.575376+0000","last_clean":"2024-09-17T05:25:48.575376+0000","last_became_active":"2024-09-17T05:25:48.564697+0000","last_became_peered":"2024-09-17T05:25:48.564697+0000","last_unstale":"2024-09-17T05:25:48.575376+0000","last_undegraded":"2024-09-17T05:25:48.575376+0000","last_fullsized":"2024-09-17T05:25:48.575376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:45.818147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.548534+0000","last_change":"2024-09-17T05:25:47.548297+0000","last_active":"2024-09-17T05:25:48.548534+0000","last_peered":"2024-09-17T05:25:48.548534+0000","last_clean":"2024-09-17T05:25:48.548534+0000","last_became_active":"2024-09-17T05:25:47.548063+0000","last_became_peered":"2024-09-17T05:25:47.548063+0000","last_unstale":"2024-09-17T05:25:48.548534+0000","last_undegraded":"2024-09-17T05:25:48.548534+0000","last_fullsized":"2024-09-17T05:25:48.548534+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:34.378981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.564574+0000","last_change":"2024-09-17T05:25:48.562700+0000","last_active":"2024-09-17T05:25:48.564574+0000","last_peered":"2024-09-17T05:25:48.564574+0000","last_clean":"2024-09-17T05:25:48.564574+0000","last_became_active":"2024-09-17T05:25:48.562183+0000","last_became_peered":"2024-09-17T05:25:48.562183+0000","last_unstale":"2024-09-17T05:25:48.564574+0000","last_undegraded":"2024-09-17T05:25:48.564574+0000","last_fullsized":"2024-09-17T05:25:48.564574+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:13.905356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.550500+0000","last_change":"2024-09-17T05:25:47.551652+0000","last_active":"2024-09-17T05:25:48.550500+0000","last_peered":"2024-09-17T05:25:48.550500+0000","last_clean":"2024-09-17T05:25:48.550500+0000","last_became_active":"2024-09-17T05:25:47.551332+0000","last_became_peered":"2024-09-17T05:25:47.551332+0000","last_unstale":"2024-09-17T05:25:48.550500+0000","last_undegraded":"2024-09-17T05:25:48.550500+0000","last_fullsized":"2024-09-17T05:25:48.550500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:38.212230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.565013+0000","last_change":"2024-09-17T05:25:48.559526+0000","last_active":"2024-09-17T05:25:48.565013+0000","last_peered":"2024-09-17T05:25:48.565013+0000","last_clean":"2024-09-17T05:25:48.565013+0000","last_became_active":"2024-09-17T05:25:48.559292+0000","last_became_peered":"2024-09-17T05:25:48.559292+0000","last_unstale":"2024-09-17T05:25:48.565013+0000","last_undegraded":"2024-09-17T05:25:48.565013+0000","last_fullsized":"2024-09-17T05:25:48.565013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:29.296945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.545263+0000","last_change":"2024-09-17T05:25:47.546172+0000","last_active":"2024-09-17T05:25:48.545263+0000","last_peered":"2024-09-17T05:25:48.545263+0000","last_clean":"2024-09-17T05:25:48.545263+0000","last_became_active":"2024-09-17T05:25:47.545644+0000","last_became_peered":"2024-09-17T05:25:47.545644+0000","last_unstale":"2024-09-17T05:25:48.545263+0000","last_undegraded":"2024-09-17T05:25:48.545263+0000","last_fullsized":"2024-09-17T05:25:48.545263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:46.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":130,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-17T05:25:48.541021+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:48.541021+0000","last_peered":"2024-09-17T05:25:48.541021+0000","last_clean":"2024-09-17T05:25:48.541021+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:48.541021+0000","last_undegraded":"2024-09-17T05:25:48.541021+0000","last_fullsized":"2024-09-17T05:25:48.541021+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":368640,"data_stored":134818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":723622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":710502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:57.248 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:25:57.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.393+0000 7fa4f0c00640 1 -- 172.21.15.122:0/4084062479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ec0cdb30 msgr2=0x7fa4ec0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.393+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/4084062479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ec0cdb30 0x7fa4ec0aeef0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009bc0 tx=0x7fa4e401cba0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.393+0000 7fa4f0c00640 1 -- 172.21.15.122:0/4084062479 shutdown_connections 2024-09-17T05:25:57.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.393+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/4084062479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ec0cdb30 0x7fa4ec0aeef0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.393+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/4084062479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ec0ced80 0x7fa4ec0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.393+0000 7fa4f0c00640 1 -- 172.21.15.122:0/4084062479 >> 172.21.15.122:0/4084062479 conn(0x7fa4ec0d1dc0 msgr2=0x7fa4ec0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:57.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.393+0000 7fa4f0c00640 1 -- 172.21.15.122:0/4084062479 shutdown_connections 2024-09-17T05:25:57.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.393+0000 7fa4f0c00640 1 -- 172.21.15.122:0/4084062479 wait complete. 2024-09-17T05:25:57.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 Processor -- start 2024-09-17T05:25:57.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 -- start start 2024-09-17T05:25:57.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec0cdb30 0x7fa4ec0c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ec0ced80 0x7fa4ec157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ec15bea0 0x7fa4ec0b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec169340 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1691c0 con 0x7fa4ec15bea0 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec1694c0 con 0x7fa4ec0ced80 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec0cdb30 0x7fa4ec0c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec0cdb30 0x7fa4ec0c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53838/0 (socket says 172.21.15.122:53838) 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 -- 172.21.15.122:0/2929203797 learned_addr learned my addr 172.21.15.122:0/2929203797 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e1400640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ec0ced80 0x7fa4ec157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4ea800640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ec15bea0 0x7fa4ec0b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 -- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ec0ced80 msgr2=0x7fa4ec157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ec0ced80 0x7fa4ec157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 -- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ec15bea0 msgr2=0x7fa4ec0b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ec15bea0 0x7fa4ec0b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 -- 172.21.15.122:0/2929203797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0c5ab0 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e9e00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec0cdb30 0x7fa4ec0c5570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00c3c0 tx=0x7fa4dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:57.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e2a00640 1 -- 172.21.15.122:0/2929203797 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc017070 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e2a00640 1 -- 172.21.15.122:0/2929203797 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4dc0053c0 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec0c9030 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4e2a00640 1 -- 172.21.15.122:0/2929203797 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc003860 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.397+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec0c95f0 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.401+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec0ce670 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.401+0000 7fa4e2a00640 1 -- 172.21.15.122:0/2929203797 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4dc01a070 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.401+0000 7fa4e2a00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4c8073200 0x7fa4c80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.401+0000 7fa4e2a00640 1 -- 172.21.15.122:0/2929203797 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa4dc093570 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.401+0000 7fa4e1400640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4c8073200 0x7fa4c80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.401+0000 7fa4e1400640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4c8073200 0x7fa4c80756c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec072800 tx=0x7fa4d4009260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:57.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.405+0000 7fa4e2a00640 1 -- 172.21.15.122:0/2929203797 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa4dc05dea0 con 0x7fa4ec0cdb30 2024-09-17T05:25:57.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.569+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4ec0d23f0 con 0x7fa4c8073200 2024-09-17T05:25:57.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.573+0000 7fa4e2a00640 1 -- 172.21.15.122:0/2929203797 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7fa4ec0d23f0 con 0x7fa4c8073200 2024-09-17T05:25:57.581 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:57.582 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:25:57.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4c8073200 msgr2=0x7fa4c80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4c8073200 0x7fa4c80756c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec072800 tx=0x7fa4d4009260 comp rx=0 tx=0).stop 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec0cdb30 msgr2=0x7fa4ec0c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec0cdb30 0x7fa4ec0c5570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00c3c0 tx=0x7fa4dc00c880 comp rx=0 tx=0).stop 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 shutdown_connections 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4c8073200 0x7fa4c80756c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ec15bea0 0x7fa4ec0b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ec0ced80 0x7fa4ec157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 --2- 172.21.15.122:0/2929203797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4ec0cdb30 0x7fa4ec0c5570 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 >> 172.21.15.122:0/2929203797 conn(0x7fa4ec0d1dc0 msgr2=0x7fa4ec0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 shutdown_connections 2024-09-17T05:25:57.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.577+0000 7fa4f0c00640 1 -- 172.21.15.122:0/2929203797 wait complete. 2024-09-17T05:25:57.602 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-09-17T05:25:56.455123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224468,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750252,"statfs":{"total":773094113280,"available":772864258048,"internally_reserved":0,"allocated":4169728,"data_stored":2301428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001801"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550928+0000","last_change":"2024-09-17T05:25:48.563496+0000","last_active":"2024-09-17T05:25:49.550928+0000","last_peered":"2024-09-17T05:25:49.550928+0000","last_clean":"2024-09-17T05:25:49.550928+0000","last_became_active":"2024-09-17T05:25:48.563114+0000","last_became_peered":"2024-09-17T05:25:48.563114+0000","last_unstale":"2024-09-17T05:25:49.550928+0000","last_undegraded":"2024-09-17T05:25:49.550928+0000","last_fullsized":"2024-09-17T05:25:49.550928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:11.419877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569754+0000","last_change":"2024-09-17T05:25:48.566399+0000","last_active":"2024-09-17T05:25:49.569754+0000","last_peered":"2024-09-17T05:25:49.569754+0000","last_clean":"2024-09-17T05:25:49.569754+0000","last_became_active":"2024-09-17T05:25:48.566092+0000","last_became_peered":"2024-09-17T05:25:48.566092+0000","last_unstale":"2024-09-17T05:25:49.569754+0000","last_undegraded":"2024-09-17T05:25:49.569754+0000","last_fullsized":"2024-09-17T05:25:49.569754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:55.166653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.575114+0000","last_change":"2024-09-17T05:25:48.549461+0000","last_active":"2024-09-17T05:25:49.575114+0000","last_peered":"2024-09-17T05:25:49.575114+0000","last_clean":"2024-09-17T05:25:49.575114+0000","last_became_active":"2024-09-17T05:25:48.549168+0000","last_became_peered":"2024-09-17T05:25:48.549168+0000","last_unstale":"2024-09-17T05:25:49.575114+0000","last_undegraded":"2024-09-17T05:25:49.575114+0000","last_fullsized":"2024-09-17T05:25:49.575114+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:33.938274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578183+0000","last_change":"2024-09-17T05:25:48.565056+0000","last_active":"2024-09-17T05:25:49.578183+0000","last_peered":"2024-09-17T05:25:49.578183+0000","last_clean":"2024-09-17T05:25:49.578183+0000","last_became_active":"2024-09-17T05:25:48.564810+0000","last_became_peered":"2024-09-17T05:25:48.564810+0000","last_unstale":"2024-09-17T05:25:49.578183+0000","last_undegraded":"2024-09-17T05:25:49.578183+0000","last_fullsized":"2024-09-17T05:25:49.578183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:11.419877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569476+0000","last_change":"2024-09-17T05:25:48.579269+0000","last_active":"2024-09-17T05:25:49.569476+0000","last_peered":"2024-09-17T05:25:49.569476+0000","last_clean":"2024-09-17T05:25:49.569476+0000","last_became_active":"2024-09-17T05:25:48.578917+0000","last_became_peered":"2024-09-17T05:25:48.578917+0000","last_unstale":"2024-09-17T05:25:49.569476+0000","last_undegraded":"2024-09-17T05:25:49.569476+0000","last_fullsized":"2024-09-17T05:25:49.569476+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:49.740512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.557935+0000","last_change":"2024-09-17T05:25:48.563410+0000","last_active":"2024-09-17T05:25:49.557935+0000","last_peered":"2024-09-17T05:25:49.557935+0000","last_clean":"2024-09-17T05:25:49.557935+0000","last_became_active":"2024-09-17T05:25:48.563094+0000","last_became_peered":"2024-09-17T05:25:48.563094+0000","last_unstale":"2024-09-17T05:25:49.557935+0000","last_undegraded":"2024-09-17T05:25:49.557935+0000","last_fullsized":"2024-09-17T05:25:49.557935+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:46.362496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550003+0000","last_change":"2024-09-17T05:25:48.555613+0000","last_active":"2024-09-17T05:25:49.550003+0000","last_peered":"2024-09-17T05:25:49.550003+0000","last_clean":"2024-09-17T05:25:49.550003+0000","last_became_active":"2024-09-17T05:25:48.555015+0000","last_became_peered":"2024-09-17T05:25:48.555015+0000","last_unstale":"2024-09-17T05:25:49.550003+0000","last_undegraded":"2024-09-17T05:25:49.550003+0000","last_fullsized":"2024-09-17T05:25:49.550003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:34.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.549878+0000","last_change":"2024-09-17T05:25:48.571453+0000","last_active":"2024-09-17T05:25:49.549878+0000","last_peered":"2024-09-17T05:25:49.549878+0000","last_clean":"2024-09-17T05:25:49.549878+0000","last_became_active":"2024-09-17T05:25:48.571191+0000","last_became_peered":"2024-09-17T05:25:48.571191+0000","last_unstale":"2024-09-17T05:25:49.549878+0000","last_undegraded":"2024-09-17T05:25:49.549878+0000","last_fullsized":"2024-09-17T05:25:49.549878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.858418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.575403+0000","last_change":"2024-09-17T05:25:48.563932+0000","last_active":"2024-09-17T05:25:49.575403+0000","last_peered":"2024-09-17T05:25:49.575403+0000","last_clean":"2024-09-17T05:25:49.575403+0000","last_became_active":"2024-09-17T05:25:48.563651+0000","last_became_peered":"2024-09-17T05:25:48.563651+0000","last_unstale":"2024-09-17T05:25:49.575403+0000","last_undegraded":"2024-09-17T05:25:49.575403+0000","last_fullsized":"2024-09-17T05:25:49.575403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:46.362496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578334+0000","last_change":"2024-09-17T05:25:48.569820+0000","last_active":"2024-09-17T05:25:49.578334+0000","last_peered":"2024-09-17T05:25:49.578334+0000","last_clean":"2024-09-17T05:25:49.578334+0000","last_became_active":"2024-09-17T05:25:48.569549+0000","last_became_peered":"2024-09-17T05:25:48.569549+0000","last_unstale":"2024-09-17T05:25:49.578334+0000","last_undegraded":"2024-09-17T05:25:49.578334+0000","last_fullsized":"2024-09-17T05:25:49.578334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:43.799177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.554969+0000","last_change":"2024-09-17T05:25:48.556416+0000","last_active":"2024-09-17T05:25:49.554969+0000","last_peered":"2024-09-17T05:25:49.554969+0000","last_clean":"2024-09-17T05:25:49.554969+0000","last_became_active":"2024-09-17T05:25:48.556113+0000","last_became_peered":"2024-09-17T05:25:48.556113+0000","last_unstale":"2024-09-17T05:25:49.554969+0000","last_undegraded":"2024-09-17T05:25:49.554969+0000","last_fullsized":"2024-09-17T05:25:49.554969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:17.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.575035+0000","last_change":"2024-09-17T05:25:48.578672+0000","last_active":"2024-09-17T05:25:49.575035+0000","last_peered":"2024-09-17T05:25:49.575035+0000","last_clean":"2024-09-17T05:25:49.575035+0000","last_became_active":"2024-09-17T05:25:48.577799+0000","last_became_peered":"2024-09-17T05:25:48.577799+0000","last_unstale":"2024-09-17T05:25:49.575035+0000","last_undegraded":"2024-09-17T05:25:49.575035+0000","last_fullsized":"2024-09-17T05:25:49.575035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:13.837862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577677+0000","last_change":"2024-09-17T05:25:48.583732+0000","last_active":"2024-09-17T05:25:49.577677+0000","last_peered":"2024-09-17T05:25:49.577677+0000","last_clean":"2024-09-17T05:25:49.577677+0000","last_became_active":"2024-09-17T05:25:48.583467+0000","last_became_peered":"2024-09-17T05:25:48.583467+0000","last_unstale":"2024-09-17T05:25:49.577677+0000","last_undegraded":"2024-09-17T05:25:49.577677+0000","last_fullsized":"2024-09-17T05:25:49.577677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:45.850018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578446+0000","last_change":"2024-09-17T05:25:48.568509+0000","last_active":"2024-09-17T05:25:49.578446+0000","last_peered":"2024-09-17T05:25:49.578446+0000","last_clean":"2024-09-17T05:25:49.578446+0000","last_became_active":"2024-09-17T05:25:48.568196+0000","last_became_peered":"2024-09-17T05:25:48.568196+0000","last_unstale":"2024-09-17T05:25:49.578446+0000","last_undegraded":"2024-09-17T05:25:49.578446+0000","last_fullsized":"2024-09-17T05:25:49.578446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:09:36.397096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574113+0000","last_change":"2024-09-17T05:25:48.557116+0000","last_active":"2024-09-17T05:25:49.574113+0000","last_peered":"2024-09-17T05:25:49.574113+0000","last_clean":"2024-09-17T05:25:49.574113+0000","last_became_active":"2024-09-17T05:25:48.556674+0000","last_became_peered":"2024-09-17T05:25:48.556674+0000","last_unstale":"2024-09-17T05:25:49.574113+0000","last_undegraded":"2024-09-17T05:25:49.574113+0000","last_fullsized":"2024-09-17T05:25:49.574113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:34.781954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574071+0000","last_change":"2024-09-17T05:25:48.555359+0000","last_active":"2024-09-17T05:25:49.574071+0000","last_peered":"2024-09-17T05:25:49.574071+0000","last_clean":"2024-09-17T05:25:49.574071+0000","last_became_active":"2024-09-17T05:25:48.554078+0000","last_became_peered":"2024-09-17T05:25:48.554078+0000","last_unstale":"2024-09-17T05:25:49.574071+0000","last_undegraded":"2024-09-17T05:25:49.574071+0000","last_fullsized":"2024-09-17T05:25:49.574071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.166901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.555180+0000","last_change":"2024-09-17T05:25:48.555168+0000","last_active":"2024-09-17T05:25:49.555180+0000","last_peered":"2024-09-17T05:25:49.555180+0000","last_clean":"2024-09-17T05:25:49.555180+0000","last_became_active":"2024-09-17T05:25:48.554887+0000","last_became_peered":"2024-09-17T05:25:48.554887+0000","last_unstale":"2024-09-17T05:25:49.555180+0000","last_undegraded":"2024-09-17T05:25:49.555180+0000","last_fullsized":"2024-09-17T05:25:49.555180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:54.533502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.558404+0000","last_change":"2024-09-17T05:25:48.580714+0000","last_active":"2024-09-17T05:25:49.558404+0000","last_peered":"2024-09-17T05:25:49.558404+0000","last_clean":"2024-09-17T05:25:49.558404+0000","last_became_active":"2024-09-17T05:25:48.580355+0000","last_became_peered":"2024-09-17T05:25:48.580355+0000","last_unstale":"2024-09-17T05:25:49.558404+0000","last_undegraded":"2024-09-17T05:25:49.558404+0000","last_fullsized":"2024-09-17T05:25:49.558404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:49.685150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.557943+0000","last_change":"2024-09-17T05:25:48.559308+0000","last_active":"2024-09-17T05:25:49.557943+0000","last_peered":"2024-09-17T05:25:49.557943+0000","last_clean":"2024-09-17T05:25:49.557943+0000","last_became_active":"2024-09-17T05:25:48.558922+0000","last_became_peered":"2024-09-17T05:25:48.558922+0000","last_unstale":"2024-09-17T05:25:49.557943+0000","last_undegraded":"2024-09-17T05:25:49.557943+0000","last_fullsized":"2024-09-17T05:25:49.557943+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:33.938274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578215+0000","last_change":"2024-09-17T05:25:48.578440+0000","last_active":"2024-09-17T05:25:49.578215+0000","last_peered":"2024-09-17T05:25:49.578215+0000","last_clean":"2024-09-17T05:25:49.578215+0000","last_became_active":"2024-09-17T05:25:48.578166+0000","last_became_peered":"2024-09-17T05:25:48.578166+0000","last_unstale":"2024-09-17T05:25:49.578215+0000","last_undegraded":"2024-09-17T05:25:49.578215+0000","last_fullsized":"2024-09-17T05:25:49.578215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:31.499889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574243+0000","last_change":"2024-09-17T05:25:48.576041+0000","last_active":"2024-09-17T05:25:49.574243+0000","last_peered":"2024-09-17T05:25:49.574243+0000","last_clean":"2024-09-17T05:25:49.574243+0000","last_became_active":"2024-09-17T05:25:48.575747+0000","last_became_peered":"2024-09-17T05:25:48.575747+0000","last_unstale":"2024-09-17T05:25:49.574243+0000","last_undegraded":"2024-09-17T05:25:49.574243+0000","last_fullsized":"2024-09-17T05:25:49.574243+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:48.318207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550335+0000","last_change":"2024-09-17T05:25:48.564740+0000","last_active":"2024-09-17T05:25:49.550335+0000","last_peered":"2024-09-17T05:25:49.550335+0000","last_clean":"2024-09-17T05:25:49.550335+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:49.550335+0000","last_undegraded":"2024-09-17T05:25:49.550335+0000","last_fullsized":"2024-09-17T05:25:49.550335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:39.261176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577186+0000","last_change":"2024-09-17T05:25:48.575949+0000","last_active":"2024-09-17T05:25:49.577186+0000","last_peered":"2024-09-17T05:25:49.577186+0000","last_clean":"2024-09-17T05:25:49.577186+0000","last_became_active":"2024-09-17T05:25:48.573427+0000","last_became_peered":"2024-09-17T05:25:48.573427+0000","last_unstale":"2024-09-17T05:25:49.577186+0000","last_undegraded":"2024-09-17T05:25:49.577186+0000","last_fullsized":"2024-09-17T05:25:49.577186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:02.623784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.549928+0000","last_change":"2024-09-17T05:25:48.566361+0000","last_active":"2024-09-17T05:25:49.549928+0000","last_peered":"2024-09-17T05:25:49.549928+0000","last_clean":"2024-09-17T05:25:49.549928+0000","last_became_active":"2024-09-17T05:25:48.566054+0000","last_became_peered":"2024-09-17T05:25:48.566054+0000","last_unstale":"2024-09-17T05:25:49.549928+0000","last_undegraded":"2024-09-17T05:25:49.549928+0000","last_fullsized":"2024-09-17T05:25:49.549928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:12.166901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577503+0000","last_change":"2024-09-17T05:25:48.565723+0000","last_active":"2024-09-17T05:25:49.577503+0000","last_peered":"2024-09-17T05:25:49.577503+0000","last_clean":"2024-09-17T05:25:49.577503+0000","last_became_active":"2024-09-17T05:25:48.565457+0000","last_became_peered":"2024-09-17T05:25:48.565457+0000","last_unstale":"2024-09-17T05:25:49.577503+0000","last_undegraded":"2024-09-17T05:25:49.577503+0000","last_fullsized":"2024-09-17T05:25:49.577503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:55.153291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578627+0000","last_change":"2024-09-17T05:25:48.555370+0000","last_active":"2024-09-17T05:25:49.578627+0000","last_peered":"2024-09-17T05:25:49.578627+0000","last_clean":"2024-09-17T05:25:49.578627+0000","last_became_active":"2024-09-17T05:25:48.554994+0000","last_became_peered":"2024-09-17T05:25:48.554994+0000","last_unstale":"2024-09-17T05:25:49.578627+0000","last_undegraded":"2024-09-17T05:25:49.578627+0000","last_fullsized":"2024-09-17T05:25:49.578627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:20.062760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569576+0000","last_change":"2024-09-17T05:25:48.566190+0000","last_active":"2024-09-17T05:25:49.569576+0000","last_peered":"2024-09-17T05:25:49.569576+0000","last_clean":"2024-09-17T05:25:49.569576+0000","last_became_active":"2024-09-17T05:25:48.565934+0000","last_became_peered":"2024-09-17T05:25:48.565934+0000","last_unstale":"2024-09-17T05:25:49.569576+0000","last_undegraded":"2024-09-17T05:25:49.569576+0000","last_fullsized":"2024-09-17T05:25:49.569576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:23.352539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550876+0000","last_change":"2024-09-17T05:25:48.576799+0000","last_active":"2024-09-17T05:25:49.550876+0000","last_peered":"2024-09-17T05:25:49.550876+0000","last_clean":"2024-09-17T05:25:49.550876+0000","last_became_active":"2024-09-17T05:25:48.576460+0000","last_became_peered":"2024-09-17T05:25:48.576460+0000","last_unstale":"2024-09-17T05:25:49.550876+0000","last_undegraded":"2024-09-17T05:25:49.550876+0000","last_fullsized":"2024-09-17T05:25:49.550876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:30.399223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574322+0000","last_change":"2024-09-17T05:25:48.555831+0000","last_active":"2024-09-17T05:25:49.574322+0000","last_peered":"2024-09-17T05:25:49.574322+0000","last_clean":"2024-09-17T05:25:49.574322+0000","last_became_active":"2024-09-17T05:25:48.555546+0000","last_became_peered":"2024-09-17T05:25:48.555546+0000","last_unstale":"2024-09-17T05:25:49.574322+0000","last_undegraded":"2024-09-17T05:25:49.574322+0000","last_fullsized":"2024-09-17T05:25:49.574322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:55.153291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574212+0000","last_change":"2024-09-17T05:25:48.554655+0000","last_active":"2024-09-17T05:25:49.574212+0000","last_peered":"2024-09-17T05:25:49.574212+0000","last_clean":"2024-09-17T05:25:49.574212+0000","last_became_active":"2024-09-17T05:25:48.553919+0000","last_became_peered":"2024-09-17T05:25:48.553919+0000","last_unstale":"2024-09-17T05:25:49.574212+0000","last_undegraded":"2024-09-17T05:25:49.574212+0000","last_fullsized":"2024-09-17T05:25:49.574212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:30.341407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550067+0000","last_change":"2024-09-17T05:25:48.560930+0000","last_active":"2024-09-17T05:25:49.550067+0000","last_peered":"2024-09-17T05:25:49.550067+0000","last_clean":"2024-09-17T05:25:49.550067+0000","last_became_active":"2024-09-17T05:25:48.560700+0000","last_became_peered":"2024-09-17T05:25:48.560700+0000","last_unstale":"2024-09-17T05:25:49.550067+0000","last_undegraded":"2024-09-17T05:25:49.550067+0000","last_fullsized":"2024-09-17T05:25:49.550067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:30.341407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.554634+0000","last_change":"2024-09-17T05:25:48.549526+0000","last_active":"2024-09-17T05:25:49.554634+0000","last_peered":"2024-09-17T05:25:49.554634+0000","last_clean":"2024-09-17T05:25:49.554634+0000","last_became_active":"2024-09-17T05:25:48.549278+0000","last_became_peered":"2024-09-17T05:25:48.549278+0000","last_unstale":"2024-09-17T05:25:49.554634+0000","last_undegraded":"2024-09-17T05:25:49.554634+0000","last_fullsized":"2024-09-17T05:25:49.554634+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:20.641179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577927+0000","last_change":"2024-09-17T05:25:48.579263+0000","last_active":"2024-09-17T05:25:49.577927+0000","last_peered":"2024-09-17T05:25:49.577927+0000","last_clean":"2024-09-17T05:25:49.577927+0000","last_became_active":"2024-09-17T05:25:48.578226+0000","last_became_peered":"2024-09-17T05:25:48.578226+0000","last_unstale":"2024-09-17T05:25:49.577927+0000","last_undegraded":"2024-09-17T05:25:49.577927+0000","last_fullsized":"2024-09-17T05:25:49.577927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:48.364658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.555278+0000","last_change":"2024-09-17T05:25:47.542992+0000","last_active":"2024-09-17T05:25:49.555278+0000","last_peered":"2024-09-17T05:25:49.555278+0000","last_clean":"2024-09-17T05:25:49.555278+0000","last_became_active":"2024-09-17T05:25:47.542678+0000","last_became_peered":"2024-09-17T05:25:47.542678+0000","last_unstale":"2024-09-17T05:25:49.555278+0000","last_undegraded":"2024-09-17T05:25:49.555278+0000","last_fullsized":"2024-09-17T05:25:49.555278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:18.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550940+0000","last_change":"2024-09-17T05:25:48.558850+0000","last_active":"2024-09-17T05:25:49.550940+0000","last_peered":"2024-09-17T05:25:49.550940+0000","last_clean":"2024-09-17T05:25:49.550940+0000","last_became_active":"2024-09-17T05:25:48.558570+0000","last_became_peered":"2024-09-17T05:25:48.558570+0000","last_unstale":"2024-09-17T05:25:49.550940+0000","last_undegraded":"2024-09-17T05:25:49.550940+0000","last_fullsized":"2024-09-17T05:25:49.550940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:28.696391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578750+0000","last_change":"2024-09-17T05:25:47.550543+0000","last_active":"2024-09-17T05:25:49.578750+0000","last_peered":"2024-09-17T05:25:49.578750+0000","last_clean":"2024-09-17T05:25:49.578750+0000","last_became_active":"2024-09-17T05:25:47.550230+0000","last_became_peered":"2024-09-17T05:25:47.550230+0000","last_unstale":"2024-09-17T05:25:49.578750+0000","last_undegraded":"2024-09-17T05:25:49.578750+0000","last_fullsized":"2024-09-17T05:25:49.578750+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550816+0000","last_change":"2024-09-17T05:25:48.561559+0000","last_active":"2024-09-17T05:25:49.550816+0000","last_peered":"2024-09-17T05:25:49.550816+0000","last_clean":"2024-09-17T05:25:49.550816+0000","last_became_active":"2024-09-17T05:25:48.561199+0000","last_became_peered":"2024-09-17T05:25:48.561199+0000","last_unstale":"2024-09-17T05:25:49.550816+0000","last_undegraded":"2024-09-17T05:25:49.550816+0000","last_fullsized":"2024-09-17T05:25:49.550816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:21.858398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.558446+0000","last_change":"2024-09-17T05:25:47.555529+0000","last_active":"2024-09-17T05:25:49.558446+0000","last_peered":"2024-09-17T05:25:49.558446+0000","last_clean":"2024-09-17T05:25:49.558446+0000","last_became_active":"2024-09-17T05:25:47.554844+0000","last_became_peered":"2024-09-17T05:25:47.554844+0000","last_unstale":"2024-09-17T05:25:49.558446+0000","last_undegraded":"2024-09-17T05:25:49.558446+0000","last_fullsized":"2024-09-17T05:25:49.558446+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:18.141436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577674+0000","last_change":"2024-09-17T05:25:48.577744+0000","last_active":"2024-09-17T05:25:49.577674+0000","last_peered":"2024-09-17T05:25:49.577674+0000","last_clean":"2024-09-17T05:25:49.577674+0000","last_became_active":"2024-09-17T05:25:48.577492+0000","last_became_peered":"2024-09-17T05:25:48.577492+0000","last_unstale":"2024-09-17T05:25:49.577674+0000","last_undegraded":"2024-09-17T05:25:49.577674+0000","last_fullsized":"2024-09-17T05:25:49.577674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:39.261176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578508+0000","last_change":"2024-09-17T05:25:47.557387+0000","last_active":"2024-09-17T05:25:49.578508+0000","last_peered":"2024-09-17T05:25:49.578508+0000","last_clean":"2024-09-17T05:25:49.578508+0000","last_became_active":"2024-09-17T05:25:47.557000+0000","last_became_peered":"2024-09-17T05:25:47.557000+0000","last_unstale":"2024-09-17T05:25:49.578508+0000","last_undegraded":"2024-09-17T05:25:49.578508+0000","last_fullsized":"2024-09-17T05:25:49.578508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:04.492367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574855+0000","last_change":"2024-09-17T05:25:48.563244+0000","last_active":"2024-09-17T05:25:49.574855+0000","last_peered":"2024-09-17T05:25:49.574855+0000","last_clean":"2024-09-17T05:25:49.574855+0000","last_became_active":"2024-09-17T05:25:48.562910+0000","last_became_peered":"2024-09-17T05:25:48.562910+0000","last_unstale":"2024-09-17T05:25:49.574855+0000","last_undegraded":"2024-09-17T05:25:49.574855+0000","last_fullsized":"2024-09-17T05:25:49.574855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:47.445644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550576+0000","last_change":"2024-09-17T05:25:47.542868+0000","last_active":"2024-09-17T05:25:49.550576+0000","last_peered":"2024-09-17T05:25:49.550576+0000","last_clean":"2024-09-17T05:25:49.550576+0000","last_became_active":"2024-09-17T05:25:47.542655+0000","last_became_peered":"2024-09-17T05:25:47.542655+0000","last_unstale":"2024-09-17T05:25:49.550576+0000","last_undegraded":"2024-09-17T05:25:49.550576+0000","last_fullsized":"2024-09-17T05:25:49.550576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:02.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550555+0000","last_change":"2024-09-17T05:25:48.561829+0000","last_active":"2024-09-17T05:25:49.550555+0000","last_peered":"2024-09-17T05:25:49.550555+0000","last_clean":"2024-09-17T05:25:49.550555+0000","last_became_active":"2024-09-17T05:25:48.561530+0000","last_became_peered":"2024-09-17T05:25:48.561530+0000","last_unstale":"2024-09-17T05:25:49.550555+0000","last_undegraded":"2024-09-17T05:25:49.550555+0000","last_fullsized":"2024-09-17T05:25:49.550555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:55.153291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550449+0000","last_change":"2024-09-17T05:25:47.552636+0000","last_active":"2024-09-17T05:25:49.550449+0000","last_peered":"2024-09-17T05:25:49.550449+0000","last_clean":"2024-09-17T05:25:49.550449+0000","last_became_active":"2024-09-17T05:25:47.552361+0000","last_became_peered":"2024-09-17T05:25:47.552361+0000","last_unstale":"2024-09-17T05:25:49.550449+0000","last_undegraded":"2024-09-17T05:25:49.550449+0000","last_fullsized":"2024-09-17T05:25:49.550449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:05.583300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574843+0000","last_change":"2024-09-17T05:25:48.553264+0000","last_active":"2024-09-17T05:25:49.574843+0000","last_peered":"2024-09-17T05:25:49.574843+0000","last_clean":"2024-09-17T05:25:49.574843+0000","last_became_active":"2024-09-17T05:25:48.552992+0000","last_became_peered":"2024-09-17T05:25:48.552992+0000","last_unstale":"2024-09-17T05:25:49.574843+0000","last_undegraded":"2024-09-17T05:25:49.574843+0000","last_fullsized":"2024-09-17T05:25:49.574843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:29.084922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.557741+0000","last_change":"2024-09-17T05:25:47.544385+0000","last_active":"2024-09-17T05:25:49.557741+0000","last_peered":"2024-09-17T05:25:49.557741+0000","last_clean":"2024-09-17T05:25:49.557741+0000","last_became_active":"2024-09-17T05:25:47.544020+0000","last_became_peered":"2024-09-17T05:25:47.544020+0000","last_unstale":"2024-09-17T05:25:49.557741+0000","last_undegraded":"2024-09-17T05:25:49.557741+0000","last_fullsized":"2024-09-17T05:25:49.557741+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:03.450940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577587+0000","last_change":"2024-09-17T05:25:48.555818+0000","last_active":"2024-09-17T05:25:49.577587+0000","last_peered":"2024-09-17T05:25:49.577587+0000","last_clean":"2024-09-17T05:25:49.577587+0000","last_became_active":"2024-09-17T05:25:48.555523+0000","last_became_peered":"2024-09-17T05:25:48.555523+0000","last_unstale":"2024-09-17T05:25:49.577587+0000","last_undegraded":"2024-09-17T05:25:49.577587+0000","last_fullsized":"2024-09-17T05:25:49.577587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:59.579161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550158+0000","last_change":"2024-09-17T05:25:47.544235+0000","last_active":"2024-09-17T05:25:49.550158+0000","last_peered":"2024-09-17T05:25:49.550158+0000","last_clean":"2024-09-17T05:25:49.550158+0000","last_became_active":"2024-09-17T05:25:47.543878+0000","last_became_peered":"2024-09-17T05:25:47.543878+0000","last_unstale":"2024-09-17T05:25:49.550158+0000","last_undegraded":"2024-09-17T05:25:49.550158+0000","last_fullsized":"2024-09-17T05:25:49.550158+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:29.899308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.555049+0000","last_change":"2024-09-17T05:25:48.578808+0000","last_active":"2024-09-17T05:25:49.555049+0000","last_peered":"2024-09-17T05:25:49.555049+0000","last_clean":"2024-09-17T05:25:49.555049+0000","last_became_active":"2024-09-17T05:25:48.578551+0000","last_became_peered":"2024-09-17T05:25:48.578551+0000","last_unstale":"2024-09-17T05:25:49.555049+0000","last_undegraded":"2024-09-17T05:25:49.555049+0000","last_fullsized":"2024-09-17T05:25:49.555049+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:31.821647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569936+0000","last_change":"2024-09-17T05:25:47.556128+0000","last_active":"2024-09-17T05:25:49.569936+0000","last_peered":"2024-09-17T05:25:49.569936+0000","last_clean":"2024-09-17T05:25:49.569936+0000","last_became_active":"2024-09-17T05:25:47.554436+0000","last_became_peered":"2024-09-17T05:25:47.554436+0000","last_unstale":"2024-09-17T05:25:49.569936+0000","last_undegraded":"2024-09-17T05:25:49.569936+0000","last_fullsized":"2024-09-17T05:25:49.569936+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:42.677350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578039+0000","last_change":"2024-09-17T05:25:48.569539+0000","last_active":"2024-09-17T05:25:49.578039+0000","last_peered":"2024-09-17T05:25:49.578039+0000","last_clean":"2024-09-17T05:25:49.578039+0000","last_became_active":"2024-09-17T05:25:48.569292+0000","last_became_peered":"2024-09-17T05:25:48.569292+0000","last_unstale":"2024-09-17T05:25:49.578039+0000","last_undegraded":"2024-09-17T05:25:49.578039+0000","last_fullsized":"2024-09-17T05:25:49.578039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:21.858398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569721+0000","last_change":"2024-09-17T05:25:47.549653+0000","last_active":"2024-09-17T05:25:49.569721+0000","last_peered":"2024-09-17T05:25:49.569721+0000","last_clean":"2024-09-17T05:25:49.569721+0000","last_became_active":"2024-09-17T05:25:47.548638+0000","last_became_peered":"2024-09-17T05:25:47.548638+0000","last_unstale":"2024-09-17T05:25:49.569721+0000","last_undegraded":"2024-09-17T05:25:49.569721+0000","last_fullsized":"2024-09-17T05:25:49.569721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:48:15.053151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578385+0000","last_change":"2024-09-17T05:25:48.568840+0000","last_active":"2024-09-17T05:25:49.578385+0000","last_peered":"2024-09-17T05:25:49.578385+0000","last_clean":"2024-09-17T05:25:49.578385+0000","last_became_active":"2024-09-17T05:25:48.568324+0000","last_became_peered":"2024-09-17T05:25:48.568324+0000","last_unstale":"2024-09-17T05:25:49.578385+0000","last_undegraded":"2024-09-17T05:25:49.578385+0000","last_fullsized":"2024-09-17T05:25:49.578385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:39.743920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.555153+0000","last_change":"2024-09-17T05:25:47.541605+0000","last_active":"2024-09-17T05:25:49.555153+0000","last_peered":"2024-09-17T05:25:49.555153+0000","last_clean":"2024-09-17T05:25:49.555153+0000","last_became_active":"2024-09-17T05:25:47.541312+0000","last_became_peered":"2024-09-17T05:25:47.541312+0000","last_unstale":"2024-09-17T05:25:49.555153+0000","last_undegraded":"2024-09-17T05:25:49.555153+0000","last_fullsized":"2024-09-17T05:25:49.555153+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:27.690980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.554931+0000","last_change":"2024-09-17T05:25:48.556785+0000","last_active":"2024-09-17T05:25:49.554931+0000","last_peered":"2024-09-17T05:25:49.554931+0000","last_clean":"2024-09-17T05:25:49.554931+0000","last_became_active":"2024-09-17T05:25:48.556443+0000","last_became_peered":"2024-09-17T05:25:48.556443+0000","last_unstale":"2024-09-17T05:25:49.554931+0000","last_undegraded":"2024-09-17T05:25:49.554931+0000","last_fullsized":"2024-09-17T05:25:49.554931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:35.613637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578386+0000","last_change":"2024-09-17T05:25:47.555429+0000","last_active":"2024-09-17T05:25:49.578386+0000","last_peered":"2024-09-17T05:25:49.578386+0000","last_clean":"2024-09-17T05:25:49.578386+0000","last_became_active":"2024-09-17T05:25:47.555144+0000","last_became_peered":"2024-09-17T05:25:47.555144+0000","last_unstale":"2024-09-17T05:25:49.578386+0000","last_undegraded":"2024-09-17T05:25:49.578386+0000","last_fullsized":"2024-09-17T05:25:49.578386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:24.436569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577406+0000","last_change":"2024-09-17T05:25:48.579894+0000","last_active":"2024-09-17T05:25:49.577406+0000","last_peered":"2024-09-17T05:25:49.577406+0000","last_clean":"2024-09-17T05:25:49.577406+0000","last_became_active":"2024-09-17T05:25:48.578255+0000","last_became_peered":"2024-09-17T05:25:48.578255+0000","last_unstale":"2024-09-17T05:25:49.577406+0000","last_undegraded":"2024-09-17T05:25:49.577406+0000","last_fullsized":"2024-09-17T05:25:49.577406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:48.376239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550678+0000","last_change":"2024-09-17T05:25:47.548118+0000","last_active":"2024-09-17T05:25:49.550678+0000","last_peered":"2024-09-17T05:25:49.550678+0000","last_clean":"2024-09-17T05:25:49.550678+0000","last_became_active":"2024-09-17T05:25:47.547871+0000","last_became_peered":"2024-09-17T05:25:47.547871+0000","last_unstale":"2024-09-17T05:25:49.550678+0000","last_undegraded":"2024-09-17T05:25:49.550678+0000","last_fullsized":"2024-09-17T05:25:49.550678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578859+0000","last_change":"2024-09-17T05:25:48.582169+0000","last_active":"2024-09-17T05:25:49.578859+0000","last_peered":"2024-09-17T05:25:49.578859+0000","last_clean":"2024-09-17T05:25:49.578859+0000","last_became_active":"2024-09-17T05:25:48.581834+0000","last_became_peered":"2024-09-17T05:25:48.581834+0000","last_unstale":"2024-09-17T05:25:49.578859+0000","last_undegraded":"2024-09-17T05:25:49.578859+0000","last_fullsized":"2024-09-17T05:25:49.578859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:24.768597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578071+0000","last_change":"2024-09-17T05:25:47.547173+0000","last_active":"2024-09-17T05:25:49.578071+0000","last_peered":"2024-09-17T05:25:49.578071+0000","last_clean":"2024-09-17T05:25:49.578071+0000","last_became_active":"2024-09-17T05:25:47.546912+0000","last_became_peered":"2024-09-17T05:25:47.546912+0000","last_unstale":"2024-09-17T05:25:49.578071+0000","last_undegraded":"2024-09-17T05:25:49.578071+0000","last_fullsized":"2024-09-17T05:25:49.578071+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:26.869603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550079+0000","last_change":"2024-09-17T05:25:48.553296+0000","last_active":"2024-09-17T05:25:49.550079+0000","last_peered":"2024-09-17T05:25:49.550079+0000","last_clean":"2024-09-17T05:25:49.550079+0000","last_became_active":"2024-09-17T05:25:48.553058+0000","last_became_peered":"2024-09-17T05:25:48.553058+0000","last_unstale":"2024-09-17T05:25:49.550079+0000","last_undegraded":"2024-09-17T05:25:49.550079+0000","last_fullsized":"2024-09-17T05:25:49.550079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:26.478618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577766+0000","last_change":"2024-09-17T05:25:47.553779+0000","last_active":"2024-09-17T05:25:49.577766+0000","last_peered":"2024-09-17T05:25:49.577766+0000","last_clean":"2024-09-17T05:25:49.577766+0000","last_became_active":"2024-09-17T05:25:47.553533+0000","last_became_peered":"2024-09-17T05:25:47.553533+0000","last_unstale":"2024-09-17T05:25:49.577766+0000","last_undegraded":"2024-09-17T05:25:49.577766+0000","last_fullsized":"2024-09-17T05:25:49.577766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.686043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.554711+0000","last_change":"2024-09-17T05:25:48.549363+0000","last_active":"2024-09-17T05:25:49.554711+0000","last_peered":"2024-09-17T05:25:49.554711+0000","last_clean":"2024-09-17T05:25:49.554711+0000","last_became_active":"2024-09-17T05:25:48.549077+0000","last_became_peered":"2024-09-17T05:25:48.549077+0000","last_unstale":"2024-09-17T05:25:49.554711+0000","last_undegraded":"2024-09-17T05:25:49.554711+0000","last_fullsized":"2024-09-17T05:25:49.554711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:22.334918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.557628+0000","last_change":"2024-09-17T05:25:47.545375+0000","last_active":"2024-09-17T05:25:49.557628+0000","last_peered":"2024-09-17T05:25:49.557628+0000","last_clean":"2024-09-17T05:25:49.557628+0000","last_became_active":"2024-09-17T05:25:47.545098+0000","last_became_peered":"2024-09-17T05:25:47.545098+0000","last_unstale":"2024-09-17T05:25:49.557628+0000","last_undegraded":"2024-09-17T05:25:49.557628+0000","last_fullsized":"2024-09-17T05:25:49.557628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:10.436228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.555085+0000","last_change":"2024-09-17T05:25:48.557134+0000","last_active":"2024-09-17T05:25:49.555085+0000","last_peered":"2024-09-17T05:25:49.555085+0000","last_clean":"2024-09-17T05:25:49.555085+0000","last_became_active":"2024-09-17T05:25:48.556815+0000","last_became_peered":"2024-09-17T05:25:48.556815+0000","last_unstale":"2024-09-17T05:25:49.555085+0000","last_undegraded":"2024-09-17T05:25:49.555085+0000","last_fullsized":"2024-09-17T05:25:49.555085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:13.067249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.558270+0000","last_change":"2024-09-17T05:25:47.551471+0000","last_active":"2024-09-17T05:25:49.558270+0000","last_peered":"2024-09-17T05:25:49.558270+0000","last_clean":"2024-09-17T05:25:49.558270+0000","last_became_active":"2024-09-17T05:25:47.550374+0000","last_became_peered":"2024-09-17T05:25:47.550374+0000","last_unstale":"2024-09-17T05:25:49.558270+0000","last_undegraded":"2024-09-17T05:25:49.558270+0000","last_fullsized":"2024-09-17T05:25:49.558270+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:34.028724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.575531+0000","last_change":"2024-09-17T05:25:48.559461+0000","last_active":"2024-09-17T05:25:49.575531+0000","last_peered":"2024-09-17T05:25:49.575531+0000","last_clean":"2024-09-17T05:25:49.575531+0000","last_became_active":"2024-09-17T05:25:48.558513+0000","last_became_peered":"2024-09-17T05:25:48.558513+0000","last_unstale":"2024-09-17T05:25:49.575531+0000","last_undegraded":"2024-09-17T05:25:49.575531+0000","last_fullsized":"2024-09-17T05:25:49.575531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:20.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550804+0000","last_change":"2024-09-17T05:25:47.547687+0000","last_active":"2024-09-17T05:25:49.550804+0000","last_peered":"2024-09-17T05:25:49.550804+0000","last_clean":"2024-09-17T05:25:49.550804+0000","last_became_active":"2024-09-17T05:25:47.547426+0000","last_became_peered":"2024-09-17T05:25:47.547426+0000","last_unstale":"2024-09-17T05:25:49.550804+0000","last_undegraded":"2024-09-17T05:25:49.550804+0000","last_fullsized":"2024-09-17T05:25:49.550804+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:06.630653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569586+0000","last_change":"2024-09-17T05:25:48.559037+0000","last_active":"2024-09-17T05:25:49.569586+0000","last_peered":"2024-09-17T05:25:49.569586+0000","last_clean":"2024-09-17T05:25:49.569586+0000","last_became_active":"2024-09-17T05:25:48.558809+0000","last_became_peered":"2024-09-17T05:25:48.558809+0000","last_unstale":"2024-09-17T05:25:49.569586+0000","last_undegraded":"2024-09-17T05:25:49.569586+0000","last_fullsized":"2024-09-17T05:25:49.569586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:03.480077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.555049+0000","last_change":"2024-09-17T05:25:47.546486+0000","last_active":"2024-09-17T05:25:49.555049+0000","last_peered":"2024-09-17T05:25:49.555049+0000","last_clean":"2024-09-17T05:25:49.555049+0000","last_became_active":"2024-09-17T05:25:47.545952+0000","last_became_peered":"2024-09-17T05:25:47.545952+0000","last_unstale":"2024-09-17T05:25:49.555049+0000","last_undegraded":"2024-09-17T05:25:49.555049+0000","last_fullsized":"2024-09-17T05:25:49.555049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:17.918336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.558193+0000","last_change":"2024-09-17T05:25:48.563376+0000","last_active":"2024-09-17T05:25:49.558193+0000","last_peered":"2024-09-17T05:25:49.558193+0000","last_clean":"2024-09-17T05:25:49.558193+0000","last_became_active":"2024-09-17T05:25:48.563128+0000","last_became_peered":"2024-09-17T05:25:48.563128+0000","last_unstale":"2024-09-17T05:25:49.558193+0000","last_undegraded":"2024-09-17T05:25:49.558193+0000","last_fullsized":"2024-09-17T05:25:49.558193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:20.762762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574443+0000","last_change":"2024-09-17T05:25:47.550977+0000","last_active":"2024-09-17T05:25:49.574443+0000","last_peered":"2024-09-17T05:25:49.574443+0000","last_clean":"2024-09-17T05:25:49.574443+0000","last_became_active":"2024-09-17T05:25:47.550677+0000","last_became_peered":"2024-09-17T05:25:47.550677+0000","last_unstale":"2024-09-17T05:25:49.574443+0000","last_undegraded":"2024-09-17T05:25:49.574443+0000","last_fullsized":"2024-09-17T05:25:49.574443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:12.600379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577501+0000","last_change":"2024-09-17T05:25:48.578870+0000","last_active":"2024-09-17T05:25:49.577501+0000","last_peered":"2024-09-17T05:25:49.577501+0000","last_clean":"2024-09-17T05:25:49.577501+0000","last_became_active":"2024-09-17T05:25:48.578503+0000","last_became_peered":"2024-09-17T05:25:48.578503+0000","last_unstale":"2024-09-17T05:25:49.577501+0000","last_undegraded":"2024-09-17T05:25:49.577501+0000","last_fullsized":"2024-09-17T05:25:49.577501+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:29.372259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569796+0000","last_change":"2024-09-17T05:25:47.553626+0000","last_active":"2024-09-17T05:25:49.569796+0000","last_peered":"2024-09-17T05:25:49.569796+0000","last_clean":"2024-09-17T05:25:49.569796+0000","last_became_active":"2024-09-17T05:25:47.553385+0000","last_became_peered":"2024-09-17T05:25:47.553385+0000","last_unstale":"2024-09-17T05:25:49.569796+0000","last_undegraded":"2024-09-17T05:25:49.569796+0000","last_fullsized":"2024-09-17T05:25:49.569796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:13.686877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569497+0000","last_change":"2024-09-17T05:25:48.566784+0000","last_active":"2024-09-17T05:25:49.569497+0000","last_peered":"2024-09-17T05:25:49.569497+0000","last_clean":"2024-09-17T05:25:49.569497+0000","last_became_active":"2024-09-17T05:25:48.566510+0000","last_became_peered":"2024-09-17T05:25:48.566510+0000","last_unstale":"2024-09-17T05:25:49.569497+0000","last_undegraded":"2024-09-17T05:25:49.569497+0000","last_fullsized":"2024-09-17T05:25:49.569497+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:16.873656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550492+0000","last_change":"2024-09-17T05:25:47.549913+0000","last_active":"2024-09-17T05:25:49.550492+0000","last_peered":"2024-09-17T05:25:49.550492+0000","last_clean":"2024-09-17T05:25:49.550492+0000","last_became_active":"2024-09-17T05:25:47.549593+0000","last_became_peered":"2024-09-17T05:25:47.549593+0000","last_unstale":"2024-09-17T05:25:49.550492+0000","last_undegraded":"2024-09-17T05:25:49.550492+0000","last_fullsized":"2024-09-17T05:25:49.550492+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:08.157058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577631+0000","last_change":"2024-09-17T05:25:48.583799+0000","last_active":"2024-09-17T05:25:49.577631+0000","last_peered":"2024-09-17T05:25:49.577631+0000","last_clean":"2024-09-17T05:25:49.577631+0000","last_became_active":"2024-09-17T05:25:48.583489+0000","last_became_peered":"2024-09-17T05:25:48.583489+0000","last_unstale":"2024-09-17T05:25:49.577631+0000","last_undegraded":"2024-09-17T05:25:49.577631+0000","last_fullsized":"2024-09-17T05:25:49.577631+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:31.937396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.575073+0000","last_change":"2024-09-17T05:25:47.542635+0000","last_active":"2024-09-17T05:25:49.575073+0000","last_peered":"2024-09-17T05:25:49.575073+0000","last_clean":"2024-09-17T05:25:49.575073+0000","last_became_active":"2024-09-17T05:25:47.542160+0000","last_became_peered":"2024-09-17T05:25:47.542160+0000","last_unstale":"2024-09-17T05:25:49.575073+0000","last_undegraded":"2024-09-17T05:25:49.575073+0000","last_fullsized":"2024-09-17T05:25:49.575073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:33.388849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574969+0000","last_change":"2024-09-17T05:25:48.552387+0000","last_active":"2024-09-17T05:25:49.574969+0000","last_peered":"2024-09-17T05:25:49.574969+0000","last_clean":"2024-09-17T05:25:49.574969+0000","last_became_active":"2024-09-17T05:25:48.552075+0000","last_became_peered":"2024-09-17T05:25:48.552075+0000","last_unstale":"2024-09-17T05:25:49.574969+0000","last_undegraded":"2024-09-17T05:25:49.574969+0000","last_fullsized":"2024-09-17T05:25:49.574969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:25.441981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.554652+0000","last_change":"2024-09-17T05:25:47.545902+0000","last_active":"2024-09-17T05:25:49.554652+0000","last_peered":"2024-09-17T05:25:49.554652+0000","last_clean":"2024-09-17T05:25:49.554652+0000","last_became_active":"2024-09-17T05:25:47.545602+0000","last_became_peered":"2024-09-17T05:25:47.545602+0000","last_unstale":"2024-09-17T05:25:49.554652+0000","last_undegraded":"2024-09-17T05:25:49.554652+0000","last_fullsized":"2024-09-17T05:25:49.554652+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:34.536424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.558278+0000","last_change":"2024-09-17T05:25:48.566441+0000","last_active":"2024-09-17T05:25:49.558278+0000","last_peered":"2024-09-17T05:25:49.558278+0000","last_clean":"2024-09-17T05:25:49.558278+0000","last_became_active":"2024-09-17T05:25:48.566282+0000","last_became_peered":"2024-09-17T05:25:48.566282+0000","last_unstale":"2024-09-17T05:25:49.558278+0000","last_undegraded":"2024-09-17T05:25:49.558278+0000","last_fullsized":"2024-09-17T05:25:49.558278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:54.356062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578411+0000","last_change":"2024-09-17T05:25:47.549606+0000","last_active":"2024-09-17T05:25:49.578411+0000","last_peered":"2024-09-17T05:25:49.578411+0000","last_clean":"2024-09-17T05:25:49.578411+0000","last_became_active":"2024-09-17T05:25:47.549353+0000","last_became_peered":"2024-09-17T05:25:47.549353+0000","last_unstale":"2024-09-17T05:25:49.578411+0000","last_undegraded":"2024-09-17T05:25:49.578411+0000","last_fullsized":"2024-09-17T05:25:49.578411+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:38:46.440233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.555310+0000","last_change":"2024-09-17T05:25:48.560961+0000","last_active":"2024-09-17T05:25:49.555310+0000","last_peered":"2024-09-17T05:25:49.555310+0000","last_clean":"2024-09-17T05:25:49.555310+0000","last_became_active":"2024-09-17T05:25:48.560715+0000","last_became_peered":"2024-09-17T05:25:48.560715+0000","last_unstale":"2024-09-17T05:25:49.555310+0000","last_undegraded":"2024-09-17T05:25:49.555310+0000","last_fullsized":"2024-09-17T05:25:49.555310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:55.053909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.569875+0000","last_change":"2024-09-17T05:25:47.552693+0000","last_active":"2024-09-17T05:25:49.569875+0000","last_peered":"2024-09-17T05:25:49.569875+0000","last_clean":"2024-09-17T05:25:49.569875+0000","last_became_active":"2024-09-17T05:25:47.548298+0000","last_became_peered":"2024-09-17T05:25:47.548298+0000","last_unstale":"2024-09-17T05:25:49.569875+0000","last_undegraded":"2024-09-17T05:25:49.569875+0000","last_fullsized":"2024-09-17T05:25:49.569875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:37.335913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.558284+0000","last_change":"2024-09-17T05:25:48.559435+0000","last_active":"2024-09-17T05:25:49.558284+0000","last_peered":"2024-09-17T05:25:49.558284+0000","last_clean":"2024-09-17T05:25:49.558284+0000","last_became_active":"2024-09-17T05:25:48.558731+0000","last_became_peered":"2024-09-17T05:25:48.558731+0000","last_unstale":"2024-09-17T05:25:49.558284+0000","last_undegraded":"2024-09-17T05:25:49.558284+0000","last_fullsized":"2024-09-17T05:25:49.558284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:00.303537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578496+0000","last_change":"2024-09-17T05:25:47.547920+0000","last_active":"2024-09-17T05:25:49.578496+0000","last_peered":"2024-09-17T05:25:49.578496+0000","last_clean":"2024-09-17T05:25:49.578496+0000","last_became_active":"2024-09-17T05:25:47.547593+0000","last_became_peered":"2024-09-17T05:25:47.547593+0000","last_unstale":"2024-09-17T05:25:49.578496+0000","last_undegraded":"2024-09-17T05:25:49.578496+0000","last_fullsized":"2024-09-17T05:25:49.578496+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:12.832450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574232+0000","last_change":"2024-09-17T05:25:48.556873+0000","last_active":"2024-09-17T05:25:49.574232+0000","last_peered":"2024-09-17T05:25:49.574232+0000","last_clean":"2024-09-17T05:25:49.574232+0000","last_became_active":"2024-09-17T05:25:48.556481+0000","last_became_peered":"2024-09-17T05:25:48.556481+0000","last_unstale":"2024-09-17T05:25:49.574232+0000","last_undegraded":"2024-09-17T05:25:49.574232+0000","last_fullsized":"2024-09-17T05:25:49.574232+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:59.579161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577945+0000","last_change":"2024-09-17T05:25:47.552565+0000","last_active":"2024-09-17T05:25:49.577945+0000","last_peered":"2024-09-17T05:25:49.577945+0000","last_clean":"2024-09-17T05:25:49.577945+0000","last_became_active":"2024-09-17T05:25:47.549868+0000","last_became_peered":"2024-09-17T05:25:47.549868+0000","last_unstale":"2024-09-17T05:25:49.577945+0000","last_undegraded":"2024-09-17T05:25:49.577945+0000","last_fullsized":"2024-09-17T05:25:49.577945+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:45.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578862+0000","last_change":"2024-09-17T05:25:48.579795+0000","last_active":"2024-09-17T05:25:49.578862+0000","last_peered":"2024-09-17T05:25:49.578862+0000","last_clean":"2024-09-17T05:25:49.578862+0000","last_became_active":"2024-09-17T05:25:48.579382+0000","last_became_peered":"2024-09-17T05:25:48.579382+0000","last_unstale":"2024-09-17T05:25:49.578862+0000","last_undegraded":"2024-09-17T05:25:49.578862+0000","last_fullsized":"2024-09-17T05:25:49.578862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:01.838706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577595+0000","last_change":"2024-09-17T05:25:47.548934+0000","last_active":"2024-09-17T05:25:49.577595+0000","last_peered":"2024-09-17T05:25:49.577595+0000","last_clean":"2024-09-17T05:25:49.577595+0000","last_became_active":"2024-09-17T05:25:47.548388+0000","last_became_peered":"2024-09-17T05:25:47.548388+0000","last_unstale":"2024-09-17T05:25:49.577595+0000","last_undegraded":"2024-09-17T05:25:49.577595+0000","last_fullsized":"2024-09-17T05:25:49.577595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:32.932862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.578126+0000","last_change":"2024-09-17T05:25:48.565031+0000","last_active":"2024-09-17T05:25:49.578126+0000","last_peered":"2024-09-17T05:25:49.578126+0000","last_clean":"2024-09-17T05:25:49.578126+0000","last_became_active":"2024-09-17T05:25:48.564697+0000","last_became_peered":"2024-09-17T05:25:48.564697+0000","last_unstale":"2024-09-17T05:25:49.578126+0000","last_undegraded":"2024-09-17T05:25:49.578126+0000","last_fullsized":"2024-09-17T05:25:49.578126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:30.399223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.550230+0000","last_change":"2024-09-17T05:25:47.548297+0000","last_active":"2024-09-17T05:25:49.550230+0000","last_peered":"2024-09-17T05:25:49.550230+0000","last_clean":"2024-09-17T05:25:49.550230+0000","last_became_active":"2024-09-17T05:25:47.548063+0000","last_became_peered":"2024-09-17T05:25:47.548063+0000","last_unstale":"2024-09-17T05:25:49.550230+0000","last_undegraded":"2024-09-17T05:25:49.550230+0000","last_fullsized":"2024-09-17T05:25:49.550230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:34.378981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.554762+0000","last_change":"2024-09-17T05:25:48.562700+0000","last_active":"2024-09-17T05:25:49.554762+0000","last_peered":"2024-09-17T05:25:49.554762+0000","last_clean":"2024-09-17T05:25:49.554762+0000","last_became_active":"2024-09-17T05:25:48.562183+0000","last_became_peered":"2024-09-17T05:25:48.562183+0000","last_unstale":"2024-09-17T05:25:49.554762+0000","last_undegraded":"2024-09-17T05:25:49.554762+0000","last_fullsized":"2024-09-17T05:25:49.554762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:39.176453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.577842+0000","last_change":"2024-09-17T05:25:47.551652+0000","last_active":"2024-09-17T05:25:49.577842+0000","last_peered":"2024-09-17T05:25:49.577842+0000","last_clean":"2024-09-17T05:25:49.577842+0000","last_became_active":"2024-09-17T05:25:47.551332+0000","last_became_peered":"2024-09-17T05:25:47.551332+0000","last_unstale":"2024-09-17T05:25:49.577842+0000","last_undegraded":"2024-09-17T05:25:49.577842+0000","last_fullsized":"2024-09-17T05:25:49.577842+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:38.212230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.574496+0000","last_change":"2024-09-17T05:25:48.559526+0000","last_active":"2024-09-17T05:25:49.574496+0000","last_peered":"2024-09-17T05:25:49.574496+0000","last_clean":"2024-09-17T05:25:49.574496+0000","last_became_active":"2024-09-17T05:25:48.559292+0000","last_became_peered":"2024-09-17T05:25:48.559292+0000","last_unstale":"2024-09-17T05:25:49.574496+0000","last_undegraded":"2024-09-17T05:25:49.574496+0000","last_fullsized":"2024-09-17T05:25:49.574496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:47.520978+0000","last_clean_scrub_stamp":"2024-09-17T05:25:47.520978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:21.858398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.555121+0000","last_change":"2024-09-17T05:25:47.546172+0000","last_active":"2024-09-17T05:25:49.555121+0000","last_peered":"2024-09-17T05:25:49.555121+0000","last_clean":"2024-09-17T05:25:49.555121+0000","last_became_active":"2024-09-17T05:25:47.545644+0000","last_became_peered":"2024-09-17T05:25:47.545644+0000","last_unstale":"2024-09-17T05:25:49.555121+0000","last_undegraded":"2024-09-17T05:25:49.555121+0000","last_fullsized":"2024-09-17T05:25:49.555121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:25:46.515567+0000","last_clean_scrub_stamp":"2024-09-17T05:25:46.515567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:46.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":132,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T05:25:49.575048+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:25:49.575048+0000","last_peered":"2024-09-17T05:25:49.575048+0000","last_clean":"2024-09-17T05:25:49.575048+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:25:49.575048+0000","last_undegraded":"2024-09-17T05:25:49.575048+0000","last_fullsized":"2024-09-17T05:25:49.575048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":141096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":141118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":140260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":139518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":962560,"data_stored":729922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":139574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":730434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:25:57.606 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:25:57.606 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:25:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- 172.21.15.122:0/4250481750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0b49e0 msgr2=0x7f6dcc06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 --2- 172.21.15.122:0/4250481750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0b49e0 0x7f6dcc06f7b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009c80 tx=0x7f6dc401cd10 comp rx=0 tx=0).stop 2024-09-17T05:25:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- 172.21.15.122:0/4250481750 shutdown_connections 2024-09-17T05:25:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 --2- 172.21.15.122:0/4250481750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0b49e0 0x7f6dcc06f7b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 --2- 172.21.15.122:0/4250481750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6dcc15dae0 0x7f6dcc0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- 172.21.15.122:0/4250481750 >> 172.21.15.122:0/4250481750 conn(0x7f6dcc0b18f0 msgr2=0x7f6dcc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- 172.21.15.122:0/4250481750 shutdown_connections 2024-09-17T05:25:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- 172.21.15.122:0/4250481750 wait complete. 2024-09-17T05:25:57.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 Processor -- start 2024-09-17T05:25:57.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- start start 2024-09-17T05:25:57.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6dcc0b49e0 0x7f6dcc153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0cd100 0x7f6dcc154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dcc15dae0 0x7f6dcc154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc0cf1a0 con 0x7f6dcc0cd100 2024-09-17T05:25:57.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc0cf020 con 0x7f6dcc15dae0 2024-09-17T05:25:57.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd4000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc0cf320 con 0x7f6dcc0b49e0 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0cd100 0x7f6dcc154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0cd100 0x7f6dcc154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53848/0 (socket says 172.21.15.122:53848) 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd0c00640 1 -- 172.21.15.122:0/269712799 learned_addr learned my addr 172.21.15.122:0/269712799 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd2000640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dcc15dae0 0x7f6dcc154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.793+0000 7f6dd1600640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6dcc0b49e0 0x7f6dcc153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd0c00640 1 -- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6dcc0b49e0 msgr2=0x7f6dcc153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd0c00640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6dcc0b49e0 0x7f6dcc153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd0c00640 1 -- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dcc15dae0 msgr2=0x7f6dcc154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd0c00640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dcc15dae0 0x7f6dcc154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd0c00640 1 -- 172.21.15.122:0/269712799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc153a50 con 0x7f6dcc0cd100 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd0c00640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0cd100 0x7f6dcc154650 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c400 tx=0x7f6db800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:57.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dc9600640 1 -- 172.21.15.122:0/269712799 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8017070 con 0x7f6dcc0cd100 2024-09-17T05:25:57.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dc9600640 1 -- 172.21.15.122:0/269712799 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6db80053e0 con 0x7f6dcc0cd100 2024-09-17T05:25:57.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dc9600640 1 -- 172.21.15.122:0/269712799 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8003860 con 0x7f6dcc0cd100 2024-09-17T05:25:57.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc153d30 con 0x7f6dcc0cd100 2024-09-17T05:25:57.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc155820 con 0x7f6dcc0cd100 2024-09-17T05:25:57.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dcc069410 con 0x7f6dcc0cd100 2024-09-17T05:25:57.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dc9600640 1 -- 172.21.15.122:0/269712799 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6db801a070 con 0x7f6dcc0cd100 2024-09-17T05:25:57.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dc9600640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6dac073230 0x7f6dac0756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:57.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.797+0000 7f6dc9600640 1 -- 172.21.15.122:0/269712799 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6db8093560 con 0x7f6dcc0cd100 2024-09-17T05:25:57.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.801+0000 7f6dd1600640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6dac073230 0x7f6dac0756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:57.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.801+0000 7f6dd1600640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6dac073230 0x7f6dac0756f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000bc50 tx=0x7f6dc000b3e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:57.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:57.801+0000 7f6dc9600640 1 -- 172.21.15.122:0/269712799 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6db8001130 con 0x7f6dcc0cd100 2024-09-17T05:25:58.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.029+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6dcc0acd40 con 0x7f6dcc0cd100 2024-09-17T05:25:58.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.029+0000 7f6dc9600640 1 -- 172.21.15.122:0/269712799 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f6db805df20 con 0x7f6dcc0cd100 2024-09-17T05:25:58.035 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:25:58.035 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:25:58.035 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:25:58.035 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:25:58.035 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:25:58.036 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:25:58.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6dac073230 msgr2=0x7f6dac0756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6dac073230 0x7f6dac0756f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000bc50 tx=0x7f6dc000b3e0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0cd100 msgr2=0x7f6dcc154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0cd100 0x7f6dcc154650 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c400 tx=0x7f6db800c8c0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 shutdown_connections 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6dac073230 0x7f6dac0756f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6dcc15dae0 0x7f6dcc154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dcc0cd100 0x7f6dcc154650 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 --2- 172.21.15.122:0/269712799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6dcc0b49e0 0x7f6dcc153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 >> 172.21.15.122:0/269712799 conn(0x7f6dcc0b18f0 msgr2=0x7f6dcc0aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 shutdown_connections 2024-09-17T05:25:58.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.033+0000 7f6dd4000640 1 -- 172.21.15.122:0/269712799 wait complete. 2024-09-17T05:25:58.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:25:58.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.193+0000 7fe700a00640 1 -- 172.21.15.122:0/779038061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc15bea0 msgr2=0x7fe6fc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.193+0000 7fe700a00640 1 --2- 172.21.15.122:0/779038061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc15bea0 0x7fe6fc0ce670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc073d10 tx=0x7fe6ec01caf0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.193+0000 7fe700a00640 1 -- 172.21.15.122:0/779038061 shutdown_connections 2024-09-17T05:25:58.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.193+0000 7fe700a00640 1 --2- 172.21.15.122:0/779038061 >> v2:172.21.15.173:3300/0 conn(0x7fe6fc0cdb30 0x7fe6fc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.193+0000 7fe700a00640 1 --2- 172.21.15.122:0/779038061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc15bea0 0x7fe6fc0ce670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.193+0000 7fe700a00640 1 -- 172.21.15.122:0/779038061 >> 172.21.15.122:0/779038061 conn(0x7fe6fc0d1dc0 msgr2=0x7fe6fc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:58.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.193+0000 7fe700a00640 1 -- 172.21.15.122:0/779038061 shutdown_connections 2024-09-17T05:25:58.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.193+0000 7fe700a00640 1 -- 172.21.15.122:0/779038061 wait complete. 2024-09-17T05:25:58.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 Processor -- start 2024-09-17T05:25:58.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 -- start start 2024-09-17T05:25:58.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc0cdb30 0x7fe6fc157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6fc0d28e0 0x7fe6fc0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6fc15bea0 0x7fe6fc0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc168f10 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc168d90 con 0x7fe6fc15bea0 2024-09-17T05:25:58.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc169090 con 0x7fe6fc0d28e0 2024-09-17T05:25:58.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc0cdb30 0x7fe6fc157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc0cdb30 0x7fe6fc157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53860/0 (socket says 172.21.15.122:53860) 2024-09-17T05:25:58.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 -- 172.21.15.122:0/3854436150 learned_addr learned my addr 172.21.15.122:0/3854436150 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:58.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6fa800640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6fc15bea0 0x7fe6fc0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9400640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6fc0d28e0 0x7fe6fc0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 -- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6fc0d28e0 msgr2=0x7fe6fc0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6fc0d28e0 0x7fe6fc0b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 -- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6fc15bea0 msgr2=0x7fe6fc0b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6fc15bea0 0x7fe6fc0b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 -- 172.21.15.122:0/3854436150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc06f750 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f9e00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc0cdb30 0x7fe6fc157560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec01c560 tx=0x7fe6ec005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f2000640 1 -- 172.21.15.122:0/3854436150 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0b2040 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f2000640 1 -- 172.21.15.122:0/3854436150 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6ec0bb070 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc06e990 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe6f2000640 1 -- 172.21.15.122:0/3854436150 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec004530 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.197+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc06ef50 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.201+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc0cc9d0 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.201+0000 7fe6f2000640 1 -- 172.21.15.122:0/3854436150 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6ec0046d0 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.201+0000 7fe6f2000640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6dc073200 0x7fe6dc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.201+0000 7fe6f2000640 1 -- 172.21.15.122:0/3854436150 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe6ec12eb80 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.201+0000 7fe6f9400640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6dc073200 0x7fe6dc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.201+0000 7fe6f9400640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6dc073200 0x7fe6dc0756c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0b1ad0 tx=0x7fe6e80091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.205+0000 7fe6f2000640 1 -- 172.21.15.122:0/3854436150 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe6fc06f750 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.409+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6fc06add0 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe6f2000640 1 -- 172.21.15.122:0/3854436150 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5235 (secure 0 0 0) 0x7fe6ec016770 con 0x7fe6fc0cdb30 2024-09-17T05:25:58.416 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:25:58.416 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":16,"btime":"2024-09-17T05:25:51:564038+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2936956280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2936956280},{"type":"v1","addr":"172.21.15.173:6836","nonce":2936956280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2308148662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2308148662},{"type":"v1","addr":"172.21.15.122:6834","nonce":2308148662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3699957348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":3699957348},{"type":"v1","addr":"172.21.15.122:6835","nonce":3699957348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:47.923640+0000","modified":"2024-09-17T05:25:50.952955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2399039177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2399039177},{"type":"v1","addr":"172.21.15.173:6837","nonce":2399039177}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4873,"qdb_cluster":[4873]},"id":2}]} 2024-09-17T05:25:58.416 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 16 2024-09-17T05:25:58.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6dc073200 msgr2=0x7fe6dc0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6dc073200 0x7fe6dc0756c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0b1ad0 tx=0x7fe6e80091e0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc0cdb30 msgr2=0x7fe6fc157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc0cdb30 0x7fe6fc157560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec01c560 tx=0x7fe6ec005ae0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 shutdown_connections 2024-09-17T05:25:58.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe6dc073200 0x7fe6dc0756c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6fc15bea0 0x7fe6fc0b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6fc0d28e0 0x7fe6fc0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 --2- 172.21.15.122:0/3854436150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6fc0cdb30 0x7fe6fc157560 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 >> 172.21.15.122:0/3854436150 conn(0x7fe6fc0d1dc0 msgr2=0x7fe6fc0b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:58.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 shutdown_connections 2024-09-17T05:25:58.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.413+0000 7fe700a00640 1 -- 172.21.15.122:0/3854436150 wait complete. 2024-09-17T05:25:58.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:25:47.923640+0000', 'modified': '2024-09-17T05:25:50.952955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4873}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4873': {'gid': 4873, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6837/2399039177', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6835', 'nonce': 2399039177}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 2399039177}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4873, 'qdb_cluster': [4873]} 2024-09-17T05:25:58.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:25:58.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:25:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- 172.21.15.122:0/3873713557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff27815bea0 msgr2=0x7ff2780ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 --2- 172.21.15.122:0/3873713557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff27815bea0 0x7ff2780ce670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff278073d10 tx=0x7ff26801c830 comp rx=0 tx=0).stop 2024-09-17T05:25:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- 172.21.15.122:0/3873713557 shutdown_connections 2024-09-17T05:25:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 --2- 172.21.15.122:0/3873713557 >> v2:172.21.15.173:3300/0 conn(0x7ff2780cdb30 0x7ff2780cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 --2- 172.21.15.122:0/3873713557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff27815bea0 0x7ff2780ce670 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- 172.21.15.122:0/3873713557 >> 172.21.15.122:0/3873713557 conn(0x7ff2780d1dc0 msgr2=0x7ff2780d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:58.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- 172.21.15.122:0/3873713557 shutdown_connections 2024-09-17T05:25:58.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- 172.21.15.122:0/3873713557 wait complete. 2024-09-17T05:25:58.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 Processor -- start 2024-09-17T05:25:58.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- start start 2024-09-17T05:25:58.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2780cdb30 0x7ff2780b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2780d28e0 0x7ff2780b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff27815bea0 0x7ff2780b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff278168fc0 con 0x7ff2780cdb30 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff278168e40 con 0x7ff27815bea0 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.985+0000 7ff27de00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff278169140 con 0x7ff2780d28e0 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2780cdb30 0x7ff2780b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2780cdb30 0x7ff2780b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53878/0 (socket says 172.21.15.122:53878) 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 -- 172.21.15.122:0/2756103356 learned_addr learned my addr 172.21.15.122:0/2756103356 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff277400640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff27815bea0 0x7ff2780b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff26e000640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2780d28e0 0x7ff2780b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 -- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2780d28e0 msgr2=0x7ff2780b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2780d28e0 0x7ff2780b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 -- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff27815bea0 msgr2=0x7ff2780b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff27815bea0 0x7ff2780b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 -- 172.21.15.122:0/2756103356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2780b3530 con 0x7ff2780cdb30 2024-09-17T05:25:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff276a00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2780cdb30 0x7ff2780b2c90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff2680b0040 tx=0x7ff2680a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff26fe00640 1 -- 172.21.15.122:0/2756103356 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2680b2040 con 0x7ff2780cdb30 2024-09-17T05:25:58.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff26fe00640 1 -- 172.21.15.122:0/2756103356 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2680043b0 con 0x7ff2780cdb30 2024-09-17T05:25:58.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff27de00640 1 -- 172.21.15.122:0/2756103356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2780b3810 con 0x7ff2780cdb30 2024-09-17T05:25:58.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff26fe00640 1 -- 172.21.15.122:0/2756103356 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268005280 con 0x7ff2780cdb30 2024-09-17T05:25:58.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff27de00640 1 -- 172.21.15.122:0/2756103356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2780b3dd0 con 0x7ff2780cdb30 2024-09-17T05:25:58.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.989+0000 7ff26fe00640 1 -- 172.21.15.122:0/2756103356 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff268005420 con 0x7ff2780cdb30 2024-09-17T05:25:58.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26fe00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff254073200 0x7ff2540756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26fe00640 1 -- 172.21.15.122:0/2756103356 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff26812ddc0 con 0x7ff2780cdb30 2024-09-17T05:25:58.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 Processor -- start 2024-09-17T05:25:58.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff254073200 0x7ff2540756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- start start 2024-09-17T05:25:58.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff254073200 0x7ff2540756c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2600058d0 tx=0x7ff260005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8d022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8cb7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 0x556ce8cb8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556ce8bd23f0 con 0x556ce8d02820 2024-09-17T05:25:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ce8b0fb20 con 0x556ce8d01f00 2024-09-17T05:25:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x556ce8afef50 con 0x556ce8cb8230 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8cb7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8cb7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53886/0 (socket says 172.21.15.122:53886) 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 -- 172.21.15.122:0/1384345001 learned_addr learned my addr 172.21.15.122:0/1384345001 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff276a00640 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8d022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff277400640 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 0x556ce8cb8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 -- 172.21.15.122:0/1384345001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 msgr2=0x556ce8cb8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 0x556ce8cb8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 -- 172.21.15.122:0/1384345001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 msgr2=0x556ce8d022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8d022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 -- 172.21.15.122:0/1384345001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce8ac6590 con 0x556ce8d02820 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff26e000640 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8cb7cf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff26000fdb0 tx=0x7ff260012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72e153fc83551b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff267e00640 1 -- 172.21.15.122:0/1384345001 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260022070 con 0x556ce8d02820 2024-09-17T05:25:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff267e00640 1 -- 172.21.15.122:0/1384345001 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff26001e450 con 0x556ce8d02820 2024-09-17T05:25:59.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff267e00640 1 -- 172.21.15.122:0/1384345001 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26001e860 con 0x556ce8d02820 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- 172.21.15.122:0/1384345001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 msgr2=0x556ce8cb7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8cb7cf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff26000fdb0 tx=0x7ff260012000 comp rx=0 tx=0).stop 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- 172.21.15.122:0/1384345001 shutdown_connections 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 0x556ce8cb8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8cb7cf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 --2- 172.21.15.122:0/1384345001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8d022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- 172.21.15.122:0/1384345001 >> 172.21.15.122:0/1384345001 conn(0x556ce8b9b130 msgr2=0x556ce8cb9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- 172.21.15.122:0/1384345001 shutdown_connections 2024-09-17T05:25:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.993+0000 7ff28117c000 1 -- 172.21.15.122:0/1384345001 wait complete. 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 -- start start 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 Processor -- start 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8cb8230 0x556ce8cec230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8cecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d02820 0x556ce8ced2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556ce8a995d0 con 0x556ce8cb8230 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ce8b02d60 con 0x556ce8d01f00 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x556ce8aa5500 con 0x556ce8d02820 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 10 client.0 _ll_register_callbacks cb 0x556ce8aa9810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8cb8230 0x556ce8cec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8cb8230 0x556ce8cec230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53898/0 (socket says 172.21.15.122:53898) 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 -- 172.21.15.122:0/4291698722 learned_addr learned my addr 172.21.15.122:0/4291698722 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff277400640 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d02820 0x556ce8ced2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff26e000640 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8cecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 -- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d02820 msgr2=0x556ce8ced2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d02820 0x556ce8ced2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 -- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 msgr2=0x556ce8cecd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8cecd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 -- 172.21.15.122:0/4291698722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce8a9d170 con 0x556ce8cb8230 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff26e000640 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8cecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff276a00640 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8cb8230 0x556ce8cec230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff26801cda0 tx=0x7ff2680f8570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e17d0da02c3a630 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff264c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff264c00640 1 -- 172.21.15.122:0/4291698722 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2680f8de0 con 0x556ce8cb8230 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff264c00640 1 -- 172.21.15.122:0/4291698722 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff268112ba0 con 0x556ce8cb8230 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 -- 172.21.15.122:0/4291698722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ce8a970f0 con 0x556ce8cb8230 2024-09-17T05:25:59.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff264c00640 1 -- 172.21.15.122:0/4291698722 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7ff2680f86e0 con 0x556ce8cb8230 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 10 client.5052 fetch_fsmap learned FSMap version 16 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 -- 172.21.15.122:0/4291698722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x556ce8b83ef0 con 0x556ce8cb8230 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff264c00640 1 -- 172.21.15.122:0/4291698722 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556ce8a970f0 con 0x556ce8cb8230 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff264c00640 1 -- 172.21.15.122:0/4291698722 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7ff26811c030 con 0x556ce8cb8230 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 10 client.5052 fetch_fsmap finished waiting for FSMap version 16 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 10 client.5052 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 20 client.5052 populate_metadata read hostname 'smithi122' 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] conn(0x556ce8cedcb0 0x556ce8cee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 4 client.5052 mds_command: new command op to 4873 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff28117c000 1 -- 172.21.15.122:0/4291698722 --> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ce8b11a70 con 0x556ce8cedcb0 2024-09-17T05:25:59.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:58.997+0000 7ff26e000640 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] conn(0x556ce8cedcb0 0x556ce8cee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff26e000640 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] conn(0x556ce8cedcb0 0x556ce8cee090 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f742f796be13720 server_cookie=6913712152844a90 in_seq=0 out_seq=0 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff264c00640 10 client.5052 ms_handle_connect on v2:172.21.15.173:6835/2399039177 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff264c00640 1 -- 172.21.15.122:0/4291698722 <== mds.0 v2:172.21.15.173:6835/2399039177 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ce8b11a70 con 0x556ce8cedcb0 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff264c00640 10 client.5052 handle_command_reply: tid=0 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 client.5052 shutdown 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 -- 172.21.15.122:0/4291698722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8cb8230 msgr2=0x556ce8cec230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8cb8230 0x556ce8cec230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff26801cda0 tx=0x7ff2680f8570 comp rx=0 tx=0).stop 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 -- 172.21.15.122:0/4291698722 shutdown_connections 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] conn(0x556ce8cedcb0 0x556ce8cee090 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d02820 0x556ce8ced2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff264c00640 0 client.5052 ms_handle_reset on v2:172.21.15.173:6835/2399039177 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d01f00 0x556ce8cecd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 --2- 172.21.15.122:0/4291698722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8cb8230 0x556ce8cec230 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 -- 172.21.15.122:0/4291698722 >> 172.21.15.122:0/4291698722 conn(0x556ce8b9b130 msgr2=0x556ce8cb8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 -- 172.21.15.122:0/4291698722 shutdown_connections 2024-09-17T05:25:59.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 1 -- 172.21.15.122:0/4291698722 wait complete. 2024-09-17T05:25:59.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.001+0000 7ff28117c000 20 client.5052 trim_cache size 0 max 16384 2024-09-17T05:25:59.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 Processor -- start 2024-09-17T05:25:59.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- start start 2024-09-17T05:25:59.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 0x556ce8d19840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d01f00 0x556ce8d00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d02820 0x556ce8d00660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2781ffd30 con 0x556ce8d01f00 2024-09-17T05:25:59.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff27806ef40 con 0x556ce8d02820 2024-09-17T05:25:59.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff278144b10 con 0x556ce8cb8230 2024-09-17T05:25:59.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d01f00 0x556ce8d00120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d01f00 0x556ce8d00120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53902/0 (socket says 172.21.15.122:53902) 2024-09-17T05:25:59.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 -- 172.21.15.122:0/1242106801 learned_addr learned my addr 172.21.15.122:0/1242106801 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 -- 172.21.15.122:0/1242106801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 msgr2=0x556ce8d19840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 --2- 172.21.15.122:0/1242106801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 0x556ce8d19840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 -- 172.21.15.122:0/1242106801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d02820 msgr2=0x556ce8d00660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 --2- 172.21.15.122:0/1242106801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d02820 0x556ce8d00660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 -- 172.21.15.122:0/1242106801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce8ced610 con 0x556ce8d01f00 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff26e000640 1 --2- 172.21.15.122:0/1242106801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d01f00 0x556ce8d00120 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556ce8b0e890 tx=0x7ff260004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7139871f2ecbb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff267400640 1 -- 172.21.15.122:0/1242106801 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260022040 con 0x556ce8d01f00 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff267400640 1 -- 172.21.15.122:0/1242106801 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff26000b370 con 0x556ce8d01f00 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff267400640 1 -- 172.21.15.122:0/1242106801 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26001ea50 con 0x556ce8d01f00 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- 172.21.15.122:0/1242106801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d01f00 msgr2=0x556ce8d00120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 --2- 172.21.15.122:0/1242106801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d01f00 0x556ce8d00120 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556ce8b0e890 tx=0x7ff260004740 comp rx=0 tx=0).stop 2024-09-17T05:25:59.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- 172.21.15.122:0/1242106801 shutdown_connections 2024-09-17T05:25:59.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 --2- 172.21.15.122:0/1242106801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8d02820 0x556ce8d00660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 --2- 172.21.15.122:0/1242106801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d01f00 0x556ce8d00120 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 --2- 172.21.15.122:0/1242106801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8cb8230 0x556ce8d19840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- 172.21.15.122:0/1242106801 >> 172.21.15.122:0/1242106801 conn(0x556ce8b9b130 msgr2=0x556ce8d0ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- 172.21.15.122:0/1242106801 shutdown_connections 2024-09-17T05:25:59.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- 172.21.15.122:0/1242106801 wait complete. 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 -- start start 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.021+0000 7ff28117c000 1 Processor -- start 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8cb8230 0x556ce8d785c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d01f00 0x556ce8d78b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8d79040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556ce8acaec0 con 0x556ce8d02820 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff2780cacf0 con 0x556ce8cb8230 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x556ce8a995d0 con 0x556ce8d01f00 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 10 client.0 _ll_register_callbacks cb 0x7ff268009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8d79040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8d79040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53910/0 (socket says 172.21.15.122:53910) 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 -- 172.21.15.122:0/170075160 learned_addr learned my addr 172.21.15.122:0/170075160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff26e000640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d01f00 0x556ce8d78b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff276a00640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8cb8230 0x556ce8d785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 -- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d01f00 msgr2=0x556ce8d78b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d01f00 0x556ce8d78b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 -- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8cb8230 msgr2=0x556ce8d785c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8cb8230 0x556ce8d785c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 -- 172.21.15.122:0/170075160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce8b5ad90 con 0x556ce8d02820 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff26e000640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d01f00 0x556ce8d78b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff276a00640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8cb8230 0x556ce8d785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff277400640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8d79040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff2700146e0 tx=0x7ff270014ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d66d41b3e568b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff264c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:25:59.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff264c00640 1 -- 172.21.15.122:0/170075160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27001a070 con 0x556ce8d02820 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff264c00640 1 -- 172.21.15.122:0/170075160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ff270003b90 con 0x556ce8d02820 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 -- 172.21.15.122:0/170075160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff26000b370 con 0x556ce8d02820 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff264c00640 1 -- 172.21.15.122:0/170075160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7ff270014e50 con 0x556ce8d02820 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 10 client.5058 fetch_fsmap learned FSMap version 16 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 -- 172.21.15.122:0/170075160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x556ce8ac5a90 con 0x556ce8d02820 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff264c00640 1 -- 172.21.15.122:0/170075160 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff26000b370 con 0x556ce8d02820 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff264c00640 1 -- 172.21.15.122:0/170075160 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7ff270028020 con 0x556ce8d02820 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 10 client.5058 fetch_fsmap finished waiting for FSMap version 16 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 10 client.5058 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 20 client.5058 populate_metadata read hostname 'smithi122' 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] conn(0x556ce8d169b0 0x556ce8d81820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 4 client.5058 mds_command: new command op to 4873 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:25:59.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff28117c000 1 -- 172.21.15.122:0/170075160 --> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] -- command(tid 0: {"prefix": "status"}) -- 0x556ce8b0d520 con 0x556ce8d169b0 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff276a00640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] conn(0x556ce8d169b0 0x556ce8d81820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff276a00640 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] conn(0x556ce8d169b0 0x556ce8d81820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f47acb0983b39a server_cookie=e1449d9bba7fdaa7 in_seq=0 out_seq=0 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.025+0000 7ff264c00640 10 client.5058 ms_handle_connect on v2:172.21.15.173:6835/2399039177 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff264c00640 1 -- 172.21.15.122:0/170075160 <== mds.0 v2:172.21.15.173:6835/2399039177 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x556ce8b0d520 con 0x556ce8d169b0 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff264c00640 10 client.5058 handle_command_reply: tid=0 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 client.5058 shutdown 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 -- 172.21.15.122:0/170075160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 msgr2=0x556ce8d79040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8d79040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff2700146e0 tx=0x7ff270014ba0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 -- 172.21.15.122:0/170075160 shutdown_connections 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:6835/2399039177,v1:172.21.15.173:6837/2399039177] conn(0x556ce8d169b0 0x556ce8d81820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556ce8d02820 0x556ce8d79040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556ce8d01f00 0x556ce8d78b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff264c00640 0 client.5058 ms_handle_reset on v2:172.21.15.173:6835/2399039177 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 --2- 172.21.15.122:0/170075160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce8cb8230 0x556ce8d785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 -- 172.21.15.122:0/170075160 >> 172.21.15.122:0/170075160 conn(0x556ce8b9b130 msgr2=0x556ce8d0f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 -- 172.21.15.122:0/170075160 shutdown_connections 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 1 -- 172.21.15.122:0/170075160 wait complete. 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff28117c000 20 client.5058 trim_cache size 0 max 16384 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:25:59.036 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4873, 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 10.479301655, 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 16, 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 33, 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 33, 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 17.353121138999999 2024-09-17T05:25:59.037 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 -- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff254073200 msgr2=0x7ff2540756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff254073200 0x7ff2540756c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2600058d0 tx=0x7ff260005860 comp rx=0 tx=0).stop 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 -- 172.21.15.122:0/2756103356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2780cdb30 msgr2=0x7ff2780b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2780cdb30 0x7ff2780b2c90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff2680b0040 tx=0x7ff2680a5680 comp rx=0 tx=0).stop 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 -- 172.21.15.122:0/2756103356 shutdown_connections 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff254073200 0x7ff2540756c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff27815bea0 0x7ff2780b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2780d28e0 0x7ff2780b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 --2- 172.21.15.122:0/2756103356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2780cdb30 0x7ff2780b2c90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 -- 172.21.15.122:0/2756103356 >> 172.21.15.122:0/2756103356 conn(0x7ff2780d1dc0 msgr2=0x7ff2780b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 -- 172.21.15.122:0/2756103356 shutdown_connections 2024-09-17T05:25:59.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.029+0000 7ff27de00640 1 -- 172.21.15.122:0/2756103356 wait complete. 2024-09-17T05:25:59.053 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:25:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.189+0000 7f01e4600640 1 -- 172.21.15.122:0/70303946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 msgr2=0x7f01dc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.189+0000 7f01e4600640 1 --2- 172.21.15.122:0/70303946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 0x7f01dc0ce670 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009900 tx=0x7f01d001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.189+0000 7f01e4600640 1 -- 172.21.15.122:0/70303946 shutdown_connections 2024-09-17T05:25:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.189+0000 7f01e4600640 1 --2- 172.21.15.122:0/70303946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0cdb30 0x7f01dc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.189+0000 7f01e4600640 1 --2- 172.21.15.122:0/70303946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 0x7f01dc0ce670 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.189+0000 7f01e4600640 1 -- 172.21.15.122:0/70303946 >> 172.21.15.122:0/70303946 conn(0x7f01dc0d1dc0 msgr2=0x7f01dc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.189+0000 7f01e4600640 1 -- 172.21.15.122:0/70303946 shutdown_connections 2024-09-17T05:25:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.189+0000 7f01e4600640 1 -- 172.21.15.122:0/70303946 wait complete. 2024-09-17T05:25:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 Processor -- start 2024-09-17T05:25:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 -- start start 2024-09-17T05:25:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0cdb30 0x7f01dc1432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0d28e0 0x7f01dc06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 0x7f01dc06f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc169020 con 0x7f01dc15bea0 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc168ea0 con 0x7f01dc0d28e0 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc1691a0 con 0x7f01dc0cdb30 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 0x7f01dc06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 0x7f01dc06f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53932/0 (socket says 172.21.15.122:53932) 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 -- 172.21.15.122:0/3320769824 learned_addr learned my addr 172.21.15.122:0/3320769824 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e1c00640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0cdb30 0x7f01dc1432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e1200640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0d28e0 0x7f01dc06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 -- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0cdb30 msgr2=0x7f01dc1432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0cdb30 0x7f01dc1432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 -- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0d28e0 msgr2=0x7f01dc06eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0d28e0 0x7f01dc06eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 -- 172.21.15.122:0/3320769824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc0cc0f0 con 0x7f01dc15bea0 2024-09-17T05:25:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e1c00640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0cdb30 0x7f01dc1432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:25:59.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e2600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 0x7f01dc06f030 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f01d400c780 tx=0x7f01d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:59.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01d9600640 1 -- 172.21.15.122:0/3320769824 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4004b70 con 0x7f01dc15bea0 2024-09-17T05:25:59.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01d9600640 1 -- 172.21.15.122:0/3320769824 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01d4004e00 con 0x7f01dc15bea0 2024-09-17T05:25:59.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc0b4580 con 0x7f01dc15bea0 2024-09-17T05:25:59.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01d9600640 1 -- 172.21.15.122:0/3320769824 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d400aa90 con 0x7f01dc15bea0 2024-09-17T05:25:59.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.193+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc06e2f0 con 0x7f01dc15bea0 2024-09-17T05:25:59.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.197+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc0cc9d0 con 0x7f01dc15bea0 2024-09-17T05:25:59.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.197+0000 7f01d9600640 1 -- 172.21.15.122:0/3320769824 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f01d400ac30 con 0x7f01dc15bea0 2024-09-17T05:25:59.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.197+0000 7f01d9600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01bc073200 0x7f01bc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:25:59.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.197+0000 7f01d9600640 1 -- 172.21.15.122:0/3320769824 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f01d4094550 con 0x7f01dc15bea0 2024-09-17T05:25:59.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.197+0000 7f01e1c00640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01bc073200 0x7f01bc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:25:59.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.197+0000 7f01e1c00640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01bc073200 0x7f01bc0756c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f01d00af000 tx=0x7f01d001b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:25:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.201+0000 7f01d9600640 1 -- 172.21.15.122:0/3320769824 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f01d4020070 con 0x7f01dc15bea0 2024-09-17T05:25:59.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.405+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f01dc0b5aa0 con 0x7f01dc15bea0 2024-09-17T05:25:59.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.405+0000 7f01d9600640 1 -- 172.21.15.122:0/3320769824 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f01d405f560 con 0x7f01dc15bea0 2024-09-17T05:25:59.410 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:25:59.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01bc073200 msgr2=0x7f01bc0756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01bc073200 0x7f01bc0756c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f01d00af000 tx=0x7f01d001b120 comp rx=0 tx=0).stop 2024-09-17T05:25:59.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 msgr2=0x7f01dc06f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:25:59.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 0x7f01dc06f030 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f01d400c780 tx=0x7f01d400cc40 comp rx=0 tx=0).stop 2024-09-17T05:25:59.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 shutdown_connections 2024-09-17T05:25:59.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01bc073200 0x7f01bc0756c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15bea0 0x7f01dc06f030 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0d28e0 0x7f01dc06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 --2- 172.21.15.122:0/3320769824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0cdb30 0x7f01dc1432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:25:59.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 >> 172.21.15.122:0/3320769824 conn(0x7f01dc0d1dc0 msgr2=0x7f01dc0cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:25:59.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 shutdown_connections 2024-09-17T05:25:59.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59.409+0000 7f01e4600640 1 -- 172.21.15.122:0/3320769824 wait complete. 2024-09-17T05:25:59.430 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:25:59.431 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:25:59.431 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:25:59.431 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:25:59.431 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:59.431 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:25:59.431 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:25:59.431 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:25:59.431 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:25:59.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:59.432 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:25:59.440 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:25:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:25:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:25:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:59.440 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:25:59.441 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42408sec preferred_lft 42408sec 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:59.442 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:25:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:25:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:25:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:59.443 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:25:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:25:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:25:59.443 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:25:59.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:59.444 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:25:59.504 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:25:59.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:59.506 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:59.559 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:25:59.560 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:25:59.560 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:25:59.560 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:25:59.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:25:59.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:25:59.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:25:59.560 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:25:59.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:25:59.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:25:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:25:59.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:59.699 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:25:59.707 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:25:59.708 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:25:59.708 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:25:59.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:59.709 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:25:59.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:59.771 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:59.823 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:25:59.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:25:59.823 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:25:59.874 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:25:59.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:59.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:59.965 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:59.965 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:59.965 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:59.977 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:59.977 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:59.977 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:59.977 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:59.977 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:59.988 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:25:59.989 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:25:59.989 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:25:59.989 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:25:59.989 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:25:59.989 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:25:59.989 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:25:59.989 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:25:59.992 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:25:59.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:25:59.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:26:00.013 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:26:00.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:26:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:26:00.125 INFO:teuthology.orchestra.run.smithi122.stdout:client4781 2024-09-17T05:26:00.126 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:26:00.126 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:00.134 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:26:00.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 -- 172.21.15.122:0/2093314028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa18815e030 msgr2=0x7fa18806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:00.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2093314028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa18815e030 0x7fa18806ecb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa178009840 tx=0x7fa17801f010 comp rx=0 tx=0).stop 2024-09-17T05:26:00.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 -- 172.21.15.122:0/2093314028 shutdown_connections 2024-09-17T05:26:00.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2093314028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1880d1000 0x7fa1880ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2093314028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa18815e030 0x7fa18806ecb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 -- 172.21.15.122:0/2093314028 >> 172.21.15.122:0/2093314028 conn(0x7fa1880d1da0 msgr2=0x7fa1880d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:00.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 -- 172.21.15.122:0/2093314028 shutdown_connections 2024-09-17T05:26:00.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 -- 172.21.15.122:0/2093314028 wait complete. 2024-09-17T05:26:00.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 Processor -- start 2024-09-17T05:26:00.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.317+0000 7fa18ea00640 1 -- start start 2024-09-17T05:26:00.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1880d00c0 0x7fa1880cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:00.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa18815e030 0x7fa1881541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:00.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18815c0c0 0x7fa1881f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:00.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880cc830 con 0x7fa18815c0c0 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1880cc6b0 con 0x7fa18815e030 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa1880cc9b0 con 0x7fa1880d00c0 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18815c0c0 0x7fa1881f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18815c0c0 0x7fa1881f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53936/0 (socket says 172.21.15.122:53936) 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 -- 172.21.15.122:0/2203897156 learned_addr learned my addr 172.21.15.122:0/2203897156 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa17fe00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa18815e030 0x7fa1881541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa187e00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1880d00c0 0x7fa1880cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 -- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1880d00c0 msgr2=0x7fa1880cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1880d00c0 0x7fa1880cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 -- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa18815e030 msgr2=0x7fa1881541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa18815e030 0x7fa1881541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 -- 172.21.15.122:0/2203897156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18815fda0 con 0x7fa18815c0c0 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ca00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18815c0c0 0x7fa1881f4d50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa18000cbc0 tx=0x7fa18000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:00.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa184c00640 1 -- 172.21.15.122:0/2203897156 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180016020 con 0x7fa18815c0c0 2024-09-17T05:26:00.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa184c00640 1 -- 172.21.15.122:0/2203897156 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa18000ab00 con 0x7fa18815c0c0 2024-09-17T05:26:00.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa188160040 con 0x7fa18815c0c0 2024-09-17T05:26:00.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa184c00640 1 -- 172.21.15.122:0/2203897156 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180012420 con 0x7fa18815c0c0 2024-09-17T05:26:00.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa188160600 con 0x7fa18815c0c0 2024-09-17T05:26:00.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.321+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1880cce60 con 0x7fa18815c0c0 2024-09-17T05:26:00.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.325+0000 7fa184c00640 1 -- 172.21.15.122:0/2203897156 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa180019020 con 0x7fa18815c0c0 2024-09-17T05:26:00.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.325+0000 7fa184c00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa164073230 0x7fa1640756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:00.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.325+0000 7fa184c00640 1 -- 172.21.15.122:0/2203897156 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa1800942c0 con 0x7fa18815c0c0 2024-09-17T05:26:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.325+0000 7fa187e00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa164073230 0x7fa1640756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.325+0000 7fa187e00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa164073230 0x7fa1640756f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b2000 tx=0x7fa17801d850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:00.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.329+0000 7fa184c00640 1 -- 172.21.15.122:0/2203897156 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa180020020 con 0x7fa18815c0c0 2024-09-17T05:26:00.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:00.533+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa188070350 con 0x7fa18815c0c0 2024-09-17T05:26:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.041+0000 7fa184c00640 1 -- 172.21.15.122:0/2203897156 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) ==== 136+0+0 (secure 0 0 0) 0x7fa18005edb0 con 0x7fa18815c0c0 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa164073230 msgr2=0x7fa1640756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa164073230 0x7fa1640756f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b2000 tx=0x7fa17801d850 comp rx=0 tx=0).stop 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18815c0c0 msgr2=0x7fa1881f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18815c0c0 0x7fa1881f4d50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa18000cbc0 tx=0x7fa18000ccb0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 shutdown_connections 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa164073230 0x7fa1640756f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa18815c0c0 0x7fa1881f4d50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa18815e030 0x7fa1881541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 --2- 172.21.15.122:0/2203897156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1880d00c0 0x7fa1880cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 >> 172.21.15.122:0/2203897156 conn(0x7fa1880d1da0 msgr2=0x7fa1880ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 shutdown_connections 2024-09-17T05:26:01.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.045+0000 7fa18ea00640 1 -- 172.21.15.122:0/2203897156 wait complete. 2024-09-17T05:26:01.069 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:26:01.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- 172.21.15.122:0/1257291490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881617b0 msgr2=0x7f86880d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 --2- 172.21.15.122:0/1257291490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881617b0 0x7f86880d0a80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86880d4aa0 tx=0x7f867401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- 172.21.15.122:0/1257291490 shutdown_connections 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 --2- 172.21.15.122:0/1257291490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881617b0 0x7f86880d0a80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 --2- 172.21.15.122:0/1257291490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86881613d0 0x7f86880d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- 172.21.15.122:0/1257291490 >> 172.21.15.122:0/1257291490 conn(0x7f86880d3e00 msgr2=0x7f86880d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- 172.21.15.122:0/1257291490 shutdown_connections 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- 172.21.15.122:0/1257291490 wait complete. 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 Processor -- start 2024-09-17T05:26:01.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- start start 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86880d0fc0 0x7f8688157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881613d0 0x7f86880c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86881617b0 0x7f86880c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f868816b050 con 0x7f86881613d0 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f868816aed0 con 0x7f86881617b0 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868da00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f868816b1d0 con 0x7f86880d0fc0 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f8687400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881613d0 0x7f86880c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f8687400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881613d0 0x7f86880c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53950/0 (socket says 172.21.15.122:53950) 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f8687400640 1 -- 172.21.15.122:0/2373583917 learned_addr learned my addr 172.21.15.122:0/2373583917 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f8687e00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86880d0fc0 0x7f8688157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.209+0000 7f868d000640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86881617b0 0x7f86880c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f8687400640 1 -- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86880d0fc0 msgr2=0x7f8688157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f8687400640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86880d0fc0 0x7f8688157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f8687400640 1 -- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86881617b0 msgr2=0x7f86880c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f8687400640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86881617b0 0x7f86880c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f8687400640 1 -- 172.21.15.122:0/2373583917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86881f8a20 con 0x7f86881613d0 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f8687400640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881613d0 0x7f86880c88e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86740b0040 tx=0x7f8674004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f867fe00640 1 -- 172.21.15.122:0/2373583917 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86740b2040 con 0x7f86881613d0 2024-09-17T05:26:01.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f867fe00640 1 -- 172.21.15.122:0/2373583917 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8674005120 con 0x7f86881613d0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86881f7ea0 con 0x7f86881613d0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f867fe00640 1 -- 172.21.15.122:0/2373583917 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674005570 con 0x7f86881613d0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86881f8460 con 0x7f86881613d0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86880d19d0 con 0x7f86881613d0 2024-09-17T05:26:01.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f867fe00640 1 -- 172.21.15.122:0/2373583917 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f86740bd060 con 0x7f86881613d0 2024-09-17T05:26:01.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f867fe00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8668073200 0x7f86680756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.213+0000 7f867fe00640 1 -- 172.21.15.122:0/2373583917 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f867412d8a0 con 0x7f86881613d0 2024-09-17T05:26:01.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.217+0000 7f8687e00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8668073200 0x7f86680756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.217+0000 7f8687e00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8668073200 0x7f86680756c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86881f5640 tx=0x7f86780091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.217+0000 7f867fe00640 1 -- 172.21.15.122:0/2373583917 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f86740164d0 con 0x7f86881613d0 2024-09-17T05:26:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.405+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f868815ac30 con 0x7f8668073200 2024-09-17T05:26:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f867fe00640 1 -- 172.21.15.122:0/2373583917 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f868815ac30 con 0x7f8668073200 2024-09-17T05:26:01.412 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:26:01.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:26:01.412 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:26:01.412 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:26:01.412 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8668073200 msgr2=0x7f86680756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8668073200 0x7f86680756c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86881f5640 tx=0x7f86780091e0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881613d0 msgr2=0x7f86880c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881613d0 0x7f86880c88e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f86740b0040 tx=0x7f8674004600 comp rx=0 tx=0).stop 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 shutdown_connections 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8668073200 0x7f86680756c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86881617b0 0x7f86880c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86881613d0 0x7f86880c88e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 --2- 172.21.15.122:0/2373583917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86880d0fc0 0x7f8688157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 >> 172.21.15.122:0/2373583917 conn(0x7f86880d3e00 msgr2=0x7f868806e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 shutdown_connections 2024-09-17T05:26:01.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.409+0000 7f868da00640 1 -- 172.21.15.122:0/2373583917 wait complete. 2024-09-17T05:26:01.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:26:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 -- 172.21.15.122:0/1526862351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 msgr2=0x7f24a00ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 --2- 172.21.15.122:0/1526862351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 0x7f24a00ce5e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2490009840 tx=0x7f249001c3e0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 -- 172.21.15.122:0/1526862351 shutdown_connections 2024-09-17T05:26:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 --2- 172.21.15.122:0/1526862351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24a00cf4e0 0x7f24a00cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 --2- 172.21.15.122:0/1526862351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 0x7f24a00ce5e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 -- 172.21.15.122:0/1526862351 >> 172.21.15.122:0/1526862351 conn(0x7f24a00d1ea0 msgr2=0x7f24a00d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 -- 172.21.15.122:0/1526862351 shutdown_connections 2024-09-17T05:26:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 -- 172.21.15.122:0/1526862351 wait complete. 2024-09-17T05:26:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 Processor -- start 2024-09-17T05:26:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.573+0000 7f24a5c00640 1 -- start start 2024-09-17T05:26:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a00cf4e0 0x7f24a0148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 0x7f24a009bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24a015c130 0x7f24a01582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0169b80 con 0x7f24a015bd50 2024-09-17T05:26:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0169a00 con 0x7f24a00cf4e0 2024-09-17T05:26:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f24a0169d00 con 0x7f24a015c130 2024-09-17T05:26:01.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 0x7f24a009bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 0x7f24a009bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36178/0 (socket says 172.21.15.122:36178) 2024-09-17T05:26:01.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 -- 172.21.15.122:0/694168679 learned_addr learned my addr 172.21.15.122:0/694168679 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249f400640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24a015c130 0x7f24a01582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249ea00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a00cf4e0 0x7f24a0148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 -- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24a015c130 msgr2=0x7f24a01582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24a015c130 0x7f24a01582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 -- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a00cf4e0 msgr2=0x7f24a0148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a00cf4e0 0x7f24a0148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 -- 172.21.15.122:0/694168679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a01f99d0 con 0x7f24a015bd50 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249ea00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a00cf4e0 0x7f24a0148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249f400640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24a015c130 0x7f24a01582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f249e000640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 0x7f24a009bba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c390 tx=0x7f248c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f2496a00640 1 -- 172.21.15.122:0/694168679 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c017070 con 0x7f24a015bd50 2024-09-17T05:26:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f2496a00640 1 -- 172.21.15.122:0/694168679 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f248c003600 con 0x7f24a015bd50 2024-09-17T05:26:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f2496a00640 1 -- 172.21.15.122:0/694168679 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c003a50 con 0x7f24a015bd50 2024-09-17T05:26:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a01f9cb0 con 0x7f24a015bd50 2024-09-17T05:26:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a01fa270 con 0x7f24a015bd50 2024-09-17T05:26:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.577+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a00ce5e0 con 0x7f24a015bd50 2024-09-17T05:26:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.581+0000 7f2496a00640 1 -- 172.21.15.122:0/694168679 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f248c01a070 con 0x7f24a015bd50 2024-09-17T05:26:01.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.581+0000 7f2496a00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2480073230 0x7f24800756f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.581+0000 7f2496a00640 1 -- 172.21.15.122:0/694168679 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f248c093390 con 0x7f24a015bd50 2024-09-17T05:26:01.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.581+0000 7f249ea00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2480073230 0x7f24800756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.581+0000 7f249ea00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2480073230 0x7f24800756f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24900ae000 tx=0x7f249001ac20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.585+0000 7f2496a00640 1 -- 172.21.15.122:0/694168679 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f248c05e3b0 con 0x7f24a015bd50 2024-09-17T05:26:01.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:26:01.668 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:26:01.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.753+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f24a00c8c30 con 0x7f24a015bd50 2024-09-17T05:26:01.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.753+0000 7f2496a00640 1 -- 172.21.15.122:0/694168679 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f248c05dcb0 con 0x7f24a015bd50 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2480073230 msgr2=0x7f24800756f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2480073230 0x7f24800756f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24900ae000 tx=0x7f249001ac20 comp rx=0 tx=0).stop 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 msgr2=0x7f24a009bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 0x7f24a009bba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c390 tx=0x7f248c00c850 comp rx=0 tx=0).stop 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 shutdown_connections 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2480073230 0x7f24800756f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24a015c130 0x7f24a01582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24a015bd50 0x7f24a009bba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 --2- 172.21.15.122:0/694168679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24a00cf4e0 0x7f24a0148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 >> 172.21.15.122:0/694168679 conn(0x7f24a00d1ea0 msgr2=0x7f24a00b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 shutdown_connections 2024-09-17T05:26:01.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.757+0000 7f24a5c00640 1 -- 172.21.15.122:0/694168679 wait complete. 2024-09-17T05:26:01.780 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:26:01.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- 172.21.15.122:0/1929403776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 msgr2=0x7f98d40d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 --2- 172.21.15.122:0/1929403776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 0x7f98d40d0240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009900 tx=0x7f98c401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- 172.21.15.122:0/1929403776 shutdown_connections 2024-09-17T05:26:01.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 --2- 172.21.15.122:0/1929403776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d40b0d10 0x7f98d40d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 --2- 172.21.15.122:0/1929403776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 0x7f98d40d0240 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- 172.21.15.122:0/1929403776 >> 172.21.15.122:0/1929403776 conn(0x7f98d409aa10 msgr2=0x7f98d409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:01.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- 172.21.15.122:0/1929403776 shutdown_connections 2024-09-17T05:26:01.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- 172.21.15.122:0/1929403776 wait complete. 2024-09-17T05:26:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 Processor -- start 2024-09-17T05:26:01.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- start start 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d40b0d10 0x7f98d415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d40d20f0 0x7f98d40afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 0x7f98d4148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98d4168cb0 con 0x7f98d415bd40 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f98d4168b30 con 0x7f98d40d20f0 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d8c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f98d4168e30 con 0x7f98d40b0d10 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 0x7f98d4148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d3400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d40b0d10 0x7f98d415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d40d20f0 0x7f98d40afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d3400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d40b0d10 0x7f98d415b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:36358/0 (socket says 172.21.15.122:36358) 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d3400640 1 -- 172.21.15.122:0/2738389089 learned_addr learned my addr 172.21.15.122:0/2738389089 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.921+0000 7f98d3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 0x7f98d4148050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36190/0 (socket says 172.21.15.122:36190) 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d3400640 1 -- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d40d20f0 msgr2=0x7f98d40afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d3400640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d40d20f0 0x7f98d40afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d3400640 1 -- 172.21.15.122:0/2738389089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 msgr2=0x7f98d4148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d3400640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 0x7f98d4148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d3400640 1 -- 172.21.15.122:0/2738389089 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d4148590 con 0x7f98d40b0d10 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d3e00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 0x7f98d4148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:26:01.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d3400640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d40b0d10 0x7f98d415b4c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98c40b0040 tx=0x7f98c40a66b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98cb400640 1 -- 172.21.15.122:0/2738389089 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40bd070 con 0x7f98d40b0d10 2024-09-17T05:26:01.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98cb400640 1 -- 172.21.15.122:0/2738389089 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98c4004470 con 0x7f98d40b0d10 2024-09-17T05:26:01.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d415e780 con 0x7f98d40b0d10 2024-09-17T05:26:01.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98cb400640 1 -- 172.21.15.122:0/2738389089 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4005290 con 0x7f98d40b0d10 2024-09-17T05:26:01.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d415ea20 con 0x7f98d40b0d10 2024-09-17T05:26:01.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.925+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d40d1bb0 con 0x7f98d40b0d10 2024-09-17T05:26:01.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.929+0000 7f98cb400640 1 -- 172.21.15.122:0/2738389089 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f98c40a6820 con 0x7f98d40b0d10 2024-09-17T05:26:01.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.929+0000 7f98cb400640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b4073200 0x7f98b40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:01.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.929+0000 7f98cb400640 1 -- 172.21.15.122:0/2738389089 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f98c412e5e0 con 0x7f98d40b0d10 2024-09-17T05:26:01.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.929+0000 7f98d2a00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b4073200 0x7f98b40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:01.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.929+0000 7f98d2a00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b4073200 0x7f98b40756c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98d406f170 tx=0x7f98c000b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:01.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:01.933+0000 7f98cb400640 1 -- 172.21.15.122:0/2738389089 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f98c40164d0 con 0x7f98d40b0d10 2024-09-17T05:26:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.101+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f98d40ce9f0 con 0x7f98d40b0d10 2024-09-17T05:26:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.101+0000 7f98cb400640 1 -- 172.21.15.122:0/2738389089 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v28) ==== 116+0+2 (secure 0 0 0) 0x7f98c40f8fa0 con 0x7f98d40b0d10 2024-09-17T05:26:02.106 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:26:02.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b4073200 msgr2=0x7f98b40756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b4073200 0x7f98b40756c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98d406f170 tx=0x7f98c000b390 comp rx=0 tx=0).stop 2024-09-17T05:26:02.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d40b0d10 msgr2=0x7f98d415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d40b0d10 0x7f98d415b4c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98c40b0040 tx=0x7f98c40a66b0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 shutdown_connections 2024-09-17T05:26:02.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b4073200 0x7f98b40756c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98d415bd40 0x7f98d4148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d40d20f0 0x7f98d40afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 --2- 172.21.15.122:0/2738389089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d40b0d10 0x7f98d415b4c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 >> 172.21.15.122:0/2738389089 conn(0x7f98d409aa10 msgr2=0x7f98d415a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 shutdown_connections 2024-09-17T05:26:02.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.105+0000 7f98d8c00640 1 -- 172.21.15.122:0/2738389089 wait complete. 2024-09-17T05:26:02.128 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:26:02.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9119000640 1 -- 172.21.15.122:0/2697402822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 msgr2=0x7f90f8004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9119000640 1 --2- 172.21.15.122:0/2697402822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 0x7f90f8004020 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9104001860 tx=0x7f910401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9112800640 1 -- 172.21.15.122:0/2697402822 reap_dead start 2024-09-17T05:26:02.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9119000640 1 -- 172.21.15.122:0/2697402822 shutdown_connections 2024-09-17T05:26:02.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9119000640 1 -- 172.21.15.122:0/2697402822 >> 172.21.15.122:0/2697402822 conn(0x7f91140d1dd0 msgr2=0x7f91140d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9119000640 1 -- 172.21.15.122:0/2697402822 shutdown_connections 2024-09-17T05:26:02.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9119000640 1 -- 172.21.15.122:0/2697402822 wait complete. 2024-09-17T05:26:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9119000640 1 Processor -- start 2024-09-17T05:26:02.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.269+0000 7f9119000640 1 -- start start 2024-09-17T05:26:02.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 0x7f91141f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91140b3df0 0x7f9114156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9114156db0 0x7f9114157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9114167eb0 con 0x7f90f8003c20 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9114167d30 con 0x7f9114156db0 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9114168030 con 0x7f91140b3df0 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 0x7f91141f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 0x7f91141f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36206/0 (socket says 172.21.15.122:36206) 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 -- 172.21.15.122:0/1923911753 learned_addr learned my addr 172.21.15.122:0/1923911753 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f910be00640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91140b3df0 0x7f9114156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 -- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91140b3df0 msgr2=0x7f9114156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91140b3df0 0x7f9114156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9112800640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9114156db0 0x7f9114157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 -- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9114156db0 msgr2=0x7f9114157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9114156db0 0x7f9114157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 -- 172.21.15.122:0/1923911753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91141f9000 con 0x7f90f8003c20 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f910be00640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91140b3df0 0x7f9114156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9112800640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9114156db0 0x7f9114157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9111e00640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 0x7f91141f8ac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9104005040 tx=0x7f9104003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f910a000640 1 -- 172.21.15.122:0/1923911753 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104001480 con 0x7f90f8003c20 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f910a000640 1 -- 172.21.15.122:0/1923911753 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91040b4030 con 0x7f90f8003c20 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f910a000640 1 -- 172.21.15.122:0/1923911753 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91040068d0 con 0x7f90f8003c20 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9114156b70 con 0x7f90f8003c20 2024-09-17T05:26:02.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91140cb490 con 0x7f90f8003c20 2024-09-17T05:26:02.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.273+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91140b1a30 con 0x7f90f8003c20 2024-09-17T05:26:02.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.277+0000 7f910a000640 1 -- 172.21.15.122:0/1923911753 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f91040a5b10 con 0x7f90f8003c20 2024-09-17T05:26:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.277+0000 7f910a000640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90f0073200 0x7f90f00756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.277+0000 7f910a000640 1 -- 172.21.15.122:0/1923911753 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f910412e1f0 con 0x7f90f8003c20 2024-09-17T05:26:02.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.277+0000 7f910be00640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90f0073200 0x7f90f00756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.277+0000 7f910be00640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90f0073200 0x7f90f00756c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90fc001870 tx=0x7f90fc0011a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.281+0000 7f910a000640 1 -- 172.21.15.122:0/1923911753 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9104016690 con 0x7f90f8003c20 2024-09-17T05:26:02.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.449+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f91140d1080 con 0x7f90f8003c20 2024-09-17T05:26:02.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f910a000640 1 -- 172.21.15.122:0/1923911753 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7f91040f9310 con 0x7f90f8003c20 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90f0073200 msgr2=0x7f90f00756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90f0073200 0x7f90f00756c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90fc001870 tx=0x7f90fc0011a0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 msgr2=0x7f91141f8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 0x7f91141f8ac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9104005040 tx=0x7f9104003c80 comp rx=0 tx=0).stop 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 shutdown_connections 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90f0073200 0x7f90f00756c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9114156db0 0x7f9114157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91140b3df0 0x7f9114156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 --2- 172.21.15.122:0/1923911753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90f8003c20 0x7f91141f8ac0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 >> 172.21.15.122:0/1923911753 conn(0x7f91140d1dd0 msgr2=0x7f91140cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 shutdown_connections 2024-09-17T05:26:02.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.453+0000 7f9119000640 1 -- 172.21.15.122:0/1923911753 wait complete. 2024-09-17T05:26:02.479 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group1 2024-09-17T05:26:02.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.617+0000 7fdfcac00640 1 -- 172.21.15.122:0/1664354586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40b2c80 msgr2=0x7fdfc4073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.617+0000 7fdfcac00640 1 --2- 172.21.15.122:0/1664354586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40b2c80 0x7fdfc4073790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009c80 tx=0x7fdfbc01caa0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.617+0000 7fdfcac00640 1 -- 172.21.15.122:0/1664354586 shutdown_connections 2024-09-17T05:26:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.617+0000 7fdfcac00640 1 --2- 172.21.15.122:0/1664354586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40b2c80 0x7fdfc4073790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.617+0000 7fdfcac00640 1 --2- 172.21.15.122:0/1664354586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc40cbdb0 0x7fdfc40b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.617+0000 7fdfcac00640 1 --2- 172.21.15.122:0/1664354586 >> v2:172.21.15.173:3300/0 conn(0x7fdfc415ac70 0x7fdfc40cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.617+0000 7fdfcac00640 1 -- 172.21.15.122:0/1664354586 >> 172.21.15.122:0/1664354586 conn(0x7fdfc40cd740 msgr2=0x7fdfc40cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.617+0000 7fdfcac00640 1 -- 172.21.15.122:0/1664354586 shutdown_connections 2024-09-17T05:26:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 -- 172.21.15.122:0/1664354586 wait complete. 2024-09-17T05:26:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 Processor -- start 2024-09-17T05:26:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 -- start start 2024-09-17T05:26:02.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc40b2c80 0x7fdfc415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40cbdb0 0x7fdfc41596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc415ac70 0x7fdfc4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4070030 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc406feb0 con 0x7fdfc40b2c80 2024-09-17T05:26:02.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc40701b0 con 0x7fdfc415ac70 2024-09-17T05:26:02.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40cbdb0 0x7fdfc41596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40cbdb0 0x7fdfc41596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36230/0 (socket says 172.21.15.122:36230) 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 -- 172.21.15.122:0/3003799833 learned_addr learned my addr 172.21.15.122:0/3003799833 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfc3e00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc40b2c80 0x7fdfc415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfc8c00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc415ac70 0x7fdfc4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 -- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc415ac70 msgr2=0x7fdfc4159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc415ac70 0x7fdfc4159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 -- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc40b2c80 msgr2=0x7fdfc415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc40b2c80 0x7fdfc415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 -- 172.21.15.122:0/3003799833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc415b3a0 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfc8c00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc415ac70 0x7fdfc4159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfbb400640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40cbdb0 0x7fdfc41596e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d780 tx=0x7fdfac00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfc0c00640 1 -- 172.21.15.122:0/3003799833 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac017020 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfc0c00640 1 -- 172.21.15.122:0/3003799833 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfac00de90 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc415b680 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.621+0000 7fdfc0c00640 1 -- 172.21.15.122:0/3003799833 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac009a20 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.625+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc415bc40 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.625+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc415be40 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.625+0000 7fdfc0c00640 1 -- 172.21.15.122:0/3003799833 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdfac01a020 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.625+0000 7fdfc0c00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa0073200 0x7fdfa00756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:02.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.625+0000 7fdfc0c00640 1 -- 172.21.15.122:0/3003799833 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdfac094690 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.625+0000 7fdfc3e00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa0073200 0x7fdfa00756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:02.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.629+0000 7fdfc3e00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa0073200 0x7fdfa00756c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4005fd0 tx=0x7fdfb4005e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:02.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.629+0000 7fdfc0c00640 1 -- 172.21.15.122:0/3003799833 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdfac021020 con 0x7fdfc40cbdb0 2024-09-17T05:26:02.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.817+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fdfc4158d30 con 0x7fdfa0073200 2024-09-17T05:26:02.828 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:26:02.826+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:26:02.828 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:26:02.826+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:26:02.828 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:26:02.826+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:26:02.828 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:26:02.826+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:26:02.828 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:26:02.826+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:26:02.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.841+0000 7fdfc0c00640 1 -- 172.21.15.122:0/3003799833 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdfc4158d30 con 0x7fdfa0073200 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa0073200 msgr2=0x7fdfa00756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa0073200 0x7fdfa00756c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4005fd0 tx=0x7fdfb4005e40 comp rx=0 tx=0).stop 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40cbdb0 msgr2=0x7fdfc41596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40cbdb0 0x7fdfc41596e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d780 tx=0x7fdfac00dc40 comp rx=0 tx=0).stop 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 shutdown_connections 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa0073200 0x7fdfa00756c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc415ac70 0x7fdfc4159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc40cbdb0 0x7fdfc41596e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 --2- 172.21.15.122:0/3003799833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc40b2c80 0x7fdfc415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 >> 172.21.15.122:0/3003799833 conn(0x7fdfc40cd740 msgr2=0x7fdfc41628c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:02.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 shutdown_connections 2024-09-17T05:26:02.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:02.845+0000 7fdfcac00640 1 -- 172.21.15.122:0/3003799833 wait complete. 2024-09-17T05:26:02.869 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 group1 --mode=777 2024-09-17T05:26:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 -- 172.21.15.122:0/71955559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 msgr2=0x7fb2980ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 --2- 172.21.15.122:0/71955559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 0x7fb2980ce700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb2980746b0 tx=0x7fb28801f0b0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 -- 172.21.15.122:0/71955559 shutdown_connections 2024-09-17T05:26:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 --2- 172.21.15.122:0/71955559 >> v2:172.21.15.173:3300/0 conn(0x7fb29815c1f0 0x7fb2980cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 --2- 172.21.15.122:0/71955559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 0x7fb2980ce700 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 -- 172.21.15.122:0/71955559 >> 172.21.15.122:0/71955559 conn(0x7fb2980d1da0 msgr2=0x7fb2980d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 -- 172.21.15.122:0/71955559 shutdown_connections 2024-09-17T05:26:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 -- 172.21.15.122:0/71955559 wait complete. 2024-09-17T05:26:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 Processor -- start 2024-09-17T05:26:03.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.013+0000 7fb29da00640 1 -- start start 2024-09-17T05:26:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2980b0b60 0x7fb2980b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 0x7fb2981f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb29815c1f0 0x7fb2981563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980cf510 con 0x7fb29815be10 2024-09-17T05:26:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980cf390 con 0x7fb2980b0b60 2024-09-17T05:26:03.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb2980cdda0 con 0x7fb29815c1f0 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 0x7fb2981f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb296a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2980b0b60 0x7fb2980b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 0x7fb2981f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36248/0 (socket says 172.21.15.122:36248) 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 -- 172.21.15.122:0/3310283397 learned_addr learned my addr 172.21.15.122:0/3310283397 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb297400640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb29815c1f0 0x7fb2981563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 -- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb29815c1f0 msgr2=0x7fb2981563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb29815c1f0 0x7fb2981563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 -- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2980b0b60 msgr2=0x7fb2980b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2980b0b60 0x7fb2980b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 -- 172.21.15.122:0/3310283397 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2981f5f90 con 0x7fb29815be10 2024-09-17T05:26:03.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28fe00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 0x7fb2981f1f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e7d0 tx=0x7fb28000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28ea00640 1 -- 172.21.15.122:0/3310283397 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28000ccc0 con 0x7fb29815be10 2024-09-17T05:26:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28ea00640 1 -- 172.21.15.122:0/3310283397 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb280010430 con 0x7fb29815be10 2024-09-17T05:26:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2981f6270 con 0x7fb29815be10 2024-09-17T05:26:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28ea00640 1 -- 172.21.15.122:0/3310283397 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280010860 con 0x7fb29815be10 2024-09-17T05:26:03.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981f6830 con 0x7fb29815be10 2024-09-17T05:26:03.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb298160550 con 0x7fb29815be10 2024-09-17T05:26:03.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.017+0000 7fb28ea00640 1 -- 172.21.15.122:0/3310283397 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb280010a00 con 0x7fb29815be10 2024-09-17T05:26:03.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.021+0000 7fb28ea00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb274073200 0x7fb2740756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.021+0000 7fb28ea00640 1 -- 172.21.15.122:0/3310283397 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb280094bb0 con 0x7fb29815be10 2024-09-17T05:26:03.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.021+0000 7fb296a00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb274073200 0x7fb2740756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.021+0000 7fb296a00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb274073200 0x7fb2740756c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb2880b2000 tx=0x7fb2880aa040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.025+0000 7fb28ea00640 1 -- 172.21.15.122:0/3310283397 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb28005f600 con 0x7fb29815be10 2024-09-17T05:26:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.217+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fb2980d2320 con 0x7fb274073200 2024-09-17T05:26:03.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.365+0000 7fb28ea00640 1 -- 172.21.15.122:0/3310283397 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb2980d2320 con 0x7fb274073200 2024-09-17T05:26:03.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.365+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb274073200 msgr2=0x7fb2740756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.365+0000 7fb29da00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb274073200 0x7fb2740756c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb2880b2000 tx=0x7fb2880aa040 comp rx=0 tx=0).stop 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.365+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 msgr2=0x7fb2981f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.365+0000 7fb29da00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 0x7fb2981f1f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb28000e7d0 tx=0x7fb28000ec90 comp rx=0 tx=0).stop 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.369+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 shutdown_connections 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.369+0000 7fb29da00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb274073200 0x7fb2740756c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.369+0000 7fb29da00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb29815c1f0 0x7fb2981563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.369+0000 7fb29da00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb29815be10 0x7fb2981f1f40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.369+0000 7fb29da00640 1 --2- 172.21.15.122:0/3310283397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2980b0b60 0x7fb2980b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.369+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 >> 172.21.15.122:0/3310283397 conn(0x7fb2980d1da0 msgr2=0x7fb2980716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.369+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 shutdown_connections 2024-09-17T05:26:03.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.369+0000 7fb29da00640 1 -- 172.21.15.122:0/3310283397 wait complete. 2024-09-17T05:26:03.392 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-17T05:26:03.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.533+0000 7f5bfde00640 1 -- 172.21.15.122:0/3568895974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf815e360 msgr2=0x7f5bf80adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.533+0000 7f5bfde00640 1 --2- 172.21.15.122:0/3568895974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf815e360 0x7f5bf80adf50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009c80 tx=0x7f5be801cd10 comp rx=0 tx=0).stop 2024-09-17T05:26:03.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.533+0000 7f5bfde00640 1 -- 172.21.15.122:0/3568895974 shutdown_connections 2024-09-17T05:26:03.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.533+0000 7f5bfde00640 1 --2- 172.21.15.122:0/3568895974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf815e360 0x7f5bf80adf50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.533+0000 7f5bfde00640 1 -- 172.21.15.122:0/3568895974 >> 172.21.15.122:0/3568895974 conn(0x7f5bf80cfbc0 msgr2=0x7f5bf80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.533+0000 7f5bfde00640 1 -- 172.21.15.122:0/3568895974 shutdown_connections 2024-09-17T05:26:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.533+0000 7f5bfde00640 1 -- 172.21.15.122:0/3568895974 wait complete. 2024-09-17T05:26:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 Processor -- start 2024-09-17T05:26:03.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 -- start start 2024-09-17T05:26:03.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf80aef60 0x7f5bf8146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bf815e360 0x7f5bf80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bf815e740 0x7f5bf8149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf8169540 con 0x7f5bf80aef60 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf815f900 con 0x7f5bf815e360 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf81696c0 con 0x7f5bf815e740 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf80aef60 0x7f5bf8146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf80aef60 0x7f5bf8146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36256/0 (socket says 172.21.15.122:36256) 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 -- 172.21.15.122:0/103102767 learned_addr learned my addr 172.21.15.122:0/103102767 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bee000640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bf815e360 0x7f5bf80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf7400640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bf815e740 0x7f5bf8149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 -- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bf815e740 msgr2=0x7f5bf8149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bf815e740 0x7f5bf8149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 -- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bf815e360 msgr2=0x7f5bf80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bf815e360 0x7f5bf80b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 -- 172.21.15.122:0/103102767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf81552c0 con 0x7f5bf80aef60 2024-09-17T05:26:03.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bee000640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bf815e360 0x7f5bf80b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:26:03.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bf6a00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf80aef60 0x7f5bf8146fc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009bd0 tx=0x7f5be80a5af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5befe00640 1 -- 172.21.15.122:0/103102767 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80b2040 con 0x7f5bf80aef60 2024-09-17T05:26:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5befe00640 1 -- 172.21.15.122:0/103102767 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be80042c0 con 0x7f5bf80aef60 2024-09-17T05:26:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5befe00640 1 -- 172.21.15.122:0/103102767 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80046f0 con 0x7f5bf80aef60 2024-09-17T05:26:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf81555a0 con 0x7f5bf80aef60 2024-09-17T05:26:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.537+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf8155b60 con 0x7f5bf80aef60 2024-09-17T05:26:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.541+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf815ebd0 con 0x7f5bf80aef60 2024-09-17T05:26:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.541+0000 7f5befe00640 1 -- 172.21.15.122:0/103102767 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5be80ad440 con 0x7f5bf80aef60 2024-09-17T05:26:03.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.541+0000 7f5befe00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bd4073200 0x7f5bd40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:26:03.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.541+0000 7f5befe00640 1 -- 172.21.15.122:0/103102767 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5be812e990 con 0x7f5bf80aef60 2024-09-17T05:26:03.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.541+0000 7f5bee000640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bd4073200 0x7f5bd40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:26:03.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.541+0000 7f5bee000640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bd4073200 0x7f5bd40756c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5bf815d9f0 tx=0x7f5be0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:26:03.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.545+0000 7f5befe00640 1 -- 172.21.15.122:0/103102767 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5be8016990 con 0x7f5bf80aef60 2024-09-17T05:26:03.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.741+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f5bf80d0830 con 0x7f5bd4073200 2024-09-17T05:26:03.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.745+0000 7f5befe00640 1 -- 172.21.15.122:0/103102767 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f5bf80d0830 con 0x7f5bd4073200 2024-09-17T05:26:03.749 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bd4073200 msgr2=0x7f5bd40756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bd4073200 0x7f5bd40756c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5bf815d9f0 tx=0x7f5be0009040 comp rx=0 tx=0).stop 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf80aef60 msgr2=0x7f5bf8146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf80aef60 0x7f5bf8146fc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009bd0 tx=0x7f5be80a5af0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 shutdown_connections 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bd4073200 0x7f5bd40756c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bf815e740 0x7f5bf8149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bf815e360 0x7f5bf80b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 --2- 172.21.15.122:0/103102767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bf80aef60 0x7f5bf8146fc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 >> 172.21.15.122:0/103102767 conn(0x7f5bf80cfbc0 msgr2=0x7f5bf80cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 shutdown_connections 2024-09-17T05:26:03.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:26:03.749+0000 7f5bfde00640 1 -- 172.21.15.122:0/103102767 wait complete. 2024-09-17T05:26:03.771 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:26:03.771 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:26:03.772 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:03.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:03.780 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:26:12.538 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:26:12.538 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:26:12.538 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.69224 s, 124 MB/s 2024-09-17T05:26:12.539 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:26:12.539 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:12.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:12.548 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:26:21.560 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:26:21.560 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:26:21.560 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.95877 s, 120 MB/s 2024-09-17T05:26:21.561 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:26:21.561 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:21.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:21.570 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:26:30.336 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:26:30.336 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:26:30.336 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.71495 s, 123 MB/s 2024-09-17T05:26:30.337 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:26:30.337 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:30.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:30.346 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:26:31.723 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:26:31.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:26:39.289 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:26:39.289 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:26:39.289 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.89287 s, 121 MB/s 2024-09-17T05:26:39.290 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:26:39.290 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:39.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:39.299 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:26:47.895 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:26:47.895 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:26:47.895 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.54289 s, 126 MB/s 2024-09-17T05:26:47.896 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:26:47.896 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:47.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:47.906 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:26:56.846 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:26:56.846 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:26:56.846 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.88585 s, 121 MB/s 2024-09-17T05:26:56.847 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:26:56.848 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:26:56.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:26:56.857 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:27:01.779 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:27:01.782 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:27:05.744 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:27:05.744 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:27:05.744 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.83256 s, 122 MB/s 2024-09-17T05:27:05.745 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:27:05.746 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:05.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:27:05.754 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:27:14.529 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:27:14.529 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:27:14.529 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.72079 s, 123 MB/s 2024-09-17T05:27:14.531 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:27:14.531 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:14.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:27:14.541 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:27:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:27:23.403 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:27:23.403 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.81089 s, 122 MB/s 2024-09-17T05:27:23.404 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:27:23.404 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:23.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:27:23.413 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:27:31.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:27:31.837 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:27:32.307 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:27:32.307 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:27:32.307 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.83992 s, 121 MB/s 2024-09-17T05:27:32.308 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 group1 2024-09-17T05:27:32.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 -- 172.21.15.122:0/2054410277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb440b2cf0 msgr2=0x7feb440b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:32.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 --2- 172.21.15.122:0/2054410277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb440b2cf0 0x7feb440b1450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb34009840 tx=0x7feb3401f010 comp rx=0 tx=0).stop 2024-09-17T05:27:32.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 -- 172.21.15.122:0/2054410277 shutdown_connections 2024-09-17T05:27:32.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 --2- 172.21.15.122:0/2054410277 >> v2:172.21.15.173:3300/0 conn(0x7feb440b1990 0x7feb440b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 --2- 172.21.15.122:0/2054410277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb440b2cf0 0x7feb440b1450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 -- 172.21.15.122:0/2054410277 >> 172.21.15.122:0/2054410277 conn(0x7feb440d1dc0 msgr2=0x7feb440d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 -- 172.21.15.122:0/2054410277 shutdown_connections 2024-09-17T05:27:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 -- 172.21.15.122:0/2054410277 wait complete. 2024-09-17T05:27:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 Processor -- start 2024-09-17T05:27:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.452+0000 7feb4c800640 1 -- start start 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb440b1990 0x7feb441600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb440b2cf0 0x7feb4414be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb440cfe90 0x7feb4414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb440afb80 con 0x7feb440cfe90 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb440afa00 con 0x7feb440b2cf0 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7feb440afd00 con 0x7feb440b1990 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb440cfe90 0x7feb4414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb440cfe90 0x7feb4414a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36926/0 (socket says 172.21.15.122:36926) 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 -- 172.21.15.122:0/4164599216 learned_addr learned my addr 172.21.15.122:0/4164599216 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:32.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb49e00640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb440b1990 0x7feb441600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:32.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb49400640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb440b2cf0 0x7feb4414be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:32.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 -- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb440b1990 msgr2=0x7feb441600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:32.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb440b1990 0x7feb441600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 -- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb440b2cf0 msgr2=0x7feb4414be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:32.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb440b2cf0 0x7feb4414be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 -- 172.21.15.122:0/4164599216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb440cdb50 con 0x7feb440cfe90 2024-09-17T05:27:32.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4a800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb440cfe90 0x7feb4414a070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feb4000eb20 tx=0x7feb4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:32.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb3e000640 1 -- 172.21.15.122:0/4164599216 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40004b30 con 0x7feb440cfe90 2024-09-17T05:27:32.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb3e000640 1 -- 172.21.15.122:0/4164599216 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb40004cf0 con 0x7feb440cfe90 2024-09-17T05:27:32.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb3e000640 1 -- 172.21.15.122:0/4164599216 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40005280 con 0x7feb440cfe90 2024-09-17T05:27:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb440cde30 con 0x7feb440cfe90 2024-09-17T05:27:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb440ce3f0 con 0x7feb440cfe90 2024-09-17T05:27:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb440b1e20 con 0x7feb440cfe90 2024-09-17T05:27:32.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.456+0000 7feb3e000640 1 -- 172.21.15.122:0/4164599216 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 7) ==== 97234+0+0 (secure 0 0 0) 0x7feb40005420 con 0x7feb440cfe90 2024-09-17T05:27:32.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.460+0000 7feb3e000640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb28073200 0x7feb280756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:32.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.460+0000 7feb3e000640 1 -- 172.21.15.122:0/4164599216 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7feb40094f40 con 0x7feb440cfe90 2024-09-17T05:27:32.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.460+0000 7feb49e00640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb28073200 0x7feb280756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:32.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.460+0000 7feb49e00640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb28073200 0x7feb280756c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feb44154a70 tx=0x7feb380013a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:32.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.464+0000 7feb3e000640 1 -- 172.21.15.122:0/4164599216 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7feb4005f880 con 0x7feb440cfe90 2024-09-17T05:27:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.664+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7feb44074c70 con 0x7feb28073200 2024-09-17T05:27:32.674 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:32.673+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:32.674 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:32.673+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:32.674 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:32.673+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:32.674 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:32.673+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:32.674 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:32.673+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:32.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.692+0000 7feb3e000640 1 -- 172.21.15.122:0/4164599216 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feb44074c70 con 0x7feb28073200 2024-09-17T05:27:32.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb28073200 msgr2=0x7feb280756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:32.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb28073200 0x7feb280756c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feb44154a70 tx=0x7feb380013a0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb440cfe90 msgr2=0x7feb4414a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:32.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb440cfe90 0x7feb4414a070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feb4000eb20 tx=0x7feb4000c2d0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 shutdown_connections 2024-09-17T05:27:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb28073200 0x7feb280756c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb440cfe90 0x7feb4414a070 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb440b2cf0 0x7feb4414be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 --2- 172.21.15.122:0/4164599216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb440b1990 0x7feb441600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 >> 172.21.15.122:0/4164599216 conn(0x7feb440d1dc0 msgr2=0x7feb440721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 shutdown_connections 2024-09-17T05:27:32.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.696+0000 7feb4c800640 1 -- 172.21.15.122:0/4164599216 wait complete. 2024-09-17T05:27:32.717 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-17T05:27:32.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 -- 172.21.15.122:0/840272404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 msgr2=0x7f20f01660f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:32.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 --2- 172.21.15.122:0/840272404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 0x7f20f01660f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009bf0 tx=0x7f20e801cb50 comp rx=0 tx=0).stop 2024-09-17T05:27:32.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 -- 172.21.15.122:0/840272404 shutdown_connections 2024-09-17T05:27:32.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 --2- 172.21.15.122:0/840272404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 0x7f20f01660f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 -- 172.21.15.122:0/840272404 >> 172.21.15.122:0/840272404 conn(0x7f20f00cfbc0 msgr2=0x7f20f00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:32.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 -- 172.21.15.122:0/840272404 shutdown_connections 2024-09-17T05:27:32.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 -- 172.21.15.122:0/840272404 wait complete. 2024-09-17T05:27:32.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 Processor -- start 2024-09-17T05:27:32.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.860+0000 7f20f8000640 1 -- start start 2024-09-17T05:27:32.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f00aef60 0x7f20f00cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:32.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f015e360 0x7f20f0165130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:32.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 0x7f20f0165670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01691e0 con 0x7f20f00aef60 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0169060 con 0x7f20f015e360 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f20f0169360 con 0x7f20f0163d00 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f00aef60 0x7f20f00cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f00aef60 0x7f20f00cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36928/0 (socket says 172.21.15.122:36928) 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 -- 172.21.15.122:0/1670717617 learned_addr learned my addr 172.21.15.122:0/1670717617 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f4c00640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f015e360 0x7f20f0165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f6000640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 0x7f20f0165670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 -- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 msgr2=0x7f20f0165670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 0x7f20f0165670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 -- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f015e360 msgr2=0x7f20f0165130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f015e360 0x7f20f0165130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 -- 172.21.15.122:0/1670717617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f01f7ed0 con 0x7f20f00aef60 2024-09-17T05:27:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f6000640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 0x7f20f0165670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:27:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f5600640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f00aef60 0x7f20f00cb6a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c500 tx=0x7f20e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20ed600640 1 -- 172.21.15.122:0/1670717617 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4016020 con 0x7f20f00aef60 2024-09-17T05:27:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20ed600640 1 -- 172.21.15.122:0/1670717617 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20e4012920 con 0x7f20f00aef60 2024-09-17T05:27:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20ed600640 1 -- 172.21.15.122:0/1670717617 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4012d70 con 0x7f20f00aef60 2024-09-17T05:27:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f01f81b0 con 0x7f20f00aef60 2024-09-17T05:27:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01f8770 con 0x7f20f00aef60 2024-09-17T05:27:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.864+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f00ae710 con 0x7f20f00aef60 2024-09-17T05:27:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.868+0000 7f20ed600640 1 -- 172.21.15.122:0/1670717617 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 7) ==== 97234+0+0 (secure 0 0 0) 0x7f20e4019020 con 0x7f20f00aef60 2024-09-17T05:27:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.868+0000 7f20ed600640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f20d0073200 0x7f20d00756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.868+0000 7f20ed600640 1 -- 172.21.15.122:0/1670717617 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f20e4094250 con 0x7f20f00aef60 2024-09-17T05:27:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.868+0000 7f20f4c00640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f20d0073200 0x7f20d00756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.868+0000 7f20f4c00640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f20d0073200 0x7f20d00756c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ca120 tx=0x7f20dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:32.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:32.872+0000 7f20ed600640 1 -- 172.21.15.122:0/1670717617 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f20e4020020 con 0x7f20f00aef60 2024-09-17T05:27:33.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.064+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f20f015a240 con 0x7f20d0073200 2024-09-17T05:27:33.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.068+0000 7f20ed600640 1 -- 172.21.15.122:0/1670717617 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f20f015a240 con 0x7f20d0073200 2024-09-17T05:27:33.074 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:27:33.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f20d0073200 msgr2=0x7f20d00756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:33.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f20d0073200 0x7f20d00756c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ca120 tx=0x7f20dc009040 comp rx=0 tx=0).stop 2024-09-17T05:27:33.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f00aef60 msgr2=0x7f20f00cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:33.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f00aef60 0x7f20f00cb6a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c500 tx=0x7f20e400c9c0 comp rx=0 tx=0).stop 2024-09-17T05:27:33.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 shutdown_connections 2024-09-17T05:27:33.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f20d0073200 0x7f20d00756c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:33.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f0163d00 0x7f20f0165670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:33.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f015e360 0x7f20f0165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:33.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 --2- 172.21.15.122:0/1670717617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f00aef60 0x7f20f00cb6a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:33.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 >> 172.21.15.122:0/1670717617 conn(0x7f20f00cfbc0 msgr2=0x7f20f00ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:33.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 shutdown_connections 2024-09-17T05:27:33.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:33.072+0000 7f20f8000640 1 -- 172.21.15.122:0/1670717617 wait complete. 2024-09-17T05:27:33.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:27:33.095 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:27:33.095 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:27:33.104 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:27:33.104 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="10488365056" 2024-09-17T05:27:33.105 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:33.105 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-17T05:27:35.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:27:35.106 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:27:35.106 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:27:35.117 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:27:35.117 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="10488365056" 2024-09-17T05:27:35.118 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:35.118 INFO:tasks.cephfs.test_volumes:r_size = 10488365056 exp_size = 10737418240 2024-09-17T05:27:37.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:27:37.119 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:27:37.119 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:27:37.129 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/group1/sv1/20e0455a-2a18-42b7-ba19-cb7e70fc5039 2024-09-17T05:27:37.129 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="10737418240" 2024-09-17T05:27:37.129 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:37.130 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-17T05:27:37.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 --group-name group1 2024-09-17T05:27:37.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 -- 172.21.15.122:0/273528523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300d2760 msgr2=0x7f56300d0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:37.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 --2- 172.21.15.122:0/273528523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300d2760 0x7f56300d0060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f56300b35e0 tx=0x7f561801f010 comp rx=0 tx=0).stop 2024-09-17T05:27:37.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 -- 172.21.15.122:0/273528523 shutdown_connections 2024-09-17T05:27:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 --2- 172.21.15.122:0/273528523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56300b2570 0x7f5630160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 --2- 172.21.15.122:0/273528523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300d2760 0x7f56300d0060 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 -- 172.21.15.122:0/273528523 >> 172.21.15.122:0/273528523 conn(0x7f56300d1c40 msgr2=0x7f56300d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 -- 172.21.15.122:0/273528523 shutdown_connections 2024-09-17T05:27:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 -- 172.21.15.122:0/273528523 wait complete. 2024-09-17T05:27:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 Processor -- start 2024-09-17T05:27:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.320+0000 7f5635c00640 1 -- start start 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56300b2570 0x7f5630156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300cfc80 0x7f56300b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56300d2760 0x7f56300b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56300af780 con 0x7f56300cfc80 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f56300af600 con 0x7f56300d2760 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f56300af900 con 0x7f56300b2570 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300cfc80 0x7f56300b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300cfc80 0x7f56300b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36948/0 (socket says 172.21.15.122:36948) 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 -- 172.21.15.122:0/466679614 learned_addr learned my addr 172.21.15.122:0/466679614 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635200640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56300d2760 0x7f56300b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f562fe00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56300b2570 0x7f5630156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 -- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56300b2570 msgr2=0x7f5630156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56300b2570 0x7f5630156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 -- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56300d2760 msgr2=0x7f56300b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56300d2760 0x7f56300b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 -- 172.21.15.122:0/466679614 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56301573c0 con 0x7f56300cfc80 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635200640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56300d2760 0x7f56300b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5627400640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300cfc80 0x7f56300b6a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5618009900 tx=0x7f5618005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f562cc00640 1 -- 172.21.15.122:0/466679614 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56180b5040 con 0x7f56300cfc80 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f562cc00640 1 -- 172.21.15.122:0/466679614 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56180be070 con 0x7f56300cfc80 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f562cc00640 1 -- 172.21.15.122:0/466679614 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618003800 con 0x7f56300cfc80 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563015cbd0 con 0x7f56300cfc80 2024-09-17T05:27:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563015d190 con 0x7f56300cfc80 2024-09-17T05:27:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56300d0060 con 0x7f56300cfc80 2024-09-17T05:27:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.324+0000 7f562cc00640 1 -- 172.21.15.122:0/466679614 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 97285+0+0 (secure 0 0 0) 0x7f56180039a0 con 0x7f56300cfc80 2024-09-17T05:27:37.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.328+0000 7f562cc00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f560c0732c0 0x7f560c075780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:37.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.328+0000 7f562cc00640 1 -- 172.21.15.122:0/466679614 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f5618131d30 con 0x7f56300cfc80 2024-09-17T05:27:37.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.328+0000 7f562fe00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f560c0732c0 0x7f560c075780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:37.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.328+0000 7f562fe00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f560c0732c0 0x7f560c075780 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5620000f70 tx=0x7f5620009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:37.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.332+0000 7f562cc00640 1 -- 172.21.15.122:0/466679614 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5618018b70 con 0x7f56300cfc80 2024-09-17T05:27:37.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.528+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f56300b1cd0 con 0x7f560c0732c0 2024-09-17T05:27:37.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.604+0000 7f562cc00640 1 -- 172.21.15.122:0/466679614 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f56300b1cd0 con 0x7f560c0732c0 2024-09-17T05:27:37.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.604+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f560c0732c0 msgr2=0x7f560c075780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.604+0000 7f5635c00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f560c0732c0 0x7f560c075780 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5620000f70 tx=0x7f5620009040 comp rx=0 tx=0).stop 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.604+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300cfc80 msgr2=0x7f56300b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.604+0000 7f5635c00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300cfc80 0x7f56300b6a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5618009900 tx=0x7f5618005760 comp rx=0 tx=0).stop 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.608+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 shutdown_connections 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.608+0000 7f5635c00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f560c0732c0 0x7f560c075780 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.608+0000 7f5635c00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56300d2760 0x7f56300b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.608+0000 7f5635c00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56300cfc80 0x7f56300b6a30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.608+0000 7f5635c00640 1 --2- 172.21.15.122:0/466679614 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56300b2570 0x7f5630156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.608+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 >> 172.21.15.122:0/466679614 conn(0x7f56300d1c40 msgr2=0x7f563006f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.608+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 shutdown_connections 2024-09-17T05:27:37.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.608+0000 7f5635c00640 1 -- 172.21.15.122:0/466679614 wait complete. 2024-09-17T05:27:37.612 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.613+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.612 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.613+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.612 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.613+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.612 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.613+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.612 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.613+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.629 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.629+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.630 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.629+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.630 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.629+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.630 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.629+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.630 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:37.629+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:37.631 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:27:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 -- 172.21.15.122:0/2977105691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0070 msgr2=0x7f281c072a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 --2- 172.21.15.122:0/2977105691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0070 0x7f281c072a70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f281c0b3290 tx=0x7f280401c830 comp rx=0 tx=0).stop 2024-09-17T05:27:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 -- 172.21.15.122:0/2977105691 shutdown_connections 2024-09-17T05:27:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 --2- 172.21.15.122:0/2977105691 >> v2:172.21.15.173:3300/0 conn(0x7f281c0d0490 0x7f281c073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 --2- 172.21.15.122:0/2977105691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0070 0x7f281c072a70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 --2- 172.21.15.122:0/2977105691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f281c0cf910 0x7f281c15a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 -- 172.21.15.122:0/2977105691 >> 172.21.15.122:0/2977105691 conn(0x7f281c06cdf0 msgr2=0x7f281c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 -- 172.21.15.122:0/2977105691 shutdown_connections 2024-09-17T05:27:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.768+0000 7f2823000640 1 -- 172.21.15.122:0/2977105691 wait complete. 2024-09-17T05:27:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 Processor -- start 2024-09-17T05:27:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 -- start start 2024-09-17T05:27:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c0cf910 0x7f281c13f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f281c0d0070 0x7f281c14f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0490 0x7f281c0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f281c15ece0 con 0x7f281c0d0490 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f281c15eb60 con 0x7f281c0cf910 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f281c1670e0 con 0x7f281c0d0070 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0490 0x7f281c0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0490 0x7f281c0b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36962/0 (socket says 172.21.15.122:36962) 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 -- 172.21.15.122:0/615684840 learned_addr learned my addr 172.21.15.122:0/615684840 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f281be00640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c0cf910 0x7f281c13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2813400640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f281c0d0070 0x7f281c14f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 -- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f281c0d0070 msgr2=0x7f281c14f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f281c0d0070 0x7f281c14f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 -- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c0cf910 msgr2=0x7f281c13f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c0cf910 0x7f281c13f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 -- 172.21.15.122:0/615684840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c13fd80 con 0x7f281c0d0490 2024-09-17T05:27:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2821000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0490 0x7f281c0b1950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f281400da10 tx=0x7f281400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2818c00640 1 -- 172.21.15.122:0/615684840 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814017070 con 0x7f281c0d0490 2024-09-17T05:27:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2818c00640 1 -- 172.21.15.122:0/615684840 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2814004b30 con 0x7f281c0d0490 2024-09-17T05:27:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2818c00640 1 -- 172.21.15.122:0/615684840 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281400be60 con 0x7f281c0d0490 2024-09-17T05:27:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c150290 con 0x7f281c0d0490 2024-09-17T05:27:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c150850 con 0x7f281c0d0490 2024-09-17T05:27:37.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.772+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c072a70 con 0x7f281c0d0490 2024-09-17T05:27:37.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.776+0000 7f2818c00640 1 -- 172.21.15.122:0/615684840 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 97285+0+0 (secure 0 0 0) 0x7f2814010460 con 0x7f281c0d0490 2024-09-17T05:27:37.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.776+0000 7f2818c00640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27f8073290 0x7f27f8075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:37.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.776+0000 7f2818c00640 1 -- 172.21.15.122:0/615684840 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f28140947d0 con 0x7f281c0d0490 2024-09-17T05:27:37.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.776+0000 7f281be00640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27f8073290 0x7f27f8075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:37.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.776+0000 7f281be00640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27f8073290 0x7f27f8075750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f280c009b60 tx=0x7f280c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:37.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:37.780+0000 7f2818c00640 1 -- 172.21.15.122:0/615684840 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2814020070 con 0x7f281c0d0490 2024-09-17T05:27:38.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.008+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f281c06d3e0 con 0x7f281c0d0490 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.008+0000 7f2818c00640 1 -- 172.21.15.122:0/615684840 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1911 (secure 0 0 0) 0x7f281405efb0 con 0x7f281c0d0490 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:27:38.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 178, 2024-09-17T05:27:38.015 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 35, 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:27:38.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:27:38.017 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 2585, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 10738556262, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 21735759872, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 751358353408, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 85, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 90154104, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 0, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 22 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 17, 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:26:01:039133+0000", 2024-09-17T05:27:38.018 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2, 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 2, 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "d", 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 4873 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:38.019 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:27:38.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:38.021 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:27:38.021 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 7, 2024-09-17T05:27:38.021 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:26:54.470306+0000", 2024-09-17T05:27:38.021 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:27:38.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:38.021 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": {} 2024-09-17T05:27:38.021 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:27:38.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27f8073290 msgr2=0x7f27f8075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:38.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27f8073290 0x7f27f8075750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f280c009b60 tx=0x7f280c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:27:38.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0490 msgr2=0x7f281c0b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:38.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0490 0x7f281c0b1950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f281400da10 tx=0x7f281400ded0 comp rx=0 tx=0).stop 2024-09-17T05:27:38.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 shutdown_connections 2024-09-17T05:27:38.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f27f8073290 0x7f27f8075750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:38.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f281c0d0490 0x7f281c0b1950 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:38.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f281c0d0070 0x7f281c14f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:38.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 --2- 172.21.15.122:0/615684840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281c0cf910 0x7f281c13f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:38.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 >> 172.21.15.122:0/615684840 conn(0x7f281c06cdf0 msgr2=0x7f281c14fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:38.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 shutdown_connections 2024-09-17T05:27:38.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:38.012+0000 7f2823000640 1 -- 172.21.15.122:0/615684840 wait complete. 2024-09-17T05:27:38.616 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:38.617+0000 7fc08c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:38.617 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:38.617+0000 7fc08c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:38.617 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:38.617+0000 7fc08c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:38.617 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:38.617+0000 7fc08c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:38.617 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:38.617+0000 7fc08c400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:39.035 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:27:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.264+0000 7fd3a3200640 1 -- 172.21.15.122:0/3365934277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a40d0780 msgr2=0x7fd3a40b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.264+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3365934277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a40d0780 0x7fd3a40b1670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4073f10 tx=0x7fd39c020710 comp rx=0 tx=0).stop 2024-09-17T05:27:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.264+0000 7fd3a3200640 1 -- 172.21.15.122:0/3365934277 shutdown_connections 2024-09-17T05:27:39.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.264+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3365934277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a40d0780 0x7fd3a40b1670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:39.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.264+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3365934277 >> v2:172.21.15.173:3300/0 conn(0x7fd3a415f190 0x7fd3a40d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:39.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.264+0000 7fd3a3200640 1 -- 172.21.15.122:0/3365934277 >> 172.21.15.122:0/3365934277 conn(0x7fd3a40d1c30 msgr2=0x7fd3a40d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:39.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.264+0000 7fd3a3200640 1 -- 172.21.15.122:0/3365934277 shutdown_connections 2024-09-17T05:27:39.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.264+0000 7fd3a3200640 1 -- 172.21.15.122:0/3365934277 wait complete. 2024-09-17T05:27:39.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a3200640 1 Processor -- start 2024-09-17T05:27:39.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a3200640 1 -- start start 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3a40b1190 0x7fd3a414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3a40d0780 0x7fd3a40b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a415f190 0x7fd3a415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a4166b60 con 0x7fd3a415f190 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a3200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a41669e0 con 0x7fd3a40d0780 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a3200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a4166ce0 con 0x7fd3a40b1190 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a415f190 0x7fd3a415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a415f190 0x7fd3a415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36972/0 (socket says 172.21.15.122:36972) 2024-09-17T05:27:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.268+0000 7fd3a2800640 1 -- 172.21.15.122:0/3276655011 learned_addr learned my addr 172.21.15.122:0/3276655011 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a2800640 1 -- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3a40b1190 msgr2=0x7fd3a414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a2800640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3a40b1190 0x7fd3a414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a2800640 1 -- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3a40d0780 msgr2=0x7fd3a40b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a2800640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3a40d0780 0x7fd3a40b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a2800640 1 -- 172.21.15.122:0/3276655011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a415adc0 con 0x7fd3a415f190 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a2800640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a415f190 0x7fd3a415be20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00dc60 tx=0x7fd39c00d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd39a000640 1 -- 172.21.15.122:0/3276655011 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0b0040 con 0x7fd3a415f190 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd39a000640 1 -- 172.21.15.122:0/3276655011 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd39c0046d0 con 0x7fd3a415f190 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a415b060 con 0x7fd3a415f190 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd39a000640 1 -- 172.21.15.122:0/3276655011 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c004b20 con 0x7fd3a415f190 2024-09-17T05:27:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a415b620 con 0x7fd3a415f190 2024-09-17T05:27:39.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.272+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a40b1670 con 0x7fd3a415f190 2024-09-17T05:27:39.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.276+0000 7fd39a000640 1 -- 172.21.15.122:0/3276655011 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 9) ==== 97387+0+0 (secure 0 0 0) 0x7fd39c0b3070 con 0x7fd3a415f190 2024-09-17T05:27:39.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.276+0000 7fd39a000640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3800733b0 0x7fd380075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:39.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.276+0000 7fd39a000640 1 -- 172.21.15.122:0/3276655011 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fd39c132a80 con 0x7fd3a415f190 2024-09-17T05:27:39.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.280+0000 7fd3a1e00640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3800733b0 0x7fd380075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:39.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.284+0000 7fd39a000640 1 -- 172.21.15.122:0/3276655011 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd39c0bb070 con 0x7fd3a415f190 2024-09-17T05:27:39.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.288+0000 7fd3a1e00640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3800733b0 0x7fd380075870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd39400ab90 tx=0x7fd394009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:39.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.644+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd3a40cde10 con 0x7fd3a415f190 2024-09-17T05:27:39.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.648+0000 7fd39a000640 1 -- 172.21.15.122:0/3276655011 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1911 (secure 0 0 0) 0x7fd39c01a420 con 0x7fd3a415f190 2024-09-17T05:27:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:39.654 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:27:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:27:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:27:39.654 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:27:39.655 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 180, 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 35, 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:27:39.656 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:27:39.657 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 2585, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 10738556262, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 21735759872, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 751358353408, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 85, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 90156021, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 0, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 22 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 17, 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:26:01:039133+0000", 2024-09-17T05:27:39.658 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2, 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 2, 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "d", 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 4873 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:39.659 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 7, 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:26:54.470306+0000", 2024-09-17T05:27:39.660 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:27:39.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:39.661 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": {} 2024-09-17T05:27:39.661 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3800733b0 msgr2=0x7fd380075870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3800733b0 0x7fd380075870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd39400ab90 tx=0x7fd394009250 comp rx=0 tx=0).stop 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a415f190 msgr2=0x7fd3a415be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a415f190 0x7fd3a415be20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00dc60 tx=0x7fd39c00d370 comp rx=0 tx=0).stop 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 shutdown_connections 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3800733b0 0x7fd380075870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3a415f190 0x7fd3a415be20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3a40d0780 0x7fd3a40b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 --2- 172.21.15.122:0/3276655011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3a40b1190 0x7fd3a414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:39.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 >> 172.21.15.122:0/3276655011 conn(0x7fd3a40d1c30 msgr2=0x7fd3a40d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:39.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 shutdown_connections 2024-09-17T05:27:39.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:39.652+0000 7fd3a3200640 1 -- 172.21.15.122:0/3276655011 wait complete. 2024-09-17T05:27:40.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:27:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.864+0000 7f7550000640 1 -- 172.21.15.122:0/3552100227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480b2760 msgr2=0x7f75481605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.864+0000 7f7550000640 1 --2- 172.21.15.122:0/3552100227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480b2760 0x7f75481605d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7540009bc0 tx=0x7f754001cb10 comp rx=0 tx=0).stop 2024-09-17T05:27:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 -- 172.21.15.122:0/3552100227 shutdown_connections 2024-09-17T05:27:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 --2- 172.21.15.122:0/3552100227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480b2760 0x7f75481605d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 --2- 172.21.15.122:0/3552100227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75480cfe80 0x7f75480d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 -- 172.21.15.122:0/3552100227 >> 172.21.15.122:0/3552100227 conn(0x7f75480d1e40 msgr2=0x7f75480d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 -- 172.21.15.122:0/3552100227 shutdown_connections 2024-09-17T05:27:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 -- 172.21.15.122:0/3552100227 wait complete. 2024-09-17T05:27:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 Processor -- start 2024-09-17T05:27:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 -- start start 2024-09-17T05:27:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75480b2760 0x7f754815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75480b2d00 0x7f754815af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480cfe80 0x7f75480b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75480b0940 con 0x7f75480cfe80 2024-09-17T05:27:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75480b07c0 con 0x7f75480b2d00 2024-09-17T05:27:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f7550000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f75480b0ac0 con 0x7f75480b2760 2024-09-17T05:27:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f754e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480cfe80 0x7f75480b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f754e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480cfe80 0x7f75480b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36976/0 (socket says 172.21.15.122:36976) 2024-09-17T05:27:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f754e000640 1 -- 172.21.15.122:0/1617754750 learned_addr learned my addr 172.21.15.122:0/1617754750 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f754e000640 1 -- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75480b2760 msgr2=0x7f754815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f754e000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75480b2760 0x7f754815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f754e000640 1 -- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75480b2d00 msgr2=0x7f754815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f754e000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75480b2d00 0x7f754815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.868+0000 7f754e000640 1 -- 172.21.15.122:0/1617754750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75480cafd0 con 0x7f75480cfe80 2024-09-17T05:27:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.872+0000 7f754e000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480cfe80 0x7f75480b5cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f754001a740 tx=0x7f75400a5960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.872+0000 7f7545600640 1 -- 172.21.15.122:0/1617754750 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75400bb070 con 0x7f75480cfe80 2024-09-17T05:27:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.872+0000 7f7545600640 1 -- 172.21.15.122:0/1617754750 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7540004e00 con 0x7f75480cfe80 2024-09-17T05:27:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.872+0000 7f7545600640 1 -- 172.21.15.122:0/1617754750 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75400b2040 con 0x7f75480cfe80 2024-09-17T05:27:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.872+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75480cb2b0 con 0x7f75480cfe80 2024-09-17T05:27:40.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.872+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75480cb870 con 0x7f75480cfe80 2024-09-17T05:27:40.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.872+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75480d0260 con 0x7f75480cfe80 2024-09-17T05:27:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.872+0000 7f7545600640 1 -- 172.21.15.122:0/1617754750 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f7540004380 con 0x7f75480cfe80 2024-09-17T05:27:40.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.876+0000 7f7545600640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7528073440 0x7f7528075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:40.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.876+0000 7f7545600640 1 -- 172.21.15.122:0/1617754750 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f754012e7c0 con 0x7f75480cfe80 2024-09-17T05:27:40.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.880+0000 7f754d600640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7528073440 0x7f7528075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:40.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.880+0000 7f7545600640 1 -- 172.21.15.122:0/1617754750 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7540016790 con 0x7f75480cfe80 2024-09-17T05:27:40.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:40.880+0000 7f754d600640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7528073440 0x7f7528075900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f753c009690 tx=0x7f753c006c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:41.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.272+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f75480cc3b0 con 0x7f75480cfe80 2024-09-17T05:27:41.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.272+0000 7f7545600640 1 -- 172.21.15.122:0/1617754750 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2166 (secure 0 0 0) 0x7f75400f9370 con 0x7f75480cfe80 2024-09-17T05:27:41.275 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:27:41.276 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 182, 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:27:41.277 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 35, 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:41.278 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 2607, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 10818345892, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 21773705216, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 751320408064, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 9435084, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 96106197, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 2, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 26 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:41.279 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 17, 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:26:01:039133+0000", 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 2, 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 2, 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "d", 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:27:41.280 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 4873 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:27:41.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 7, 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:26:54.470306+0000", 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "1 ongoing clones - average progress is 3.125% (1s)\n [............................] (remaining: 34s)", 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.03125, 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:41.282 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:27:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7528073440 msgr2=0x7f7528075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7528073440 0x7f7528075900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f753c009690 tx=0x7f753c006c00 comp rx=0 tx=0).stop 2024-09-17T05:27:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480cfe80 msgr2=0x7f75480b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480cfe80 0x7f75480b5cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f754001a740 tx=0x7f75400a5960 comp rx=0 tx=0).stop 2024-09-17T05:27:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 shutdown_connections 2024-09-17T05:27:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7528073440 0x7f7528075900 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75480cfe80 0x7f75480b5cc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75480b2d00 0x7f754815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 --2- 172.21.15.122:0/1617754750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75480b2760 0x7f754815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 >> 172.21.15.122:0/1617754750 conn(0x7f75480d1e40 msgr2=0x7f75480aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 shutdown_connections 2024-09-17T05:27:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.276+0000 7f7550000640 1 -- 172.21.15.122:0/1617754750 wait complete. 2024-09-17T05:27:41.306 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-17T05:27:41.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.536+0000 7f6340800640 1 -- 172.21.15.122:0/3783932897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63380b4cf0 msgr2=0x7f63380b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:41.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.536+0000 7f6340800640 1 --2- 172.21.15.122:0/3783932897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63380b4cf0 0x7f63380b3130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6334009c80 tx=0x7f633401cd10 comp rx=0 tx=0).stop 2024-09-17T05:27:41.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.536+0000 7f6340800640 1 -- 172.21.15.122:0/3783932897 shutdown_connections 2024-09-17T05:27:41.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.536+0000 7f6340800640 1 --2- 172.21.15.122:0/3783932897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63380b4cf0 0x7f63380b3130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:41.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.536+0000 7f6340800640 1 --2- 172.21.15.122:0/3783932897 >> v2:172.21.15.173:3300/0 conn(0x7f633815e260 0x7f63380b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:41.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.536+0000 7f6340800640 1 -- 172.21.15.122:0/3783932897 >> 172.21.15.122:0/3783932897 conn(0x7f63380b18f0 msgr2=0x7f63380b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:41.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.540+0000 7f6340800640 1 -- 172.21.15.122:0/3783932897 shutdown_connections 2024-09-17T05:27:41.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.540+0000 7f6340800640 1 -- 172.21.15.122:0/3783932897 wait complete. 2024-09-17T05:27:41.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.540+0000 7f6340800640 1 Processor -- start 2024-09-17T05:27:41.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.540+0000 7f6340800640 1 -- start start 2024-09-17T05:27:41.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63380b4cf0 0x7f63380ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:41.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633815e260 0x7f63381549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:41.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633809c3a0 0x7f633809c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:41.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63381663a0 con 0x7f633809c3a0 2024-09-17T05:27:41.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6338166220 con 0x7f633815e260 2024-09-17T05:27:41.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6338166520 con 0x7f63380b4cf0 2024-09-17T05:27:41.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633809c3a0 0x7f633809c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:41.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633809c3a0 0x7f633809c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59382/0 (socket says 172.21.15.122:59382) 2024-09-17T05:27:41.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 -- 172.21.15.122:0/3773618193 learned_addr learned my addr 172.21.15.122:0/3773618193 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:41.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633d400640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633815e260 0x7f63381549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:41.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633de00640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63380b4cf0 0x7f63380ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:41.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 -- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63380b4cf0 msgr2=0x7f63380ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:41.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63380b4cf0 0x7f63380ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:41.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 -- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633815e260 msgr2=0x7f63381549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:41.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633815e260 0x7f63381549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:41.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 -- 172.21.15.122:0/3773618193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6338155940 con 0x7f633809c3a0 2024-09-17T05:27:41.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f633e800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633809c3a0 0x7f633809c850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6334009bd0 tx=0x7f6334004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:41.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6332000640 1 -- 172.21.15.122:0/3773618193 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340bb070 con 0x7f633809c3a0 2024-09-17T05:27:41.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633809cf10 con 0x7f633809c3a0 2024-09-17T05:27:41.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6332000640 1 -- 172.21.15.122:0/3773618193 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6334004b40 con 0x7f633809c3a0 2024-09-17T05:27:41.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6332000640 1 -- 172.21.15.122:0/3773618193 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340b2040 con 0x7f633809c3a0 2024-09-17T05:27:41.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63380acd40 con 0x7f633809c3a0 2024-09-17T05:27:41.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.544+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6338069410 con 0x7f633809c3a0 2024-09-17T05:27:41.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.548+0000 7f6332000640 1 -- 172.21.15.122:0/3773618193 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f63340be070 con 0x7f633809c3a0 2024-09-17T05:27:41.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.548+0000 7f6332000640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f631c073470 0x7f631c075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:41.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.548+0000 7f6332000640 1 -- 172.21.15.122:0/3773618193 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f633412e780 con 0x7f633809c3a0 2024-09-17T05:27:41.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.552+0000 7f6332000640 1 -- 172.21.15.122:0/3773618193 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f63340f3120 con 0x7f633809c3a0 2024-09-17T05:27:41.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.556+0000 7f633de00640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f631c073470 0x7f631c075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:41.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.560+0000 7f633de00640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f631c073470 0x7f631c075930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f632c00bb50 tx=0x7f632c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:41.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:41.904+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f6338154f20 con 0x7f631c073470 2024-09-17T05:27:42.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.104+0000 7f6332000640 1 -- 172.21.15.122:0/3773618193 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6338154f20 con 0x7f631c073470 2024-09-17T05:27:42.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f631c073470 msgr2=0x7f631c075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f631c073470 0x7f631c075930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f632c00bb50 tx=0x7f632c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633809c3a0 msgr2=0x7f633809c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633809c3a0 0x7f633809c850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6334009bd0 tx=0x7f6334004270 comp rx=0 tx=0).stop 2024-09-17T05:27:42.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 shutdown_connections 2024-09-17T05:27:42.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f631c073470 0x7f631c075930 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633809c3a0 0x7f633809c850 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f633815e260 0x7f63381549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 --2- 172.21.15.122:0/3773618193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63380b4cf0 0x7f63380ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 >> 172.21.15.122:0/3773618193 conn(0x7f63380b18f0 msgr2=0x7f63380ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:42.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 shutdown_connections 2024-09-17T05:27:42.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.108+0000 7f6340800640 1 -- 172.21.15.122:0/3773618193 wait complete. 2024-09-17T05:27:42.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:27:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- 172.21.15.122:0/1193190278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde40b2d00 msgr2=0x7fbde40b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 --2- 172.21.15.122:0/1193190278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde40b2d00 0x7fbde40b0870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009bf0 tx=0x7fbddc01ca90 comp rx=0 tx=0).stop 2024-09-17T05:27:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- 172.21.15.122:0/1193190278 shutdown_connections 2024-09-17T05:27:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 --2- 172.21.15.122:0/1193190278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde40b2d00 0x7fbde40b0870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- 172.21.15.122:0/1193190278 >> 172.21.15.122:0/1193190278 conn(0x7fbde40d1e40 msgr2=0x7fbde40d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- 172.21.15.122:0/1193190278 shutdown_connections 2024-09-17T05:27:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- 172.21.15.122:0/1193190278 wait complete. 2024-09-17T05:27:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 Processor -- start 2024-09-17T05:27:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- start start 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde40b2760 0x7fbde4155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde40b2d00 0x7fbde41fb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde40cfe80 0x7fbde4154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbde416a1b0 con 0x7fbde40cfe80 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbde416a030 con 0x7fbde40b2760 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde8a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbde416a330 con 0x7fbde40b2d00 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde40cfe80 0x7fbde4154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde40cfe80 0x7fbde4154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59390/0 (socket says 172.21.15.122:59390) 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde2800640 1 -- 172.21.15.122:0/341308199 learned_addr learned my addr 172.21.15.122:0/341308199 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbde1e00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde40b2760 0x7fbde4155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.272+0000 7fbdd9400640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde40b2d00 0x7fbde41fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde2800640 1 -- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde40b2d00 msgr2=0x7fbde41fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde2800640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde40b2d00 0x7fbde41fb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde2800640 1 -- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde40b2760 msgr2=0x7fbde4155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde2800640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde40b2760 0x7fbde4155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde2800640 1 -- 172.21.15.122:0/341308199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde41f7aa0 con 0x7fbde40cfe80 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde2800640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde40cfe80 0x7fbde4154c80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbddc01a6c0 tx=0x7fbddc009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbddaa00640 1 -- 172.21.15.122:0/341308199 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0bb070 con 0x7fbde40cfe80 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbddaa00640 1 -- 172.21.15.122:0/341308199 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbddc004d40 con 0x7fbde40cfe80 2024-09-17T05:27:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbddaa00640 1 -- 172.21.15.122:0/341308199 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0b2040 con 0x7fbde40cfe80 2024-09-17T05:27:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde41f7d80 con 0x7fbde40cfe80 2024-09-17T05:27:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde41f8340 con 0x7fbde40cfe80 2024-09-17T05:27:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde4160680 con 0x7fbde40cfe80 2024-09-17T05:27:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbddaa00640 1 -- 172.21.15.122:0/341308199 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7fbddc0a5a40 con 0x7fbde40cfe80 2024-09-17T05:27:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbddaa00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0073440 0x7fbdc0075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.276+0000 7fbddaa00640 1 -- 172.21.15.122:0/341308199 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fbddc12e560 con 0x7fbde40cfe80 2024-09-17T05:27:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.280+0000 7fbde1e00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0073440 0x7fbdc0075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.280+0000 7fbde1e00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0073440 0x7fbdc0075900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4007f80 tx=0x7fbdd4007f10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:42.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.280+0000 7fbddaa00640 1 -- 172.21.15.122:0/341308199 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbddc0167d0 con 0x7fbde40cfe80 2024-09-17T05:27:42.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.476+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbde41f8840 con 0x7fbdc0073440 2024-09-17T05:27:42.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.476+0000 7fbddaa00640 1 -- 172.21.15.122:0/341308199 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbdd400f880 con 0x7fbdc0073440 2024-09-17T05:27:42.482 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:27:42.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:27:42.482 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:27:42.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:42.482 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:27:42.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0073440 msgr2=0x7fbdc0075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0073440 0x7fbdc0075900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4007f80 tx=0x7fbdd4007f10 comp rx=0 tx=0).stop 2024-09-17T05:27:42.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde40cfe80 msgr2=0x7fbde4154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde40cfe80 0x7fbde4154c80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbddc01a6c0 tx=0x7fbddc009200 comp rx=0 tx=0).stop 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 shutdown_connections 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0073440 0x7fbdc0075900 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde40cfe80 0x7fbde4154c80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde40b2d00 0x7fbde41fb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 --2- 172.21.15.122:0/341308199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde40b2760 0x7fbde4155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 >> 172.21.15.122:0/341308199 conn(0x7fbde40d1e40 msgr2=0x7fbde4071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 shutdown_connections 2024-09-17T05:27:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.480+0000 7fbde8a00640 1 -- 172.21.15.122:0/341308199 wait complete. 2024-09-17T05:27:42.503 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:27:42.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- 172.21.15.122:0/298596677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a01603f0 msgr2=0x7fb1a00b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 --2- 172.21.15.122:0/298596677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a01603f0 0x7fb1a00b2390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb190009840 tx=0x7fb19001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- 172.21.15.122:0/298596677 shutdown_connections 2024-09-17T05:27:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 --2- 172.21.15.122:0/298596677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1a00b28d0 0x7fb1a00ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 --2- 172.21.15.122:0/298596677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a01603f0 0x7fb1a00b2390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- 172.21.15.122:0/298596677 >> 172.21.15.122:0/298596677 conn(0x7fb1a00d1df0 msgr2=0x7fb1a00d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- 172.21.15.122:0/298596677 shutdown_connections 2024-09-17T05:27:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- 172.21.15.122:0/298596677 wait complete. 2024-09-17T05:27:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 Processor -- start 2024-09-17T05:27:42.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- start start 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1a00b1190 0x7fb1a0159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a00b28d0 0x7fb1a01fc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1a01603f0 0x7fb1a015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01694f0 con 0x7fb1a00b28d0 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00b34a0 con 0x7fb1a00b1190 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb1a5400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a0169670 con 0x7fb1a01603f0 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb195e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a00b28d0 0x7fb1a01fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb195e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a00b28d0 0x7fb1a01fc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59412/0 (socket says 172.21.15.122:59412) 2024-09-17T05:27:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb195e00640 1 -- 172.21.15.122:0/1712230243 learned_addr learned my addr 172.21.15.122:0/1712230243 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb19f200640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1a01603f0 0x7fb1a015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.644+0000 7fb19e800640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1a00b1190 0x7fb1a0159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb195e00640 1 -- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1a01603f0 msgr2=0x7fb1a015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb195e00640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1a01603f0 0x7fb1a015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb195e00640 1 -- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1a00b1190 msgr2=0x7fb1a0159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb195e00640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1a00b1190 0x7fb1a0159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb195e00640 1 -- 172.21.15.122:0/1712230243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a01fc5a0 con 0x7fb1a00b28d0 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb195e00640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a00b28d0 0x7fb1a01fc060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb18800e9c0 tx=0x7fb18800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb197400640 1 -- 172.21.15.122:0/1712230243 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18800cd80 con 0x7fb1a00b28d0 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb197400640 1 -- 172.21.15.122:0/1712230243 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb188004020 con 0x7fb1a00b28d0 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a00b5360 con 0x7fb1a00b28d0 2024-09-17T05:27:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb197400640 1 -- 172.21.15.122:0/1712230243 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188004450 con 0x7fb1a00b28d0 2024-09-17T05:27:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a00b5920 con 0x7fb1a00b28d0 2024-09-17T05:27:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a00ce270 con 0x7fb1a00b28d0 2024-09-17T05:27:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb197400640 1 -- 172.21.15.122:0/1712230243 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7fb188003600 con 0x7fb1a00b28d0 2024-09-17T05:27:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb197400640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb17c073440 0x7fb17c075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb197400640 1 -- 172.21.15.122:0/1712230243 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fb188095170 con 0x7fb1a00b28d0 2024-09-17T05:27:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.648+0000 7fb19e800640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb17c073440 0x7fb17c075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.652+0000 7fb19e800640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb17c073440 0x7fb17c075900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1900af000 tx=0x7fb19001b030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:42.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.652+0000 7fb197400640 1 -- 172.21.15.122:0/1712230243 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb1a01fc5a0 con 0x7fb1a00b28d0 2024-09-17T05:27:42.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.848+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1a00b7ba0 con 0x7fb17c073440 2024-09-17T05:27:42.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb197400640 1 -- 172.21.15.122:0/1712230243 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7fb1a00b7ba0 con 0x7fb17c073440 2024-09-17T05:27:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:42.855 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:27:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:27:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1clone1" 2024-09-17T05:27:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:27:42.856 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb17c073440 msgr2=0x7fb17c075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb17c073440 0x7fb17c075900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1900af000 tx=0x7fb19001b030 comp rx=0 tx=0).stop 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a00b28d0 msgr2=0x7fb1a01fc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a00b28d0 0x7fb1a01fc060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb18800e9c0 tx=0x7fb18800ee80 comp rx=0 tx=0).stop 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 shutdown_connections 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb17c073440 0x7fb17c075900 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1a01603f0 0x7fb1a015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1a00b28d0 0x7fb1a01fc060 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 --2- 172.21.15.122:0/1712230243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1a00b1190 0x7fb1a0159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 >> 172.21.15.122:0/1712230243 conn(0x7fb1a00d1df0 msgr2=0x7fb1a0071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:42.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 shutdown_connections 2024-09-17T05:27:42.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:42.852+0000 7fb1a5400640 1 -- 172.21.15.122:0/1712230243 wait complete. 2024-09-17T05:27:42.877 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-17T05:27:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.016+0000 7fbdb1800640 1 -- 172.21.15.122:0/2949494544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 msgr2=0x7fbdac0733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.016+0000 7fbdb1800640 1 --2- 172.21.15.122:0/2949494544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 0x7fbdac0733a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbdac0adc70 tx=0x7fbd9c01caf0 comp rx=0 tx=0).stop 2024-09-17T05:27:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.016+0000 7fbdb1800640 1 -- 172.21.15.122:0/2949494544 shutdown_connections 2024-09-17T05:27:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.016+0000 7fbdb1800640 1 --2- 172.21.15.122:0/2949494544 >> v2:172.21.15.173:3300/0 conn(0x7fbdac0cdac0 0x7fbdac0cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.016+0000 7fbdb1800640 1 --2- 172.21.15.122:0/2949494544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 0x7fbdac0733a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.016+0000 7fbdb1800640 1 -- 172.21.15.122:0/2949494544 >> 172.21.15.122:0/2949494544 conn(0x7fbdac0cfbc0 msgr2=0x7fbdac0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.016+0000 7fbdb1800640 1 -- 172.21.15.122:0/2949494544 shutdown_connections 2024-09-17T05:27:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.016+0000 7fbdb1800640 1 -- 172.21.15.122:0/2949494544 wait complete. 2024-09-17T05:27:43.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 Processor -- start 2024-09-17T05:27:43.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 -- start start 2024-09-17T05:27:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdac0b5170 0x7fbdac149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdac0cdac0 0x7fbdac15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 0x7fbdac15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac169020 con 0x7fbdac15bf60 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac168ea0 con 0x7fbdac0b5170 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbdac1691a0 con 0x7fbdac0cdac0 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 0x7fbdac15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 0x7fbdac15aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59432/0 (socket says 172.21.15.122:59432) 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 -- 172.21.15.122:0/1927436966 learned_addr learned my addr 172.21.15.122:0/1927436966 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdaaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdac0b5170 0x7fbdac149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbda2000640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdac0cdac0 0x7fbdac15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 -- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdac0cdac0 msgr2=0x7fbdac15c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdac0cdac0 0x7fbdac15c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 -- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdac0b5170 msgr2=0x7fbdac149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdac0b5170 0x7fbdac149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 -- 172.21.15.122:0/1927436966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdac152630 con 0x7fbdac15bf60 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdab400640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 0x7fbdac15aff0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbda400d780 tx=0x7fbda400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbda3400640 1 -- 172.21.15.122:0/1927436966 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4017070 con 0x7fbdac15bf60 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbda3400640 1 -- 172.21.15.122:0/1927436966 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbda400de90 con 0x7fbdac15bf60 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdac152910 con 0x7fbdac15bf60 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbda3400640 1 -- 172.21.15.122:0/1927436966 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4005020 con 0x7fbdac15bf60 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.020+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdac15b530 con 0x7fbdac15bf60 2024-09-17T05:27:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.024+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdac069410 con 0x7fbdac15bf60 2024-09-17T05:27:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.024+0000 7fbda3400640 1 -- 172.21.15.122:0/1927436966 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7fbda40051c0 con 0x7fbdac15bf60 2024-09-17T05:27:43.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.024+0000 7fbda3400640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbd88073470 0x7fbd88075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:43.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.024+0000 7fbda3400640 1 -- 172.21.15.122:0/1927436966 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fbda4094880 con 0x7fbdac15bf60 2024-09-17T05:27:43.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.024+0000 7fbdaaa00640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbd88073470 0x7fbd88075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.028+0000 7fbdaaa00640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbd88073470 0x7fbd88075930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0af000 tx=0x7fbd9c01b330 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.028+0000 7fbda3400640 1 -- 172.21.15.122:0/1927436966 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbda405ed50 con 0x7fbdac15bf60 2024-09-17T05:27:43.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:43.220+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fbdac158430 con 0x7fbd88073470 2024-09-17T05:27:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.708+0000 7fbda3400640 1 -- 172.21.15.122:0/1927436966 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbdac158430 con 0x7fbd88073470 2024-09-17T05:27:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbd88073470 msgr2=0x7fbd88075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbd88073470 0x7fbd88075930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0af000 tx=0x7fbd9c01b330 comp rx=0 tx=0).stop 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 msgr2=0x7fbdac15aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 0x7fbdac15aff0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbda400d780 tx=0x7fbda400dc40 comp rx=0 tx=0).stop 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 shutdown_connections 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbd88073470 0x7fbd88075930 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdac15bf60 0x7fbdac15aff0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdac0cdac0 0x7fbdac15c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 --2- 172.21.15.122:0/1927436966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdac0b5170 0x7fbdac149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 >> 172.21.15.122:0/1927436966 conn(0x7fbdac0cfbc0 msgr2=0x7fbdac165dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 shutdown_connections 2024-09-17T05:27:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.712+0000 7fbdb1800640 1 -- 172.21.15.122:0/1927436966 wait complete. 2024-09-17T05:27:46.717 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.717+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.717 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.717+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.717 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.717+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.717 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.717+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.717 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.717+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.735 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.733+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.735 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.733+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.735 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.733+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.735 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.733+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.735 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:27:46.733+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:27:46.738 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:27:46.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:27:46.795 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:27:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:27:46.802 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:27:46.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.944+0000 7f7201600640 1 -- 172.21.15.122:0/684838206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc15bd40 msgr2=0x7f71fc0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:46.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.944+0000 7f7201600640 1 --2- 172.21.15.122:0/684838206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc15bd40 0x7f71fc0d0240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009900 tx=0x7f71ec01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:27:46.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.944+0000 7f7201600640 1 -- 172.21.15.122:0/684838206 shutdown_connections 2024-09-17T05:27:46.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.944+0000 7f7201600640 1 --2- 172.21.15.122:0/684838206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc15bd40 0x7f71fc0d0240 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:46.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.944+0000 7f7201600640 1 -- 172.21.15.122:0/684838206 >> 172.21.15.122:0/684838206 conn(0x7f71fc09aa10 msgr2=0x7f71fc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:46.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.944+0000 7f7201600640 1 -- 172.21.15.122:0/684838206 shutdown_connections 2024-09-17T05:27:46.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.944+0000 7f7201600640 1 -- 172.21.15.122:0/684838206 wait complete. 2024-09-17T05:27:46.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 Processor -- start 2024-09-17T05:27:46.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 -- start start 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71fc0b0d10 0x7f71fc166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc0d22c0 0x7f71fc1fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71fc15bd40 0x7f71fc1fad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc169ee0 con 0x7f71fc0d22c0 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc169d60 con 0x7f71fc15bd40 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f71fc16a060 con 0x7f71fc0b0d10 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc0d22c0 0x7f71fc1fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc0d22c0 0x7f71fc1fa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59456/0 (socket says 172.21.15.122:59456) 2024-09-17T05:27:46.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 -- 172.21.15.122:0/1541085869 learned_addr learned my addr 172.21.15.122:0/1541085869 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7200c00640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71fc15bd40 0x7f71fc1fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71fbe00640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71fc0b0d10 0x7f71fc166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 -- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71fc0b0d10 msgr2=0x7f71fc166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71fc0b0d10 0x7f71fc166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 -- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71fc15bd40 msgr2=0x7f71fc1fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71fc15bd40 0x7f71fc1fad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 -- 172.21.15.122:0/1541085869 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc1fb4a0 con 0x7f71fc0d22c0 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7200c00640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71fc15bd40 0x7f71fc1fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:46.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f3400640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc0d22c0 0x7f71fc1fa7d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71e400c500 tx=0x7f71e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:46.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f8c00640 1 -- 172.21.15.122:0/1541085869 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4018070 con 0x7f71fc0d22c0 2024-09-17T05:27:46.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f8c00640 1 -- 172.21.15.122:0/1541085869 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e4003600 con 0x7f71fc0d22c0 2024-09-17T05:27:46.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f71f8c00640 1 -- 172.21.15.122:0/1541085869 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4003ad0 con 0x7f71fc0d22c0 2024-09-17T05:27:46.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc15eeb0 con 0x7f71fc0d22c0 2024-09-17T05:27:46.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc06ea60 con 0x7f71fc0d22c0 2024-09-17T05:27:46.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.948+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71fc0c8d90 con 0x7f71fc0d22c0 2024-09-17T05:27:46.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.952+0000 7f71f8c00640 1 -- 172.21.15.122:0/1541085869 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f71e4003c70 con 0x7f71fc0d22c0 2024-09-17T05:27:46.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.952+0000 7f71f8c00640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71d8073440 0x7f71d8075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:46.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.952+0000 7f71f8c00640 1 -- 172.21.15.122:0/1541085869 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f71e4093b80 con 0x7f71fc0d22c0 2024-09-17T05:27:46.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.952+0000 7f71fbe00640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71d8073440 0x7f71d8075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:46.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.952+0000 7f71fbe00640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71d8073440 0x7f71d8075900 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0af000 tx=0x7f71ec01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:46.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:46.956+0000 7f71f8c00640 1 -- 172.21.15.122:0/1541085869 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f71e405e120 con 0x7f71fc0d22c0 2024-09-17T05:27:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.128+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f71fc076160 con 0x7f71fc0d22c0 2024-09-17T05:27:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.128+0000 7f71f8c00640 1 -- 172.21.15.122:0/1541085869 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7f71e407f0c0 con 0x7f71fc0d22c0 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71d8073440 msgr2=0x7f71d8075900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71d8073440 0x7f71d8075900 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0af000 tx=0x7f71ec01b120 comp rx=0 tx=0).stop 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc0d22c0 msgr2=0x7f71fc1fa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc0d22c0 0x7f71fc1fa7d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71e400c500 tx=0x7f71e400c9c0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 shutdown_connections 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71d8073440 0x7f71d8075900 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71fc15bd40 0x7f71fc1fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71fc0d22c0 0x7f71fc1fa7d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 --2- 172.21.15.122:0/1541085869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71fc0b0d10 0x7f71fc166fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 >> 172.21.15.122:0/1541085869 conn(0x7f71fc09aa10 msgr2=0x7f71fc0aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 shutdown_connections 2024-09-17T05:27:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.132+0000 7f7201600640 1 -- 172.21.15.122:0/1541085869 wait complete. 2024-09-17T05:27:47.154 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:27:47.154 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:27:47.253 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:27:47.354 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:27:47.354 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:27:47.354 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:27:47.355 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:47.363 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:27:47.364 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:47.414 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:27:47.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.596+0000 7f75a3e00640 1 -- 172.21.15.122:0/2411532086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75a40cefb0 msgr2=0x7f75a40cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:47.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.596+0000 7f75a3e00640 1 --2- 172.21.15.122:0/2411532086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75a40cefb0 0x7f75a40cc5b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7590009900 tx=0x7f759001c850 comp rx=0 tx=0).stop 2024-09-17T05:27:47.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.596+0000 7f75a3e00640 1 -- 172.21.15.122:0/2411532086 shutdown_connections 2024-09-17T05:27:47.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.596+0000 7f75a3e00640 1 --2- 172.21.15.122:0/2411532086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75a40cefb0 0x7f75a40cc5b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.596+0000 7f75a3e00640 1 --2- 172.21.15.122:0/2411532086 >> v2:172.21.15.173:3300/0 conn(0x7f75a415f000 0x7f75a40d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.596+0000 7f75a3e00640 1 -- 172.21.15.122:0/2411532086 >> 172.21.15.122:0/2411532086 conn(0x7f75a409aa10 msgr2=0x7f75a409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:47.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.596+0000 7f75a3e00640 1 -- 172.21.15.122:0/2411532086 shutdown_connections 2024-09-17T05:27:47.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.596+0000 7f75a3e00640 1 -- 172.21.15.122:0/2411532086 wait complete. 2024-09-17T05:27:47.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 Processor -- start 2024-09-17T05:27:47.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 -- start start 2024-09-17T05:27:47.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75a40ccaf0 0x7f75a415aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:47.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75a40cefb0 0x7f75a40ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:47.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75a415f000 0x7f75a406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:47.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75a41692c0 con 0x7f75a40ccaf0 2024-09-17T05:27:47.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75a4169140 con 0x7f75a40cefb0 2024-09-17T05:27:47.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f75a4169440 con 0x7f75a415f000 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75a40ccaf0 0x7f75a415aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75a40ccaf0 0x7f75a415aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59468/0 (socket says 172.21.15.122:59468) 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 -- 172.21.15.122:0/3203209684 learned_addr learned my addr 172.21.15.122:0/3203209684 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3400640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75a415f000 0x7f75a406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2000640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75a40cefb0 0x7f75a40ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 -- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75a415f000 msgr2=0x7f75a406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75a415f000 0x7f75a406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 -- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75a40cefb0 msgr2=0x7f75a40ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75a40cefb0 0x7f75a40ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 -- 172.21.15.122:0/3203209684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a41f7cc0 con 0x7f75a40ccaf0 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2000640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75a40cefb0 0x7f75a40ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a2a00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75a40ccaf0 0x7f75a415aa50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f759400e7a0 tx=0x7f759400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f759aa00640 1 -- 172.21.15.122:0/3203209684 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759400ccc0 con 0x7f75a40ccaf0 2024-09-17T05:27:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f759aa00640 1 -- 172.21.15.122:0/3203209684 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7594010430 con 0x7f75a40ccaf0 2024-09-17T05:27:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f759aa00640 1 -- 172.21.15.122:0/3203209684 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594010860 con 0x7f75a40ccaf0 2024-09-17T05:27:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a41f7fa0 con 0x7f75a40ccaf0 2024-09-17T05:27:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.600+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a41f8560 con 0x7f75a40ccaf0 2024-09-17T05:27:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.604+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a40cc5b0 con 0x7f75a40ccaf0 2024-09-17T05:27:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.604+0000 7f759aa00640 1 -- 172.21.15.122:0/3203209684 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 10) ==== 97438+0+0 (secure 0 0 0) 0x7f7594010a00 con 0x7f75a40ccaf0 2024-09-17T05:27:47.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.604+0000 7f759aa00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7584073470 0x7f7584075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:47.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.604+0000 7f759aa00640 1 -- 172.21.15.122:0/3203209684 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f7594095130 con 0x7f75a40ccaf0 2024-09-17T05:27:47.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.604+0000 7f75a2000640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7584073470 0x7f7584075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:47.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.604+0000 7f75a2000640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7584073470 0x7f7584075930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f75900af000 tx=0x7f759001b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:47.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.608+0000 7f759aa00640 1 -- 172.21.15.122:0/3203209684 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f759405f690 con 0x7f75a40ccaf0 2024-09-17T05:27:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.784+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f75a40b02f0 con 0x7f75a40ccaf0 2024-09-17T05:27:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.784+0000 7f759aa00640 1 -- 172.21.15.122:0/3203209684 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f75940640c0 con 0x7f75a40ccaf0 2024-09-17T05:27:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:27:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7584073470 msgr2=0x7f7584075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7584073470 0x7f7584075930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f75900af000 tx=0x7f759001b090 comp rx=0 tx=0).stop 2024-09-17T05:27:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75a40ccaf0 msgr2=0x7f75a415aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75a40ccaf0 0x7f75a415aa50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f759400e7a0 tx=0x7f759400ec60 comp rx=0 tx=0).stop 2024-09-17T05:27:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 shutdown_connections 2024-09-17T05:27:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7584073470 0x7f7584075930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75a415f000 0x7f75a406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75a40cefb0 0x7f75a40ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 --2- 172.21.15.122:0/3203209684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75a40ccaf0 0x7f75a415aa50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 >> 172.21.15.122:0/3203209684 conn(0x7f75a409aa10 msgr2=0x7f75a415e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 shutdown_connections 2024-09-17T05:27:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:47.788+0000 7f75a3e00640 1 -- 172.21.15.122:0/3203209684 wait complete. 2024-09-17T05:27:47.810 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:27:47.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:27:47.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:27:49.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:27:49.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:27:49.529 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:27:49.543 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:27:49.543 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:27:49.543 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:49.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:27:49.561 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:27:49.561 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:27:49.561 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:49.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:27:49.594 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:27:49.594 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:27:49.595 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:49.615 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:27:49.615 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:27:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.864+0000 7f1225e00640 1 -- 172.21.15.122:0/1528757660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 msgr2=0x7f1220073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.864+0000 7f1225e00640 1 --2- 172.21.15.122:0/1528757660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 0x7f1220073ed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f120c009900 tx=0x7f120c01c8b0 comp rx=0 tx=0).stop 2024-09-17T05:27:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.864+0000 7f1225e00640 1 -- 172.21.15.122:0/1528757660 shutdown_connections 2024-09-17T05:27:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.864+0000 7f1225e00640 1 --2- 172.21.15.122:0/1528757660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122006f9a0 0x7f122006fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.864+0000 7f1225e00640 1 --2- 172.21.15.122:0/1528757660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 0x7f1220073ed0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.864+0000 7f1225e00640 1 -- 172.21.15.122:0/1528757660 >> 172.21.15.122:0/1528757660 conn(0x7f12200d0c90 msgr2=0x7f12200d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.864+0000 7f1225e00640 1 -- 172.21.15.122:0/1528757660 shutdown_connections 2024-09-17T05:27:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.864+0000 7f1225e00640 1 -- 172.21.15.122:0/1528757660 wait complete. 2024-09-17T05:27:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f1225e00640 1 Processor -- start 2024-09-17T05:27:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f1225e00640 1 -- start start 2024-09-17T05:27:49.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f1225e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122006f9a0 0x7f12200c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f1225e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 0x7f12201f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f1225e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f122015af60 0x7f1220156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f1225e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1220167cc0 con 0x7f12200cffc0 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f1225e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1220167b40 con 0x7f122015af60 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f1225e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1220167e40 con 0x7f122006f9a0 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f121e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 0x7f12201f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f121f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f122015af60 0x7f1220156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f121f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f122015af60 0x7f1220156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:50874/0 (socket says 172.21.15.122:50874) 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f121f400640 1 -- 172.21.15.122:0/1950656523 learned_addr learned my addr 172.21.15.122:0/1950656523 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.868+0000 7f121ea00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122006f9a0 0x7f12200c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:49.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f121e000640 1 -- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122006f9a0 msgr2=0x7f12200c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:49.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f121e000640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122006f9a0 0x7f12200c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:49.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f121e000640 1 -- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f122015af60 msgr2=0x7f1220156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:49.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f121e000640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f122015af60 0x7f1220156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:49.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f121e000640 1 -- 172.21.15.122:0/1950656523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12201f9d50 con 0x7f12200cffc0 2024-09-17T05:27:49.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f121e000640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 0x7f12201f9810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f120c0ac000 tx=0x7f120c0a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:49.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f1217400640 1 -- 172.21.15.122:0/1950656523 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c0b2040 con 0x7f12200cffc0 2024-09-17T05:27:49.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f1217400640 1 -- 172.21.15.122:0/1950656523 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f120c004450 con 0x7f12200cffc0 2024-09-17T05:27:49.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f1217400640 1 -- 172.21.15.122:0/1950656523 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c005290 con 0x7f12200cffc0 2024-09-17T05:27:49.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12200b28c0 con 0x7f12200cffc0 2024-09-17T05:27:49.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12200b2e40 con 0x7f12200cffc0 2024-09-17T05:27:49.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.872+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122006fe00 con 0x7f12200cffc0 2024-09-17T05:27:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.876+0000 7f1217400640 1 -- 172.21.15.122:0/1950656523 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f120c005430 con 0x7f12200cffc0 2024-09-17T05:27:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.876+0000 7f1217400640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1200073630 0x7f1200075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.876+0000 7f1217400640 1 -- 172.21.15.122:0/1950656523 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f120c12e420 con 0x7f12200cffc0 2024-09-17T05:27:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.876+0000 7f121ea00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1200073630 0x7f1200075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.880+0000 7f121ea00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1200073630 0x7f1200075af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1210007fd0 tx=0x7f1210007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:49.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:49.884+0000 7f1217400640 1 -- 172.21.15.122:0/1950656523 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f12201f9d50 con 0x7f12200cffc0 2024-09-17T05:27:50.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12200b5f40 con 0x7f12200cffc0 2024-09-17T05:27:50.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1217400640 1 -- 172.21.15.122:0/1950656523 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5235 (secure 0 0 0) 0x7f120c0164d0 con 0x7f12200cffc0 2024-09-17T05:27:50.092 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:50.092 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"btime":"2024-09-17T05:26:01:039133+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2936956280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2936956280},{"type":"v1","addr":"172.21.15.173:6836","nonce":2936956280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2308148662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2308148662},{"type":"v1","addr":"172.21.15.122:6834","nonce":2308148662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3699957348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":3699957348},{"type":"v1","addr":"172.21.15.122:6835","nonce":3699957348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:47.923640+0000","modified":"2024-09-17T05:25:50.952955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2399039177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2399039177},{"type":"v1","addr":"172.21.15.173:6837","nonce":2399039177}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4873,"qdb_cluster":[4873]},"id":2}]} 2024-09-17T05:27:50.092 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 17 2024-09-17T05:27:50.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1200073630 msgr2=0x7f1200075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1200073630 0x7f1200075af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1210007fd0 tx=0x7f1210007e40 comp rx=0 tx=0).stop 2024-09-17T05:27:50.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 msgr2=0x7f12201f9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 0x7f12201f9810 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f120c0ac000 tx=0x7f120c0a57e0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 shutdown_connections 2024-09-17T05:27:50.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1200073630 0x7f1200075af0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f122015af60 0x7f1220156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12200cffc0 0x7f12201f9810 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 --2- 172.21.15.122:0/1950656523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122006f9a0 0x7f12200c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 >> 172.21.15.122:0/1950656523 conn(0x7f12200d0c90 msgr2=0x7f1220155a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:50.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.088+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 shutdown_connections 2024-09-17T05:27:50.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.092+0000 7f1225e00640 1 -- 172.21.15.122:0/1950656523 wait complete. 2024-09-17T05:27:50.114 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:27:50.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.252+0000 7fa0c6400640 1 -- 172.21.15.122:0/2882971379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c00cc740 msgr2=0x7fa0c00ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.252+0000 7fa0c6400640 1 --2- 172.21.15.122:0/2882971379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c00cc740 0x7fa0c00ccb80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009840 tx=0x7fa0ac01c790 comp rx=0 tx=0).stop 2024-09-17T05:27:50.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.252+0000 7fa0c6400640 1 -- 172.21.15.122:0/2882971379 shutdown_connections 2024-09-17T05:27:50.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.252+0000 7fa0c6400640 1 --2- 172.21.15.122:0/2882971379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c00cc740 0x7fa0c00ccb80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.252+0000 7fa0c6400640 1 --2- 172.21.15.122:0/2882971379 >> v2:172.21.15.173:3300/0 conn(0x7fa0c006f8c0 0x7fa0c006fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.252+0000 7fa0c6400640 1 -- 172.21.15.122:0/2882971379 >> 172.21.15.122:0/2882971379 conn(0x7fa0c00b18f0 msgr2=0x7fa0c00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:50.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.252+0000 7fa0c6400640 1 -- 172.21.15.122:0/2882971379 shutdown_connections 2024-09-17T05:27:50.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.252+0000 7fa0c6400640 1 -- 172.21.15.122:0/2882971379 wait complete. 2024-09-17T05:27:50.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 Processor -- start 2024-09-17T05:27:50.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 -- start start 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c006e7a0 0x7fa0c0141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c006f8c0 0x7fa0c01568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c00cc740 0x7fa0c0158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0167970 con 0x7fa0c00cc740 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01677f0 con 0x7fa0c006e7a0 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c0167af0 con 0x7fa0c006f8c0 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c00cc740 0x7fa0c0158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bea00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c006f8c0 0x7fa0c01568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c00cc740 0x7fa0c0158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59496/0 (socket says 172.21.15.122:59496) 2024-09-17T05:27:50.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 -- 172.21.15.122:0/1473029831 learned_addr learned my addr 172.21.15.122:0/1473029831 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bf400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c006e7a0 0x7fa0c0141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 -- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c006f8c0 msgr2=0x7fa0c01568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c006f8c0 0x7fa0c01568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 -- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c006e7a0 msgr2=0x7fa0c0141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c006e7a0 0x7fa0c0141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 -- 172.21.15.122:0/1473029831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c00afff0 con 0x7fa0c00cc740 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bea00640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c006f8c0 0x7fa0c01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0bfe00640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c00cc740 0x7fa0c0158ce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800cbc0 tx=0x7fa0b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0b7400640 1 -- 172.21.15.122:0/1473029831 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8016020 con 0x7fa0c00cc740 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0b7400640 1 -- 172.21.15.122:0/1473029831 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0b800ab00 con 0x7fa0c00cc740 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c00b02d0 con 0x7fa0c00cc740 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0b7400640 1 -- 172.21.15.122:0/1473029831 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800ce40 con 0x7fa0c00cc740 2024-09-17T05:27:50.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c00b0890 con 0x7fa0c00cc740 2024-09-17T05:27:50.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.256+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c0069410 con 0x7fa0c00cc740 2024-09-17T05:27:50.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.260+0000 7fa0b7400640 1 -- 172.21.15.122:0/1473029831 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fa0b8019020 con 0x7fa0c00cc740 2024-09-17T05:27:50.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.260+0000 7fa0b7400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0a0073590 0x7fa0a0075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.260+0000 7fa0b7400640 1 -- 172.21.15.122:0/1473029831 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fa0b8094a90 con 0x7fa0c00cc740 2024-09-17T05:27:50.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.260+0000 7fa0bf400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0a0073590 0x7fa0a0075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.260+0000 7fa0bf400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0a0073590 0x7fa0a0075a50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000a7e0 tx=0x7fa0b000c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:50.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.264+0000 7fa0b7400640 1 -- 172.21.15.122:0/1473029831 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa0b8020020 con 0x7fa0c00cc740 2024-09-17T05:27:50.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.460+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0c00ccb80 con 0x7fa0c00cc740 2024-09-17T05:27:50.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0b7400640 1 -- 172.21.15.122:0/1473029831 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5235 (secure 0 0 0) 0x7fa0b805f560 con 0x7fa0c00cc740 2024-09-17T05:27:50.467 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:50.467 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"btime":"2024-09-17T05:26:01:039133+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2936956280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2936956280},{"type":"v1","addr":"172.21.15.173:6836","nonce":2936956280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2308148662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2308148662},{"type":"v1","addr":"172.21.15.122:6834","nonce":2308148662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3699957348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":3699957348},{"type":"v1","addr":"172.21.15.122:6835","nonce":3699957348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:47.923640+0000","modified":"2024-09-17T05:25:50.952955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2399039177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2399039177},{"type":"v1","addr":"172.21.15.173:6837","nonce":2399039177}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4873,"qdb_cluster":[4873]},"id":2}]} 2024-09-17T05:27:50.467 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 17 2024-09-17T05:27:50.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0a0073590 msgr2=0x7fa0a0075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0a0073590 0x7fa0a0075a50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000a7e0 tx=0x7fa0b000c040 comp rx=0 tx=0).stop 2024-09-17T05:27:50.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c00cc740 msgr2=0x7fa0c0158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c00cc740 0x7fa0c0158ce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800cbc0 tx=0x7fa0b800cbf0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 shutdown_connections 2024-09-17T05:27:50.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa0a0073590 0x7fa0a0075a50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0c00cc740 0x7fa0c0158ce0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c006f8c0 0x7fa0c01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 --2- 172.21.15.122:0/1473029831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c006e7a0 0x7fa0c0141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 >> 172.21.15.122:0/1473029831 conn(0x7fa0c00b18f0 msgr2=0x7fa0c00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:50.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 shutdown_connections 2024-09-17T05:27:50.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.464+0000 7fa0c6400640 1 -- 172.21.15.122:0/1473029831 wait complete. 2024-09-17T05:27:50.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:27:50.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50f3e00640 1 -- 172.21.15.122:0/3259129351 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e80a5cf0 con 0x7f50f41563f0 2024-09-17T05:27:50.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 -- 172.21.15.122:0/3259129351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f41563f0 msgr2=0x7f50f415cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 --2- 172.21.15.122:0/3259129351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f41563f0 0x7f50f415cde0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f50f415cad0 tx=0x7f50e801c820 comp rx=0 tx=0).stop 2024-09-17T05:27:50.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 -- 172.21.15.122:0/3259129351 shutdown_connections 2024-09-17T05:27:50.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 --2- 172.21.15.122:0/3259129351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f406cc20 0x7f50f406d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 --2- 172.21.15.122:0/3259129351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f41563f0 0x7f50f415cde0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 -- 172.21.15.122:0/3259129351 >> 172.21.15.122:0/3259129351 conn(0x7f50f40b2890 msgr2=0x7f50f40b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:50.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 -- 172.21.15.122:0/3259129351 shutdown_connections 2024-09-17T05:27:50.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 -- 172.21.15.122:0/3259129351 wait complete. 2024-09-17T05:27:50.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 Processor -- start 2024-09-17T05:27:50.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.628+0000 7f50fc000640 1 -- start start 2024-09-17T05:27:50.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f406cc20 0x7f50f41436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f40b55e0 0x7f50f40d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f41563f0 0x7f50f4155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4168d00 con 0x7f50f41563f0 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4168b80 con 0x7f50f406cc20 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f50f4168e80 con 0x7f50f40b55e0 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f41563f0 0x7f50f4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f41563f0 0x7f50f4155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59504/0 (socket says 172.21.15.122:59504) 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 -- 172.21.15.122:0/2086957534 learned_addr learned my addr 172.21.15.122:0/2086957534 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50f9600640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f406cc20 0x7f50f41436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50f8c00640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f40b55e0 0x7f50f40d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 -- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f40b55e0 msgr2=0x7f50f40d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f40b55e0 0x7f50f40d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 -- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f406cc20 msgr2=0x7f50f41436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f406cc20 0x7f50f41436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 -- 172.21.15.122:0/2086957534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f406f710 con 0x7f50f41563f0 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50f9600640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f406cc20 0x7f50f41436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fa000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f41563f0 0x7f50f4155310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00efd0 tx=0x7f50ec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50f1600640 1 -- 172.21.15.122:0/2086957534 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec003b80 con 0x7f50f41563f0 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50f1600640 1 -- 172.21.15.122:0/2086957534 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50ec003d20 con 0x7f50f41563f0 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50f1600640 1 -- 172.21.15.122:0/2086957534 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec004280 con 0x7f50f41563f0 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f4143c30 con 0x7f50f41563f0 2024-09-17T05:27:50.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f406fc90 con 0x7f50f41563f0 2024-09-17T05:27:50.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f406d0d0 con 0x7f50f41563f0 2024-09-17T05:27:50.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.632+0000 7f50f1600640 1 -- 172.21.15.122:0/2086957534 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f50ec004420 con 0x7f50f41563f0 2024-09-17T05:27:50.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.636+0000 7f50f1600640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50d4073560 0x7f50d4075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.636+0000 7f50f1600640 1 -- 172.21.15.122:0/2086957534 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f50ec095830 con 0x7f50f41563f0 2024-09-17T05:27:50.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.636+0000 7f50f9600640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50d4073560 0x7f50d4075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.636+0000 7f50f9600640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50d4073560 0x7f50d4075a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f50e80af000 tx=0x7f50e801b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:50.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.640+0000 7f50f1600640 1 -- 172.21.15.122:0/2086957534 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f50ec060230 con 0x7f50f41563f0 2024-09-17T05:27:50.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.812+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50f41573f0 con 0x7f50f41563f0 2024-09-17T05:27:50.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.812+0000 7f50f1600640 1 -- 172.21.15.122:0/2086957534 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16978 (secure 0 0 0) 0x7f50ec05fb30 con 0x7f50f41563f0 2024-09-17T05:27:50.818 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:50.818 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":35,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:26:46.142691+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:46.061761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:46.889922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.8","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.23","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:27:50.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50d4073560 msgr2=0x7f50d4075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50d4073560 0x7f50d4075a20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f50e80af000 tx=0x7f50e801b060 comp rx=0 tx=0).stop 2024-09-17T05:27:50.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f41563f0 msgr2=0x7f50f4155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f41563f0 0x7f50f4155310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00efd0 tx=0x7f50ec00c6a0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 shutdown_connections 2024-09-17T05:27:50.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50d4073560 0x7f50d4075a20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f41563f0 0x7f50f4155310 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f40b55e0 0x7f50f40d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 --2- 172.21.15.122:0/2086957534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50f406cc20 0x7f50f41436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 >> 172.21.15.122:0/2086957534 conn(0x7f50f40b2890 msgr2=0x7f50f40b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:50.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 shutdown_connections 2024-09-17T05:27:50.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.816+0000 7f50fc000640 1 -- 172.21.15.122:0/2086957534 wait complete. 2024-09-17T05:27:50.839 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:27:50.840 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:27:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.976+0000 7ff48f800640 1 -- 172.21.15.122:0/3567558927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880cdb30 msgr2=0x7ff4880cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.976+0000 7ff48f800640 1 --2- 172.21.15.122:0/3567558927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880cdb30 0x7ff4880cebb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff4880745d0 tx=0x7ff47401c830 comp rx=0 tx=0).stop 2024-09-17T05:27:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.976+0000 7ff48f800640 1 -- 172.21.15.122:0/3567558927 shutdown_connections 2024-09-17T05:27:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.976+0000 7ff48f800640 1 --2- 172.21.15.122:0/3567558927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880cdb30 0x7ff4880cebb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.976+0000 7ff48f800640 1 --2- 172.21.15.122:0/3567558927 >> v2:172.21.15.173:3300/0 conn(0x7ff48815bea0 0x7ff4880ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.976+0000 7ff48f800640 1 -- 172.21.15.122:0/3567558927 >> 172.21.15.122:0/3567558927 conn(0x7ff4880d1dc0 msgr2=0x7ff4880d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.976+0000 7ff48f800640 1 -- 172.21.15.122:0/3567558927 shutdown_connections 2024-09-17T05:27:50.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.976+0000 7ff48f800640 1 -- 172.21.15.122:0/3567558927 wait complete. 2024-09-17T05:27:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 Processor -- start 2024-09-17T05:27:50.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 -- start start 2024-09-17T05:27:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4880cdb30 0x7ff48814a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880d28e0 0x7ff4880b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48815bea0 0x7ff4880b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff488168fc0 con 0x7ff4880d28e0 2024-09-17T05:27:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff488168e40 con 0x7ff48815bea0 2024-09-17T05:27:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff488169140 con 0x7ff4880cdb30 2024-09-17T05:27:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880d28e0 0x7ff4880b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880d28e0 0x7ff4880b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59526/0 (socket says 172.21.15.122:59526) 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 -- 172.21.15.122:0/4143674515 learned_addr learned my addr 172.21.15.122:0/4143674515 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48ce00640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4880cdb30 0x7ff48814a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48d800640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48815bea0 0x7ff4880b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 -- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4880cdb30 msgr2=0x7ff48814a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4880cdb30 0x7ff48814a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 -- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48815bea0 msgr2=0x7ff4880b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48815bea0 0x7ff4880b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 -- 172.21.15.122:0/4143674515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48814a710 con 0x7ff4880d28e0 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48d800640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48815bea0 0x7ff4880b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:50.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff487e00640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880d28e0 0x7ff4880b72d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff4740b0040 tx=0x7ff4740a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff484c00640 1 -- 172.21.15.122:0/4143674515 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4740b2040 con 0x7ff4880d28e0 2024-09-17T05:27:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff484c00640 1 -- 172.21.15.122:0/4143674515 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff474005020 con 0x7ff4880d28e0 2024-09-17T05:27:50.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4880b1640 con 0x7ff4880d28e0 2024-09-17T05:27:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff484c00640 1 -- 172.21.15.122:0/4143674515 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474005470 con 0x7ff4880d28e0 2024-09-17T05:27:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.980+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4880b1c00 con 0x7ff4880d28e0 2024-09-17T05:27:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.984+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4880cc9d0 con 0x7ff4880d28e0 2024-09-17T05:27:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.984+0000 7ff484c00640 1 -- 172.21.15.122:0/4143674515 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7ff4740046d0 con 0x7ff4880d28e0 2024-09-17T05:27:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.984+0000 7ff484c00640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff468073560 0x7ff468075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:50.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.984+0000 7ff484c00640 1 -- 172.21.15.122:0/4143674515 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7ff47412e440 con 0x7ff4880d28e0 2024-09-17T05:27:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.984+0000 7ff48ce00640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff468073560 0x7ff468075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:50.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.984+0000 7ff48ce00640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff468073560 0x7ff468075a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff47800bb50 tx=0x7ff47800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:50.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:50.988+0000 7ff484c00640 1 -- 172.21.15.122:0/4143674515 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff4740164b0 con 0x7ff4880d28e0 2024-09-17T05:27:51.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.188+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff48806ed10 con 0x7ff4880d28e0 2024-09-17T05:27:51.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff484c00640 1 -- 172.21.15.122:0/4143674515 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5235 (secure 0 0 0) 0x7ff4740f8f10 con 0x7ff4880d28e0 2024-09-17T05:27:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:51.195 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"btime":"2024-09-17T05:26:01:039133+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2936956280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2936956280},{"type":"v1","addr":"172.21.15.173:6836","nonce":2936956280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2308148662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2308148662},{"type":"v1","addr":"172.21.15.122:6834","nonce":2308148662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3699957348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":3699957348},{"type":"v1","addr":"172.21.15.122:6835","nonce":3699957348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:47.923640+0000","modified":"2024-09-17T05:25:50.952955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2399039177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2399039177},{"type":"v1","addr":"172.21.15.173:6837","nonce":2399039177}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4873,"qdb_cluster":[4873]},"id":2}]} 2024-09-17T05:27:51.195 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 17 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff468073560 msgr2=0x7ff468075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff468073560 0x7ff468075a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff47800bb50 tx=0x7ff47800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880d28e0 msgr2=0x7ff4880b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880d28e0 0x7ff4880b72d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff4740b0040 tx=0x7ff4740a5710 comp rx=0 tx=0).stop 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 shutdown_connections 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff468073560 0x7ff468075a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48815bea0 0x7ff4880b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4880d28e0 0x7ff4880b72d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 --2- 172.21.15.122:0/4143674515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4880cdb30 0x7ff48814a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 >> 172.21.15.122:0/4143674515 conn(0x7ff4880d1dc0 msgr2=0x7ff4881662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 shutdown_connections 2024-09-17T05:27:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.192+0000 7ff48f800640 1 -- 172.21.15.122:0/4143674515 wait complete. 2024-09-17T05:27:51.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:27:51.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.352+0000 7fe132400640 1 -- 172.21.15.122:0/2173508508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 msgr2=0x7fe12c15dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.352+0000 7fe132400640 1 --2- 172.21.15.122:0/2173508508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 0x7fe12c15dae0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe114009900 tx=0x7fe11401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.352+0000 7fe132400640 1 -- 172.21.15.122:0/2173508508 shutdown_connections 2024-09-17T05:27:51.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.352+0000 7fe132400640 1 --2- 172.21.15.122:0/2173508508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 0x7fe12c15dae0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.352+0000 7fe132400640 1 -- 172.21.15.122:0/2173508508 >> 172.21.15.122:0/2173508508 conn(0x7fe12c0b18f0 msgr2=0x7fe12c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:51.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.352+0000 7fe132400640 1 -- 172.21.15.122:0/2173508508 shutdown_connections 2024-09-17T05:27:51.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.352+0000 7fe132400640 1 -- 172.21.15.122:0/2173508508 wait complete. 2024-09-17T05:27:51.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 Processor -- start 2024-09-17T05:27:51.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 -- start start 2024-09-17T05:27:51.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe12c06f8b0 0x7fe12c158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:51.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe12c0acd40 0x7fe12c155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 0x7fe12c150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1692c0 con 0x7fe12c0cd100 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c169140 con 0x7fe12c0acd40 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe12c169440 con 0x7fe12c06f8b0 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 0x7fe12c150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 0x7fe12c150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43270/0 (socket says 172.21.15.122:43270) 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 -- 172.21.15.122:0/4085612229 learned_addr learned my addr 172.21.15.122:0/4085612229 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe122a00640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe12c0acd40 0x7fe12c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12b400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe12c06f8b0 0x7fe12c158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 -- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe12c06f8b0 msgr2=0x7fe12c158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe12c06f8b0 0x7fe12c158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 -- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe12c0acd40 msgr2=0x7fe12c155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe12c0acd40 0x7fe12c155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 -- 172.21.15.122:0/4085612229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c1561f0 con 0x7fe12c0cd100 2024-09-17T05:27:51.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe12be00640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 0x7fe12c150bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe12400c3a0 tx=0x7fe12400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:51.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe123e00640 1 -- 172.21.15.122:0/4085612229 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124017070 con 0x7fe12c0cd100 2024-09-17T05:27:51.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe123e00640 1 -- 172.21.15.122:0/4085612229 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe124003600 con 0x7fe12c0cd100 2024-09-17T05:27:51.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c152420 con 0x7fe12c0cd100 2024-09-17T05:27:51.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe123e00640 1 -- 172.21.15.122:0/4085612229 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124003a50 con 0x7fe12c0cd100 2024-09-17T05:27:51.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.356+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c1529e0 con 0x7fe12c0cd100 2024-09-17T05:27:51.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.360+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12c069410 con 0x7fe12c0cd100 2024-09-17T05:27:51.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.360+0000 7fe123e00640 1 -- 172.21.15.122:0/4085612229 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fe12401a070 con 0x7fe12c0cd100 2024-09-17T05:27:51.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.360+0000 7fe123e00640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe108073560 0x7fe108075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:51.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.360+0000 7fe123e00640 1 -- 172.21.15.122:0/4085612229 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fe124093b50 con 0x7fe12c0cd100 2024-09-17T05:27:51.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.360+0000 7fe12b400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe108073560 0x7fe108075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:51.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.360+0000 7fe12b400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe108073560 0x7fe108075a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0012f0 tx=0x7fe11c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:51.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.364+0000 7fe123e00640 1 -- 172.21.15.122:0/4085612229 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe12405dfe0 con 0x7fe12c0cd100 2024-09-17T05:27:51.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe12c152bb0 con 0x7fe12c0cd100 2024-09-17T05:27:51.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe123e00640 1 -- 172.21.15.122:0/4085612229 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5235 (secure 0 0 0) 0x7fe124062a10 con 0x7fe12c0cd100 2024-09-17T05:27:51.571 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:51.571 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"btime":"2024-09-17T05:26:01:039133+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2936956280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2936956280},{"type":"v1","addr":"172.21.15.173:6836","nonce":2936956280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2308148662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2308148662},{"type":"v1","addr":"172.21.15.122:6834","nonce":2308148662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3699957348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":3699957348},{"type":"v1","addr":"172.21.15.122:6835","nonce":3699957348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:25:47.923640+0000","modified":"2024-09-17T05:25:50.952955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4873},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4873":{"gid":4873,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/2399039177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2399039177},{"type":"v1","addr":"172.21.15.173:6837","nonce":2399039177}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4873,"qdb_cluster":[4873]},"id":2}]} 2024-09-17T05:27:51.571 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 17 2024-09-17T05:27:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe108073560 msgr2=0x7fe108075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe108073560 0x7fe108075a20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0012f0 tx=0x7fe11c009040 comp rx=0 tx=0).stop 2024-09-17T05:27:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 msgr2=0x7fe12c150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 0x7fe12c150bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe12400c3a0 tx=0x7fe12400c860 comp rx=0 tx=0).stop 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 shutdown_connections 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe108073560 0x7fe108075a20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe12c0cd100 0x7fe12c150bf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe12c0acd40 0x7fe12c155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 --2- 172.21.15.122:0/4085612229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe12c06f8b0 0x7fe12c158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 >> 172.21.15.122:0/4085612229 conn(0x7fe12c0b18f0 msgr2=0x7fe12c151950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 shutdown_connections 2024-09-17T05:27:51.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.568+0000 7fe132400640 1 -- 172.21.15.122:0/4085612229 wait complete. 2024-09-17T05:27:51.592 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:27:51.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.728+0000 7f6fba400640 1 -- 172.21.15.122:0/2975325638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f98003a60 msgr2=0x7f6f98003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.728+0000 7f6fba400640 1 --2- 172.21.15.122:0/2975325638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f98003a60 0x7f6f98003e80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4006fd0 tx=0x7f6fa401cee0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- 172.21.15.122:0/2975325638 shutdown_connections 2024-09-17T05:27:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 --2- 172.21.15.122:0/2975325638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f98003a60 0x7f6f98003e80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- 172.21.15.122:0/2975325638 >> 172.21.15.122:0/2975325638 conn(0x7f6fb406dd30 msgr2=0x7f6fb406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- 172.21.15.122:0/2975325638 shutdown_connections 2024-09-17T05:27:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- 172.21.15.122:0/2975325638 wait complete. 2024-09-17T05:27:51.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 Processor -- start 2024-09-17T05:27:51.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- start start 2024-09-17T05:27:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f98003a60 0x7f6fb4158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fb41fb250 0x7f6fb41f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fb41fbbc0 0x7f6fb40af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb40b52b0 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb40b5130 con 0x7f6f98003a60 2024-09-17T05:27:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb40b5430 con 0x7f6fb41fb250 2024-09-17T05:27:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fb41fbbc0 0x7f6fb40af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fb41fbbc0 0x7f6fb40af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43288/0 (socket says 172.21.15.122:43288) 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 -- 172.21.15.122:0/100707423 learned_addr learned my addr 172.21.15.122:0/100707423 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f98003a60 0x7f6fb4158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6faaa00640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fb41fb250 0x7f6fb41f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 -- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fb41fb250 msgr2=0x7f6fb41f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fb41fb250 0x7f6fb41f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 -- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f98003a60 msgr2=0x7f6fb4158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f98003a60 0x7f6fb4158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 -- 172.21.15.122:0/100707423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb4158540 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f98003a60 0x7f6fb4158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6faaa00640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fb41fb250 0x7f6fb41f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fb3e00640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fb41fbbc0 0x7f6fb40af920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40aee30 tx=0x7f6fac00fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fabe00640 1 -- 172.21.15.122:0/100707423 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac011910 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fabe00640 1 -- 172.21.15.122:0/100707423 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fac011ba0 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fabe00640 1 -- 172.21.15.122:0/100707423 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac01b410 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.732+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb41f81a0 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.736+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb41f8700 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.736+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb40af710 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.736+0000 7f6fabe00640 1 -- 172.21.15.122:0/100707423 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6fac019020 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.736+0000 7f6fabe00640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6f90073560 0x7f6f90075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.736+0000 7f6fabe00640 1 -- 172.21.15.122:0/100707423 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f6fac094d20 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.736+0000 7f6fb3400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6f90073560 0x7f6f90075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.740+0000 7f6fb3400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6f90073560 0x7f6f90075a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40ab000 tx=0x7f6fa4019d20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.740+0000 7f6fabe00640 1 -- 172.21.15.122:0/100707423 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6fb41f81a0 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.912+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fb40b0910 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.912+0000 7f6fabe00640 1 -- 172.21.15.122:0/100707423 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16978 (secure 0 0 0) 0x7f6fac05f6f0 con 0x7f6fb41fbbc0 2024-09-17T05:27:51.917 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:51.917 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":35,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:26:46.142691+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:25:46.061761+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T05:25:46.889922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.8","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.23","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:27:51.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6f90073560 msgr2=0x7f6f90075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6f90073560 0x7f6f90075a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40ab000 tx=0x7f6fa4019d20 comp rx=0 tx=0).stop 2024-09-17T05:27:51.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fb41fbbc0 msgr2=0x7f6fb40af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fb41fbbc0 0x7f6fb40af920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40aee30 tx=0x7f6fac00fa20 comp rx=0 tx=0).stop 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 shutdown_connections 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6f90073560 0x7f6f90075a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fb41fbbc0 0x7f6fb40af920 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fb41fb250 0x7f6fb41f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 --2- 172.21.15.122:0/100707423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f98003a60 0x7f6fb4158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 >> 172.21.15.122:0/100707423 conn(0x7f6fb406dd30 msgr2=0x7f6fb415c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 shutdown_connections 2024-09-17T05:27:51.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:51.916+0000 7f6fba400640 1 -- 172.21.15.122:0/100707423 wait complete. 2024-09-17T05:27:51.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:27:52.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 -- 172.21.15.122:0/2588142527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 msgr2=0x7f61740cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:52.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 --2- 172.21.15.122:0/2588142527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 0x7f61740cc940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f61740d0f60 tx=0x7f615c01c6e0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 -- 172.21.15.122:0/2588142527 shutdown_connections 2024-09-17T05:27:52.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 --2- 172.21.15.122:0/2588142527 >> v2:172.21.15.173:3300/0 conn(0x7f617415c130 0x7f61740cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 --2- 172.21.15.122:0/2588142527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 0x7f61740cc940 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 -- 172.21.15.122:0/2588142527 >> 172.21.15.122:0/2588142527 conn(0x7f61740d1ea0 msgr2=0x7f61740d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:52.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 -- 172.21.15.122:0/2588142527 shutdown_connections 2024-09-17T05:27:52.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 -- 172.21.15.122:0/2588142527 wait complete. 2024-09-17T05:27:52.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 Processor -- start 2024-09-17T05:27:52.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.080+0000 7f6179a00640 1 -- start start 2024-09-17T05:27:52.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 0x7f61740c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:52.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f617415bd50 0x7f61740b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f617415c130 0x7f6174156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f61741694a0 con 0x7f61740cecf0 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6174169320 con 0x7f617415c130 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6174169620 con 0x7f617415bd50 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 0x7f61740c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 0x7f61740c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43304/0 (socket says 172.21.15.122:43304) 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 -- 172.21.15.122:0/3198229882 learned_addr learned my addr 172.21.15.122:0/3198229882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 -- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f617415bd50 msgr2=0x7f61740b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f617415bd50 0x7f61740b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6173400640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f617415c130 0x7f6174156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 -- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f617415c130 msgr2=0x7f6174156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f617415c130 0x7f6174156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 -- 172.21.15.122:0/3198229882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61740ca3b0 con 0x7f61740cecf0 2024-09-17T05:27:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6173400640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f617415c130 0x7f6174156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:27:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6172a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 0x7f61740c9980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f616400c4c0 tx=0x7f616400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f616aa00640 1 -- 172.21.15.122:0/3198229882 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164016020 con 0x7f61740cecf0 2024-09-17T05:27:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f616aa00640 1 -- 172.21.15.122:0/3198229882 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6164012940 con 0x7f61740cecf0 2024-09-17T05:27:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f616aa00640 1 -- 172.21.15.122:0/3198229882 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164012d90 con 0x7f61740cecf0 2024-09-17T05:27:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61740ca690 con 0x7f61740cecf0 2024-09-17T05:27:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61740cac50 con 0x7f61740cecf0 2024-09-17T05:27:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61740cb360 con 0x7f61740cecf0 2024-09-17T05:27:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.084+0000 7f616aa00640 1 -- 172.21.15.122:0/3198229882 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6164019020 con 0x7f61740cecf0 2024-09-17T05:27:52.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.088+0000 7f616aa00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6150073560 0x7f6150075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:52.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.088+0000 7f616aa00640 1 -- 172.21.15.122:0/3198229882 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f61640948a0 con 0x7f61740cecf0 2024-09-17T05:27:52.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.088+0000 7f616be00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6150073560 0x7f6150075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.088+0000 7f616be00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6150073560 0x7f6150075a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f615c0af000 tx=0x7f615c019480 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:52.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.092+0000 7f616aa00640 1 -- 172.21.15.122:0/3198229882 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6164020020 con 0x7f61740cecf0 2024-09-17T05:27:52.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.296+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f61740c8c90 con 0x7f61740cecf0 2024-09-17T05:27:52.302 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:27:52.296+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:27:52.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f616aa00640 1 -- 172.21.15.122:0/3198229882 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f616405f430 con 0x7f61740cecf0 2024-09-17T05:27:52.304 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:27:52.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6150073560 msgr2=0x7f6150075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:52.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6150073560 0x7f6150075a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f615c0af000 tx=0x7f615c019480 comp rx=0 tx=0).stop 2024-09-17T05:27:52.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 msgr2=0x7f61740c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 0x7f61740c9980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f616400c4c0 tx=0x7f616400c980 comp rx=0 tx=0).stop 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 shutdown_connections 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6150073560 0x7f6150075a20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f617415c130 0x7f6174156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f617415bd50 0x7f61740b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.300+0000 7f6179a00640 1 --2- 172.21.15.122:0/3198229882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f61740cecf0 0x7f61740c9980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.304+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 >> 172.21.15.122:0/3198229882 conn(0x7f61740d1ea0 msgr2=0x7f6174070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.304+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 shutdown_connections 2024-09-17T05:27:52.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.304+0000 7f6179a00640 1 -- 172.21.15.122:0/3198229882 wait complete. 2024-09-17T05:27:52.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:27:52.375 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:27:52.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- 172.21.15.122:0/2260763400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 msgr2=0x7f6b3c0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:52.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 --2- 172.21.15.122:0/2260763400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 0x7f6b3c0cf9c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009bc0 tx=0x7f6b3401cb70 comp rx=0 tx=0).stop 2024-09-17T05:27:52.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- 172.21.15.122:0/2260763400 shutdown_connections 2024-09-17T05:27:52.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 --2- 172.21.15.122:0/2260763400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 0x7f6b3c0cf9c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- 172.21.15.122:0/2260763400 >> 172.21.15.122:0/2260763400 conn(0x7f6b3c0d1df0 msgr2=0x7f6b3c0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:52.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- 172.21.15.122:0/2260763400 shutdown_connections 2024-09-17T05:27:52.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- 172.21.15.122:0/2260763400 wait complete. 2024-09-17T05:27:52.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 Processor -- start 2024-09-17T05:27:52.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- start start 2024-09-17T05:27:52.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b3c0b1190 0x7f6b3c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:52.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 0x7f6b3c1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:52.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b3c1603f0 0x7f6b3c167440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:52.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c0b34a0 con 0x7f6b3c0ce980 2024-09-17T05:27:52.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c160910 con 0x7f6b3c1603f0 2024-09-17T05:27:52.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b40a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3c16a500 con 0x7f6b3c0b1190 2024-09-17T05:27:52.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.468+0000 7f6b31400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 0x7f6b3c1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:52.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b31400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 0x7f6b3c1584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43320/0 (socket says 172.21.15.122:43320) 2024-09-17T05:27:52.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b31400640 1 -- 172.21.15.122:0/2665709032 learned_addr learned my addr 172.21.15.122:0/2665709032 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:52.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b3a800640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b3c1603f0 0x7f6b3c167440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:52.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b39e00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b3c0b1190 0x7f6b3c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:52.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b31400640 1 -- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b3c0b1190 msgr2=0x7f6b3c153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:52.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b31400640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b3c0b1190 0x7f6b3c153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b31400640 1 -- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b3c1603f0 msgr2=0x7f6b3c167440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:52.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b31400640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b3c1603f0 0x7f6b3c167440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:52.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b31400640 1 -- 172.21.15.122:0/2665709032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c0aede0 con 0x7f6b3c0ce980 2024-09-17T05:27:52.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b3a800640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b3c1603f0 0x7f6b3c167440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:52.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b31400640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 0x7f6b3c1584a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400c4b0 tx=0x7f6b2400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:52.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b32a00640 1 -- 172.21.15.122:0/2665709032 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24016020 con 0x7f6b3c0ce980 2024-09-17T05:27:52.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b32a00640 1 -- 172.21.15.122:0/2665709032 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b24012920 con 0x7f6b3c0ce980 2024-09-17T05:27:52.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b32a00640 1 -- 172.21.15.122:0/2665709032 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24012d50 con 0x7f6b3c0ce980 2024-09-17T05:27:52.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c0af0c0 con 0x7f6b3c0ce980 2024-09-17T05:27:52.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c0af680 con 0x7f6b3c0ce980 2024-09-17T05:27:52.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c0cf9c0 con 0x7f6b3c0ce980 2024-09-17T05:27:52.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.472+0000 7f6b32a00640 1 -- 172.21.15.122:0/2665709032 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6b24019020 con 0x7f6b3c0ce980 2024-09-17T05:27:52.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.476+0000 7f6b32a00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b18073560 0x7f6b18075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:52.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.476+0000 7f6b32a00640 1 -- 172.21.15.122:0/2665709032 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5911+0+0 (secure 0 0 0) 0x7f6b24094970 con 0x7f6b3c0ce980 2024-09-17T05:27:52.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.476+0000 7f6b39e00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b18073560 0x7f6b18075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:52.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.476+0000 7f6b39e00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b18073560 0x7f6b18075a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0b7390 tx=0x7f6b2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:52.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.476+0000 7f6b32a00640 1 -- 172.21.15.122:0/2665709032 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6b24020020 con 0x7f6b3c0ce980 2024-09-17T05:27:52.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:52.680+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6b3c1589e0 con 0x7f6b3c0ce980 2024-09-17T05:27:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b32a00640 1 -- 172.21.15.122:0/2665709032 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f6b2405f3f0 con 0x7f6b3c0ce980 2024-09-17T05:27:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b18073560 msgr2=0x7f6b18075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b18073560 0x7f6b18075a20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0b7390 tx=0x7f6b2c009040 comp rx=0 tx=0).stop 2024-09-17T05:27:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 msgr2=0x7f6b3c1584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 0x7f6b3c1584a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400c4b0 tx=0x7f6b2400c970 comp rx=0 tx=0).stop 2024-09-17T05:27:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 shutdown_connections 2024-09-17T05:27:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6b18073560 0x7f6b18075a20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b3c1603f0 0x7f6b3c167440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b3c0ce980 0x7f6b3c1584a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 --2- 172.21.15.122:0/2665709032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b3c0b1190 0x7f6b3c153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 >> 172.21.15.122:0/2665709032 conn(0x7f6b3c0d1df0 msgr2=0x7f6b3c06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 shutdown_connections 2024-09-17T05:27:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.308+0000 7f6b40a00640 1 -- 172.21.15.122:0/2665709032 wait complete. 2024-09-17T05:27:53.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:27:53.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.476+0000 7f19b2e00640 1 -- 172.21.15.122:0/815865202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19940051d0 msgr2=0x7f199400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:53.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.476+0000 7f19b2e00640 1 --2- 172.21.15.122:0/815865202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19940051d0 0x7f199400b9d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f19a400c660 tx=0x7f19a401d2e0 comp rx=0 tx=0).stop 2024-09-17T05:27:53.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.476+0000 7f19b2e00640 1 -- 172.21.15.122:0/815865202 shutdown_connections 2024-09-17T05:27:53.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.476+0000 7f19b2e00640 1 --2- 172.21.15.122:0/815865202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19940051d0 0x7f199400b9d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:53.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.476+0000 7f19b2e00640 1 -- 172.21.15.122:0/815865202 >> 172.21.15.122:0/815865202 conn(0x7f19ac0d1da0 msgr2=0x7f19ac0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:53.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.476+0000 7f19b2e00640 1 -- 172.21.15.122:0/815865202 shutdown_connections 2024-09-17T05:27:53.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.476+0000 7f19b2e00640 1 -- 172.21.15.122:0/815865202 wait complete. 2024-09-17T05:27:53.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 Processor -- start 2024-09-17T05:27:53.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 -- start start 2024-09-17T05:27:53.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19940051d0 0x7f19ac1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:53.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19ac0b4bf0 0x7f19ac0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:53.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19ac15dd30 0x7f19ac15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:53.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac0d0460 con 0x7f19ac15dd30 2024-09-17T05:27:53.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac0d02e0 con 0x7f19940051d0 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f19ac0d05e0 con 0x7f19ac0b4bf0 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19ac15dd30 0x7f19ac15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19ac15dd30 0x7f19ac15e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43340/0 (socket says 172.21.15.122:43340) 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 -- 172.21.15.122:0/529250267 learned_addr learned my addr 172.21.15.122:0/529250267 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b1000640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19ac0b4bf0 0x7f19ac0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b1a00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19940051d0 0x7f19ac1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 -- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19ac0b4bf0 msgr2=0x7f19ac0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19ac0b4bf0 0x7f19ac0b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 -- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19940051d0 msgr2=0x7f19ac1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19940051d0 0x7f19ac1555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 -- 172.21.15.122:0/529250267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac0c5650 con 0x7f19ac15dd30 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b1000640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19ac0b4bf0 0x7f19ac0b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:53.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2400640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19ac15dd30 0x7f19ac15e190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f19a4098eb0 tx=0x7f19a40ad940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:53.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19a9600640 1 -- 172.21.15.122:0/529250267 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a400c280 con 0x7f19ac15dd30 2024-09-17T05:27:53.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19a9600640 1 -- 172.21.15.122:0/529250267 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19a40ade20 con 0x7f19ac15dd30 2024-09-17T05:27:53.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19a9600640 1 -- 172.21.15.122:0/529250267 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4002090 con 0x7f19ac15dd30 2024-09-17T05:27:53.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ac0cb620 con 0x7f19ac15dd30 2024-09-17T05:27:53.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.480+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ac0cbba0 con 0x7f19ac15dd30 2024-09-17T05:27:53.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.484+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ac0cbd70 con 0x7f19ac15dd30 2024-09-17T05:27:53.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.484+0000 7f19a9600640 1 -- 172.21.15.122:0/529250267 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f19a40ab030 con 0x7f19ac15dd30 2024-09-17T05:27:53.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.484+0000 7f19a9600640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f198c073560 0x7f198c075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:53.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.484+0000 7f19a9600640 1 -- 172.21.15.122:0/529250267 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5911+0+0 (secure 0 0 0) 0x7f19a412e870 con 0x7f19ac15dd30 2024-09-17T05:27:53.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.484+0000 7f19b1a00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f198c073560 0x7f198c075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:53.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.484+0000 7f19b1a00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f198c073560 0x7f198c075a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19a0000c10 tx=0x7f19a0006b80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:53.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.488+0000 7f19a9600640 1 -- 172.21.15.122:0/529250267 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f19a4017090 con 0x7f19ac15dd30 2024-09-17T05:27:53.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:53.660+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19ac15f410 con 0x7f19ac15dd30 2024-09-17T05:27:54.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19a9600640 1 -- 172.21.15.122:0/529250267 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7f19a40f9420 con 0x7f19ac15dd30 2024-09-17T05:27:54.313 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:27:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f198c073560 msgr2=0x7f198c075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f198c073560 0x7f198c075a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19a0000c10 tx=0x7f19a0006b80 comp rx=0 tx=0).stop 2024-09-17T05:27:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19ac15dd30 msgr2=0x7f19ac15e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19ac15dd30 0x7f19ac15e190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f19a4098eb0 tx=0x7f19a40ad940 comp rx=0 tx=0).stop 2024-09-17T05:27:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 shutdown_connections 2024-09-17T05:27:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f198c073560 0x7f198c075a20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19ac15dd30 0x7f19ac15e190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19ac0b4bf0 0x7f19ac0b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 --2- 172.21.15.122:0/529250267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19940051d0 0x7f19ac1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 >> 172.21.15.122:0/529250267 conn(0x7f19ac0d1da0 msgr2=0x7f19ac0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 shutdown_connections 2024-09-17T05:27:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.312+0000 7f19b2e00640 1 -- 172.21.15.122:0/529250267 wait complete. 2024-09-17T05:27:54.342 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:27:54.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 -- 172.21.15.122:0/1333784060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd28c0b1ad0 msgr2=0x7fd28c0aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:54.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 --2- 172.21.15.122:0/1333784060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd28c0b1ad0 0x7fd28c0aff60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd278009870 tx=0x7fd27801c700 comp rx=0 tx=0).stop 2024-09-17T05:27:54.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 -- 172.21.15.122:0/1333784060 shutdown_connections 2024-09-17T05:27:54.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 --2- 172.21.15.122:0/1333784060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd28c0b1ad0 0x7fd28c0aff60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:54.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 --2- 172.21.15.122:0/1333784060 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd28c06df90 0x7fd28c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:54.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 -- 172.21.15.122:0/1333784060 >> 172.21.15.122:0/1333784060 conn(0x7fd28c0d1da0 msgr2=0x7fd28c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:54.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 -- 172.21.15.122:0/1333784060 shutdown_connections 2024-09-17T05:27:54.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 -- 172.21.15.122:0/1333784060 wait complete. 2024-09-17T05:27:54.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.484+0000 7fd28b200640 1 Processor -- start 2024-09-17T05:27:54.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 -- start start 2024-09-17T05:27:54.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd28c06df90 0x7fd28c152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:54.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd28c0b04a0 0x7fd28c15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0b1ad0 0x7fd28c15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c0d0c40 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c0d0ac0 con 0x7fd28c06df90 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd28c16b180 con 0x7fd28c0b04a0 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0b1ad0 0x7fd28c15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0b1ad0 0x7fd28c15b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43348/0 (socket says 172.21.15.122:43348) 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 -- 172.21.15.122:0/1600418579 learned_addr learned my addr 172.21.15.122:0/1600418579 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd289e00640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd28c06df90 0x7fd28c152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd289400640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd28c0b04a0 0x7fd28c15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 -- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd28c0b04a0 msgr2=0x7fd28c15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:54.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd28c0b04a0 0x7fd28c15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:54.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 -- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd28c06df90 msgr2=0x7fd28c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:54.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd28c06df90 0x7fd28c152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:54.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 -- 172.21.15.122:0/1600418579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c15d3d0 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28a800640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0b1ad0 0x7fd28c15b560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd28400cbf0 tx=0x7fd28400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:54.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd282000640 1 -- 172.21.15.122:0/1600418579 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284016020 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd282000640 1 -- 172.21.15.122:0/1600418579 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd28400ab00 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c140130 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd282000640 1 -- 172.21.15.122:0/1600418579 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284012420 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1406f0 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.488+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c0b1590 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.492+0000 7fd282000640 1 -- 172.21.15.122:0/1600418579 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd284019020 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.492+0000 7fd282000640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd26c073560 0x7fd26c075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:54.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.492+0000 7fd282000640 1 -- 172.21.15.122:0/1600418579 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5473+0+0 (secure 0 0 0) 0x7fd2840948b0 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.492+0000 7fd289e00640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd26c073560 0x7fd26c075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:54.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.492+0000 7fd289e00640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd26c073560 0x7fd26c075a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28c06aea0 tx=0x7fd27c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:54.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.496+0000 7fd282000640 1 -- 172.21.15.122:0/1600418579 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd284020020 con 0x7fd28c0b1ad0 2024-09-17T05:27:54.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:54.664+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd28c15f0a0 con 0x7fd28c0b1ad0 2024-09-17T05:27:55.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd282000640 1 -- 172.21.15.122:0/1600418579 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7fd28c15d3d0 con 0x7fd28c0b1ad0 2024-09-17T05:27:55.320 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:27:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd26c073560 msgr2=0x7fd26c075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd26c073560 0x7fd26c075a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd28c06aea0 tx=0x7fd27c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0b1ad0 msgr2=0x7fd28c15b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0b1ad0 0x7fd28c15b560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd28400cbf0 tx=0x7fd28400cce0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 shutdown_connections 2024-09-17T05:27:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd26c073560 0x7fd26c075a20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0b1ad0 0x7fd28c15b560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd28c0b04a0 0x7fd28c15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 --2- 172.21.15.122:0/1600418579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd28c06df90 0x7fd28c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.320+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 >> 172.21.15.122:0/1600418579 conn(0x7fd28c0d1da0 msgr2=0x7fd28c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.324+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 shutdown_connections 2024-09-17T05:27:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.324+0000 7fd28b200640 1 -- 172.21.15.122:0/1600418579 wait complete. 2024-09-17T05:27:55.356 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:27:55.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.580+0000 7f03cee00640 1 -- 172.21.15.122:0/3892917871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c8073690 msgr2=0x7f03c8073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.580+0000 7f03cee00640 1 --2- 172.21.15.122:0/3892917871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c8073690 0x7f03c8073af0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009900 tx=0x7f03b401c850 comp rx=0 tx=0).stop 2024-09-17T05:27:55.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.580+0000 7f03cee00640 1 -- 172.21.15.122:0/3892917871 shutdown_connections 2024-09-17T05:27:55.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.580+0000 7f03cee00640 1 --2- 172.21.15.122:0/3892917871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c8073690 0x7f03c8073af0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.580+0000 7f03cee00640 1 --2- 172.21.15.122:0/3892917871 >> v2:172.21.15.173:3300/0 conn(0x7f03c80c6250 0x7f03c80c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.580+0000 7f03cee00640 1 -- 172.21.15.122:0/3892917871 >> 172.21.15.122:0/3892917871 conn(0x7f03c806dd30 msgr2=0x7f03c806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:55.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.580+0000 7f03cee00640 1 -- 172.21.15.122:0/3892917871 shutdown_connections 2024-09-17T05:27:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.580+0000 7f03cee00640 1 -- 172.21.15.122:0/3892917871 wait complete. 2024-09-17T05:27:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 Processor -- start 2024-09-17T05:27:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 -- start start 2024-09-17T05:27:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03c8073690 0x7f03c81460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03c80c6250 0x7f03c81fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c815e5c0 0x7f03c814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81682f0 con 0x7f03c80c6250 2024-09-17T05:27:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8168170 con 0x7f03c8073690 2024-09-17T05:27:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f03c8168470 con 0x7f03c815e5c0 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03c80c6250 0x7f03c81fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03c80c6250 0x7f03c81fbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43354/0 (socket says 172.21.15.122:43354) 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 -- 172.21.15.122:0/2321123923 learned_addr learned my addr 172.21.15.122:0/2321123923 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03ce400640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c815e5c0 0x7f03c814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cda00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03c8073690 0x7f03c81460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 -- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c815e5c0 msgr2=0x7f03c814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c815e5c0 0x7f03c814e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 -- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03c8073690 msgr2=0x7f03c81460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03c8073690 0x7f03c81460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 -- 172.21.15.122:0/2321123923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c815e3f0 con 0x7f03c80c6250 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03ce400640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c815e5c0 0x7f03c814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cd000640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03c80c6250 0x7f03c81fbd00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009850 tx=0x7f03b4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03c5600640 1 -- 172.21.15.122:0/2321123923 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40b2040 con 0x7f03c80c6250 2024-09-17T05:27:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03c5600640 1 -- 172.21.15.122:0/2321123923 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03b4004300 con 0x7f03c80c6250 2024-09-17T05:27:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03c5600640 1 -- 172.21.15.122:0/2321123923 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b40052d0 con 0x7f03c80c6250 2024-09-17T05:27:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c80ceb10 con 0x7f03c80c6250 2024-09-17T05:27:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c81f8680 con 0x7f03c80c6250 2024-09-17T05:27:55.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.584+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c8073af0 con 0x7f03c80c6250 2024-09-17T05:27:55.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.588+0000 7f03c5600640 1 -- 172.21.15.122:0/2321123923 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f03b4005470 con 0x7f03c80c6250 2024-09-17T05:27:55.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.588+0000 7f03c5600640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03a8073560 0x7f03a8075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:55.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.588+0000 7f03c5600640 1 -- 172.21.15.122:0/2321123923 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f03b412df70 con 0x7f03c80c6250 2024-09-17T05:27:55.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.588+0000 7f03cda00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03a8073560 0x7f03a8075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:55.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.588+0000 7f03cda00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03a8073560 0x7f03a8075a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009b70 tx=0x7f03bc0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:55.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.592+0000 7f03c5600640 1 -- 172.21.15.122:0/2321123923 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f03b40164d0 con 0x7f03c80c6250 2024-09-17T05:27:55.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.759+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f03c815be00 con 0x7f03c80c6250 2024-09-17T05:27:55.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.763+0000 7f03c5600640 1 -- 172.21.15.122:0/2321123923 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v30) ==== 71+0+0 (secure 0 0 0) 0x7f03b40f8d50 con 0x7f03c80c6250 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03a8073560 msgr2=0x7f03a8075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03a8073560 0x7f03a8075a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009b70 tx=0x7f03bc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03c80c6250 msgr2=0x7f03c81fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03c80c6250 0x7f03c81fbd00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009850 tx=0x7f03b4005e70 comp rx=0 tx=0).stop 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 shutdown_connections 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03a8073560 0x7f03a8075a20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03c815e5c0 0x7f03c814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03c80c6250 0x7f03c81fbd00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 --2- 172.21.15.122:0/2321123923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03c8073690 0x7f03c81460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 >> 172.21.15.122:0/2321123923 conn(0x7f03c806dd30 msgr2=0x7f03c815c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:55.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 shutdown_connections 2024-09-17T05:27:55.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.767+0000 7f03cee00640 1 -- 172.21.15.122:0/2321123923 wait complete. 2024-09-17T05:27:55.786 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 -- 172.21.15.122:0/1448142034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc15a660 msgr2=0x7f5edc0cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 --2- 172.21.15.122:0/1448142034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc15a660 0x7f5edc0cd100 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5edc070d60 tx=0x7f5ed001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 -- 172.21.15.122:0/1448142034 shutdown_connections 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 --2- 172.21.15.122:0/1448142034 >> v2:172.21.15.173:3300/0 conn(0x7f5edc0b42d0 0x7f5edc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 --2- 172.21.15.122:0/1448142034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5edc072d00 0x7f5edc073170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 --2- 172.21.15.122:0/1448142034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc15a660 0x7f5edc0cd100 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 -- 172.21.15.122:0/1448142034 >> 172.21.15.122:0/1448142034 conn(0x7f5edc0b18f0 msgr2=0x7f5edc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 -- 172.21.15.122:0/1448142034 shutdown_connections 2024-09-17T05:27:55.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 -- 172.21.15.122:0/1448142034 wait complete. 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 Processor -- start 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.935+0000 7f5ee4800640 1 -- start start 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc072d00 0x7f5edc14b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5edc0b42d0 0x7f5edc158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5edc15a660 0x7f5edc1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc073ef0 con 0x7f5edc072d00 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc073d70 con 0x7f5edc15a660 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc074070 con 0x7f5edc0b42d0 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc072d00 0x7f5edc14b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc072d00 0x7f5edc14b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43372/0 (socket says 172.21.15.122:43372) 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 -- 172.21.15.122:0/3449430519 learned_addr learned my addr 172.21.15.122:0/3449430519 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:55.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1400640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5edc0b42d0 0x7f5edc158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee2800640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5edc15a660 0x7f5edc1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 -- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5edc0b42d0 msgr2=0x7f5edc158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5edc0b42d0 0x7f5edc158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 -- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5edc15a660 msgr2=0x7f5edc1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5edc15a660 0x7f5edc1589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 -- 172.21.15.122:0/3449430519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc1fbd50 con 0x7f5edc072d00 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee1e00640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc072d00 0x7f5edc14b610 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00b0040 tx=0x7f5ed00a66b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ed6000640 1 -- 172.21.15.122:0/3449430519 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0004380 con 0x7f5edc072d00 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ed6000640 1 -- 172.21.15.122:0/3449430519 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ed0005020 con 0x7f5edc072d00 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ed6000640 1 -- 172.21.15.122:0/3449430519 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0005470 con 0x7f5edc072d00 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc09cac0 con 0x7f5edc072d00 2024-09-17T05:27:55.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc09d080 con 0x7f5edc072d00 2024-09-17T05:27:55.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc069410 con 0x7f5edc072d00 2024-09-17T05:27:55.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.939+0000 7f5ed6000640 1 -- 172.21.15.122:0/3449430519 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5ed00be060 con 0x7f5edc072d00 2024-09-17T05:27:55.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.943+0000 7f5ed6000640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ec0073590 0x7f5ec0075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:55.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.943+0000 7f5ed6000640 1 -- 172.21.15.122:0/3449430519 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f5ed012e8c0 con 0x7f5edc072d00 2024-09-17T05:27:55.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.943+0000 7f5ee1400640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ec0073590 0x7f5ec0075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:55.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.943+0000 7f5ee1400640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ec0073590 0x7f5ec0075a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0aeea0 tx=0x7f5ecc00a2d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:55.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:55.943+0000 7f5ed6000640 1 -- 172.21.15.122:0/3449430519 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5ed00164b0 con 0x7f5edc072d00 2024-09-17T05:27:56.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.171+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f5edc0b4740 con 0x7f5edc072d00 2024-09-17T05:27:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.319+0000 7f5ed6000640 1 -- 172.21.15.122:0/3449430519 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v147) ==== 184+0+0 (secure 0 0 0) 0x7f5ed00f9820 con 0x7f5edc072d00 2024-09-17T05:27:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ec0073590 msgr2=0x7f5ec0075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ec0073590 0x7f5ec0075a50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0aeea0 tx=0x7f5ecc00a2d0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc072d00 msgr2=0x7f5edc14b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc072d00 0x7f5edc14b610 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00b0040 tx=0x7f5ed00a66b0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 shutdown_connections 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ec0073590 0x7f5ec0075a50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5edc15a660 0x7f5edc1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5edc0b42d0 0x7f5edc158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 --2- 172.21.15.122:0/3449430519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5edc072d00 0x7f5edc14b610 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 >> 172.21.15.122:0/3449430519 conn(0x7f5edc0b18f0 msgr2=0x7f5edc1598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 shutdown_connections 2024-09-17T05:27:56.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.323+0000 7f5ee4800640 1 -- 172.21.15.122:0/3449430519 wait complete. 2024-09-17T05:27:56.344 INFO:tasks.cephfs_test_runner:Initiate cloning where clone subvolume and source subvolume are located ... ok 2024-09-17T05:27:56.344 INFO:tasks.cephfs_test_runner:Starting test: test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-17T05:27:56.344 INFO:tasks.cephfs_test_runner:test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:27:56.348 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:27:56.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- 172.21.15.122:0/556457991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84004810 msgr2=0x7f0a84004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 --2- 172.21.15.122:0/556457991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84004810 0x7f0a84004c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001890 tx=0x7f0a8801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa2200640 1 -- 172.21.15.122:0/556457991 reap_dead start 2024-09-17T05:27:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- 172.21.15.122:0/556457991 shutdown_connections 2024-09-17T05:27:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- 172.21.15.122:0/556457991 >> 172.21.15.122:0/556457991 conn(0x7f0a9c0d1dc0 msgr2=0x7f0a9c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- 172.21.15.122:0/556457991 shutdown_connections 2024-09-17T05:27:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- 172.21.15.122:0/556457991 wait complete. 2024-09-17T05:27:56.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 Processor -- start 2024-09-17T05:27:56.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- start start 2024-09-17T05:27:56.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84004810 0x7f0a9c0cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c0cbfb0 0x7f0a9c15ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1582b0 0x7f0a9c15f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c167d00 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c167b80 con 0x7f0a9c1582b0 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa4200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9c167e80 con 0x7f0a84004810 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c0cbfb0 0x7f0a9c15ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c0cbfb0 0x7f0a9c15ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43388/0 (socket says 172.21.15.122:43388) 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 -- 172.21.15.122:0/142475357 learned_addr learned my addr 172.21.15.122:0/142475357 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa1800640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84004810 0x7f0a9c0cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa2200640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1582b0 0x7f0a9c15f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:56.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 -- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84004810 msgr2=0x7f0a9c0cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84004810 0x7f0a9c0cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 -- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1582b0 msgr2=0x7f0a9c15f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1582b0 0x7f0a9c15f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 -- 172.21.15.122:0/142475357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c0b2990 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.491+0000 7f0aa0e00640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c0cbfb0 0x7f0a9c15ed20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001890 tx=0x7f0a88003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0a99600640 1 -- 172.21.15.122:0/142475357 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88001480 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0a99600640 1 -- 172.21.15.122:0/142475357 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a880a5880 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0a99600640 1 -- 172.21.15.122:0/142475357 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a880068d0 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c06e900 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c06eec0 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9c0cc9d0 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0a99600640 1 -- 172.21.15.122:0/142475357 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f0a880a5cd0 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0a99600640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a7c073500 0x7f0a7c0759c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:56.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0a99600640 1 -- 172.21.15.122:0/142475357 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f0a8812e3e0 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.495+0000 7f0aa1800640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a7c073500 0x7f0a7c0759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:56.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.499+0000 7f0aa1800640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a7c073500 0x7f0a7c0759c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0745d0 tx=0x7f0a900091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.499+0000 7f0a99600640 1 -- 172.21.15.122:0/142475357 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0a88016670 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.671+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a9c0d2370 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.671+0000 7f0a99600640 1 -- 172.21.15.122:0/142475357 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13094 (secure 0 0 0) 0x7f0a88117340 con 0x7f0a9c0cbfb0 2024-09-17T05:27:56.673 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:56.673 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":38,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:27:55.314509+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.173:6837/2399039177":"2024-09-18T05:27:52.303492+0000","172.21.15.173:6835/2399039177":"2024-09-18T05:27:52.303492+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:27:56.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a7c073500 msgr2=0x7f0a7c0759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a7c073500 0x7f0a7c0759c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0745d0 tx=0x7f0a900091e0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c0cbfb0 msgr2=0x7f0a9c15ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c0cbfb0 0x7f0a9c15ed20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001890 tx=0x7f0a88003c80 comp rx=0 tx=0).stop 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 shutdown_connections 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0a7c073500 0x7f0a7c0759c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1582b0 0x7f0a9c15f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c0cbfb0 0x7f0a9c15ed20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 --2- 172.21.15.122:0/142475357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84004810 0x7f0a9c0cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 >> 172.21.15.122:0/142475357 conn(0x7f0a9c0d1dc0 msgr2=0x7f0a9c15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 shutdown_connections 2024-09-17T05:27:56.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.675+0000 7f0aa4200640 1 -- 172.21.15.122:0/142475357 wait complete. 2024-09-17T05:27:56.694 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:27:56.694 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:27:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.899+0000 7ff52e000640 1 -- 172.21.15.122:0/2983785592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5280b0d10 msgr2=0x7ff5280d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.899+0000 7ff52e000640 1 --2- 172.21.15.122:0/2983785592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5280b0d10 0x7ff5280d1bb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff518009c80 tx=0x7ff51801cd60 comp rx=0 tx=0).stop 2024-09-17T05:27:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.899+0000 7ff52e000640 1 -- 172.21.15.122:0/2983785592 shutdown_connections 2024-09-17T05:27:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.899+0000 7ff52e000640 1 --2- 172.21.15.122:0/2983785592 >> v2:172.21.15.122:3300/0 conn(0x7ff5280d20f0 0x7ff5280cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.899+0000 7ff52e000640 1 --2- 172.21.15.122:0/2983785592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5280b0d10 0x7ff5280d1bb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.899+0000 7ff52e000640 1 -- 172.21.15.122:0/2983785592 >> 172.21.15.122:0/2983785592 conn(0x7ff52809aa10 msgr2=0x7ff52809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:56.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.903+0000 7ff52e000640 1 -- 172.21.15.122:0/2983785592 shutdown_connections 2024-09-17T05:27:56.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.903+0000 7ff52e000640 1 -- 172.21.15.122:0/2983785592 wait complete. 2024-09-17T05:27:56.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.903+0000 7ff52e000640 1 Processor -- start 2024-09-17T05:27:56.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.903+0000 7ff52e000640 1 -- start start 2024-09-17T05:27:56.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.903+0000 7ff52e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5280b0d10 0x7ff5280b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:56.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5280d20f0 0x7ff5280b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:56.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52e000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff52815bd40 0x7ff528159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:56.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52e000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff528168cb0 con 0x7ff5280d20f0 2024-09-17T05:27:56.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52e000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff528168b30 con 0x7ff5280b0d10 2024-09-17T05:27:56.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52e000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff528168e30 con 0x7ff52815bd40 2024-09-17T05:27:56.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52cc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5280b0d10 0x7ff5280b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:56.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff52815bd40 0x7ff528159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:56.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff527e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5280d20f0 0x7ff5280b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:56.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff52815bd40 0x7ff528159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:50450/0 (socket says 172.21.15.122:50450) 2024-09-17T05:27:56.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52cc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5280b0d10 0x7ff5280b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:44324/0 (socket says 172.21.15.122:44324) 2024-09-17T05:27:56.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 -- 172.21.15.122:0/724221666 learned_addr learned my addr 172.21.15.122:0/724221666 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:56.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 -- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5280b0d10 msgr2=0x7ff5280b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5280b0d10 0x7ff5280b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 -- 172.21.15.122:0/724221666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5280d20f0 msgr2=0x7ff5280b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:56.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5280d20f0 0x7ff5280b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:56.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 -- 172.21.15.122:0/724221666 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5280b71e0 con 0x7ff52815bd40 2024-09-17T05:27:56.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff52d600640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff52815bd40 0x7ff528159f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff51000b410 tx=0x7ff51000b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:56.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff524c00640 1 -- 172.21.15.122:0/724221666 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510015020 con 0x7ff52815bd40 2024-09-17T05:27:56.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.907+0000 7ff524c00640 1 -- 172.21.15.122:0/724221666 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff510011920 con 0x7ff52815bd40 2024-09-17T05:27:56.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.911+0000 7ff524c00640 1 -- 172.21.15.122:0/724221666 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510011dd0 con 0x7ff52815bd40 2024-09-17T05:27:56.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.911+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5280b6f00 con 0x7ff52815bd40 2024-09-17T05:27:56.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.911+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5280b4e30 con 0x7ff52815bd40 2024-09-17T05:27:56.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.911+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5280d1bb0 con 0x7ff52815bd40 2024-09-17T05:27:56.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.915+0000 7ff524c00640 1 -- 172.21.15.122:0/724221666 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7ff510018020 con 0x7ff52815bd40 2024-09-17T05:27:56.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.915+0000 7ff524c00640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff504073630 0x7ff504075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:56.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.915+0000 7ff524c00640 1 -- 172.21.15.122:0/724221666 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7ff5100948e0 con 0x7ff52815bd40 2024-09-17T05:27:56.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.915+0000 7ff52cc00640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff504073630 0x7ff504075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:56.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.919+0000 7ff524c00640 1 -- 172.21.15.122:0/724221666 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff51005f1f0 con 0x7ff52815bd40 2024-09-17T05:27:56.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:56.919+0000 7ff52cc00640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff504073630 0x7ff504075af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5280b6390 tx=0x7ff520009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:57.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.155+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff528071800 con 0x7ff52815bd40 2024-09-17T05:27:57.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.155+0000 7ff524c00640 1 -- 172.21.15.122:0/724221666 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7ff510063c20 con 0x7ff52815bd40 2024-09-17T05:27:57.157 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:27:57.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff504073630 msgr2=0x7ff504075af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff504073630 0x7ff504075af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5280b6390 tx=0x7ff520009060 comp rx=0 tx=0).stop 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff52815bd40 msgr2=0x7ff528159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff52815bd40 0x7ff528159f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff51000b410 tx=0x7ff51000b8d0 comp rx=0 tx=0).stop 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 shutdown_connections 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff504073630 0x7ff504075af0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff52815bd40 0x7ff528159f40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5280d20f0 0x7ff5280b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 --2- 172.21.15.122:0/724221666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5280b0d10 0x7ff5280b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 >> 172.21.15.122:0/724221666 conn(0x7ff52809aa10 msgr2=0x7ff52806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 shutdown_connections 2024-09-17T05:27:57.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.159+0000 7ff52e000640 1 -- 172.21.15.122:0/724221666 wait complete. 2024-09-17T05:27:57.178 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-17T05:27:57.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 -- 172.21.15.122:0/894008305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3c06f8b0 msgr2=0x7fdd3c15dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:57.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 --2- 172.21.15.122:0/894008305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3c06f8b0 0x7fdd3c15dae0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009840 tx=0x7fdd2c01c700 comp rx=0 tx=0).stop 2024-09-17T05:27:57.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 -- 172.21.15.122:0/894008305 shutdown_connections 2024-09-17T05:27:57.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 --2- 172.21.15.122:0/894008305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3c06f8b0 0x7fdd3c15dae0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:57.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 -- 172.21.15.122:0/894008305 >> 172.21.15.122:0/894008305 conn(0x7fdd3c0b18f0 msgr2=0x7fdd3c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:57.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 -- 172.21.15.122:0/894008305 shutdown_connections 2024-09-17T05:27:57.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 -- 172.21.15.122:0/894008305 wait complete. 2024-09-17T05:27:57.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 Processor -- start 2024-09-17T05:27:57.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.319+0000 7fdd42c00640 1 -- start start 2024-09-17T05:27:57.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd42c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3c06f8b0 0x7fdd3c09d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd42c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd3c0acd40 0x7fdd3c149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd42c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd3c0cd100 0x7fdd3c14a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd42c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c1692c0 con 0x7fdd3c0cd100 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd42c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c169140 con 0x7fdd3c06f8b0 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd42c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c169440 con 0x7fdd3c0acd40 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd3c0cd100 0x7fdd3c14a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd3c0cd100 0x7fdd3c14a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43394/0 (socket says 172.21.15.122:43394) 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 -- 172.21.15.122:0/2731547556 learned_addr learned my addr 172.21.15.122:0/2731547556 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd3be00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3c06f8b0 0x7fdd3c09d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd33400640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd3c0acd40 0x7fdd3c149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 -- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd3c0acd40 msgr2=0x7fdd3c149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd3c0acd40 0x7fdd3c149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 -- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3c06f8b0 msgr2=0x7fdd3c09d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3c06f8b0 0x7fdd3c09d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 -- 172.21.15.122:0/2731547556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c09c340 con 0x7fdd3c0cd100 2024-09-17T05:27:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd33400640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd3c0acd40 0x7fdd3c149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:27:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd40c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd3c0cd100 0x7fdd3c14a9d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400cbc0 tx=0x7fdd3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd38c00640 1 -- 172.21.15.122:0/2731547556 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34016020 con 0x7fdd3c0cd100 2024-09-17T05:27:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c09c060 con 0x7fdd3c0cd100 2024-09-17T05:27:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd38c00640 1 -- 172.21.15.122:0/2731547556 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd3400ab00 con 0x7fdd3c0cd100 2024-09-17T05:27:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd38c00640 1 -- 172.21.15.122:0/2731547556 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34012420 con 0x7fdd3c0cd100 2024-09-17T05:27:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.323+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c14a1d0 con 0x7fdd3c0cd100 2024-09-17T05:27:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.327+0000 7fdd38c00640 1 -- 172.21.15.122:0/2731547556 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fdd34019020 con 0x7fdd3c0cd100 2024-09-17T05:27:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.327+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c14a6c0 con 0x7fdd3c0cd100 2024-09-17T05:27:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.327+0000 7fdd38c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd18073560 0x7fdd18075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:57.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.327+0000 7fdd38c00640 1 -- 172.21.15.122:0/2731547556 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fdd34094890 con 0x7fdd3c0cd100 2024-09-17T05:27:57.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.331+0000 7fdd3be00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd18073560 0x7fdd18075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:57.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.331+0000 7fdd3be00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd18073560 0x7fdd18075a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0af000 tx=0x7fdd2c01af40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:57.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.335+0000 7fdd38c00640 1 -- 172.21.15.122:0/2731547556 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdd34020020 con 0x7fdd3c0cd100 2024-09-17T05:27:57.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:57.627+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7fdd3c069410 con 0x7fdd3c0cd100 2024-09-17T05:27:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.335+0000 7fdd38c00640 1 -- 172.21.15.122:0/2731547556 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v149) ==== 169+0+0 (secure 0 0 0) 0x7fdd3c09c340 con 0x7fdd3c0cd100 2024-09-17T05:27:58.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd18073560 msgr2=0x7fdd18075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd18073560 0x7fdd18075a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0af000 tx=0x7fdd2c01af40 comp rx=0 tx=0).stop 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd3c0cd100 msgr2=0x7fdd3c14a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd3c0cd100 0x7fdd3c14a9d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400cbc0 tx=0x7fdd3400ccb0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 shutdown_connections 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd18073560 0x7fdd18075a20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd3c0cd100 0x7fdd3c14a9d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd3c0acd40 0x7fdd3c149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 --2- 172.21.15.122:0/2731547556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3c06f8b0 0x7fdd3c09d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 >> 172.21.15.122:0/2731547556 conn(0x7fdd3c0b18f0 msgr2=0x7fdd3c1550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 shutdown_connections 2024-09-17T05:27:58.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.339+0000 7fdd42c00640 1 -- 172.21.15.122:0/2731547556 wait complete. 2024-09-17T05:27:58.362 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:27:58.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.503+0000 7fbc55a00640 1 -- 172.21.15.122:0/3521290625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 msgr2=0x7fbc500ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.503+0000 7fbc55a00640 1 --2- 172.21.15.122:0/3521290625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 0x7fbc500ce670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009840 tx=0x7fbc4001c790 comp rx=0 tx=0).stop 2024-09-17T05:27:58.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.503+0000 7fbc55a00640 1 -- 172.21.15.122:0/3521290625 shutdown_connections 2024-09-17T05:27:58.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.503+0000 7fbc55a00640 1 --2- 172.21.15.122:0/3521290625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 0x7fbc500ce670 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.503+0000 7fbc55a00640 1 -- 172.21.15.122:0/3521290625 >> 172.21.15.122:0/3521290625 conn(0x7fbc500d1dc0 msgr2=0x7fbc500d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:58.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.503+0000 7fbc55a00640 1 -- 172.21.15.122:0/3521290625 shutdown_connections 2024-09-17T05:27:58.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.503+0000 7fbc55a00640 1 -- 172.21.15.122:0/3521290625 wait complete. 2024-09-17T05:27:58.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 Processor -- start 2024-09-17T05:27:58.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 -- start start 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc500cdb30 0x7fbc50151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc500d28e0 0x7fbc50071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 0x7fbc50072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5016a930 con 0x7fbc500cdb30 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5016a7b0 con 0x7fbc5015bea0 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5016aab0 con 0x7fbc500d28e0 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc500cdb30 0x7fbc50151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc500cdb30 0x7fbc50151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43396/0 (socket says 172.21.15.122:43396) 2024-09-17T05:27:58.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 -- 172.21.15.122:0/3579856144 learned_addr learned my addr 172.21.15.122:0/3579856144 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:58.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 -- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc500d28e0 msgr2=0x7fbc50071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc500d28e0 0x7fbc50071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 -- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 msgr2=0x7fbc50072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4f400640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 0x7fbc50072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 0x7fbc50072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 -- 172.21.15.122:0/3579856144 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc501fc520 con 0x7fbc500cdb30 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4f400640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 0x7fbc50072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc4ea00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc500cdb30 0x7fbc50151240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbc40000c10 tx=0x7fbc400a65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc46a00640 1 -- 172.21.15.122:0/3579856144 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40005030 con 0x7fbc500cdb30 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc501fb9a0 con 0x7fbc500cdb30 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc46a00640 1 -- 172.21.15.122:0/3579856144 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc400051d0 con 0x7fbc500cdb30 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc46a00640 1 -- 172.21.15.122:0/3579856144 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc400055c0 con 0x7fbc500cdb30 2024-09-17T05:27:58.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc501fbf60 con 0x7fbc500cdb30 2024-09-17T05:27:58.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.507+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc50151cb0 con 0x7fbc500cdb30 2024-09-17T05:27:58.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.511+0000 7fbc46a00640 1 -- 172.21.15.122:0/3579856144 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fbc400b8030 con 0x7fbc500cdb30 2024-09-17T05:27:58.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.511+0000 7fbc46a00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc2c073560 0x7fbc2c075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:58.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.511+0000 7fbc46a00640 1 -- 172.21.15.122:0/3579856144 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fbc4012de80 con 0x7fbc500cdb30 2024-09-17T05:27:58.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.511+0000 7fbc47e00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc2c073560 0x7fbc2c075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:58.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.511+0000 7fbc47e00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc2c073560 0x7fbc2c075a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc38007fd0 tx=0x7fbc38007980 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:58.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.515+0000 7fbc46a00640 1 -- 172.21.15.122:0/3579856144 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbc400bf070 con 0x7fbc500cdb30 2024-09-17T05:27:58.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbc50154df0 con 0x7fbc500cdb30 2024-09-17T05:27:58.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc46a00640 1 -- 172.21.15.122:0/3579856144 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) ==== 59+0+1998 (secure 0 0 0) 0x7fbc400ac030 con 0x7fbc500cdb30 2024-09-17T05:27:58.683 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:27:58.683 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:27:58.684 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:27:58.684 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:27:58.684 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:27:58.684 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:27:58.684 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:27:58.684 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:27:58.684 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:27:58.684 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:27:58.685 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:27:58.686 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:27:58.686 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:27:58.686 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:27:58.686 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:27:58.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc2c073560 msgr2=0x7fbc2c075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc2c073560 0x7fbc2c075a20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc38007fd0 tx=0x7fbc38007980 comp rx=0 tx=0).stop 2024-09-17T05:27:58.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc500cdb30 msgr2=0x7fbc50151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc500cdb30 0x7fbc50151240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbc40000c10 tx=0x7fbc400a65a0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 shutdown_connections 2024-09-17T05:27:58.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc2c073560 0x7fbc2c075a20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc5015bea0 0x7fbc50072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc500d28e0 0x7fbc50071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 --2- 172.21.15.122:0/3579856144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc500cdb30 0x7fbc50151240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 >> 172.21.15.122:0/3579856144 conn(0x7fbc500d1dc0 msgr2=0x7fbc50154070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:58.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 shutdown_connections 2024-09-17T05:27:58.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.683+0000 7fbc55a00640 1 -- 172.21.15.122:0/3579856144 wait complete. 2024-09-17T05:27:58.705 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:27:58.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.847+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2409333423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 msgr2=0x7f1adc0cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.847+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2409333423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 0x7f1adc0cd950 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1adc073a20 tx=0x7f1ad401c7a0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.847+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2409333423 shutdown_connections 2024-09-17T05:27:58.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.847+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2409333423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 0x7f1adc0cd950 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.847+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2409333423 >> v2:172.21.15.173:3300/0 conn(0x7f1adc15bd50 0x7f1adc0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.847+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2409333423 >> 172.21.15.122:0/2409333423 conn(0x7f1adc0d1ea0 msgr2=0x7f1adc0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:58.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.847+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2409333423 shutdown_connections 2024-09-17T05:27:58.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.847+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2409333423 wait complete. 2024-09-17T05:27:58.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 Processor -- start 2024-09-17T05:27:58.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 -- start start 2024-09-17T05:27:58.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 0x7f1adc1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:58.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1adc15bd50 0x7f1adc0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:58.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1adc15c130 0x7f1adc1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:58.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc168ea0 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc168d20 con 0x7f1adc15bd50 2024-09-17T05:27:58.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1adc169020 con 0x7f1adc15c130 2024-09-17T05:27:58.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 0x7f1adc1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:58.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 0x7f1adc1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43416/0 (socket says 172.21.15.122:43416) 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 -- 172.21.15.122:0/2301973682 learned_addr learned my addr 172.21.15.122:0/2301973682 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adb400640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1adc15c130 0x7f1adc1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 -- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1adc15c130 msgr2=0x7f1adc1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ad2000640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1adc15bd50 0x7f1adc0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1adc15c130 0x7f1adc1519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 -- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1adc15bd50 msgr2=0x7f1adc0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1adc15bd50 0x7f1adc0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 -- 172.21.15.122:0/2301973682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc0b72f0 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adb400640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1adc15c130 0x7f1adc1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ad2000640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1adc15bd50 0x7f1adc0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1adaa00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 0x7f1adc1512e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00ec00 tx=0x7f1acc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ad3400640 1 -- 172.21.15.122:0/2301973682 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc010630 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ad3400640 1 -- 172.21.15.122:0/2301973682 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1acc0108c0 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1adc0b75d0 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.851+0000 7f1ad3400640 1 -- 172.21.15.122:0/2301973682 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc010cf0 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.855+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1adc154e00 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.855+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1adc0ce5e0 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.855+0000 7f1ad3400640 1 -- 172.21.15.122:0/2301973682 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f1acc018420 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.855+0000 7f1ad3400640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1ab8073560 0x7f1ab8075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:58.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.855+0000 7f1ad3400640 1 -- 172.21.15.122:0/2301973682 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f1acc095360 con 0x7f1adc0cf4e0 2024-09-17T05:27:58.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.855+0000 7f1ad2000640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1ab8073560 0x7f1ab8075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:58.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.859+0000 7f1ad2000640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1ab8073560 0x7f1ab8075a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0cd750 tx=0x7f1ac4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:58.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:58.859+0000 7f1ad3400640 1 -- 172.21.15.122:0/2301973682 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1acc060240 con 0x7f1adc0cf4e0 2024-09-17T05:27:59.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.027+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1adc155530 con 0x7f1adc0cf4e0 2024-09-17T05:27:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.031+0000 7f1ad3400640 1 -- 172.21.15.122:0/2301973682 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7f1acc05fb40 con 0x7f1adc0cf4e0 2024-09-17T05:27:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1ab8073560 msgr2=0x7f1ab8075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1ab8073560 0x7f1ab8075a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0cd750 tx=0x7f1ac4009040 comp rx=0 tx=0).stop 2024-09-17T05:27:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 msgr2=0x7f1adc1512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 0x7f1adc1512e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00ec00 tx=0x7f1acc00c6a0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 shutdown_connections 2024-09-17T05:27:59.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1ab8073560 0x7f1ab8075a20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1adc15c130 0x7f1adc1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1adc15bd50 0x7f1adc0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 --2- 172.21.15.122:0/2301973682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc0cf4e0 0x7f1adc1512e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 >> 172.21.15.122:0/2301973682 conn(0x7f1adc0d1ea0 msgr2=0x7f1adc0d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 shutdown_connections 2024-09-17T05:27:59.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.035+0000 7f1ae1a00640 1 -- 172.21.15.122:0/2301973682 wait complete. 2024-09-17T05:27:59.055 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:27:59.055 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:27:59.160 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:27:59.250 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:27:59.250 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:27:59.250 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:59.259 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:27:59.259 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:27:59.259 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:59.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:27:59.268 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:27:59.268 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:59.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:27:59.310 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:27:59.310 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:27:59.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:27:59.319 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:27:59.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:27:59.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f216ca00640 1 -- 172.21.15.122:0/335270415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c004810 msgr2=0x7f214c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f216ca00640 1 --2- 172.21.15.122:0/335270415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c004810 0x7f214c004c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2150009b60 tx=0x7f215001d240 comp rx=0 tx=0).stop 2024-09-17T05:27:59.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f2166800640 1 -- 172.21.15.122:0/335270415 reap_dead start 2024-09-17T05:27:59.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f216ca00640 1 -- 172.21.15.122:0/335270415 shutdown_connections 2024-09-17T05:27:59.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f216ca00640 1 -- 172.21.15.122:0/335270415 >> 172.21.15.122:0/335270415 conn(0x7f21680d1e30 msgr2=0x7f21680d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:59.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f216ca00640 1 -- 172.21.15.122:0/335270415 shutdown_connections 2024-09-17T05:27:59.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f216ca00640 1 -- 172.21.15.122:0/335270415 wait complete. 2024-09-17T05:27:59.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f216ca00640 1 Processor -- start 2024-09-17T05:27:59.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.495+0000 7f216ca00640 1 -- start start 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c004810 0x7f2168150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21681fb890 0x7f21680ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21680ce980 0x7f216815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2168166df0 con 0x7f21681fb890 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2168166c70 con 0x7f21680ce980 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2168166f70 con 0x7f214c004810 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21681fb890 0x7f21680ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21681fb890 0x7f21680ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43440/0 (socket says 172.21.15.122:43440) 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 -- 172.21.15.122:0/794704229 learned_addr learned my addr 172.21.15.122:0/794704229 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f2165e00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c004810 0x7f2168150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f2166800640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21680ce980 0x7f216815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 -- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c004810 msgr2=0x7f2168150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c004810 0x7f2168150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 -- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21680ce980 msgr2=0x7f216815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21680ce980 0x7f216815d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 -- 172.21.15.122:0/794704229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21680cde80 con 0x7f21681fb890 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215d400640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21681fb890 0x7f21680ce440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f215001d070 tx=0x7f2150005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215ea00640 1 -- 172.21.15.122:0/794704229 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21500ba020 con 0x7f21681fb890 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215ea00640 1 -- 172.21.15.122:0/794704229 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21500ac8a0 con 0x7f21681fb890 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215ea00640 1 -- 172.21.15.122:0/794704229 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21500b1040 con 0x7f21681fb890 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21680ce160 con 0x7f21681fb890 2024-09-17T05:27:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2168151280 con 0x7f21681fb890 2024-09-17T05:27:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21680b14c0 con 0x7f21681fb890 2024-09-17T05:27:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.499+0000 7f215ea00640 1 -- 172.21.15.122:0/794704229 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f21500bd020 con 0x7f21681fb890 2024-09-17T05:27:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.503+0000 7f215ea00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2144073590 0x7f2144075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.503+0000 7f215ea00640 1 -- 172.21.15.122:0/794704229 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f215012f1e0 con 0x7f21681fb890 2024-09-17T05:27:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.503+0000 7f2165e00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2144073590 0x7f2144075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.503+0000 7f2165e00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2144073590 0x7f2144075a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f216815a9a0 tx=0x7f215800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:59.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.507+0000 7f215ea00640 1 -- 172.21.15.122:0/794704229 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f21500c4030 con 0x7f21681fb890 2024-09-17T05:27:59.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.707+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2168164d70 con 0x7f21681fb890 2024-09-17T05:27:59.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.707+0000 7f215ea00640 1 -- 172.21.15.122:0/794704229 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3923 (secure 0 0 0) 0x7f21500a5e20 con 0x7f21681fb890 2024-09-17T05:27:59.709 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:27:59.709 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":19,"btime":"2024-09-17T05:27:53:306110+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2936956280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2936956280},{"type":"v1","addr":"172.21.15.173:6836","nonce":2936956280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2308148662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2308148662},{"type":"v1","addr":"172.21.15.122:6834","nonce":2308148662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3699957348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":3699957348},{"type":"v1","addr":"172.21.15.122:6835","nonce":3699957348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5292,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1758271269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1758271269},{"type":"v1","addr":"172.21.15.173:6837","nonce":1758271269}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-09-17T05:27:59.709 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 19 2024-09-17T05:27:59.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2144073590 msgr2=0x7f2144075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2144073590 0x7f2144075a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f216815a9a0 tx=0x7f215800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21681fb890 msgr2=0x7f21680ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21681fb890 0x7f21680ce440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f215001d070 tx=0x7f2150005620 comp rx=0 tx=0).stop 2024-09-17T05:27:59.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 shutdown_connections 2024-09-17T05:27:59.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2144073590 0x7f2144075a50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21680ce980 0x7f216815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21681fb890 0x7f21680ce440 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 --2- 172.21.15.122:0/794704229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c004810 0x7f2168150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 >> 172.21.15.122:0/794704229 conn(0x7f21680d1e30 msgr2=0x7f21680cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:59.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 shutdown_connections 2024-09-17T05:27:59.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.711+0000 7f216ca00640 1 -- 172.21.15.122:0/794704229 wait complete. 2024-09-17T05:27:59.731 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:27:59.731 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:27:59.732 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:27:59.732 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:27:59.732 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:27:59.732 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:27:59.732 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:27:59.732 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:27:59.732 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:27:59.733 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:27:59.733 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:27:59.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:27:59.733 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:27:59.733 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:27:59.733 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:27:59.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:27:59.743 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:27:59.743 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:27:59.744 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:27:59.744 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:27:59.748 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:27:59.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:27:59.749 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:27:59.749 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:27:59.788 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:27:59.789 DEBUG:teuthology.parallel:result is None 2024-09-17T05:27:59.790 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:27:59.790 DEBUG:teuthology.parallel:result is None 2024-09-17T05:27:59.793 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:27:59.794 DEBUG:teuthology.parallel:result is None 2024-09-17T05:27:59.795 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:27:59.795 DEBUG:teuthology.parallel:result is None 2024-09-17T05:27:59.795 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:27:59.876 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:27:59.877 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:27:59.884 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:27:59.884 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:27:59.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.943+0000 7f9a0d400640 1 -- 172.21.15.122:0/3509635488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a0806ecb0 msgr2=0x7f9a080b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.943+0000 7f9a0d400640 1 --2- 172.21.15.122:0/3509635488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a0806ecb0 0x7f9a080b4a60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9a080af3d0 tx=0x7f99f001cd80 comp rx=0 tx=0).stop 2024-09-17T05:27:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.943+0000 7f9a0d400640 1 -- 172.21.15.122:0/3509635488 shutdown_connections 2024-09-17T05:27:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.943+0000 7f9a0d400640 1 --2- 172.21.15.122:0/3509635488 >> v2:172.21.15.173:3300/0 conn(0x7f9a080b2bf0 0x7f9a080b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.943+0000 7f9a0d400640 1 --2- 172.21.15.122:0/3509635488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a0806ecb0 0x7f9a080b4a60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.943+0000 7f9a0d400640 1 -- 172.21.15.122:0/3509635488 >> 172.21.15.122:0/3509635488 conn(0x7f9a080b18f0 msgr2=0x7f9a080b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:27:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.943+0000 7f9a0d400640 1 -- 172.21.15.122:0/3509635488 shutdown_connections 2024-09-17T05:27:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.943+0000 7f9a0d400640 1 -- 172.21.15.122:0/3509635488 wait complete. 2024-09-17T05:27:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 Processor -- start 2024-09-17T05:27:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 -- start start 2024-09-17T05:27:59.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a0806ecb0 0x7f9a08150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:59.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a080b2bf0 0x7f9a08072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:59.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a08158a20 0x7f9a0814f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:59.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08167b80 con 0x7f9a08158a20 2024-09-17T05:27:59.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08167a00 con 0x7f9a080b2bf0 2024-09-17T05:27:59.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9a08167d00 con 0x7f9a0806ecb0 2024-09-17T05:27:59.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a08158a20 0x7f9a0814f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:59.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a06800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a0806ecb0 0x7f9a08150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:59.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a08158a20 0x7f9a0814f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43482/0 (socket says 172.21.15.122:43482) 2024-09-17T05:27:59.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 -- 172.21.15.122:0/828293307 learned_addr learned my addr 172.21.15.122:0/828293307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:27:59.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f99fde00640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a080b2bf0 0x7f9a08072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:59.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 -- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a0806ecb0 msgr2=0x7f9a08150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a0806ecb0 0x7f9a08150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 -- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a080b2bf0 msgr2=0x7f9a08072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:27:59.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a080b2bf0 0x7f9a08072870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:27:59.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 -- 172.21.15.122:0/828293307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a081508b0 con 0x7f9a08158a20 2024-09-17T05:27:59.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f99fde00640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a080b2bf0 0x7f9a08072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:27:59.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a07200640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a08158a20 0x7f9a0814f7e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e8c0 tx=0x7f9a0000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:59.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f99ff400640 1 -- 172.21.15.122:0/828293307 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000ccc0 con 0x7f9a08158a20 2024-09-17T05:27:59.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f99ff400640 1 -- 172.21.15.122:0/828293307 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a0000ce60 con 0x7f9a08158a20 2024-09-17T05:27:59.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f99ff400640 1 -- 172.21.15.122:0/828293307 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00005280 con 0x7f9a08158a20 2024-09-17T05:27:59.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a08156bb0 con 0x7f9a08158a20 2024-09-17T05:27:59.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.947+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a08156e70 con 0x7f9a08158a20 2024-09-17T05:27:59.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.951+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a08069410 con 0x7f9a08158a20 2024-09-17T05:27:59.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.951+0000 7f99ff400640 1 -- 172.21.15.122:0/828293307 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f9a00005420 con 0x7f9a08158a20 2024-09-17T05:27:59.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.951+0000 7f99ff400640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99e4073590 0x7f99e4075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:27:59.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.951+0000 7f99ff400640 1 -- 172.21.15.122:0/828293307 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f9a00094f70 con 0x7f9a08158a20 2024-09-17T05:27:59.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.951+0000 7f9a06800640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99e4073590 0x7f99e4075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:27:59.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.951+0000 7f9a06800640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99e4073590 0x7f99e4075a50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9a0809daf0 tx=0x7f99f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:27:59.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:27:59.955+0000 7f99ff400640 1 -- 172.21.15.122:0/828293307 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9a081508b0 con 0x7f9a08158a20 2024-09-17T05:28:00.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.167+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9a081657d0 con 0x7f9a08158a20 2024-09-17T05:28:00.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.355+0000 7f99ff400640 1 -- 172.21.15.122:0/828293307 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) ==== 97+0+0 (secure 0 0 0) 0x7f9a0005f700 con 0x7f9a08158a20 2024-09-17T05:28:00.357 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:28:00.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99e4073590 msgr2=0x7f99e4075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99e4073590 0x7f99e4075a50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9a0809daf0 tx=0x7f99f8009040 comp rx=0 tx=0).stop 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a08158a20 msgr2=0x7f9a0814f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a08158a20 0x7f9a0814f7e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e8c0 tx=0x7f9a0000ed80 comp rx=0 tx=0).stop 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 shutdown_connections 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99e4073590 0x7f99e4075a50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a08158a20 0x7f9a0814f7e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a080b2bf0 0x7f9a08072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 --2- 172.21.15.122:0/828293307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a0806ecb0 0x7f9a08150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 >> 172.21.15.122:0/828293307 conn(0x7f9a080b18f0 msgr2=0x7f9a08158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 shutdown_connections 2024-09-17T05:28:00.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.359+0000 7f9a0d400640 1 -- 172.21.15.122:0/828293307 wait complete. 2024-09-17T05:28:00.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:28:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 -- 172.21.15.122:0/1788107266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b40152860 msgr2=0x7f5b400cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/1788107266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b40152860 0x7f5b400cc760 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b400cdb90 tx=0x7f5b3001c7a0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 -- 172.21.15.122:0/1788107266 shutdown_connections 2024-09-17T05:28:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/1788107266 >> v2:172.21.15.173:3300/0 conn(0x7f5b40074840 0x7f5b4006ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/1788107266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b40152860 0x7f5b400cc760 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 -- 172.21.15.122:0/1788107266 >> 172.21.15.122:0/1788107266 conn(0x7f5b4006cc20 msgr2=0x7f5b4006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 -- 172.21.15.122:0/1788107266 shutdown_connections 2024-09-17T05:28:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 -- 172.21.15.122:0/1788107266 wait complete. 2024-09-17T05:28:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 Processor -- start 2024-09-17T05:28:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.527+0000 7f5b3fe00640 1 -- start start 2024-09-17T05:28:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b40074840 0x7f5b40164d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b400ccca0 0x7f5b4014ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40152860 0x7f5b40154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40169020 con 0x7f5b400ccca0 2024-09-17T05:28:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40168ea0 con 0x7f5b40074840 2024-09-17T05:28:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5b401691a0 con 0x7f5b40152860 2024-09-17T05:28:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b400ccca0 0x7f5b4014ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b400ccca0 0x7f5b4014ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43504/0 (socket says 172.21.15.122:43504) 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 -- 172.21.15.122:0/2720293721 learned_addr learned my addr 172.21.15.122:0/2720293721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3ea00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b40074840 0x7f5b40164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3f400640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40152860 0x7f5b40154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 -- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40152860 msgr2=0x7f5b40154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40152860 0x7f5b40154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 -- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b40074840 msgr2=0x7f5b40164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b40074840 0x7f5b40164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 -- 172.21.15.122:0/2720293721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4014cff0 con 0x7f5b400ccca0 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3f400640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40152860 0x7f5b40154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3e000640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b400ccca0 0x7f5b4014ae00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00c4b0 tx=0x7f5b2c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b36a00640 1 -- 172.21.15.122:0/2720293721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0052d0 con 0x7f5b400ccca0 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b36a00640 1 -- 172.21.15.122:0/2720293721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b2c005470 con 0x7f5b400ccca0 2024-09-17T05:28:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4014d2d0 con 0x7f5b400ccca0 2024-09-17T05:28:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b36a00640 1 -- 172.21.15.122:0/2720293721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c012870 con 0x7f5b400ccca0 2024-09-17T05:28:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4014d890 con 0x7f5b400ccca0 2024-09-17T05:28:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.531+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4006ec80 con 0x7f5b400ccca0 2024-09-17T05:28:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.535+0000 7f5b36a00640 1 -- 172.21.15.122:0/2720293721 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5b2c003600 con 0x7f5b400ccca0 2024-09-17T05:28:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.535+0000 7f5b36a00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b20073590 0x7f5b20075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.535+0000 7f5b36a00640 1 -- 172.21.15.122:0/2720293721 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f5b2c0940a0 con 0x7f5b400ccca0 2024-09-17T05:28:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.535+0000 7f5b3ea00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b20073590 0x7f5b20075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.535+0000 7f5b3ea00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b20073590 0x7f5b20075a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b300af000 tx=0x7f5b3001afe0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.539+0000 7f5b36a00640 1 -- 172.21.15.122:0/2720293721 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5b2c05ef10 con 0x7f5b400ccca0 2024-09-17T05:28:00.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.763+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5b400d0910 con 0x7f5b400ccca0 2024-09-17T05:28:00.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.763+0000 7f5b36a00640 1 -- 172.21.15.122:0/2720293721 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f5b2c05e810 con 0x7f5b400ccca0 2024-09-17T05:28:00.765 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:00.765 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:28:00.766 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:28:00.767 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:28:00.768 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.769 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:28:00.769 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:28:00.769 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:28:00.769 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.769 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:28:00.770 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:28:00.770 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.770 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:00.770 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:28:00.770 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.770 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.771 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:28:00.771 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:28:00.771 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.771 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:00.771 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:28:00.771 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.772 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:28:00.772 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:28:00.772 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.772 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:00.772 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:28:00.772 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.772 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.773 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.774 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:28:00.774 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:28:00.774 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.774 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:00.774 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:00.775 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:28:00.776 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.776 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:28:00.776 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:28:00.776 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.776 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:00.777 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:28:00.777 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.777 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.777 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:28:00.777 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:28:00.778 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:28:00.779 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.779 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:28:00.779 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow r", 2024-09-17T05:28:00.779 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:28:00.779 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.779 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:28:00.780 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:28:00.780 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.780 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:28:00.780 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:28:00.780 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:28:00.780 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:28:00.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:28:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:28:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:28:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:28:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:28:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:28:00.782 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.783 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:28:00.784 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:28:00.785 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:28:00.785 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.785 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:00.785 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:28:00.785 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:28:00.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b20073590 msgr2=0x7f5b20075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b20073590 0x7f5b20075a50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b300af000 tx=0x7f5b3001afe0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b400ccca0 msgr2=0x7f5b4014ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b400ccca0 0x7f5b4014ae00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00c4b0 tx=0x7f5b2c00c970 comp rx=0 tx=0).stop 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 shutdown_connections 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b20073590 0x7f5b20075a50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40152860 0x7f5b40154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b400ccca0 0x7f5b4014ae00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 --2- 172.21.15.122:0/2720293721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b40074840 0x7f5b40164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 >> 172.21.15.122:0/2720293721 conn(0x7f5b4006cc20 msgr2=0x7f5b40072a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 shutdown_connections 2024-09-17T05:28:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.767+0000 7f5b3fe00640 1 -- 172.21.15.122:0/2720293721 wait complete. 2024-09-17T05:28:00.789 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:28:00.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f881ac00640 1 -- 172.21.15.122:0/902339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 msgr2=0x7f87f8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f881ac00640 1 --2- 172.21.15.122:0/902339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 0x7f87f8004c90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009b60 tx=0x7f87fc01d250 comp rx=0 tx=0).stop 2024-09-17T05:28:00.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f8818c00640 1 -- 172.21.15.122:0/902339 reap_dead start 2024-09-17T05:28:00.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f881ac00640 1 -- 172.21.15.122:0/902339 shutdown_connections 2024-09-17T05:28:00.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f881ac00640 1 -- 172.21.15.122:0/902339 >> 172.21.15.122:0/902339 conn(0x7f88140b18f0 msgr2=0x7f88140b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:00.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f881ac00640 1 -- 172.21.15.122:0/902339 shutdown_connections 2024-09-17T05:28:00.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f881ac00640 1 -- 172.21.15.122:0/902339 wait complete. 2024-09-17T05:28:00.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f881ac00640 1 Processor -- start 2024-09-17T05:28:00.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.931+0000 7f881ac00640 1 -- start start 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 0x7f8814155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88140ca2d0 0x7f88140ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f881413fb60 0x7f881413ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88140b4d50 con 0x7f87f8004810 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88140b4bd0 con 0x7f88140ca2d0 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f88140b4ed0 con 0x7f881413fb60 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 0x7f8814155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 0x7f8814155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43512/0 (socket says 172.21.15.122:43512) 2024-09-17T05:28:00.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 -- 172.21.15.122:0/1373559536 learned_addr learned my addr 172.21.15.122:0/1373559536 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:00.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8818c00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f881413fb60 0x7f881413ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:00.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f880b400640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88140ca2d0 0x7f88140ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 -- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f881413fb60 msgr2=0x7f881413ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f881413fb60 0x7f881413ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 -- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88140ca2d0 msgr2=0x7f88140ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88140ca2d0 0x7f88140ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 -- 172.21.15.122:0/1373559536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8814142900 con 0x7f87f8004810 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8813e00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 0x7f8814155610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8804012690 tx=0x7f8804012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8810c00640 1 -- 172.21.15.122:0/1373559536 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804010b40 con 0x7f87f8004810 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8810c00640 1 -- 172.21.15.122:0/1373559536 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8804017030 con 0x7f87f8004810 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f8810c00640 1 -- 172.21.15.122:0/1373559536 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804012e90 con 0x7f87f8004810 2024-09-17T05:28:00.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8814142be0 con 0x7f87f8004810 2024-09-17T05:28:00.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88141431a0 con 0x7f87f8004810 2024-09-17T05:28:00.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.935+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8814069410 con 0x7f87f8004810 2024-09-17T05:28:00.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.939+0000 7f8810c00640 1 -- 172.21.15.122:0/1373559536 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f8804013450 con 0x7f87f8004810 2024-09-17T05:28:00.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.939+0000 7f8810c00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f87f0073730 0x7f87f0075bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:00.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.939+0000 7f8810c00640 1 -- 172.21.15.122:0/1373559536 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f8804097fe0 con 0x7f87f8004810 2024-09-17T05:28:00.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.939+0000 7f880b400640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f87f0073730 0x7f87f0075bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:00.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.939+0000 7f880b400640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f87f0073730 0x7f87f0075bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0ae000 tx=0x7f87fc01a050 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:00.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:00.943+0000 7f8810c00640 1 -- 172.21.15.122:0/1373559536 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8804062730 con 0x7f87f8004810 2024-09-17T05:28:01.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.115+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f88140cec90 con 0x7f87f8004810 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.115+0000 7f8810c00640 1 -- 172.21.15.122:0/1373559536 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7f8804067160 con 0x7f87f8004810 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:01.117 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:28:01.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f87f0073730 msgr2=0x7f87f0075bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:01.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f87f0073730 0x7f87f0075bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0ae000 tx=0x7f87fc01a050 comp rx=0 tx=0).stop 2024-09-17T05:28:01.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 msgr2=0x7f8814155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:01.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 0x7f8814155610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8804012690 tx=0x7f8804012b50 comp rx=0 tx=0).stop 2024-09-17T05:28:01.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 shutdown_connections 2024-09-17T05:28:01.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f87f0073730 0x7f87f0075bf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f881413fb60 0x7f881413ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88140ca2d0 0x7f88140ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 --2- 172.21.15.122:0/1373559536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f8004810 0x7f8814155610 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 >> 172.21.15.122:0/1373559536 conn(0x7f88140b18f0 msgr2=0x7f88140b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:01.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 shutdown_connections 2024-09-17T05:28:01.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.119+0000 7f881ac00640 1 -- 172.21.15.122:0/1373559536 wait complete. 2024-09-17T05:28:01.140 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:28:01.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:28:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.283+0000 7f2151600640 1 -- 172.21.15.122:0/995140243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 msgr2=0x7f214c0cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.283+0000 7f2151600640 1 --2- 172.21.15.122:0/995140243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 0x7f214c0cee10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f214c0b0fb0 tx=0x7f213801f0b0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.283+0000 7f2151600640 1 -- 172.21.15.122:0/995140243 shutdown_connections 2024-09-17T05:28:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.283+0000 7f2151600640 1 --2- 172.21.15.122:0/995140243 >> v2:172.21.15.173:3300/0 conn(0x7f214c0b0b60 0x7f214c0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.283+0000 7f2151600640 1 --2- 172.21.15.122:0/995140243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 0x7f214c0cee10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.283+0000 7f2151600640 1 -- 172.21.15.122:0/995140243 >> 172.21.15.122:0/995140243 conn(0x7f214c0d1da0 msgr2=0x7f214c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.283+0000 7f2151600640 1 -- 172.21.15.122:0/995140243 shutdown_connections 2024-09-17T05:28:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.283+0000 7f2151600640 1 -- 172.21.15.122:0/995140243 wait complete. 2024-09-17T05:28:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 Processor -- start 2024-09-17T05:28:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 -- start start 2024-09-17T05:28:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214c0b0b60 0x7f214c1f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c15be10 0x7f214c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 0x7f214c0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f214c0cd4a0 con 0x7f214c15c1f0 2024-09-17T05:28:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f214c0cd320 con 0x7f214c0b0b60 2024-09-17T05:28:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f214c0cd620 con 0x7f214c15be10 2024-09-17T05:28:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 0x7f214c0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 0x7f214c0c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51676/0 (socket says 172.21.15.122:51676) 2024-09-17T05:28:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 -- 172.21.15.122:0/3022855379 learned_addr learned my addr 172.21.15.122:0/3022855379 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214aa00640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214c0b0b60 0x7f214c1f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214a000640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c15be10 0x7f214c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:01.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 -- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c15be10 msgr2=0x7f214c0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:01.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c15be10 0x7f214c0c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 -- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214c0b0b60 msgr2=0x7f214c1f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:01.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214c0b0b60 0x7f214c1f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:01.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 -- 172.21.15.122:0/3022855379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c0c72e0 con 0x7f214c15c1f0 2024-09-17T05:28:01.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214aa00640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214c0b0b60 0x7f214c1f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:01.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f214b400640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 0x7f214c0c58d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f214400eb20 tx=0x7f214400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:01.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2142a00640 1 -- 172.21.15.122:0/3022855379 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144004b30 con 0x7f214c15c1f0 2024-09-17T05:28:01.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2142a00640 1 -- 172.21.15.122:0/3022855379 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2144004cf0 con 0x7f214c15c1f0 2024-09-17T05:28:01.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c0c7580 con 0x7f214c15c1f0 2024-09-17T05:28:01.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2142a00640 1 -- 172.21.15.122:0/3022855379 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144005280 con 0x7f214c15c1f0 2024-09-17T05:28:01.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.287+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c0c7b00 con 0x7f214c15c1f0 2024-09-17T05:28:01.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.291+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214c0cdb40 con 0x7f214c15c1f0 2024-09-17T05:28:01.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.291+0000 7f2142a00640 1 -- 172.21.15.122:0/3022855379 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2144005420 con 0x7f214c15c1f0 2024-09-17T05:28:01.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.291+0000 7f2142a00640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f212c073560 0x7f212c075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:01.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.291+0000 7f2142a00640 1 -- 172.21.15.122:0/3022855379 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f2144095350 con 0x7f214c15c1f0 2024-09-17T05:28:01.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.291+0000 7f214aa00640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f212c073560 0x7f212c075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:01.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.291+0000 7f214aa00640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f212c073560 0x7f212c075a20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f214c0b0b30 tx=0x7f213c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:01.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.295+0000 7f2142a00640 1 -- 172.21.15.122:0/3022855379 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2144060110 con 0x7f214c15c1f0 2024-09-17T05:28:01.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:01.467+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f214c070350 con 0x7f214c15c1f0 2024-09-17T05:28:01.889 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:01.892 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:02.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.371+0000 7f2142a00640 1 -- 172.21.15.122:0/3022855379 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f214405fa10 con 0x7f214c15c1f0 2024-09-17T05:28:02.371 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:28:02.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f212c073560 msgr2=0x7f212c075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:02.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f212c073560 0x7f212c075a20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f214c0b0b30 tx=0x7f213c009040 comp rx=0 tx=0).stop 2024-09-17T05:28:02.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 msgr2=0x7f214c0c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 0x7f214c0c58d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f214400eb20 tx=0x7f214400c2d0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 shutdown_connections 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f212c073560 0x7f212c075a20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c15c1f0 0x7f214c0c58d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f214c15be10 0x7f214c0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 --2- 172.21.15.122:0/3022855379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214c0b0b60 0x7f214c1f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 >> 172.21.15.122:0/3022855379 conn(0x7f214c0d1da0 msgr2=0x7f214c0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 shutdown_connections 2024-09-17T05:28:02.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.375+0000 7f2151600640 1 -- 172.21.15.122:0/3022855379 wait complete. 2024-09-17T05:28:02.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:28:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 -- 172.21.15.122:0/4220261290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040cd8c0 msgr2=0x7f48040cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 --2- 172.21.15.122:0/4220261290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040cd8c0 0x7f48040cea20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4804073d80 tx=0x7f47fc01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 -- 172.21.15.122:0/4220261290 shutdown_connections 2024-09-17T05:28:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 --2- 172.21.15.122:0/4220261290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040cd8c0 0x7f48040cea20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 --2- 172.21.15.122:0/4220261290 >> v2:172.21.15.173:3300/0 conn(0x7f480415be80 0x7f48040ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 -- 172.21.15.122:0/4220261290 >> 172.21.15.122:0/4220261290 conn(0x7f48040d1da0 msgr2=0x7f48040d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 -- 172.21.15.122:0/4220261290 shutdown_connections 2024-09-17T05:28:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 -- 172.21.15.122:0/4220261290 wait complete. 2024-09-17T05:28:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 Processor -- start 2024-09-17T05:28:02.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.555+0000 7f480c200640 1 -- start start 2024-09-17T05:28:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040cd8c0 0x7f480414d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040d28c0 0x7f48040c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f480415be80 0x7f48040c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4804168bd0 con 0x7f48040d28c0 2024-09-17T05:28:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4804168a50 con 0x7f48040cd8c0 2024-09-17T05:28:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4804168d50 con 0x7f480415be80 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040d28c0 0x7f48040c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040d28c0 0x7f48040c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51694/0 (socket says 172.21.15.122:51694) 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 -- 172.21.15.122:0/1525599646 learned_addr learned my addr 172.21.15.122:0/1525599646 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4809800640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040cd8c0 0x7f480414d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480a200640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f480415be80 0x7f48040c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 -- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f480415be80 msgr2=0x7f48040c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f480415be80 0x7f48040c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 -- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040cd8c0 msgr2=0x7f480414d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040cd8c0 0x7f480414d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 -- 172.21.15.122:0/1525599646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48040cfc90 con 0x7f48040d28c0 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480a200640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f480415be80 0x7f48040c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4808e00640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040d28c0 0x7f48040c67e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c4b0 tx=0x7f47f000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4801600640 1 -- 172.21.15.122:0/1525599646 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f00052d0 con 0x7f48040d28c0 2024-09-17T05:28:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4801600640 1 -- 172.21.15.122:0/1525599646 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47f0005470 con 0x7f48040d28c0 2024-09-17T05:28:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f4801600640 1 -- 172.21.15.122:0/1525599646 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0012870 con 0x7f48040d28c0 2024-09-17T05:28:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48040c6d20 con 0x7f48040d28c0 2024-09-17T05:28:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480415d990 con 0x7f48040d28c0 2024-09-17T05:28:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.559+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48040b53f0 con 0x7f48040d28c0 2024-09-17T05:28:02.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.563+0000 7f4801600640 1 -- 172.21.15.122:0/1525599646 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f47f0003600 con 0x7f48040d28c0 2024-09-17T05:28:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.563+0000 7f4801600640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f47e4073560 0x7f47e4075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.563+0000 7f4801600640 1 -- 172.21.15.122:0/1525599646 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7f47f0094230 con 0x7f48040d28c0 2024-09-17T05:28:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.563+0000 7f4809800640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f47e4073560 0x7f47e4075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:02.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.563+0000 7f4809800640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f47e4073560 0x7f47e4075a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009f30 tx=0x7f47f80096a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:02.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.567+0000 7f4801600640 1 -- 172.21.15.122:0/1525599646 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f48040c6d20 con 0x7f48040d28c0 2024-09-17T05:28:02.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:02.731+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f48040c9120 con 0x7f48040d28c0 2024-09-17T05:28:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.379+0000 7f4801600640 1 -- 172.21.15.122:0/1525599646 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7f48040d0350 con 0x7f48040d28c0 2024-09-17T05:28:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.379+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f47e4073560 msgr2=0x7f47e4075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.379+0000 7f480c200640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f47e4073560 0x7f47e4075a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009f30 tx=0x7f47f80096a0 comp rx=0 tx=0).stop 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.379+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040d28c0 msgr2=0x7f48040c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.379+0000 7f480c200640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040d28c0 0x7f48040c67e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c4b0 tx=0x7f47f000c970 comp rx=0 tx=0).stop 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.383+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 shutdown_connections 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.383+0000 7f480c200640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f47e4073560 0x7f47e4075a20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.383+0000 7f480c200640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f480415be80 0x7f48040c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.383+0000 7f480c200640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48040d28c0 0x7f48040c67e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.383+0000 7f480c200640 1 --2- 172.21.15.122:0/1525599646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040cd8c0 0x7f480414d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:03.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.383+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 >> 172.21.15.122:0/1525599646 conn(0x7f48040d1da0 msgr2=0x7f48040d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:03.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.383+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 shutdown_connections 2024-09-17T05:28:03.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.383+0000 7f480c200640 1 -- 172.21.15.122:0/1525599646 wait complete. 2024-09-17T05:28:03.422 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:28:03.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566a00640 1 -- 172.21.15.122:0/2360168956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0548004d50 msgr2=0x7f05480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:03.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566a00640 1 --2- 172.21.15.122:0/2360168956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0548004d50 0x7f05480051d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0558009ee0 tx=0x7f055801d630 comp rx=0 tx=0).stop 2024-09-17T05:28:03.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566000640 1 -- 172.21.15.122:0/2360168956 reap_dead start 2024-09-17T05:28:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566a00640 1 -- 172.21.15.122:0/2360168956 shutdown_connections 2024-09-17T05:28:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566a00640 1 -- 172.21.15.122:0/2360168956 >> 172.21.15.122:0/2360168956 conn(0x7f056009aa10 msgr2=0x7f056009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566a00640 1 -- 172.21.15.122:0/2360168956 shutdown_connections 2024-09-17T05:28:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566a00640 1 -- 172.21.15.122:0/2360168956 wait complete. 2024-09-17T05:28:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566a00640 1 Processor -- start 2024-09-17T05:28:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.567+0000 7f0566a00640 1 -- start start 2024-09-17T05:28:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0548004d50 0x7f05601510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05600b6de0 0x7f05601f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05600b57f0 0x7f05601f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05600d1f40 con 0x7f05600b57f0 2024-09-17T05:28:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05600d1dc0 con 0x7f05600b6de0 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f05600ccb60 con 0x7f0548004d50 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05600b57f0 0x7f05601f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05600b57f0 0x7f05601f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51708/0 (socket says 172.21.15.122:51708) 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 -- 172.21.15.122:0/963251858 learned_addr learned my addr 172.21.15.122:0/963251858 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0564c00640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05600b6de0 0x7f05601f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0565600640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0548004d50 0x7f05601510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 -- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0548004d50 msgr2=0x7f05601510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0548004d50 0x7f05601510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 -- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05600b6de0 msgr2=0x7f05601f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05600b6de0 0x7f05601f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 -- 172.21.15.122:0/963251858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600b5d40 con 0x7f05600b57f0 2024-09-17T05:28:03.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566000640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05600b57f0 0x7f05601f7f20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0558099160 tx=0x7f0558004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:03.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f055d600640 1 -- 172.21.15.122:0/963251858 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580ad050 con 0x7f05600b57f0 2024-09-17T05:28:03.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f055d600640 1 -- 172.21.15.122:0/963251858 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05580a7ad0 con 0x7f05600b57f0 2024-09-17T05:28:03.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f055d600640 1 -- 172.21.15.122:0/963251858 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558009200 con 0x7f05600b57f0 2024-09-17T05:28:03.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05600b6020 con 0x7f05600b57f0 2024-09-17T05:28:03.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05600b65e0 con 0x7f05600b57f0 2024-09-17T05:28:03.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05600cc620 con 0x7f05600b57f0 2024-09-17T05:28:03.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.571+0000 7f055d600640 1 -- 172.21.15.122:0/963251858 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f05580b0070 con 0x7f05600b57f0 2024-09-17T05:28:03.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.575+0000 7f055d600640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0540073590 0x7f0540075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:03.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.575+0000 7f055d600640 1 -- 172.21.15.122:0/963251858 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7f055812f5b0 con 0x7f05600b57f0 2024-09-17T05:28:03.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.575+0000 7f0565600640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0540073590 0x7f0540075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:03.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.575+0000 7f0565600640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0540073590 0x7f0540075a50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0554001ec0 tx=0x7f0554006730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.579+0000 7f055d600640 1 -- 172.21.15.122:0/963251858 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0558017340 con 0x7f05600b57f0 2024-09-17T05:28:03.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:03.779+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0560070c00 con 0x7f05600b57f0 2024-09-17T05:28:04.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.379+0000 7f055d600640 1 -- 172.21.15.122:0/963251858 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7f05580f9980 con 0x7f05600b57f0 2024-09-17T05:28:04.381 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 6 and data pool 7 2024-09-17T05:28:04.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0540073590 msgr2=0x7f0540075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0540073590 0x7f0540075a50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0554001ec0 tx=0x7f0554006730 comp rx=0 tx=0).stop 2024-09-17T05:28:04.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05600b57f0 msgr2=0x7f05601f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05600b57f0 0x7f05601f7f20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0558099160 tx=0x7f0558004f30 comp rx=0 tx=0).stop 2024-09-17T05:28:04.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 shutdown_connections 2024-09-17T05:28:04.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0540073590 0x7f0540075a50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05600b57f0 0x7f05601f7f20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05600b6de0 0x7f05601f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 --2- 172.21.15.122:0/963251858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0548004d50 0x7f05601510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 >> 172.21.15.122:0/963251858 conn(0x7f056009aa10 msgr2=0x7f05600d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:04.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 shutdown_connections 2024-09-17T05:28:04.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.383+0000 7f0566a00640 1 -- 172.21.15.122:0/963251858 wait complete. 2024-09-17T05:28:04.430 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:04.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- 172.21.15.122:0/815953717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b4d80 msgr2=0x7f901c0b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 --2- 172.21.15.122:0/815953717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b4d80 0x7f901c0b11a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f901c069e80 tx=0x7f900c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- 172.21.15.122:0/815953717 shutdown_connections 2024-09-17T05:28:04.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 --2- 172.21.15.122:0/815953717 >> v2:172.21.15.173:3300/0 conn(0x7f901c0b0ce0 0x7f901c0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 --2- 172.21.15.122:0/815953717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b4d80 0x7f901c0b11a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- 172.21.15.122:0/815953717 >> 172.21.15.122:0/815953717 conn(0x7f901c09aa10 msgr2=0x7f901c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:04.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- 172.21.15.122:0/815953717 shutdown_connections 2024-09-17T05:28:04.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- 172.21.15.122:0/815953717 wait complete. 2024-09-17T05:28:04.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 Processor -- start 2024-09-17T05:28:04.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- start start 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b0ce0 0x7f901c148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f901c0b4d80 0x7f901c147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901c0b56a0 0x7f901c134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f901c168a20 con 0x7f901c0b0ce0 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f901c1688a0 con 0x7f901c0b4d80 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901b200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f901c168ba0 con 0x7f901c0b56a0 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b0ce0 0x7f901c148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b0ce0 0x7f901c148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51726/0 (socket says 172.21.15.122:51726) 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019e00640 1 -- 172.21.15.122:0/3118131982 learned_addr learned my addr 172.21.15.122:0/3118131982 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019400640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f901c0b4d80 0x7f901c147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f901a800640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901c0b56a0 0x7f901c134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019e00640 1 -- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901c0b56a0 msgr2=0x7f901c134140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019e00640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901c0b56a0 0x7f901c134140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019e00640 1 -- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f901c0b4d80 msgr2=0x7f901c147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019e00640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f901c0b4d80 0x7f901c147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.575+0000 7f9019e00640 1 -- 172.21.15.122:0/3118131982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901c148050 con 0x7f901c0b0ce0 2024-09-17T05:28:04.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f9019e00640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b0ce0 0x7f901c148780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f900c0194d0 tx=0x7f900c0a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:04.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f9012000640 1 -- 172.21.15.122:0/3118131982 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c0bc070 con 0x7f901c0b0ce0 2024-09-17T05:28:04.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f9012000640 1 -- 172.21.15.122:0/3118131982 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f900c005030 con 0x7f901c0b0ce0 2024-09-17T05:28:04.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f9012000640 1 -- 172.21.15.122:0/3118131982 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c0b2040 con 0x7f901c0b0ce0 2024-09-17T05:28:04.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901c136ba0 con 0x7f901c0b0ce0 2024-09-17T05:28:04.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f901c14e410 con 0x7f901c0b0ce0 2024-09-17T05:28:04.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f901c0b5160 con 0x7f901c0b0ce0 2024-09-17T05:28:04.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f9012000640 1 -- 172.21.15.122:0/3118131982 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f900c005480 con 0x7f901c0b0ce0 2024-09-17T05:28:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f9012000640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ffc073560 0x7f8ffc075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.579+0000 7f9012000640 1 -- 172.21.15.122:0/3118131982 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f900c12e0a0 con 0x7f901c0b0ce0 2024-09-17T05:28:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.583+0000 7f9019400640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ffc073560 0x7f8ffc075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.583+0000 7f9019400640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ffc073560 0x7f8ffc075a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f901c153520 tx=0x7f900800b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:04.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.583+0000 7f9012000640 1 -- 172.21.15.122:0/3118131982 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f901c148050 con 0x7f901c0b0ce0 2024-09-17T05:28:04.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.751+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f901c0ce0c0 con 0x7f901c0b0ce0 2024-09-17T05:28:04.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.751+0000 7f9012000640 1 -- 172.21.15.122:0/3118131982 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16742 (secure 0 0 0) 0x7f900c0164b0 con 0x7f901c0b0ce0 2024-09-17T05:28:04.754 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:04.754 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:28:03.784321+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:01.479885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:02.745102+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ffc073560 msgr2=0x7f8ffc075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ffc073560 0x7f8ffc075a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f901c153520 tx=0x7f900800b390 comp rx=0 tx=0).stop 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b0ce0 msgr2=0x7f901c148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b0ce0 0x7f901c148780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f900c0194d0 tx=0x7f900c0a57b0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 shutdown_connections 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ffc073560 0x7f8ffc075a20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901c0b56a0 0x7f901c134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f901c0b4d80 0x7f901c147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 --2- 172.21.15.122:0/3118131982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f901c0b0ce0 0x7f901c148780 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 >> 172.21.15.122:0/3118131982 conn(0x7f901c09aa10 msgr2=0x7f901c157d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:04.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 shutdown_connections 2024-09-17T05:28:04.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.755+0000 7f901b200640 1 -- 172.21.15.122:0/3118131982 wait complete. 2024-09-17T05:28:04.776 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:04.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1458769168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c06cc20 msgr2=0x7fe49c0737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 --2- 172.21.15.122:0/1458769168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c06cc20 0x7fe49c0737c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009870 tx=0x7fe48c01c830 comp rx=0 tx=0).stop 2024-09-17T05:28:04.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1458769168 shutdown_connections 2024-09-17T05:28:04.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 --2- 172.21.15.122:0/1458769168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c06cc20 0x7fe49c0737c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1458769168 >> 172.21.15.122:0/1458769168 conn(0x7fe49c0d0c90 msgr2=0x7fe49c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:04.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1458769168 shutdown_connections 2024-09-17T05:28:04.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1458769168 wait complete. 2024-09-17T05:28:04.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 Processor -- start 2024-09-17T05:28:04.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- start start 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c06cc20 0x7fe49c0ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c071030 0x7fe49c0c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe49c15c1c0 0x7fe49c0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c16a900 con 0x7fe49c15c1c0 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c16a780 con 0x7fe49c06cc20 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe4a1c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c16aa80 con 0x7fe49c071030 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe49b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe49c15c1c0 0x7fe49c0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe49b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe49c15c1c0 0x7fe49c0c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51736/0 (socket says 172.21.15.122:51736) 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe49b400640 1 -- 172.21.15.122:0/1419891753 learned_addr learned my addr 172.21.15.122:0/1419891753 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe49aa00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c06cc20 0x7fe49c0ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.919+0000 7fe493e00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c071030 0x7fe49c0c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe49b400640 1 -- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c071030 msgr2=0x7fe49c0c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe49b400640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c071030 0x7fe49c0c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe49b400640 1 -- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c06cc20 msgr2=0x7fe49c0ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe49b400640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c06cc20 0x7fe49c0ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe49b400640 1 -- 172.21.15.122:0/1419891753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c0c1930 con 0x7fe49c15c1c0 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe49b400640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe49c15c1c0 0x7fe49c0c6bb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe49400c7d0 tx=0x7fe49400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe492a00640 1 -- 172.21.15.122:0/1419891753 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494003b80 con 0x7fe49c15c1c0 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c0c0b30 con 0x7fe49c15c1c0 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe492a00640 1 -- 172.21.15.122:0/1419891753 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe494003d20 con 0x7fe49c15c1c0 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe492a00640 1 -- 172.21.15.122:0/1419891753 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49400aab0 con 0x7fe49c15c1c0 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c0c1e80 con 0x7fe49c15c1c0 2024-09-17T05:28:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c0737c0 con 0x7fe49c15c1c0 2024-09-17T05:28:04.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.923+0000 7fe492a00640 1 -- 172.21.15.122:0/1419891753 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fe49400ac50 con 0x7fe49c15c1c0 2024-09-17T05:28:04.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.927+0000 7fe492a00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe478073560 0x7fe478075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:04.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.927+0000 7fe492a00640 1 -- 172.21.15.122:0/1419891753 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe494094640 con 0x7fe49c15c1c0 2024-09-17T05:28:04.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.927+0000 7fe49aa00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe478073560 0x7fe478075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:04.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.927+0000 7fe49aa00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe478073560 0x7fe478075a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0af000 tx=0x7fe48c01b070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:04.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:04.927+0000 7fe492a00640 1 -- 172.21.15.122:0/1419891753 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe49405ea50 con 0x7fe49c15c1c0 2024-09-17T05:28:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.099+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe49c0b6c10 con 0x7fe49c15c1c0 2024-09-17T05:28:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe492a00640 1 -- 172.21.15.122:0/1419891753 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16742 (secure 0 0 0) 0x7fe494063480 con 0x7fe49c15c1c0 2024-09-17T05:28:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:05.103 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":45,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:28:03.784321+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:01.479885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:02.745102+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe478073560 msgr2=0x7fe478075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe478073560 0x7fe478075a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0af000 tx=0x7fe48c01b070 comp rx=0 tx=0).stop 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe49c15c1c0 msgr2=0x7fe49c0c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe49c15c1c0 0x7fe49c0c6bb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe49400c7d0 tx=0x7fe49400cc90 comp rx=0 tx=0).stop 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 shutdown_connections 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe478073560 0x7fe478075a20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe49c15c1c0 0x7fe49c0c6bb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c071030 0x7fe49c0c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 --2- 172.21.15.122:0/1419891753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c06cc20 0x7fe49c0ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:05.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 >> 172.21.15.122:0/1419891753 conn(0x7fe49c0d0c90 msgr2=0x7fe49c0c4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:05.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 shutdown_connections 2024-09-17T05:28:05.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.103+0000 7fe4a1c00640 1 -- 172.21.15.122:0/1419891753 wait complete. 2024-09-17T05:28:05.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:28:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.267+0000 7f59a5400640 1 -- 172.21.15.122:0/3595384656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 msgr2=0x7f59a006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.267+0000 7f59a5400640 1 --2- 172.21.15.122:0/3595384656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 0x7f59a006f7b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f599800ad90 tx=0x7f599801dc20 comp rx=0 tx=0).stop 2024-09-17T05:28:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.267+0000 7f59a5400640 1 -- 172.21.15.122:0/3595384656 shutdown_connections 2024-09-17T05:28:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.267+0000 7f59a5400640 1 --2- 172.21.15.122:0/3595384656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 0x7f59a006f7b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.267+0000 7f59a5400640 1 --2- 172.21.15.122:0/3595384656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a015dae0 0x7f59a00acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.267+0000 7f59a5400640 1 -- 172.21.15.122:0/3595384656 >> 172.21.15.122:0/3595384656 conn(0x7f59a00b18f0 msgr2=0x7f59a00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.267+0000 7f59a5400640 1 -- 172.21.15.122:0/3595384656 shutdown_connections 2024-09-17T05:28:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.267+0000 7f59a5400640 1 -- 172.21.15.122:0/3595384656 wait complete. 2024-09-17T05:28:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 Processor -- start 2024-09-17T05:28:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 -- start start 2024-09-17T05:28:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 0x7f59a014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00cd100 0x7f59a01553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a015dae0 0x7f59a0155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59a00cf1a0 con 0x7f59a00b49e0 2024-09-17T05:28:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59a00cf020 con 0x7f59a00cd100 2024-09-17T05:28:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f59a00cf320 con 0x7f59a015dae0 2024-09-17T05:28:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 0x7f59a014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 0x7f59a014f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51760/0 (socket says 172.21.15.122:51760) 2024-09-17T05:28:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 -- 172.21.15.122:0/186716340 learned_addr learned my addr 172.21.15.122:0/186716340 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599de00640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00cd100 0x7f59a01553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599f200640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a015dae0 0x7f59a0155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 -- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a015dae0 msgr2=0x7f59a0155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a015dae0 0x7f59a0155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 -- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00cd100 msgr2=0x7f59a01553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00cd100 0x7f59a01553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:05.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 -- 172.21.15.122:0/186716340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a01545f0 con 0x7f59a00b49e0 2024-09-17T05:28:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f599e800640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 0x7f59a014f290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f599000d950 tx=0x7f599000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f5996a00640 1 -- 172.21.15.122:0/186716340 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990017070 con 0x7f59a00b49e0 2024-09-17T05:28:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f5996a00640 1 -- 172.21.15.122:0/186716340 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5990009d20 con 0x7f59a00b49e0 2024-09-17T05:28:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f5996a00640 1 -- 172.21.15.122:0/186716340 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900106c0 con 0x7f59a00b49e0 2024-09-17T05:28:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a01548d0 con 0x7f59a00b49e0 2024-09-17T05:28:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.271+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a0154e90 con 0x7f59a00b49e0 2024-09-17T05:28:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.275+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a0069410 con 0x7f59a00b49e0 2024-09-17T05:28:05.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.275+0000 7f5996a00640 1 -- 172.21.15.122:0/186716340 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f599001a070 con 0x7f59a00b49e0 2024-09-17T05:28:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.275+0000 7f5996a00640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5980073590 0x7f5980075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.275+0000 7f5996a00640 1 -- 172.21.15.122:0/186716340 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f59900949a0 con 0x7f59a00b49e0 2024-09-17T05:28:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.275+0000 7f599de00640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5980073590 0x7f5980075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.275+0000 7f599de00640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5980073590 0x7f5980075a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f59a015b610 tx=0x7f598c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:05.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.279+0000 7f5996a00640 1 -- 172.21.15.122:0/186716340 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f599005edb0 con 0x7f59a00b49e0 2024-09-17T05:28:05.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:05.483+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f59a00acd40 con 0x7f59a00b49e0 2024-09-17T05:28:06.362 INFO:tasks.ceph.mds.d.smithi173.stderr:2024-09-17T05:28:06.364+0000 7f5d54c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:28:06.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f5996a00640 1 -- 172.21.15.122:0/186716340 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f59900637e0 con 0x7f59a00b49e0 2024-09-17T05:28:06.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5980073590 msgr2=0x7f5980075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:06.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5980073590 0x7f5980075a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f59a015b610 tx=0x7f598c009040 comp rx=0 tx=0).stop 2024-09-17T05:28:06.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 msgr2=0x7f59a014f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:06.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 0x7f59a014f290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f599000d950 tx=0x7f599000de10 comp rx=0 tx=0).stop 2024-09-17T05:28:06.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 shutdown_connections 2024-09-17T05:28:06.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5980073590 0x7f5980075a50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:06.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a015dae0 0x7f59a0155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:06.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00cd100 0x7f59a01553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:06.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 --2- 172.21.15.122:0/186716340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59a00b49e0 0x7f59a014f290 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:06.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 >> 172.21.15.122:0/186716340 conn(0x7f59a00b18f0 msgr2=0x7f59a00aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:06.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 shutdown_connections 2024-09-17T05:28:06.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.411+0000 7f59a5400640 1 -- 172.21.15.122:0/186716340 wait complete. 2024-09-17T05:28:06.435 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:28:06.435 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:28:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.579+0000 7f2ddce00640 1 -- 172.21.15.122:0/3259298891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 msgr2=0x7f2dd8162130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.579+0000 7f2ddce00640 1 --2- 172.21.15.122:0/3259298891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 0x7f2dd8162130 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8073990 tx=0x7f2dc401c770 comp rx=0 tx=0).stop 2024-09-17T05:28:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.579+0000 7f2ddce00640 1 -- 172.21.15.122:0/3259298891 shutdown_connections 2024-09-17T05:28:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.579+0000 7f2ddce00640 1 --2- 172.21.15.122:0/3259298891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 0x7f2dd8162130 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.579+0000 7f2ddce00640 1 --2- 172.21.15.122:0/3259298891 >> v2:172.21.15.173:3300/0 conn(0x7f2dd81613c0 0x7f2dd8161bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.579+0000 7f2ddce00640 1 -- 172.21.15.122:0/3259298891 >> 172.21.15.122:0/3259298891 conn(0x7f2dd80d1dd0 msgr2=0x7f2dd80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.579+0000 7f2ddce00640 1 -- 172.21.15.122:0/3259298891 shutdown_connections 2024-09-17T05:28:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.579+0000 7f2ddce00640 1 -- 172.21.15.122:0/3259298891 wait complete. 2024-09-17T05:28:06.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 Processor -- start 2024-09-17T05:28:06.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 -- start start 2024-09-17T05:28:06.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80b3190 0x7f2dd8149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:06.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd81613c0 0x7f2dd81fbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:06.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 0x7f2dd80b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:06.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd816b480 con 0x7f2dd81617a0 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd816b300 con 0x7f2dd80b3190 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd816b600 con 0x7f2dd81613c0 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 0x7f2dd80b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 0x7f2dd80b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51786/0 (socket says 172.21.15.122:51786) 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 -- 172.21.15.122:0/2132628165 learned_addr learned my addr 172.21.15.122:0/2132628165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd5e00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80b3190 0x7f2dd8149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd5400640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd81613c0 0x7f2dd81fbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 -- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd81613c0 msgr2=0x7f2dd81fbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd81613c0 0x7f2dd81fbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 -- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80b3190 msgr2=0x7f2dd8149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80b3190 0x7f2dd8149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 -- 172.21.15.122:0/2132628165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd80b6cc0 con 0x7f2dd81617a0 2024-09-17T05:28:06.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd5e00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80b3190 0x7f2dd8149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:06.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dd6800640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 0x7f2dd80b6780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000c7a0 tx=0x7f2dd000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:06.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dce000640 1 -- 172.21.15.122:0/2132628165 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0003b80 con 0x7f2dd81617a0 2024-09-17T05:28:06.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dce000640 1 -- 172.21.15.122:0/2132628165 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd0003d40 con 0x7f2dd81617a0 2024-09-17T05:28:06.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd8149a90 con 0x7f2dd81617a0 2024-09-17T05:28:06.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2dce000640 1 -- 172.21.15.122:0/2132628165 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000aab0 con 0x7f2dd81617a0 2024-09-17T05:28:06.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd814a050 con 0x7f2dd81617a0 2024-09-17T05:28:06.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.583+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd814a290 con 0x7f2dd81617a0 2024-09-17T05:28:06.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.587+0000 7f2dce000640 1 -- 172.21.15.122:0/2132628165 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2dd000ac50 con 0x7f2dd81617a0 2024-09-17T05:28:06.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.587+0000 7f2dce000640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2db80734d0 0x7f2db8075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:06.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.587+0000 7f2dce000640 1 -- 172.21.15.122:0/2132628165 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f2dd00946d0 con 0x7f2dd81617a0 2024-09-17T05:28:06.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.587+0000 7f2dd5e00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2db80734d0 0x7f2db8075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:06.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.587+0000 7f2dd5e00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2db80734d0 0x7f2db8075990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80058d0 tx=0x7f2dc8005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:06.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.591+0000 7f2dce000640 1 -- 172.21.15.122:0/2132628165 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2dd814a290 con 0x7f2dd81617a0 2024-09-17T05:28:06.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:06.799+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2dd806dd30 con 0x7f2dd81617a0 2024-09-17T05:28:07.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.415+0000 7f2dce000640 1 -- 172.21.15.122:0/2132628165 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f2dd005f140 con 0x7f2dd81617a0 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2db80734d0 msgr2=0x7f2db8075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2db80734d0 0x7f2db8075990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80058d0 tx=0x7f2dc8005860 comp rx=0 tx=0).stop 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 msgr2=0x7f2dd80b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 0x7f2dd80b6780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000c7a0 tx=0x7f2dd000cc60 comp rx=0 tx=0).stop 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 shutdown_connections 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2db80734d0 0x7f2db8075990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd81617a0 0x7f2dd80b6780 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd81613c0 0x7f2dd81fbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 --2- 172.21.15.122:0/2132628165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80b3190 0x7f2dd8149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 >> 172.21.15.122:0/2132628165 conn(0x7f2dd80d1dd0 msgr2=0x7f2dd80cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:07.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 shutdown_connections 2024-09-17T05:28:07.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.419+0000 7f2ddce00640 1 -- 172.21.15.122:0/2132628165 wait complete. 2024-09-17T05:28:07.440 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:28:07.440 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:07.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.579+0000 7f3006e00640 1 -- 172.21.15.122:0/2854901361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f300006f8b0 msgr2=0x7f300015dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.579+0000 7f3006e00640 1 --2- 172.21.15.122:0/2854901361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f300006f8b0 0x7f300015dae0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3000070d30 tx=0x7f2ff001cc60 comp rx=0 tx=0).stop 2024-09-17T05:28:07.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.579+0000 7f3006e00640 1 -- 172.21.15.122:0/2854901361 shutdown_connections 2024-09-17T05:28:07.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.579+0000 7f3006e00640 1 --2- 172.21.15.122:0/2854901361 >> v2:172.21.15.173:3300/0 conn(0x7f30000cc620 0x7f30000b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.579+0000 7f3006e00640 1 --2- 172.21.15.122:0/2854901361 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30000cd100 0x7f30000acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.579+0000 7f3006e00640 1 --2- 172.21.15.122:0/2854901361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f300006f8b0 0x7f300015dae0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.579+0000 7f3006e00640 1 -- 172.21.15.122:0/2854901361 >> 172.21.15.122:0/2854901361 conn(0x7f30000b18f0 msgr2=0x7f30000b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 -- 172.21.15.122:0/2854901361 shutdown_connections 2024-09-17T05:28:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 -- 172.21.15.122:0/2854901361 wait complete. 2024-09-17T05:28:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 Processor -- start 2024-09-17T05:28:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 -- start start 2024-09-17T05:28:07.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f300006f8b0 0x7f3000152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:07.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30000cc620 0x7f300009d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:07.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30000cd100 0x7f30000b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30001672a0 con 0x7f30000cd100 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3000167120 con 0x7f30000cc620 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3006e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3000167420 con 0x7f300006f8b0 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30000cd100 0x7f30000b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30000cd100 0x7f30000b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51800/0 (socket says 172.21.15.122:51800) 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 -- 172.21.15.122:0/910976766 learned_addr learned my addr 172.21.15.122:0/910976766 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f2fffe00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f300006f8b0 0x7f3000152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f2ff7400640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30000cc620 0x7f300009d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 -- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f300006f8b0 msgr2=0x7f3000152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f300006f8b0 0x7f3000152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 -- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30000cc620 msgr2=0x7f300009d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30000cc620 0x7f300009d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 -- 172.21.15.122:0/910976766 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3000152f00 con 0x7f30000cd100 2024-09-17T05:28:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f3004e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30000cd100 0x7f30000b09e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800c940 tx=0x7f2ff800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.583+0000 7f2ffcc00640 1 -- 172.21.15.122:0/910976766 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8004220 con 0x7f30000cd100 2024-09-17T05:28:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f2ffcc00640 1 -- 172.21.15.122:0/910976766 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ff80079b0 con 0x7f30000cd100 2024-09-17T05:28:07.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30001531e0 con 0x7f30000cd100 2024-09-17T05:28:07.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f2ffcc00640 1 -- 172.21.15.122:0/910976766 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80044b0 con 0x7f30000cd100 2024-09-17T05:28:07.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30001537a0 con 0x7f30000cd100 2024-09-17T05:28:07.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3000069410 con 0x7f30000cd100 2024-09-17T05:28:07.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f2ffcc00640 1 -- 172.21.15.122:0/910976766 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f2ff8003600 con 0x7f30000cd100 2024-09-17T05:28:07.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f2ffcc00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fdc073560 0x7f2fdc075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:07.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f2ffcc00640 1 -- 172.21.15.122:0/910976766 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f2ff8094a50 con 0x7f30000cd100 2024-09-17T05:28:07.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.587+0000 7f2fffe00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fdc073560 0x7f2fdc075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:07.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.591+0000 7f2fffe00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fdc073560 0x7f2fdc075a20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00af000 tx=0x7f2ff001b4a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:07.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.591+0000 7f2ffcc00640 1 -- 172.21.15.122:0/910976766 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2ff8014030 con 0x7f30000cd100 2024-09-17T05:28:07.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.795+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3000156f20 con 0x7f30000cd100 2024-09-17T05:28:07.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f2ffcc00640 1 -- 172.21.15.122:0/910976766 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5232 (secure 0 0 0) 0x7f2ff805ee50 con 0x7f30000cd100 2024-09-17T05:28:07.799 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:07.799 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-17T05:28:07:412553+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3520614210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3520614210},{"type":"v1","addr":"172.21.15.122:6834","nonce":3520614210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5308,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2442281734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2442281734},{"type":"v1","addr":"172.21.15.173:6836","nonce":2442281734}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/638961094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":638961094},{"type":"v1","addr":"172.21.15.122:6835","nonce":638961094}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:03.784287+0000","modified":"2024-09-17T05:28:06.804180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/4117982032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4117982032},{"type":"v1","addr":"172.21.15.173:6837","nonce":4117982032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5388,"qdb_cluster":[5388]},"id":3}]} 2024-09-17T05:28:07.799 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fdc073560 msgr2=0x7f2fdc075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fdc073560 0x7f2fdc075a20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00af000 tx=0x7f2ff001b4a0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30000cd100 msgr2=0x7f30000b09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30000cd100 0x7f30000b09e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800c940 tx=0x7f2ff800ce00 comp rx=0 tx=0).stop 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 shutdown_connections 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fdc073560 0x7f2fdc075a20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30000cd100 0x7f30000b09e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30000cc620 0x7f300009d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 --2- 172.21.15.122:0/910976766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f300006f8b0 0x7f3000152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 >> 172.21.15.122:0/910976766 conn(0x7f30000b18f0 msgr2=0x7f30000ade60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:07.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 shutdown_connections 2024-09-17T05:28:07.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.799+0000 7f3006e00640 1 -- 172.21.15.122:0/910976766 wait complete. 2024-09-17T05:28:07.820 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:28:07.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- 172.21.15.122:0/4225631250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b88003a10 msgr2=0x7f1b88003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 --2- 172.21.15.122:0/4225631250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b88003a10 0x7f1b88003e30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009ee0 tx=0x7f1b9401d310 comp rx=0 tx=0).stop 2024-09-17T05:28:07.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba2800640 1 -- 172.21.15.122:0/4225631250 reap_dead start 2024-09-17T05:28:07.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- 172.21.15.122:0/4225631250 shutdown_connections 2024-09-17T05:28:07.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- 172.21.15.122:0/4225631250 >> 172.21.15.122:0/4225631250 conn(0x7f1ba40d1ea0 msgr2=0x7f1ba40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:07.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- 172.21.15.122:0/4225631250 shutdown_connections 2024-09-17T05:28:07.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- 172.21.15.122:0/4225631250 wait complete. 2024-09-17T05:28:07.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 Processor -- start 2024-09-17T05:28:07.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- start start 2024-09-17T05:28:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b88003a10 0x7f1ba41f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba40cb5f0 0x7f1ba40cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ba40c72a0 0x7f1ba40c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4166e40 con 0x7f1ba40cb5f0 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4166cc0 con 0x7f1ba40c72a0 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba8c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba4166fc0 con 0x7f1b88003a10 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1b99400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba40cb5f0 0x7f1ba40cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b88003a10 0x7f1ba41f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1b99400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba40cb5f0 0x7f1ba40cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51818/0 (socket says 172.21.15.122:51818) 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1ba1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b88003a10 0x7f1ba41f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:32976/0 (socket says 172.21.15.122:32976) 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.963+0000 7f1b99400640 1 -- 172.21.15.122:0/3912699015 learned_addr learned my addr 172.21.15.122:0/3912699015 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba2800640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ba40c72a0 0x7f1ba40c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba1e00640 1 -- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ba40c72a0 msgr2=0x7f1ba40c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba1e00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ba40c72a0 0x7f1ba40c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba1e00640 1 -- 172.21.15.122:0/3912699015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba40cb5f0 msgr2=0x7f1ba40cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:07.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba1e00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba40cb5f0 0x7f1ba40cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:07.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba1e00640 1 -- 172.21.15.122:0/3912699015 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba41f61b0 con 0x7f1b88003a10 2024-09-17T05:28:07.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1b99400640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba40cb5f0 0x7f1ba40cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:28:07.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba1e00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b88003a10 0x7f1ba41f8bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b940ac000 tx=0x7f1b94009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:07.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1b9aa00640 1 -- 172.21.15.122:0/3912699015 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940b2040 con 0x7f1b88003a10 2024-09-17T05:28:07.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba41f6490 con 0x7f1b88003a10 2024-09-17T05:28:07.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1b9aa00640 1 -- 172.21.15.122:0/3912699015 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b94003a70 con 0x7f1b88003a10 2024-09-17T05:28:07.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1b9aa00640 1 -- 172.21.15.122:0/3912699015 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94003e60 con 0x7f1b88003a10 2024-09-17T05:28:07.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba41f6a50 con 0x7f1b88003a10 2024-09-17T05:28:07.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba40727c0 con 0x7f1b88003a10 2024-09-17T05:28:07.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.967+0000 7f1b9aa00640 1 -- 172.21.15.122:0/3912699015 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f1b94004000 con 0x7f1b88003a10 2024-09-17T05:28:07.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.971+0000 7f1b9aa00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b80073560 0x7f1b80075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:07.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.971+0000 7f1b9aa00640 1 -- 172.21.15.122:0/3912699015 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1b9412e460 con 0x7f1b88003a10 2024-09-17T05:28:07.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.971+0000 7f1b99400640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b80073560 0x7f1b80075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:07.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.971+0000 7f1b99400640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b80073560 0x7f1b80075a20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005890 tx=0x7f1b8c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:07.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:07.975+0000 7f1b9aa00640 1 -- 172.21.15.122:0/3912699015 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1ba40cef30 con 0x7f1b88003a10 2024-09-17T05:28:08.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.143+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ba4165700 con 0x7f1b88003a10 2024-09-17T05:28:08.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.143+0000 7f1b9aa00640 1 -- 172.21.15.122:0/3912699015 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16729 (secure 0 0 0) 0x7f1b94017340 con 0x7f1b88003a10 2024-09-17T05:28:08.144 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:08.145 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":46,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:28:05.395887+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:01.479885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:02.745102+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:28:08.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b80073560 msgr2=0x7f1b80075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b80073560 0x7f1b80075a20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c005890 tx=0x7f1b8c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b88003a10 msgr2=0x7f1ba41f8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b88003a10 0x7f1ba41f8bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b940ac000 tx=0x7f1b94009bc0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 shutdown_connections 2024-09-17T05:28:08.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b80073560 0x7f1b80075a20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ba40c72a0 0x7f1ba40c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ba40cb5f0 0x7f1ba40cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 --2- 172.21.15.122:0/3912699015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b88003a10 0x7f1ba41f8bb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 >> 172.21.15.122:0/3912699015 conn(0x7f1ba40d1ea0 msgr2=0x7f1ba415a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:08.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 shutdown_connections 2024-09-17T05:28:08.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.147+0000 7f1ba8c00640 1 -- 172.21.15.122:0/3912699015 wait complete. 2024-09-17T05:28:08.167 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:28:08.167 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:08.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 -- 172.21.15.122:0/252745140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700cd180 msgr2=0x7f237015e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 --2- 172.21.15.122:0/252745140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700cd180 0x7f237015e780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f23700cd9f0 tx=0x7f235c01c830 comp rx=0 tx=0).stop 2024-09-17T05:28:08.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 -- 172.21.15.122:0/252745140 shutdown_connections 2024-09-17T05:28:08.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 --2- 172.21.15.122:0/252745140 >> v2:172.21.15.173:3300/0 conn(0x7f23700af4c0 0x7f23700caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 --2- 172.21.15.122:0/252745140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700cd180 0x7f237015e780 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 -- 172.21.15.122:0/252745140 >> 172.21.15.122:0/252745140 conn(0x7f2370073490 msgr2=0x7f23700d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:08.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 -- 172.21.15.122:0/252745140 shutdown_connections 2024-09-17T05:28:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 -- 172.21.15.122:0/252745140 wait complete. 2024-09-17T05:28:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 Processor -- start 2024-09-17T05:28:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.307+0000 7f236fe00640 1 -- start start 2024-09-17T05:28:08.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700af4c0 0x7f2370159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23700cd180 0x7f23701fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f237015a1e0 0x7f23700b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2370167270 con 0x7f23700af4c0 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23701670f0 con 0x7f23700cd180 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f23701673f0 con 0x7f237015a1e0 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700af4c0 0x7f2370159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700af4c0 0x7f2370159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51830/0 (socket says 172.21.15.122:51830) 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 -- 172.21.15.122:0/2405309672 learned_addr learned my addr 172.21.15.122:0/2405309672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236e000640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23700cd180 0x7f23701fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236f400640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f237015a1e0 0x7f23700b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 -- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f237015a1e0 msgr2=0x7f23700b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f237015a1e0 0x7f23700b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 -- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23700cd180 msgr2=0x7f23701fbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23700cd180 0x7f23701fbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 -- 172.21.15.122:0/2405309672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23700cf750 con 0x7f23700af4c0 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236e000640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23700cd180 0x7f23701fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236ea00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700af4c0 0x7f2370159090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f236000c400 tx=0x7f236000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f2366a00640 1 -- 172.21.15.122:0/2405309672 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360017070 con 0x7f23700af4c0 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f2366a00640 1 -- 172.21.15.122:0/2405309672 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23600053c0 con 0x7f23700af4c0 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f2366a00640 1 -- 172.21.15.122:0/2405309672 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360003880 con 0x7f23700af4c0 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237015d320 con 0x7f23700af4c0 2024-09-17T05:28:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237015d8e0 con 0x7f23700af4c0 2024-09-17T05:28:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23700caf20 con 0x7f23700af4c0 2024-09-17T05:28:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.311+0000 7f2366a00640 1 -- 172.21.15.122:0/2405309672 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f236001a070 con 0x7f23700af4c0 2024-09-17T05:28:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.315+0000 7f2366a00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2350073560 0x7f2350075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.315+0000 7f2366a00640 1 -- 172.21.15.122:0/2405309672 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f2360093c40 con 0x7f23700af4c0 2024-09-17T05:28:08.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.315+0000 7f236e000640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2350073560 0x7f2350075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:08.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.315+0000 7f236e000640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2350073560 0x7f2350075a20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f235c0af000 tx=0x7f235c01b070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:08.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.319+0000 7f2366a00640 1 -- 172.21.15.122:0/2405309672 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2360001130 con 0x7f23700af4c0 2024-09-17T05:28:08.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.483+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2370069dd0 con 0x7f2350073560 2024-09-17T05:28:08.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.487+0000 7f2366a00640 1 -- 172.21.15.122:0/2405309672 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245788 (secure 0 0 0) 0x7f2370069dd0 con 0x7f2350073560 2024-09-17T05:28:08.491 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:08.493 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:28:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2350073560 msgr2=0x7f2350075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2350073560 0x7f2350075a20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f235c0af000 tx=0x7f235c01b070 comp rx=0 tx=0).stop 2024-09-17T05:28:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700af4c0 msgr2=0x7f2370159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700af4c0 0x7f2370159090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f236000c400 tx=0x7f236000c8c0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 shutdown_connections 2024-09-17T05:28:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2350073560 0x7f2350075a20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f237015a1e0 0x7f23700b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23700cd180 0x7f23701fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 --2- 172.21.15.122:0/2405309672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23700af4c0 0x7f2370159090 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 >> 172.21.15.122:0/2405309672 conn(0x7f2370073490 msgr2=0x7f23700708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 shutdown_connections 2024-09-17T05:28:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.491+0000 7f236fe00640 1 -- 172.21.15.122:0/2405309672 wait complete. 2024-09-17T05:28:08.511 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-17T05:28:06.489602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":166,"acting":166,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262928,"kb_used_data":5080,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711792,"statfs":{"total":773094113280,"available":772824875008,"internally_reserved":0,"allocated":5201920,"data_stored":2772428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":263835190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.109626"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387945+0000","last_change":"2024-09-17T05:28:03.387945+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387945+0000","last_undegraded":"2024-09-17T05:28:03.387945+0000","last_fullsized":"2024-09-17T05:28:03.387945+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388044+0000","last_change":"2024-09-17T05:28:03.388044+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388044+0000","last_undegraded":"2024-09-17T05:28:03.388044+0000","last_fullsized":"2024-09-17T05:28:03.388044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391128+0000","last_change":"2024-09-17T05:28:03.391128+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391128+0000","last_undegraded":"2024-09-17T05:28:03.391128+0000","last_fullsized":"2024-09-17T05:28:03.391128+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387704+0000","last_change":"2024-09-17T05:28:03.387704+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387704+0000","last_undegraded":"2024-09-17T05:28:03.387704+0000","last_fullsized":"2024-09-17T05:28:03.387704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.392926+0000","last_change":"2024-09-17T05:28:03.392926+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.392926+0000","last_undegraded":"2024-09-17T05:28:03.392926+0000","last_fullsized":"2024-09-17T05:28:03.392926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398593+0000","last_change":"2024-09-17T05:28:03.398593+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398593+0000","last_undegraded":"2024-09-17T05:28:03.398593+0000","last_fullsized":"2024-09-17T05:28:03.398593+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398634+0000","last_change":"2024-09-17T05:28:03.398634+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398634+0000","last_undegraded":"2024-09-17T05:28:03.398634+0000","last_fullsized":"2024-09-17T05:28:03.398634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393796+0000","last_change":"2024-09-17T05:28:03.393796+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393796+0000","last_undegraded":"2024-09-17T05:28:03.393796+0000","last_fullsized":"2024-09-17T05:28:03.393796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389118+0000","last_change":"2024-09-17T05:28:03.389118+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389118+0000","last_undegraded":"2024-09-17T05:28:03.389118+0000","last_fullsized":"2024-09-17T05:28:03.389118+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395047+0000","last_change":"2024-09-17T05:28:03.395047+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395047+0000","last_undegraded":"2024-09-17T05:28:03.395047+0000","last_fullsized":"2024-09-17T05:28:03.395047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386968+0000","last_change":"2024-09-17T05:28:03.386968+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386968+0000","last_undegraded":"2024-09-17T05:28:03.386968+0000","last_fullsized":"2024-09-17T05:28:03.386968+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387639+0000","last_change":"2024-09-17T05:28:03.387639+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387639+0000","last_undegraded":"2024-09-17T05:28:03.387639+0000","last_fullsized":"2024-09-17T05:28:03.387639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391815+0000","last_change":"2024-09-17T05:28:03.391815+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391815+0000","last_undegraded":"2024-09-17T05:28:03.391815+0000","last_fullsized":"2024-09-17T05:28:03.391815+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387562+0000","last_change":"2024-09-17T05:28:03.387562+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387562+0000","last_undegraded":"2024-09-17T05:28:03.387562+0000","last_fullsized":"2024-09-17T05:28:03.387562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395327+0000","last_change":"2024-09-17T05:28:03.395327+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395327+0000","last_undegraded":"2024-09-17T05:28:03.395327+0000","last_fullsized":"2024-09-17T05:28:03.395327+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389470+0000","last_change":"2024-09-17T05:28:03.389470+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389470+0000","last_undegraded":"2024-09-17T05:28:03.389470+0000","last_fullsized":"2024-09-17T05:28:03.389470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393313+0000","last_change":"2024-09-17T05:28:03.393313+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393313+0000","last_undegraded":"2024-09-17T05:28:03.393313+0000","last_fullsized":"2024-09-17T05:28:03.393313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387264+0000","last_change":"2024-09-17T05:28:03.387264+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387264+0000","last_undegraded":"2024-09-17T05:28:03.387264+0000","last_fullsized":"2024-09-17T05:28:03.387264+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386893+0000","last_change":"2024-09-17T05:28:03.386893+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386893+0000","last_undegraded":"2024-09-17T05:28:03.386893+0000","last_fullsized":"2024-09-17T05:28:03.386893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395190+0000","last_change":"2024-09-17T05:28:03.395190+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395190+0000","last_undegraded":"2024-09-17T05:28:03.395190+0000","last_fullsized":"2024-09-17T05:28:03.395190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393821+0000","last_change":"2024-09-17T05:28:03.393821+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393821+0000","last_undegraded":"2024-09-17T05:28:03.393821+0000","last_fullsized":"2024-09-17T05:28:03.393821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386847+0000","last_change":"2024-09-17T05:28:03.386847+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386847+0000","last_undegraded":"2024-09-17T05:28:03.386847+0000","last_fullsized":"2024-09-17T05:28:03.386847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387350+0000","last_change":"2024-09-17T05:28:03.387350+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387350+0000","last_undegraded":"2024-09-17T05:28:03.387350+0000","last_fullsized":"2024-09-17T05:28:03.387350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394182+0000","last_change":"2024-09-17T05:28:03.394182+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394182+0000","last_undegraded":"2024-09-17T05:28:03.394182+0000","last_fullsized":"2024-09-17T05:28:03.394182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389389+0000","last_change":"2024-09-17T05:28:03.389389+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389389+0000","last_undegraded":"2024-09-17T05:28:03.389389+0000","last_fullsized":"2024-09-17T05:28:03.389389+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411758+0000","last_change":"2024-09-17T05:28:03.408603+0000","last_active":"2024-09-17T05:28:03.411758+0000","last_peered":"2024-09-17T05:28:03.411758+0000","last_clean":"2024-09-17T05:28:03.411758+0000","last_became_active":"2024-09-17T05:28:03.408354+0000","last_became_peered":"2024-09-17T05:28:03.408354+0000","last_unstale":"2024-09-17T05:28:03.411758+0000","last_undegraded":"2024-09-17T05:28:03.411758+0000","last_fullsized":"2024-09-17T05:28:03.411758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:20.506564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386337+0000","last_change":"2024-09-17T05:28:03.386337+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386337+0000","last_undegraded":"2024-09-17T05:28:03.386337+0000","last_fullsized":"2024-09-17T05:28:03.386337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408180+0000","last_change":"2024-09-17T05:28:03.403643+0000","last_active":"2024-09-17T05:28:03.408180+0000","last_peered":"2024-09-17T05:28:03.408180+0000","last_clean":"2024-09-17T05:28:03.408180+0000","last_became_active":"2024-09-17T05:28:03.403353+0000","last_became_peered":"2024-09-17T05:28:03.403353+0000","last_unstale":"2024-09-17T05:28:03.408180+0000","last_undegraded":"2024-09-17T05:28:03.408180+0000","last_fullsized":"2024-09-17T05:28:03.408180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:30.436401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387803+0000","last_change":"2024-09-17T05:28:03.387803+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387803+0000","last_undegraded":"2024-09-17T05:28:03.387803+0000","last_fullsized":"2024-09-17T05:28:03.387803+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411701+0000","last_change":"2024-09-17T05:28:03.407955+0000","last_active":"2024-09-17T05:28:03.411701+0000","last_peered":"2024-09-17T05:28:03.411701+0000","last_clean":"2024-09-17T05:28:03.411701+0000","last_became_active":"2024-09-17T05:28:03.407597+0000","last_became_peered":"2024-09-17T05:28:03.407597+0000","last_unstale":"2024-09-17T05:28:03.411701+0000","last_undegraded":"2024-09-17T05:28:03.411701+0000","last_fullsized":"2024-09-17T05:28:03.411701+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:31.638210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410169+0000","last_change":"2024-09-17T05:28:03.406275+0000","last_active":"2024-09-17T05:28:03.410169+0000","last_peered":"2024-09-17T05:28:03.410169+0000","last_clean":"2024-09-17T05:28:03.410169+0000","last_became_active":"2024-09-17T05:28:03.405506+0000","last_became_peered":"2024-09-17T05:28:03.405506+0000","last_unstale":"2024-09-17T05:28:03.410169+0000","last_undegraded":"2024-09-17T05:28:03.410169+0000","last_fullsized":"2024-09-17T05:28:03.410169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:46.627002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395704+0000","last_change":"2024-09-17T05:28:03.395704+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395704+0000","last_undegraded":"2024-09-17T05:28:03.395704+0000","last_fullsized":"2024-09-17T05:28:03.395704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410230+0000","last_change":"2024-09-17T05:28:03.406212+0000","last_active":"2024-09-17T05:28:03.410230+0000","last_peered":"2024-09-17T05:28:03.410230+0000","last_clean":"2024-09-17T05:28:03.410230+0000","last_became_active":"2024-09-17T05:28:03.405428+0000","last_became_peered":"2024-09-17T05:28:03.405428+0000","last_unstale":"2024-09-17T05:28:03.410230+0000","last_undegraded":"2024-09-17T05:28:03.410230+0000","last_fullsized":"2024-09-17T05:28:03.410230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:08.586828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394459+0000","last_change":"2024-09-17T05:28:03.394459+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394459+0000","last_undegraded":"2024-09-17T05:28:03.394459+0000","last_fullsized":"2024-09-17T05:28:03.394459+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410102+0000","last_change":"2024-09-17T05:28:03.405006+0000","last_active":"2024-09-17T05:28:03.410102+0000","last_peered":"2024-09-17T05:28:03.410102+0000","last_clean":"2024-09-17T05:28:03.410102+0000","last_became_active":"2024-09-17T05:28:03.404741+0000","last_became_peered":"2024-09-17T05:28:03.404741+0000","last_unstale":"2024-09-17T05:28:03.410102+0000","last_undegraded":"2024-09-17T05:28:03.410102+0000","last_fullsized":"2024-09-17T05:28:03.410102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:23.814938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393871+0000","last_change":"2024-09-17T05:28:03.393871+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393871+0000","last_undegraded":"2024-09-17T05:28:03.393871+0000","last_fullsized":"2024-09-17T05:28:03.393871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.399582+0000","last_change":"2024-09-17T05:28:03.397856+0000","last_active":"2024-09-17T05:28:03.399582+0000","last_peered":"2024-09-17T05:28:03.399582+0000","last_clean":"2024-09-17T05:28:03.399582+0000","last_became_active":"2024-09-17T05:28:03.397609+0000","last_became_peered":"2024-09-17T05:28:03.397609+0000","last_unstale":"2024-09-17T05:28:03.399582+0000","last_undegraded":"2024-09-17T05:28:03.399582+0000","last_fullsized":"2024-09-17T05:28:03.399582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:40.677984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393780+0000","last_change":"2024-09-17T05:28:03.393780+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393780+0000","last_undegraded":"2024-09-17T05:28:03.393780+0000","last_fullsized":"2024-09-17T05:28:03.393780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.414117+0000","last_change":"2024-09-17T05:28:03.410366+0000","last_active":"2024-09-17T05:28:03.414117+0000","last_peered":"2024-09-17T05:28:03.414117+0000","last_clean":"2024-09-17T05:28:03.414117+0000","last_became_active":"2024-09-17T05:28:03.409222+0000","last_became_peered":"2024-09-17T05:28:03.409222+0000","last_unstale":"2024-09-17T05:28:03.414117+0000","last_undegraded":"2024-09-17T05:28:03.414117+0000","last_fullsized":"2024-09-17T05:28:03.414117+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:11.005389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391108+0000","last_change":"2024-09-17T05:28:03.391108+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391108+0000","last_undegraded":"2024-09-17T05:28:03.391108+0000","last_fullsized":"2024-09-17T05:28:03.391108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.403575+0000","last_change":"2024-09-17T05:28:03.400471+0000","last_active":"2024-09-17T05:28:03.403575+0000","last_peered":"2024-09-17T05:28:03.403575+0000","last_clean":"2024-09-17T05:28:03.403575+0000","last_became_active":"2024-09-17T05:28:03.400202+0000","last_became_peered":"2024-09-17T05:28:03.400202+0000","last_unstale":"2024-09-17T05:28:03.403575+0000","last_undegraded":"2024-09-17T05:28:03.403575+0000","last_fullsized":"2024-09-17T05:28:03.403575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:58.648337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.390844+0000","last_change":"2024-09-17T05:28:03.390844+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.390844+0000","last_undegraded":"2024-09-17T05:28:03.390844+0000","last_fullsized":"2024-09-17T05:28:03.390844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.409027+0000","last_change":"2024-09-17T05:28:03.401227+0000","last_active":"2024-09-17T05:28:03.409027+0000","last_peered":"2024-09-17T05:28:03.409027+0000","last_clean":"2024-09-17T05:28:03.409027+0000","last_became_active":"2024-09-17T05:28:03.400688+0000","last_became_peered":"2024-09-17T05:28:03.400688+0000","last_unstale":"2024-09-17T05:28:03.409027+0000","last_undegraded":"2024-09-17T05:28:03.409027+0000","last_fullsized":"2024-09-17T05:28:03.409027+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:45.856441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407650+0000","last_change":"2024-09-17T05:28:03.401672+0000","last_active":"2024-09-17T05:28:03.407650+0000","last_peered":"2024-09-17T05:28:03.407650+0000","last_clean":"2024-09-17T05:28:03.407650+0000","last_became_active":"2024-09-17T05:28:03.401380+0000","last_became_peered":"2024-09-17T05:28:03.401380+0000","last_unstale":"2024-09-17T05:28:03.407650+0000","last_undegraded":"2024-09-17T05:28:03.407650+0000","last_fullsized":"2024-09-17T05:28:03.407650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:57.580288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394819+0000","last_change":"2024-09-17T05:28:03.394819+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394819+0000","last_undegraded":"2024-09-17T05:28:03.394819+0000","last_fullsized":"2024-09-17T05:28:03.394819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411516+0000","last_change":"2024-09-17T05:28:03.408491+0000","last_active":"2024-09-17T05:28:03.411516+0000","last_peered":"2024-09-17T05:28:03.411516+0000","last_clean":"2024-09-17T05:28:03.411516+0000","last_became_active":"2024-09-17T05:28:03.407793+0000","last_became_peered":"2024-09-17T05:28:03.407793+0000","last_unstale":"2024-09-17T05:28:03.411516+0000","last_undegraded":"2024-09-17T05:28:03.411516+0000","last_fullsized":"2024-09-17T05:28:03.411516+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.426226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389843+0000","last_change":"2024-09-17T05:28:03.389843+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389843+0000","last_undegraded":"2024-09-17T05:28:03.389843+0000","last_fullsized":"2024-09-17T05:28:03.389843+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411542+0000","last_change":"2024-09-17T05:28:03.406639+0000","last_active":"2024-09-17T05:28:03.411542+0000","last_peered":"2024-09-17T05:28:03.411542+0000","last_clean":"2024-09-17T05:28:03.411542+0000","last_became_active":"2024-09-17T05:28:03.406360+0000","last_became_peered":"2024-09-17T05:28:03.406360+0000","last_unstale":"2024-09-17T05:28:03.411542+0000","last_undegraded":"2024-09-17T05:28:03.411542+0000","last_fullsized":"2024-09-17T05:28:03.411542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:17.531567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.404093+0000","last_change":"2024-09-17T05:28:03.401363+0000","last_active":"2024-09-17T05:28:03.404093+0000","last_peered":"2024-09-17T05:28:03.404093+0000","last_clean":"2024-09-17T05:28:03.404093+0000","last_became_active":"2024-09-17T05:28:03.401110+0000","last_became_peered":"2024-09-17T05:28:03.401110+0000","last_unstale":"2024-09-17T05:28:03.404093+0000","last_undegraded":"2024-09-17T05:28:03.404093+0000","last_fullsized":"2024-09-17T05:28:03.404093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:24.220817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.413897+0000","last_change":"2024-09-17T05:28:03.409353+0000","last_active":"2024-09-17T05:28:03.413897+0000","last_peered":"2024-09-17T05:28:03.413897+0000","last_clean":"2024-09-17T05:28:03.413897+0000","last_became_active":"2024-09-17T05:28:03.409087+0000","last_became_peered":"2024-09-17T05:28:03.409087+0000","last_unstale":"2024-09-17T05:28:03.413897+0000","last_undegraded":"2024-09-17T05:28:03.413897+0000","last_fullsized":"2024-09-17T05:28:03.413897+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:37.361308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388677+0000","last_change":"2024-09-17T05:28:03.388677+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388677+0000","last_undegraded":"2024-09-17T05:28:03.388677+0000","last_fullsized":"2024-09-17T05:28:03.388677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.409986+0000","last_change":"2024-09-17T05:28:03.403774+0000","last_active":"2024-09-17T05:28:03.409986+0000","last_peered":"2024-09-17T05:28:03.409986+0000","last_clean":"2024-09-17T05:28:03.409986+0000","last_became_active":"2024-09-17T05:28:03.403425+0000","last_became_peered":"2024-09-17T05:28:03.403425+0000","last_unstale":"2024-09-17T05:28:03.409986+0000","last_undegraded":"2024-09-17T05:28:03.409986+0000","last_fullsized":"2024-09-17T05:28:03.409986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:08.155865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388720+0000","last_change":"2024-09-17T05:28:03.388720+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388720+0000","last_undegraded":"2024-09-17T05:28:03.388720+0000","last_fullsized":"2024-09-17T05:28:03.388720+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-17T05:28:02.382518+0000","last_change":"2024-09-17T05:28:02.382518+0000","last_active":"2024-09-17T05:28:02.370138+0000","last_peered":"2024-09-17T05:28:02.370138+0000","last_clean":"2024-09-17T05:28:02.370138+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:02.382518+0000","last_undegraded":"2024-09-17T05:28:02.382518+0000","last_fullsized":"2024-09-17T05:28:02.382518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412689+0000","last_change":"2024-09-17T05:28:03.408640+0000","last_active":"2024-09-17T05:28:03.412689+0000","last_peered":"2024-09-17T05:28:03.412689+0000","last_clean":"2024-09-17T05:28:03.412689+0000","last_became_active":"2024-09-17T05:28:03.408371+0000","last_became_peered":"2024-09-17T05:28:03.408371+0000","last_unstale":"2024-09-17T05:28:03.412689+0000","last_undegraded":"2024-09-17T05:28:03.412689+0000","last_fullsized":"2024-09-17T05:28:03.412689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:51.246256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387320+0000","last_change":"2024-09-17T05:28:03.387320+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387320+0000","last_undegraded":"2024-09-17T05:28:03.387320+0000","last_fullsized":"2024-09-17T05:28:03.387320+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.403739+0000","last_change":"2024-09-17T05:28:03.398552+0000","last_active":"2024-09-17T05:28:03.403739+0000","last_peered":"2024-09-17T05:28:03.403739+0000","last_clean":"2024-09-17T05:28:03.403739+0000","last_became_active":"2024-09-17T05:28:03.398282+0000","last_became_peered":"2024-09-17T05:28:03.398282+0000","last_unstale":"2024-09-17T05:28:03.403739+0000","last_undegraded":"2024-09-17T05:28:03.403739+0000","last_fullsized":"2024-09-17T05:28:03.403739+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:54.593079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.390768+0000","last_change":"2024-09-17T05:28:03.390768+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.390768+0000","last_undegraded":"2024-09-17T05:28:03.390768+0000","last_fullsized":"2024-09-17T05:28:03.390768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410212+0000","last_change":"2024-09-17T05:28:03.404997+0000","last_active":"2024-09-17T05:28:03.410212+0000","last_peered":"2024-09-17T05:28:03.410212+0000","last_clean":"2024-09-17T05:28:03.410212+0000","last_became_active":"2024-09-17T05:28:03.404512+0000","last_became_peered":"2024-09-17T05:28:03.404512+0000","last_unstale":"2024-09-17T05:28:03.410212+0000","last_undegraded":"2024-09-17T05:28:03.410212+0000","last_fullsized":"2024-09-17T05:28:03.410212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:07.584572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398333+0000","last_change":"2024-09-17T05:28:03.398333+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398333+0000","last_undegraded":"2024-09-17T05:28:03.398333+0000","last_fullsized":"2024-09-17T05:28:03.398333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408284+0000","last_change":"2024-09-17T05:28:03.400873+0000","last_active":"2024-09-17T05:28:03.408284+0000","last_peered":"2024-09-17T05:28:03.408284+0000","last_clean":"2024-09-17T05:28:03.408284+0000","last_became_active":"2024-09-17T05:28:03.400549+0000","last_became_peered":"2024-09-17T05:28:03.400549+0000","last_unstale":"2024-09-17T05:28:03.408284+0000","last_undegraded":"2024-09-17T05:28:03.408284+0000","last_fullsized":"2024-09-17T05:28:03.408284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:31.777509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394647+0000","last_change":"2024-09-17T05:28:03.394647+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394647+0000","last_undegraded":"2024-09-17T05:28:03.394647+0000","last_fullsized":"2024-09-17T05:28:03.394647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.398760+0000","last_change":"2024-09-17T05:28:03.395798+0000","last_active":"2024-09-17T05:28:03.398760+0000","last_peered":"2024-09-17T05:28:03.398760+0000","last_clean":"2024-09-17T05:28:03.398760+0000","last_became_active":"2024-09-17T05:28:03.395543+0000","last_became_peered":"2024-09-17T05:28:03.395543+0000","last_unstale":"2024-09-17T05:28:03.398760+0000","last_undegraded":"2024-09-17T05:28:03.398760+0000","last_fullsized":"2024-09-17T05:28:03.398760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:37.361308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408407+0000","last_change":"2024-09-17T05:28:03.400992+0000","last_active":"2024-09-17T05:28:03.408407+0000","last_peered":"2024-09-17T05:28:03.408407+0000","last_clean":"2024-09-17T05:28:03.408407+0000","last_became_active":"2024-09-17T05:28:03.400690+0000","last_became_peered":"2024-09-17T05:28:03.400690+0000","last_unstale":"2024-09-17T05:28:03.408407+0000","last_undegraded":"2024-09-17T05:28:03.408407+0000","last_fullsized":"2024-09-17T05:28:03.408407+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:49.611905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412599+0000","last_change":"2024-09-17T05:28:03.407318+0000","last_active":"2024-09-17T05:28:03.412599+0000","last_peered":"2024-09-17T05:28:03.412599+0000","last_clean":"2024-09-17T05:28:03.412599+0000","last_became_active":"2024-09-17T05:28:03.407059+0000","last_became_peered":"2024-09-17T05:28:03.407059+0000","last_unstale":"2024-09-17T05:28:03.412599+0000","last_undegraded":"2024-09-17T05:28:03.412599+0000","last_fullsized":"2024-09-17T05:28:03.412599+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:15.545411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393056+0000","last_change":"2024-09-17T05:28:03.393056+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393056+0000","last_undegraded":"2024-09-17T05:28:03.393056+0000","last_fullsized":"2024-09-17T05:28:03.393056+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-09-17T05:28:02.382350+0000","last_change":"2024-09-17T05:28:02.382350+0000","last_active":"2024-09-17T05:28:02.370138+0000","last_peered":"2024-09-17T05:28:02.370138+0000","last_clean":"2024-09-17T05:28:02.370138+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:02.382350+0000","last_undegraded":"2024-09-17T05:28:02.382350+0000","last_fullsized":"2024-09-17T05:28:02.382350+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394936+0000","last_change":"2024-09-17T05:28:03.394936+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394936+0000","last_undegraded":"2024-09-17T05:28:03.394936+0000","last_fullsized":"2024-09-17T05:28:03.394936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.402356+0000","last_change":"2024-09-17T05:28:03.397060+0000","last_active":"2024-09-17T05:28:03.402356+0000","last_peered":"2024-09-17T05:28:03.402356+0000","last_clean":"2024-09-17T05:28:03.402356+0000","last_became_active":"2024-09-17T05:28:03.396798+0000","last_became_peered":"2024-09-17T05:28:03.396798+0000","last_unstale":"2024-09-17T05:28:03.402356+0000","last_undegraded":"2024-09-17T05:28:03.402356+0000","last_fullsized":"2024-09-17T05:28:03.402356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.747610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.392714+0000","last_change":"2024-09-17T05:28:03.392714+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.392714+0000","last_undegraded":"2024-09-17T05:28:03.392714+0000","last_fullsized":"2024-09-17T05:28:03.392714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407241+0000","last_change":"2024-09-17T05:28:03.402041+0000","last_active":"2024-09-17T05:28:03.407241+0000","last_peered":"2024-09-17T05:28:03.407241+0000","last_clean":"2024-09-17T05:28:03.407241+0000","last_became_active":"2024-09-17T05:28:03.401757+0000","last_became_peered":"2024-09-17T05:28:03.401757+0000","last_unstale":"2024-09-17T05:28:03.407241+0000","last_undegraded":"2024-09-17T05:28:03.407241+0000","last_fullsized":"2024-09-17T05:28:03.407241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:39.617757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.397895+0000","last_change":"2024-09-17T05:28:03.397895+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.397895+0000","last_undegraded":"2024-09-17T05:28:03.397895+0000","last_fullsized":"2024-09-17T05:28:03.397895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407808+0000","last_change":"2024-09-17T05:28:03.406099+0000","last_active":"2024-09-17T05:28:03.407808+0000","last_peered":"2024-09-17T05:28:03.407808+0000","last_clean":"2024-09-17T05:28:03.407808+0000","last_became_active":"2024-09-17T05:28:03.405763+0000","last_became_peered":"2024-09-17T05:28:03.405763+0000","last_unstale":"2024-09-17T05:28:03.407808+0000","last_undegraded":"2024-09-17T05:28:03.407808+0000","last_fullsized":"2024-09-17T05:28:03.407808+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:57.580288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.405322+0000","last_change":"2024-09-17T05:28:03.403069+0000","last_active":"2024-09-17T05:28:03.405322+0000","last_peered":"2024-09-17T05:28:03.405322+0000","last_clean":"2024-09-17T05:28:03.405322+0000","last_became_active":"2024-09-17T05:28:03.402652+0000","last_became_peered":"2024-09-17T05:28:03.402652+0000","last_unstale":"2024-09-17T05:28:03.405322+0000","last_undegraded":"2024-09-17T05:28:03.405322+0000","last_fullsized":"2024-09-17T05:28:03.405322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:42.875418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389143+0000","last_change":"2024-09-17T05:28:03.389143+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389143+0000","last_undegraded":"2024-09-17T05:28:03.389143+0000","last_fullsized":"2024-09-17T05:28:03.389143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411465+0000","last_change":"2024-09-17T05:28:03.407068+0000","last_active":"2024-09-17T05:28:03.411465+0000","last_peered":"2024-09-17T05:28:03.411465+0000","last_clean":"2024-09-17T05:28:03.411465+0000","last_became_active":"2024-09-17T05:28:03.406674+0000","last_became_peered":"2024-09-17T05:28:03.406674+0000","last_unstale":"2024-09-17T05:28:03.411465+0000","last_undegraded":"2024-09-17T05:28:03.411465+0000","last_fullsized":"2024-09-17T05:28:03.411465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:54.447845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387799+0000","last_change":"2024-09-17T05:28:03.387799+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387799+0000","last_undegraded":"2024-09-17T05:28:03.387799+0000","last_fullsized":"2024-09-17T05:28:03.387799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407397+0000","last_change":"2024-09-17T05:28:03.402396+0000","last_active":"2024-09-17T05:28:03.407397+0000","last_peered":"2024-09-17T05:28:03.407397+0000","last_clean":"2024-09-17T05:28:03.407397+0000","last_became_active":"2024-09-17T05:28:03.402089+0000","last_became_peered":"2024-09-17T05:28:03.402089+0000","last_unstale":"2024-09-17T05:28:03.407397+0000","last_undegraded":"2024-09-17T05:28:03.407397+0000","last_fullsized":"2024-09-17T05:28:03.407397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:31.994460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:28:03.373314+0000","last_change":"2024-09-17T05:28:03.373314+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.373314+0000","last_undegraded":"2024-09-17T05:28:03.373314+0000","last_fullsized":"2024-09-17T05:28:03.373314+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412718+0000","last_change":"2024-09-17T05:28:03.408678+0000","last_active":"2024-09-17T05:28:03.412718+0000","last_peered":"2024-09-17T05:28:03.412718+0000","last_clean":"2024-09-17T05:28:03.412718+0000","last_became_active":"2024-09-17T05:28:03.408375+0000","last_became_peered":"2024-09-17T05:28:03.408375+0000","last_unstale":"2024-09-17T05:28:03.412718+0000","last_undegraded":"2024-09-17T05:28:03.412718+0000","last_fullsized":"2024-09-17T05:28:03.412718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:10.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":153,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.393074+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:28:03.393074+0000","last_peered":"2024-09-17T05:28:03.393074+0000","last_clean":"2024-09-17T05:28:03.393074+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:28:03.393074+0000","last_undegraded":"2024-09-17T05:28:03.393074+0000","last_fullsized":"2024-09-17T05:28:03.393074+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":792467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":786192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:08.516 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:08.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- 172.21.15.122:0/745971066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8014004810 msgr2=0x7f8014004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 --2- 172.21.15.122:0/745971066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8014004810 0x7f8014004c90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8018009b60 tx=0x7f801801d210 comp rx=0 tx=0).stop 2024-09-17T05:28:08.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802f400640 1 -- 172.21.15.122:0/745971066 reap_dead start 2024-09-17T05:28:08.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- 172.21.15.122:0/745971066 shutdown_connections 2024-09-17T05:28:08.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- 172.21.15.122:0/745971066 >> 172.21.15.122:0/745971066 conn(0x7f80300d1d90 msgr2=0x7f80300d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:08.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- 172.21.15.122:0/745971066 shutdown_connections 2024-09-17T05:28:08.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- 172.21.15.122:0/745971066 wait complete. 2024-09-17T05:28:08.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 Processor -- start 2024-09-17T05:28:08.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- start start 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80301f2520 0x7f803015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80301f2900 0x7f803015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803014e6c0 0x7f80300c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80301604e0 con 0x7f80301f2520 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8030160360 con 0x7f80301f2900 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8035a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8030160660 con 0x7f803014e6c0 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80301f2520 0x7f803015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802f400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803014e6c0 0x7f80300c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80301f2520 0x7f803015d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51848/0 (socket says 172.21.15.122:51848) 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802ea00640 1 -- 172.21.15.122:0/203211843 learned_addr learned my addr 172.21.15.122:0/203211843 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:08.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f8026000640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80301f2900 0x7f803015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:08.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802ea00640 1 -- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803014e6c0 msgr2=0x7f80300c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802ea00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803014e6c0 0x7f80300c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802ea00640 1 -- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80301f2900 msgr2=0x7f803015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802ea00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80301f2900 0x7f803015dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.695+0000 7f802ea00640 1 -- 172.21.15.122:0/203211843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803015a9c0 con 0x7f80301f2520 2024-09-17T05:28:08.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f802ea00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80301f2520 0x7f803015d9a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f80200017d0 tx=0x7f802000e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:08.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8027400640 1 -- 172.21.15.122:0/203211843 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020012850 con 0x7f80301f2520 2024-09-17T05:28:08.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8027400640 1 -- 172.21.15.122:0/203211843 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8020012b00 con 0x7f80301f2520 2024-09-17T05:28:08.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8027400640 1 -- 172.21.15.122:0/203211843 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802001a400 con 0x7f80301f2520 2024-09-17T05:28:08.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803015e650 con 0x7f80301f2520 2024-09-17T05:28:08.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803015ec10 con 0x7f80301f2520 2024-09-17T05:28:08.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80300b53e0 con 0x7f80301f2520 2024-09-17T05:28:08.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8027400640 1 -- 172.21.15.122:0/203211843 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f802000d030 con 0x7f80301f2520 2024-09-17T05:28:08.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8027400640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c073590 0x7f800c075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:08.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8027400640 1 -- 172.21.15.122:0/203211843 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f80200979c0 con 0x7f80301f2520 2024-09-17T05:28:08.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.699+0000 7f8026000640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c073590 0x7f800c075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:08.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.703+0000 7f8026000640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c073590 0x7f800c075a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80180ae000 tx=0x7f801801a010 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:08.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.703+0000 7f8027400640 1 -- 172.21.15.122:0/203211843 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8020016030 con 0x7f80301f2520 2024-09-17T05:28:08.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.867+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80300d1b70 con 0x7f800c073590 2024-09-17T05:28:08.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.875+0000 7f8027400640 1 -- 172.21.15.122:0/203211843 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247807 (secure 0 0 0) 0x7f80300d1b70 con 0x7f800c073590 2024-09-17T05:28:08.877 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:08.878 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:28:08.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c073590 msgr2=0x7f800c075a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c073590 0x7f800c075a50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f80180ae000 tx=0x7f801801a010 comp rx=0 tx=0).stop 2024-09-17T05:28:08.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80301f2520 msgr2=0x7f803015d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:08.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80301f2520 0x7f803015d9a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f80200017d0 tx=0x7f802000e770 comp rx=0 tx=0).stop 2024-09-17T05:28:08.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 shutdown_connections 2024-09-17T05:28:08.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c073590 0x7f800c075a50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f803014e6c0 0x7f80300c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80301f2900 0x7f803015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 --2- 172.21.15.122:0/203211843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80301f2520 0x7f803015d9a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:08.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 >> 172.21.15.122:0/203211843 conn(0x7f80300d1d90 msgr2=0x7f80300cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:08.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 shutdown_connections 2024-09-17T05:28:08.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:08.879+0000 7f8035a00640 1 -- 172.21.15.122:0/203211843 wait complete. 2024-09-17T05:28:08.898 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-17T05:28:08.492320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262972,"kb_used_data":5120,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711748,"statfs":{"total":773094113280,"available":772824829952,"internally_reserved":0,"allocated":5242880,"data_stored":2791800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":263835190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003592"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387945+0000","last_change":"2024-09-17T05:28:03.387945+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387945+0000","last_undegraded":"2024-09-17T05:28:03.387945+0000","last_fullsized":"2024-09-17T05:28:03.387945+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388044+0000","last_change":"2024-09-17T05:28:03.388044+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388044+0000","last_undegraded":"2024-09-17T05:28:03.388044+0000","last_fullsized":"2024-09-17T05:28:03.388044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391128+0000","last_change":"2024-09-17T05:28:03.391128+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391128+0000","last_undegraded":"2024-09-17T05:28:03.391128+0000","last_fullsized":"2024-09-17T05:28:03.391128+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409677+0000","last_change":"2024-09-17T05:28:04.416129+0000","last_active":"2024-09-17T05:28:05.409677+0000","last_peered":"2024-09-17T05:28:05.409677+0000","last_clean":"2024-09-17T05:28:05.409677+0000","last_became_active":"2024-09-17T05:28:04.415869+0000","last_became_peered":"2024-09-17T05:28:04.415869+0000","last_unstale":"2024-09-17T05:28:05.409677+0000","last_undegraded":"2024-09-17T05:28:05.409677+0000","last_fullsized":"2024-09-17T05:28:05.409677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.172592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387704+0000","last_change":"2024-09-17T05:28:03.387704+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387704+0000","last_undegraded":"2024-09-17T05:28:03.387704+0000","last_fullsized":"2024-09-17T05:28:03.387704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.392926+0000","last_change":"2024-09-17T05:28:03.392926+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.392926+0000","last_undegraded":"2024-09-17T05:28:03.392926+0000","last_fullsized":"2024-09-17T05:28:03.392926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398593+0000","last_change":"2024-09-17T05:28:03.398593+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398593+0000","last_undegraded":"2024-09-17T05:28:03.398593+0000","last_fullsized":"2024-09-17T05:28:03.398593+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.408875+0000","last_change":"2024-09-17T05:28:04.421456+0000","last_active":"2024-09-17T05:28:05.408875+0000","last_peered":"2024-09-17T05:28:05.408875+0000","last_clean":"2024-09-17T05:28:05.408875+0000","last_became_active":"2024-09-17T05:28:04.421034+0000","last_became_peered":"2024-09-17T05:28:04.421034+0000","last_unstale":"2024-09-17T05:28:05.408875+0000","last_undegraded":"2024-09-17T05:28:05.408875+0000","last_fullsized":"2024-09-17T05:28:05.408875+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:41.119416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398634+0000","last_change":"2024-09-17T05:28:03.398634+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398634+0000","last_undegraded":"2024-09-17T05:28:03.398634+0000","last_fullsized":"2024-09-17T05:28:03.398634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393796+0000","last_change":"2024-09-17T05:28:03.393796+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393796+0000","last_undegraded":"2024-09-17T05:28:03.393796+0000","last_fullsized":"2024-09-17T05:28:03.393796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409905+0000","last_change":"2024-09-17T05:28:04.419275+0000","last_active":"2024-09-17T05:28:05.409905+0000","last_peered":"2024-09-17T05:28:05.409905+0000","last_clean":"2024-09-17T05:28:05.409905+0000","last_became_active":"2024-09-17T05:28:04.418991+0000","last_became_peered":"2024-09-17T05:28:04.418991+0000","last_unstale":"2024-09-17T05:28:05.409905+0000","last_undegraded":"2024-09-17T05:28:05.409905+0000","last_fullsized":"2024-09-17T05:28:05.409905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:23.031211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389118+0000","last_change":"2024-09-17T05:28:03.389118+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389118+0000","last_undegraded":"2024-09-17T05:28:03.389118+0000","last_fullsized":"2024-09-17T05:28:03.389118+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395047+0000","last_change":"2024-09-17T05:28:03.395047+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395047+0000","last_undegraded":"2024-09-17T05:28:03.395047+0000","last_fullsized":"2024-09-17T05:28:03.395047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409281+0000","last_change":"2024-09-17T05:28:04.411688+0000","last_active":"2024-09-17T05:28:05.409281+0000","last_peered":"2024-09-17T05:28:05.409281+0000","last_clean":"2024-09-17T05:28:05.409281+0000","last_became_active":"2024-09-17T05:28:04.411403+0000","last_became_peered":"2024-09-17T05:28:04.411403+0000","last_unstale":"2024-09-17T05:28:05.409281+0000","last_undegraded":"2024-09-17T05:28:05.409281+0000","last_fullsized":"2024-09-17T05:28:05.409281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:17.638839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409054+0000","last_change":"2024-09-17T05:28:04.423517+0000","last_active":"2024-09-17T05:28:05.409054+0000","last_peered":"2024-09-17T05:28:05.409054+0000","last_clean":"2024-09-17T05:28:05.409054+0000","last_became_active":"2024-09-17T05:28:04.423243+0000","last_became_peered":"2024-09-17T05:28:04.423243+0000","last_unstale":"2024-09-17T05:28:05.409054+0000","last_undegraded":"2024-09-17T05:28:05.409054+0000","last_fullsized":"2024-09-17T05:28:05.409054+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:24.878942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386968+0000","last_change":"2024-09-17T05:28:03.386968+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386968+0000","last_undegraded":"2024-09-17T05:28:03.386968+0000","last_fullsized":"2024-09-17T05:28:03.386968+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387639+0000","last_change":"2024-09-17T05:28:03.387639+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387639+0000","last_undegraded":"2024-09-17T05:28:03.387639+0000","last_fullsized":"2024-09-17T05:28:03.387639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391815+0000","last_change":"2024-09-17T05:28:03.391815+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391815+0000","last_undegraded":"2024-09-17T05:28:03.391815+0000","last_fullsized":"2024-09-17T05:28:03.391815+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387562+0000","last_change":"2024-09-17T05:28:03.387562+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387562+0000","last_undegraded":"2024-09-17T05:28:03.387562+0000","last_fullsized":"2024-09-17T05:28:03.387562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395327+0000","last_change":"2024-09-17T05:28:03.395327+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395327+0000","last_undegraded":"2024-09-17T05:28:03.395327+0000","last_fullsized":"2024-09-17T05:28:03.395327+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409823+0000","last_change":"2024-09-17T05:28:04.421922+0000","last_active":"2024-09-17T05:28:05.409823+0000","last_peered":"2024-09-17T05:28:05.409823+0000","last_clean":"2024-09-17T05:28:05.409823+0000","last_became_active":"2024-09-17T05:28:04.414061+0000","last_became_peered":"2024-09-17T05:28:04.414061+0000","last_unstale":"2024-09-17T05:28:05.409823+0000","last_undegraded":"2024-09-17T05:28:05.409823+0000","last_fullsized":"2024-09-17T05:28:05.409823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:35.698197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389470+0000","last_change":"2024-09-17T05:28:03.389470+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389470+0000","last_undegraded":"2024-09-17T05:28:03.389470+0000","last_fullsized":"2024-09-17T05:28:03.389470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393313+0000","last_change":"2024-09-17T05:28:03.393313+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393313+0000","last_undegraded":"2024-09-17T05:28:03.393313+0000","last_fullsized":"2024-09-17T05:28:03.393313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387264+0000","last_change":"2024-09-17T05:28:03.387264+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387264+0000","last_undegraded":"2024-09-17T05:28:03.387264+0000","last_fullsized":"2024-09-17T05:28:03.387264+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386893+0000","last_change":"2024-09-17T05:28:03.386893+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386893+0000","last_undegraded":"2024-09-17T05:28:03.386893+0000","last_fullsized":"2024-09-17T05:28:03.386893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410396+0000","last_change":"2024-09-17T05:28:04.419337+0000","last_active":"2024-09-17T05:28:05.410396+0000","last_peered":"2024-09-17T05:28:05.410396+0000","last_clean":"2024-09-17T05:28:05.410396+0000","last_became_active":"2024-09-17T05:28:04.419066+0000","last_became_peered":"2024-09-17T05:28:04.419066+0000","last_unstale":"2024-09-17T05:28:05.410396+0000","last_undegraded":"2024-09-17T05:28:05.410396+0000","last_fullsized":"2024-09-17T05:28:05.410396+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:28.650511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395190+0000","last_change":"2024-09-17T05:28:03.395190+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395190+0000","last_undegraded":"2024-09-17T05:28:03.395190+0000","last_fullsized":"2024-09-17T05:28:03.395190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393821+0000","last_change":"2024-09-17T05:28:03.393821+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393821+0000","last_undegraded":"2024-09-17T05:28:03.393821+0000","last_fullsized":"2024-09-17T05:28:03.393821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386847+0000","last_change":"2024-09-17T05:28:03.386847+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386847+0000","last_undegraded":"2024-09-17T05:28:03.386847+0000","last_fullsized":"2024-09-17T05:28:03.386847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387350+0000","last_change":"2024-09-17T05:28:03.387350+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387350+0000","last_undegraded":"2024-09-17T05:28:03.387350+0000","last_fullsized":"2024-09-17T05:28:03.387350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394182+0000","last_change":"2024-09-17T05:28:03.394182+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394182+0000","last_undegraded":"2024-09-17T05:28:03.394182+0000","last_fullsized":"2024-09-17T05:28:03.394182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409026+0000","last_change":"2024-09-17T05:28:04.414079+0000","last_active":"2024-09-17T05:28:05.409026+0000","last_peered":"2024-09-17T05:28:05.409026+0000","last_clean":"2024-09-17T05:28:05.409026+0000","last_became_active":"2024-09-17T05:28:04.413733+0000","last_became_peered":"2024-09-17T05:28:04.413733+0000","last_unstale":"2024-09-17T05:28:05.409026+0000","last_undegraded":"2024-09-17T05:28:05.409026+0000","last_fullsized":"2024-09-17T05:28:05.409026+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:15.069773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389389+0000","last_change":"2024-09-17T05:28:03.389389+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389389+0000","last_undegraded":"2024-09-17T05:28:03.389389+0000","last_fullsized":"2024-09-17T05:28:03.389389+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411758+0000","last_change":"2024-09-17T05:28:03.408603+0000","last_active":"2024-09-17T05:28:03.411758+0000","last_peered":"2024-09-17T05:28:03.411758+0000","last_clean":"2024-09-17T05:28:03.411758+0000","last_became_active":"2024-09-17T05:28:03.408354+0000","last_became_peered":"2024-09-17T05:28:03.408354+0000","last_unstale":"2024-09-17T05:28:03.411758+0000","last_undegraded":"2024-09-17T05:28:03.411758+0000","last_fullsized":"2024-09-17T05:28:03.411758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:20.506564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386337+0000","last_change":"2024-09-17T05:28:03.386337+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386337+0000","last_undegraded":"2024-09-17T05:28:03.386337+0000","last_fullsized":"2024-09-17T05:28:03.386337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408180+0000","last_change":"2024-09-17T05:28:03.403643+0000","last_active":"2024-09-17T05:28:03.408180+0000","last_peered":"2024-09-17T05:28:03.408180+0000","last_clean":"2024-09-17T05:28:03.408180+0000","last_became_active":"2024-09-17T05:28:03.403353+0000","last_became_peered":"2024-09-17T05:28:03.403353+0000","last_unstale":"2024-09-17T05:28:03.408180+0000","last_undegraded":"2024-09-17T05:28:03.408180+0000","last_fullsized":"2024-09-17T05:28:03.408180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:30.436401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387803+0000","last_change":"2024-09-17T05:28:03.387803+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387803+0000","last_undegraded":"2024-09-17T05:28:03.387803+0000","last_fullsized":"2024-09-17T05:28:03.387803+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411701+0000","last_change":"2024-09-17T05:28:03.407955+0000","last_active":"2024-09-17T05:28:03.411701+0000","last_peered":"2024-09-17T05:28:03.411701+0000","last_clean":"2024-09-17T05:28:03.411701+0000","last_became_active":"2024-09-17T05:28:03.407597+0000","last_became_peered":"2024-09-17T05:28:03.407597+0000","last_unstale":"2024-09-17T05:28:03.411701+0000","last_undegraded":"2024-09-17T05:28:03.411701+0000","last_fullsized":"2024-09-17T05:28:03.411701+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:31.638210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410107+0000","last_change":"2024-09-17T05:28:04.411454+0000","last_active":"2024-09-17T05:28:05.410107+0000","last_peered":"2024-09-17T05:28:05.410107+0000","last_clean":"2024-09-17T05:28:05.410107+0000","last_became_active":"2024-09-17T05:28:04.411144+0000","last_became_peered":"2024-09-17T05:28:04.411144+0000","last_unstale":"2024-09-17T05:28:05.410107+0000","last_undegraded":"2024-09-17T05:28:05.410107+0000","last_fullsized":"2024-09-17T05:28:05.410107+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:10.404469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410169+0000","last_change":"2024-09-17T05:28:03.406275+0000","last_active":"2024-09-17T05:28:03.410169+0000","last_peered":"2024-09-17T05:28:03.410169+0000","last_clean":"2024-09-17T05:28:03.410169+0000","last_became_active":"2024-09-17T05:28:03.405506+0000","last_became_peered":"2024-09-17T05:28:03.405506+0000","last_unstale":"2024-09-17T05:28:03.410169+0000","last_undegraded":"2024-09-17T05:28:03.410169+0000","last_fullsized":"2024-09-17T05:28:03.410169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:46.627002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395704+0000","last_change":"2024-09-17T05:28:03.395704+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395704+0000","last_undegraded":"2024-09-17T05:28:03.395704+0000","last_fullsized":"2024-09-17T05:28:03.395704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410230+0000","last_change":"2024-09-17T05:28:03.406212+0000","last_active":"2024-09-17T05:28:03.410230+0000","last_peered":"2024-09-17T05:28:03.410230+0000","last_clean":"2024-09-17T05:28:03.410230+0000","last_became_active":"2024-09-17T05:28:03.405428+0000","last_became_peered":"2024-09-17T05:28:03.405428+0000","last_unstale":"2024-09-17T05:28:03.410230+0000","last_undegraded":"2024-09-17T05:28:03.410230+0000","last_fullsized":"2024-09-17T05:28:03.410230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:08.586828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394459+0000","last_change":"2024-09-17T05:28:03.394459+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394459+0000","last_undegraded":"2024-09-17T05:28:03.394459+0000","last_fullsized":"2024-09-17T05:28:03.394459+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410102+0000","last_change":"2024-09-17T05:28:03.405006+0000","last_active":"2024-09-17T05:28:03.410102+0000","last_peered":"2024-09-17T05:28:03.410102+0000","last_clean":"2024-09-17T05:28:03.410102+0000","last_became_active":"2024-09-17T05:28:03.404741+0000","last_became_peered":"2024-09-17T05:28:03.404741+0000","last_unstale":"2024-09-17T05:28:03.410102+0000","last_undegraded":"2024-09-17T05:28:03.410102+0000","last_fullsized":"2024-09-17T05:28:03.410102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:23.814938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393871+0000","last_change":"2024-09-17T05:28:03.393871+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393871+0000","last_undegraded":"2024-09-17T05:28:03.393871+0000","last_fullsized":"2024-09-17T05:28:03.393871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.399582+0000","last_change":"2024-09-17T05:28:03.397856+0000","last_active":"2024-09-17T05:28:03.399582+0000","last_peered":"2024-09-17T05:28:03.399582+0000","last_clean":"2024-09-17T05:28:03.399582+0000","last_became_active":"2024-09-17T05:28:03.397609+0000","last_became_peered":"2024-09-17T05:28:03.397609+0000","last_unstale":"2024-09-17T05:28:03.399582+0000","last_undegraded":"2024-09-17T05:28:03.399582+0000","last_fullsized":"2024-09-17T05:28:03.399582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:40.677984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393780+0000","last_change":"2024-09-17T05:28:03.393780+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393780+0000","last_undegraded":"2024-09-17T05:28:03.393780+0000","last_fullsized":"2024-09-17T05:28:03.393780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.414117+0000","last_change":"2024-09-17T05:28:03.410366+0000","last_active":"2024-09-17T05:28:03.414117+0000","last_peered":"2024-09-17T05:28:03.414117+0000","last_clean":"2024-09-17T05:28:03.414117+0000","last_became_active":"2024-09-17T05:28:03.409222+0000","last_became_peered":"2024-09-17T05:28:03.409222+0000","last_unstale":"2024-09-17T05:28:03.414117+0000","last_undegraded":"2024-09-17T05:28:03.414117+0000","last_fullsized":"2024-09-17T05:28:03.414117+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:11.005389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391108+0000","last_change":"2024-09-17T05:28:03.391108+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391108+0000","last_undegraded":"2024-09-17T05:28:03.391108+0000","last_fullsized":"2024-09-17T05:28:03.391108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.403575+0000","last_change":"2024-09-17T05:28:03.400471+0000","last_active":"2024-09-17T05:28:03.403575+0000","last_peered":"2024-09-17T05:28:03.403575+0000","last_clean":"2024-09-17T05:28:03.403575+0000","last_became_active":"2024-09-17T05:28:03.400202+0000","last_became_peered":"2024-09-17T05:28:03.400202+0000","last_unstale":"2024-09-17T05:28:03.403575+0000","last_undegraded":"2024-09-17T05:28:03.403575+0000","last_fullsized":"2024-09-17T05:28:03.403575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:58.648337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.390844+0000","last_change":"2024-09-17T05:28:03.390844+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.390844+0000","last_undegraded":"2024-09-17T05:28:03.390844+0000","last_fullsized":"2024-09-17T05:28:03.390844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.409027+0000","last_change":"2024-09-17T05:28:03.401227+0000","last_active":"2024-09-17T05:28:03.409027+0000","last_peered":"2024-09-17T05:28:03.409027+0000","last_clean":"2024-09-17T05:28:03.409027+0000","last_became_active":"2024-09-17T05:28:03.400688+0000","last_became_peered":"2024-09-17T05:28:03.400688+0000","last_unstale":"2024-09-17T05:28:03.409027+0000","last_undegraded":"2024-09-17T05:28:03.409027+0000","last_fullsized":"2024-09-17T05:28:03.409027+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:45.856441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409693+0000","last_change":"2024-09-17T05:28:04.413706+0000","last_active":"2024-09-17T05:28:05.409693+0000","last_peered":"2024-09-17T05:28:05.409693+0000","last_clean":"2024-09-17T05:28:05.409693+0000","last_became_active":"2024-09-17T05:28:04.413217+0000","last_became_peered":"2024-09-17T05:28:04.413217+0000","last_unstale":"2024-09-17T05:28:05.409693+0000","last_undegraded":"2024-09-17T05:28:05.409693+0000","last_fullsized":"2024-09-17T05:28:05.409693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:33.217682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407650+0000","last_change":"2024-09-17T05:28:03.401672+0000","last_active":"2024-09-17T05:28:03.407650+0000","last_peered":"2024-09-17T05:28:03.407650+0000","last_clean":"2024-09-17T05:28:03.407650+0000","last_became_active":"2024-09-17T05:28:03.401380+0000","last_became_peered":"2024-09-17T05:28:03.401380+0000","last_unstale":"2024-09-17T05:28:03.407650+0000","last_undegraded":"2024-09-17T05:28:03.407650+0000","last_fullsized":"2024-09-17T05:28:03.407650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:57.580288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394819+0000","last_change":"2024-09-17T05:28:03.394819+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394819+0000","last_undegraded":"2024-09-17T05:28:03.394819+0000","last_fullsized":"2024-09-17T05:28:03.394819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411516+0000","last_change":"2024-09-17T05:28:03.408491+0000","last_active":"2024-09-17T05:28:03.411516+0000","last_peered":"2024-09-17T05:28:03.411516+0000","last_clean":"2024-09-17T05:28:03.411516+0000","last_became_active":"2024-09-17T05:28:03.407793+0000","last_became_peered":"2024-09-17T05:28:03.407793+0000","last_unstale":"2024-09-17T05:28:03.411516+0000","last_undegraded":"2024-09-17T05:28:03.411516+0000","last_fullsized":"2024-09-17T05:28:03.411516+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.426226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389843+0000","last_change":"2024-09-17T05:28:03.389843+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389843+0000","last_undegraded":"2024-09-17T05:28:03.389843+0000","last_fullsized":"2024-09-17T05:28:03.389843+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411542+0000","last_change":"2024-09-17T05:28:03.406639+0000","last_active":"2024-09-17T05:28:03.411542+0000","last_peered":"2024-09-17T05:28:03.411542+0000","last_clean":"2024-09-17T05:28:03.411542+0000","last_became_active":"2024-09-17T05:28:03.406360+0000","last_became_peered":"2024-09-17T05:28:03.406360+0000","last_unstale":"2024-09-17T05:28:03.411542+0000","last_undegraded":"2024-09-17T05:28:03.411542+0000","last_fullsized":"2024-09-17T05:28:03.411542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:17.531567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.404093+0000","last_change":"2024-09-17T05:28:03.401363+0000","last_active":"2024-09-17T05:28:03.404093+0000","last_peered":"2024-09-17T05:28:03.404093+0000","last_clean":"2024-09-17T05:28:03.404093+0000","last_became_active":"2024-09-17T05:28:03.401110+0000","last_became_peered":"2024-09-17T05:28:03.401110+0000","last_unstale":"2024-09-17T05:28:03.404093+0000","last_undegraded":"2024-09-17T05:28:03.404093+0000","last_fullsized":"2024-09-17T05:28:03.404093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:24.220817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.413897+0000","last_change":"2024-09-17T05:28:03.409353+0000","last_active":"2024-09-17T05:28:03.413897+0000","last_peered":"2024-09-17T05:28:03.413897+0000","last_clean":"2024-09-17T05:28:03.413897+0000","last_became_active":"2024-09-17T05:28:03.409087+0000","last_became_peered":"2024-09-17T05:28:03.409087+0000","last_unstale":"2024-09-17T05:28:03.413897+0000","last_undegraded":"2024-09-17T05:28:03.413897+0000","last_fullsized":"2024-09-17T05:28:03.413897+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:37.361308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388677+0000","last_change":"2024-09-17T05:28:03.388677+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388677+0000","last_undegraded":"2024-09-17T05:28:03.388677+0000","last_fullsized":"2024-09-17T05:28:03.388677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.409986+0000","last_change":"2024-09-17T05:28:03.403774+0000","last_active":"2024-09-17T05:28:03.409986+0000","last_peered":"2024-09-17T05:28:03.409986+0000","last_clean":"2024-09-17T05:28:03.409986+0000","last_became_active":"2024-09-17T05:28:03.403425+0000","last_became_peered":"2024-09-17T05:28:03.403425+0000","last_unstale":"2024-09-17T05:28:03.409986+0000","last_undegraded":"2024-09-17T05:28:03.409986+0000","last_fullsized":"2024-09-17T05:28:03.409986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:08.155865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388720+0000","last_change":"2024-09-17T05:28:03.388720+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388720+0000","last_undegraded":"2024-09-17T05:28:03.388720+0000","last_fullsized":"2024-09-17T05:28:03.388720+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410531+0000","last_change":"2024-09-17T05:28:03.409845+0000","last_active":"2024-09-17T05:28:05.410531+0000","last_peered":"2024-09-17T05:28:05.410531+0000","last_clean":"2024-09-17T05:28:05.410531+0000","last_became_active":"2024-09-17T05:28:03.409519+0000","last_became_peered":"2024-09-17T05:28:03.409519+0000","last_unstale":"2024-09-17T05:28:05.410531+0000","last_undegraded":"2024-09-17T05:28:05.410531+0000","last_fullsized":"2024-09-17T05:28:05.410531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:04.018422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412689+0000","last_change":"2024-09-17T05:28:03.408640+0000","last_active":"2024-09-17T05:28:03.412689+0000","last_peered":"2024-09-17T05:28:03.412689+0000","last_clean":"2024-09-17T05:28:03.412689+0000","last_became_active":"2024-09-17T05:28:03.408371+0000","last_became_peered":"2024-09-17T05:28:03.408371+0000","last_unstale":"2024-09-17T05:28:03.412689+0000","last_undegraded":"2024-09-17T05:28:03.412689+0000","last_fullsized":"2024-09-17T05:28:03.412689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:51.246256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387320+0000","last_change":"2024-09-17T05:28:03.387320+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387320+0000","last_undegraded":"2024-09-17T05:28:03.387320+0000","last_fullsized":"2024-09-17T05:28:03.387320+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.403739+0000","last_change":"2024-09-17T05:28:03.398552+0000","last_active":"2024-09-17T05:28:03.403739+0000","last_peered":"2024-09-17T05:28:03.403739+0000","last_clean":"2024-09-17T05:28:03.403739+0000","last_became_active":"2024-09-17T05:28:03.398282+0000","last_became_peered":"2024-09-17T05:28:03.398282+0000","last_unstale":"2024-09-17T05:28:03.403739+0000","last_undegraded":"2024-09-17T05:28:03.403739+0000","last_fullsized":"2024-09-17T05:28:03.403739+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:54.593079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.390768+0000","last_change":"2024-09-17T05:28:03.390768+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.390768+0000","last_undegraded":"2024-09-17T05:28:03.390768+0000","last_fullsized":"2024-09-17T05:28:03.390768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410212+0000","last_change":"2024-09-17T05:28:03.404997+0000","last_active":"2024-09-17T05:28:03.410212+0000","last_peered":"2024-09-17T05:28:03.410212+0000","last_clean":"2024-09-17T05:28:03.410212+0000","last_became_active":"2024-09-17T05:28:03.404512+0000","last_became_peered":"2024-09-17T05:28:03.404512+0000","last_unstale":"2024-09-17T05:28:03.410212+0000","last_undegraded":"2024-09-17T05:28:03.410212+0000","last_fullsized":"2024-09-17T05:28:03.410212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:07.584572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398333+0000","last_change":"2024-09-17T05:28:03.398333+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398333+0000","last_undegraded":"2024-09-17T05:28:03.398333+0000","last_fullsized":"2024-09-17T05:28:03.398333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408284+0000","last_change":"2024-09-17T05:28:03.400873+0000","last_active":"2024-09-17T05:28:03.408284+0000","last_peered":"2024-09-17T05:28:03.408284+0000","last_clean":"2024-09-17T05:28:03.408284+0000","last_became_active":"2024-09-17T05:28:03.400549+0000","last_became_peered":"2024-09-17T05:28:03.400549+0000","last_unstale":"2024-09-17T05:28:03.408284+0000","last_undegraded":"2024-09-17T05:28:03.408284+0000","last_fullsized":"2024-09-17T05:28:03.408284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:31.777509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394647+0000","last_change":"2024-09-17T05:28:03.394647+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394647+0000","last_undegraded":"2024-09-17T05:28:03.394647+0000","last_fullsized":"2024-09-17T05:28:03.394647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.398760+0000","last_change":"2024-09-17T05:28:03.395798+0000","last_active":"2024-09-17T05:28:03.398760+0000","last_peered":"2024-09-17T05:28:03.398760+0000","last_clean":"2024-09-17T05:28:03.398760+0000","last_became_active":"2024-09-17T05:28:03.395543+0000","last_became_peered":"2024-09-17T05:28:03.395543+0000","last_unstale":"2024-09-17T05:28:03.398760+0000","last_undegraded":"2024-09-17T05:28:03.398760+0000","last_fullsized":"2024-09-17T05:28:03.398760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:37.361308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409491+0000","last_change":"2024-09-17T05:28:04.423847+0000","last_active":"2024-09-17T05:28:05.409491+0000","last_peered":"2024-09-17T05:28:05.409491+0000","last_clean":"2024-09-17T05:28:05.409491+0000","last_became_active":"2024-09-17T05:28:04.423569+0000","last_became_peered":"2024-09-17T05:28:04.423569+0000","last_unstale":"2024-09-17T05:28:05.409491+0000","last_undegraded":"2024-09-17T05:28:05.409491+0000","last_fullsized":"2024-09-17T05:28:05.409491+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:58.595615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408407+0000","last_change":"2024-09-17T05:28:03.400992+0000","last_active":"2024-09-17T05:28:03.408407+0000","last_peered":"2024-09-17T05:28:03.408407+0000","last_clean":"2024-09-17T05:28:03.408407+0000","last_became_active":"2024-09-17T05:28:03.400690+0000","last_became_peered":"2024-09-17T05:28:03.400690+0000","last_unstale":"2024-09-17T05:28:03.408407+0000","last_undegraded":"2024-09-17T05:28:03.408407+0000","last_fullsized":"2024-09-17T05:28:03.408407+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:49.611905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409358+0000","last_change":"2024-09-17T05:28:04.426697+0000","last_active":"2024-09-17T05:28:05.409358+0000","last_peered":"2024-09-17T05:28:05.409358+0000","last_clean":"2024-09-17T05:28:05.409358+0000","last_became_active":"2024-09-17T05:28:04.426419+0000","last_became_peered":"2024-09-17T05:28:04.426419+0000","last_unstale":"2024-09-17T05:28:05.409358+0000","last_undegraded":"2024-09-17T05:28:05.409358+0000","last_fullsized":"2024-09-17T05:28:05.409358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.432704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412599+0000","last_change":"2024-09-17T05:28:03.407318+0000","last_active":"2024-09-17T05:28:03.412599+0000","last_peered":"2024-09-17T05:28:03.412599+0000","last_clean":"2024-09-17T05:28:03.412599+0000","last_became_active":"2024-09-17T05:28:03.407059+0000","last_became_peered":"2024-09-17T05:28:03.407059+0000","last_unstale":"2024-09-17T05:28:03.412599+0000","last_undegraded":"2024-09-17T05:28:03.412599+0000","last_fullsized":"2024-09-17T05:28:03.412599+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:15.545411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393056+0000","last_change":"2024-09-17T05:28:03.393056+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393056+0000","last_undegraded":"2024-09-17T05:28:03.393056+0000","last_fullsized":"2024-09-17T05:28:03.393056+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410708+0000","last_change":"2024-09-17T05:28:03.408701+0000","last_active":"2024-09-17T05:28:05.410708+0000","last_peered":"2024-09-17T05:28:05.410708+0000","last_clean":"2024-09-17T05:28:05.410708+0000","last_became_active":"2024-09-17T05:28:03.408379+0000","last_became_peered":"2024-09-17T05:28:03.408379+0000","last_unstale":"2024-09-17T05:28:05.410708+0000","last_undegraded":"2024-09-17T05:28:05.410708+0000","last_fullsized":"2024-09-17T05:28:05.410708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:24.330713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394936+0000","last_change":"2024-09-17T05:28:03.394936+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394936+0000","last_undegraded":"2024-09-17T05:28:03.394936+0000","last_fullsized":"2024-09-17T05:28:03.394936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.402356+0000","last_change":"2024-09-17T05:28:03.397060+0000","last_active":"2024-09-17T05:28:03.402356+0000","last_peered":"2024-09-17T05:28:03.402356+0000","last_clean":"2024-09-17T05:28:03.402356+0000","last_became_active":"2024-09-17T05:28:03.396798+0000","last_became_peered":"2024-09-17T05:28:03.396798+0000","last_unstale":"2024-09-17T05:28:03.402356+0000","last_undegraded":"2024-09-17T05:28:03.402356+0000","last_fullsized":"2024-09-17T05:28:03.402356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.747610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.392714+0000","last_change":"2024-09-17T05:28:03.392714+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.392714+0000","last_undegraded":"2024-09-17T05:28:03.392714+0000","last_fullsized":"2024-09-17T05:28:03.392714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407241+0000","last_change":"2024-09-17T05:28:03.402041+0000","last_active":"2024-09-17T05:28:03.407241+0000","last_peered":"2024-09-17T05:28:03.407241+0000","last_clean":"2024-09-17T05:28:03.407241+0000","last_became_active":"2024-09-17T05:28:03.401757+0000","last_became_peered":"2024-09-17T05:28:03.401757+0000","last_unstale":"2024-09-17T05:28:03.407241+0000","last_undegraded":"2024-09-17T05:28:03.407241+0000","last_fullsized":"2024-09-17T05:28:03.407241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:39.617757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.397895+0000","last_change":"2024-09-17T05:28:03.397895+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.397895+0000","last_undegraded":"2024-09-17T05:28:03.397895+0000","last_fullsized":"2024-09-17T05:28:03.397895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407808+0000","last_change":"2024-09-17T05:28:03.406099+0000","last_active":"2024-09-17T05:28:03.407808+0000","last_peered":"2024-09-17T05:28:03.407808+0000","last_clean":"2024-09-17T05:28:03.407808+0000","last_became_active":"2024-09-17T05:28:03.405763+0000","last_became_peered":"2024-09-17T05:28:03.405763+0000","last_unstale":"2024-09-17T05:28:03.407808+0000","last_undegraded":"2024-09-17T05:28:03.407808+0000","last_fullsized":"2024-09-17T05:28:03.407808+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:57.580288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409499+0000","last_change":"2024-09-17T05:28:04.405966+0000","last_active":"2024-09-17T05:28:05.409499+0000","last_peered":"2024-09-17T05:28:05.409499+0000","last_clean":"2024-09-17T05:28:05.409499+0000","last_became_active":"2024-09-17T05:28:04.405723+0000","last_became_peered":"2024-09-17T05:28:04.405723+0000","last_unstale":"2024-09-17T05:28:05.409499+0000","last_undegraded":"2024-09-17T05:28:05.409499+0000","last_fullsized":"2024-09-17T05:28:05.409499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:09:42.528631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.405322+0000","last_change":"2024-09-17T05:28:03.403069+0000","last_active":"2024-09-17T05:28:03.405322+0000","last_peered":"2024-09-17T05:28:03.405322+0000","last_clean":"2024-09-17T05:28:03.405322+0000","last_became_active":"2024-09-17T05:28:03.402652+0000","last_became_peered":"2024-09-17T05:28:03.402652+0000","last_unstale":"2024-09-17T05:28:03.405322+0000","last_undegraded":"2024-09-17T05:28:03.405322+0000","last_fullsized":"2024-09-17T05:28:03.405322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:42.875418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389143+0000","last_change":"2024-09-17T05:28:03.389143+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389143+0000","last_undegraded":"2024-09-17T05:28:03.389143+0000","last_fullsized":"2024-09-17T05:28:03.389143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411465+0000","last_change":"2024-09-17T05:28:03.407068+0000","last_active":"2024-09-17T05:28:03.411465+0000","last_peered":"2024-09-17T05:28:03.411465+0000","last_clean":"2024-09-17T05:28:03.411465+0000","last_became_active":"2024-09-17T05:28:03.406674+0000","last_became_peered":"2024-09-17T05:28:03.406674+0000","last_unstale":"2024-09-17T05:28:03.411465+0000","last_undegraded":"2024-09-17T05:28:03.411465+0000","last_fullsized":"2024-09-17T05:28:03.411465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:54.447845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387799+0000","last_change":"2024-09-17T05:28:03.387799+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387799+0000","last_undegraded":"2024-09-17T05:28:03.387799+0000","last_fullsized":"2024-09-17T05:28:03.387799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407397+0000","last_change":"2024-09-17T05:28:03.402396+0000","last_active":"2024-09-17T05:28:03.407397+0000","last_peered":"2024-09-17T05:28:03.407397+0000","last_clean":"2024-09-17T05:28:03.407397+0000","last_became_active":"2024-09-17T05:28:03.402089+0000","last_became_peered":"2024-09-17T05:28:03.402089+0000","last_unstale":"2024-09-17T05:28:03.407397+0000","last_undegraded":"2024-09-17T05:28:03.407397+0000","last_fullsized":"2024-09-17T05:28:03.407397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:31.994460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410638+0000","last_change":"2024-09-17T05:28:04.407978+0000","last_active":"2024-09-17T05:28:05.410638+0000","last_peered":"2024-09-17T05:28:05.410638+0000","last_clean":"2024-09-17T05:28:05.410638+0000","last_became_active":"2024-09-17T05:28:04.407717+0000","last_became_peered":"2024-09-17T05:28:04.407717+0000","last_unstale":"2024-09-17T05:28:05.410638+0000","last_undegraded":"2024-09-17T05:28:05.410638+0000","last_fullsized":"2024-09-17T05:28:05.410638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:52.263630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412718+0000","last_change":"2024-09-17T05:28:03.408678+0000","last_active":"2024-09-17T05:28:03.412718+0000","last_peered":"2024-09-17T05:28:03.412718+0000","last_clean":"2024-09-17T05:28:03.412718+0000","last_became_active":"2024-09-17T05:28:03.408375+0000","last_became_peered":"2024-09-17T05:28:03.408375+0000","last_unstale":"2024-09-17T05:28:03.412718+0000","last_undegraded":"2024-09-17T05:28:03.412718+0000","last_fullsized":"2024-09-17T05:28:03.412718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:10.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":153,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.393074+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:28:03.393074+0000","last_peered":"2024-09-17T05:28:03.393074+0000","last_clean":"2024-09-17T05:28:03.393074+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:28:03.393074+0000","last_undegraded":"2024-09-17T05:28:03.393074+0000","last_fullsized":"2024-09-17T05:28:03.393074+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":792467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1130496,"data_stored":805564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:08.902 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:09.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- 172.21.15.122:0/3423887422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 msgr2=0x7f5ec015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/3423887422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 0x7f5ec015f910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0076610 tx=0x7f5eb801f580 comp rx=0 tx=0).stop 2024-09-17T05:28:09.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- 172.21.15.122:0/3423887422 shutdown_connections 2024-09-17T05:28:09.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/3423887422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 0x7f5ec015f910 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/3423887422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ec015bfa0 0x7f5ec00cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/3423887422 >> v2:172.21.15.173:3300/0 conn(0x7f5ec015c3e0 0x7f5ec015c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- 172.21.15.122:0/3423887422 >> 172.21.15.122:0/3423887422 conn(0x7f5ec00b18f0 msgr2=0x7f5ec00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:09.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- 172.21.15.122:0/3423887422 shutdown_connections 2024-09-17T05:28:09.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- 172.21.15.122:0/3423887422 wait complete. 2024-09-17T05:28:09.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 Processor -- start 2024-09-17T05:28:09.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- start start 2024-09-17T05:28:09.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 0x7f5ec01f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ec015bfa0 0x7f5ec00acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ec015c3e0 0x7f5ec014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec01601e0 con 0x7f5ec00cd540 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec0160060 con 0x7f5ec015c3e0 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec7c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec0160360 con 0x7f5ec015bfa0 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 0x7f5ec01f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 0x7f5ec01f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51860/0 (socket says 172.21.15.122:51860) 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5200640 1 -- 172.21.15.122:0/2847030452 learned_addr learned my addr 172.21.15.122:0/2847030452 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ebfe00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ec015bfa0 0x7f5ec00acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5c00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ec015c3e0 0x7f5ec014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5200640 1 -- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ec015bfa0 msgr2=0x7f5ec00acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5200640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ec015bfa0 0x7f5ec00acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5200640 1 -- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ec015c3e0 msgr2=0x7f5ec014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5200640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ec015c3e0 0x7f5ec014d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5200640 1 -- 172.21.15.122:0/2847030452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec00b47d0 con 0x7f5ec00cd540 2024-09-17T05:28:09.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ec5c00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ec015c3e0 0x7f5ec014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:09.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.087+0000 7f5ebfe00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ec015bfa0 0x7f5ec00acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:09.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ec5200640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 0x7f5ec01f8af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400c9f0 tx=0x7f5eb400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:09.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ebcc00640 1 -- 172.21.15.122:0/2847030452 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4007bb0 con 0x7f5ec00cd540 2024-09-17T05:28:09.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ebcc00640 1 -- 172.21.15.122:0/2847030452 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5eb4007e40 con 0x7f5ec00cd540 2024-09-17T05:28:09.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ebcc00640 1 -- 172.21.15.122:0/2847030452 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb400f690 con 0x7f5ec00cd540 2024-09-17T05:28:09.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec01f8330 con 0x7f5ec00cd540 2024-09-17T05:28:09.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec01f9030 con 0x7f5ec00cd540 2024-09-17T05:28:09.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec00cd000 con 0x7f5ec00cd540 2024-09-17T05:28:09.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ebcc00640 1 -- 172.21.15.122:0/2847030452 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f5eb400f830 con 0x7f5ec00cd540 2024-09-17T05:28:09.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ebcc00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ea0073560 0x7f5ea0075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.091+0000 7f5ebcc00640 1 -- 172.21.15.122:0/2847030452 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f5eb4095060 con 0x7f5ec00cd540 2024-09-17T05:28:09.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.095+0000 7f5ebfe00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ea0073560 0x7f5ea0075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.095+0000 7f5ebfe00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ea0073560 0x7f5ea0075a20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5eac005fd0 tx=0x7f5eac005ee0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:09.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.095+0000 7f5ebcc00640 1 -- 172.21.15.122:0/2847030452 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5eb4014030 con 0x7f5ec00cd540 2024-09-17T05:28:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.263+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ec00cbf00 con 0x7f5ea0073560 2024-09-17T05:28:09.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.267+0000 7f5ebcc00640 1 -- 172.21.15.122:0/2847030452 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247807 (secure 0 0 0) 0x7f5ec00cbf00 con 0x7f5ea0073560 2024-09-17T05:28:09.271 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:28:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ea0073560 msgr2=0x7f5ea0075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ea0073560 0x7f5ea0075a20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5eac005fd0 tx=0x7f5eac005ee0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 msgr2=0x7f5ec01f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 0x7f5ec01f8af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400c9f0 tx=0x7f5eb400ceb0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 shutdown_connections 2024-09-17T05:28:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ea0073560 0x7f5ea0075a20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ec015c3e0 0x7f5ec014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ec015bfa0 0x7f5ec00acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 --2- 172.21.15.122:0/2847030452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ec00cd540 0x7f5ec01f8af0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 >> 172.21.15.122:0/2847030452 conn(0x7f5ec00b18f0 msgr2=0x7f5ec0155180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 shutdown_connections 2024-09-17T05:28:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.271+0000 7f5ec7c00640 1 -- 172.21.15.122:0/2847030452 wait complete. 2024-09-17T05:28:09.290 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-17T05:28:08.492320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262972,"kb_used_data":5120,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711748,"statfs":{"total":773094113280,"available":772824829952,"internally_reserved":0,"allocated":5242880,"data_stored":2791800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":263835190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003592"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387945+0000","last_change":"2024-09-17T05:28:03.387945+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387945+0000","last_undegraded":"2024-09-17T05:28:03.387945+0000","last_fullsized":"2024-09-17T05:28:03.387945+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388044+0000","last_change":"2024-09-17T05:28:03.388044+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388044+0000","last_undegraded":"2024-09-17T05:28:03.388044+0000","last_fullsized":"2024-09-17T05:28:03.388044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391128+0000","last_change":"2024-09-17T05:28:03.391128+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391128+0000","last_undegraded":"2024-09-17T05:28:03.391128+0000","last_fullsized":"2024-09-17T05:28:03.391128+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409677+0000","last_change":"2024-09-17T05:28:04.416129+0000","last_active":"2024-09-17T05:28:05.409677+0000","last_peered":"2024-09-17T05:28:05.409677+0000","last_clean":"2024-09-17T05:28:05.409677+0000","last_became_active":"2024-09-17T05:28:04.415869+0000","last_became_peered":"2024-09-17T05:28:04.415869+0000","last_unstale":"2024-09-17T05:28:05.409677+0000","last_undegraded":"2024-09-17T05:28:05.409677+0000","last_fullsized":"2024-09-17T05:28:05.409677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.172592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387704+0000","last_change":"2024-09-17T05:28:03.387704+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387704+0000","last_undegraded":"2024-09-17T05:28:03.387704+0000","last_fullsized":"2024-09-17T05:28:03.387704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.392926+0000","last_change":"2024-09-17T05:28:03.392926+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.392926+0000","last_undegraded":"2024-09-17T05:28:03.392926+0000","last_fullsized":"2024-09-17T05:28:03.392926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398593+0000","last_change":"2024-09-17T05:28:03.398593+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398593+0000","last_undegraded":"2024-09-17T05:28:03.398593+0000","last_fullsized":"2024-09-17T05:28:03.398593+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.408875+0000","last_change":"2024-09-17T05:28:04.421456+0000","last_active":"2024-09-17T05:28:05.408875+0000","last_peered":"2024-09-17T05:28:05.408875+0000","last_clean":"2024-09-17T05:28:05.408875+0000","last_became_active":"2024-09-17T05:28:04.421034+0000","last_became_peered":"2024-09-17T05:28:04.421034+0000","last_unstale":"2024-09-17T05:28:05.408875+0000","last_undegraded":"2024-09-17T05:28:05.408875+0000","last_fullsized":"2024-09-17T05:28:05.408875+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:41.119416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398634+0000","last_change":"2024-09-17T05:28:03.398634+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398634+0000","last_undegraded":"2024-09-17T05:28:03.398634+0000","last_fullsized":"2024-09-17T05:28:03.398634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393796+0000","last_change":"2024-09-17T05:28:03.393796+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393796+0000","last_undegraded":"2024-09-17T05:28:03.393796+0000","last_fullsized":"2024-09-17T05:28:03.393796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409905+0000","last_change":"2024-09-17T05:28:04.419275+0000","last_active":"2024-09-17T05:28:05.409905+0000","last_peered":"2024-09-17T05:28:05.409905+0000","last_clean":"2024-09-17T05:28:05.409905+0000","last_became_active":"2024-09-17T05:28:04.418991+0000","last_became_peered":"2024-09-17T05:28:04.418991+0000","last_unstale":"2024-09-17T05:28:05.409905+0000","last_undegraded":"2024-09-17T05:28:05.409905+0000","last_fullsized":"2024-09-17T05:28:05.409905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:23.031211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389118+0000","last_change":"2024-09-17T05:28:03.389118+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389118+0000","last_undegraded":"2024-09-17T05:28:03.389118+0000","last_fullsized":"2024-09-17T05:28:03.389118+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395047+0000","last_change":"2024-09-17T05:28:03.395047+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395047+0000","last_undegraded":"2024-09-17T05:28:03.395047+0000","last_fullsized":"2024-09-17T05:28:03.395047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409281+0000","last_change":"2024-09-17T05:28:04.411688+0000","last_active":"2024-09-17T05:28:05.409281+0000","last_peered":"2024-09-17T05:28:05.409281+0000","last_clean":"2024-09-17T05:28:05.409281+0000","last_became_active":"2024-09-17T05:28:04.411403+0000","last_became_peered":"2024-09-17T05:28:04.411403+0000","last_unstale":"2024-09-17T05:28:05.409281+0000","last_undegraded":"2024-09-17T05:28:05.409281+0000","last_fullsized":"2024-09-17T05:28:05.409281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:17.638839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409054+0000","last_change":"2024-09-17T05:28:04.423517+0000","last_active":"2024-09-17T05:28:05.409054+0000","last_peered":"2024-09-17T05:28:05.409054+0000","last_clean":"2024-09-17T05:28:05.409054+0000","last_became_active":"2024-09-17T05:28:04.423243+0000","last_became_peered":"2024-09-17T05:28:04.423243+0000","last_unstale":"2024-09-17T05:28:05.409054+0000","last_undegraded":"2024-09-17T05:28:05.409054+0000","last_fullsized":"2024-09-17T05:28:05.409054+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:24.878942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386968+0000","last_change":"2024-09-17T05:28:03.386968+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386968+0000","last_undegraded":"2024-09-17T05:28:03.386968+0000","last_fullsized":"2024-09-17T05:28:03.386968+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387639+0000","last_change":"2024-09-17T05:28:03.387639+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387639+0000","last_undegraded":"2024-09-17T05:28:03.387639+0000","last_fullsized":"2024-09-17T05:28:03.387639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391815+0000","last_change":"2024-09-17T05:28:03.391815+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391815+0000","last_undegraded":"2024-09-17T05:28:03.391815+0000","last_fullsized":"2024-09-17T05:28:03.391815+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387562+0000","last_change":"2024-09-17T05:28:03.387562+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387562+0000","last_undegraded":"2024-09-17T05:28:03.387562+0000","last_fullsized":"2024-09-17T05:28:03.387562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395327+0000","last_change":"2024-09-17T05:28:03.395327+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395327+0000","last_undegraded":"2024-09-17T05:28:03.395327+0000","last_fullsized":"2024-09-17T05:28:03.395327+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409823+0000","last_change":"2024-09-17T05:28:04.421922+0000","last_active":"2024-09-17T05:28:05.409823+0000","last_peered":"2024-09-17T05:28:05.409823+0000","last_clean":"2024-09-17T05:28:05.409823+0000","last_became_active":"2024-09-17T05:28:04.414061+0000","last_became_peered":"2024-09-17T05:28:04.414061+0000","last_unstale":"2024-09-17T05:28:05.409823+0000","last_undegraded":"2024-09-17T05:28:05.409823+0000","last_fullsized":"2024-09-17T05:28:05.409823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:35.698197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389470+0000","last_change":"2024-09-17T05:28:03.389470+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389470+0000","last_undegraded":"2024-09-17T05:28:03.389470+0000","last_fullsized":"2024-09-17T05:28:03.389470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393313+0000","last_change":"2024-09-17T05:28:03.393313+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393313+0000","last_undegraded":"2024-09-17T05:28:03.393313+0000","last_fullsized":"2024-09-17T05:28:03.393313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387264+0000","last_change":"2024-09-17T05:28:03.387264+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387264+0000","last_undegraded":"2024-09-17T05:28:03.387264+0000","last_fullsized":"2024-09-17T05:28:03.387264+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386893+0000","last_change":"2024-09-17T05:28:03.386893+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386893+0000","last_undegraded":"2024-09-17T05:28:03.386893+0000","last_fullsized":"2024-09-17T05:28:03.386893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410396+0000","last_change":"2024-09-17T05:28:04.419337+0000","last_active":"2024-09-17T05:28:05.410396+0000","last_peered":"2024-09-17T05:28:05.410396+0000","last_clean":"2024-09-17T05:28:05.410396+0000","last_became_active":"2024-09-17T05:28:04.419066+0000","last_became_peered":"2024-09-17T05:28:04.419066+0000","last_unstale":"2024-09-17T05:28:05.410396+0000","last_undegraded":"2024-09-17T05:28:05.410396+0000","last_fullsized":"2024-09-17T05:28:05.410396+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:28.650511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395190+0000","last_change":"2024-09-17T05:28:03.395190+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395190+0000","last_undegraded":"2024-09-17T05:28:03.395190+0000","last_fullsized":"2024-09-17T05:28:03.395190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393821+0000","last_change":"2024-09-17T05:28:03.393821+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393821+0000","last_undegraded":"2024-09-17T05:28:03.393821+0000","last_fullsized":"2024-09-17T05:28:03.393821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386847+0000","last_change":"2024-09-17T05:28:03.386847+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386847+0000","last_undegraded":"2024-09-17T05:28:03.386847+0000","last_fullsized":"2024-09-17T05:28:03.386847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387350+0000","last_change":"2024-09-17T05:28:03.387350+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387350+0000","last_undegraded":"2024-09-17T05:28:03.387350+0000","last_fullsized":"2024-09-17T05:28:03.387350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394182+0000","last_change":"2024-09-17T05:28:03.394182+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394182+0000","last_undegraded":"2024-09-17T05:28:03.394182+0000","last_fullsized":"2024-09-17T05:28:03.394182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409026+0000","last_change":"2024-09-17T05:28:04.414079+0000","last_active":"2024-09-17T05:28:05.409026+0000","last_peered":"2024-09-17T05:28:05.409026+0000","last_clean":"2024-09-17T05:28:05.409026+0000","last_became_active":"2024-09-17T05:28:04.413733+0000","last_became_peered":"2024-09-17T05:28:04.413733+0000","last_unstale":"2024-09-17T05:28:05.409026+0000","last_undegraded":"2024-09-17T05:28:05.409026+0000","last_fullsized":"2024-09-17T05:28:05.409026+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:15.069773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389389+0000","last_change":"2024-09-17T05:28:03.389389+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389389+0000","last_undegraded":"2024-09-17T05:28:03.389389+0000","last_fullsized":"2024-09-17T05:28:03.389389+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411758+0000","last_change":"2024-09-17T05:28:03.408603+0000","last_active":"2024-09-17T05:28:03.411758+0000","last_peered":"2024-09-17T05:28:03.411758+0000","last_clean":"2024-09-17T05:28:03.411758+0000","last_became_active":"2024-09-17T05:28:03.408354+0000","last_became_peered":"2024-09-17T05:28:03.408354+0000","last_unstale":"2024-09-17T05:28:03.411758+0000","last_undegraded":"2024-09-17T05:28:03.411758+0000","last_fullsized":"2024-09-17T05:28:03.411758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:20.506564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386337+0000","last_change":"2024-09-17T05:28:03.386337+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386337+0000","last_undegraded":"2024-09-17T05:28:03.386337+0000","last_fullsized":"2024-09-17T05:28:03.386337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408180+0000","last_change":"2024-09-17T05:28:03.403643+0000","last_active":"2024-09-17T05:28:03.408180+0000","last_peered":"2024-09-17T05:28:03.408180+0000","last_clean":"2024-09-17T05:28:03.408180+0000","last_became_active":"2024-09-17T05:28:03.403353+0000","last_became_peered":"2024-09-17T05:28:03.403353+0000","last_unstale":"2024-09-17T05:28:03.408180+0000","last_undegraded":"2024-09-17T05:28:03.408180+0000","last_fullsized":"2024-09-17T05:28:03.408180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:30.436401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387803+0000","last_change":"2024-09-17T05:28:03.387803+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387803+0000","last_undegraded":"2024-09-17T05:28:03.387803+0000","last_fullsized":"2024-09-17T05:28:03.387803+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411701+0000","last_change":"2024-09-17T05:28:03.407955+0000","last_active":"2024-09-17T05:28:03.411701+0000","last_peered":"2024-09-17T05:28:03.411701+0000","last_clean":"2024-09-17T05:28:03.411701+0000","last_became_active":"2024-09-17T05:28:03.407597+0000","last_became_peered":"2024-09-17T05:28:03.407597+0000","last_unstale":"2024-09-17T05:28:03.411701+0000","last_undegraded":"2024-09-17T05:28:03.411701+0000","last_fullsized":"2024-09-17T05:28:03.411701+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:31.638210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410107+0000","last_change":"2024-09-17T05:28:04.411454+0000","last_active":"2024-09-17T05:28:05.410107+0000","last_peered":"2024-09-17T05:28:05.410107+0000","last_clean":"2024-09-17T05:28:05.410107+0000","last_became_active":"2024-09-17T05:28:04.411144+0000","last_became_peered":"2024-09-17T05:28:04.411144+0000","last_unstale":"2024-09-17T05:28:05.410107+0000","last_undegraded":"2024-09-17T05:28:05.410107+0000","last_fullsized":"2024-09-17T05:28:05.410107+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:10.404469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410169+0000","last_change":"2024-09-17T05:28:03.406275+0000","last_active":"2024-09-17T05:28:03.410169+0000","last_peered":"2024-09-17T05:28:03.410169+0000","last_clean":"2024-09-17T05:28:03.410169+0000","last_became_active":"2024-09-17T05:28:03.405506+0000","last_became_peered":"2024-09-17T05:28:03.405506+0000","last_unstale":"2024-09-17T05:28:03.410169+0000","last_undegraded":"2024-09-17T05:28:03.410169+0000","last_fullsized":"2024-09-17T05:28:03.410169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:46.627002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395704+0000","last_change":"2024-09-17T05:28:03.395704+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395704+0000","last_undegraded":"2024-09-17T05:28:03.395704+0000","last_fullsized":"2024-09-17T05:28:03.395704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410230+0000","last_change":"2024-09-17T05:28:03.406212+0000","last_active":"2024-09-17T05:28:03.410230+0000","last_peered":"2024-09-17T05:28:03.410230+0000","last_clean":"2024-09-17T05:28:03.410230+0000","last_became_active":"2024-09-17T05:28:03.405428+0000","last_became_peered":"2024-09-17T05:28:03.405428+0000","last_unstale":"2024-09-17T05:28:03.410230+0000","last_undegraded":"2024-09-17T05:28:03.410230+0000","last_fullsized":"2024-09-17T05:28:03.410230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:08.586828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394459+0000","last_change":"2024-09-17T05:28:03.394459+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394459+0000","last_undegraded":"2024-09-17T05:28:03.394459+0000","last_fullsized":"2024-09-17T05:28:03.394459+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410102+0000","last_change":"2024-09-17T05:28:03.405006+0000","last_active":"2024-09-17T05:28:03.410102+0000","last_peered":"2024-09-17T05:28:03.410102+0000","last_clean":"2024-09-17T05:28:03.410102+0000","last_became_active":"2024-09-17T05:28:03.404741+0000","last_became_peered":"2024-09-17T05:28:03.404741+0000","last_unstale":"2024-09-17T05:28:03.410102+0000","last_undegraded":"2024-09-17T05:28:03.410102+0000","last_fullsized":"2024-09-17T05:28:03.410102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:23.814938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393871+0000","last_change":"2024-09-17T05:28:03.393871+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393871+0000","last_undegraded":"2024-09-17T05:28:03.393871+0000","last_fullsized":"2024-09-17T05:28:03.393871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.399582+0000","last_change":"2024-09-17T05:28:03.397856+0000","last_active":"2024-09-17T05:28:03.399582+0000","last_peered":"2024-09-17T05:28:03.399582+0000","last_clean":"2024-09-17T05:28:03.399582+0000","last_became_active":"2024-09-17T05:28:03.397609+0000","last_became_peered":"2024-09-17T05:28:03.397609+0000","last_unstale":"2024-09-17T05:28:03.399582+0000","last_undegraded":"2024-09-17T05:28:03.399582+0000","last_fullsized":"2024-09-17T05:28:03.399582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:40.677984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393780+0000","last_change":"2024-09-17T05:28:03.393780+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393780+0000","last_undegraded":"2024-09-17T05:28:03.393780+0000","last_fullsized":"2024-09-17T05:28:03.393780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.414117+0000","last_change":"2024-09-17T05:28:03.410366+0000","last_active":"2024-09-17T05:28:03.414117+0000","last_peered":"2024-09-17T05:28:03.414117+0000","last_clean":"2024-09-17T05:28:03.414117+0000","last_became_active":"2024-09-17T05:28:03.409222+0000","last_became_peered":"2024-09-17T05:28:03.409222+0000","last_unstale":"2024-09-17T05:28:03.414117+0000","last_undegraded":"2024-09-17T05:28:03.414117+0000","last_fullsized":"2024-09-17T05:28:03.414117+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:11.005389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391108+0000","last_change":"2024-09-17T05:28:03.391108+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391108+0000","last_undegraded":"2024-09-17T05:28:03.391108+0000","last_fullsized":"2024-09-17T05:28:03.391108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.403575+0000","last_change":"2024-09-17T05:28:03.400471+0000","last_active":"2024-09-17T05:28:03.403575+0000","last_peered":"2024-09-17T05:28:03.403575+0000","last_clean":"2024-09-17T05:28:03.403575+0000","last_became_active":"2024-09-17T05:28:03.400202+0000","last_became_peered":"2024-09-17T05:28:03.400202+0000","last_unstale":"2024-09-17T05:28:03.403575+0000","last_undegraded":"2024-09-17T05:28:03.403575+0000","last_fullsized":"2024-09-17T05:28:03.403575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:58.648337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.390844+0000","last_change":"2024-09-17T05:28:03.390844+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.390844+0000","last_undegraded":"2024-09-17T05:28:03.390844+0000","last_fullsized":"2024-09-17T05:28:03.390844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.409027+0000","last_change":"2024-09-17T05:28:03.401227+0000","last_active":"2024-09-17T05:28:03.409027+0000","last_peered":"2024-09-17T05:28:03.409027+0000","last_clean":"2024-09-17T05:28:03.409027+0000","last_became_active":"2024-09-17T05:28:03.400688+0000","last_became_peered":"2024-09-17T05:28:03.400688+0000","last_unstale":"2024-09-17T05:28:03.409027+0000","last_undegraded":"2024-09-17T05:28:03.409027+0000","last_fullsized":"2024-09-17T05:28:03.409027+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:45.856441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409693+0000","last_change":"2024-09-17T05:28:04.413706+0000","last_active":"2024-09-17T05:28:05.409693+0000","last_peered":"2024-09-17T05:28:05.409693+0000","last_clean":"2024-09-17T05:28:05.409693+0000","last_became_active":"2024-09-17T05:28:04.413217+0000","last_became_peered":"2024-09-17T05:28:04.413217+0000","last_unstale":"2024-09-17T05:28:05.409693+0000","last_undegraded":"2024-09-17T05:28:05.409693+0000","last_fullsized":"2024-09-17T05:28:05.409693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:33.217682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407650+0000","last_change":"2024-09-17T05:28:03.401672+0000","last_active":"2024-09-17T05:28:03.407650+0000","last_peered":"2024-09-17T05:28:03.407650+0000","last_clean":"2024-09-17T05:28:03.407650+0000","last_became_active":"2024-09-17T05:28:03.401380+0000","last_became_peered":"2024-09-17T05:28:03.401380+0000","last_unstale":"2024-09-17T05:28:03.407650+0000","last_undegraded":"2024-09-17T05:28:03.407650+0000","last_fullsized":"2024-09-17T05:28:03.407650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:57.580288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394819+0000","last_change":"2024-09-17T05:28:03.394819+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394819+0000","last_undegraded":"2024-09-17T05:28:03.394819+0000","last_fullsized":"2024-09-17T05:28:03.394819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411516+0000","last_change":"2024-09-17T05:28:03.408491+0000","last_active":"2024-09-17T05:28:03.411516+0000","last_peered":"2024-09-17T05:28:03.411516+0000","last_clean":"2024-09-17T05:28:03.411516+0000","last_became_active":"2024-09-17T05:28:03.407793+0000","last_became_peered":"2024-09-17T05:28:03.407793+0000","last_unstale":"2024-09-17T05:28:03.411516+0000","last_undegraded":"2024-09-17T05:28:03.411516+0000","last_fullsized":"2024-09-17T05:28:03.411516+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.426226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389843+0000","last_change":"2024-09-17T05:28:03.389843+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389843+0000","last_undegraded":"2024-09-17T05:28:03.389843+0000","last_fullsized":"2024-09-17T05:28:03.389843+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411542+0000","last_change":"2024-09-17T05:28:03.406639+0000","last_active":"2024-09-17T05:28:03.411542+0000","last_peered":"2024-09-17T05:28:03.411542+0000","last_clean":"2024-09-17T05:28:03.411542+0000","last_became_active":"2024-09-17T05:28:03.406360+0000","last_became_peered":"2024-09-17T05:28:03.406360+0000","last_unstale":"2024-09-17T05:28:03.411542+0000","last_undegraded":"2024-09-17T05:28:03.411542+0000","last_fullsized":"2024-09-17T05:28:03.411542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:17.531567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.404093+0000","last_change":"2024-09-17T05:28:03.401363+0000","last_active":"2024-09-17T05:28:03.404093+0000","last_peered":"2024-09-17T05:28:03.404093+0000","last_clean":"2024-09-17T05:28:03.404093+0000","last_became_active":"2024-09-17T05:28:03.401110+0000","last_became_peered":"2024-09-17T05:28:03.401110+0000","last_unstale":"2024-09-17T05:28:03.404093+0000","last_undegraded":"2024-09-17T05:28:03.404093+0000","last_fullsized":"2024-09-17T05:28:03.404093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:24.220817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.413897+0000","last_change":"2024-09-17T05:28:03.409353+0000","last_active":"2024-09-17T05:28:03.413897+0000","last_peered":"2024-09-17T05:28:03.413897+0000","last_clean":"2024-09-17T05:28:03.413897+0000","last_became_active":"2024-09-17T05:28:03.409087+0000","last_became_peered":"2024-09-17T05:28:03.409087+0000","last_unstale":"2024-09-17T05:28:03.413897+0000","last_undegraded":"2024-09-17T05:28:03.413897+0000","last_fullsized":"2024-09-17T05:28:03.413897+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:37.361308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388677+0000","last_change":"2024-09-17T05:28:03.388677+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388677+0000","last_undegraded":"2024-09-17T05:28:03.388677+0000","last_fullsized":"2024-09-17T05:28:03.388677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.409986+0000","last_change":"2024-09-17T05:28:03.403774+0000","last_active":"2024-09-17T05:28:03.409986+0000","last_peered":"2024-09-17T05:28:03.409986+0000","last_clean":"2024-09-17T05:28:03.409986+0000","last_became_active":"2024-09-17T05:28:03.403425+0000","last_became_peered":"2024-09-17T05:28:03.403425+0000","last_unstale":"2024-09-17T05:28:03.409986+0000","last_undegraded":"2024-09-17T05:28:03.409986+0000","last_fullsized":"2024-09-17T05:28:03.409986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:08.155865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388720+0000","last_change":"2024-09-17T05:28:03.388720+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388720+0000","last_undegraded":"2024-09-17T05:28:03.388720+0000","last_fullsized":"2024-09-17T05:28:03.388720+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410531+0000","last_change":"2024-09-17T05:28:03.409845+0000","last_active":"2024-09-17T05:28:05.410531+0000","last_peered":"2024-09-17T05:28:05.410531+0000","last_clean":"2024-09-17T05:28:05.410531+0000","last_became_active":"2024-09-17T05:28:03.409519+0000","last_became_peered":"2024-09-17T05:28:03.409519+0000","last_unstale":"2024-09-17T05:28:05.410531+0000","last_undegraded":"2024-09-17T05:28:05.410531+0000","last_fullsized":"2024-09-17T05:28:05.410531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:04.018422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412689+0000","last_change":"2024-09-17T05:28:03.408640+0000","last_active":"2024-09-17T05:28:03.412689+0000","last_peered":"2024-09-17T05:28:03.412689+0000","last_clean":"2024-09-17T05:28:03.412689+0000","last_became_active":"2024-09-17T05:28:03.408371+0000","last_became_peered":"2024-09-17T05:28:03.408371+0000","last_unstale":"2024-09-17T05:28:03.412689+0000","last_undegraded":"2024-09-17T05:28:03.412689+0000","last_fullsized":"2024-09-17T05:28:03.412689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:51.246256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387320+0000","last_change":"2024-09-17T05:28:03.387320+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387320+0000","last_undegraded":"2024-09-17T05:28:03.387320+0000","last_fullsized":"2024-09-17T05:28:03.387320+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.403739+0000","last_change":"2024-09-17T05:28:03.398552+0000","last_active":"2024-09-17T05:28:03.403739+0000","last_peered":"2024-09-17T05:28:03.403739+0000","last_clean":"2024-09-17T05:28:03.403739+0000","last_became_active":"2024-09-17T05:28:03.398282+0000","last_became_peered":"2024-09-17T05:28:03.398282+0000","last_unstale":"2024-09-17T05:28:03.403739+0000","last_undegraded":"2024-09-17T05:28:03.403739+0000","last_fullsized":"2024-09-17T05:28:03.403739+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:54.593079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.390768+0000","last_change":"2024-09-17T05:28:03.390768+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.390768+0000","last_undegraded":"2024-09-17T05:28:03.390768+0000","last_fullsized":"2024-09-17T05:28:03.390768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410212+0000","last_change":"2024-09-17T05:28:03.404997+0000","last_active":"2024-09-17T05:28:03.410212+0000","last_peered":"2024-09-17T05:28:03.410212+0000","last_clean":"2024-09-17T05:28:03.410212+0000","last_became_active":"2024-09-17T05:28:03.404512+0000","last_became_peered":"2024-09-17T05:28:03.404512+0000","last_unstale":"2024-09-17T05:28:03.410212+0000","last_undegraded":"2024-09-17T05:28:03.410212+0000","last_fullsized":"2024-09-17T05:28:03.410212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:07.584572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398333+0000","last_change":"2024-09-17T05:28:03.398333+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398333+0000","last_undegraded":"2024-09-17T05:28:03.398333+0000","last_fullsized":"2024-09-17T05:28:03.398333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408284+0000","last_change":"2024-09-17T05:28:03.400873+0000","last_active":"2024-09-17T05:28:03.408284+0000","last_peered":"2024-09-17T05:28:03.408284+0000","last_clean":"2024-09-17T05:28:03.408284+0000","last_became_active":"2024-09-17T05:28:03.400549+0000","last_became_peered":"2024-09-17T05:28:03.400549+0000","last_unstale":"2024-09-17T05:28:03.408284+0000","last_undegraded":"2024-09-17T05:28:03.408284+0000","last_fullsized":"2024-09-17T05:28:03.408284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:31.777509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394647+0000","last_change":"2024-09-17T05:28:03.394647+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394647+0000","last_undegraded":"2024-09-17T05:28:03.394647+0000","last_fullsized":"2024-09-17T05:28:03.394647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.398760+0000","last_change":"2024-09-17T05:28:03.395798+0000","last_active":"2024-09-17T05:28:03.398760+0000","last_peered":"2024-09-17T05:28:03.398760+0000","last_clean":"2024-09-17T05:28:03.398760+0000","last_became_active":"2024-09-17T05:28:03.395543+0000","last_became_peered":"2024-09-17T05:28:03.395543+0000","last_unstale":"2024-09-17T05:28:03.398760+0000","last_undegraded":"2024-09-17T05:28:03.398760+0000","last_fullsized":"2024-09-17T05:28:03.398760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:37.361308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409491+0000","last_change":"2024-09-17T05:28:04.423847+0000","last_active":"2024-09-17T05:28:05.409491+0000","last_peered":"2024-09-17T05:28:05.409491+0000","last_clean":"2024-09-17T05:28:05.409491+0000","last_became_active":"2024-09-17T05:28:04.423569+0000","last_became_peered":"2024-09-17T05:28:04.423569+0000","last_unstale":"2024-09-17T05:28:05.409491+0000","last_undegraded":"2024-09-17T05:28:05.409491+0000","last_fullsized":"2024-09-17T05:28:05.409491+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:58.595615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408407+0000","last_change":"2024-09-17T05:28:03.400992+0000","last_active":"2024-09-17T05:28:03.408407+0000","last_peered":"2024-09-17T05:28:03.408407+0000","last_clean":"2024-09-17T05:28:03.408407+0000","last_became_active":"2024-09-17T05:28:03.400690+0000","last_became_peered":"2024-09-17T05:28:03.400690+0000","last_unstale":"2024-09-17T05:28:03.408407+0000","last_undegraded":"2024-09-17T05:28:03.408407+0000","last_fullsized":"2024-09-17T05:28:03.408407+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:49.611905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409358+0000","last_change":"2024-09-17T05:28:04.426697+0000","last_active":"2024-09-17T05:28:05.409358+0000","last_peered":"2024-09-17T05:28:05.409358+0000","last_clean":"2024-09-17T05:28:05.409358+0000","last_became_active":"2024-09-17T05:28:04.426419+0000","last_became_peered":"2024-09-17T05:28:04.426419+0000","last_unstale":"2024-09-17T05:28:05.409358+0000","last_undegraded":"2024-09-17T05:28:05.409358+0000","last_fullsized":"2024-09-17T05:28:05.409358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.432704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412599+0000","last_change":"2024-09-17T05:28:03.407318+0000","last_active":"2024-09-17T05:28:03.412599+0000","last_peered":"2024-09-17T05:28:03.412599+0000","last_clean":"2024-09-17T05:28:03.412599+0000","last_became_active":"2024-09-17T05:28:03.407059+0000","last_became_peered":"2024-09-17T05:28:03.407059+0000","last_unstale":"2024-09-17T05:28:03.412599+0000","last_undegraded":"2024-09-17T05:28:03.412599+0000","last_fullsized":"2024-09-17T05:28:03.412599+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:15.545411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393056+0000","last_change":"2024-09-17T05:28:03.393056+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393056+0000","last_undegraded":"2024-09-17T05:28:03.393056+0000","last_fullsized":"2024-09-17T05:28:03.393056+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410708+0000","last_change":"2024-09-17T05:28:03.408701+0000","last_active":"2024-09-17T05:28:05.410708+0000","last_peered":"2024-09-17T05:28:05.410708+0000","last_clean":"2024-09-17T05:28:05.410708+0000","last_became_active":"2024-09-17T05:28:03.408379+0000","last_became_peered":"2024-09-17T05:28:03.408379+0000","last_unstale":"2024-09-17T05:28:05.410708+0000","last_undegraded":"2024-09-17T05:28:05.410708+0000","last_fullsized":"2024-09-17T05:28:05.410708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:24.330713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394936+0000","last_change":"2024-09-17T05:28:03.394936+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394936+0000","last_undegraded":"2024-09-17T05:28:03.394936+0000","last_fullsized":"2024-09-17T05:28:03.394936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.402356+0000","last_change":"2024-09-17T05:28:03.397060+0000","last_active":"2024-09-17T05:28:03.402356+0000","last_peered":"2024-09-17T05:28:03.402356+0000","last_clean":"2024-09-17T05:28:03.402356+0000","last_became_active":"2024-09-17T05:28:03.396798+0000","last_became_peered":"2024-09-17T05:28:03.396798+0000","last_unstale":"2024-09-17T05:28:03.402356+0000","last_undegraded":"2024-09-17T05:28:03.402356+0000","last_fullsized":"2024-09-17T05:28:03.402356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.747610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.392714+0000","last_change":"2024-09-17T05:28:03.392714+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.392714+0000","last_undegraded":"2024-09-17T05:28:03.392714+0000","last_fullsized":"2024-09-17T05:28:03.392714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407241+0000","last_change":"2024-09-17T05:28:03.402041+0000","last_active":"2024-09-17T05:28:03.407241+0000","last_peered":"2024-09-17T05:28:03.407241+0000","last_clean":"2024-09-17T05:28:03.407241+0000","last_became_active":"2024-09-17T05:28:03.401757+0000","last_became_peered":"2024-09-17T05:28:03.401757+0000","last_unstale":"2024-09-17T05:28:03.407241+0000","last_undegraded":"2024-09-17T05:28:03.407241+0000","last_fullsized":"2024-09-17T05:28:03.407241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:39.617757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.397895+0000","last_change":"2024-09-17T05:28:03.397895+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.397895+0000","last_undegraded":"2024-09-17T05:28:03.397895+0000","last_fullsized":"2024-09-17T05:28:03.397895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407808+0000","last_change":"2024-09-17T05:28:03.406099+0000","last_active":"2024-09-17T05:28:03.407808+0000","last_peered":"2024-09-17T05:28:03.407808+0000","last_clean":"2024-09-17T05:28:03.407808+0000","last_became_active":"2024-09-17T05:28:03.405763+0000","last_became_peered":"2024-09-17T05:28:03.405763+0000","last_unstale":"2024-09-17T05:28:03.407808+0000","last_undegraded":"2024-09-17T05:28:03.407808+0000","last_fullsized":"2024-09-17T05:28:03.407808+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:57.580288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409499+0000","last_change":"2024-09-17T05:28:04.405966+0000","last_active":"2024-09-17T05:28:05.409499+0000","last_peered":"2024-09-17T05:28:05.409499+0000","last_clean":"2024-09-17T05:28:05.409499+0000","last_became_active":"2024-09-17T05:28:04.405723+0000","last_became_peered":"2024-09-17T05:28:04.405723+0000","last_unstale":"2024-09-17T05:28:05.409499+0000","last_undegraded":"2024-09-17T05:28:05.409499+0000","last_fullsized":"2024-09-17T05:28:05.409499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:09:42.528631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.405322+0000","last_change":"2024-09-17T05:28:03.403069+0000","last_active":"2024-09-17T05:28:03.405322+0000","last_peered":"2024-09-17T05:28:03.405322+0000","last_clean":"2024-09-17T05:28:03.405322+0000","last_became_active":"2024-09-17T05:28:03.402652+0000","last_became_peered":"2024-09-17T05:28:03.402652+0000","last_unstale":"2024-09-17T05:28:03.405322+0000","last_undegraded":"2024-09-17T05:28:03.405322+0000","last_fullsized":"2024-09-17T05:28:03.405322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:42.875418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389143+0000","last_change":"2024-09-17T05:28:03.389143+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389143+0000","last_undegraded":"2024-09-17T05:28:03.389143+0000","last_fullsized":"2024-09-17T05:28:03.389143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411465+0000","last_change":"2024-09-17T05:28:03.407068+0000","last_active":"2024-09-17T05:28:03.411465+0000","last_peered":"2024-09-17T05:28:03.411465+0000","last_clean":"2024-09-17T05:28:03.411465+0000","last_became_active":"2024-09-17T05:28:03.406674+0000","last_became_peered":"2024-09-17T05:28:03.406674+0000","last_unstale":"2024-09-17T05:28:03.411465+0000","last_undegraded":"2024-09-17T05:28:03.411465+0000","last_fullsized":"2024-09-17T05:28:03.411465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:54.447845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387799+0000","last_change":"2024-09-17T05:28:03.387799+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387799+0000","last_undegraded":"2024-09-17T05:28:03.387799+0000","last_fullsized":"2024-09-17T05:28:03.387799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407397+0000","last_change":"2024-09-17T05:28:03.402396+0000","last_active":"2024-09-17T05:28:03.407397+0000","last_peered":"2024-09-17T05:28:03.407397+0000","last_clean":"2024-09-17T05:28:03.407397+0000","last_became_active":"2024-09-17T05:28:03.402089+0000","last_became_peered":"2024-09-17T05:28:03.402089+0000","last_unstale":"2024-09-17T05:28:03.407397+0000","last_undegraded":"2024-09-17T05:28:03.407397+0000","last_fullsized":"2024-09-17T05:28:03.407397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:31.994460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410638+0000","last_change":"2024-09-17T05:28:04.407978+0000","last_active":"2024-09-17T05:28:05.410638+0000","last_peered":"2024-09-17T05:28:05.410638+0000","last_clean":"2024-09-17T05:28:05.410638+0000","last_became_active":"2024-09-17T05:28:04.407717+0000","last_became_peered":"2024-09-17T05:28:04.407717+0000","last_unstale":"2024-09-17T05:28:05.410638+0000","last_undegraded":"2024-09-17T05:28:05.410638+0000","last_fullsized":"2024-09-17T05:28:05.410638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:52.263630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412718+0000","last_change":"2024-09-17T05:28:03.408678+0000","last_active":"2024-09-17T05:28:03.412718+0000","last_peered":"2024-09-17T05:28:03.412718+0000","last_clean":"2024-09-17T05:28:03.412718+0000","last_became_active":"2024-09-17T05:28:03.408375+0000","last_became_peered":"2024-09-17T05:28:03.408375+0000","last_unstale":"2024-09-17T05:28:03.412718+0000","last_undegraded":"2024-09-17T05:28:03.412718+0000","last_fullsized":"2024-09-17T05:28:03.412718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:10.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":153,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.393074+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:28:03.393074+0000","last_peered":"2024-09-17T05:28:03.393074+0000","last_clean":"2024-09-17T05:28:03.393074+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:28:03.393074+0000","last_undegraded":"2024-09-17T05:28:03.393074+0000","last_fullsized":"2024-09-17T05:28:03.393074+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":792467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1130496,"data_stored":805564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:09.294 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3f is not active+clean 2024-09-17T05:28:09.294 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387945+0000', 'last_change': '2024-09-17T05:28:03.387945+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387945+0000', 'last_undegraded': '2024-09-17T05:28:03.387945+0000', 'last_fullsized': '2024-09-17T05:28:03.387945+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:09.294 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3e is not active+clean 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.388044+0000', 'last_change': '2024-09-17T05:28:03.388044+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.388044+0000', 'last_undegraded': '2024-09-17T05:28:03.388044+0000', 'last_fullsized': '2024-09-17T05:28:03.388044+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3d is not active+clean 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.391128+0000', 'last_change': '2024-09-17T05:28:03.391128+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.391128+0000', 'last_undegraded': '2024-09-17T05:28:03.391128+0000', 'last_fullsized': '2024-09-17T05:28:03.391128+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3b is not active+clean 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387704+0000', 'last_change': '2024-09-17T05:28:03.387704+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387704+0000', 'last_undegraded': '2024-09-17T05:28:03.387704+0000', 'last_fullsized': '2024-09-17T05:28:03.387704+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.392926+0000', 'last_change': '2024-09-17T05:28:03.392926+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.392926+0000', 'last_undegraded': '2024-09-17T05:28:03.392926+0000', 'last_fullsized': '2024-09-17T05:28:03.392926+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:PG 7.39 is not active+clean 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.398593+0000', 'last_change': '2024-09-17T05:28:03.398593+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.398593+0000', 'last_undegraded': '2024-09-17T05:28:03.398593+0000', 'last_fullsized': '2024-09-17T05:28:03.398593+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:PG 7.37 is not active+clean 2024-09-17T05:28:09.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.398634+0000', 'last_change': '2024-09-17T05:28:03.398634+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.398634+0000', 'last_undegraded': '2024-09-17T05:28:03.398634+0000', 'last_fullsized': '2024-09-17T05:28:03.398634+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:PG 7.36 is not active+clean 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.393796+0000', 'last_change': '2024-09-17T05:28:03.393796+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.393796+0000', 'last_undegraded': '2024-09-17T05:28:03.393796+0000', 'last_fullsized': '2024-09-17T05:28:03.393796+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:PG 7.34 is not active+clean 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.389118+0000', 'last_change': '2024-09-17T05:28:03.389118+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.389118+0000', 'last_undegraded': '2024-09-17T05:28:03.389118+0000', 'last_fullsized': '2024-09-17T05:28:03.389118+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:PG 7.33 is not active+clean 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.395047+0000', 'last_change': '2024-09-17T05:28:03.395047+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.395047+0000', 'last_undegraded': '2024-09-17T05:28:03.395047+0000', 'last_fullsized': '2024-09-17T05:28:03.395047+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:PG 7.30 is not active+clean 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.386968+0000', 'last_change': '2024-09-17T05:28:03.386968+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.386968+0000', 'last_undegraded': '2024-09-17T05:28:03.386968+0000', 'last_fullsized': '2024-09-17T05:28:03.386968+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2f is not active+clean 2024-09-17T05:28:09.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387639+0000', 'last_change': '2024-09-17T05:28:03.387639+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387639+0000', 'last_undegraded': '2024-09-17T05:28:03.387639+0000', 'last_fullsized': '2024-09-17T05:28:03.387639+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2e is not active+clean 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.391815+0000', 'last_change': '2024-09-17T05:28:03.391815+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.391815+0000', 'last_undegraded': '2024-09-17T05:28:03.391815+0000', 'last_fullsized': '2024-09-17T05:28:03.391815+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2d is not active+clean 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387562+0000', 'last_change': '2024-09-17T05:28:03.387562+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387562+0000', 'last_undegraded': '2024-09-17T05:28:03.387562+0000', 'last_fullsized': '2024-09-17T05:28:03.387562+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2c is not active+clean 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.395327+0000', 'last_change': '2024-09-17T05:28:03.395327+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.395327+0000', 'last_undegraded': '2024-09-17T05:28:03.395327+0000', 'last_fullsized': '2024-09-17T05:28:03.395327+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2a is not active+clean 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.389470+0000', 'last_change': '2024-09-17T05:28:03.389470+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.389470+0000', 'last_undegraded': '2024-09-17T05:28:03.389470+0000', 'last_fullsized': '2024-09-17T05:28:03.389470+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:PG 7.29 is not active+clean 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.393313+0000', 'last_change': '2024-09-17T05:28:03.393313+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.393313+0000', 'last_undegraded': '2024-09-17T05:28:03.393313+0000', 'last_fullsized': '2024-09-17T05:28:03.393313+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.297 INFO:tasks.ceph.ceph_manager.ceph:PG 7.28 is not active+clean 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387264+0000', 'last_change': '2024-09-17T05:28:03.387264+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387264+0000', 'last_undegraded': '2024-09-17T05:28:03.387264+0000', 'last_fullsized': '2024-09-17T05:28:03.387264+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:PG 7.27 is not active+clean 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.386893+0000', 'last_change': '2024-09-17T05:28:03.386893+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.386893+0000', 'last_undegraded': '2024-09-17T05:28:03.386893+0000', 'last_fullsized': '2024-09-17T05:28:03.386893+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:PG 7.25 is not active+clean 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.395190+0000', 'last_change': '2024-09-17T05:28:03.395190+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.395190+0000', 'last_undegraded': '2024-09-17T05:28:03.395190+0000', 'last_fullsized': '2024-09-17T05:28:03.395190+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:PG 7.24 is not active+clean 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.393821+0000', 'last_change': '2024-09-17T05:28:03.393821+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.393821+0000', 'last_undegraded': '2024-09-17T05:28:03.393821+0000', 'last_fullsized': '2024-09-17T05:28:03.393821+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:PG 7.23 is not active+clean 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.386847+0000', 'last_change': '2024-09-17T05:28:03.386847+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.386847+0000', 'last_undegraded': '2024-09-17T05:28:03.386847+0000', 'last_fullsized': '2024-09-17T05:28:03.386847+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:PG 7.22 is not active+clean 2024-09-17T05:28:09.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387350+0000', 'last_change': '2024-09-17T05:28:03.387350+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387350+0000', 'last_undegraded': '2024-09-17T05:28:03.387350+0000', 'last_fullsized': '2024-09-17T05:28:03.387350+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.394182+0000', 'last_change': '2024-09-17T05:28:03.394182+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.394182+0000', 'last_undegraded': '2024-09-17T05:28:03.394182+0000', 'last_fullsized': '2024-09-17T05:28:03.394182+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1e is not active+clean 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.389389+0000', 'last_change': '2024-09-17T05:28:03.389389+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.389389+0000', 'last_undegraded': '2024-09-17T05:28:03.389389+0000', 'last_fullsized': '2024-09-17T05:28:03.389389+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1f is not active+clean 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.386337+0000', 'last_change': '2024-09-17T05:28:03.386337+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.386337+0000', 'last_undegraded': '2024-09-17T05:28:03.386337+0000', 'last_fullsized': '2024-09-17T05:28:03.386337+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1c is not active+clean 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387803+0000', 'last_change': '2024-09-17T05:28:03.387803+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387803+0000', 'last_undegraded': '2024-09-17T05:28:03.387803+0000', 'last_fullsized': '2024-09-17T05:28:03.387803+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1a is not active+clean 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.395704+0000', 'last_change': '2024-09-17T05:28:03.395704+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.395704+0000', 'last_undegraded': '2024-09-17T05:28:03.395704+0000', 'last_fullsized': '2024-09-17T05:28:03.395704+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1b is not active+clean 2024-09-17T05:28:09.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.394459+0000', 'last_change': '2024-09-17T05:28:03.394459+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.394459+0000', 'last_undegraded': '2024-09-17T05:28:03.394459+0000', 'last_fullsized': '2024-09-17T05:28:03.394459+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:PG 7.18 is not active+clean 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.393871+0000', 'last_change': '2024-09-17T05:28:03.393871+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.393871+0000', 'last_undegraded': '2024-09-17T05:28:03.393871+0000', 'last_fullsized': '2024-09-17T05:28:03.393871+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:PG 7.19 is not active+clean 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.393780+0000', 'last_change': '2024-09-17T05:28:03.393780+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.393780+0000', 'last_undegraded': '2024-09-17T05:28:03.393780+0000', 'last_fullsized': '2024-09-17T05:28:03.393780+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:PG 7.16 is not active+clean 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.391108+0000', 'last_change': '2024-09-17T05:28:03.391108+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.391108+0000', 'last_undegraded': '2024-09-17T05:28:03.391108+0000', 'last_fullsized': '2024-09-17T05:28:03.391108+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:PG 7.17 is not active+clean 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.390844+0000', 'last_change': '2024-09-17T05:28:03.390844+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.390844+0000', 'last_undegraded': '2024-09-17T05:28:03.390844+0000', 'last_fullsized': '2024-09-17T05:28:03.390844+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:PG 7.15 is not active+clean 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.394819+0000', 'last_change': '2024-09-17T05:28:03.394819+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.394819+0000', 'last_undegraded': '2024-09-17T05:28:03.394819+0000', 'last_fullsized': '2024-09-17T05:28:03.394819+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.300 INFO:tasks.ceph.ceph_manager.ceph:PG 7.12 is not active+clean 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.389843+0000', 'last_change': '2024-09-17T05:28:03.389843+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.389843+0000', 'last_undegraded': '2024-09-17T05:28:03.389843+0000', 'last_fullsized': '2024-09-17T05:28:03.389843+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:PG 7.13 is not active+clean 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.388693+0000', 'last_change': '2024-09-17T05:28:03.388693+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.388693+0000', 'last_undegraded': '2024-09-17T05:28:03.388693+0000', 'last_fullsized': '2024-09-17T05:28:03.388693+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.388693+0000', 'last_change': '2024-09-17T05:28:03.388693+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.388693+0000', 'last_undegraded': '2024-09-17T05:28:03.388693+0000', 'last_fullsized': '2024-09-17T05:28:03.388693+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:PG 7.11 is not active+clean 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.388677+0000', 'last_change': '2024-09-17T05:28:03.388677+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.388677+0000', 'last_undegraded': '2024-09-17T05:28:03.388677+0000', 'last_fullsized': '2024-09-17T05:28:03.388677+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.388720+0000', 'last_change': '2024-09-17T05:28:03.388720+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.388720+0000', 'last_undegraded': '2024-09-17T05:28:03.388720+0000', 'last_fullsized': '2024-09-17T05:28:03.388720+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:PG 7.f is not active+clean 2024-09-17T05:28:09.301 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.388693+0000', 'last_change': '2024-09-17T05:28:03.388693+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.388693+0000', 'last_undegraded': '2024-09-17T05:28:03.388693+0000', 'last_fullsized': '2024-09-17T05:28:03.388693+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:PG 7.c is not active+clean 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387320+0000', 'last_change': '2024-09-17T05:28:03.387320+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387320+0000', 'last_undegraded': '2024-09-17T05:28:03.387320+0000', 'last_fullsized': '2024-09-17T05:28:03.387320+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:PG 7.d is not active+clean 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.390768+0000', 'last_change': '2024-09-17T05:28:03.390768+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.390768+0000', 'last_undegraded': '2024-09-17T05:28:03.390768+0000', 'last_fullsized': '2024-09-17T05:28:03.390768+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:PG 7.a is not active+clean 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.398333+0000', 'last_change': '2024-09-17T05:28:03.398333+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.398333+0000', 'last_undegraded': '2024-09-17T05:28:03.398333+0000', 'last_fullsized': '2024-09-17T05:28:03.398333+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:PG 7.b is not active+clean 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.394647+0000', 'last_change': '2024-09-17T05:28:03.394647+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.394647+0000', 'last_undegraded': '2024-09-17T05:28:03.394647+0000', 'last_fullsized': '2024-09-17T05:28:03.394647+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:PG 7.7 is not active+clean 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.393056+0000', 'last_change': '2024-09-17T05:28:03.393056+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.393056+0000', 'last_undegraded': '2024-09-17T05:28:03.393056+0000', 'last_fullsized': '2024-09-17T05:28:03.393056+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:PG 7.4 is not active+clean 2024-09-17T05:28:09.302 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.394936+0000', 'last_change': '2024-09-17T05:28:03.394936+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.394936+0000', 'last_undegraded': '2024-09-17T05:28:03.394936+0000', 'last_fullsized': '2024-09-17T05:28:03.394936+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:09.303 INFO:tasks.ceph.ceph_manager.ceph:PG 7.5 is not active+clean 2024-09-17T05:28:09.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.392714+0000', 'last_change': '2024-09-17T05:28:03.392714+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.392714+0000', 'last_undegraded': '2024-09-17T05:28:03.392714+0000', 'last_fullsized': '2024-09-17T05:28:03.392714+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:28:09.303 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2 is not active+clean 2024-09-17T05:28:09.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.397895+0000', 'last_change': '2024-09-17T05:28:03.397895+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.397895+0000', 'last_undegraded': '2024-09-17T05:28:03.397895+0000', 'last_fullsized': '2024-09-17T05:28:03.397895+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:28:09.303 INFO:tasks.ceph.ceph_manager.ceph:PG 7.0 is not active+clean 2024-09-17T05:28:09.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.389143+0000', 'last_change': '2024-09-17T05:28:03.389143+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.389143+0000', 'last_undegraded': '2024-09-17T05:28:03.389143+0000', 'last_fullsized': '2024-09-17T05:28:03.389143+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:28:09.303 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1 is not active+clean 2024-09-17T05:28:09.303 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:28:03.387799+0000', 'last_change': '2024-09-17T05:28:03.387799+0000', 'last_active': '2024-09-17T05:28:03.373314+0000', 'last_peered': '2024-09-17T05:28:03.373314+0000', 'last_clean': '2024-09-17T05:28:03.373314+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:28:03.387799+0000', 'last_undegraded': '2024-09-17T05:28:03.387799+0000', 'last_fullsized': '2024-09-17T05:28:03.387799+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'last_clean_scrub_stamp': '2024-09-17T05:28:03.373314+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:28:09.304 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:28:09.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- 172.21.15.122:0/3753219676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 msgr2=0x7f8e34071150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 --2- 172.21.15.122:0/3753219676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 0x7f8e34071150 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009bc0 tx=0x7f8e2c01cb70 comp rx=0 tx=0).stop 2024-09-17T05:28:09.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- 172.21.15.122:0/3753219676 shutdown_connections 2024-09-17T05:28:09.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 --2- 172.21.15.122:0/3753219676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 0x7f8e34071150 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- 172.21.15.122:0/3753219676 >> 172.21.15.122:0/3753219676 conn(0x7f8e340d0b10 msgr2=0x7f8e340d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:09.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- 172.21.15.122:0/3753219676 shutdown_connections 2024-09-17T05:28:09.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- 172.21.15.122:0/3753219676 wait complete. 2024-09-17T05:28:09.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 Processor -- start 2024-09-17T05:28:09.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- start start 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3406d330 0x7f8e3415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 0x7f8e3415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3415beb0 0x7f8e34164df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3416a5f0 con 0x7f8e340a1ca0 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3416a470 con 0x7f8e3406d330 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e38a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3416a770 con 0x7f8e3415beb0 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e31e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 0x7f8e3415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e31e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 0x7f8e3415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51886/0 (socket says 172.21.15.122:51886) 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e31e00640 1 -- 172.21.15.122:0/2617006009 learned_addr learned my addr 172.21.15.122:0/2617006009 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e33200640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3415beb0 0x7f8e34164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.479+0000 7f8e32800640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3406d330 0x7f8e3415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e31e00640 1 -- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3415beb0 msgr2=0x7f8e34164df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e31e00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3415beb0 0x7f8e34164df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e31e00640 1 -- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3406d330 msgr2=0x7f8e3415d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e31e00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3406d330 0x7f8e3415d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e31e00640 1 -- 172.21.15.122:0/2617006009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e34166d40 con 0x7f8e340a1ca0 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e31e00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 0x7f8e3415d880 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000c4b0 tx=0x7f8e2000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e2aa00640 1 -- 172.21.15.122:0/2617006009 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20016020 con 0x7f8e340a1ca0 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e2aa00640 1 -- 172.21.15.122:0/2617006009 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e20012920 con 0x7f8e340a1ca0 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e34165f80 con 0x7f8e340a1ca0 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e2aa00640 1 -- 172.21.15.122:0/2617006009 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20012d90 con 0x7f8e340a1ca0 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e34166540 con 0x7f8e340a1ca0 2024-09-17T05:28:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3406cc20 con 0x7f8e340a1ca0 2024-09-17T05:28:09.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e2aa00640 1 -- 172.21.15.122:0/2617006009 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f8e20019020 con 0x7f8e340a1ca0 2024-09-17T05:28:09.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.483+0000 7f8e2aa00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e14073560 0x7f8e14075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.487+0000 7f8e2aa00640 1 -- 172.21.15.122:0/2617006009 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8e20094410 con 0x7f8e340a1ca0 2024-09-17T05:28:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.487+0000 7f8e32800640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e14073560 0x7f8e14075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.487+0000 7f8e32800640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e14073560 0x7f8e14075a20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e3415f7c0 tx=0x7f8e24009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:09.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.487+0000 7f8e2aa00640 1 -- 172.21.15.122:0/2617006009 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8e20020030 con 0x7f8e340a1ca0 2024-09-17T05:28:09.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.711+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8e340c41e0 con 0x7f8e340a1ca0 2024-09-17T05:28:09.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e2aa00640 1 -- 172.21.15.122:0/2617006009 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1139 (secure 0 0 0) 0x7f8e2005e910 con 0x7f8e340a1ca0 2024-09-17T05:28:09.714 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:09.714 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":210,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550693,"num_in_osds":8,"osd_in_since":1726550682,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":50},{"state_name":"active+clean","count":47}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590390,"bytes_used":269283328,"bytes_avail":772824829952,"bytes_total":773094113280,"inactive_pgs_ratio":0.51546388864517212,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":26,"btime":"2024-09-17T05:28:07:412553+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5388}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T05:26:54.470306+0000","services":{}},"progress_events":{}} 2024-09-17T05:28:09.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e14073560 msgr2=0x7f8e14075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e14073560 0x7f8e14075a20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e3415f7c0 tx=0x7f8e24009040 comp rx=0 tx=0).stop 2024-09-17T05:28:09.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 msgr2=0x7f8e3415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 0x7f8e3415d880 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000c4b0 tx=0x7f8e2000c970 comp rx=0 tx=0).stop 2024-09-17T05:28:09.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 shutdown_connections 2024-09-17T05:28:09.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e14073560 0x7f8e14075a20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3415beb0 0x7f8e34164df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e340a1ca0 0x7f8e3415d880 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 --2- 172.21.15.122:0/2617006009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3406d330 0x7f8e3415d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 >> 172.21.15.122:0/2617006009 conn(0x7f8e340d0b10 msgr2=0x7f8e340b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:09.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 shutdown_connections 2024-09-17T05:28:09.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.715+0000 7f8e38a00640 1 -- 172.21.15.122:0/2617006009 wait complete. 2024-09-17T05:28:09.736 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:28:09.736 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 -- 172.21.15.122:0/1549715258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415e330 msgr2=0x7f3d3406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1549715258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415e330 0x7f3d3406ec80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3d340c58a0 tx=0x7f3d2001c7a0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 -- 172.21.15.122:0/1549715258 shutdown_connections 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1549715258 >> v2:172.21.15.173:3300/0 conn(0x7f3d340b11d0 0x7f3d340afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1549715258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415e330 0x7f3d3406ec80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1549715258 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3415df50 0x7f3d3406e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 -- 172.21.15.122:0/1549715258 >> 172.21.15.122:0/1549715258 conn(0x7f3d340d1dc0 msgr2=0x7f3d340d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 -- 172.21.15.122:0/1549715258 shutdown_connections 2024-09-17T05:28:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.875+0000 7f3d3b400640 1 -- 172.21.15.122:0/1549715258 wait complete. 2024-09-17T05:28:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 Processor -- start 2024-09-17T05:28:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 -- start start 2024-09-17T05:28:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d340b11d0 0x7f3d341530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415df50 0x7f3d340d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3415e330 0x7f3d340d0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d340b2ec0 con 0x7f3d3415df50 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d340b2d40 con 0x7f3d340b11d0 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3d340b3040 con 0x7f3d3415e330 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415df50 0x7f3d340d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415df50 0x7f3d340d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51890/0 (socket says 172.21.15.122:51890) 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 -- 172.21.15.122:0/1068521746 learned_addr learned my addr 172.21.15.122:0/1068521746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d39400640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3415e330 0x7f3d340d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d38a00640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d340b11d0 0x7f3d341530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 -- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3415e330 msgr2=0x7f3d340d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3415e330 0x7f3d340d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 -- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d340b11d0 msgr2=0x7f3d341530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d340b11d0 0x7f3d341530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 -- 172.21.15.122:0/1068521746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d341f7460 con 0x7f3d3415df50 2024-09-17T05:28:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d33e00640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415df50 0x7f3d340d00f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d2001ae50 tx=0x7f3d200a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:09.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d30c00640 1 -- 172.21.15.122:0/1068521746 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200b2040 con 0x7f3d3415df50 2024-09-17T05:28:09.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d30c00640 1 -- 172.21.15.122:0/1068521746 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d200044b0 con 0x7f3d3415df50 2024-09-17T05:28:09.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d341f7720 con 0x7f3d3415df50 2024-09-17T05:28:09.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d30c00640 1 -- 172.21.15.122:0/1068521746 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20003840 con 0x7f3d3415df50 2024-09-17T05:28:09.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.879+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d341f7ce0 con 0x7f3d3415df50 2024-09-17T05:28:09.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.883+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d340b35a0 con 0x7f3d3415df50 2024-09-17T05:28:09.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.883+0000 7f3d30c00640 1 -- 172.21.15.122:0/1068521746 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f3d200bf060 con 0x7f3d3415df50 2024-09-17T05:28:09.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.883+0000 7f3d30c00640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3d14073560 0x7f3d14075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:09.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.883+0000 7f3d30c00640 1 -- 172.21.15.122:0/1068521746 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f3d2012e0a0 con 0x7f3d3415df50 2024-09-17T05:28:09.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.883+0000 7f3d38a00640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3d14073560 0x7f3d14075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:09.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.883+0000 7f3d38a00640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3d14073560 0x7f3d14075a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3d341f4fc0 tx=0x7f3d24009110 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:09.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:09.887+0000 7f3d30c00640 1 -- 172.21.15.122:0/1068521746 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3d200164b0 con 0x7f3d3415df50 2024-09-17T05:28:10.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.055+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d340ce6d0 con 0x7f3d14073560 2024-09-17T05:28:10.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.059+0000 7f3d30c00640 1 -- 172.21.15.122:0/1068521746 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247807 (secure 0 0 0) 0x7f3d340ce6d0 con 0x7f3d14073560 2024-09-17T05:28:10.064 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:10.065 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:28:10.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3d14073560 msgr2=0x7f3d14075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:10.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3d14073560 0x7f3d14075a20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3d341f4fc0 tx=0x7f3d24009110 comp rx=0 tx=0).stop 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415df50 msgr2=0x7f3d340d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415df50 0x7f3d340d00f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d2001ae50 tx=0x7f3d200a5750 comp rx=0 tx=0).stop 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 shutdown_connections 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3d14073560 0x7f3d14075a20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3415e330 0x7f3d340d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d3415df50 0x7f3d340d00f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 --2- 172.21.15.122:0/1068521746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d340b11d0 0x7f3d341530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 >> 172.21.15.122:0/1068521746 conn(0x7f3d340d1dc0 msgr2=0x7f3d34070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 shutdown_connections 2024-09-17T05:28:10.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:10.063+0000 7f3d3b400640 1 -- 172.21.15.122:0/1068521746 wait complete. 2024-09-17T05:28:10.086 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-17T05:28:08.492320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262972,"kb_used_data":5120,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711748,"statfs":{"total":773094113280,"available":772824829952,"internally_reserved":0,"allocated":5242880,"data_stored":2791800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":263835190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003592"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387945+0000","last_change":"2024-09-17T05:28:03.387945+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387945+0000","last_undegraded":"2024-09-17T05:28:03.387945+0000","last_fullsized":"2024-09-17T05:28:03.387945+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388044+0000","last_change":"2024-09-17T05:28:03.388044+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388044+0000","last_undegraded":"2024-09-17T05:28:03.388044+0000","last_fullsized":"2024-09-17T05:28:03.388044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391128+0000","last_change":"2024-09-17T05:28:03.391128+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391128+0000","last_undegraded":"2024-09-17T05:28:03.391128+0000","last_fullsized":"2024-09-17T05:28:03.391128+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409677+0000","last_change":"2024-09-17T05:28:04.416129+0000","last_active":"2024-09-17T05:28:05.409677+0000","last_peered":"2024-09-17T05:28:05.409677+0000","last_clean":"2024-09-17T05:28:05.409677+0000","last_became_active":"2024-09-17T05:28:04.415869+0000","last_became_peered":"2024-09-17T05:28:04.415869+0000","last_unstale":"2024-09-17T05:28:05.409677+0000","last_undegraded":"2024-09-17T05:28:05.409677+0000","last_fullsized":"2024-09-17T05:28:05.409677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.172592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387704+0000","last_change":"2024-09-17T05:28:03.387704+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387704+0000","last_undegraded":"2024-09-17T05:28:03.387704+0000","last_fullsized":"2024-09-17T05:28:03.387704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.392926+0000","last_change":"2024-09-17T05:28:03.392926+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.392926+0000","last_undegraded":"2024-09-17T05:28:03.392926+0000","last_fullsized":"2024-09-17T05:28:03.392926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398593+0000","last_change":"2024-09-17T05:28:03.398593+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398593+0000","last_undegraded":"2024-09-17T05:28:03.398593+0000","last_fullsized":"2024-09-17T05:28:03.398593+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.408875+0000","last_change":"2024-09-17T05:28:04.421456+0000","last_active":"2024-09-17T05:28:05.408875+0000","last_peered":"2024-09-17T05:28:05.408875+0000","last_clean":"2024-09-17T05:28:05.408875+0000","last_became_active":"2024-09-17T05:28:04.421034+0000","last_became_peered":"2024-09-17T05:28:04.421034+0000","last_unstale":"2024-09-17T05:28:05.408875+0000","last_undegraded":"2024-09-17T05:28:05.408875+0000","last_fullsized":"2024-09-17T05:28:05.408875+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:41.119416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398634+0000","last_change":"2024-09-17T05:28:03.398634+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398634+0000","last_undegraded":"2024-09-17T05:28:03.398634+0000","last_fullsized":"2024-09-17T05:28:03.398634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393796+0000","last_change":"2024-09-17T05:28:03.393796+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393796+0000","last_undegraded":"2024-09-17T05:28:03.393796+0000","last_fullsized":"2024-09-17T05:28:03.393796+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409905+0000","last_change":"2024-09-17T05:28:04.419275+0000","last_active":"2024-09-17T05:28:05.409905+0000","last_peered":"2024-09-17T05:28:05.409905+0000","last_clean":"2024-09-17T05:28:05.409905+0000","last_became_active":"2024-09-17T05:28:04.418991+0000","last_became_peered":"2024-09-17T05:28:04.418991+0000","last_unstale":"2024-09-17T05:28:05.409905+0000","last_undegraded":"2024-09-17T05:28:05.409905+0000","last_fullsized":"2024-09-17T05:28:05.409905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:23.031211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389118+0000","last_change":"2024-09-17T05:28:03.389118+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389118+0000","last_undegraded":"2024-09-17T05:28:03.389118+0000","last_fullsized":"2024-09-17T05:28:03.389118+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395047+0000","last_change":"2024-09-17T05:28:03.395047+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395047+0000","last_undegraded":"2024-09-17T05:28:03.395047+0000","last_fullsized":"2024-09-17T05:28:03.395047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409281+0000","last_change":"2024-09-17T05:28:04.411688+0000","last_active":"2024-09-17T05:28:05.409281+0000","last_peered":"2024-09-17T05:28:05.409281+0000","last_clean":"2024-09-17T05:28:05.409281+0000","last_became_active":"2024-09-17T05:28:04.411403+0000","last_became_peered":"2024-09-17T05:28:04.411403+0000","last_unstale":"2024-09-17T05:28:05.409281+0000","last_undegraded":"2024-09-17T05:28:05.409281+0000","last_fullsized":"2024-09-17T05:28:05.409281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:17.638839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409054+0000","last_change":"2024-09-17T05:28:04.423517+0000","last_active":"2024-09-17T05:28:05.409054+0000","last_peered":"2024-09-17T05:28:05.409054+0000","last_clean":"2024-09-17T05:28:05.409054+0000","last_became_active":"2024-09-17T05:28:04.423243+0000","last_became_peered":"2024-09-17T05:28:04.423243+0000","last_unstale":"2024-09-17T05:28:05.409054+0000","last_undegraded":"2024-09-17T05:28:05.409054+0000","last_fullsized":"2024-09-17T05:28:05.409054+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:24.878942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386968+0000","last_change":"2024-09-17T05:28:03.386968+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386968+0000","last_undegraded":"2024-09-17T05:28:03.386968+0000","last_fullsized":"2024-09-17T05:28:03.386968+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387639+0000","last_change":"2024-09-17T05:28:03.387639+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387639+0000","last_undegraded":"2024-09-17T05:28:03.387639+0000","last_fullsized":"2024-09-17T05:28:03.387639+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391815+0000","last_change":"2024-09-17T05:28:03.391815+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391815+0000","last_undegraded":"2024-09-17T05:28:03.391815+0000","last_fullsized":"2024-09-17T05:28:03.391815+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387562+0000","last_change":"2024-09-17T05:28:03.387562+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387562+0000","last_undegraded":"2024-09-17T05:28:03.387562+0000","last_fullsized":"2024-09-17T05:28:03.387562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395327+0000","last_change":"2024-09-17T05:28:03.395327+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395327+0000","last_undegraded":"2024-09-17T05:28:03.395327+0000","last_fullsized":"2024-09-17T05:28:03.395327+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409823+0000","last_change":"2024-09-17T05:28:04.421922+0000","last_active":"2024-09-17T05:28:05.409823+0000","last_peered":"2024-09-17T05:28:05.409823+0000","last_clean":"2024-09-17T05:28:05.409823+0000","last_became_active":"2024-09-17T05:28:04.414061+0000","last_became_peered":"2024-09-17T05:28:04.414061+0000","last_unstale":"2024-09-17T05:28:05.409823+0000","last_undegraded":"2024-09-17T05:28:05.409823+0000","last_fullsized":"2024-09-17T05:28:05.409823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:35.698197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389470+0000","last_change":"2024-09-17T05:28:03.389470+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389470+0000","last_undegraded":"2024-09-17T05:28:03.389470+0000","last_fullsized":"2024-09-17T05:28:03.389470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393313+0000","last_change":"2024-09-17T05:28:03.393313+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393313+0000","last_undegraded":"2024-09-17T05:28:03.393313+0000","last_fullsized":"2024-09-17T05:28:03.393313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387264+0000","last_change":"2024-09-17T05:28:03.387264+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387264+0000","last_undegraded":"2024-09-17T05:28:03.387264+0000","last_fullsized":"2024-09-17T05:28:03.387264+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386893+0000","last_change":"2024-09-17T05:28:03.386893+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386893+0000","last_undegraded":"2024-09-17T05:28:03.386893+0000","last_fullsized":"2024-09-17T05:28:03.386893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410396+0000","last_change":"2024-09-17T05:28:04.419337+0000","last_active":"2024-09-17T05:28:05.410396+0000","last_peered":"2024-09-17T05:28:05.410396+0000","last_clean":"2024-09-17T05:28:05.410396+0000","last_became_active":"2024-09-17T05:28:04.419066+0000","last_became_peered":"2024-09-17T05:28:04.419066+0000","last_unstale":"2024-09-17T05:28:05.410396+0000","last_undegraded":"2024-09-17T05:28:05.410396+0000","last_fullsized":"2024-09-17T05:28:05.410396+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:28.650511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395190+0000","last_change":"2024-09-17T05:28:03.395190+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395190+0000","last_undegraded":"2024-09-17T05:28:03.395190+0000","last_fullsized":"2024-09-17T05:28:03.395190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393821+0000","last_change":"2024-09-17T05:28:03.393821+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393821+0000","last_undegraded":"2024-09-17T05:28:03.393821+0000","last_fullsized":"2024-09-17T05:28:03.393821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386847+0000","last_change":"2024-09-17T05:28:03.386847+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386847+0000","last_undegraded":"2024-09-17T05:28:03.386847+0000","last_fullsized":"2024-09-17T05:28:03.386847+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387350+0000","last_change":"2024-09-17T05:28:03.387350+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387350+0000","last_undegraded":"2024-09-17T05:28:03.387350+0000","last_fullsized":"2024-09-17T05:28:03.387350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394182+0000","last_change":"2024-09-17T05:28:03.394182+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394182+0000","last_undegraded":"2024-09-17T05:28:03.394182+0000","last_fullsized":"2024-09-17T05:28:03.394182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409026+0000","last_change":"2024-09-17T05:28:04.414079+0000","last_active":"2024-09-17T05:28:05.409026+0000","last_peered":"2024-09-17T05:28:05.409026+0000","last_clean":"2024-09-17T05:28:05.409026+0000","last_became_active":"2024-09-17T05:28:04.413733+0000","last_became_peered":"2024-09-17T05:28:04.413733+0000","last_unstale":"2024-09-17T05:28:05.409026+0000","last_undegraded":"2024-09-17T05:28:05.409026+0000","last_fullsized":"2024-09-17T05:28:05.409026+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:15.069773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389389+0000","last_change":"2024-09-17T05:28:03.389389+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389389+0000","last_undegraded":"2024-09-17T05:28:03.389389+0000","last_fullsized":"2024-09-17T05:28:03.389389+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411758+0000","last_change":"2024-09-17T05:28:03.408603+0000","last_active":"2024-09-17T05:28:03.411758+0000","last_peered":"2024-09-17T05:28:03.411758+0000","last_clean":"2024-09-17T05:28:03.411758+0000","last_became_active":"2024-09-17T05:28:03.408354+0000","last_became_peered":"2024-09-17T05:28:03.408354+0000","last_unstale":"2024-09-17T05:28:03.411758+0000","last_undegraded":"2024-09-17T05:28:03.411758+0000","last_fullsized":"2024-09-17T05:28:03.411758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:20.506564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.386337+0000","last_change":"2024-09-17T05:28:03.386337+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.386337+0000","last_undegraded":"2024-09-17T05:28:03.386337+0000","last_fullsized":"2024-09-17T05:28:03.386337+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408180+0000","last_change":"2024-09-17T05:28:03.403643+0000","last_active":"2024-09-17T05:28:03.408180+0000","last_peered":"2024-09-17T05:28:03.408180+0000","last_clean":"2024-09-17T05:28:03.408180+0000","last_became_active":"2024-09-17T05:28:03.403353+0000","last_became_peered":"2024-09-17T05:28:03.403353+0000","last_unstale":"2024-09-17T05:28:03.408180+0000","last_undegraded":"2024-09-17T05:28:03.408180+0000","last_fullsized":"2024-09-17T05:28:03.408180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:30.436401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387803+0000","last_change":"2024-09-17T05:28:03.387803+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387803+0000","last_undegraded":"2024-09-17T05:28:03.387803+0000","last_fullsized":"2024-09-17T05:28:03.387803+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411701+0000","last_change":"2024-09-17T05:28:03.407955+0000","last_active":"2024-09-17T05:28:03.411701+0000","last_peered":"2024-09-17T05:28:03.411701+0000","last_clean":"2024-09-17T05:28:03.411701+0000","last_became_active":"2024-09-17T05:28:03.407597+0000","last_became_peered":"2024-09-17T05:28:03.407597+0000","last_unstale":"2024-09-17T05:28:03.411701+0000","last_undegraded":"2024-09-17T05:28:03.411701+0000","last_fullsized":"2024-09-17T05:28:03.411701+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:31.638210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410107+0000","last_change":"2024-09-17T05:28:04.411454+0000","last_active":"2024-09-17T05:28:05.410107+0000","last_peered":"2024-09-17T05:28:05.410107+0000","last_clean":"2024-09-17T05:28:05.410107+0000","last_became_active":"2024-09-17T05:28:04.411144+0000","last_became_peered":"2024-09-17T05:28:04.411144+0000","last_unstale":"2024-09-17T05:28:05.410107+0000","last_undegraded":"2024-09-17T05:28:05.410107+0000","last_fullsized":"2024-09-17T05:28:05.410107+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:10.404469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410169+0000","last_change":"2024-09-17T05:28:03.406275+0000","last_active":"2024-09-17T05:28:03.410169+0000","last_peered":"2024-09-17T05:28:03.410169+0000","last_clean":"2024-09-17T05:28:03.410169+0000","last_became_active":"2024-09-17T05:28:03.405506+0000","last_became_peered":"2024-09-17T05:28:03.405506+0000","last_unstale":"2024-09-17T05:28:03.410169+0000","last_undegraded":"2024-09-17T05:28:03.410169+0000","last_fullsized":"2024-09-17T05:28:03.410169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:46.627002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.395704+0000","last_change":"2024-09-17T05:28:03.395704+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.395704+0000","last_undegraded":"2024-09-17T05:28:03.395704+0000","last_fullsized":"2024-09-17T05:28:03.395704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410230+0000","last_change":"2024-09-17T05:28:03.406212+0000","last_active":"2024-09-17T05:28:03.410230+0000","last_peered":"2024-09-17T05:28:03.410230+0000","last_clean":"2024-09-17T05:28:03.410230+0000","last_became_active":"2024-09-17T05:28:03.405428+0000","last_became_peered":"2024-09-17T05:28:03.405428+0000","last_unstale":"2024-09-17T05:28:03.410230+0000","last_undegraded":"2024-09-17T05:28:03.410230+0000","last_fullsized":"2024-09-17T05:28:03.410230+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:08.586828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394459+0000","last_change":"2024-09-17T05:28:03.394459+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394459+0000","last_undegraded":"2024-09-17T05:28:03.394459+0000","last_fullsized":"2024-09-17T05:28:03.394459+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410102+0000","last_change":"2024-09-17T05:28:03.405006+0000","last_active":"2024-09-17T05:28:03.410102+0000","last_peered":"2024-09-17T05:28:03.410102+0000","last_clean":"2024-09-17T05:28:03.410102+0000","last_became_active":"2024-09-17T05:28:03.404741+0000","last_became_peered":"2024-09-17T05:28:03.404741+0000","last_unstale":"2024-09-17T05:28:03.410102+0000","last_undegraded":"2024-09-17T05:28:03.410102+0000","last_fullsized":"2024-09-17T05:28:03.410102+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:23.814938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393871+0000","last_change":"2024-09-17T05:28:03.393871+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393871+0000","last_undegraded":"2024-09-17T05:28:03.393871+0000","last_fullsized":"2024-09-17T05:28:03.393871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.399582+0000","last_change":"2024-09-17T05:28:03.397856+0000","last_active":"2024-09-17T05:28:03.399582+0000","last_peered":"2024-09-17T05:28:03.399582+0000","last_clean":"2024-09-17T05:28:03.399582+0000","last_became_active":"2024-09-17T05:28:03.397609+0000","last_became_peered":"2024-09-17T05:28:03.397609+0000","last_unstale":"2024-09-17T05:28:03.399582+0000","last_undegraded":"2024-09-17T05:28:03.399582+0000","last_fullsized":"2024-09-17T05:28:03.399582+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:40.677984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393780+0000","last_change":"2024-09-17T05:28:03.393780+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393780+0000","last_undegraded":"2024-09-17T05:28:03.393780+0000","last_fullsized":"2024-09-17T05:28:03.393780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.414117+0000","last_change":"2024-09-17T05:28:03.410366+0000","last_active":"2024-09-17T05:28:03.414117+0000","last_peered":"2024-09-17T05:28:03.414117+0000","last_clean":"2024-09-17T05:28:03.414117+0000","last_became_active":"2024-09-17T05:28:03.409222+0000","last_became_peered":"2024-09-17T05:28:03.409222+0000","last_unstale":"2024-09-17T05:28:03.414117+0000","last_undegraded":"2024-09-17T05:28:03.414117+0000","last_fullsized":"2024-09-17T05:28:03.414117+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:11.005389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.391108+0000","last_change":"2024-09-17T05:28:03.391108+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.391108+0000","last_undegraded":"2024-09-17T05:28:03.391108+0000","last_fullsized":"2024-09-17T05:28:03.391108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.403575+0000","last_change":"2024-09-17T05:28:03.400471+0000","last_active":"2024-09-17T05:28:03.403575+0000","last_peered":"2024-09-17T05:28:03.403575+0000","last_clean":"2024-09-17T05:28:03.403575+0000","last_became_active":"2024-09-17T05:28:03.400202+0000","last_became_peered":"2024-09-17T05:28:03.400202+0000","last_unstale":"2024-09-17T05:28:03.403575+0000","last_undegraded":"2024-09-17T05:28:03.403575+0000","last_fullsized":"2024-09-17T05:28:03.403575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:58.648337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.390844+0000","last_change":"2024-09-17T05:28:03.390844+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.390844+0000","last_undegraded":"2024-09-17T05:28:03.390844+0000","last_fullsized":"2024-09-17T05:28:03.390844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.409027+0000","last_change":"2024-09-17T05:28:03.401227+0000","last_active":"2024-09-17T05:28:03.409027+0000","last_peered":"2024-09-17T05:28:03.409027+0000","last_clean":"2024-09-17T05:28:03.409027+0000","last_became_active":"2024-09-17T05:28:03.400688+0000","last_became_peered":"2024-09-17T05:28:03.400688+0000","last_unstale":"2024-09-17T05:28:03.409027+0000","last_undegraded":"2024-09-17T05:28:03.409027+0000","last_fullsized":"2024-09-17T05:28:03.409027+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:45.856441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409693+0000","last_change":"2024-09-17T05:28:04.413706+0000","last_active":"2024-09-17T05:28:05.409693+0000","last_peered":"2024-09-17T05:28:05.409693+0000","last_clean":"2024-09-17T05:28:05.409693+0000","last_became_active":"2024-09-17T05:28:04.413217+0000","last_became_peered":"2024-09-17T05:28:04.413217+0000","last_unstale":"2024-09-17T05:28:05.409693+0000","last_undegraded":"2024-09-17T05:28:05.409693+0000","last_fullsized":"2024-09-17T05:28:05.409693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:33.217682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407650+0000","last_change":"2024-09-17T05:28:03.401672+0000","last_active":"2024-09-17T05:28:03.407650+0000","last_peered":"2024-09-17T05:28:03.407650+0000","last_clean":"2024-09-17T05:28:03.407650+0000","last_became_active":"2024-09-17T05:28:03.401380+0000","last_became_peered":"2024-09-17T05:28:03.401380+0000","last_unstale":"2024-09-17T05:28:03.407650+0000","last_undegraded":"2024-09-17T05:28:03.407650+0000","last_fullsized":"2024-09-17T05:28:03.407650+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:57.580288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394819+0000","last_change":"2024-09-17T05:28:03.394819+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394819+0000","last_undegraded":"2024-09-17T05:28:03.394819+0000","last_fullsized":"2024-09-17T05:28:03.394819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411516+0000","last_change":"2024-09-17T05:28:03.408491+0000","last_active":"2024-09-17T05:28:03.411516+0000","last_peered":"2024-09-17T05:28:03.411516+0000","last_clean":"2024-09-17T05:28:03.411516+0000","last_became_active":"2024-09-17T05:28:03.407793+0000","last_became_peered":"2024-09-17T05:28:03.407793+0000","last_unstale":"2024-09-17T05:28:03.411516+0000","last_undegraded":"2024-09-17T05:28:03.411516+0000","last_fullsized":"2024-09-17T05:28:03.411516+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.426226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389843+0000","last_change":"2024-09-17T05:28:03.389843+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389843+0000","last_undegraded":"2024-09-17T05:28:03.389843+0000","last_fullsized":"2024-09-17T05:28:03.389843+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411542+0000","last_change":"2024-09-17T05:28:03.406639+0000","last_active":"2024-09-17T05:28:03.411542+0000","last_peered":"2024-09-17T05:28:03.411542+0000","last_clean":"2024-09-17T05:28:03.411542+0000","last_became_active":"2024-09-17T05:28:03.406360+0000","last_became_peered":"2024-09-17T05:28:03.406360+0000","last_unstale":"2024-09-17T05:28:03.411542+0000","last_undegraded":"2024-09-17T05:28:03.411542+0000","last_fullsized":"2024-09-17T05:28:03.411542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:17.531567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.404093+0000","last_change":"2024-09-17T05:28:03.401363+0000","last_active":"2024-09-17T05:28:03.404093+0000","last_peered":"2024-09-17T05:28:03.404093+0000","last_clean":"2024-09-17T05:28:03.404093+0000","last_became_active":"2024-09-17T05:28:03.401110+0000","last_became_peered":"2024-09-17T05:28:03.401110+0000","last_unstale":"2024-09-17T05:28:03.404093+0000","last_undegraded":"2024-09-17T05:28:03.404093+0000","last_fullsized":"2024-09-17T05:28:03.404093+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:24.220817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.413897+0000","last_change":"2024-09-17T05:28:03.409353+0000","last_active":"2024-09-17T05:28:03.413897+0000","last_peered":"2024-09-17T05:28:03.413897+0000","last_clean":"2024-09-17T05:28:03.413897+0000","last_became_active":"2024-09-17T05:28:03.409087+0000","last_became_peered":"2024-09-17T05:28:03.409087+0000","last_unstale":"2024-09-17T05:28:03.413897+0000","last_undegraded":"2024-09-17T05:28:03.413897+0000","last_fullsized":"2024-09-17T05:28:03.413897+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:37.361308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388677+0000","last_change":"2024-09-17T05:28:03.388677+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388677+0000","last_undegraded":"2024-09-17T05:28:03.388677+0000","last_fullsized":"2024-09-17T05:28:03.388677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.409986+0000","last_change":"2024-09-17T05:28:03.403774+0000","last_active":"2024-09-17T05:28:03.409986+0000","last_peered":"2024-09-17T05:28:03.409986+0000","last_clean":"2024-09-17T05:28:03.409986+0000","last_became_active":"2024-09-17T05:28:03.403425+0000","last_became_peered":"2024-09-17T05:28:03.403425+0000","last_unstale":"2024-09-17T05:28:03.409986+0000","last_undegraded":"2024-09-17T05:28:03.409986+0000","last_fullsized":"2024-09-17T05:28:03.409986+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:08.155865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388720+0000","last_change":"2024-09-17T05:28:03.388720+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388720+0000","last_undegraded":"2024-09-17T05:28:03.388720+0000","last_fullsized":"2024-09-17T05:28:03.388720+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410531+0000","last_change":"2024-09-17T05:28:03.409845+0000","last_active":"2024-09-17T05:28:05.410531+0000","last_peered":"2024-09-17T05:28:05.410531+0000","last_clean":"2024-09-17T05:28:05.410531+0000","last_became_active":"2024-09-17T05:28:03.409519+0000","last_became_peered":"2024-09-17T05:28:03.409519+0000","last_unstale":"2024-09-17T05:28:05.410531+0000","last_undegraded":"2024-09-17T05:28:05.410531+0000","last_fullsized":"2024-09-17T05:28:05.410531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:04.018422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.388693+0000","last_change":"2024-09-17T05:28:03.388693+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.388693+0000","last_undegraded":"2024-09-17T05:28:03.388693+0000","last_fullsized":"2024-09-17T05:28:03.388693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412689+0000","last_change":"2024-09-17T05:28:03.408640+0000","last_active":"2024-09-17T05:28:03.412689+0000","last_peered":"2024-09-17T05:28:03.412689+0000","last_clean":"2024-09-17T05:28:03.412689+0000","last_became_active":"2024-09-17T05:28:03.408371+0000","last_became_peered":"2024-09-17T05:28:03.408371+0000","last_unstale":"2024-09-17T05:28:03.412689+0000","last_undegraded":"2024-09-17T05:28:03.412689+0000","last_fullsized":"2024-09-17T05:28:03.412689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:11:51.246256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387320+0000","last_change":"2024-09-17T05:28:03.387320+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387320+0000","last_undegraded":"2024-09-17T05:28:03.387320+0000","last_fullsized":"2024-09-17T05:28:03.387320+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.403739+0000","last_change":"2024-09-17T05:28:03.398552+0000","last_active":"2024-09-17T05:28:03.403739+0000","last_peered":"2024-09-17T05:28:03.403739+0000","last_clean":"2024-09-17T05:28:03.403739+0000","last_became_active":"2024-09-17T05:28:03.398282+0000","last_became_peered":"2024-09-17T05:28:03.398282+0000","last_unstale":"2024-09-17T05:28:03.403739+0000","last_undegraded":"2024-09-17T05:28:03.403739+0000","last_fullsized":"2024-09-17T05:28:03.403739+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:54.593079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.390768+0000","last_change":"2024-09-17T05:28:03.390768+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.390768+0000","last_undegraded":"2024-09-17T05:28:03.390768+0000","last_fullsized":"2024-09-17T05:28:03.390768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.410212+0000","last_change":"2024-09-17T05:28:03.404997+0000","last_active":"2024-09-17T05:28:03.410212+0000","last_peered":"2024-09-17T05:28:03.410212+0000","last_clean":"2024-09-17T05:28:03.410212+0000","last_became_active":"2024-09-17T05:28:03.404512+0000","last_became_peered":"2024-09-17T05:28:03.404512+0000","last_unstale":"2024-09-17T05:28:03.410212+0000","last_undegraded":"2024-09-17T05:28:03.410212+0000","last_fullsized":"2024-09-17T05:28:03.410212+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:07.584572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.398333+0000","last_change":"2024-09-17T05:28:03.398333+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.398333+0000","last_undegraded":"2024-09-17T05:28:03.398333+0000","last_fullsized":"2024-09-17T05:28:03.398333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408284+0000","last_change":"2024-09-17T05:28:03.400873+0000","last_active":"2024-09-17T05:28:03.408284+0000","last_peered":"2024-09-17T05:28:03.408284+0000","last_clean":"2024-09-17T05:28:03.408284+0000","last_became_active":"2024-09-17T05:28:03.400549+0000","last_became_peered":"2024-09-17T05:28:03.400549+0000","last_unstale":"2024-09-17T05:28:03.408284+0000","last_undegraded":"2024-09-17T05:28:03.408284+0000","last_fullsized":"2024-09-17T05:28:03.408284+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:36:31.777509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394647+0000","last_change":"2024-09-17T05:28:03.394647+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394647+0000","last_undegraded":"2024-09-17T05:28:03.394647+0000","last_fullsized":"2024-09-17T05:28:03.394647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.398760+0000","last_change":"2024-09-17T05:28:03.395798+0000","last_active":"2024-09-17T05:28:03.398760+0000","last_peered":"2024-09-17T05:28:03.398760+0000","last_clean":"2024-09-17T05:28:03.398760+0000","last_became_active":"2024-09-17T05:28:03.395543+0000","last_became_peered":"2024-09-17T05:28:03.395543+0000","last_unstale":"2024-09-17T05:28:03.398760+0000","last_undegraded":"2024-09-17T05:28:03.398760+0000","last_fullsized":"2024-09-17T05:28:03.398760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:37.361308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409491+0000","last_change":"2024-09-17T05:28:04.423847+0000","last_active":"2024-09-17T05:28:05.409491+0000","last_peered":"2024-09-17T05:28:05.409491+0000","last_clean":"2024-09-17T05:28:05.409491+0000","last_became_active":"2024-09-17T05:28:04.423569+0000","last_became_peered":"2024-09-17T05:28:04.423569+0000","last_unstale":"2024-09-17T05:28:05.409491+0000","last_undegraded":"2024-09-17T05:28:05.409491+0000","last_fullsized":"2024-09-17T05:28:05.409491+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:58.595615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.408407+0000","last_change":"2024-09-17T05:28:03.400992+0000","last_active":"2024-09-17T05:28:03.408407+0000","last_peered":"2024-09-17T05:28:03.408407+0000","last_clean":"2024-09-17T05:28:03.408407+0000","last_became_active":"2024-09-17T05:28:03.400690+0000","last_became_peered":"2024-09-17T05:28:03.400690+0000","last_unstale":"2024-09-17T05:28:03.408407+0000","last_undegraded":"2024-09-17T05:28:03.408407+0000","last_fullsized":"2024-09-17T05:28:03.408407+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:49.611905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409358+0000","last_change":"2024-09-17T05:28:04.426697+0000","last_active":"2024-09-17T05:28:05.409358+0000","last_peered":"2024-09-17T05:28:05.409358+0000","last_clean":"2024-09-17T05:28:05.409358+0000","last_became_active":"2024-09-17T05:28:04.426419+0000","last_became_peered":"2024-09-17T05:28:04.426419+0000","last_unstale":"2024-09-17T05:28:05.409358+0000","last_undegraded":"2024-09-17T05:28:05.409358+0000","last_fullsized":"2024-09-17T05:28:05.409358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.432704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412599+0000","last_change":"2024-09-17T05:28:03.407318+0000","last_active":"2024-09-17T05:28:03.412599+0000","last_peered":"2024-09-17T05:28:03.412599+0000","last_clean":"2024-09-17T05:28:03.412599+0000","last_became_active":"2024-09-17T05:28:03.407059+0000","last_became_peered":"2024-09-17T05:28:03.407059+0000","last_unstale":"2024-09-17T05:28:03.412599+0000","last_undegraded":"2024-09-17T05:28:03.412599+0000","last_fullsized":"2024-09-17T05:28:03.412599+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:15.545411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.393056+0000","last_change":"2024-09-17T05:28:03.393056+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.393056+0000","last_undegraded":"2024-09-17T05:28:03.393056+0000","last_fullsized":"2024-09-17T05:28:03.393056+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410708+0000","last_change":"2024-09-17T05:28:03.408701+0000","last_active":"2024-09-17T05:28:05.410708+0000","last_peered":"2024-09-17T05:28:05.410708+0000","last_clean":"2024-09-17T05:28:05.410708+0000","last_became_active":"2024-09-17T05:28:03.408379+0000","last_became_peered":"2024-09-17T05:28:03.408379+0000","last_unstale":"2024-09-17T05:28:05.410708+0000","last_undegraded":"2024-09-17T05:28:05.410708+0000","last_fullsized":"2024-09-17T05:28:05.410708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:24.330713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.394936+0000","last_change":"2024-09-17T05:28:03.394936+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.394936+0000","last_undegraded":"2024-09-17T05:28:03.394936+0000","last_fullsized":"2024-09-17T05:28:03.394936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.402356+0000","last_change":"2024-09-17T05:28:03.397060+0000","last_active":"2024-09-17T05:28:03.402356+0000","last_peered":"2024-09-17T05:28:03.402356+0000","last_clean":"2024-09-17T05:28:03.402356+0000","last_became_active":"2024-09-17T05:28:03.396798+0000","last_became_peered":"2024-09-17T05:28:03.396798+0000","last_unstale":"2024-09-17T05:28:03.402356+0000","last_undegraded":"2024-09-17T05:28:03.402356+0000","last_fullsized":"2024-09-17T05:28:03.402356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.747610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.392714+0000","last_change":"2024-09-17T05:28:03.392714+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.392714+0000","last_undegraded":"2024-09-17T05:28:03.392714+0000","last_fullsized":"2024-09-17T05:28:03.392714+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407241+0000","last_change":"2024-09-17T05:28:03.402041+0000","last_active":"2024-09-17T05:28:03.407241+0000","last_peered":"2024-09-17T05:28:03.407241+0000","last_clean":"2024-09-17T05:28:03.407241+0000","last_became_active":"2024-09-17T05:28:03.401757+0000","last_became_peered":"2024-09-17T05:28:03.401757+0000","last_unstale":"2024-09-17T05:28:03.407241+0000","last_undegraded":"2024-09-17T05:28:03.407241+0000","last_fullsized":"2024-09-17T05:28:03.407241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:39.617757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.397895+0000","last_change":"2024-09-17T05:28:03.397895+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.397895+0000","last_undegraded":"2024-09-17T05:28:03.397895+0000","last_fullsized":"2024-09-17T05:28:03.397895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407808+0000","last_change":"2024-09-17T05:28:03.406099+0000","last_active":"2024-09-17T05:28:03.407808+0000","last_peered":"2024-09-17T05:28:03.407808+0000","last_clean":"2024-09-17T05:28:03.407808+0000","last_became_active":"2024-09-17T05:28:03.405763+0000","last_became_peered":"2024-09-17T05:28:03.405763+0000","last_unstale":"2024-09-17T05:28:03.407808+0000","last_undegraded":"2024-09-17T05:28:03.407808+0000","last_fullsized":"2024-09-17T05:28:03.407808+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:57.580288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409499+0000","last_change":"2024-09-17T05:28:04.405966+0000","last_active":"2024-09-17T05:28:05.409499+0000","last_peered":"2024-09-17T05:28:05.409499+0000","last_clean":"2024-09-17T05:28:05.409499+0000","last_became_active":"2024-09-17T05:28:04.405723+0000","last_became_peered":"2024-09-17T05:28:04.405723+0000","last_unstale":"2024-09-17T05:28:05.409499+0000","last_undegraded":"2024-09-17T05:28:05.409499+0000","last_fullsized":"2024-09-17T05:28:05.409499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:09:42.528631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.405322+0000","last_change":"2024-09-17T05:28:03.403069+0000","last_active":"2024-09-17T05:28:03.405322+0000","last_peered":"2024-09-17T05:28:03.405322+0000","last_clean":"2024-09-17T05:28:03.405322+0000","last_became_active":"2024-09-17T05:28:03.402652+0000","last_became_peered":"2024-09-17T05:28:03.402652+0000","last_unstale":"2024-09-17T05:28:03.405322+0000","last_undegraded":"2024-09-17T05:28:03.405322+0000","last_fullsized":"2024-09-17T05:28:03.405322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:42.875418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.389143+0000","last_change":"2024-09-17T05:28:03.389143+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.389143+0000","last_undegraded":"2024-09-17T05:28:03.389143+0000","last_fullsized":"2024-09-17T05:28:03.389143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.411465+0000","last_change":"2024-09-17T05:28:03.407068+0000","last_active":"2024-09-17T05:28:03.411465+0000","last_peered":"2024-09-17T05:28:03.411465+0000","last_clean":"2024-09-17T05:28:03.411465+0000","last_became_active":"2024-09-17T05:28:03.406674+0000","last_became_peered":"2024-09-17T05:28:03.406674+0000","last_unstale":"2024-09-17T05:28:03.411465+0000","last_undegraded":"2024-09-17T05:28:03.411465+0000","last_fullsized":"2024-09-17T05:28:03.411465+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:54.447845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-09-17T05:28:03.387799+0000","last_change":"2024-09-17T05:28:03.387799+0000","last_active":"2024-09-17T05:28:03.373314+0000","last_peered":"2024-09-17T05:28:03.373314+0000","last_clean":"2024-09-17T05:28:03.373314+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:28:03.387799+0000","last_undegraded":"2024-09-17T05:28:03.387799+0000","last_fullsized":"2024-09-17T05:28:03.387799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.407397+0000","last_change":"2024-09-17T05:28:03.402396+0000","last_active":"2024-09-17T05:28:03.407397+0000","last_peered":"2024-09-17T05:28:03.407397+0000","last_clean":"2024-09-17T05:28:03.407397+0000","last_became_active":"2024-09-17T05:28:03.402089+0000","last_became_peered":"2024-09-17T05:28:03.402089+0000","last_unstale":"2024-09-17T05:28:03.407397+0000","last_undegraded":"2024-09-17T05:28:03.407397+0000","last_fullsized":"2024-09-17T05:28:03.407397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:31.994460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410638+0000","last_change":"2024-09-17T05:28:04.407978+0000","last_active":"2024-09-17T05:28:05.410638+0000","last_peered":"2024-09-17T05:28:05.410638+0000","last_clean":"2024-09-17T05:28:05.410638+0000","last_became_active":"2024-09-17T05:28:04.407717+0000","last_became_peered":"2024-09-17T05:28:04.407717+0000","last_unstale":"2024-09-17T05:28:05.410638+0000","last_undegraded":"2024-09-17T05:28:05.410638+0000","last_fullsized":"2024-09-17T05:28:05.410638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:52.263630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.412718+0000","last_change":"2024-09-17T05:28:03.408678+0000","last_active":"2024-09-17T05:28:03.412718+0000","last_peered":"2024-09-17T05:28:03.412718+0000","last_clean":"2024-09-17T05:28:03.412718+0000","last_became_active":"2024-09-17T05:28:03.408375+0000","last_became_peered":"2024-09-17T05:28:03.408375+0000","last_unstale":"2024-09-17T05:28:03.412718+0000","last_undegraded":"2024-09-17T05:28:03.412718+0000","last_fullsized":"2024-09-17T05:28:03.412718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:10.377520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":153,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T05:28:03.393074+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:28:03.393074+0000","last_peered":"2024-09-17T05:28:03.393074+0000","last_clean":"2024-09-17T05:28:03.393074+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:28:03.393074+0000","last_undegraded":"2024-09-17T05:28:03.393074+0000","last_fullsized":"2024-09-17T05:28:03.393074+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":195824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":792467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":507904,"data_stored":202099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1130496,"data_stored":805564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:13.091 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:28:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 -- 172.21.15.122:0/4172868705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 msgr2=0x7f6a180cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 --2- 172.21.15.122:0/4172868705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 0x7f6a180cccd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009c80 tx=0x7f6a1001cce0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 -- 172.21.15.122:0/4172868705 shutdown_connections 2024-09-17T05:28:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 --2- 172.21.15.122:0/4172868705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 0x7f6a180cccd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 -- 172.21.15.122:0/4172868705 >> 172.21.15.122:0/4172868705 conn(0x7f6a180d0c90 msgr2=0x7f6a180d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 -- 172.21.15.122:0/4172868705 shutdown_connections 2024-09-17T05:28:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 -- 172.21.15.122:0/4172868705 wait complete. 2024-09-17T05:28:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 Processor -- start 2024-09-17T05:28:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.243+0000 7f6a20600640 1 -- start start 2024-09-17T05:28:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 0x7f6a180afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a18158240 0x7f6a180c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a18158620 0x7f6a180c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180ce9e0 con 0x7f6a180cc8c0 2024-09-17T05:28:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180ce860 con 0x7f6a18158620 2024-09-17T05:28:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6a180ceb60 con 0x7f6a18158240 2024-09-17T05:28:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 0x7f6a180afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 0x7f6a180afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60278/0 (socket says 172.21.15.122:60278) 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 -- 172.21.15.122:0/540805529 learned_addr learned my addr 172.21.15.122:0/540805529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1d200640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a18158240 0x7f6a180c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1e600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a18158620 0x7f6a180c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 -- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a18158240 msgr2=0x7f6a180c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a18158240 0x7f6a180c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 -- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a18158620 msgr2=0x7f6a180c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a18158620 0x7f6a180c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 -- 172.21.15.122:0/540805529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a180c74f0 con 0x7f6a180cc8c0 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1e600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a18158620 0x7f6a180c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1d200640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a18158240 0x7f6a180c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a1dc00640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 0x7f6a180afa30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00c450 tx=0x7f6a0c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a15600640 1 -- 172.21.15.122:0/540805529 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c012830 con 0x7f6a180cc8c0 2024-09-17T05:28:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a15600640 1 -- 172.21.15.122:0/540805529 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a0c017070 con 0x7f6a180cc8c0 2024-09-17T05:28:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a15600640 1 -- 172.21.15.122:0/540805529 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c012d80 con 0x7f6a180cc8c0 2024-09-17T05:28:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a180c77d0 con 0x7f6a180cc8c0 2024-09-17T05:28:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a180c4aa0 con 0x7f6a180cc8c0 2024-09-17T05:28:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.247+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a180cf090 con 0x7f6a180cc8c0 2024-09-17T05:28:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.251+0000 7f6a15600640 1 -- 172.21.15.122:0/540805529 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6a0c01c460 con 0x7f6a180cc8c0 2024-09-17T05:28:13.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.251+0000 7f6a15600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69f8073560 0x7f69f8075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:13.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.251+0000 7f6a15600640 1 -- 172.21.15.122:0/540805529 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f6a0c094630 con 0x7f6a180cc8c0 2024-09-17T05:28:13.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.251+0000 7f6a1d200640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69f8073560 0x7f69f8075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:13.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.251+0000 7f6a1d200640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69f8073560 0x7f69f8075a20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a180b6b70 tx=0x7f6a04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:13.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.255+0000 7f6a15600640 1 -- 172.21.15.122:0/540805529 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6a0c05f210 con 0x7f6a180cc8c0 2024-09-17T05:28:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.419+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a180c4fa0 con 0x7f69f8073560 2024-09-17T05:28:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.423+0000 7f6a15600640 1 -- 172.21.15.122:0/540805529 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f6a04001c00 con 0x7f69f8073560 2024-09-17T05:28:13.426 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:13.428 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:28:13.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69f8073560 msgr2=0x7f69f8075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69f8073560 0x7f69f8075a20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a180b6b70 tx=0x7f6a04009040 comp rx=0 tx=0).stop 2024-09-17T05:28:13.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 msgr2=0x7f6a180afa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 0x7f6a180afa30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00c450 tx=0x7f6a0c00c910 comp rx=0 tx=0).stop 2024-09-17T05:28:13.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 shutdown_connections 2024-09-17T05:28:13.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69f8073560 0x7f69f8075a20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a18158620 0x7f6a180c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a18158240 0x7f6a180c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 --2- 172.21.15.122:0/540805529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a180cc8c0 0x7f6a180afa30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 >> 172.21.15.122:0/540805529 conn(0x7f6a180d0c90 msgr2=0x7f6a180d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:13.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 shutdown_connections 2024-09-17T05:28:13.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.427+0000 7f6a20600640 1 -- 172.21.15.122:0/540805529 wait complete. 2024-09-17T05:28:13.447 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-17T05:28:12.493721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263216,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754711504,"statfs":{"total":773094113280,"available":772824580096,"internally_reserved":0,"allocated":5513216,"data_stored":2906948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":263835190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.106636"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.404387+0000","last_change":"2024-09-17T05:28:04.410815+0000","last_active":"2024-09-17T05:28:05.404387+0000","last_peered":"2024-09-17T05:28:05.404387+0000","last_clean":"2024-09-17T05:28:05.404387+0000","last_became_active":"2024-09-17T05:28:04.408969+0000","last_became_peered":"2024-09-17T05:28:04.408969+0000","last_unstale":"2024-09-17T05:28:05.404387+0000","last_undegraded":"2024-09-17T05:28:05.404387+0000","last_fullsized":"2024-09-17T05:28:05.404387+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:35.739947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230219+0000","last_change":"2024-09-17T05:28:04.400163+0000","last_active":"2024-09-17T05:28:07.230219+0000","last_peered":"2024-09-17T05:28:07.230219+0000","last_clean":"2024-09-17T05:28:07.230219+0000","last_became_active":"2024-09-17T05:28:04.399793+0000","last_became_peered":"2024-09-17T05:28:04.399793+0000","last_unstale":"2024-09-17T05:28:07.230219+0000","last_undegraded":"2024-09-17T05:28:07.230219+0000","last_fullsized":"2024-09-17T05:28:07.230219+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:33.416236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230185+0000","last_change":"2024-09-17T05:28:04.401659+0000","last_active":"2024-09-17T05:28:07.230185+0000","last_peered":"2024-09-17T05:28:07.230185+0000","last_clean":"2024-09-17T05:28:07.230185+0000","last_became_active":"2024-09-17T05:28:04.401166+0000","last_became_peered":"2024-09-17T05:28:04.401166+0000","last_unstale":"2024-09-17T05:28:07.230185+0000","last_undegraded":"2024-09-17T05:28:07.230185+0000","last_fullsized":"2024-09-17T05:28:07.230185+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:21.750787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409677+0000","last_change":"2024-09-17T05:28:04.416129+0000","last_active":"2024-09-17T05:28:05.409677+0000","last_peered":"2024-09-17T05:28:05.409677+0000","last_clean":"2024-09-17T05:28:05.409677+0000","last_became_active":"2024-09-17T05:28:04.415869+0000","last_became_peered":"2024-09-17T05:28:04.415869+0000","last_unstale":"2024-09-17T05:28:05.409677+0000","last_undegraded":"2024-09-17T05:28:05.409677+0000","last_fullsized":"2024-09-17T05:28:05.409677+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:02.172592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.403758+0000","last_change":"2024-09-17T05:28:04.411113+0000","last_active":"2024-09-17T05:28:05.403758+0000","last_peered":"2024-09-17T05:28:05.403758+0000","last_clean":"2024-09-17T05:28:05.403758+0000","last_became_active":"2024-09-17T05:28:04.410849+0000","last_became_peered":"2024-09-17T05:28:04.410849+0000","last_unstale":"2024-09-17T05:28:05.403758+0000","last_undegraded":"2024-09-17T05:28:05.403758+0000","last_fullsized":"2024-09-17T05:28:05.403758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:58.248539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606522+0000","last_change":"2024-09-17T05:28:04.414974+0000","last_active":"2024-09-17T05:28:05.606522+0000","last_peered":"2024-09-17T05:28:05.606522+0000","last_clean":"2024-09-17T05:28:05.606522+0000","last_became_active":"2024-09-17T05:28:04.414571+0000","last_became_peered":"2024-09-17T05:28:04.414571+0000","last_unstale":"2024-09-17T05:28:05.606522+0000","last_undegraded":"2024-09-17T05:28:05.606522+0000","last_fullsized":"2024-09-17T05:28:05.606522+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:45.429403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.393976+0000","last_change":"2024-09-17T05:28:04.413403+0000","last_active":"2024-09-17T05:28:06.393976+0000","last_peered":"2024-09-17T05:28:06.393976+0000","last_clean":"2024-09-17T05:28:06.393976+0000","last_became_active":"2024-09-17T05:28:04.413106+0000","last_became_peered":"2024-09-17T05:28:04.413106+0000","last_unstale":"2024-09-17T05:28:06.393976+0000","last_undegraded":"2024-09-17T05:28:06.393976+0000","last_fullsized":"2024-09-17T05:28:06.393976+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:00.782058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.408875+0000","last_change":"2024-09-17T05:28:04.421456+0000","last_active":"2024-09-17T05:28:05.408875+0000","last_peered":"2024-09-17T05:28:05.408875+0000","last_clean":"2024-09-17T05:28:05.408875+0000","last_became_active":"2024-09-17T05:28:04.421034+0000","last_became_peered":"2024-09-17T05:28:04.421034+0000","last_unstale":"2024-09-17T05:28:05.408875+0000","last_undegraded":"2024-09-17T05:28:05.408875+0000","last_fullsized":"2024-09-17T05:28:05.408875+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:41.119416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606592+0000","last_change":"2024-09-17T05:28:04.411323+0000","last_active":"2024-09-17T05:28:05.606592+0000","last_peered":"2024-09-17T05:28:05.606592+0000","last_clean":"2024-09-17T05:28:05.606592+0000","last_became_active":"2024-09-17T05:28:04.410483+0000","last_became_peered":"2024-09-17T05:28:04.410483+0000","last_unstale":"2024-09-17T05:28:05.606592+0000","last_undegraded":"2024-09-17T05:28:05.606592+0000","last_fullsized":"2024-09-17T05:28:05.606592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:12.008566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251748+0000","last_change":"2024-09-17T05:28:04.406933+0000","last_active":"2024-09-17T05:28:06.251748+0000","last_peered":"2024-09-17T05:28:06.251748+0000","last_clean":"2024-09-17T05:28:06.251748+0000","last_became_active":"2024-09-17T05:28:04.406587+0000","last_became_peered":"2024-09-17T05:28:04.406587+0000","last_unstale":"2024-09-17T05:28:06.251748+0000","last_undegraded":"2024-09-17T05:28:06.251748+0000","last_fullsized":"2024-09-17T05:28:06.251748+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:47.490222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409905+0000","last_change":"2024-09-17T05:28:04.419275+0000","last_active":"2024-09-17T05:28:05.409905+0000","last_peered":"2024-09-17T05:28:05.409905+0000","last_clean":"2024-09-17T05:28:05.409905+0000","last_became_active":"2024-09-17T05:28:04.418991+0000","last_became_peered":"2024-09-17T05:28:04.418991+0000","last_unstale":"2024-09-17T05:28:05.409905+0000","last_undegraded":"2024-09-17T05:28:05.409905+0000","last_fullsized":"2024-09-17T05:28:05.409905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:23.031211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229933+0000","last_change":"2024-09-17T05:28:04.399932+0000","last_active":"2024-09-17T05:28:07.229933+0000","last_peered":"2024-09-17T05:28:07.229933+0000","last_clean":"2024-09-17T05:28:07.229933+0000","last_became_active":"2024-09-17T05:28:04.399693+0000","last_became_peered":"2024-09-17T05:28:04.399693+0000","last_unstale":"2024-09-17T05:28:07.229933+0000","last_undegraded":"2024-09-17T05:28:07.229933+0000","last_fullsized":"2024-09-17T05:28:07.229933+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:59.431986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606297+0000","last_change":"2024-09-17T05:28:04.427371+0000","last_active":"2024-09-17T05:28:05.606297+0000","last_peered":"2024-09-17T05:28:05.606297+0000","last_clean":"2024-09-17T05:28:05.606297+0000","last_became_active":"2024-09-17T05:28:04.427101+0000","last_became_peered":"2024-09-17T05:28:04.427101+0000","last_unstale":"2024-09-17T05:28:05.606297+0000","last_undegraded":"2024-09-17T05:28:05.606297+0000","last_fullsized":"2024-09-17T05:28:05.606297+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:46.859617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409281+0000","last_change":"2024-09-17T05:28:04.411688+0000","last_active":"2024-09-17T05:28:05.409281+0000","last_peered":"2024-09-17T05:28:05.409281+0000","last_clean":"2024-09-17T05:28:05.409281+0000","last_became_active":"2024-09-17T05:28:04.411403+0000","last_became_peered":"2024-09-17T05:28:04.411403+0000","last_unstale":"2024-09-17T05:28:05.409281+0000","last_undegraded":"2024-09-17T05:28:05.409281+0000","last_fullsized":"2024-09-17T05:28:05.409281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:17.638839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409054+0000","last_change":"2024-09-17T05:28:04.423517+0000","last_active":"2024-09-17T05:28:05.409054+0000","last_peered":"2024-09-17T05:28:05.409054+0000","last_clean":"2024-09-17T05:28:05.409054+0000","last_became_active":"2024-09-17T05:28:04.423243+0000","last_became_peered":"2024-09-17T05:28:04.423243+0000","last_unstale":"2024-09-17T05:28:05.409054+0000","last_undegraded":"2024-09-17T05:28:05.409054+0000","last_fullsized":"2024-09-17T05:28:05.409054+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:24.878942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.403648+0000","last_change":"2024-09-17T05:28:04.411942+0000","last_active":"2024-09-17T05:28:05.403648+0000","last_peered":"2024-09-17T05:28:05.403648+0000","last_clean":"2024-09-17T05:28:05.403648+0000","last_became_active":"2024-09-17T05:28:04.411682+0000","last_became_peered":"2024-09-17T05:28:04.411682+0000","last_unstale":"2024-09-17T05:28:05.403648+0000","last_undegraded":"2024-09-17T05:28:05.403648+0000","last_fullsized":"2024-09-17T05:28:05.403648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:59.431986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230493+0000","last_change":"2024-09-17T05:28:04.406853+0000","last_active":"2024-09-17T05:28:07.230493+0000","last_peered":"2024-09-17T05:28:07.230493+0000","last_clean":"2024-09-17T05:28:07.230493+0000","last_became_active":"2024-09-17T05:28:04.406600+0000","last_became_peered":"2024-09-17T05:28:04.406600+0000","last_unstale":"2024-09-17T05:28:07.230493+0000","last_undegraded":"2024-09-17T05:28:07.230493+0000","last_fullsized":"2024-09-17T05:28:07.230493+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:50.794813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229880+0000","last_change":"2024-09-17T05:28:04.413166+0000","last_active":"2024-09-17T05:28:07.229880+0000","last_peered":"2024-09-17T05:28:07.229880+0000","last_clean":"2024-09-17T05:28:07.229880+0000","last_became_active":"2024-09-17T05:28:04.412896+0000","last_became_peered":"2024-09-17T05:28:04.412896+0000","last_unstale":"2024-09-17T05:28:07.229880+0000","last_undegraded":"2024-09-17T05:28:07.229880+0000","last_fullsized":"2024-09-17T05:28:07.229880+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:21.509740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.404197+0000","last_change":"2024-09-17T05:28:04.408844+0000","last_active":"2024-09-17T05:28:05.404197+0000","last_peered":"2024-09-17T05:28:05.404197+0000","last_clean":"2024-09-17T05:28:05.404197+0000","last_became_active":"2024-09-17T05:28:04.408508+0000","last_became_peered":"2024-09-17T05:28:04.408508+0000","last_unstale":"2024-09-17T05:28:05.404197+0000","last_undegraded":"2024-09-17T05:28:05.404197+0000","last_fullsized":"2024-09-17T05:28:05.404197+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:33.416236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251549+0000","last_change":"2024-09-17T05:28:04.422495+0000","last_active":"2024-09-17T05:28:06.251549+0000","last_peered":"2024-09-17T05:28:06.251549+0000","last_clean":"2024-09-17T05:28:06.251549+0000","last_became_active":"2024-09-17T05:28:04.420411+0000","last_became_peered":"2024-09-17T05:28:04.420411+0000","last_unstale":"2024-09-17T05:28:06.251549+0000","last_undegraded":"2024-09-17T05:28:06.251549+0000","last_fullsized":"2024-09-17T05:28:06.251549+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:07.002120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409823+0000","last_change":"2024-09-17T05:28:04.421922+0000","last_active":"2024-09-17T05:28:05.409823+0000","last_peered":"2024-09-17T05:28:05.409823+0000","last_clean":"2024-09-17T05:28:05.409823+0000","last_became_active":"2024-09-17T05:28:04.414061+0000","last_became_peered":"2024-09-17T05:28:04.414061+0000","last_unstale":"2024-09-17T05:28:05.409823+0000","last_undegraded":"2024-09-17T05:28:05.409823+0000","last_fullsized":"2024-09-17T05:28:05.409823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:35.698197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410707+0000","last_change":"2024-09-17T05:28:04.405634+0000","last_active":"2024-09-17T05:28:05.410707+0000","last_peered":"2024-09-17T05:28:05.410707+0000","last_clean":"2024-09-17T05:28:05.410707+0000","last_became_active":"2024-09-17T05:28:04.405404+0000","last_became_peered":"2024-09-17T05:28:04.405404+0000","last_unstale":"2024-09-17T05:28:05.410707+0000","last_undegraded":"2024-09-17T05:28:05.410707+0000","last_fullsized":"2024-09-17T05:28:05.410707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:38.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394225+0000","last_change":"2024-09-17T05:28:04.397685+0000","last_active":"2024-09-17T05:28:06.394225+0000","last_peered":"2024-09-17T05:28:06.394225+0000","last_clean":"2024-09-17T05:28:06.394225+0000","last_became_active":"2024-09-17T05:28:04.397341+0000","last_became_peered":"2024-09-17T05:28:04.397341+0000","last_unstale":"2024-09-17T05:28:06.394225+0000","last_undegraded":"2024-09-17T05:28:06.394225+0000","last_fullsized":"2024-09-17T05:28:06.394225+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:58.972972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229414+0000","last_change":"2024-09-17T05:28:04.415623+0000","last_active":"2024-09-17T05:28:07.229414+0000","last_peered":"2024-09-17T05:28:07.229414+0000","last_clean":"2024-09-17T05:28:07.229414+0000","last_became_active":"2024-09-17T05:28:04.415437+0000","last_became_peered":"2024-09-17T05:28:04.415437+0000","last_unstale":"2024-09-17T05:28:07.229414+0000","last_undegraded":"2024-09-17T05:28:07.229414+0000","last_fullsized":"2024-09-17T05:28:07.229414+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:58.248539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.404413+0000","last_change":"2024-09-17T05:28:04.409812+0000","last_active":"2024-09-17T05:28:05.404413+0000","last_peered":"2024-09-17T05:28:05.404413+0000","last_clean":"2024-09-17T05:28:05.404413+0000","last_became_active":"2024-09-17T05:28:04.408623+0000","last_became_peered":"2024-09-17T05:28:04.408623+0000","last_unstale":"2024-09-17T05:28:05.404413+0000","last_undegraded":"2024-09-17T05:28:05.404413+0000","last_fullsized":"2024-09-17T05:28:05.404413+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:50.794813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410396+0000","last_change":"2024-09-17T05:28:04.419337+0000","last_active":"2024-09-17T05:28:05.410396+0000","last_peered":"2024-09-17T05:28:05.410396+0000","last_clean":"2024-09-17T05:28:05.410396+0000","last_became_active":"2024-09-17T05:28:04.419066+0000","last_became_peered":"2024-09-17T05:28:04.419066+0000","last_unstale":"2024-09-17T05:28:05.410396+0000","last_undegraded":"2024-09-17T05:28:05.410396+0000","last_fullsized":"2024-09-17T05:28:05.410396+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:28.650511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394349+0000","last_change":"2024-09-17T05:28:04.403488+0000","last_active":"2024-09-17T05:28:06.394349+0000","last_peered":"2024-09-17T05:28:06.394349+0000","last_clean":"2024-09-17T05:28:06.394349+0000","last_became_active":"2024-09-17T05:28:04.403190+0000","last_became_peered":"2024-09-17T05:28:04.403190+0000","last_unstale":"2024-09-17T05:28:06.394349+0000","last_undegraded":"2024-09-17T05:28:06.394349+0000","last_fullsized":"2024-09-17T05:28:06.394349+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:45.695123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394224+0000","last_change":"2024-09-17T05:28:04.401638+0000","last_active":"2024-09-17T05:28:06.394224+0000","last_peered":"2024-09-17T05:28:06.394224+0000","last_clean":"2024-09-17T05:28:06.394224+0000","last_became_active":"2024-09-17T05:28:04.401218+0000","last_became_peered":"2024-09-17T05:28:04.401218+0000","last_unstale":"2024-09-17T05:28:06.394224+0000","last_undegraded":"2024-09-17T05:28:06.394224+0000","last_fullsized":"2024-09-17T05:28:06.394224+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:52.886579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.403939+0000","last_change":"2024-09-17T05:28:04.405151+0000","last_active":"2024-09-17T05:28:05.403939+0000","last_peered":"2024-09-17T05:28:05.403939+0000","last_clean":"2024-09-17T05:28:05.403939+0000","last_became_active":"2024-09-17T05:28:04.404864+0000","last_became_peered":"2024-09-17T05:28:04.404864+0000","last_unstale":"2024-09-17T05:28:05.403939+0000","last_undegraded":"2024-09-17T05:28:05.403939+0000","last_fullsized":"2024-09-17T05:28:05.403939+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:35:50.545868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229445+0000","last_change":"2024-09-17T05:28:04.403008+0000","last_active":"2024-09-17T05:28:07.229445+0000","last_peered":"2024-09-17T05:28:07.229445+0000","last_clean":"2024-09-17T05:28:07.229445+0000","last_became_active":"2024-09-17T05:28:04.402738+0000","last_became_peered":"2024-09-17T05:28:04.402738+0000","last_unstale":"2024-09-17T05:28:07.229445+0000","last_undegraded":"2024-09-17T05:28:07.229445+0000","last_fullsized":"2024-09-17T05:28:07.229445+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:51.683121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606271+0000","last_change":"2024-09-17T05:28:04.417065+0000","last_active":"2024-09-17T05:28:05.606271+0000","last_peered":"2024-09-17T05:28:05.606271+0000","last_clean":"2024-09-17T05:28:05.606271+0000","last_became_active":"2024-09-17T05:28:04.416781+0000","last_became_peered":"2024-09-17T05:28:04.416781+0000","last_unstale":"2024-09-17T05:28:05.606271+0000","last_undegraded":"2024-09-17T05:28:05.606271+0000","last_fullsized":"2024-09-17T05:28:05.606271+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:19.883152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409026+0000","last_change":"2024-09-17T05:28:04.414079+0000","last_active":"2024-09-17T05:28:05.409026+0000","last_peered":"2024-09-17T05:28:05.409026+0000","last_clean":"2024-09-17T05:28:05.409026+0000","last_became_active":"2024-09-17T05:28:04.413733+0000","last_became_peered":"2024-09-17T05:28:04.413733+0000","last_unstale":"2024-09-17T05:28:05.409026+0000","last_undegraded":"2024-09-17T05:28:05.409026+0000","last_fullsized":"2024-09-17T05:28:05.409026+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:15.069773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.411128+0000","last_change":"2024-09-17T05:28:04.407601+0000","last_active":"2024-09-17T05:28:05.411128+0000","last_peered":"2024-09-17T05:28:05.411128+0000","last_clean":"2024-09-17T05:28:05.411128+0000","last_became_active":"2024-09-17T05:28:04.406508+0000","last_became_peered":"2024-09-17T05:28:04.406508+0000","last_unstale":"2024-09-17T05:28:05.411128+0000","last_undegraded":"2024-09-17T05:28:05.411128+0000","last_fullsized":"2024-09-17T05:28:05.411128+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.223994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606631+0000","last_change":"2024-09-17T05:28:03.408603+0000","last_active":"2024-09-17T05:28:05.606631+0000","last_peered":"2024-09-17T05:28:05.606631+0000","last_clean":"2024-09-17T05:28:05.606631+0000","last_became_active":"2024-09-17T05:28:03.408354+0000","last_became_peered":"2024-09-17T05:28:03.408354+0000","last_unstale":"2024-09-17T05:28:05.606631+0000","last_undegraded":"2024-09-17T05:28:05.606631+0000","last_fullsized":"2024-09-17T05:28:05.606631+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:42:57.995249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.404588+0000","last_change":"2024-09-17T05:28:04.411252+0000","last_active":"2024-09-17T05:28:05.404588+0000","last_peered":"2024-09-17T05:28:05.404588+0000","last_clean":"2024-09-17T05:28:05.404588+0000","last_became_active":"2024-09-17T05:28:04.410903+0000","last_became_peered":"2024-09-17T05:28:04.410903+0000","last_unstale":"2024-09-17T05:28:05.404588+0000","last_undegraded":"2024-09-17T05:28:05.404588+0000","last_fullsized":"2024-09-17T05:28:05.404588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:10.914818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394379+0000","last_change":"2024-09-17T05:28:03.403643+0000","last_active":"2024-09-17T05:28:06.394379+0000","last_peered":"2024-09-17T05:28:06.394379+0000","last_clean":"2024-09-17T05:28:06.394379+0000","last_became_active":"2024-09-17T05:28:03.403353+0000","last_became_peered":"2024-09-17T05:28:03.403353+0000","last_unstale":"2024-09-17T05:28:06.394379+0000","last_undegraded":"2024-09-17T05:28:06.394379+0000","last_fullsized":"2024-09-17T05:28:06.394379+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:08.847793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229970+0000","last_change":"2024-09-17T05:28:04.405514+0000","last_active":"2024-09-17T05:28:07.229970+0000","last_peered":"2024-09-17T05:28:07.229970+0000","last_clean":"2024-09-17T05:28:07.229970+0000","last_became_active":"2024-09-17T05:28:04.405249+0000","last_became_peered":"2024-09-17T05:28:04.405249+0000","last_unstale":"2024-09-17T05:28:07.229970+0000","last_undegraded":"2024-09-17T05:28:07.229970+0000","last_fullsized":"2024-09-17T05:28:07.229970+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:54.642116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606646+0000","last_change":"2024-09-17T05:28:03.407955+0000","last_active":"2024-09-17T05:28:05.606646+0000","last_peered":"2024-09-17T05:28:05.606646+0000","last_clean":"2024-09-17T05:28:05.606646+0000","last_became_active":"2024-09-17T05:28:03.407597+0000","last_became_peered":"2024-09-17T05:28:03.407597+0000","last_unstale":"2024-09-17T05:28:05.606646+0000","last_undegraded":"2024-09-17T05:28:05.606646+0000","last_fullsized":"2024-09-17T05:28:05.606646+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:09.903069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410107+0000","last_change":"2024-09-17T05:28:04.411454+0000","last_active":"2024-09-17T05:28:05.410107+0000","last_peered":"2024-09-17T05:28:05.410107+0000","last_clean":"2024-09-17T05:28:05.410107+0000","last_became_active":"2024-09-17T05:28:04.411144+0000","last_became_peered":"2024-09-17T05:28:04.411144+0000","last_unstale":"2024-09-17T05:28:05.410107+0000","last_undegraded":"2024-09-17T05:28:05.410107+0000","last_fullsized":"2024-09-17T05:28:05.410107+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:10.404469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394214+0000","last_change":"2024-09-17T05:28:03.406275+0000","last_active":"2024-09-17T05:28:06.394214+0000","last_peered":"2024-09-17T05:28:06.394214+0000","last_clean":"2024-09-17T05:28:06.394214+0000","last_became_active":"2024-09-17T05:28:03.405506+0000","last_became_peered":"2024-09-17T05:28:03.405506+0000","last_unstale":"2024-09-17T05:28:06.394214+0000","last_undegraded":"2024-09-17T05:28:06.394214+0000","last_fullsized":"2024-09-17T05:28:06.394214+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:16.444124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.393968+0000","last_change":"2024-09-17T05:28:04.403545+0000","last_active":"2024-09-17T05:28:06.393968+0000","last_peered":"2024-09-17T05:28:06.393968+0000","last_clean":"2024-09-17T05:28:06.393968+0000","last_became_active":"2024-09-17T05:28:04.403280+0000","last_became_peered":"2024-09-17T05:28:04.403280+0000","last_unstale":"2024-09-17T05:28:06.393968+0000","last_undegraded":"2024-09-17T05:28:06.393968+0000","last_fullsized":"2024-09-17T05:28:06.393968+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:59.236390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606827+0000","last_change":"2024-09-17T05:28:03.406212+0000","last_active":"2024-09-17T05:28:05.606827+0000","last_peered":"2024-09-17T05:28:05.606827+0000","last_clean":"2024-09-17T05:28:05.606827+0000","last_became_active":"2024-09-17T05:28:03.405428+0000","last_became_peered":"2024-09-17T05:28:03.405428+0000","last_unstale":"2024-09-17T05:28:05.606827+0000","last_undegraded":"2024-09-17T05:28:05.606827+0000","last_fullsized":"2024-09-17T05:28:05.606827+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:10.302286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251512+0000","last_change":"2024-09-17T05:28:04.416318+0000","last_active":"2024-09-17T05:28:06.251512+0000","last_peered":"2024-09-17T05:28:06.251512+0000","last_clean":"2024-09-17T05:28:06.251512+0000","last_became_active":"2024-09-17T05:28:04.415988+0000","last_became_peered":"2024-09-17T05:28:04.415988+0000","last_unstale":"2024-09-17T05:28:06.251512+0000","last_undegraded":"2024-09-17T05:28:06.251512+0000","last_fullsized":"2024-09-17T05:28:06.251512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:12.126916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.393983+0000","last_change":"2024-09-17T05:28:03.405006+0000","last_active":"2024-09-17T05:28:06.393983+0000","last_peered":"2024-09-17T05:28:06.393983+0000","last_clean":"2024-09-17T05:28:06.393983+0000","last_became_active":"2024-09-17T05:28:03.404741+0000","last_became_peered":"2024-09-17T05:28:03.404741+0000","last_unstale":"2024-09-17T05:28:06.393983+0000","last_undegraded":"2024-09-17T05:28:06.393983+0000","last_fullsized":"2024-09-17T05:28:06.393983+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:22.627351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251215+0000","last_change":"2024-09-17T05:28:04.408123+0000","last_active":"2024-09-17T05:28:06.251215+0000","last_peered":"2024-09-17T05:28:06.251215+0000","last_clean":"2024-09-17T05:28:06.251215+0000","last_became_active":"2024-09-17T05:28:04.401419+0000","last_became_peered":"2024-09-17T05:28:04.401419+0000","last_unstale":"2024-09-17T05:28:06.251215+0000","last_undegraded":"2024-09-17T05:28:06.251215+0000","last_fullsized":"2024-09-17T05:28:06.251215+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:49:29.686287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.403714+0000","last_change":"2024-09-17T05:28:03.397856+0000","last_active":"2024-09-17T05:28:05.403714+0000","last_peered":"2024-09-17T05:28:05.403714+0000","last_clean":"2024-09-17T05:28:05.403714+0000","last_became_active":"2024-09-17T05:28:03.397609+0000","last_became_peered":"2024-09-17T05:28:03.397609+0000","last_unstale":"2024-09-17T05:28:05.403714+0000","last_undegraded":"2024-09-17T05:28:05.403714+0000","last_fullsized":"2024-09-17T05:28:05.403714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:41.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.393950+0000","last_change":"2024-09-17T05:28:04.413766+0000","last_active":"2024-09-17T05:28:06.393950+0000","last_peered":"2024-09-17T05:28:06.393950+0000","last_clean":"2024-09-17T05:28:06.393950+0000","last_became_active":"2024-09-17T05:28:04.413452+0000","last_became_peered":"2024-09-17T05:28:04.413452+0000","last_unstale":"2024-09-17T05:28:06.393950+0000","last_undegraded":"2024-09-17T05:28:06.393950+0000","last_fullsized":"2024-09-17T05:28:06.393950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:52.758735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251446+0000","last_change":"2024-09-17T05:28:03.410366+0000","last_active":"2024-09-17T05:28:06.251446+0000","last_peered":"2024-09-17T05:28:06.251446+0000","last_clean":"2024-09-17T05:28:06.251446+0000","last_became_active":"2024-09-17T05:28:03.409222+0000","last_became_peered":"2024-09-17T05:28:03.409222+0000","last_unstale":"2024-09-17T05:28:06.251446+0000","last_undegraded":"2024-09-17T05:28:06.251446+0000","last_fullsized":"2024-09-17T05:28:06.251446+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:32.413060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230081+0000","last_change":"2024-09-17T05:28:04.417527+0000","last_active":"2024-09-17T05:28:07.230081+0000","last_peered":"2024-09-17T05:28:07.230081+0000","last_clean":"2024-09-17T05:28:07.230081+0000","last_became_active":"2024-09-17T05:28:04.417183+0000","last_became_peered":"2024-09-17T05:28:04.417183+0000","last_unstale":"2024-09-17T05:28:07.230081+0000","last_undegraded":"2024-09-17T05:28:07.230081+0000","last_fullsized":"2024-09-17T05:28:07.230081+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:38.364484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230384+0000","last_change":"2024-09-17T05:28:03.400471+0000","last_active":"2024-09-17T05:28:07.230384+0000","last_peered":"2024-09-17T05:28:07.230384+0000","last_clean":"2024-09-17T05:28:07.230384+0000","last_became_active":"2024-09-17T05:28:03.400202+0000","last_became_peered":"2024-09-17T05:28:03.400202+0000","last_unstale":"2024-09-17T05:28:07.230384+0000","last_undegraded":"2024-09-17T05:28:07.230384+0000","last_fullsized":"2024-09-17T05:28:07.230384+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:28.754516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230177+0000","last_change":"2024-09-17T05:28:04.416350+0000","last_active":"2024-09-17T05:28:07.230177+0000","last_peered":"2024-09-17T05:28:07.230177+0000","last_clean":"2024-09-17T05:28:07.230177+0000","last_became_active":"2024-09-17T05:28:04.415630+0000","last_became_peered":"2024-09-17T05:28:04.415630+0000","last_unstale":"2024-09-17T05:28:07.230177+0000","last_undegraded":"2024-09-17T05:28:07.230177+0000","last_fullsized":"2024-09-17T05:28:07.230177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:10.906245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251767+0000","last_change":"2024-09-17T05:28:03.401227+0000","last_active":"2024-09-17T05:28:06.251767+0000","last_peered":"2024-09-17T05:28:06.251767+0000","last_clean":"2024-09-17T05:28:06.251767+0000","last_became_active":"2024-09-17T05:28:03.400688+0000","last_became_peered":"2024-09-17T05:28:03.400688+0000","last_unstale":"2024-09-17T05:28:06.251767+0000","last_undegraded":"2024-09-17T05:28:06.251767+0000","last_fullsized":"2024-09-17T05:28:06.251767+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:57.245363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409693+0000","last_change":"2024-09-17T05:28:04.413706+0000","last_active":"2024-09-17T05:28:05.409693+0000","last_peered":"2024-09-17T05:28:05.409693+0000","last_clean":"2024-09-17T05:28:05.409693+0000","last_became_active":"2024-09-17T05:28:04.413217+0000","last_became_peered":"2024-09-17T05:28:04.413217+0000","last_unstale":"2024-09-17T05:28:05.409693+0000","last_undegraded":"2024-09-17T05:28:05.409693+0000","last_fullsized":"2024-09-17T05:28:05.409693+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:33.217682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229967+0000","last_change":"2024-09-17T05:28:03.401672+0000","last_active":"2024-09-17T05:28:07.229967+0000","last_peered":"2024-09-17T05:28:07.229967+0000","last_clean":"2024-09-17T05:28:07.229967+0000","last_became_active":"2024-09-17T05:28:03.401380+0000","last_became_peered":"2024-09-17T05:28:03.401380+0000","last_unstale":"2024-09-17T05:28:07.229967+0000","last_undegraded":"2024-09-17T05:28:07.229967+0000","last_fullsized":"2024-09-17T05:28:07.229967+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:04.572530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394382+0000","last_change":"2024-09-17T05:28:04.409973+0000","last_active":"2024-09-17T05:28:06.394382+0000","last_peered":"2024-09-17T05:28:06.394382+0000","last_clean":"2024-09-17T05:28:06.394382+0000","last_became_active":"2024-09-17T05:28:04.409702+0000","last_became_peered":"2024-09-17T05:28:04.409702+0000","last_unstale":"2024-09-17T05:28:06.394382+0000","last_undegraded":"2024-09-17T05:28:06.394382+0000","last_fullsized":"2024-09-17T05:28:06.394382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:20.822462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251899+0000","last_change":"2024-09-17T05:28:03.408491+0000","last_active":"2024-09-17T05:28:06.251899+0000","last_peered":"2024-09-17T05:28:06.251899+0000","last_clean":"2024-09-17T05:28:06.251899+0000","last_became_active":"2024-09-17T05:28:03.407793+0000","last_became_peered":"2024-09-17T05:28:03.407793+0000","last_unstale":"2024-09-17T05:28:06.251899+0000","last_undegraded":"2024-09-17T05:28:06.251899+0000","last_fullsized":"2024-09-17T05:28:06.251899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:35:49.542692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229729+0000","last_change":"2024-09-17T05:28:04.415045+0000","last_active":"2024-09-17T05:28:07.229729+0000","last_peered":"2024-09-17T05:28:07.229729+0000","last_clean":"2024-09-17T05:28:07.229729+0000","last_became_active":"2024-09-17T05:28:04.414749+0000","last_became_peered":"2024-09-17T05:28:04.414749+0000","last_unstale":"2024-09-17T05:28:07.229729+0000","last_undegraded":"2024-09-17T05:28:07.229729+0000","last_fullsized":"2024-09-17T05:28:07.229729+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:32.641386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410921+0000","last_change":"2024-09-17T05:28:03.406639+0000","last_active":"2024-09-17T05:28:05.410921+0000","last_peered":"2024-09-17T05:28:05.410921+0000","last_clean":"2024-09-17T05:28:05.410921+0000","last_became_active":"2024-09-17T05:28:03.406360+0000","last_became_peered":"2024-09-17T05:28:03.406360+0000","last_unstale":"2024-09-17T05:28:05.410921+0000","last_undegraded":"2024-09-17T05:28:05.410921+0000","last_fullsized":"2024-09-17T05:28:05.410921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:43.750161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410940+0000","last_change":"2024-09-17T05:28:04.409285+0000","last_active":"2024-09-17T05:28:05.410940+0000","last_peered":"2024-09-17T05:28:05.410940+0000","last_clean":"2024-09-17T05:28:05.410940+0000","last_became_active":"2024-09-17T05:28:04.409003+0000","last_became_peered":"2024-09-17T05:28:04.409003+0000","last_unstale":"2024-09-17T05:28:05.410940+0000","last_undegraded":"2024-09-17T05:28:05.410940+0000","last_fullsized":"2024-09-17T05:28:05.410940+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:21.750787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.403786+0000","last_change":"2024-09-17T05:28:03.401363+0000","last_active":"2024-09-17T05:28:05.403786+0000","last_peered":"2024-09-17T05:28:05.403786+0000","last_clean":"2024-09-17T05:28:05.403786+0000","last_became_active":"2024-09-17T05:28:03.401110+0000","last_became_peered":"2024-09-17T05:28:03.401110+0000","last_unstale":"2024-09-17T05:28:05.403786+0000","last_undegraded":"2024-09-17T05:28:05.403786+0000","last_fullsized":"2024-09-17T05:28:05.403786+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:40.122035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410375+0000","last_change":"2024-09-17T05:28:04.417294+0000","last_active":"2024-09-17T05:28:05.410375+0000","last_peered":"2024-09-17T05:28:05.410375+0000","last_clean":"2024-09-17T05:28:05.410375+0000","last_became_active":"2024-09-17T05:28:04.416974+0000","last_became_peered":"2024-09-17T05:28:04.416974+0000","last_unstale":"2024-09-17T05:28:05.410375+0000","last_undegraded":"2024-09-17T05:28:05.410375+0000","last_fullsized":"2024-09-17T05:28:05.410375+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:21.509740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606279+0000","last_change":"2024-09-17T05:28:03.409353+0000","last_active":"2024-09-17T05:28:05.606279+0000","last_peered":"2024-09-17T05:28:05.606279+0000","last_clean":"2024-09-17T05:28:05.606279+0000","last_became_active":"2024-09-17T05:28:03.409087+0000","last_became_peered":"2024-09-17T05:28:03.409087+0000","last_unstale":"2024-09-17T05:28:05.606279+0000","last_undegraded":"2024-09-17T05:28:05.606279+0000","last_fullsized":"2024-09-17T05:28:05.606279+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:39.382226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229416+0000","last_change":"2024-09-17T05:28:04.406993+0000","last_active":"2024-09-17T05:28:07.229416+0000","last_peered":"2024-09-17T05:28:07.229416+0000","last_clean":"2024-09-17T05:28:07.229416+0000","last_became_active":"2024-09-17T05:28:04.406691+0000","last_became_peered":"2024-09-17T05:28:04.406691+0000","last_unstale":"2024-09-17T05:28:07.229416+0000","last_undegraded":"2024-09-17T05:28:07.229416+0000","last_fullsized":"2024-09-17T05:28:07.229416+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:45.429403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":23,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251596+0000","last_change":"2024-09-17T05:28:03.403774+0000","last_active":"2024-09-17T05:28:06.251596+0000","last_peered":"2024-09-17T05:28:06.251596+0000","last_clean":"2024-09-17T05:28:06.251596+0000","last_became_active":"2024-09-17T05:28:03.403425+0000","last_became_peered":"2024-09-17T05:28:03.403425+0000","last_unstale":"2024-09-17T05:28:06.251596+0000","last_undegraded":"2024-09-17T05:28:06.251596+0000","last_fullsized":"2024-09-17T05:28:06.251596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:34.736771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.411285+0000","last_change":"2024-09-17T05:28:04.407740+0000","last_active":"2024-09-17T05:28:05.411285+0000","last_peered":"2024-09-17T05:28:05.411285+0000","last_clean":"2024-09-17T05:28:05.411285+0000","last_became_active":"2024-09-17T05:28:04.407351+0000","last_became_peered":"2024-09-17T05:28:04.407351+0000","last_unstale":"2024-09-17T05:28:05.411285+0000","last_undegraded":"2024-09-17T05:28:05.411285+0000","last_fullsized":"2024-09-17T05:28:05.411285+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:58.583465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410531+0000","last_change":"2024-09-17T05:28:03.409845+0000","last_active":"2024-09-17T05:28:05.410531+0000","last_peered":"2024-09-17T05:28:05.410531+0000","last_clean":"2024-09-17T05:28:05.410531+0000","last_became_active":"2024-09-17T05:28:03.409519+0000","last_became_peered":"2024-09-17T05:28:03.409519+0000","last_unstale":"2024-09-17T05:28:05.410531+0000","last_undegraded":"2024-09-17T05:28:05.410531+0000","last_fullsized":"2024-09-17T05:28:05.410531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:04.018422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.411135+0000","last_change":"2024-09-17T05:28:04.402268+0000","last_active":"2024-09-17T05:28:05.411135+0000","last_peered":"2024-09-17T05:28:05.411135+0000","last_clean":"2024-09-17T05:28:05.411135+0000","last_became_active":"2024-09-17T05:28:04.401996+0000","last_became_peered":"2024-09-17T05:28:04.401996+0000","last_unstale":"2024-09-17T05:28:05.411135+0000","last_undegraded":"2024-09-17T05:28:05.411135+0000","last_fullsized":"2024-09-17T05:28:05.411135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:41.681160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230234+0000","last_change":"2024-09-17T05:28:03.408640+0000","last_active":"2024-09-17T05:28:07.230234+0000","last_peered":"2024-09-17T05:28:07.230234+0000","last_clean":"2024-09-17T05:28:07.230234+0000","last_became_active":"2024-09-17T05:28:03.408371+0000","last_became_peered":"2024-09-17T05:28:03.408371+0000","last_unstale":"2024-09-17T05:28:07.230234+0000","last_undegraded":"2024-09-17T05:28:07.230234+0000","last_fullsized":"2024-09-17T05:28:07.230234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:04.792542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230346+0000","last_change":"2024-09-17T05:28:04.415415+0000","last_active":"2024-09-17T05:28:07.230346+0000","last_peered":"2024-09-17T05:28:07.230346+0000","last_clean":"2024-09-17T05:28:07.230346+0000","last_became_active":"2024-09-17T05:28:04.414944+0000","last_became_peered":"2024-09-17T05:28:04.414944+0000","last_unstale":"2024-09-17T05:28:07.230346+0000","last_undegraded":"2024-09-17T05:28:07.230346+0000","last_fullsized":"2024-09-17T05:28:07.230346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:35.739947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230364+0000","last_change":"2024-09-17T05:28:03.398552+0000","last_active":"2024-09-17T05:28:07.230364+0000","last_peered":"2024-09-17T05:28:07.230364+0000","last_clean":"2024-09-17T05:28:07.230364+0000","last_became_active":"2024-09-17T05:28:03.398282+0000","last_became_peered":"2024-09-17T05:28:03.398282+0000","last_unstale":"2024-09-17T05:28:07.230364+0000","last_undegraded":"2024-09-17T05:28:07.230364+0000","last_fullsized":"2024-09-17T05:28:07.230364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:56.640383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.230223+0000","last_change":"2024-09-17T05:28:04.401400+0000","last_active":"2024-09-17T05:28:07.230223+0000","last_peered":"2024-09-17T05:28:07.230223+0000","last_clean":"2024-09-17T05:28:07.230223+0000","last_became_active":"2024-09-17T05:28:04.400856+0000","last_became_peered":"2024-09-17T05:28:04.400856+0000","last_unstale":"2024-09-17T05:28:07.230223+0000","last_undegraded":"2024-09-17T05:28:07.230223+0000","last_fullsized":"2024-09-17T05:28:07.230223+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.223994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606235+0000","last_change":"2024-09-17T05:28:03.404997+0000","last_active":"2024-09-17T05:28:05.606235+0000","last_peered":"2024-09-17T05:28:05.606235+0000","last_clean":"2024-09-17T05:28:05.606235+0000","last_became_active":"2024-09-17T05:28:03.404512+0000","last_became_peered":"2024-09-17T05:28:03.404512+0000","last_unstale":"2024-09-17T05:28:05.606235+0000","last_undegraded":"2024-09-17T05:28:05.606235+0000","last_fullsized":"2024-09-17T05:28:05.606235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:59:41.144238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251332+0000","last_change":"2024-09-17T05:28:04.425612+0000","last_active":"2024-09-17T05:28:06.251332+0000","last_peered":"2024-09-17T05:28:06.251332+0000","last_clean":"2024-09-17T05:28:06.251332+0000","last_became_active":"2024-09-17T05:28:04.425306+0000","last_became_peered":"2024-09-17T05:28:04.425306+0000","last_unstale":"2024-09-17T05:28:06.251332+0000","last_undegraded":"2024-09-17T05:28:06.251332+0000","last_fullsized":"2024-09-17T05:28:06.251332+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:25.183839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251307+0000","last_change":"2024-09-17T05:28:03.400873+0000","last_active":"2024-09-17T05:28:06.251307+0000","last_peered":"2024-09-17T05:28:06.251307+0000","last_clean":"2024-09-17T05:28:06.251307+0000","last_became_active":"2024-09-17T05:28:03.400549+0000","last_became_peered":"2024-09-17T05:28:03.400549+0000","last_unstale":"2024-09-17T05:28:06.251307+0000","last_undegraded":"2024-09-17T05:28:06.251307+0000","last_fullsized":"2024-09-17T05:28:06.251307+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:58.428810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606654+0000","last_change":"2024-09-17T05:28:04.412609+0000","last_active":"2024-09-17T05:28:05.606654+0000","last_peered":"2024-09-17T05:28:05.606654+0000","last_clean":"2024-09-17T05:28:05.606654+0000","last_became_active":"2024-09-17T05:28:04.412351+0000","last_became_peered":"2024-09-17T05:28:04.412351+0000","last_unstale":"2024-09-17T05:28:05.606654+0000","last_undegraded":"2024-09-17T05:28:05.606654+0000","last_fullsized":"2024-09-17T05:28:05.606654+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:51.683121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.403874+0000","last_change":"2024-09-17T05:28:03.395798+0000","last_active":"2024-09-17T05:28:05.403874+0000","last_peered":"2024-09-17T05:28:05.403874+0000","last_clean":"2024-09-17T05:28:05.403874+0000","last_became_active":"2024-09-17T05:28:03.395543+0000","last_became_peered":"2024-09-17T05:28:03.395543+0000","last_unstale":"2024-09-17T05:28:05.403874+0000","last_undegraded":"2024-09-17T05:28:05.403874+0000","last_fullsized":"2024-09-17T05:28:05.403874+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:34.710243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409491+0000","last_change":"2024-09-17T05:28:04.423847+0000","last_active":"2024-09-17T05:28:05.409491+0000","last_peered":"2024-09-17T05:28:05.409491+0000","last_clean":"2024-09-17T05:28:05.409491+0000","last_became_active":"2024-09-17T05:28:04.423569+0000","last_became_peered":"2024-09-17T05:28:04.423569+0000","last_unstale":"2024-09-17T05:28:05.409491+0000","last_undegraded":"2024-09-17T05:28:05.409491+0000","last_fullsized":"2024-09-17T05:28:05.409491+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:58.595615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251127+0000","last_change":"2024-09-17T05:28:03.400992+0000","last_active":"2024-09-17T05:28:06.251127+0000","last_peered":"2024-09-17T05:28:06.251127+0000","last_clean":"2024-09-17T05:28:06.251127+0000","last_became_active":"2024-09-17T05:28:03.400690+0000","last_became_peered":"2024-09-17T05:28:03.400690+0000","last_unstale":"2024-09-17T05:28:06.251127+0000","last_undegraded":"2024-09-17T05:28:06.251127+0000","last_fullsized":"2024-09-17T05:28:06.251127+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:50.679945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409358+0000","last_change":"2024-09-17T05:28:04.426697+0000","last_active":"2024-09-17T05:28:05.409358+0000","last_peered":"2024-09-17T05:28:05.409358+0000","last_clean":"2024-09-17T05:28:05.409358+0000","last_became_active":"2024-09-17T05:28:04.426419+0000","last_became_peered":"2024-09-17T05:28:04.426419+0000","last_unstale":"2024-09-17T05:28:05.409358+0000","last_undegraded":"2024-09-17T05:28:05.409358+0000","last_fullsized":"2024-09-17T05:28:05.409358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.432704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.393853+0000","last_change":"2024-09-17T05:28:03.407318+0000","last_active":"2024-09-17T05:28:06.393853+0000","last_peered":"2024-09-17T05:28:06.393853+0000","last_clean":"2024-09-17T05:28:06.393853+0000","last_became_active":"2024-09-17T05:28:03.407059+0000","last_became_peered":"2024-09-17T05:28:03.407059+0000","last_unstale":"2024-09-17T05:28:06.393853+0000","last_undegraded":"2024-09-17T05:28:06.393853+0000","last_fullsized":"2024-09-17T05:28:06.393853+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:11.123739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394452+0000","last_change":"2024-09-17T05:28:04.415514+0000","last_active":"2024-09-17T05:28:06.394452+0000","last_peered":"2024-09-17T05:28:06.394452+0000","last_clean":"2024-09-17T05:28:06.394452+0000","last_became_active":"2024-09-17T05:28:04.413865+0000","last_became_peered":"2024-09-17T05:28:04.413865+0000","last_unstale":"2024-09-17T05:28:06.394452+0000","last_undegraded":"2024-09-17T05:28:06.394452+0000","last_fullsized":"2024-09-17T05:28:06.394452+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:45.151110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410708+0000","last_change":"2024-09-17T05:28:03.408701+0000","last_active":"2024-09-17T05:28:05.410708+0000","last_peered":"2024-09-17T05:28:05.410708+0000","last_clean":"2024-09-17T05:28:05.410708+0000","last_became_active":"2024-09-17T05:28:03.408379+0000","last_became_peered":"2024-09-17T05:28:03.408379+0000","last_unstale":"2024-09-17T05:28:05.410708+0000","last_undegraded":"2024-09-17T05:28:05.410708+0000","last_fullsized":"2024-09-17T05:28:05.410708+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:24.330713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251701+0000","last_change":"2024-09-17T05:28:04.409609+0000","last_active":"2024-09-17T05:28:06.251701+0000","last_peered":"2024-09-17T05:28:06.251701+0000","last_clean":"2024-09-17T05:28:06.251701+0000","last_became_active":"2024-09-17T05:28:04.409341+0000","last_became_peered":"2024-09-17T05:28:04.409341+0000","last_unstale":"2024-09-17T05:28:06.251701+0000","last_undegraded":"2024-09-17T05:28:06.251701+0000","last_fullsized":"2024-09-17T05:28:06.251701+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:57.023503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.404175+0000","last_change":"2024-09-17T05:28:03.397060+0000","last_active":"2024-09-17T05:28:05.404175+0000","last_peered":"2024-09-17T05:28:05.404175+0000","last_clean":"2024-09-17T05:28:05.404175+0000","last_became_active":"2024-09-17T05:28:03.396798+0000","last_became_peered":"2024-09-17T05:28:03.396798+0000","last_unstale":"2024-09-17T05:28:05.404175+0000","last_undegraded":"2024-09-17T05:28:05.404175+0000","last_fullsized":"2024-09-17T05:28:05.404175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:37.659154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394569+0000","last_change":"2024-09-17T05:28:04.412984+0000","last_active":"2024-09-17T05:28:06.394569+0000","last_peered":"2024-09-17T05:28:06.394569+0000","last_clean":"2024-09-17T05:28:06.394569+0000","last_became_active":"2024-09-17T05:28:04.412518+0000","last_became_peered":"2024-09-17T05:28:04.412518+0000","last_unstale":"2024-09-17T05:28:06.394569+0000","last_undegraded":"2024-09-17T05:28:06.394569+0000","last_fullsized":"2024-09-17T05:28:06.394569+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:28:07.994494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.411175+0000","last_change":"2024-09-17T05:28:03.402041+0000","last_active":"2024-09-17T05:28:05.411175+0000","last_peered":"2024-09-17T05:28:05.411175+0000","last_clean":"2024-09-17T05:28:05.411175+0000","last_became_active":"2024-09-17T05:28:03.401757+0000","last_became_peered":"2024-09-17T05:28:03.401757+0000","last_unstale":"2024-09-17T05:28:05.411175+0000","last_undegraded":"2024-09-17T05:28:05.411175+0000","last_fullsized":"2024-09-17T05:28:05.411175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:35.456782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.251333+0000","last_change":"2024-09-17T05:28:04.425552+0000","last_active":"2024-09-17T05:28:06.251333+0000","last_peered":"2024-09-17T05:28:06.251333+0000","last_clean":"2024-09-17T05:28:06.251333+0000","last_became_active":"2024-09-17T05:28:04.425273+0000","last_became_peered":"2024-09-17T05:28:04.425273+0000","last_unstale":"2024-09-17T05:28:06.251333+0000","last_undegraded":"2024-09-17T05:28:06.251333+0000","last_fullsized":"2024-09-17T05:28:06.251333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:59.690483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.403997+0000","last_change":"2024-09-17T05:28:03.406099+0000","last_active":"2024-09-17T05:28:05.403997+0000","last_peered":"2024-09-17T05:28:05.403997+0000","last_clean":"2024-09-17T05:28:05.403997+0000","last_became_active":"2024-09-17T05:28:03.405763+0000","last_became_peered":"2024-09-17T05:28:03.405763+0000","last_unstale":"2024-09-17T05:28:05.403997+0000","last_undegraded":"2024-09-17T05:28:05.403997+0000","last_fullsized":"2024-09-17T05:28:05.403997+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:23.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.409499+0000","last_change":"2024-09-17T05:28:04.405966+0000","last_active":"2024-09-17T05:28:05.409499+0000","last_peered":"2024-09-17T05:28:05.409499+0000","last_clean":"2024-09-17T05:28:05.409499+0000","last_became_active":"2024-09-17T05:28:04.405723+0000","last_became_peered":"2024-09-17T05:28:04.405723+0000","last_unstale":"2024-09-17T05:28:05.409499+0000","last_undegraded":"2024-09-17T05:28:05.409499+0000","last_fullsized":"2024-09-17T05:28:05.409499+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:09:42.528631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394078+0000","last_change":"2024-09-17T05:28:03.403069+0000","last_active":"2024-09-17T05:28:06.394078+0000","last_peered":"2024-09-17T05:28:06.394078+0000","last_clean":"2024-09-17T05:28:06.394078+0000","last_became_active":"2024-09-17T05:28:03.402652+0000","last_became_peered":"2024-09-17T05:28:03.402652+0000","last_unstale":"2024-09-17T05:28:06.394078+0000","last_undegraded":"2024-09-17T05:28:06.394078+0000","last_fullsized":"2024-09-17T05:28:06.394078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:21.688445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229432+0000","last_change":"2024-09-17T05:28:04.406867+0000","last_active":"2024-09-17T05:28:07.229432+0000","last_peered":"2024-09-17T05:28:07.229432+0000","last_clean":"2024-09-17T05:28:07.229432+0000","last_became_active":"2024-09-17T05:28:04.406562+0000","last_became_peered":"2024-09-17T05:28:04.406562+0000","last_unstale":"2024-09-17T05:28:07.229432+0000","last_undegraded":"2024-09-17T05:28:07.229432+0000","last_fullsized":"2024-09-17T05:28:07.229432+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:55.451022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.606671+0000","last_change":"2024-09-17T05:28:03.407068+0000","last_active":"2024-09-17T05:28:05.606671+0000","last_peered":"2024-09-17T05:28:05.606671+0000","last_clean":"2024-09-17T05:28:05.606671+0000","last_became_active":"2024-09-17T05:28:03.406674+0000","last_became_peered":"2024-09-17T05:28:03.406674+0000","last_unstale":"2024-09-17T05:28:05.606671+0000","last_undegraded":"2024-09-17T05:28:05.606671+0000","last_fullsized":"2024-09-17T05:28:05.606671+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:37.659154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229567+0000","last_change":"2024-09-17T05:28:04.398338+0000","last_active":"2024-09-17T05:28:07.229567+0000","last_peered":"2024-09-17T05:28:07.229567+0000","last_clean":"2024-09-17T05:28:07.229567+0000","last_became_active":"2024-09-17T05:28:04.398036+0000","last_became_peered":"2024-09-17T05:28:04.398036+0000","last_unstale":"2024-09-17T05:28:07.229567+0000","last_undegraded":"2024-09-17T05:28:07.229567+0000","last_fullsized":"2024-09-17T05:28:07.229567+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:12.008566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410615+0000","last_change":"2024-09-17T05:28:03.402396+0000","last_active":"2024-09-17T05:28:05.410615+0000","last_peered":"2024-09-17T05:28:05.410615+0000","last_clean":"2024-09-17T05:28:05.410615+0000","last_became_active":"2024-09-17T05:28:03.402089+0000","last_became_peered":"2024-09-17T05:28:03.402089+0000","last_unstale":"2024-09-17T05:28:05.410615+0000","last_undegraded":"2024-09-17T05:28:05.410615+0000","last_fullsized":"2024-09-17T05:28:05.410615+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:45.934108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:05.410638+0000","last_change":"2024-09-17T05:28:04.407978+0000","last_active":"2024-09-17T05:28:05.410638+0000","last_peered":"2024-09-17T05:28:05.410638+0000","last_clean":"2024-09-17T05:28:05.410638+0000","last_became_active":"2024-09-17T05:28:04.407717+0000","last_became_peered":"2024-09-17T05:28:04.407717+0000","last_unstale":"2024-09-17T05:28:05.410638+0000","last_undegraded":"2024-09-17T05:28:05.410638+0000","last_fullsized":"2024-09-17T05:28:05.410638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:03.373314+0000","last_clean_scrub_stamp":"2024-09-17T05:28:03.373314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:52.263630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:06.394430+0000","last_change":"2024-09-17T05:28:03.408678+0000","last_active":"2024-09-17T05:28:06.394430+0000","last_peered":"2024-09-17T05:28:06.394430+0000","last_clean":"2024-09-17T05:28:06.394430+0000","last_became_active":"2024-09-17T05:28:03.408375+0000","last_became_peered":"2024-09-17T05:28:03.408375+0000","last_unstale":"2024-09-17T05:28:06.394430+0000","last_undegraded":"2024-09-17T05:28:06.394430+0000","last_fullsized":"2024-09-17T05:28:06.394430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:28:02.370138+0000","last_clean_scrub_stamp":"2024-09-17T05:28:02.370138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:03.076561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":157,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T05:28:07.229586+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:28:07.229586+0000","last_peered":"2024-09-17T05:28:07.229586+0000","last_clean":"2024-09-17T05:28:07.229586+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:28:07.229586+0000","last_undegraded":"2024-09-17T05:28:07.229586+0000","last_fullsized":"2024-09-17T05:28:07.229586+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":536576,"data_stored":215208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35876,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335964,"statfs":{"total":96636764160,"available":96600027136,"internally_reserved":0,"allocated":540672,"data_stored":215790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35888,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335952,"statfs":{"total":96636764160,"available":96600014848,"internally_reserved":0,"allocated":552960,"data_stored":216878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":216786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":215264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":805612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":540672,"data_stored":215846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1130496,"data_stored":805564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:28:13.450 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:28:13.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- 172.21.15.122:0/126489835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d80b4bd0 msgr2=0x7f69d815e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 --2- 172.21.15.122:0/126489835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d80b4bd0 0x7f69d815e900 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f69d806aab0 tx=0x7f69d001c5d0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- 172.21.15.122:0/126489835 shutdown_connections 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 --2- 172.21.15.122:0/126489835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d80b4bd0 0x7f69d815e900 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 --2- 172.21.15.122:0/126489835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d806e810 0x7f69d80cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 --2- 172.21.15.122:0/126489835 >> v2:172.21.15.173:3300/0 conn(0x7f69d815ade0 0x7f69d80cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- 172.21.15.122:0/126489835 >> 172.21.15.122:0/126489835 conn(0x7f69d80b18f0 msgr2=0x7f69d80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- 172.21.15.122:0/126489835 shutdown_connections 2024-09-17T05:28:13.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- 172.21.15.122:0/126489835 wait complete. 2024-09-17T05:28:13.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 Processor -- start 2024-09-17T05:28:13.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- start start 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d806e810 0x7f69d8143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d80b4bd0 0x7f69d81538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d815ade0 0x7f69d80ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69d806f040 con 0x7f69d815ade0 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69d806eec0 con 0x7f69d80b4bd0 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69de600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f69d806f1c0 con 0x7f69d806e810 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d815ade0 0x7f69d80ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d815ade0 0x7f69d80ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60298/0 (socket says 172.21.15.122:60298) 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7e00640 1 -- 172.21.15.122:0/1662079916 learned_addr learned my addr 172.21.15.122:0/1662079916 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:13.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7400640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d806e810 0x7f69d8143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:13.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69cea00640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d80b4bd0 0x7f69d81538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:13.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7e00640 1 -- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d806e810 msgr2=0x7f69d8143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7e00640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d806e810 0x7f69d8143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7e00640 1 -- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d80b4bd0 msgr2=0x7f69d81538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7e00640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d80b4bd0 0x7f69d81538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.639+0000 7f69d7e00640 1 -- 172.21.15.122:0/1662079916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d8143ad0 con 0x7f69d815ade0 2024-09-17T05:28:13.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69d7e00640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d815ade0 0x7f69d80ce460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f69d001ac80 tx=0x7f69d00a6460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:13.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69cfe00640 1 -- 172.21.15.122:0/1662079916 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d00ac070 con 0x7f69d815ade0 2024-09-17T05:28:13.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69cfe00640 1 -- 172.21.15.122:0/1662079916 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69d00053b0 con 0x7f69d815ade0 2024-09-17T05:28:13.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d81521d0 con 0x7f69d815ade0 2024-09-17T05:28:13.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69cfe00640 1 -- 172.21.15.122:0/1662079916 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d00046d0 con 0x7f69d815ade0 2024-09-17T05:28:13.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d8153440 con 0x7f69d815ade0 2024-09-17T05:28:13.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d8153640 con 0x7f69d815ade0 2024-09-17T05:28:13.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69cfe00640 1 -- 172.21.15.122:0/1662079916 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f69d00af070 con 0x7f69d815ade0 2024-09-17T05:28:13.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69cfe00640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69b4073560 0x7f69b4075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:13.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.643+0000 7f69cfe00640 1 -- 172.21.15.122:0/1662079916 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f69d012e8a0 con 0x7f69d815ade0 2024-09-17T05:28:13.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.647+0000 7f69d7400640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69b4073560 0x7f69b4075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:13.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.647+0000 7f69d7400640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69b4073560 0x7f69b4075a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69d81f92b0 tx=0x7f69c8009630 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:13.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.647+0000 7f69cfe00640 1 -- 172.21.15.122:0/1662079916 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f69d8153640 con 0x7f69d815ade0 2024-09-17T05:28:13.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.875+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f69d815e8e0 con 0x7f69d815ade0 2024-09-17T05:28:13.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.879+0000 7f69cfe00640 1 -- 172.21.15.122:0/1662079916 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f69d012e020 con 0x7f69d815ade0 2024-09-17T05:28:13.881 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:28:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:28:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:28:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:28:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:28:13.881 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:28:13.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69b4073560 msgr2=0x7f69b4075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69b4073560 0x7f69b4075a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69d81f92b0 tx=0x7f69c8009630 comp rx=0 tx=0).stop 2024-09-17T05:28:13.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d815ade0 msgr2=0x7f69d80ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:13.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d815ade0 0x7f69d80ce460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f69d001ac80 tx=0x7f69d00a6460 comp rx=0 tx=0).stop 2024-09-17T05:28:13.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 shutdown_connections 2024-09-17T05:28:13.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69b4073560 0x7f69b4075a20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69d815ade0 0x7f69d80ce460 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d80b4bd0 0x7f69d81538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 --2- 172.21.15.122:0/1662079916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69d806e810 0x7f69d8143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:13.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 >> 172.21.15.122:0/1662079916 conn(0x7f69d80b18f0 msgr2=0x7f69d81589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:13.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 shutdown_connections 2024-09-17T05:28:13.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:13.883+0000 7f69de600640 1 -- 172.21.15.122:0/1662079916 wait complete. 2024-09-17T05:28:13.902 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:28:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 -- 172.21.15.122:0/380339289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c0cc2a0 msgr2=0x7f6d3c0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 --2- 172.21.15.122:0/380339289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c0cc2a0 0x7f6d3c0b3130 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0b2680 tx=0x7f6d2401c830 comp rx=0 tx=0).stop 2024-09-17T05:28:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 -- 172.21.15.122:0/380339289 shutdown_connections 2024-09-17T05:28:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 --2- 172.21.15.122:0/380339289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d3c0b4cf0 0x7f6d3c15e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 --2- 172.21.15.122:0/380339289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c0cc2a0 0x7f6d3c0b3130 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 --2- 172.21.15.122:0/380339289 >> v2:172.21.15.173:3300/0 conn(0x7f6d3c15e260 0x7f6d3c0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 -- 172.21.15.122:0/380339289 >> 172.21.15.122:0/380339289 conn(0x7f6d3c0b18f0 msgr2=0x7f6d3c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 -- 172.21.15.122:0/380339289 shutdown_connections 2024-09-17T05:28:14.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 -- 172.21.15.122:0/380339289 wait complete. 2024-09-17T05:28:14.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 Processor -- start 2024-09-17T05:28:14.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.043+0000 7f6d43a00640 1 -- start start 2024-09-17T05:28:14.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d3c0b4cf0 0x7f6d3c1f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d3c15e260 0x7f6d3c165390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c14b570 0x7f6d3c0ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c06f460 con 0x7f6d3c14b570 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c06f2e0 con 0x7f6d3c15e260 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c06f5e0 con 0x7f6d3c0b4cf0 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c14b570 0x7f6d3c0ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c14b570 0x7f6d3c0ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60316/0 (socket says 172.21.15.122:60316) 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 -- 172.21.15.122:0/398155655 learned_addr learned my addr 172.21.15.122:0/398155655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41000640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d3c0b4cf0 0x7f6d3c1f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d33e00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d3c15e260 0x7f6d3c165390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 -- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d3c0b4cf0 msgr2=0x7f6d3c1f99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d3c0b4cf0 0x7f6d3c1f99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 -- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d3c15e260 msgr2=0x7f6d3c165390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d3c15e260 0x7f6d3c165390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 -- 172.21.15.122:0/398155655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c15f900 con 0x7f6d3c14b570 2024-09-17T05:28:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d41a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c14b570 0x7f6d3c0ae600 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400b9a0 tx=0x7f6d3400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d39600640 1 -- 172.21.15.122:0/398155655 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34019070 con 0x7f6d3c14b570 2024-09-17T05:28:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d39600640 1 -- 172.21.15.122:0/398155655 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d3400ccb0 con 0x7f6d3c14b570 2024-09-17T05:28:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d39600640 1 -- 172.21.15.122:0/398155655 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d340146a0 con 0x7f6d3c14b570 2024-09-17T05:28:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c0ad1d0 con 0x7f6d3c14b570 2024-09-17T05:28:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c163140 con 0x7f6d3c14b570 2024-09-17T05:28:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c06faf0 con 0x7f6d3c14b570 2024-09-17T05:28:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.047+0000 7f6d39600640 1 -- 172.21.15.122:0/398155655 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f6d3401c070 con 0x7f6d3c14b570 2024-09-17T05:28:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.051+0000 7f6d39600640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d18073560 0x7f6d18075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.051+0000 7f6d39600640 1 -- 172.21.15.122:0/398155655 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f6d340944d0 con 0x7f6d3c14b570 2024-09-17T05:28:14.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.051+0000 7f6d41000640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d18073560 0x7f6d18075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.051+0000 7f6d41000640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d18073560 0x7f6d18075a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0098f0 tx=0x7f6d2c006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:14.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.055+0000 7f6d39600640 1 -- 172.21.15.122:0/398155655 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6d3c15f900 con 0x7f6d3c14b570 2024-09-17T05:28:14.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.255+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d3c1589c0 con 0x7f6d3c14b570 2024-09-17T05:28:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.255+0000 7f6d39600640 1 -- 172.21.15.122:0/398155655 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5232 (secure 0 0 0) 0x7f6d3405e9e0 con 0x7f6d3c14b570 2024-09-17T05:28:14.258 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:28:14.258 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"btime":"2024-09-17T05:28:07:412553+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3520614210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3520614210},{"type":"v1","addr":"172.21.15.122:6834","nonce":3520614210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5308,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2442281734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2442281734},{"type":"v1","addr":"172.21.15.173:6836","nonce":2442281734}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/638961094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":638961094},{"type":"v1","addr":"172.21.15.122:6835","nonce":638961094}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:03.784287+0000","modified":"2024-09-17T05:28:06.804180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/4117982032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4117982032},{"type":"v1","addr":"172.21.15.173:6837","nonce":4117982032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5388,"qdb_cluster":[5388]},"id":3}]} 2024-09-17T05:28:14.258 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 26 2024-09-17T05:28:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d18073560 msgr2=0x7f6d18075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d18073560 0x7f6d18075a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0098f0 tx=0x7f6d2c006f90 comp rx=0 tx=0).stop 2024-09-17T05:28:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c14b570 msgr2=0x7f6d3c0ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c14b570 0x7f6d3c0ae600 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400b9a0 tx=0x7f6d3400be60 comp rx=0 tx=0).stop 2024-09-17T05:28:14.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 shutdown_connections 2024-09-17T05:28:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d18073560 0x7f6d18075a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d3c14b570 0x7f6d3c0ae600 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d3c15e260 0x7f6d3c165390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 --2- 172.21.15.122:0/398155655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d3c0b4cf0 0x7f6d3c1f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 >> 172.21.15.122:0/398155655 conn(0x7f6d3c0b18f0 msgr2=0x7f6d3c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 shutdown_connections 2024-09-17T05:28:14.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.259+0000 7f6d43a00640 1 -- 172.21.15.122:0/398155655 wait complete. 2024-09-17T05:28:14.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:28:03.784287+0000', 'modified': '2024-09-17T05:28:06.804180+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5388}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5388': {'gid': 5388, 'name': 'd', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6837/4117982032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6835', 'nonce': 4117982032}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 4117982032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5388, 'qdb_cluster': [5388]} 2024-09-17T05:28:14.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:28:14.682 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:28:14.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.827+0000 7fb5b6800640 1 -- 172.21.15.122:0/3861589927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5b00d17b0 msgr2=0x7fb5b006fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.827+0000 7fb5b6800640 1 --2- 172.21.15.122:0/3861589927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5b00d17b0 0x7fb5b006fa40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb598009900 tx=0x7fb59801c850 comp rx=0 tx=0).stop 2024-09-17T05:28:14.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.827+0000 7fb5b6800640 1 -- 172.21.15.122:0/3861589927 shutdown_connections 2024-09-17T05:28:14.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.827+0000 7fb5b6800640 1 --2- 172.21.15.122:0/3861589927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5b00b3ae0 0x7fb5b006ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.827+0000 7fb5b6800640 1 --2- 172.21.15.122:0/3861589927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5b00d17b0 0x7fb5b006fa40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.827+0000 7fb5b6800640 1 -- 172.21.15.122:0/3861589927 >> 172.21.15.122:0/3861589927 conn(0x7fb5b00d0c90 msgr2=0x7fb5b00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:14.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.827+0000 7fb5b6800640 1 -- 172.21.15.122:0/3861589927 shutdown_connections 2024-09-17T05:28:14.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.827+0000 7fb5b6800640 1 -- 172.21.15.122:0/3861589927 wait complete. 2024-09-17T05:28:14.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 Processor -- start 2024-09-17T05:28:14.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 -- start start 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5b00b3ae0 0x7fb5b00b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5b00d17b0 0x7fb5b01f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5b015be80 0x7fb5b0152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0169490 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0169310 con 0x7fb5b015be80 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b0169610 con 0x7fb5b00d17b0 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5b00b3ae0 0x7fb5b00b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5b00b3ae0 0x7fb5b00b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60328/0 (socket says 172.21.15.122:60328) 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 -- 172.21.15.122:0/1496966931 learned_addr learned my addr 172.21.15.122:0/1496966931 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5b00d17b0 0x7fb5b01f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5afe00640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5b015be80 0x7fb5b0152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 -- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5b00d17b0 msgr2=0x7fb5b01f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5b00d17b0 0x7fb5b01f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 -- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5b015be80 msgr2=0x7fb5b0152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5b015be80 0x7fb5b0152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 -- 172.21.15.122:0/1496966931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b00c6d80 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5af400640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5b00b3ae0 0x7fb5b00b3080 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000c3f0 tx=0x7fb5a000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:14.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5a7e00640 1 -- 172.21.15.122:0/1496966931 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0017070 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5a7e00640 1 -- 172.21.15.122:0/1496966931 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5a00053c0 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 -- 172.21.15.122:0/1496966931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b00c7040 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5a7e00640 1 -- 172.21.15.122:0/1496966931 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0003860 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.831+0000 7fb5b6800640 1 -- 172.21.15.122:0/1496966931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b0153380 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5a7e00640 1 -- 172.21.15.122:0/1496966931 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fb5a001a070 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5a7e00640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb58c073560 0x7fb58c075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5b9bd5000 1 Processor -- start 2024-09-17T05:28:14.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5a7e00640 1 -- 172.21.15.122:0/1496966931 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb5a00939d0 con 0x7fb5b00b3ae0 2024-09-17T05:28:14.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5b9bd5000 1 -- start start 2024-09-17T05:28:14.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb58c073560 0x7fb58c075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7a078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a07e10 0x5603e7a081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603e79283f0 con 0x5603e7a539d0 2024-09-17T05:28:14.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5603e7865b20 con 0x5603e7a542f0 2024-09-17T05:28:14.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5603e7854f50 con 0x5603e7a07e10 2024-09-17T05:28:14.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5af400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5af400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a53db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60334/0 (socket says 172.21.15.122:60334) 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5af400640 1 -- 172.21.15.122:0/1566421428 learned_addr learned my addr 172.21.15.122:0/1566421428 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb58c073560 0x7fb58c075a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb5980af000 tx=0x7fb59801b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7a078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5afe00640 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a07e10 0x5603e7a081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5af400640 1 -- 172.21.15.122:0/1566421428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a07e10 msgr2=0x5603e7a081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.835+0000 7fb5af400640 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a07e10 0x5603e7a081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5af400640 1 -- 172.21.15.122:0/1566421428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 msgr2=0x5603e7a078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5af400640 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7a078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5af400640 1 -- 172.21.15.122:0/1566421428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e781c590 con 0x5603e7a539d0 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5af400640 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a53db0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb5a005de60 tx=0x7fb5a005d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148bd2dafd515b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb59fe00640 1 -- 172.21.15.122:0/1566421428 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a005d330 con 0x5603e7a539d0 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb59fe00640 1 -- 172.21.15.122:0/1566421428 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5a00677b0 con 0x5603e7a539d0 2024-09-17T05:28:14.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb59fe00640 1 -- 172.21.15.122:0/1566421428 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0059a40 con 0x5603e7a539d0 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/1566421428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 msgr2=0x5603e7a53db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a53db0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb5a005de60 tx=0x7fb5a005d700 comp rx=0 tx=0).stop 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/1566421428 shutdown_connections 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a07e10 0x5603e7a081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7a078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/1566421428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a53db0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/1566421428 >> 172.21.15.122:0/1566421428 conn(0x5603e78f1130 msgr2=0x5603e7a36e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/1566421428 shutdown_connections 2024-09-17T05:28:14.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/1566421428 wait complete. 2024-09-17T05:28:14.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- start start 2024-09-17T05:28:14.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 Processor -- start 2024-09-17T05:28:14.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a07e10 0x5603e7a346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a34e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 0x5603e7a353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603e77ef5d0 con 0x5603e7a539d0 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5603e7858d60 con 0x5603e7a07e10 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5603e77fb500 con 0x5603e7a542f0 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5b9bd5000 10 client.0 _ll_register_callbacks cb 0x5603e77ff810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a34e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a34e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60338/0 (socket says 172.21.15.122:60338) 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 -- 172.21.15.122:0/2665222255 learned_addr learned my addr 172.21.15.122:0/2665222255 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5af400640 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a07e10 0x5603e7a346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5afe00640 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 0x5603e7a353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 -- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 msgr2=0x5603e7a353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 0x5603e7a353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 -- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a07e10 msgr2=0x5603e7a346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a07e10 0x5603e7a346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 -- 172.21.15.122:0/2665222255 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e77f3170 con 0x5603e7a539d0 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a34e70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb5980b0040 tx=0x7fb59801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e00b64e29a2f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.839+0000 7fb59cc00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 1 -- 172.21.15.122:0/2665222255 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5980b2040 con 0x5603e7a539d0 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 1 -- 172.21.15.122:0/2665222255 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb5980c1040 con 0x5603e7a539d0 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2665222255 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603e77ed0f0 con 0x5603e7a539d0 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 1 -- 172.21.15.122:0/2665222255 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb5980cf050 con 0x5603e7a539d0 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 10 client.5529 fetch_fsmap learned FSMap version 26 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2665222255 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5603e78d9ef0 con 0x5603e7a539d0 2024-09-17T05:28:14.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 1 -- 172.21.15.122:0/2665222255 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5603e77ed0f0 con 0x5603e7a539d0 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 1 -- 172.21.15.122:0/2665222255 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fb5980d2020 con 0x5603e7a539d0 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 10 client.5529 fetch_fsmap finished waiting for FSMap version 26 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 10 client.5529 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 20 client.5529 populate_metadata read hostname 'smithi122' 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] conn(0x5603e7a360f0 0x5603e7a65c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 4 client.5529 mds_command: new command op to 5388 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2665222255 --> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603e7867a70 con 0x5603e7a360f0 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5af400640 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] conn(0x5603e7a360f0 0x5603e7a65c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 10 client.5529 ms_handle_connect on v2:172.21.15.173:6835/4117982032 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5af400640 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] conn(0x5603e7a360f0 0x5603e7a65c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56705645789fdfda server_cookie=25d39c9bb84f818c in_seq=0 out_seq=0 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 1 -- 172.21.15.122:0/2665222255 <== mds.0 v2:172.21.15.173:6835/4117982032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603e7867a70 con 0x5603e7a360f0 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 10 client.5529 handle_command_reply: tid=0 2024-09-17T05:28:14.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 client.5529 shutdown 2024-09-17T05:28:14.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2665222255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 msgr2=0x5603e7a34e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a34e70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb5980b0040 tx=0x7fb59801c850 comp rx=0 tx=0).stop 2024-09-17T05:28:14.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2665222255 shutdown_connections 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] conn(0x5603e7a360f0 0x5603e7a65c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 0x5603e7a353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a539d0 0x5603e7a34e70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2665222255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a07e10 0x5603e7a346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb59cc00640 0 client.5529 ms_handle_reset on v2:172.21.15.173:6835/4117982032 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2665222255 >> 172.21.15.122:0/2665222255 conn(0x5603e78f1130 msgr2=0x5603e7a06fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2665222255 shutdown_connections 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2665222255 wait complete. 2024-09-17T05:28:14.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.843+0000 7fb5b9bd5000 20 client.5529 trim_cache size 0 max 16384 2024-09-17T05:28:14.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5b9bd5000 1 Processor -- start 2024-09-17T05:28:14.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5b9bd5000 1 -- start start 2024-09-17T05:28:14.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a07e10 0x5603e7a52cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a539d0 0x5603e7a53200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 0x5603e7a65c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b01fc570 con 0x5603e7a07e10 2024-09-17T05:28:14.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b014eef0 con 0x5603e7a539d0 2024-09-17T05:28:14.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00c4d20 con 0x5603e7a542f0 2024-09-17T05:28:14.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a07e10 0x5603e7a52cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a07e10 0x5603e7a52cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60346/0 (socket says 172.21.15.122:60346) 2024-09-17T05:28:14.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 -- 172.21.15.122:0/283840437 learned_addr learned my addr 172.21.15.122:0/283840437 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 -- 172.21.15.122:0/283840437 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 msgr2=0x5603e7a65c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 --2- 172.21.15.122:0/283840437 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 0x5603e7a65c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 -- 172.21.15.122:0/283840437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a539d0 msgr2=0x5603e7a53200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 --2- 172.21.15.122:0/283840437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a539d0 0x5603e7a53200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 -- 172.21.15.122:0/283840437 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603e78b0d90 con 0x5603e7a07e10 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb5af400640 1 --2- 172.21.15.122:0/283840437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a07e10 0x5603e7a52cc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb5a005e090 tx=0x7fb5a0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a421c41e3920c038 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb59f400640 1 -- 172.21.15.122:0/283840437 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a005d130 con 0x5603e7a07e10 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb59f400640 1 -- 172.21.15.122:0/283840437 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5a005d2d0 con 0x5603e7a07e10 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.863+0000 7fb59f400640 1 -- 172.21.15.122:0/283840437 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0059a60 con 0x5603e7a07e10 2024-09-17T05:28:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/283840437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a07e10 msgr2=0x5603e7a52cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/283840437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a07e10 0x5603e7a52cc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb5a005e090 tx=0x7fb5a0000bd0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/283840437 shutdown_connections 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/283840437 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a542f0 0x5603e7a65c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/283840437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a539d0 0x5603e7a53200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/283840437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7a07e10 0x5603e7a52cc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/283840437 >> 172.21.15.122:0/283840437 conn(0x5603e78f1130 msgr2=0x5603e7a525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/283840437 shutdown_connections 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/283840437 wait complete. 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- start start 2024-09-17T05:28:14.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 Processor -- start 2024-09-17T05:28:14.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a539d0 0x5603e7aca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7acada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7acb2e0 0x5603e7a67fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b014e8a0 con 0x5603e7acb2e0 2024-09-17T05:28:14.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5603e7a6ace0 con 0x5603e7a542f0 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5603e785d390 con 0x5603e7a539d0 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 10 client.0 _ll_register_callbacks cb 0x7fb5a80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7acb2e0 0x5603e7a67fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7acb2e0 0x5603e7a67fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60360/0 (socket says 172.21.15.122:60360) 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 -- 172.21.15.122:0/2733509439 learned_addr learned my addr 172.21.15.122:0/2733509439 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7acada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5af400640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a539d0 0x5603e7aca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 -- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a539d0 msgr2=0x5603e7aca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a539d0 0x5603e7aca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 -- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 msgr2=0x5603e7acada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7acada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 -- 172.21.15.122:0/2733509439 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603e781ba90 con 0x5603e7acb2e0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5af400640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a539d0 0x5603e7aca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5a6a00640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7acada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5afe00640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7acb2e0 0x5603e7a67fd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8010d70 tx=0x7fb5a800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47fefaee4a5400c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb59cc00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb59cc00640 1 -- 172.21.15.122:0/2733509439 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8009dd0 con 0x5603e7acb2e0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb59cc00640 1 -- 172.21.15.122:0/2733509439 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fb5a801eaf0 con 0x5603e7acb2e0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2733509439 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a005d2d0 con 0x5603e7acb2e0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb59cc00640 1 -- 172.21.15.122:0/2733509439 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb5a801ecf0 con 0x5603e7acb2e0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 10 client.5535 fetch_fsmap learned FSMap version 26 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2733509439 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5603e77fa050 con 0x5603e7acb2e0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb59cc00640 1 -- 172.21.15.122:0/2733509439 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5a005d2d0 con 0x5603e7acb2e0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb59cc00640 1 -- 172.21.15.122:0/2733509439 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fb5a801b020 con 0x5603e7acb2e0 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 10 client.5535 fetch_fsmap finished waiting for FSMap version 26 2024-09-17T05:28:14.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 10 client.5535 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 20 client.5535 populate_metadata read hostname 'smithi122' 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] conn(0x5603e7ad1ab0 0x5603e7a68c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 4 client.5535 mds_command: new command op to 5388 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.867+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2733509439 --> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] -- command(tid 0: {"prefix": "status"}) -- 0x5603e7863520 con 0x5603e7ad1ab0 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5af400640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] conn(0x5603e7ad1ab0 0x5603e7a68c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5af400640 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] conn(0x5603e7ad1ab0 0x5603e7a68c40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b8f1d45936ac44 server_cookie=3e0d63bac9e84801 in_seq=0 out_seq=0 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb59cc00640 10 client.5535 ms_handle_connect on v2:172.21.15.173:6835/4117982032 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb59cc00640 1 -- 172.21.15.122:0/2733509439 <== mds.0 v2:172.21.15.173:6835/4117982032 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5603e7863520 con 0x5603e7ad1ab0 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb59cc00640 10 client.5535 handle_command_reply: tid=0 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 client.5535 shutdown 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2733509439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7acb2e0 msgr2=0x5603e7a67fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7acb2e0 0x5603e7a67fd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8010d70 tx=0x7fb5a800f790 comp rx=0 tx=0).stop 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2733509439 shutdown_connections 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:6835/4117982032,v1:172.21.15.173:6837/4117982032] conn(0x5603e7ad1ab0 0x5603e7a68c40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603e7acb2e0 0x5603e7a67fd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603e7a542f0 0x5603e7acada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 --2- 172.21.15.122:0/2733509439 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5603e7a539d0 0x5603e7aca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb59cc00640 0 client.5535 ms_handle_reset on v2:172.21.15.173:6835/4117982032 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2733509439 >> 172.21.15.122:0/2733509439 conn(0x5603e78f1130 msgr2=0x5603e7a525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2733509439 shutdown_connections 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 1 -- 172.21.15.122:0/2733509439 wait complete. 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b9bd5000 20 client.5535 trim_cache size 0 max 16384 2024-09-17T05:28:14.875 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 5388, 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 10.489225619000001, 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 26, 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 46, 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 46, 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 14.982527421 2024-09-17T05:28:14.876 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:28:14.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 -- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb58c073560 msgr2=0x7fb58c075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb58c073560 0x7fb58c075a20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb5980af000 tx=0x7fb59801b090 comp rx=0 tx=0).stop 2024-09-17T05:28:14.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 -- 172.21.15.122:0/1496966931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5b00b3ae0 msgr2=0x7fb5b00b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:14.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5b00b3ae0 0x7fb5b00b3080 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000c3f0 tx=0x7fb5a000c7f0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 -- 172.21.15.122:0/1496966931 shutdown_connections 2024-09-17T05:28:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb58c073560 0x7fb58c075a20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5b015be80 0x7fb5b0152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5b00d17b0 0x7fb5b01f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 --2- 172.21.15.122:0/1496966931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5b00b3ae0 0x7fb5b00b3080 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 -- 172.21.15.122:0/1496966931 >> 172.21.15.122:0/1496966931 conn(0x7fb5b00d0c90 msgr2=0x7fb5b0155290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 -- 172.21.15.122:0/1496966931 shutdown_connections 2024-09-17T05:28:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:14.871+0000 7fb5b6800640 1 -- 172.21.15.122:0/1496966931 wait complete. 2024-09-17T05:28:14.892 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:28:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- 172.21.15.122:0/2520066690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 msgr2=0x7f7668003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 --2- 172.21.15.122:0/2520066690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 0x7f7668003e30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f767400c2e0 tx=0x7f767401cf30 comp rx=0 tx=0).stop 2024-09-17T05:28:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7682800640 1 -- 172.21.15.122:0/2520066690 reap_dead start 2024-09-17T05:28:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- 172.21.15.122:0/2520066690 shutdown_connections 2024-09-17T05:28:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- 172.21.15.122:0/2520066690 >> 172.21.15.122:0/2520066690 conn(0x7f76840d1e30 msgr2=0x7f76840d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- 172.21.15.122:0/2520066690 shutdown_connections 2024-09-17T05:28:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- 172.21.15.122:0/2520066690 wait complete. 2024-09-17T05:28:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 Processor -- start 2024-09-17T05:28:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- start start 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 0x7f76841570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76840cdbe0 0x7f768415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76840ce060 0x7f768415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76840b1f60 con 0x7f7668003a10 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f76840b1de0 con 0x7f76840ce060 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7688e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f76840b20e0 con 0x7f76840cdbe0 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7681e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 0x7f76841570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7681e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 0x7f76841570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60386/0 (socket says 172.21.15.122:60386) 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7681e00640 1 -- 172.21.15.122:0/1491143289 learned_addr learned my addr 172.21.15.122:0/1491143289 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7679400640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76840cdbe0 0x7f768415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.035+0000 7f7682800640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76840ce060 0x7f768415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7681e00640 1 -- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76840cdbe0 msgr2=0x7f768415b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7681e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76840cdbe0 0x7f768415b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7681e00640 1 -- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76840ce060 msgr2=0x7f768415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7681e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76840ce060 0x7f768415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7681e00640 1 -- 172.21.15.122:0/1491143289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768409fe50 con 0x7f7668003a10 2024-09-17T05:28:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7681e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 0x7f76841570e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f767400c2e0 tx=0x7f76740010c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f767aa00640 1 -- 172.21.15.122:0/1491143289 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674006440 con 0x7f7668003a10 2024-09-17T05:28:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f767aa00640 1 -- 172.21.15.122:0/1491143289 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7674003600 con 0x7f7668003a10 2024-09-17T05:28:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f767aa00640 1 -- 172.21.15.122:0/1491143289 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674003a30 con 0x7f7668003a10 2024-09-17T05:28:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76841f5760 con 0x7f7668003a10 2024-09-17T05:28:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841f5d20 con 0x7f7668003a10 2024-09-17T05:28:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76840b0130 con 0x7f7668003a10 2024-09-17T05:28:15.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f767aa00640 1 -- 172.21.15.122:0/1491143289 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f76740b8020 con 0x7f7668003a10 2024-09-17T05:28:15.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f767aa00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7660073560 0x7f7660075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:15.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.039+0000 7f767aa00640 1 -- 172.21.15.122:0/1491143289 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f767412e710 con 0x7f7668003a10 2024-09-17T05:28:15.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.043+0000 7f7679400640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7660073560 0x7f7660075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:15.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.043+0000 7f7679400640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7660073560 0x7f7660075a20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7684071490 tx=0x7f766c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:15.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.047+0000 7f767aa00640 1 -- 172.21.15.122:0/1491143289 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f76740c1070 con 0x7f7668003a10 2024-09-17T05:28:15.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7684166930 con 0x7f7668003a10 2024-09-17T05:28:15.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f767aa00640 1 -- 172.21.15.122:0/1491143289 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f7674016d10 con 0x7f7668003a10 2024-09-17T05:28:15.247 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:28:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7660073560 msgr2=0x7f7660075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7660073560 0x7f7660075a20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7684071490 tx=0x7f766c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:28:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 msgr2=0x7f76841570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 0x7f76841570e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f767400c2e0 tx=0x7f76740010c0 comp rx=0 tx=0).stop 2024-09-17T05:28:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 shutdown_connections 2024-09-17T05:28:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7660073560 0x7f7660075a20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76840ce060 0x7f768415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76840cdbe0 0x7f768415b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 --2- 172.21.15.122:0/1491143289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7668003a10 0x7f76841570e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 >> 172.21.15.122:0/1491143289 conn(0x7f76840d1e30 msgr2=0x7f768415a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 shutdown_connections 2024-09-17T05:28:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15.247+0000 7f7688e00640 1 -- 172.21.15.122:0/1491143289 wait complete. 2024-09-17T05:28:15.268 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:28:15.268 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:28:15.268 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:28:15.269 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:28:15.269 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:15.269 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:28:15.269 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:28:15.269 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:28:15.269 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:15.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:15.269 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:28:15.278 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:28:15.278 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:28:15.278 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:28:15.278 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:15.278 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:28:15.278 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:15.278 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:15.278 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42272sec preferred_lft 42272sec 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:28:15.279 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:15.280 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:28:15.281 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:28:15.281 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:28:15.281 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:28:15.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:15.282 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:28:15.343 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:28:15.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:15.345 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:28:15.400 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:28:15.400 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:28:15.400 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:28:15.400 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:28:15.401 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:28:15.401 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:28:15.401 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:28:15.401 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:28:15.401 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:28:15.401 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:28:15.401 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:28:15.401 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:28:15.401 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:28:15.401 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:28:15.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:28:15.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:28:15.402 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:28:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:28:15.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:28:15.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:15.533 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:28:15.542 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:28:15.543 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:28:15.543 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:15.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:15.543 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:28:15.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:28:15.612 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:15.661 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:28:15.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:28:15.662 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:15.717 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:28:15.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:15.717 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:15.804 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:15.805 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:15.805 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:28:15.819 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:15.819 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:15.820 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:28:15.820 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:28:15.820 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:28:15.830 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:28:15.830 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:28:15.831 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:28:15.831 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:28:15.831 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:28:15.831 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:28:15.831 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:28:15.831 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:28:15.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:28:15.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:15.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:28:15.903 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:28:15.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:28:16.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:28:16.016 INFO:teuthology.orchestra.run.smithi122.stdout:client5446 2024-09-17T05:28:16.017 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:28:16.017 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:16.026 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:28:16.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd181200640 1 -- 172.21.15.122:0/3408030447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd160004b20 msgr2=0x7fd160004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:16.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd181200640 1 --2- 172.21.15.122:0/3408030447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd160004b20 0x7fd160004f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009f10 tx=0x7fd16c01d510 comp rx=0 tx=0).stop 2024-09-17T05:28:16.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd17a800640 1 -- 172.21.15.122:0/3408030447 reap_dead start 2024-09-17T05:28:16.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd181200640 1 -- 172.21.15.122:0/3408030447 shutdown_connections 2024-09-17T05:28:16.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd181200640 1 -- 172.21.15.122:0/3408030447 >> 172.21.15.122:0/3408030447 conn(0x7fd17c0cfbc0 msgr2=0x7fd17c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:16.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd181200640 1 -- 172.21.15.122:0/3408030447 shutdown_connections 2024-09-17T05:28:16.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd181200640 1 -- 172.21.15.122:0/3408030447 wait complete. 2024-09-17T05:28:16.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd181200640 1 Processor -- start 2024-09-17T05:28:16.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.215+0000 7fd181200640 1 -- start start 2024-09-17T05:28:16.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd160004b20 0x7fd17c161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:16.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd17c1640a0 0x7fd17c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:16.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd17c0b4900 0x7fd17c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:16.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c06f9f0 con 0x7fd17c0b4900 2024-09-17T05:28:16.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c06f870 con 0x7fd160004b20 2024-09-17T05:28:16.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd17c06fb70 con 0x7fd17c1640a0 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd17c0b4900 0x7fd17c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd17c0b4900 0x7fd17c0b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60404/0 (socket says 172.21.15.122:60404) 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 -- 172.21.15.122:0/4055854112 learned_addr learned my addr 172.21.15.122:0/4055854112 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd179e00640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd160004b20 0x7fd17c161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd171400640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd17c1640a0 0x7fd17c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 -- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd17c1640a0 msgr2=0x7fd17c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd17c1640a0 0x7fd17c153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 -- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd160004b20 msgr2=0x7fd17c161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd160004b20 0x7fd17c161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 -- 172.21.15.122:0/4055854112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c09ae60 con 0x7fd17c0b4900 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd17a800640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd17c0b4900 0x7fd17c0b4d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0cc1f0 tx=0x7fd174011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:16.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd172a00640 1 -- 172.21.15.122:0/4055854112 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174004440 con 0x7fd17c0b4900 2024-09-17T05:28:16.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd172a00640 1 -- 172.21.15.122:0/4055854112 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd17400f8f0 con 0x7fd17c0b4900 2024-09-17T05:28:16.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c0b3e20 con 0x7fd17c0b4900 2024-09-17T05:28:16.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd172a00640 1 -- 172.21.15.122:0/4055854112 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17400fd40 con 0x7fd17c0b4900 2024-09-17T05:28:16.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c154200 con 0x7fd17c0b4900 2024-09-17T05:28:16.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.219+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17c069410 con 0x7fd17c0b4900 2024-09-17T05:28:16.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.223+0000 7fd172a00640 1 -- 172.21.15.122:0/4055854112 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fd174003600 con 0x7fd17c0b4900 2024-09-17T05:28:16.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.223+0000 7fd172a00640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd158073560 0x7fd158075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:16.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.223+0000 7fd172a00640 1 -- 172.21.15.122:0/4055854112 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fd174095110 con 0x7fd17c0b4900 2024-09-17T05:28:16.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.223+0000 7fd179e00640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd158073560 0x7fd158075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:16.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.223+0000 7fd179e00640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd158073560 0x7fd158075a20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0ae000 tx=0x7fd16c01a2b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:16.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.227+0000 7fd172a00640 1 -- 172.21.15.122:0/4055854112 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd17405f510 con 0x7fd17c0b4900 2024-09-17T05:28:16.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.431+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd17c0b3c40 con 0x7fd17c0b4900 2024-09-17T05:28:16.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.891+0000 7fd172a00640 1 -- 172.21.15.122:0/4055854112 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7fd17c0b3c40 con 0x7fd17c0b4900 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd158073560 msgr2=0x7fd158075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd158073560 0x7fd158075a20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0ae000 tx=0x7fd16c01a2b0 comp rx=0 tx=0).stop 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd17c0b4900 msgr2=0x7fd17c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd17c0b4900 0x7fd17c0b4d90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0cc1f0 tx=0x7fd174011c60 comp rx=0 tx=0).stop 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 shutdown_connections 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd158073560 0x7fd158075a20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd17c0b4900 0x7fd17c0b4d90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd17c1640a0 0x7fd17c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 --2- 172.21.15.122:0/4055854112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd160004b20 0x7fd17c161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 >> 172.21.15.122:0/4055854112 conn(0x7fd17c0cfbc0 msgr2=0x7fd17c0b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:16.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 shutdown_connections 2024-09-17T05:28:16.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:16.895+0000 7fd181200640 1 -- 172.21.15.122:0/4055854112 wait complete. 2024-09-17T05:28:16.916 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:28:17.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- 172.21.15.122:0/2674272500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 msgr2=0x7f9b24004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 --2- 172.21.15.122:0/2674272500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 0x7f9b24004020 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009b90 tx=0x7f9b3001d270 comp rx=0 tx=0).stop 2024-09-17T05:28:17.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3e800640 1 -- 172.21.15.122:0/2674272500 reap_dead start 2024-09-17T05:28:17.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- 172.21.15.122:0/2674272500 shutdown_connections 2024-09-17T05:28:17.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- 172.21.15.122:0/2674272500 >> 172.21.15.122:0/2674272500 conn(0x7f9b400d1da0 msgr2=0x7f9b400d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:17.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- 172.21.15.122:0/2674272500 shutdown_connections 2024-09-17T05:28:17.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- 172.21.15.122:0/2674272500 wait complete. 2024-09-17T05:28:17.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 Processor -- start 2024-09-17T05:28:17.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- start start 2024-09-17T05:28:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 0x7f9b401f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b4015a640 0x7f9b4015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b400c5a40 0x7f9b400c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40167ff0 con 0x7f9b24003c20 2024-09-17T05:28:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40167e70 con 0x7f9b4015a640 2024-09-17T05:28:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b44c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9b40168170 con 0x7f9b400c5a40 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 0x7f9b401f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 0x7f9b401f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60422/0 (socket says 172.21.15.122:60422) 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 -- 172.21.15.122:0/3721591386 learned_addr learned my addr 172.21.15.122:0/3721591386 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3e800640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b400c5a40 0x7f9b400c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3d400640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b4015a640 0x7f9b4015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 -- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b400c5a40 msgr2=0x7f9b400c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b400c5a40 0x7f9b400c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 -- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b4015a640 msgr2=0x7f9b4015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b4015a640 0x7f9b4015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 -- 172.21.15.122:0/3721591386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b40159880 con 0x7f9b24003c20 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3d400640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b4015a640 0x7f9b4015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3e800640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b400c5a40 0x7f9b400c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b3de00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 0x7f9b401f9d30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9b3001d0a0 tx=0x7f9b30005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.059+0000 7f9b36000640 1 -- 172.21.15.122:0/3721591386 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b300ba070 con 0x7f9b24003c20 2024-09-17T05:28:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b36000640 1 -- 172.21.15.122:0/3721591386 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b300ac8a0 con 0x7f9b24003c20 2024-09-17T05:28:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b36000640 1 -- 172.21.15.122:0/3721591386 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b300b1040 con 0x7f9b24003c20 2024-09-17T05:28:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b40159b60 con 0x7f9b24003c20 2024-09-17T05:28:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4015a120 con 0x7f9b24003c20 2024-09-17T05:28:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b400b53f0 con 0x7f9b24003c20 2024-09-17T05:28:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b36000640 1 -- 172.21.15.122:0/3721591386 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f9b300046d0 con 0x7f9b24003c20 2024-09-17T05:28:17.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b36000640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b20073560 0x7f9b20075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b36000640 1 -- 172.21.15.122:0/3721591386 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f9b3012f7d0 con 0x7f9b24003c20 2024-09-17T05:28:17.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.063+0000 7f9b3d400640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b20073560 0x7f9b20075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.067+0000 7f9b3d400640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b20073560 0x7f9b20075a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c000f70 tx=0x7f9b2c001e50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.067+0000 7f9b36000640 1 -- 172.21.15.122:0/3721591386 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9b300c4020 con 0x7f9b24003c20 2024-09-17T05:28:17.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.255+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9b40160630 con 0x7f9b20073560 2024-09-17T05:28:17.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b36000640 1 -- 172.21.15.122:0/3721591386 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9b40160630 con 0x7f9b20073560 2024-09-17T05:28:17.259 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:28:17.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:28:17.259 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:28:17.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:28:17.259 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b20073560 msgr2=0x7f9b20075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b20073560 0x7f9b20075a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c000f70 tx=0x7f9b2c001e50 comp rx=0 tx=0).stop 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 msgr2=0x7f9b401f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 0x7f9b401f9d30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9b3001d0a0 tx=0x7f9b30005620 comp rx=0 tx=0).stop 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 shutdown_connections 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b20073560 0x7f9b20075a20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b400c5a40 0x7f9b400c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b4015a640 0x7f9b4015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 --2- 172.21.15.122:0/3721591386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b24003c20 0x7f9b401f9d30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 >> 172.21.15.122:0/3721591386 conn(0x7f9b400d1da0 msgr2=0x7f9b40072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:17.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.259+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 shutdown_connections 2024-09-17T05:28:17.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.263+0000 7f9b44c00640 1 -- 172.21.15.122:0/3721591386 wait complete. 2024-09-17T05:28:17.283 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:28:17.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 -- 172.21.15.122:0/2332762935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15bd50 msgr2=0x7fad5c0ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 --2- 172.21.15.122:0/2332762935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15bd50 0x7fad5c0ce5e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0d0a60 tx=0x7fad5001c780 comp rx=0 tx=0).stop 2024-09-17T05:28:17.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 -- 172.21.15.122:0/2332762935 shutdown_connections 2024-09-17T05:28:17.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 --2- 172.21.15.122:0/2332762935 >> v2:172.21.15.173:3300/0 conn(0x7fad5c15c130 0x7fad5c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 --2- 172.21.15.122:0/2332762935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad5c0cf4e0 0x7fad5c0c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 --2- 172.21.15.122:0/2332762935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15bd50 0x7fad5c0ce5e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 -- 172.21.15.122:0/2332762935 >> 172.21.15.122:0/2332762935 conn(0x7fad5c0d1ea0 msgr2=0x7fad5c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:17.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 -- 172.21.15.122:0/2332762935 shutdown_connections 2024-09-17T05:28:17.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 -- 172.21.15.122:0/2332762935 wait complete. 2024-09-17T05:28:17.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.427+0000 7fad64000640 1 Processor -- start 2024-09-17T05:28:17.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 -- start start 2024-09-17T05:28:17.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad5c0cf4e0 0x7fad5c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad5c15bd50 0x7fad5c0cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15c130 0x7fad5c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0ccac0 con 0x7fad5c15c130 2024-09-17T05:28:17.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0cc940 con 0x7fad5c0cf4e0 2024-09-17T05:28:17.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c0ccc40 con 0x7fad5c15bd50 2024-09-17T05:28:17.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15c130 0x7fad5c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15c130 0x7fad5c0cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60432/0 (socket says 172.21.15.122:60432) 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 -- 172.21.15.122:0/2058819104 learned_addr learned my addr 172.21.15.122:0/2058819104 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad60c00640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad5c15bd50 0x7fad5c0cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad61600640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad5c0cf4e0 0x7fad5c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 -- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad5c15bd50 msgr2=0x7fad5c0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad5c15bd50 0x7fad5c0cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 -- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad5c0cf4e0 msgr2=0x7fad5c1577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad5c0cf4e0 0x7fad5c1577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 -- 172.21.15.122:0/2058819104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c157d30 con 0x7fad5c15c130 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad62000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15c130 0x7fad5c0cd6b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e9c0 tx=0x7fad5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad59600640 1 -- 172.21.15.122:0/2058819104 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5400cd80 con 0x7fad5c15c130 2024-09-17T05:28:17.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad59600640 1 -- 172.21.15.122:0/2058819104 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad54004b30 con 0x7fad5c15c130 2024-09-17T05:28:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad59600640 1 -- 172.21.15.122:0/2058819104 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54005020 con 0x7fad5c15c130 2024-09-17T05:28:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c157fd0 con 0x7fad5c15c130 2024-09-17T05:28:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c158590 con 0x7fad5c15c130 2024-09-17T05:28:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.431+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c0c5dd0 con 0x7fad5c15c130 2024-09-17T05:28:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.435+0000 7fad59600640 1 -- 172.21.15.122:0/2058819104 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7fad540051c0 con 0x7fad5c15c130 2024-09-17T05:28:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.435+0000 7fad59600640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad3c073560 0x7fad3c075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.435+0000 7fad59600640 1 -- 172.21.15.122:0/2058819104 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fad54095150 con 0x7fad5c15c130 2024-09-17T05:28:17.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.435+0000 7fad61600640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad3c073560 0x7fad3c075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.435+0000 7fad61600640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad3c073560 0x7fad3c075a20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad500af000 tx=0x7fad5001afc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:17.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.439+0000 7fad59600640 1 -- 172.21.15.122:0/2058819104 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fad54016030 con 0x7fad5c15c130 2024-09-17T05:28:17.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fad5c0c7e50 con 0x7fad5c15c130 2024-09-17T05:28:17.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad59600640 1 -- 172.21.15.122:0/2058819104 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7fad5405f590 con 0x7fad5c15c130 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad3c073560 msgr2=0x7fad3c075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad3c073560 0x7fad3c075a20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad500af000 tx=0x7fad5001afc0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15c130 msgr2=0x7fad5c0cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15c130 0x7fad5c0cd6b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e9c0 tx=0x7fad5400ee80 comp rx=0 tx=0).stop 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 shutdown_connections 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad3c073560 0x7fad3c075a20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad5c15c130 0x7fad5c0cd6b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad5c15bd50 0x7fad5c0cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 --2- 172.21.15.122:0/2058819104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad5c0cf4e0 0x7fad5c1577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 >> 172.21.15.122:0/2058819104 conn(0x7fad5c0d1ea0 msgr2=0x7fad5c072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 shutdown_connections 2024-09-17T05:28:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.607+0000 7fad64000640 1 -- 172.21.15.122:0/2058819104 wait complete. 2024-09-17T05:28:17.630 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:28:17.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.771+0000 7efc45e00640 1 -- 172.21.15.122:0/170578372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 msgr2=0x7efc4015e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.771+0000 7efc45e00640 1 --2- 172.21.15.122:0/170578372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 0x7efc4015e7f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efc30009900 tx=0x7efc3001c850 comp rx=0 tx=0).stop 2024-09-17T05:28:17.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.771+0000 7efc45e00640 1 -- 172.21.15.122:0/170578372 shutdown_connections 2024-09-17T05:28:17.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.771+0000 7efc45e00640 1 --2- 172.21.15.122:0/170578372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 0x7efc4015e7f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.771+0000 7efc45e00640 1 -- 172.21.15.122:0/170578372 >> 172.21.15.122:0/170578372 conn(0x7efc400cfc20 msgr2=0x7efc400d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:17.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.771+0000 7efc45e00640 1 -- 172.21.15.122:0/170578372 shutdown_connections 2024-09-17T05:28:17.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.771+0000 7efc45e00640 1 -- 172.21.15.122:0/170578372 wait complete. 2024-09-17T05:28:17.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 Processor -- start 2024-09-17T05:28:17.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 -- start start 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 0x7efc4014d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc400af220 0x7efc4015e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc400cddd0 0x7efc4015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc40169430 con 0x7efc4006fea0 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc401692b0 con 0x7efc400af220 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efc401695b0 con 0x7efc400cddd0 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 0x7efc4014d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 0x7efc4014d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60454/0 (socket says 172.21.15.122:60454) 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 -- 172.21.15.122:0/2102298266 learned_addr learned my addr 172.21.15.122:0/2102298266 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3e000640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc400af220 0x7efc4015e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3f400640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc400cddd0 0x7efc4015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 -- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc400cddd0 msgr2=0x7efc4015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc400cddd0 0x7efc4015ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 -- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc400af220 msgr2=0x7efc4015e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc400af220 0x7efc4015e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 -- 172.21.15.122:0/2102298266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc401f9930 con 0x7efc4006fea0 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3e000640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc400af220 0x7efc4015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc3ea00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 0x7efc4014d080 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efc30009850 tx=0x7efc300a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc37400640 1 -- 172.21.15.122:0/2102298266 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc300b2040 con 0x7efc4006fea0 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc37400640 1 -- 172.21.15.122:0/2102298266 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc300044b0 con 0x7efc4006fea0 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc401f9c10 con 0x7efc4006fea0 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc37400640 1 -- 172.21.15.122:0/2102298266 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30003860 con 0x7efc4006fea0 2024-09-17T05:28:17.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.775+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401fa1d0 con 0x7efc4006fea0 2024-09-17T05:28:17.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.779+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc401524e0 con 0x7efc4006fea0 2024-09-17T05:28:17.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.779+0000 7efc37400640 1 -- 172.21.15.122:0/2102298266 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7efc30003a00 con 0x7efc4006fea0 2024-09-17T05:28:17.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.779+0000 7efc37400640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc20073560 0x7efc20075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:17.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.779+0000 7efc37400640 1 -- 172.21.15.122:0/2102298266 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7efc3012e610 con 0x7efc4006fea0 2024-09-17T05:28:17.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.779+0000 7efc3e000640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc20073560 0x7efc20075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:17.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.779+0000 7efc3e000640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc20073560 0x7efc20075a20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efc40154640 tx=0x7efc2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:17.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.783+0000 7efc37400640 1 -- 172.21.15.122:0/2102298266 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efc300bd060 con 0x7efc4006fea0 2024-09-17T05:28:17.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.951+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7efc400d0810 con 0x7efc4006fea0 2024-09-17T05:28:17.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.951+0000 7efc37400640 1 -- 172.21.15.122:0/2102298266 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v31) ==== 116+0+2 (secure 0 0 0) 0x7efc300164d0 con 0x7efc4006fea0 2024-09-17T05:28:17.953 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:28:17.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc20073560 msgr2=0x7efc20075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc20073560 0x7efc20075a20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efc40154640 tx=0x7efc2c009040 comp rx=0 tx=0).stop 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 msgr2=0x7efc4014d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 0x7efc4014d080 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efc30009850 tx=0x7efc300a5750 comp rx=0 tx=0).stop 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 shutdown_connections 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc20073560 0x7efc20075a20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc400cddd0 0x7efc4015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc400af220 0x7efc4015e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 --2- 172.21.15.122:0/2102298266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc4006fea0 0x7efc4014d080 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 >> 172.21.15.122:0/2102298266 conn(0x7efc400cfc20 msgr2=0x7efc40158470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 shutdown_connections 2024-09-17T05:28:17.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:17.955+0000 7efc45e00640 1 -- 172.21.15.122:0/2102298266 wait complete. 2024-09-17T05:28:17.974 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:28:18.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 -- 172.21.15.122:0/286169459 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 msgr2=0x7f8f740d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:18.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 --2- 172.21.15.122:0/286169459 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 0x7f8f740d15d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009bf0 tx=0x7f8f6c01ca70 comp rx=0 tx=0).stop 2024-09-17T05:28:18.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 -- 172.21.15.122:0/286169459 shutdown_connections 2024-09-17T05:28:18.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 --2- 172.21.15.122:0/286169459 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 0x7f8f740d15d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 -- 172.21.15.122:0/286169459 >> 172.21.15.122:0/286169459 conn(0x7f8f740d0ab0 msgr2=0x7f8f740d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:18.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 -- 172.21.15.122:0/286169459 shutdown_connections 2024-09-17T05:28:18.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 -- 172.21.15.122:0/286169459 wait complete. 2024-09-17T05:28:18.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 Processor -- start 2024-09-17T05:28:18.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.119+0000 7f8f7a400640 1 -- start start 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f74071060 0x7f8f74150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 0x7f8f740c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f740cc630 0x7f8f7406f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7416cc50 con 0x7f8f740cc630 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7416cad0 con 0x7f8f74071060 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7416cdd0 con 0x7f8f74073d40 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f740cc630 0x7f8f7406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f740cc630 0x7f8f7406f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60468/0 (socket says 172.21.15.122:60468) 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 -- 172.21.15.122:0/3680830214 learned_addr learned my addr 172.21.15.122:0/3680830214 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f6be00640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 0x7f8f740c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79000640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f74071060 0x7f8f74150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:18.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 -- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 msgr2=0x7f8f740c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:18.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 0x7f8f740c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 -- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f74071060 msgr2=0x7f8f74150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:18.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f74071060 0x7f8f74150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 -- 172.21.15.122:0/3680830214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f74151030 con 0x7f8f740cc630 2024-09-17T05:28:18.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f6be00640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 0x7f8f740c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:28:18.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f79a00640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f740cc630 0x7f8f7406f550 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c01b120 tx=0x7f8f6c009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:18.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f71600640 1 -- 172.21.15.122:0/3680830214 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0bb070 con 0x7f8f740cc630 2024-09-17T05:28:18.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f71600640 1 -- 172.21.15.122:0/3680830214 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f6c004d40 con 0x7f8f740cc630 2024-09-17T05:28:18.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f71600640 1 -- 172.21.15.122:0/3680830214 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0b2040 con 0x7f8f740cc630 2024-09-17T05:28:18.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f74161240 con 0x7f8f740cc630 2024-09-17T05:28:18.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f740c96f0 con 0x7f8f740cc630 2024-09-17T05:28:18.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f740b6a30 con 0x7f8f740cc630 2024-09-17T05:28:18.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.123+0000 7f8f71600640 1 -- 172.21.15.122:0/3680830214 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f8f6c0be070 con 0x7f8f740cc630 2024-09-17T05:28:18.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.127+0000 7f8f71600640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f50073560 0x7f8f50075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:18.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.127+0000 7f8f71600640 1 -- 172.21.15.122:0/3680830214 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f8f6c12e560 con 0x7f8f740cc630 2024-09-17T05:28:18.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.127+0000 7f8f79000640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f50073560 0x7f8f50075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:18.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.127+0000 7f8f79000640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f50073560 0x7f8f50075a20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8f64007fd0 tx=0x7f8f64009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:18.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.131+0000 7f8f71600640 1 -- 172.21.15.122:0/3680830214 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8f74151030 con 0x7f8f740cc630 2024-09-17T05:28:18.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.299+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f8f74156e50 con 0x7f8f740cc630 2024-09-17T05:28:18.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f71600640 1 -- 172.21.15.122:0/3680830214 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7f8f6c0166f0 con 0x7f8f740cc630 2024-09-17T05:28:18.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f50073560 msgr2=0x7f8f50075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f50073560 0x7f8f50075a20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8f64007fd0 tx=0x7f8f64009040 comp rx=0 tx=0).stop 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f740cc630 msgr2=0x7f8f7406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f740cc630 0x7f8f7406f550 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c01b120 tx=0x7f8f6c009280 comp rx=0 tx=0).stop 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 shutdown_connections 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f50073560 0x7f8f50075a20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f740cc630 0x7f8f7406f550 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f74073d40 0x7f8f740c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 --2- 172.21.15.122:0/3680830214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f74071060 0x7f8f74150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 >> 172.21.15.122:0/3680830214 conn(0x7f8f740d0ab0 msgr2=0x7f8f74157370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 shutdown_connections 2024-09-17T05:28:18.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.303+0000 7f8f7a400640 1 -- 172.21.15.122:0/3680830214 wait complete. 2024-09-17T05:28:18.319 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:18.320+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:18.320 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:18.320+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:18.320 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:18.320+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:18.320 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:18.320+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:18.320 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:18.320+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:18.325 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T05:28:18.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- 172.21.15.122:0/3843344161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 msgr2=0x7f25e406eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:18.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 --2- 172.21.15.122:0/3843344161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 0x7f25e406eb40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009c80 tx=0x7f25dc01cda0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- 172.21.15.122:0/3843344161 shutdown_connections 2024-09-17T05:28:18.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 --2- 172.21.15.122:0/3843344161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 0x7f25e406eb40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 --2- 172.21.15.122:0/3843344161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25e415a9d0 0x7f25e40ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- 172.21.15.122:0/3843344161 >> 172.21.15.122:0/3843344161 conn(0x7f25e40cfbc0 msgr2=0x7f25e40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:18.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- 172.21.15.122:0/3843344161 shutdown_connections 2024-09-17T05:28:18.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- 172.21.15.122:0/3843344161 wait complete. 2024-09-17T05:28:18.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 Processor -- start 2024-09-17T05:28:18.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- start start 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25e40734f0 0x7f25e40ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 0x7f25e414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25e415a9d0 0x7f25e40b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25e40cdc40 con 0x7f25e40ce890 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f25e40cdac0 con 0x7f25e415a9d0 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e8a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f25e40cddc0 con 0x7f25e40734f0 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25d9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 0x7f25e414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25d9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 0x7f25e414f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60488/0 (socket says 172.21.15.122:60488) 2024-09-17T05:28:18.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25d9400640 1 -- 172.21.15.122:0/1695583097 learned_addr learned my addr 172.21.15.122:0/1695583097 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e1e00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25e40734f0 0x7f25e40ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25e2800640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25e415a9d0 0x7f25e40b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25d9400640 1 -- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25e40734f0 msgr2=0x7f25e40ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25d9400640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25e40734f0 0x7f25e40ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25d9400640 1 -- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25e415a9d0 msgr2=0x7f25e40b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25d9400640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25e415a9d0 0x7f25e40b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.471+0000 7f25d9400640 1 -- 172.21.15.122:0/1695583097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e40aecd0 con 0x7f25e40ce890 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.475+0000 7f25d9400640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 0x7f25e414f8c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00d950 tx=0x7f25cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:18.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.475+0000 7f25daa00640 1 -- 172.21.15.122:0/1695583097 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc017070 con 0x7f25e40ce890 2024-09-17T05:28:18.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.475+0000 7f25daa00640 1 -- 172.21.15.122:0/1695583097 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25cc009d20 con 0x7f25e40ce890 2024-09-17T05:28:18.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.475+0000 7f25daa00640 1 -- 172.21.15.122:0/1695583097 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0106c0 con 0x7f25e40ce890 2024-09-17T05:28:18.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.475+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e40b4ee0 con 0x7f25e40ce890 2024-09-17T05:28:18.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.475+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e40b54a0 con 0x7f25e40ce890 2024-09-17T05:28:18.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.475+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e4069410 con 0x7f25e40ce890 2024-09-17T05:28:18.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.475+0000 7f25daa00640 1 -- 172.21.15.122:0/1695583097 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 97540+0+0 (secure 0 0 0) 0x7f25cc01a070 con 0x7f25e40ce890 2024-09-17T05:28:18.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.479+0000 7f25daa00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25c0073560 0x7f25c0075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:18.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.479+0000 7f25daa00640 1 -- 172.21.15.122:0/1695583097 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f25cc0949c0 con 0x7f25e40ce890 2024-09-17T05:28:18.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.479+0000 7f25e1e00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25c0073560 0x7f25c0075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:18.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.479+0000 7f25e1e00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25c0073560 0x7f25c0075a20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f25e40b48f0 tx=0x7f25d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:18.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.483+0000 7f25daa00640 1 -- 172.21.15.122:0/1695583097 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f25cc05ee50 con 0x7f25e40ce890 2024-09-17T05:28:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:18.671+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f25e406eb40 con 0x7f25c0073560 2024-09-17T05:28:19.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.311+0000 7f25daa00640 1 -- 172.21.15.122:0/1695583097 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 12) ==== 97591+0+0 (secure 0 0 0) 0x7f25cc05b0c0 con 0x7f25e40ce890 2024-09-17T05:28:19.633 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:19.632+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:19.633 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:19.632+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:19.633 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:19.632+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:19.633 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:19.632+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:19.633 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:28:19.632+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:28:19.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.675+0000 7f25daa00640 1 -- 172.21.15.122:0/1695583097 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25e406eb40 con 0x7f25c0073560 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25c0073560 msgr2=0x7f25c0075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25c0073560 0x7f25c0075a20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f25e40b48f0 tx=0x7f25d4009040 comp rx=0 tx=0).stop 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 msgr2=0x7f25e414f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 0x7f25e414f8c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00d950 tx=0x7f25cc00de10 comp rx=0 tx=0).stop 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 shutdown_connections 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25c0073560 0x7f25c0075a20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25e415a9d0 0x7f25e40b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40ce890 0x7f25e414f8c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 --2- 172.21.15.122:0/1695583097 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25e40734f0 0x7f25e40ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:19.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 >> 172.21.15.122:0/1695583097 conn(0x7f25e40cfbc0 msgr2=0x7f25e40af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:19.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 shutdown_connections 2024-09-17T05:28:19.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.679+0000 7f25e8a00640 1 -- 172.21.15.122:0/1695583097 wait complete. 2024-09-17T05:28:19.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:28:19.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- 172.21.15.122:0/961180961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 msgr2=0x7fdfc40ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:19.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 --2- 172.21.15.122:0/961180961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 0x7fdfc40ce5e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009c80 tx=0x7fdfb401cc80 comp rx=0 tx=0).stop 2024-09-17T05:28:19.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- 172.21.15.122:0/961180961 shutdown_connections 2024-09-17T05:28:19.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 --2- 172.21.15.122:0/961180961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc415c130 0x7fdfc40ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:19.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 --2- 172.21.15.122:0/961180961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 0x7fdfc40ce5e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:19.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- 172.21.15.122:0/961180961 >> 172.21.15.122:0/961180961 conn(0x7fdfc40d1ea0 msgr2=0x7fdfc40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:19.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- 172.21.15.122:0/961180961 shutdown_connections 2024-09-17T05:28:19.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- 172.21.15.122:0/961180961 wait complete. 2024-09-17T05:28:19.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 Processor -- start 2024-09-17T05:28:19.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- start start 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc40cf4e0 0x7fdfc4152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 0x7fdfc4153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc415c130 0x7fdfc4153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4168e50 con 0x7fdfc415bd50 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4168cd0 con 0x7fdfc415c130 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfca600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc4168fd0 con 0x7fdfc40cf4e0 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 0x7fdfc4153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 0x7fdfc4153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60518/0 (socket says 172.21.15.122:60518) 2024-09-17T05:28:19.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 -- 172.21.15.122:0/385708445 learned_addr learned my addr 172.21.15.122:0/385708445 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc3400640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc40cf4e0 0x7fdfc4152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc3e00640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc415c130 0x7fdfc4153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 -- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc40cf4e0 msgr2=0x7fdfc4152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc40cf4e0 0x7fdfc4152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 -- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc415c130 msgr2=0x7fdfc4153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc415c130 0x7fdfc4153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 -- 172.21.15.122:0/385708445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc41f5ea0 con 0x7fdfc415bd50 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfc2a00640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 0x7fdfc4153590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000d750 tx=0x7fdfb000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfbb400640 1 -- 172.21.15.122:0/385708445 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0017070 con 0x7fdfc415bd50 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfbb400640 1 -- 172.21.15.122:0/385708445 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfb0009d20 con 0x7fdfc415bd50 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.851+0000 7fdfbb400640 1 -- 172.21.15.122:0/385708445 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00106c0 con 0x7fdfc415bd50 2024-09-17T05:28:19.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.855+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc41540e0 con 0x7fdfc415bd50 2024-09-17T05:28:19.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.855+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc41546a0 con 0x7fdfc415bd50 2024-09-17T05:28:19.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.855+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc40c9370 con 0x7fdfc415bd50 2024-09-17T05:28:19.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.855+0000 7fdfbb400640 1 -- 172.21.15.122:0/385708445 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 12) ==== 97591+0+0 (secure 0 0 0) 0x7fdfb001a070 con 0x7fdfc415bd50 2024-09-17T05:28:19.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.855+0000 7fdfbb400640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa40735f0 0x7fdfa4075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:28:19.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.855+0000 7fdfbb400640 1 -- 172.21.15.122:0/385708445 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fdfb0094ad0 con 0x7fdfc415bd50 2024-09-17T05:28:19.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.855+0000 7fdfc3400640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa40735f0 0x7fdfa4075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:28:19.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.859+0000 7fdfc3400640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa40735f0 0x7fdfa4075ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40af000 tx=0x7fdfb401b4c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:28:19.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:19.859+0000 7fdfbb400640 1 -- 172.21.15.122:0/385708445 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdfb000de60 con 0x7fdfc415bd50 2024-09-17T05:28:20.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.051+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fdfc40cfaf0 con 0x7fdfa40735f0 2024-09-17T05:28:20.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.059+0000 7fdfbb400640 1 -- 172.21.15.122:0/385708445 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fdfc40cfaf0 con 0x7fdfa40735f0 2024-09-17T05:28:20.060 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0 2024-09-17T05:28:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.059+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa40735f0 msgr2=0x7fdfa4075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.059+0000 7fdfca600640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa40735f0 0x7fdfa4075ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40af000 tx=0x7fdfb401b4c0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.059+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 msgr2=0x7fdfc4153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:28:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.059+0000 7fdfca600640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 0x7fdfc4153590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000d750 tx=0x7fdfb000dc10 comp rx=0 tx=0).stop 2024-09-17T05:28:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.063+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 shutdown_connections 2024-09-17T05:28:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.063+0000 7fdfca600640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfa40735f0 0x7fdfa4075ab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.063+0000 7fdfca600640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfc415c130 0x7fdfc4153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.063+0000 7fdfca600640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc415bd50 0x7fdfc4153590 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.063+0000 7fdfca600640 1 --2- 172.21.15.122:0/385708445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfc40cf4e0 0x7fdfc4152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:28:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.063+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 >> 172.21.15.122:0/385708445 conn(0x7fdfc40d1ea0 msgr2=0x7fdfc40d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:28:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.063+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 shutdown_connections 2024-09-17T05:28:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:28:20.063+0000 7fdfca600640 1 -- 172.21.15.122:0/385708445 wait complete. 2024-09-17T05:28:20.082 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0 2024-09-17T05:28:20.082 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:28:20.082 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:20.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:20.091 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:28:28.900 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:28:28.901 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:28:28.901 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.74907 s, 123 MB/s 2024-09-17T05:28:28.902 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:28:28.902 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:28.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:28.911 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:28:31.953 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:31.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:28:37.467 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:28:37.467 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:28:37.467 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.49973 s, 126 MB/s 2024-09-17T05:28:37.468 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:28:37.468 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:37.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:37.477 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:28:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:28:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:28:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.49554 s, 113 MB/s 2024-09-17T05:28:47.023 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:28:47.023 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:47.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:47.033 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:28:56.064 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:28:56.065 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:28:56.065 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.97695 s, 120 MB/s 2024-09-17T05:28:56.066 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:28:56.066 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:28:56.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:28:56.075 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:29:02.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:29:02.037 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:29:04.865 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:29:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:29:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.7388 s, 123 MB/s 2024-09-17T05:29:04.867 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:29:04.867 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:04.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:04.876 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:29:13.222 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:29:13.222 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:29:13.222 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.2964 s, 129 MB/s 2024-09-17T05:29:13.223 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:29:13.223 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:13.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:13.233 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:29:21.942 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:29:21.943 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:29:21.943 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.66049 s, 124 MB/s 2024-09-17T05:29:21.944 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:29:21.944 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:21.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:21.953 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:29:30.381 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:29:30.381 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:29:30.382 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.37473 s, 128 MB/s 2024-09-17T05:29:30.383 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:29:30.383 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:30.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:30.392 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:29:32.092 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:29:32.095 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:29:39.539 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:29:39.539 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:29:39.539 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.09344 s, 118 MB/s 2024-09-17T05:29:39.540 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:29:39.540 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:29:39.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:39.549 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:29:48.613 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:29:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:29:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.00782 s, 119 MB/s 2024-09-17T05:29:48.615 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T05:29:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 -- 172.21.15.122:0/2685728219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 msgr2=0x7f2c940cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 --2- 172.21.15.122:0/2685728219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 0x7f2c940cee10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2c940b0fb0 tx=0x7f2c7c01f0b0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 -- 172.21.15.122:0/2685728219 shutdown_connections 2024-09-17T05:29:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 --2- 172.21.15.122:0/2685728219 >> v2:172.21.15.173:3300/0 conn(0x7f2c940b0b60 0x7f2c940ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 --2- 172.21.15.122:0/2685728219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 0x7f2c940cee10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 -- 172.21.15.122:0/2685728219 >> 172.21.15.122:0/2685728219 conn(0x7f2c940d1da0 msgr2=0x7f2c940d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 -- 172.21.15.122:0/2685728219 shutdown_connections 2024-09-17T05:29:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 -- 172.21.15.122:0/2685728219 wait complete. 2024-09-17T05:29:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 Processor -- start 2024-09-17T05:29:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.762+0000 7f2c99000640 1 -- start start 2024-09-17T05:29:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c940b0b60 0x7f2c9415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c9415be10 0x7f2c940b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 0x7f2c941f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940cd4a0 con 0x7f2c9415c1f0 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940cd320 con 0x7f2c940b0b60 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2c940cd620 con 0x7f2c9415be10 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 0x7f2c941f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c91e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c940b0b60 0x7f2c9415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c89400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c9415be10 0x7f2c940b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 0x7f2c941f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60292/0 (socket says 172.21.15.122:60292) 2024-09-17T05:29:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 -- 172.21.15.122:0/1529342998 learned_addr learned my addr 172.21.15.122:0/1529342998 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 -- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c9415be10 msgr2=0x7f2c940b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c9415be10 0x7f2c940b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 -- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c940b0b60 msgr2=0x7f2c9415e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c940b0b60 0x7f2c9415e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 -- 172.21.15.122:0/1529342998 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c940b79c0 con 0x7f2c9415c1f0 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c91e00640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c940b0b60 0x7f2c9415e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c92800640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 0x7f2c941f8590 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00eb20 tx=0x7f2c8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c8aa00640 1 -- 172.21.15.122:0/1529342998 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c004b30 con 0x7f2c9415c1f0 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c8aa00640 1 -- 172.21.15.122:0/1529342998 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c8c004cf0 con 0x7f2c9415c1f0 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c8aa00640 1 -- 172.21.15.122:0/1529342998 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c005280 con 0x7f2c9415c1f0 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9415ec90 con 0x7f2c9415c1f0 2024-09-17T05:29:48.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c940c5c50 con 0x7f2c9415c1f0 2024-09-17T05:29:48.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.766+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c940cdb40 con 0x7f2c9415c1f0 2024-09-17T05:29:48.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.770+0000 7f2c8aa00640 1 -- 172.21.15.122:0/1529342998 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 14) ==== 97540+0+0 (secure 0 0 0) 0x7f2c8c005420 con 0x7f2c9415c1f0 2024-09-17T05:29:48.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.770+0000 7f2c8aa00640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c70073560 0x7f2c70075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:48.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.770+0000 7f2c8aa00640 1 -- 172.21.15.122:0/1529342998 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f2c8c095530 con 0x7f2c9415c1f0 2024-09-17T05:29:48.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.770+0000 7f2c91e00640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c70073560 0x7f2c70075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:48.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.770+0000 7f2c91e00640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c70073560 0x7f2c70075a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c940b6d40 tx=0x7f2c84009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:48.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.774+0000 7f2c8aa00640 1 -- 172.21.15.122:0/1529342998 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2c8c05f930 con 0x7f2c9415c1f0 2024-09-17T05:29:48.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:48.974+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f2c9415f7d0 con 0x7f2c70073560 2024-09-17T05:29:48.981 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:48.983+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:48.982 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:48.983+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:48.982 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:48.983+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:48.982 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:48.983+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:48.982 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:48.983+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:49.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c8aa00640 1 -- 172.21.15.122:0/1529342998 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c9415f7d0 con 0x7f2c70073560 2024-09-17T05:29:49.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c70073560 msgr2=0x7f2c70075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c70073560 0x7f2c70075a20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c940b6d40 tx=0x7f2c84009040 comp rx=0 tx=0).stop 2024-09-17T05:29:49.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 msgr2=0x7f2c941f8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 0x7f2c941f8590 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00eb20 tx=0x7f2c8c00c2d0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 shutdown_connections 2024-09-17T05:29:49.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c70073560 0x7f2c70075a20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c9415c1f0 0x7f2c941f8590 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c9415be10 0x7f2c940b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 --2- 172.21.15.122:0/1529342998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c940b0b60 0x7f2c9415e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.002+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 >> 172.21.15.122:0/1529342998 conn(0x7f2c940d1da0 msgr2=0x7f2c94070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.006+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 shutdown_connections 2024-09-17T05:29:49.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.006+0000 7f2c99000640 1 -- 172.21.15.122:0/1529342998 wait complete. 2024-09-17T05:29:49.024 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:29:49.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- 172.21.15.122:0/2149841759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a9015ade0 msgr2=0x7f6a900cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 --2- 172.21.15.122:0/2149841759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a9015ade0 0x7f6a900cebf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009870 tx=0x7f6a8401c860 comp rx=0 tx=0).stop 2024-09-17T05:29:49.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- 172.21.15.122:0/2149841759 shutdown_connections 2024-09-17T05:29:49.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 --2- 172.21.15.122:0/2149841759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a9015ade0 0x7f6a900cebf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- 172.21.15.122:0/2149841759 >> 172.21.15.122:0/2149841759 conn(0x7f6a900b18f0 msgr2=0x7f6a900b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- 172.21.15.122:0/2149841759 shutdown_connections 2024-09-17T05:29:49.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- 172.21.15.122:0/2149841759 wait complete. 2024-09-17T05:29:49.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 Processor -- start 2024-09-17T05:29:49.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- start start 2024-09-17T05:29:49.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a9006e810 0x7f6a9015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a900b4bd0 0x7f6a90152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a9015ade0 0x7f6a901f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90169850 con 0x7f6a9006e810 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901696d0 con 0x7f6a900b4bd0 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a98600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6a901699d0 con 0x7f6a9015ade0 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a95c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a9006e810 0x7f6a9015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a95c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a9006e810 0x7f6a9015ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60302/0 (socket says 172.21.15.122:60302) 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.170+0000 7f6a95c00640 1 -- 172.21.15.122:0/3577435960 learned_addr learned my addr 172.21.15.122:0/3577435960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:49.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a96600640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a9015ade0 0x7f6a901f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a95200640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a900b4bd0 0x7f6a90152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a95c00640 1 -- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a9015ade0 msgr2=0x7f6a901f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a95c00640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a9015ade0 0x7f6a901f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a95c00640 1 -- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a900b4bd0 msgr2=0x7f6a90152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a95c00640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a900b4bd0 0x7f6a90152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a95c00640 1 -- 172.21.15.122:0/3577435960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a90158570 con 0x7f6a9006e810 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a95c00640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a9006e810 0x7f6a9015ef10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6a8401cd40 tx=0x7f6a840056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a8d600640 1 -- 172.21.15.122:0/3577435960 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a840b2040 con 0x7f6a9006e810 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a8d600640 1 -- 172.21.15.122:0/3577435960 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a840bb070 con 0x7f6a9006e810 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a90153110 con 0x7f6a9006e810 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a8d600640 1 -- 172.21.15.122:0/3577435960 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84003600 con 0x7f6a9006e810 2024-09-17T05:29:49.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a901587f0 con 0x7f6a9006e810 2024-09-17T05:29:49.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a900cebf0 con 0x7f6a9006e810 2024-09-17T05:29:49.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a8d600640 1 -- 172.21.15.122:0/3577435960 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 14) ==== 97540+0+0 (secure 0 0 0) 0x7f6a840a55e0 con 0x7f6a9006e810 2024-09-17T05:29:49.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.174+0000 7f6a8d600640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a70073560 0x7f6a70075a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:49.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.178+0000 7f6a8d600640 1 -- 172.21.15.122:0/3577435960 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f6a8412ea50 con 0x7f6a9006e810 2024-09-17T05:29:49.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.178+0000 7f6a95200640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a70073560 0x7f6a70075a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:49.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.178+0000 7f6a95200640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a70073560 0x7f6a70075a20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a901580b0 tx=0x7f6a7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:49.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.178+0000 7f6a8d600640 1 -- 172.21.15.122:0/3577435960 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6a90158570 con 0x7f6a9006e810 2024-09-17T05:29:49.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.374+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f6a900b0c70 con 0x7f6a70073560 2024-09-17T05:29:49.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.378+0000 7f6a8d600640 1 -- 172.21.15.122:0/3577435960 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f6a900b0c70 con 0x7f6a70073560 2024-09-17T05:29:49.381 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0 2024-09-17T05:29:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a70073560 msgr2=0x7f6a70075a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a70073560 0x7f6a70075a20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a901580b0 tx=0x7f6a7c009040 comp rx=0 tx=0).stop 2024-09-17T05:29:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a9006e810 msgr2=0x7f6a9015ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a9006e810 0x7f6a9015ef10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6a8401cd40 tx=0x7f6a840056e0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 shutdown_connections 2024-09-17T05:29:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a70073560 0x7f6a70075a20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a9015ade0 0x7f6a901f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a900b4bd0 0x7f6a90152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 --2- 172.21.15.122:0/3577435960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a9006e810 0x7f6a9015ef10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:49.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 >> 172.21.15.122:0/3577435960 conn(0x7f6a900b18f0 msgr2=0x7f6a900b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:49.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 shutdown_connections 2024-09-17T05:29:49.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:49.382+0000 7f6a98600640 1 -- 172.21.15.122:0/3577435960 wait complete. 2024-09-17T05:29:49.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:49.403 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0 2024-09-17T05:29:49.403 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:29:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0 2024-09-17T05:29:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="10405347328" 2024-09-17T05:29:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:29:49.413 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-17T05:29:51.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:29:51.415 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0 2024-09-17T05:29:51.415 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:29:51.426 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/941d07cc-a3b7-4d9e-8a71-298e61fdcfa0 2024-09-17T05:29:51.426 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="10737418240" 2024-09-17T05:29:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:29:51.427 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-17T05:29:51.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone23140 2024-09-17T05:29:51.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.622+0000 7f5b4aa00640 1 -- 172.21.15.122:0/760549976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b440b1650 msgr2=0x7f5b440b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.622+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/760549976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b440b1650 0x7f5b440b1a30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5b44073810 tx=0x7f5b3001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.622+0000 7f5b4aa00640 1 -- 172.21.15.122:0/760549976 shutdown_connections 2024-09-17T05:29:51.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.622+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/760549976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b440b1650 0x7f5b440b1a30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.622+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/760549976 >> v2:172.21.15.173:3300/0 conn(0x7f5b441605d0 0x7f5b440d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.622+0000 7f5b4aa00640 1 -- 172.21.15.122:0/760549976 >> 172.21.15.122:0/760549976 conn(0x7f5b440d1dd0 msgr2=0x7f5b440d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:51.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.622+0000 7f5b4aa00640 1 -- 172.21.15.122:0/760549976 shutdown_connections 2024-09-17T05:29:51.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.622+0000 7f5b4aa00640 1 -- 172.21.15.122:0/760549976 wait complete. 2024-09-17T05:29:51.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 Processor -- start 2024-09-17T05:29:51.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 -- start start 2024-09-17T05:29:51.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b440aeb00 0x7f5b4414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b440b1650 0x7f5b4415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b441605d0 0x7f5b441552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b440b0810 con 0x7f5b441605d0 2024-09-17T05:29:51.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b440b0ee0 con 0x7f5b440aeb00 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5b440b1220 con 0x7f5b440b1650 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b441605d0 0x7f5b441552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b441605d0 0x7f5b441552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58694/0 (socket says 172.21.15.122:58694) 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 -- 172.21.15.122:0/3354105603 learned_addr learned my addr 172.21.15.122:0/3354105603 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b43e00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b440aeb00 0x7f5b4414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b43400640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b440b1650 0x7f5b4415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 -- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b440b1650 msgr2=0x7f5b4415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b440b1650 0x7f5b4415eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 -- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b440aeb00 msgr2=0x7f5b4414f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b440aeb00 0x7f5b4414f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 -- 172.21.15.122:0/3354105603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4415de20 con 0x7f5b441605d0 2024-09-17T05:29:51.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b48a00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b441605d0 0x7f5b441552e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00c7c0 tx=0x7f5b3c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:51.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b3be00640 1 -- 172.21.15.122:0/3354105603 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c003b80 con 0x7f5b441605d0 2024-09-17T05:29:51.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b3be00640 1 -- 172.21.15.122:0/3354105603 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b3c003d20 con 0x7f5b441605d0 2024-09-17T05:29:51.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b3be00640 1 -- 172.21.15.122:0/3354105603 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00aab0 con 0x7f5b441605d0 2024-09-17T05:29:51.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4415e100 con 0x7f5b441605d0 2024-09-17T05:29:51.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4415e6c0 con 0x7f5b441605d0 2024-09-17T05:29:51.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.626+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b440b1a30 con 0x7f5b441605d0 2024-09-17T05:29:51.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.630+0000 7f5b3be00640 1 -- 172.21.15.122:0/3354105603 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 97591+0+0 (secure 0 0 0) 0x7f5b3c00ac50 con 0x7f5b441605d0 2024-09-17T05:29:51.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.630+0000 7f5b3be00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b240735f0 0x7f5b24075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:51.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.630+0000 7f5b3be00640 1 -- 172.21.15.122:0/3354105603 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f5b3c094990 con 0x7f5b441605d0 2024-09-17T05:29:51.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.630+0000 7f5b43e00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b240735f0 0x7f5b24075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:51.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.630+0000 7f5b43e00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b240735f0 0x7f5b24075ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b340058d0 tx=0x7f5b34005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:51.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.634+0000 7f5b3be00640 1 -- 172.21.15.122:0/3354105603 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5b3c05f280 con 0x7f5b441605d0 2024-09-17T05:29:51.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.834+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone23140", "target": ["mon-mgr", ""]}) -- 0x7f5b4415dbd0 con 0x7f5b240735f0 2024-09-17T05:29:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b3be00640 1 -- 172.21.15.122:0/3354105603 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b4415dbd0 con 0x7f5b240735f0 2024-09-17T05:29:51.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b240735f0 msgr2=0x7f5b24075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b240735f0 0x7f5b24075ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b340058d0 tx=0x7f5b34005860 comp rx=0 tx=0).stop 2024-09-17T05:29:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b441605d0 msgr2=0x7f5b441552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b441605d0 0x7f5b441552e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00c7c0 tx=0x7f5b3c00cc80 comp rx=0 tx=0).stop 2024-09-17T05:29:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 shutdown_connections 2024-09-17T05:29:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b240735f0 0x7f5b24075ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b441605d0 0x7f5b441552e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b440b1650 0x7f5b4415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 --2- 172.21.15.122:0/3354105603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b440aeb00 0x7f5b4414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:51.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 >> 172.21.15.122:0/3354105603 conn(0x7f5b440d1dd0 msgr2=0x7f5b440b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:51.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 shutdown_connections 2024-09-17T05:29:51.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:51.914+0000 7f5b4aa00640 1 -- 172.21.15.122:0/3354105603 wait complete. 2024-09-17T05:29:51.918 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.919+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.918 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.919+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.918 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.919+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.919 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.919+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.919 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.919+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.936 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone64901 2024-09-17T05:29:51.936 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.939+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.936 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.939+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.937 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.939+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.937 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.939+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:51.937 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:51.939+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.086+0000 7f95f4a00640 1 -- 172.21.15.122:0/3850480505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d4990 msgr2=0x7f95f015ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.086+0000 7f95f4a00640 1 --2- 172.21.15.122:0/3850480505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d4990 0x7f95f015ef20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009bc0 tx=0x7f95e801cac0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.086+0000 7f95f4a00640 1 -- 172.21.15.122:0/3850480505 shutdown_connections 2024-09-17T05:29:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.086+0000 7f95f4a00640 1 --2- 172.21.15.122:0/3850480505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d4990 0x7f95f015ef20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.086+0000 7f95f4a00640 1 -- 172.21.15.122:0/3850480505 >> 172.21.15.122:0/3850480505 conn(0x7f95f00d3e70 msgr2=0x7f95f00d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.086+0000 7f95f4a00640 1 -- 172.21.15.122:0/3850480505 shutdown_connections 2024-09-17T05:29:52.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.086+0000 7f95f4a00640 1 -- 172.21.15.122:0/3850480505 wait complete. 2024-09-17T05:29:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 Processor -- start 2024-09-17T05:29:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 -- start start 2024-09-17T05:29:52.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d2590 0x7f95f0152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f00d4990 0x7f95f00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95f015df50 0x7f95f015c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95f016c1e0 con 0x7f95f015df50 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f95f016c060 con 0x7f95f00d2590 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f95f016c360 con 0x7f95f00d4990 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95f015df50 0x7f95f015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95f015df50 0x7f95f015c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58702/0 (socket says 172.21.15.122:58702) 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 -- 172.21.15.122:0/3310081210 learned_addr learned my addr 172.21.15.122:0/3310081210 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ee800640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d2590 0x7f95f0152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95e5e00640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f00d4990 0x7f95f00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 -- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f00d4990 msgr2=0x7f95f00caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f00d4990 0x7f95f00caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 -- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d2590 msgr2=0x7f95f0152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d2590 0x7f95f0152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 -- 172.21.15.122:0/3310081210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f0153470 con 0x7f95f015df50 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ee800640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d2590 0x7f95f0152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95e5e00640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f00d4990 0x7f95f00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95ef200640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95f015df50 0x7f95f015c180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f95e80196d0 tx=0x7f95e80a5f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95e7400640 1 -- 172.21.15.122:0/3310081210 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80bb070 con 0x7f95f015df50 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95e7400640 1 -- 172.21.15.122:0/3310081210 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95e8004e00 con 0x7f95f015df50 2024-09-17T05:29:52.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f00cb990 con 0x7f95f015df50 2024-09-17T05:29:52.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95e7400640 1 -- 172.21.15.122:0/3310081210 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80b2040 con 0x7f95f015df50 2024-09-17T05:29:52.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f00cbf50 con 0x7f95f015df50 2024-09-17T05:29:52.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.090+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f015ef20 con 0x7f95f015df50 2024-09-17T05:29:52.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.094+0000 7f95e7400640 1 -- 172.21.15.122:0/3310081210 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 97591+0+0 (secure 0 0 0) 0x7f95e80be070 con 0x7f95f015df50 2024-09-17T05:29:52.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.094+0000 7f95e7400640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95cc0735f0 0x7f95cc075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:52.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.094+0000 7f95e7400640 1 -- 172.21.15.122:0/3310081210 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f95e812e260 con 0x7f95f015df50 2024-09-17T05:29:52.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.094+0000 7f95ee800640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95cc0735f0 0x7f95cc075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:52.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.094+0000 7f95ee800640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95cc0735f0 0x7f95cc075ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f95e0001230 tx=0x7f95e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:52.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.098+0000 7f95e7400640 1 -- 172.21.15.122:0/3310081210 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f95e80166b0 con 0x7f95f015df50 2024-09-17T05:29:52.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:52.298+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone64901", "target": ["mon-mgr", ""]}) -- 0x7f95f00d0660 con 0x7f95cc0735f0 2024-09-17T05:29:52.919 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:52.923+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:52.920 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:52.923+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:52.920 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:52.923+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:52.920 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:52.923+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:52.920 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:52.923+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:53.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95e7400640 1 -- 172.21.15.122:0/3310081210 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95f00d0660 con 0x7f95cc0735f0 2024-09-17T05:29:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95cc0735f0 msgr2=0x7f95cc075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95cc0735f0 0x7f95cc075ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f95e0001230 tx=0x7f95e0009040 comp rx=0 tx=0).stop 2024-09-17T05:29:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95f015df50 msgr2=0x7f95f015c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95f015df50 0x7f95f015c180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f95e80196d0 tx=0x7f95e80a5f90 comp rx=0 tx=0).stop 2024-09-17T05:29:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 shutdown_connections 2024-09-17T05:29:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95cc0735f0 0x7f95cc075ab0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95f015df50 0x7f95f015c180 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f00d4990 0x7f95f00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 --2- 172.21.15.122:0/3310081210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f00d2590 0x7f95f0152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 >> 172.21.15.122:0/3310081210 conn(0x7f95f00d3e70 msgr2=0x7f95f015f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 shutdown_connections 2024-09-17T05:29:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.026+0000 7f95f4a00640 1 -- 172.21.15.122:0/3310081210 wait complete. 2024-09-17T05:29:53.062 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone10202 2024-09-17T05:29:53.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.258+0000 7f4e63200640 1 -- 172.21.15.122:0/4183964654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415bcf0 msgr2=0x7f4e6406cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.258+0000 7f4e63200640 1 --2- 172.21.15.122:0/4183964654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415bcf0 0x7f4e6406cc20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4e640cb0f0 tx=0x7f4e5401f0b0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 -- 172.21.15.122:0/4183964654 shutdown_connections 2024-09-17T05:29:53.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 --2- 172.21.15.122:0/4183964654 >> v2:172.21.15.173:3300/0 conn(0x7f4e6415f6e0 0x7f4e640cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 --2- 172.21.15.122:0/4183964654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415bcf0 0x7f4e6406cc20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 -- 172.21.15.122:0/4183964654 >> 172.21.15.122:0/4183964654 conn(0x7f4e640d0b10 msgr2=0x7f4e640d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:53.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 -- 172.21.15.122:0/4183964654 shutdown_connections 2024-09-17T05:29:53.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 -- 172.21.15.122:0/4183964654 wait complete. 2024-09-17T05:29:53.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 Processor -- start 2024-09-17T05:29:53.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 -- start start 2024-09-17T05:29:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e64070b20 0x7f4e640ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415f6e0 0x7f4e641582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6415a920 0x7f4e641f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640ccad0 con 0x7f4e6415f6e0 2024-09-17T05:29:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640cc950 con 0x7f4e64070b20 2024-09-17T05:29:53.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e63200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4e640ccc50 con 0x7f4e6415a920 2024-09-17T05:29:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e61400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415f6e0 0x7f4e641582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e61400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415f6e0 0x7f4e641582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58716/0 (socket says 172.21.15.122:58716) 2024-09-17T05:29:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.262+0000 7f4e61400640 1 -- 172.21.15.122:0/2170404822 learned_addr learned my addr 172.21.15.122:0/2170404822 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e61e00640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e64070b20 0x7f4e640ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e62800640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6415a920 0x7f4e641f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e61400640 1 -- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6415a920 msgr2=0x7f4e641f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e61400640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6415a920 0x7f4e641f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e61400640 1 -- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e64070b20 msgr2=0x7f4e640ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e61400640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e64070b20 0x7f4e640ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e61400640 1 -- 172.21.15.122:0/2170404822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6415ae00 con 0x7f4e6415f6e0 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e61400640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415f6e0 0x7f4e641582a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c410 tx=0x7f4e5000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e5a000640 1 -- 172.21.15.122:0/2170404822 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50017070 con 0x7f4e6415f6e0 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e5a000640 1 -- 172.21.15.122:0/2170404822 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e50003600 con 0x7f4e6415f6e0 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e5a000640 1 -- 172.21.15.122:0/2170404822 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50003a50 con 0x7f4e6415f6e0 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6415b0e0 con 0x7f4e6415f6e0 2024-09-17T05:29:53.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.266+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6415cd60 con 0x7f4e6415f6e0 2024-09-17T05:29:53.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.270+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e640cd1e0 con 0x7f4e6415f6e0 2024-09-17T05:29:53.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.270+0000 7f4e5a000640 1 -- 172.21.15.122:0/2170404822 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 97693+0+0 (secure 0 0 0) 0x7f4e50003bf0 con 0x7f4e6415f6e0 2024-09-17T05:29:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.270+0000 7f4e5a000640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e44073710 0x7f4e44075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:53.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.270+0000 7f4e5a000640 1 -- 172.21.15.122:0/2170404822 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f4e50094040 con 0x7f4e6415f6e0 2024-09-17T05:29:53.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.274+0000 7f4e61e00640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e44073710 0x7f4e44075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:53.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.278+0000 7f4e5a000640 1 -- 172.21.15.122:0/2170404822 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4e5005e7c0 con 0x7f4e6415f6e0 2024-09-17T05:29:53.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.278+0000 7f4e61e00640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e44073710 0x7f4e44075bd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4e540b2000 tx=0x7f4e5401d8f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:53.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:53.574+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone10202", "target": ["mon-mgr", ""]}) -- 0x7f4e6415b2b0 con 0x7f4e44073710 2024-09-17T05:29:54.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.430+0000 7f4e5a000640 1 -- 172.21.15.122:0/2170404822 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e6415b2b0 con 0x7f4e44073710 2024-09-17T05:29:54.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e44073710 msgr2=0x7f4e44075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e44073710 0x7f4e44075bd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4e540b2000 tx=0x7f4e5401d8f0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415f6e0 msgr2=0x7f4e641582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415f6e0 0x7f4e641582a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c410 tx=0x7f4e5000c8d0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 shutdown_connections 2024-09-17T05:29:54.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e44073710 0x7f4e44075bd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6415a920 0x7f4e641f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6415f6e0 0x7f4e641582a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 --2- 172.21.15.122:0/2170404822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e64070b20 0x7f4e640ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 >> 172.21.15.122:0/2170404822 conn(0x7f4e640d0b10 msgr2=0x7f4e64157030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 shutdown_connections 2024-09-17T05:29:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.434+0000 7f4e63200640 1 -- 172.21.15.122:0/2170404822 wait complete. 2024-09-17T05:29:54.460 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone95203 2024-09-17T05:29:54.507 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:54.502+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:54.507 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:54.502+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:54.507 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:54.502+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:54.507 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:54.502+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:54.508 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:54.502+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:54.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.706+0000 7f7a09c00640 1 -- 172.21.15.122:0/483707291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79e8005220 msgr2=0x7f79e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.706+0000 7f7a09c00640 1 --2- 172.21.15.122:0/483707291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79e8005220 0x7f79e800b9d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f79fc001cb0 tx=0x7f79fc01ca40 comp rx=0 tx=0).stop 2024-09-17T05:29:54.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.706+0000 7f7a03400640 1 -- 172.21.15.122:0/483707291 reap_dead start 2024-09-17T05:29:54.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.706+0000 7f7a09c00640 1 -- 172.21.15.122:0/483707291 shutdown_connections 2024-09-17T05:29:54.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.706+0000 7f7a09c00640 1 -- 172.21.15.122:0/483707291 >> 172.21.15.122:0/483707291 conn(0x7f7a040cd740 msgr2=0x7f7a040cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:54.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.706+0000 7f7a09c00640 1 -- 172.21.15.122:0/483707291 shutdown_connections 2024-09-17T05:29:54.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.706+0000 7f7a09c00640 1 -- 172.21.15.122:0/483707291 wait complete. 2024-09-17T05:29:54.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 Processor -- start 2024-09-17T05:29:54.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 -- start start 2024-09-17T05:29:54.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79e8005220 0x7f7a041f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040b3120 0x7f7a0415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a0415c7d0 0x7f7a0415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0406d460 con 0x7f7a0415c7d0 2024-09-17T05:29:54.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0406d2e0 con 0x7f79e8005220 2024-09-17T05:29:54.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7a04160730 con 0x7f7a040b3120 2024-09-17T05:29:54.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a0415c7d0 0x7f7a0415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a0415c7d0 0x7f7a0415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58734/0 (socket says 172.21.15.122:58734) 2024-09-17T05:29:54.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 -- 172.21.15.122:0/1950105159 learned_addr learned my addr 172.21.15.122:0/1950105159 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a02a00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79e8005220 0x7f7a041f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 -- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040b3120 msgr2=0x7f7a0415c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040b3120 0x7f7a0415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 -- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79e8005220 msgr2=0x7f7a041f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79e8005220 0x7f7a041f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 -- 172.21.15.122:0/1950105159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a040c7e30 con 0x7f7a0415c7d0 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a02a00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79e8005220 0x7f7a041f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a03400640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a0415c7d0 0x7f7a0415cc30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f79fc01cf20 tx=0x7f79fc0a5d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f79faa00640 1 -- 172.21.15.122:0/1950105159 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc001880 con 0x7f7a0415c7d0 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f79faa00640 1 -- 172.21.15.122:0/1950105159 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79fc0062a0 con 0x7f7a0415c7d0 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0415e7b0 con 0x7f7a0415c7d0 2024-09-17T05:29:54.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f79faa00640 1 -- 172.21.15.122:0/1950105159 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc006c90 con 0x7f7a0415c7d0 2024-09-17T05:29:54.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.710+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a0415ed70 con 0x7f7a0415c7d0 2024-09-17T05:29:54.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.714+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a04074480 con 0x7f7a0415c7d0 2024-09-17T05:29:54.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.714+0000 7f79faa00640 1 -- 172.21.15.122:0/1950105159 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 97693+0+0 (secure 0 0 0) 0x7f79fc006690 con 0x7f7a0415c7d0 2024-09-17T05:29:54.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.714+0000 7f79faa00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79e4073710 0x7f79e4075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:54.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.714+0000 7f79faa00640 1 -- 172.21.15.122:0/1950105159 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f79fc12ecb0 con 0x7f7a0415c7d0 2024-09-17T05:29:54.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.722+0000 7f79faa00640 1 -- 172.21.15.122:0/1950105159 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f79fc016750 con 0x7f7a0415c7d0 2024-09-17T05:29:54.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.738+0000 7f7a02a00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79e4073710 0x7f79e4075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:54.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:54.746+0000 7f7a02a00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79e4073710 0x7f79e4075bd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f79f4005d10 tx=0x7f79f4005de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:55.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:55.030+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone95203", "target": ["mon-mgr", ""]}) -- 0x7f7a0415f2b0 con 0x7f79e4073710 2024-09-17T05:29:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:55.250+0000 7f79faa00640 1 -- 172.21.15.122:0/1950105159 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 17) ==== 97795+0+0 (secure 0 0 0) 0x7f79fc0f5380 con 0x7f7a0415c7d0 2024-09-17T05:29:56.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.102+0000 7f79faa00640 1 -- 172.21.15.122:0/1950105159 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7a0415f2b0 con 0x7f79e4073710 2024-09-17T05:29:56.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79e4073710 msgr2=0x7f79e4075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79e4073710 0x7f79e4075bd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f79f4005d10 tx=0x7f79f4005de0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a0415c7d0 msgr2=0x7f7a0415cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a0415c7d0 0x7f7a0415cc30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f79fc01cf20 tx=0x7f79fc0a5d90 comp rx=0 tx=0).stop 2024-09-17T05:29:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 shutdown_connections 2024-09-17T05:29:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79e4073710 0x7f79e4075bd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a0415c7d0 0x7f7a0415cc30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040b3120 0x7f7a0415c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 --2- 172.21.15.122:0/1950105159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79e8005220 0x7f7a041f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.110+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 >> 172.21.15.122:0/1950105159 conn(0x7f7a040cd740 msgr2=0x7f7a04070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.114+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 shutdown_connections 2024-09-17T05:29:56.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.114+0000 7f7a09c00640 1 -- 172.21.15.122:0/1950105159 wait complete. 2024-09-17T05:29:56.139 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:29:56.204 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:56.202+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:56.204 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:56.202+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:56.205 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:56.202+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:56.205 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:56.202+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:56.205 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:56.202+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:56.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.338+0000 7f88fa800640 1 -- 172.21.15.122:0/1301122200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415c1f0 msgr2=0x7f88f40cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.338+0000 7f88fa800640 1 --2- 172.21.15.122:0/1301122200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415c1f0 0x7f88f40cee10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f88f40b0fb0 tx=0x7f88e001eff0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.338+0000 7f88fa800640 1 -- 172.21.15.122:0/1301122200 shutdown_connections 2024-09-17T05:29:56.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.338+0000 7f88fa800640 1 --2- 172.21.15.122:0/1301122200 >> v2:172.21.15.173:3300/0 conn(0x7f88f40b0b60 0x7f88f40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.338+0000 7f88fa800640 1 --2- 172.21.15.122:0/1301122200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415c1f0 0x7f88f40cee10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.338+0000 7f88fa800640 1 -- 172.21.15.122:0/1301122200 >> 172.21.15.122:0/1301122200 conn(0x7f88f40d1da0 msgr2=0x7f88f40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.338+0000 7f88fa800640 1 -- 172.21.15.122:0/1301122200 shutdown_connections 2024-09-17T05:29:56.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.338+0000 7f88fa800640 1 -- 172.21.15.122:0/1301122200 wait complete. 2024-09-17T05:29:56.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 Processor -- start 2024-09-17T05:29:56.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 -- start start 2024-09-17T05:29:56.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40b0b60 0x7f88f415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415be10 0x7f88f40c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415c1f0 0x7f88f41f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88f40cd4a0 con 0x7f88f415be10 2024-09-17T05:29:56.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88f40cd320 con 0x7f88f40b0b60 2024-09-17T05:29:56.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f88f40cd620 con 0x7f88f415c1f0 2024-09-17T05:29:56.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415be10 0x7f88f40c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415be10 0x7f88f40c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58760/0 (socket says 172.21.15.122:58760) 2024-09-17T05:29:56.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 -- 172.21.15.122:0/164715651 learned_addr learned my addr 172.21.15.122:0/164715651 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:56.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 -- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415c1f0 msgr2=0x7f88f41f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415c1f0 0x7f88f41f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 -- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40b0b60 msgr2=0x7f88f415aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40b0b60 0x7f88f415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 -- 172.21.15.122:0/164715651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f40c7d10 con 0x7f88f415be10 2024-09-17T05:29:56.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88f2a00640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415be10 0x7f88f40c77d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f88e0000c10 tx=0x7f88e00a8d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88eb400640 1 -- 172.21.15.122:0/164715651 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e00bf070 con 0x7f88f415be10 2024-09-17T05:29:56.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88eb400640 1 -- 172.21.15.122:0/164715651 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88e0005320 con 0x7f88f415be10 2024-09-17T05:29:56.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f40c7fb0 con 0x7f88f415be10 2024-09-17T05:29:56.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88eb400640 1 -- 172.21.15.122:0/164715651 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e00b5040 con 0x7f88f415be10 2024-09-17T05:29:56.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.342+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f41f60e0 con 0x7f88f415be10 2024-09-17T05:29:56.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.346+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f40cdb60 con 0x7f88f415be10 2024-09-17T05:29:56.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.346+0000 7f88eb400640 1 -- 172.21.15.122:0/164715651 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 97795+0+0 (secure 0 0 0) 0x7f88e00b1700 con 0x7f88f415be10 2024-09-17T05:29:56.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.346+0000 7f88eb400640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4073830 0x7f88d4075cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.346+0000 7f88eb400640 1 -- 172.21.15.122:0/164715651 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f88e0131db0 con 0x7f88f415be10 2024-09-17T05:29:56.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.350+0000 7f88eb400640 1 -- 172.21.15.122:0/164715651 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f88e0018b50 con 0x7f88f415be10 2024-09-17T05:29:56.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.374+0000 7f88f3400640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4073830 0x7f88d4075cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.386+0000 7f88f3400640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4073830 0x7f88d4075cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88e4007fd0 tx=0x7f88e40078e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.706+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f88f4070350 con 0x7f88f415be10 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.706+0000 7f88eb400640 1 -- 172.21.15.122:0/164715651 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2168 (secure 0 0 0) 0x7f88e00fbe50 con 0x7f88f415be10 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:29:56.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 317, 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:29:56.709 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 48, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:29:56.710 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 2696, 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 11174931436, 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 22138183680, 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 750955929600, 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:29:56.711 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 42281168, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 140492823, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 10, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 40 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 27, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:28:16:887773+0000", 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 3, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:29:56.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 3, 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "d", 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 5388 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:29:56.713 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 10, 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:29:12.508973+0000", 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:29:56.714 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:29:56.715 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "1 ongoing clones - average progress is 0.859% (0s)\n [............................] ", 2024-09-17T05:29:56.715 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.0085899997502565384, 2024-09-17T05:29:56.715 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:29:56.715 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:29:56.715 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:29:56.715 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4073830 msgr2=0x7f88d4075cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4073830 0x7f88d4075cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88e4007fd0 tx=0x7f88e40078e0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415be10 msgr2=0x7f88f40c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415be10 0x7f88f40c77d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f88e0000c10 tx=0x7f88e00a8d70 comp rx=0 tx=0).stop 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 shutdown_connections 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4073830 0x7f88d4075cf0 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f88e4007fd0 tx=0x7f88e40078e0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415c1f0 0x7f88f41f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f415be10 0x7f88f40c77d0 secure :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f88e0000c10 tx=0x7f88e00a8d70 comp rx=0 tx=0).stop 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 --2- 172.21.15.122:0/164715651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40b0b60 0x7f88f415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.710+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 >> 172.21.15.122:0/164715651 conn(0x7f88f40d1da0 msgr2=0x7f88f40d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.714+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 shutdown_connections 2024-09-17T05:29:56.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.714+0000 7f88fa800640 1 -- 172.21.15.122:0/164715651 wait complete. 2024-09-17T05:29:56.744 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone23140 2024-09-17T05:29:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.934+0000 7fbebec00640 1 -- 172.21.15.122:0/3719434148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb815ac70 msgr2=0x7fbeb80cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.934+0000 7fbebec00640 1 --2- 172.21.15.122:0/3719434148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb815ac70 0x7fbeb80cf2a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80b3320 tx=0x7fbea801c600 comp rx=0 tx=0).stop 2024-09-17T05:29:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.938+0000 7fbebec00640 1 -- 172.21.15.122:0/3719434148 shutdown_connections 2024-09-17T05:29:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.938+0000 7fbebec00640 1 --2- 172.21.15.122:0/3719434148 >> v2:172.21.15.173:3300/0 conn(0x7fbeb80cbdb0 0x7fbeb80b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.938+0000 7fbebec00640 1 --2- 172.21.15.122:0/3719434148 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb80b2c80 0x7fbeb8073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.938+0000 7fbebec00640 1 --2- 172.21.15.122:0/3719434148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb815ac70 0x7fbeb80cf2a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.938+0000 7fbebec00640 1 -- 172.21.15.122:0/3719434148 >> 172.21.15.122:0/3719434148 conn(0x7fbeb80cd740 msgr2=0x7fbeb80cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.938+0000 7fbebec00640 1 -- 172.21.15.122:0/3719434148 shutdown_connections 2024-09-17T05:29:56.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.938+0000 7fbebec00640 1 -- 172.21.15.122:0/3719434148 wait complete. 2024-09-17T05:29:56.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbebec00640 1 Processor -- start 2024-09-17T05:29:56.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbebec00640 1 -- start start 2024-09-17T05:29:56.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbebec00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb80b2c80 0x7fbeb8157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbebec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb80cbdb0 0x7fbeb81f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbebec00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb815ac70 0x7fbeb81f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbebec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb8070030 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbebec00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb806feb0 con 0x7fbeb815ac70 2024-09-17T05:29:56.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbebec00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb80701b0 con 0x7fbeb80b2c80 2024-09-17T05:29:56.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb80cbdb0 0x7fbeb81f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb80cbdb0 0x7fbeb81f8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58780/0 (socket says 172.21.15.122:58780) 2024-09-17T05:29:56.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7400640 1 -- 172.21.15.122:0/3255441193 learned_addr learned my addr 172.21.15.122:0/3255441193 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:56.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7e00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb80b2c80 0x7fbeb8157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:56.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7400640 1 -- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb80b2c80 msgr2=0x7fbeb8157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7400640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb80b2c80 0x7fbeb8157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7400640 1 -- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb815ac70 msgr2=0x7fbeb81f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:56.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7400640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb815ac70 0x7fbeb81f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:56.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.942+0000 7fbeb7400640 1 -- 172.21.15.122:0/3255441193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb8073b00 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.946+0000 7fbeb7400640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb80cbdb0 0x7fbeb81f8d50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbea400b550 tx=0x7fbea400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:56.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.946+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4014070 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.946+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbea4010980 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.946+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4010dd0 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.946+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb80c5b30 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.946+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb80c60f0 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.946+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb80c62f0 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.950+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 97795+0+0 (secure 0 0 0) 0x7fbea4017070 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.950+0000 7fbeafe00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe98073860 0x7fbe98075d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:56.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.950+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fbea4094f70 con 0x7fbeb80cbdb0 2024-09-17T05:29:56.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:56.958+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbeb80c62f0 con 0x7fbeb80cbdb0 2024-09-17T05:29:57.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:57.002+0000 7fbeb7e00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe98073860 0x7fbe98075d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:57.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:57.014+0000 7fbeb7e00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe98073860 0x7fbe98075d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbea80af000 tx=0x7fbea801ae40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:57.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:57.238+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone23140", "target": ["mon-mgr", ""]}) -- 0x7fbeb8070750 con 0x7fbe98073860 2024-09-17T05:29:57.296 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:57.294+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:57.296 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:57.294+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:57.296 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:57.294+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:57.296 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:57.294+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:57.297 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:29:57.294+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:29:57.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:57.314+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 18) ==== 97846+0+0 (secure 0 0 0) 0x7fbea405b230 con 0x7fbeb80cbdb0 2024-09-17T05:29:59.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.394+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7fbea405e220 con 0x7fbeb80cbdb0 2024-09-17T05:29:59.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbeafe00640 1 -- 172.21.15.122:0/3255441193 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbeb8070750 con 0x7fbe98073860 2024-09-17T05:29:59.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe98073860 msgr2=0x7fbe98075d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe98073860 0x7fbe98075d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbea80af000 tx=0x7fbea801ae40 comp rx=0 tx=0).stop 2024-09-17T05:29:59.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb80cbdb0 msgr2=0x7fbeb81f8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb80cbdb0 0x7fbeb81f8d50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbea400b550 tx=0x7fbea400ba10 comp rx=0 tx=0).stop 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 shutdown_connections 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe98073860 0x7fbe98075d20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb815ac70 0x7fbeb81f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb80cbdb0 0x7fbeb81f8d50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 --2- 172.21.15.122:0/3255441193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb80b2c80 0x7fbeb8157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 >> 172.21.15.122:0/3255441193 conn(0x7fbeb80cd740 msgr2=0x7fbeb815c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 shutdown_connections 2024-09-17T05:29:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.542+0000 7fbebec00640 1 -- 172.21.15.122:0/3255441193 wait complete. 2024-09-17T05:29:59.568 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone64901 2024-09-17T05:29:59.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- 172.21.15.122:0/2400112075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 msgr2=0x7fc5b00cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 --2- 172.21.15.122:0/2400112075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 0x7fc5b00cc660 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00cfbb0 tx=0x7fc5a401c890 comp rx=0 tx=0).stop 2024-09-17T05:29:59.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- 172.21.15.122:0/2400112075 shutdown_connections 2024-09-17T05:29:59.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 --2- 172.21.15.122:0/2400112075 >> v2:172.21.15.173:3300/0 conn(0x7fc5b00b2bf0 0x7fc5b00b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 --2- 172.21.15.122:0/2400112075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5b00cd520 0x7fc5b00b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 --2- 172.21.15.122:0/2400112075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 0x7fc5b00cc660 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- 172.21.15.122:0/2400112075 >> 172.21.15.122:0/2400112075 conn(0x7fc5b00b18f0 msgr2=0x7fc5b00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:29:59.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- 172.21.15.122:0/2400112075 shutdown_connections 2024-09-17T05:29:59.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- 172.21.15.122:0/2400112075 wait complete. 2024-09-17T05:29:59.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 Processor -- start 2024-09-17T05:29:59.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- start start 2024-09-17T05:29:59.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5b00b2bf0 0x7fc5b00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:59.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 0x7fc5b0152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:59.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5b00cd520 0x7fc5b0159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:59.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b015b1f0 con 0x7fc5b00cd100 2024-09-17T05:29:59.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b015b070 con 0x7fc5b00cd520 2024-09-17T05:29:59.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.766+0000 7fc5b8600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b015b370 con 0x7fc5b00b2bf0 2024-09-17T05:29:59.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 0x7fc5b0152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 0x7fc5b0152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58800/0 (socket says 172.21.15.122:58800) 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 -- 172.21.15.122:0/3151966528 learned_addr learned my addr 172.21.15.122:0/3151966528 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b6600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5b00cd520 0x7fc5b0159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 -- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5b00b2bf0 msgr2=0x7fc5b00acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5b00b2bf0 0x7fc5b00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 -- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5b00cd520 msgr2=0x7fc5b0159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5b00cd520 0x7fc5b0159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 -- 172.21.15.122:0/3151966528 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b00ad470 con 0x7fc5b00cd100 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b6600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5b00cd520 0x7fc5b0159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b5200640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 0x7fc5b0152ea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b550 tx=0x7fc59c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5ad600640 1 -- 172.21.15.122:0/3151966528 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c014070 con 0x7fc5b00cd100 2024-09-17T05:29:59.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5ad600640 1 -- 172.21.15.122:0/3151966528 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc59c010980 con 0x7fc5b00cd100 2024-09-17T05:29:59.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5ad600640 1 -- 172.21.15.122:0/3151966528 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c010dd0 con 0x7fc5b00cd100 2024-09-17T05:29:59.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b00ad750 con 0x7fc5b00cd100 2024-09-17T05:29:59.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b00add10 con 0x7fc5b00cd100 2024-09-17T05:29:59.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b01533e0 con 0x7fc5b00cd100 2024-09-17T05:29:59.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.770+0000 7fc5ad600640 1 -- 172.21.15.122:0/3151966528 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7fc59c017070 con 0x7fc5b00cd100 2024-09-17T05:29:59.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.774+0000 7fc5ad600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc590073980 0x7fc590075e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:29:59.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.774+0000 7fc5ad600640 1 -- 172.21.15.122:0/3151966528 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fc59c095360 con 0x7fc5b00cd100 2024-09-17T05:29:59.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.778+0000 7fc5ad600640 1 -- 172.21.15.122:0/3151966528 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc59c05f2e0 con 0x7fc5b00cd100 2024-09-17T05:29:59.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.806+0000 7fc5b5c00640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc590073980 0x7fc590075e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:29:59.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:29:59.822+0000 7fc5b5c00640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc590073980 0x7fc590075e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40af000 tx=0x7fc5a4019630 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:00.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:00.122+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone64901", "target": ["mon-mgr", ""]}) -- 0x7fc5b0069410 con 0x7fc590073980 2024-09-17T05:30:01.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.342+0000 7fc5ad600640 1 -- 172.21.15.122:0/3151966528 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5b0069410 con 0x7fc590073980 2024-09-17T05:30:01.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc590073980 msgr2=0x7fc590075e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:01.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc590073980 0x7fc590075e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40af000 tx=0x7fc5a4019630 comp rx=0 tx=0).stop 2024-09-17T05:30:01.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 msgr2=0x7fc5b0152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:01.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 0x7fc5b0152ea0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b550 tx=0x7fc59c00ba10 comp rx=0 tx=0).stop 2024-09-17T05:30:01.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 shutdown_connections 2024-09-17T05:30:01.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc590073980 0x7fc590075e40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:01.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5b00cd520 0x7fc5b0159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:01.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5b00cd100 0x7fc5b0152ea0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:01.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 --2- 172.21.15.122:0/3151966528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5b00b2bf0 0x7fc5b00acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:01.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 >> 172.21.15.122:0/3151966528 conn(0x7fc5b00b18f0 msgr2=0x7fc5b00739a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:01.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 shutdown_connections 2024-09-17T05:30:01.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.346+0000 7fc5b8600640 1 -- 172.21.15.122:0/3151966528 wait complete. 2024-09-17T05:30:01.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone10202 2024-09-17T05:30:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.626+0000 7f34b3a00640 1 -- 172.21.15.122:0/4206894924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 msgr2=0x7f3494003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.626+0000 7f34b3a00640 1 --2- 172.21.15.122:0/4206894924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 0x7f3494003e60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f34a0006ef0 tx=0x7f34a001ce60 comp rx=0 tx=0).stop 2024-09-17T05:30:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.626+0000 7f34b3a00640 1 -- 172.21.15.122:0/4206894924 shutdown_connections 2024-09-17T05:30:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.626+0000 7f34b3a00640 1 --2- 172.21.15.122:0/4206894924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 0x7f3494003e60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.626+0000 7f34b3a00640 1 -- 172.21.15.122:0/4206894924 >> 172.21.15.122:0/4206894924 conn(0x7f34ac0d0470 msgr2=0x7f34ac0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.626+0000 7f34b3a00640 1 -- 172.21.15.122:0/4206894924 shutdown_connections 2024-09-17T05:30:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.626+0000 7f34b3a00640 1 -- 172.21.15.122:0/4206894924 wait complete. 2024-09-17T05:30:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.626+0000 7f34b3a00640 1 Processor -- start 2024-09-17T05:30:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 -- start start 2024-09-17T05:30:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 0x7f34ac1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34ac157f70 0x7f34ac0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34ac0c8da0 0x7f34ac0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac068ae0 con 0x7f3494003a60 2024-09-17T05:30:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac068960 con 0x7f34ac0c8da0 2024-09-17T05:30:01.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac0cc550 con 0x7f34ac157f70 2024-09-17T05:30:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 0x7f34ac1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 0x7f34ac1576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41352/0 (socket says 172.21.15.122:41352) 2024-09-17T05:30:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 -- 172.21.15.122:0/955259238 learned_addr learned my addr 172.21.15.122:0/955259238 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 -- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34ac157f70 msgr2=0x7f34ac0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34ac157f70 0x7f34ac0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 -- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34ac0c8da0 msgr2=0x7f34ac0c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34ac0c8da0 0x7f34ac0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:01.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 -- 172.21.15.122:0/955259238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac15e980 con 0x7f3494003a60 2024-09-17T05:30:01.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b1000640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 0x7f34ac1576f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f34a0000fc0 tx=0x7f34a0006bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:01.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34a8c00640 1 -- 172.21.15.122:0/955259238 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0006480 con 0x7f3494003a60 2024-09-17T05:30:01.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34a8c00640 1 -- 172.21.15.122:0/955259238 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34a00060d0 con 0x7f3494003a60 2024-09-17T05:30:01.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34a8c00640 1 -- 172.21.15.122:0/955259238 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00a5910 con 0x7f3494003a60 2024-09-17T05:30:01.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac0c9760 con 0x7f3494003a60 2024-09-17T05:30:01.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.630+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac15ec20 con 0x7f3494003a60 2024-09-17T05:30:01.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.634+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac0cca60 con 0x7f3494003a60 2024-09-17T05:30:01.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.638+0000 7f34a8c00640 1 -- 172.21.15.122:0/955259238 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f34a00a5ab0 con 0x7f3494003a60 2024-09-17T05:30:01.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.638+0000 7f34a8c00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f348c073a20 0x7f348c075ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:01.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.638+0000 7f34a8c00640 1 -- 172.21.15.122:0/955259238 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f34a012e810 con 0x7f3494003a60 2024-09-17T05:30:01.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.642+0000 7f34abe00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f348c073a20 0x7f348c075ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:01.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.642+0000 7f34abe00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f348c073a20 0x7f348c075ee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0c74a0 tx=0x7f3498005b80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:01.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.642+0000 7f34a8c00640 1 -- 172.21.15.122:0/955259238 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f34a00be060 con 0x7f3494003a60 2024-09-17T05:30:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:01.998+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone10202", "target": ["mon-mgr", ""]}) -- 0x7f34ac0af460 con 0x7f348c073a20 2024-09-17T05:30:02.153 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:30:02.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:30:02.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.614+0000 7f34a8c00640 1 -- 172.21.15.122:0/955259238 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34ac0af460 con 0x7f348c073a20 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f348c073a20 msgr2=0x7f348c075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f348c073a20 0x7f348c075ee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0c74a0 tx=0x7f3498005b80 comp rx=0 tx=0).stop 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 msgr2=0x7f34ac1576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 0x7f34ac1576f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f34a0000fc0 tx=0x7f34a0006bd0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 shutdown_connections 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f348c073a20 0x7f348c075ee0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34ac0c8da0 0x7f34ac0c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34ac157f70 0x7f34ac0c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 --2- 172.21.15.122:0/955259238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3494003a60 0x7f34ac1576f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 >> 172.21.15.122:0/955259238 conn(0x7f34ac0d0470 msgr2=0x7f34ac0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 shutdown_connections 2024-09-17T05:30:02.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.618+0000 7f34b3a00640 1 -- 172.21.15.122:0/955259238 wait complete. 2024-09-17T05:30:02.650 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone95203 2024-09-17T05:30:02.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.882+0000 7f373e200640 1 -- 172.21.15.122:0/3994213511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 msgr2=0x7f37380b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:02.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.882+0000 7f373e200640 1 --2- 172.21.15.122:0/3994213511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 0x7f37380b1a30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3724009900 tx=0x7f372401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 -- 172.21.15.122:0/3994213511 shutdown_connections 2024-09-17T05:30:02.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 --2- 172.21.15.122:0/3994213511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37380afb10 0x7f37380affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 --2- 172.21.15.122:0/3994213511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 0x7f37380b1a30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 -- 172.21.15.122:0/3994213511 >> 172.21.15.122:0/3994213511 conn(0x7f37380d1dd0 msgr2=0x7f37380d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:02.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 -- 172.21.15.122:0/3994213511 shutdown_connections 2024-09-17T05:30:02.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 -- 172.21.15.122:0/3994213511 wait complete. 2024-09-17T05:30:02.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 Processor -- start 2024-09-17T05:30:02.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 -- start start 2024-09-17T05:30:02.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37380afb10 0x7f373814a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 0x7f3738159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37381605d0 0x7f37380b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3738169540 con 0x7f37380b1650 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37380b1430 con 0x7f37381605d0 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f373e200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f37381696c0 con 0x7f37380afb10 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f3736a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 0x7f3738159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f3736a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 0x7f3738159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41358/0 (socket says 172.21.15.122:41358) 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f3736a00640 1 -- 172.21.15.122:0/3776912704 learned_addr learned my addr 172.21.15.122:0/3776912704 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f3736a00640 1 -- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37380afb10 msgr2=0x7f373814a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:02.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f3736a00640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37380afb10 0x7f373814a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f3736a00640 1 -- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37381605d0 msgr2=0x7f37380b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:02.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f3736a00640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37381605d0 0x7f37380b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:02.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.886+0000 7f3736a00640 1 -- 172.21.15.122:0/3776912704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37380c7120 con 0x7f37380b1650 2024-09-17T05:30:02.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.890+0000 7f3736a00640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 0x7f3738159180 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3724009850 tx=0x7f37240a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:02.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.890+0000 7f372f400640 1 -- 172.21.15.122:0/3776912704 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37240b2040 con 0x7f37380b1650 2024-09-17T05:30:02.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.890+0000 7f372f400640 1 -- 172.21.15.122:0/3776912704 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3724004340 con 0x7f37380b1650 2024-09-17T05:30:02.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.890+0000 7f372f400640 1 -- 172.21.15.122:0/3776912704 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724005280 con 0x7f37380b1650 2024-09-17T05:30:02.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.890+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37380c7400 con 0x7f37380b1650 2024-09-17T05:30:02.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.890+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37380b6080 con 0x7f37380b1650 2024-09-17T05:30:02.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.890+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37380affc0 con 0x7f37380b1650 2024-09-17T05:30:02.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.890+0000 7f372f400640 1 -- 172.21.15.122:0/3776912704 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f3724005420 con 0x7f37380b1650 2024-09-17T05:30:02.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.894+0000 7f372f400640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3718073950 0x7f3718075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:02.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.894+0000 7f372f400640 1 -- 172.21.15.122:0/3776912704 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f372412ec80 con 0x7f37380b1650 2024-09-17T05:30:02.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.898+0000 7f372f400640 1 -- 172.21.15.122:0/3776912704 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f37240164d0 con 0x7f37380b1650 2024-09-17T05:30:02.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.902+0000 7f3737400640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3718073950 0x7f3718075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:02.906+0000 7f3737400640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3718073950 0x7f3718075e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3728009650 tx=0x7f3728006bb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.218+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone95203", "target": ["mon-mgr", ""]}) -- 0x7f37380d1080 con 0x7f3718073950 2024-09-17T05:30:03.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.374+0000 7f372f400640 1 -- 172.21.15.122:0/3776912704 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f37380d1080 con 0x7f3718073950 2024-09-17T05:30:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3718073950 msgr2=0x7f3718075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3718073950 0x7f3718075e10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3728009650 tx=0x7f3728006bb0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 msgr2=0x7f3738159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 0x7f3738159180 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3724009850 tx=0x7f37240a5630 comp rx=0 tx=0).stop 2024-09-17T05:30:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 shutdown_connections 2024-09-17T05:30:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3718073950 0x7f3718075e10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37381605d0 0x7f37380b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37380b1650 0x7f3738159180 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 --2- 172.21.15.122:0/3776912704 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37380afb10 0x7f373814a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 >> 172.21.15.122:0/3776912704 conn(0x7f37380d1dd0 msgr2=0x7f37380cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 shutdown_connections 2024-09-17T05:30:03.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.378+0000 7f373e200640 1 -- 172.21.15.122:0/3776912704 wait complete. 2024-09-17T05:30:03.397 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:30:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 -- 172.21.15.122:0/2370658300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415b890 msgr2=0x7fb3a40ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2370658300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415b890 0x7fb3a40ce010 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b04f0 tx=0x7fb39401cd50 comp rx=0 tx=0).stop 2024-09-17T05:30:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 -- 172.21.15.122:0/2370658300 shutdown_connections 2024-09-17T05:30:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2370658300 >> v2:172.21.15.173:3300/0 conn(0x7fb3a406cc20 0x7fb3a406d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2370658300 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3a415bc70 0x7fb3a40cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2370658300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415b890 0x7fb3a40ce010 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 -- 172.21.15.122:0/2370658300 >> 172.21.15.122:0/2370658300 conn(0x7fb3a406dd30 msgr2=0x7fb3a406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:03.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 -- 172.21.15.122:0/2370658300 shutdown_connections 2024-09-17T05:30:03.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.542+0000 7fb3a9200640 1 -- 172.21.15.122:0/2370658300 wait complete. 2024-09-17T05:30:03.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 Processor -- start 2024-09-17T05:30:03.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 -- start start 2024-09-17T05:30:03.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a406cc20 0x7fb3a4155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:03.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3a415b890 0x7fb3a4156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415bc70 0x7fb3a4155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a40746b0 con 0x7fb3a415bc70 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a4074530 con 0x7fb3a406cc20 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a40cb150 con 0x7fb3a415b890 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415bc70 0x7fb3a4155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415bc70 0x7fb3a4155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41388/0 (socket says 172.21.15.122:41388) 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a2a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3a415b890 0x7fb3a4156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 -- 172.21.15.122:0/2784411162 learned_addr learned my addr 172.21.15.122:0/2784411162 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3400640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a406cc20 0x7fb3a4155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 -- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3a415b890 msgr2=0x7fb3a4156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3a415b890 0x7fb3a4156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 -- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a406cc20 msgr2=0x7fb3a4155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a406cc20 0x7fb3a4155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 -- 172.21.15.122:0/2784411162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a4155770 con 0x7fb3a415bc70 2024-09-17T05:30:03.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a3e00640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415bc70 0x7fb3a4155230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00d750 tx=0x7fb39c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:03.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb39b400640 1 -- 172.21.15.122:0/2784411162 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c017070 con 0x7fb3a415bc70 2024-09-17T05:30:03.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb39b400640 1 -- 172.21.15.122:0/2784411162 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb39c009d20 con 0x7fb3a415bc70 2024-09-17T05:30:03.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a40c8e30 con 0x7fb3a415bc70 2024-09-17T05:30:03.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb39b400640 1 -- 172.21.15.122:0/2784411162 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c00de60 con 0x7fb3a415bc70 2024-09-17T05:30:03.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a415c570 con 0x7fb3a415bc70 2024-09-17T05:30:03.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.546+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a40cdff0 con 0x7fb3a415bc70 2024-09-17T05:30:03.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.550+0000 7fb39b400640 1 -- 172.21.15.122:0/2784411162 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7fb39c005280 con 0x7fb3a415bc70 2024-09-17T05:30:03.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.550+0000 7fb39b400640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb384073950 0x7fb384075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:03.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.550+0000 7fb39b400640 1 -- 172.21.15.122:0/2784411162 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fb39c095560 con 0x7fb3a415bc70 2024-09-17T05:30:03.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.550+0000 7fb3a3400640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb384073950 0x7fb384075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:03.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.550+0000 7fb3a3400640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb384073950 0x7fb384075e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb3940af000 tx=0x7fb3940a5f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:03.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.554+0000 7fb39b400640 1 -- 172.21.15.122:0/2784411162 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb39c05ef70 con 0x7fb3a415bc70 2024-09-17T05:30:03.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.742+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb3a4069e60 con 0x7fb384073950 2024-09-17T05:30:03.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.742+0000 7fb39b400640 1 -- 172.21.15.122:0/2784411162 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb3940a5d50 con 0x7fb384073950 2024-09-17T05:30:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:30:03.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:03.744 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:30:03.744 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:03.744 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:30:03.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb384073950 msgr2=0x7fb384075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb384073950 0x7fb384075e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb3940af000 tx=0x7fb3940a5f90 comp rx=0 tx=0).stop 2024-09-17T05:30:03.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415bc70 msgr2=0x7fb3a4155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415bc70 0x7fb3a4155230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00d750 tx=0x7fb39c00dc10 comp rx=0 tx=0).stop 2024-09-17T05:30:03.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 shutdown_connections 2024-09-17T05:30:03.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb384073950 0x7fb384075e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3a415bc70 0x7fb3a4155230 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3a415b890 0x7fb3a4156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 --2- 172.21.15.122:0/2784411162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3a406cc20 0x7fb3a4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 >> 172.21.15.122:0/2784411162 conn(0x7fb3a406dd30 msgr2=0x7fb3a40b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:03.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 shutdown_connections 2024-09-17T05:30:03.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.746+0000 7fb3a9200640 1 -- 172.21.15.122:0/2784411162 wait complete. 2024-09-17T05:30:03.766 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:30:03.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 -- 172.21.15.122:0/3169824547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 msgr2=0x7fc9f4073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/3169824547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 0x7fc9f4073980 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009900 tx=0x7fc9e401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 -- 172.21.15.122:0/3169824547 shutdown_connections 2024-09-17T05:30:03.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/3169824547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9f40cb410 0x7fc9f40d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/3169824547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 0x7fc9f4073980 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 -- 172.21.15.122:0/3169824547 >> 172.21.15.122:0/3169824547 conn(0x7fc9f40d0d30 msgr2=0x7fc9f40d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:03.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 -- 172.21.15.122:0/3169824547 shutdown_connections 2024-09-17T05:30:03.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 -- 172.21.15.122:0/3169824547 wait complete. 2024-09-17T05:30:03.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 Processor -- start 2024-09-17T05:30:03.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.910+0000 7fc9f8a00640 1 -- start start 2024-09-17T05:30:03.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f406cc20 0x7fc9f4151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:03.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 0x7fc9f414e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:03.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9f40cb410 0x7fc9f4152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:03.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4169310 con 0x7fc9f4074650 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4169190 con 0x7fc9f406cc20 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f4169490 con 0x7fc9f40cb410 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 0x7fc9f414e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f406cc20 0x7fc9f4151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f406cc20 0x7fc9f4151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:40778/0 (socket says 172.21.15.122:40778) 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 0x7fc9f414e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41412/0 (socket says 172.21.15.122:41412) 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f2800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9f40cb410 0x7fc9f4152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1e00640 1 -- 172.21.15.122:0/1942432052 learned_addr learned my addr 172.21.15.122:0/1942432052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1400640 1 -- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9f40cb410 msgr2=0x7fc9f4152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1400640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9f40cb410 0x7fc9f4152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1400640 1 -- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f406cc20 msgr2=0x7fc9f4151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1400640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f406cc20 0x7fc9f4151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:03.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1400640 1 -- 172.21.15.122:0/1942432052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f40708d0 con 0x7fc9f4074650 2024-09-17T05:30:03.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1e00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f406cc20 0x7fc9f4151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:03.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f1400640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 0x7fc9f414e190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000e9f0 tx=0x7fc9e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:03.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9ea000640 1 -- 172.21.15.122:0/1942432052 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000ccc0 con 0x7fc9f4074650 2024-09-17T05:30:03.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9ea000640 1 -- 172.21.15.122:0/1942432052 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9e0010430 con 0x7fc9f4074650 2024-09-17T05:30:03.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9ea000640 1 -- 172.21.15.122:0/1942432052 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0010860 con 0x7fc9f4074650 2024-09-17T05:30:03.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f4070bb0 con 0x7fc9f4074650 2024-09-17T05:30:03.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f414e9d0 con 0x7fc9f4074650 2024-09-17T05:30:03.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.914+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f4073980 con 0x7fc9f4074650 2024-09-17T05:30:03.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.918+0000 7fc9ea000640 1 -- 172.21.15.122:0/1942432052 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7fc9e0010a00 con 0x7fc9f4074650 2024-09-17T05:30:03.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.918+0000 7fc9ea000640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9d4073950 0x7fc9d4075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:03.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.918+0000 7fc9ea000640 1 -- 172.21.15.122:0/1942432052 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fc9e0095dd0 con 0x7fc9f4074650 2024-09-17T05:30:03.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.918+0000 7fc9f1e00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9d4073950 0x7fc9d4075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:03.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.918+0000 7fc9f1e00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9d4073950 0x7fc9d4075e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009230 tx=0x7fc9e401b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:03.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:03.922+0000 7fc9ea000640 1 -- 172.21.15.122:0/1942432052 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc9e005f7e0 con 0x7fc9f4074650 2024-09-17T05:30:04.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.122+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9f4070680 con 0x7fc9d4073950 2024-09-17T05:30:04.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.126+0000 7fc9ea000640 1 -- 172.21.15.122:0/1942432052 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+234 (secure 0 0 0) 0x7fc9f4070680 con 0x7fc9d4073950 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone95203" 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sv1" 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone23140" 2024-09-17T05:30:04.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:04.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:04.130 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone64901" 2024-09-17T05:30:04.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:04.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:04.130 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone10202" 2024-09-17T05:30:04.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:04.130 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:30:04.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9d4073950 msgr2=0x7fc9d4075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:04.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9d4073950 0x7fc9d4075e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009230 tx=0x7fc9e401b120 comp rx=0 tx=0).stop 2024-09-17T05:30:04.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 msgr2=0x7fc9f414e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:04.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 0x7fc9f414e190 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000e9f0 tx=0x7fc9e000eeb0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 shutdown_connections 2024-09-17T05:30:04.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9d4073950 0x7fc9d4075e10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9f40cb410 0x7fc9f4152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9f4074650 0x7fc9f414e190 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 --2- 172.21.15.122:0/1942432052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9f406cc20 0x7fc9f4151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 >> 172.21.15.122:0/1942432052 conn(0x7fc9f40d0d30 msgr2=0x7fc9f40b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:04.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 shutdown_connections 2024-09-17T05:30:04.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.130+0000 7fc9f8a00640 1 -- 172.21.15.122:0/1942432052 wait complete. 2024-09-17T05:30:04.152 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone95203 2024-09-17T05:30:04.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.298+0000 7f7528a00640 1 -- 172.21.15.122:0/1401169540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 msgr2=0x7f75240d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:04.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.298+0000 7f7528a00640 1 --2- 172.21.15.122:0/1401169540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 0x7f75240d28c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7514009c80 tx=0x7f751401cd70 comp rx=0 tx=0).stop 2024-09-17T05:30:04.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.298+0000 7f7528a00640 1 -- 172.21.15.122:0/1401169540 shutdown_connections 2024-09-17T05:30:04.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.298+0000 7f7528a00640 1 --2- 172.21.15.122:0/1401169540 >> v2:172.21.15.173:3300/0 conn(0x7f75240d02e0 0x7f75240d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.298+0000 7f7528a00640 1 --2- 172.21.15.122:0/1401169540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 0x7f75240d28c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.298+0000 7f7528a00640 1 -- 172.21.15.122:0/1401169540 >> 172.21.15.122:0/1401169540 conn(0x7f75240d1da0 msgr2=0x7f75240d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:04.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.298+0000 7f7528a00640 1 -- 172.21.15.122:0/1401169540 shutdown_connections 2024-09-17T05:30:04.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.298+0000 7f7528a00640 1 -- 172.21.15.122:0/1401169540 wait complete. 2024-09-17T05:30:04.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 Processor -- start 2024-09-17T05:30:04.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 -- start start 2024-09-17T05:30:04.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 0x7f75240cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:04.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75240d02e0 0x7f75240c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:04.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f752415eff0 0x7f75240cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:04.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75241690e0 con 0x7f752415eff0 2024-09-17T05:30:04.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7524168f60 con 0x7f75240d02e0 2024-09-17T05:30:04.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7524169260 con 0x7f75240cff00 2024-09-17T05:30:04.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 0x7f75240cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:04.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f752415eff0 0x7f75240cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:04.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 0x7f75240cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:37986/0 (socket says 172.21.15.122:37986) 2024-09-17T05:30:04.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523400640 1 -- 172.21.15.122:0/832121282 learned_addr learned my addr 172.21.15.122:0/832121282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:04.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f751aa00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75240d02e0 0x7f75240c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:04.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523e00640 1 -- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 msgr2=0x7f75240cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:04.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523e00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 0x7f75240cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523e00640 1 -- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75240d02e0 msgr2=0x7f75240c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:04.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523e00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75240d02e0 0x7f75240c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523e00640 1 -- 172.21.15.122:0/832121282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75240cea40 con 0x7f752415eff0 2024-09-17T05:30:04.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523400640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 0x7f75240cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:04.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7523e00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f752415eff0 0x7f75240cdf80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c3f0 tx=0x7f751c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:04.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f751be00640 1 -- 172.21.15.122:0/832121282 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c017070 con 0x7f752415eff0 2024-09-17T05:30:04.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f751be00640 1 -- 172.21.15.122:0/832121282 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f751c0053c0 con 0x7f752415eff0 2024-09-17T05:30:04.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f751be00640 1 -- 172.21.15.122:0/832121282 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c003860 con 0x7f752415eff0 2024-09-17T05:30:04.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75240ced20 con 0x7f752415eff0 2024-09-17T05:30:04.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75240ccdb0 con 0x7f752415eff0 2024-09-17T05:30:04.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.302+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75240d28c0 con 0x7f752415eff0 2024-09-17T05:30:04.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.306+0000 7f751be00640 1 -- 172.21.15.122:0/832121282 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f751c01a070 con 0x7f752415eff0 2024-09-17T05:30:04.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.306+0000 7f751be00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7500073950 0x7f7500075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:04.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.306+0000 7f751be00640 1 -- 172.21.15.122:0/832121282 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f751c094710 con 0x7f752415eff0 2024-09-17T05:30:04.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.306+0000 7f7523400640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7500073950 0x7f7500075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:04.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.306+0000 7f7523400640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7500073950 0x7f7500075e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f751401b420 tx=0x7f751401b5b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:04.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.310+0000 7f751be00640 1 -- 172.21.15.122:0/832121282 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f751c05e190 con 0x7f752415eff0 2024-09-17T05:30:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.506+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone95203", "target": ["mon-mgr", ""]}) -- 0x7f7524069e60 con 0x7f7500073950 2024-09-17T05:30:04.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f751be00640 1 -- 172.21.15.122:0/832121282 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f75140a5c40 con 0x7f7500073950 2024-09-17T05:30:04.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7500073950 msgr2=0x7f7500075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:04.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7500073950 0x7f7500075e10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f751401b420 tx=0x7f751401b5b0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f752415eff0 msgr2=0x7f75240cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:04.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f752415eff0 0x7f75240cdf80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c3f0 tx=0x7f751c00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 shutdown_connections 2024-09-17T05:30:04.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7500073950 0x7f7500075e10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f752415eff0 0x7f75240cdf80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75240d02e0 0x7f75240c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 --2- 172.21.15.122:0/832121282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75240cff00 0x7f75240cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:04.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 >> 172.21.15.122:0/832121282 conn(0x7f75240d1da0 msgr2=0x7f75240b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:04.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.918+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 shutdown_connections 2024-09-17T05:30:04.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:04.922+0000 7f7528a00640 1 -- 172.21.15.122:0/832121282 wait complete. 2024-09-17T05:30:04.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.926+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.926+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.926+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.926+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.926+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.943 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T05:30:04.943 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.946+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.944 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.946+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.944 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.946+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.944 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.946+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:04.944 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:04.946+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:05.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.090+0000 7f40ec800640 1 -- 172.21.15.122:0/3822120806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415c1f0 msgr2=0x7f40e40cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.090+0000 7f40ec800640 1 --2- 172.21.15.122:0/3822120806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415c1f0 0x7f40e40cee10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009900 tx=0x7f40d401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.090+0000 7f40ec800640 1 -- 172.21.15.122:0/3822120806 shutdown_connections 2024-09-17T05:30:05.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.090+0000 7f40ec800640 1 --2- 172.21.15.122:0/3822120806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40e40b0b60 0x7f40e40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.090+0000 7f40ec800640 1 --2- 172.21.15.122:0/3822120806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415c1f0 0x7f40e40cee10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.090+0000 7f40ec800640 1 -- 172.21.15.122:0/3822120806 >> 172.21.15.122:0/3822120806 conn(0x7f40e40d1da0 msgr2=0x7f40e40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.090+0000 7f40ec800640 1 -- 172.21.15.122:0/3822120806 shutdown_connections 2024-09-17T05:30:05.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.090+0000 7f40ec800640 1 -- 172.21.15.122:0/3822120806 wait complete. 2024-09-17T05:30:05.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 Processor -- start 2024-09-17T05:30:05.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 -- start start 2024-09-17T05:30:05.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40e40b0b60 0x7f40e40c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415be10 0x7f40e40c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40e415c1f0 0x7f40e41f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4169040 con 0x7f40e415be10 2024-09-17T05:30:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4168ec0 con 0x7f40e40b0b60 2024-09-17T05:30:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f40e41691c0 con 0x7f40e415c1f0 2024-09-17T05:30:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415be10 0x7f40e40c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415be10 0x7f40e40c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41438/0 (socket says 172.21.15.122:41438) 2024-09-17T05:30:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 -- 172.21.15.122:0/2235804305 learned_addr learned my addr 172.21.15.122:0/2235804305 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ea800640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40e415c1f0 0x7f40e41f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9e00640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40e40b0b60 0x7f40e40c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 -- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40e415c1f0 msgr2=0x7f40e41f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40e415c1f0 0x7f40e41f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 -- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40e40b0b60 msgr2=0x7f40e40c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40e40b0b60 0x7f40e40c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 -- 172.21.15.122:0/2235804305 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e406e4e0 con 0x7f40e415be10 2024-09-17T05:30:05.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9e00640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40e40b0b60 0x7f40e40c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:05.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40e9400640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415be10 0x7f40e40c8180 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009850 tx=0x7f40d4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40de000640 1 -- 172.21.15.122:0/2235804305 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d40b2040 con 0x7f40e415be10 2024-09-17T05:30:05.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40de000640 1 -- 172.21.15.122:0/2235804305 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40d4004130 con 0x7f40e415be10 2024-09-17T05:30:05.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40de000640 1 -- 172.21.15.122:0/2235804305 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4003600 con 0x7f40e415be10 2024-09-17T05:30:05.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e406e7c0 con 0x7f40e415be10 2024-09-17T05:30:05.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.094+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e406ed80 con 0x7f40e415be10 2024-09-17T05:30:05.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.098+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e40ccbd0 con 0x7f40e415be10 2024-09-17T05:30:05.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.098+0000 7f40de000640 1 -- 172.21.15.122:0/2235804305 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f40d40bf020 con 0x7f40e415be10 2024-09-17T05:30:05.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.098+0000 7f40de000640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f40c8073950 0x7f40c8075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.098+0000 7f40de000640 1 -- 172.21.15.122:0/2235804305 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f40d412f1e0 con 0x7f40e415be10 2024-09-17T05:30:05.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.098+0000 7f40e9e00640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f40c8073950 0x7f40c8075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.102+0000 7f40e9e00640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f40c8073950 0x7f40c8075e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000f70 tx=0x7f40d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.102+0000 7f40de000640 1 -- 172.21.15.122:0/2235804305 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f40d40c6020 con 0x7f40e415be10 2024-09-17T05:30:05.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.298+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40e4070350 con 0x7f40c8073950 2024-09-17T05:30:05.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.310+0000 7f40de000640 1 -- 172.21.15.122:0/2235804305 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f40e4070350 con 0x7f40c8073950 2024-09-17T05:30:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:05.313 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:30:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1" 2024-09-17T05:30:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:05.313 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:30:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f40c8073950 msgr2=0x7f40c8075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f40c8073950 0x7f40c8075e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f40d8000f70 tx=0x7f40d8009040 comp rx=0 tx=0).stop 2024-09-17T05:30:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415be10 msgr2=0x7f40e40c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415be10 0x7f40e40c8180 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009850 tx=0x7f40d4005e70 comp rx=0 tx=0).stop 2024-09-17T05:30:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 shutdown_connections 2024-09-17T05:30:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f40c8073950 0x7f40c8075e10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40e415c1f0 0x7f40e41f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40e415be10 0x7f40e40c8180 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 --2- 172.21.15.122:0/2235804305 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40e40b0b60 0x7f40e40c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 >> 172.21.15.122:0/2235804305 conn(0x7f40e40d1da0 msgr2=0x7f40e40aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 shutdown_connections 2024-09-17T05:30:05.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.314+0000 7f40ec800640 1 -- 172.21.15.122:0/2235804305 wait complete. 2024-09-17T05:30:05.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T05:30:05.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81e0a00640 1 -- 172.21.15.122:0/1014400659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 msgr2=0x7f81c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81e0a00640 1 --2- 172.21.15.122:0/1014400659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 0x7f81c0004810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f81c400ac40 tx=0x7f81c401e070 comp rx=0 tx=0).stop 2024-09-17T05:30:05.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81da800640 1 -- 172.21.15.122:0/1014400659 reap_dead start 2024-09-17T05:30:05.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81e0a00640 1 -- 172.21.15.122:0/1014400659 shutdown_connections 2024-09-17T05:30:05.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81e0a00640 1 -- 172.21.15.122:0/1014400659 >> 172.21.15.122:0/1014400659 conn(0x7f81dc0b18f0 msgr2=0x7f81dc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81e0a00640 1 -- 172.21.15.122:0/1014400659 shutdown_connections 2024-09-17T05:30:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81e0a00640 1 -- 172.21.15.122:0/1014400659 wait complete. 2024-09-17T05:30:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81e0a00640 1 Processor -- start 2024-09-17T05:30:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.482+0000 7f81e0a00640 1 -- start start 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 0x7f81dc156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc155fb0 0x7f81dc159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81dc1564d0 0x7f81dc09c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc0acd40 con 0x7f81c0003f80 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc06fb30 con 0x7f81dc155fb0 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc0acec0 con 0x7f81dc1564d0 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 0x7f81dc156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 0x7f81dc156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41462/0 (socket says 172.21.15.122:41462) 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 -- 172.21.15.122:0/1002639702 learned_addr learned my addr 172.21.15.122:0/1002639702 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d1400640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc155fb0 0x7f81dc159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81da800640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81dc1564d0 0x7f81dc09c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 -- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81dc1564d0 msgr2=0x7f81dc09c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81dc1564d0 0x7f81dc09c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 -- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc155fb0 msgr2=0x7f81dc159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc155fb0 0x7f81dc159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 -- 172.21.15.122:0/1002639702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc158f40 con 0x7f81c0003f80 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d9e00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 0x7f81dc156a20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0060d0 tx=0x7f81cc006590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d2a00640 1 -- 172.21.15.122:0/1002639702 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc015070 con 0x7f81c0003f80 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d2a00640 1 -- 172.21.15.122:0/1002639702 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81cc00ea40 con 0x7f81c0003f80 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc159220 con 0x7f81c0003f80 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81d2a00640 1 -- 172.21.15.122:0/1002639702 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc011460 con 0x7f81c0003f80 2024-09-17T05:30:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc1597e0 con 0x7f81c0003f80 2024-09-17T05:30:05.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.486+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc069410 con 0x7f81c0003f80 2024-09-17T05:30:05.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.490+0000 7f81d2a00640 1 -- 172.21.15.122:0/1002639702 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f81cc018070 con 0x7f81c0003f80 2024-09-17T05:30:05.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.490+0000 7f81d2a00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81b8073950 0x7f81b8075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.490+0000 7f81d2a00640 1 -- 172.21.15.122:0/1002639702 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f81cc096170 con 0x7f81c0003f80 2024-09-17T05:30:05.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.490+0000 7f81d1400640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81b8073950 0x7f81b8075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.494+0000 7f81d1400640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81b8073950 0x7f81b8075e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f81c40af000 tx=0x7f81c401ae10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.494+0000 7f81d2a00640 1 -- 172.21.15.122:0/1002639702 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f81cc05fc00 con 0x7f81c0003f80 2024-09-17T05:30:05.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.694+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81dc0b1f50 con 0x7f81b8073950 2024-09-17T05:30:05.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.722+0000 7f81d2a00640 1 -- 172.21.15.122:0/1002639702 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f81dc0b1f50 con 0x7f81b8073950 2024-09-17T05:30:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:05.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81b8073950 msgr2=0x7f81b8075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81b8073950 0x7f81b8075e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f81c40af000 tx=0x7f81c401ae10 comp rx=0 tx=0).stop 2024-09-17T05:30:05.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 msgr2=0x7f81dc156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 0x7f81dc156a20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0060d0 tx=0x7f81cc006590 comp rx=0 tx=0).stop 2024-09-17T05:30:05.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 shutdown_connections 2024-09-17T05:30:05.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81b8073950 0x7f81b8075e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81dc1564d0 0x7f81dc09c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81dc155fb0 0x7f81dc159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 --2- 172.21.15.122:0/1002639702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c0003f80 0x7f81dc156a20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 >> 172.21.15.122:0/1002639702 conn(0x7f81dc0b18f0 msgr2=0x7f81dc0aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 shutdown_connections 2024-09-17T05:30:05.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.726+0000 7f81e0a00640 1 -- 172.21.15.122:0/1002639702 wait complete. 2024-09-17T05:30:05.746 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.890+0000 7f796b200640 1 -- 172.21.15.122:0/3112455176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950003ab0 msgr2=0x7f7950003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.890+0000 7f796b200640 1 --2- 172.21.15.122:0/3112455176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950003ab0 0x7f7950003ed0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f795c009ee0 tx=0x7f795c01d250 comp rx=0 tx=0).stop 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.890+0000 7f796a800640 1 -- 172.21.15.122:0/3112455176 reap_dead start 2024-09-17T05:30:05.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.890+0000 7f796b200640 1 -- 172.21.15.122:0/3112455176 shutdown_connections 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.890+0000 7f796b200640 1 -- 172.21.15.122:0/3112455176 >> 172.21.15.122:0/3112455176 conn(0x7f796c09aa10 msgr2=0x7f796c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.890+0000 7f796b200640 1 -- 172.21.15.122:0/3112455176 shutdown_connections 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.890+0000 7f796b200640 1 -- 172.21.15.122:0/3112455176 wait complete. 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.890+0000 7f796b200640 1 Processor -- start 2024-09-17T05:30:05.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 -- start start 2024-09-17T05:30:05.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7950003ab0 0x7f796c1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f796c0b6190 0x7f796c1f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f796c161b90 0x7f796c162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f796c166b90 con 0x7f796c161b90 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f796c166a10 con 0x7f796c0b6190 2024-09-17T05:30:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f796c166d10 con 0x7f7950003ab0 2024-09-17T05:30:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f796c161b90 0x7f796c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f796c161b90 0x7f796c162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41476/0 (socket says 172.21.15.122:41476) 2024-09-17T05:30:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f7969e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7950003ab0 0x7f796c1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 -- 172.21.15.122:0/3795506206 learned_addr learned my addr 172.21.15.122:0/3795506206 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f7961400640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f796c0b6190 0x7f796c1f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 -- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7950003ab0 msgr2=0x7f796c1570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7950003ab0 0x7f796c1570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 -- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f796c0b6190 msgr2=0x7f796c1f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f796c0b6190 0x7f796c1f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 -- 172.21.15.122:0/3795506206 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c1f97d0 con 0x7f796c161b90 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796a800640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f796c161b90 0x7f796c162000 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7964001870 tx=0x7f79640039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f7962a00640 1 -- 172.21.15.122:0/3795506206 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964011840 con 0x7f796c161b90 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f7962a00640 1 -- 172.21.15.122:0/3795506206 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7964018070 con 0x7f796c161b90 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c1f9ab0 con 0x7f796c161b90 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f7962a00640 1 -- 172.21.15.122:0/3795506206 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964011c80 con 0x7f796c161b90 2024-09-17T05:30:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c1f93a0 con 0x7f796c161b90 2024-09-17T05:30:05.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.894+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c1f9cf0 con 0x7f796c161b90 2024-09-17T05:30:05.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.898+0000 7f7962a00640 1 -- 172.21.15.122:0/3795506206 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f796401d460 con 0x7f796c161b90 2024-09-17T05:30:05.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.898+0000 7f7962a00640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7948073980 0x7f7948075e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:05.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.898+0000 7f7962a00640 1 -- 172.21.15.122:0/3795506206 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f7964095b10 con 0x7f796c161b90 2024-09-17T05:30:05.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.898+0000 7f7969e00640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7948073980 0x7f7948075e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:05.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.898+0000 7f7969e00640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7948073980 0x7f7948075e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f795c0ae000 tx=0x7f795c019ff0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:05.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:05.902+0000 7f7962a00640 1 -- 172.21.15.122:0/3795506206 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f796c1f9cf0 con 0x7f796c161b90 2024-09-17T05:30:06.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.094+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f796c0c99f0 con 0x7f7948073980 2024-09-17T05:30:06.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f7962a00640 1 -- 172.21.15.122:0/3795506206 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f795c004ad0 con 0x7f7948073980 2024-09-17T05:30:06.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7948073980 msgr2=0x7f7948075e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7948073980 0x7f7948075e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f795c0ae000 tx=0x7f795c019ff0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f796c161b90 msgr2=0x7f796c162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f796c161b90 0x7f796c162000 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7964001870 tx=0x7f79640039d0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 shutdown_connections 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7948073980 0x7f7948075e40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f796c161b90 0x7f796c162000 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f796c0b6190 0x7f796c1f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 --2- 172.21.15.122:0/3795506206 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7950003ab0 0x7f796c1570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 >> 172.21.15.122:0/3795506206 conn(0x7f796c09aa10 msgr2=0x7f796c15a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 shutdown_connections 2024-09-17T05:30:06.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.114+0000 7f796b200640 1 -- 172.21.15.122:0/3795506206 wait complete. 2024-09-17T05:30:06.136 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone23140 2024-09-17T05:30:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.282+0000 7f2e41000640 1 -- 172.21.15.122:0/2477494161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0b0b60 msgr2=0x7f2e3c0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.282+0000 7f2e41000640 1 --2- 172.21.15.122:0/2477494161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0b0b60 0x7f2e3c0ccbd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009bc0 tx=0x7f2e3401cba0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.282+0000 7f2e41000640 1 -- 172.21.15.122:0/2477494161 shutdown_connections 2024-09-17T05:30:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.282+0000 7f2e41000640 1 --2- 172.21.15.122:0/2477494161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0b0b60 0x7f2e3c0ccbd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.282+0000 7f2e41000640 1 --2- 172.21.15.122:0/2477494161 >> v2:172.21.15.173:3300/0 conn(0x7f2e3c15c1f0 0x7f2e3c0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.282+0000 7f2e41000640 1 -- 172.21.15.122:0/2477494161 >> 172.21.15.122:0/2477494161 conn(0x7f2e3c0d1da0 msgr2=0x7f2e3c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:06.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.282+0000 7f2e41000640 1 -- 172.21.15.122:0/2477494161 shutdown_connections 2024-09-17T05:30:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.282+0000 7f2e41000640 1 -- 172.21.15.122:0/2477494161 wait complete. 2024-09-17T05:30:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 Processor -- start 2024-09-17T05:30:06.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 -- start start 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0b0b60 0x7f2e3c1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e3c15be10 0x7f2e3c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c15c1f0 0x7f2e3c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c169040 con 0x7f2e3c15be10 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c168ec0 con 0x7f2e3c15c1f0 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c1691c0 con 0x7f2e3c0b0b60 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e3c15be10 0x7f2e3c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e3c15be10 0x7f2e3c06e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41488/0 (socket says 172.21.15.122:41488) 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 -- 172.21.15.122:0/1426243912 learned_addr learned my addr 172.21.15.122:0/1426243912 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e3a800640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c15c1f0 0x7f2e3c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e39e00640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0b0b60 0x7f2e3c1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 -- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0b0b60 msgr2=0x7f2e3c1483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0b0b60 0x7f2e3c1483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 -- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c15c1f0 msgr2=0x7f2e3c06e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c15c1f0 0x7f2e3c06e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 -- 172.21.15.122:0/1426243912 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c165a90 con 0x7f2e3c15be10 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e33e00640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e3c15be10 0x7f2e3c06e410 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400e970 tx=0x7f2e2400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e32000640 1 -- 172.21.15.122:0/1426243912 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2400cd80 con 0x7f2e3c15be10 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e32000640 1 -- 172.21.15.122:0/1426243912 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e24004020 con 0x7f2e3c15be10 2024-09-17T05:30:06.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.286+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c165d70 con 0x7f2e3c15be10 2024-09-17T05:30:06.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.290+0000 7f2e32000640 1 -- 172.21.15.122:0/1426243912 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24004470 con 0x7f2e3c15be10 2024-09-17T05:30:06.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.290+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c166330 con 0x7f2e3c15be10 2024-09-17T05:30:06.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.290+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3c0ccbd0 con 0x7f2e3c15be10 2024-09-17T05:30:06.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.290+0000 7f2e32000640 1 -- 172.21.15.122:0/1426243912 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7f2e24003600 con 0x7f2e3c15be10 2024-09-17T05:30:06.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.290+0000 7f2e32000640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e18073950 0x7f2e18075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.290+0000 7f2e32000640 1 -- 172.21.15.122:0/1426243912 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f2e24095de0 con 0x7f2e3c15be10 2024-09-17T05:30:06.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.290+0000 7f2e39e00640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e18073950 0x7f2e18075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.294+0000 7f2e39e00640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e18073950 0x7f2e18075e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0b0660 tx=0x7f2e2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:06.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.298+0000 7f2e32000640 1 -- 172.21.15.122:0/1426243912 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2e2409c080 con 0x7f2e3c15be10 2024-09-17T05:30:06.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.494+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone23140", "target": ["mon-mgr", ""]}) -- 0x7f2e3c070350 con 0x7f2e18073950 2024-09-17T05:30:06.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.510+0000 7f2e32000640 1 -- 172.21.15.122:0/1426243912 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e3c070350 con 0x7f2e18073950 2024-09-17T05:30:06.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e18073950 msgr2=0x7f2e18075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e18073950 0x7f2e18075e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0b0660 tx=0x7f2e2c009040 comp rx=0 tx=0).stop 2024-09-17T05:30:06.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e3c15be10 msgr2=0x7f2e3c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e3c15be10 0x7f2e3c06e410 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400e970 tx=0x7f2e2400ee30 comp rx=0 tx=0).stop 2024-09-17T05:30:06.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 shutdown_connections 2024-09-17T05:30:06.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e18073950 0x7f2e18075e10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c15c1f0 0x7f2e3c06e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e3c15be10 0x7f2e3c06e410 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 --2- 172.21.15.122:0/1426243912 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0b0b60 0x7f2e3c1483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 >> 172.21.15.122:0/1426243912 conn(0x7f2e3c0d1da0 msgr2=0x7f2e3c157580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:06.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 shutdown_connections 2024-09-17T05:30:06.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.514+0000 7f2e41000640 1 -- 172.21.15.122:0/1426243912 wait complete. 2024-09-17T05:30:06.526 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:06.526+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:06.526 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:06.526+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:06.526 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:06.526+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:06.526 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:06.526+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:06.526 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:06.526+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:06.536 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone64901 2024-09-17T05:30:06.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- 172.21.15.122:0/2266300654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15c1f0 msgr2=0x7fb61c0cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 --2- 172.21.15.122:0/2266300654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15c1f0 0x7fb61c0cee10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb608009870 tx=0x7fb60801f050 comp rx=0 tx=0).stop 2024-09-17T05:30:06.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- 172.21.15.122:0/2266300654 shutdown_connections 2024-09-17T05:30:06.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 --2- 172.21.15.122:0/2266300654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15c1f0 0x7fb61c0cee10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- 172.21.15.122:0/2266300654 >> 172.21.15.122:0/2266300654 conn(0x7fb61c0d1da0 msgr2=0x7fb61c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:06.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- 172.21.15.122:0/2266300654 shutdown_connections 2024-09-17T05:30:06.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- 172.21.15.122:0/2266300654 wait complete. 2024-09-17T05:30:06.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 Processor -- start 2024-09-17T05:30:06.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- start start 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb61c0b0b60 0x7fb61c1f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15be10 0x7fb61c1488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c15c1f0 0x7fb61c148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c0cd670 con 0x7fb61c15c1f0 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c0cd4f0 con 0x7fb61c0b0b60 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb622000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb61c0cd7f0 con 0x7fb61c15be10 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb61be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c15c1f0 0x7fb61c148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb61b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb61c0b0b60 0x7fb61c1f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb61be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c15c1f0 0x7fb61c148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41494/0 (socket says 172.21.15.122:41494) 2024-09-17T05:30:06.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb61be00640 1 -- 172.21.15.122:0/693320649 learned_addr learned my addr 172.21.15.122:0/693320649 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.686+0000 7fb61aa00640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15be10 0x7fb61c1488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb61be00640 1 -- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15be10 msgr2=0x7fb61c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb61be00640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15be10 0x7fb61c1488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb61be00640 1 -- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb61c0b0b60 msgr2=0x7fb61c1f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb61be00640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb61c0b0b60 0x7fb61c1f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb61be00640 1 -- 172.21.15.122:0/693320649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c1f8070 con 0x7fb61c15c1f0 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb61b400640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb61c0b0b60 0x7fb61c1f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb61aa00640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15be10 0x7fb61c1488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb61be00640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c15c1f0 0x7fb61c148e00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb61400cbc0 tx=0x7fb61400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb613400640 1 -- 172.21.15.122:0/693320649 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb614016020 con 0x7fb61c15c1f0 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb613400640 1 -- 172.21.15.122:0/693320649 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb61400ab60 con 0x7fb61c15c1f0 2024-09-17T05:30:06.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61c14c140 con 0x7fb61c15c1f0 2024-09-17T05:30:06.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb613400640 1 -- 172.21.15.122:0/693320649 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb614012420 con 0x7fb61c15c1f0 2024-09-17T05:30:06.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61c1f4860 con 0x7fb61c15c1f0 2024-09-17T05:30:06.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb61c0ce4a0 con 0x7fb61c15c1f0 2024-09-17T05:30:06.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.690+0000 7fb613400640 1 -- 172.21.15.122:0/693320649 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 97897+0+0 (secure 0 0 0) 0x7fb614019020 con 0x7fb61c15c1f0 2024-09-17T05:30:06.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.694+0000 7fb613400640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5fc073950 0x7fb5fc075e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:06.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.694+0000 7fb613400640 1 -- 172.21.15.122:0/693320649 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fb6140956e0 con 0x7fb61c15c1f0 2024-09-17T05:30:06.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.694+0000 7fb61b400640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5fc073950 0x7fb5fc075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:06.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.694+0000 7fb61b400640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5fc073950 0x7fb5fc075e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb60c001740 tx=0x7fb60c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:06.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.694+0000 7fb613400640 1 -- 172.21.15.122:0/693320649 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb614020020 con 0x7fb61c15c1f0 2024-09-17T05:30:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:06.890+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone64901", "target": ["mon-mgr", ""]}) -- 0x7fb61c071630 con 0x7fb5fc073950 2024-09-17T05:30:07.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.506+0000 7fb613400640 1 -- 172.21.15.122:0/693320649 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fb61405b2f0 con 0x7fb61c15c1f0 2024-09-17T05:30:07.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.862+0000 7fb613400640 1 -- 172.21.15.122:0/693320649 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb61c071630 con 0x7fb5fc073950 2024-09-17T05:30:07.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5fc073950 msgr2=0x7fb5fc075e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5fc073950 0x7fb5fc075e10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb60c001740 tx=0x7fb60c009040 comp rx=0 tx=0).stop 2024-09-17T05:30:07.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c15c1f0 msgr2=0x7fb61c148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:07.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c15c1f0 0x7fb61c148e00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb61400cbc0 tx=0x7fb61400ccb0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 shutdown_connections 2024-09-17T05:30:07.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5fc073950 0x7fb5fc075e10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c15c1f0 0x7fb61c148e00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb61c15be10 0x7fb61c1488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 --2- 172.21.15.122:0/693320649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb61c0b0b60 0x7fb61c1f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:07.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 >> 172.21.15.122:0/693320649 conn(0x7fb61c0d1da0 msgr2=0x7fb61c071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:07.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 shutdown_connections 2024-09-17T05:30:07.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:07.866+0000 7fb622000640 1 -- 172.21.15.122:0/693320649 wait complete. 2024-09-17T05:30:07.886 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone10202 2024-09-17T05:30:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 -- 172.21.15.122:0/3586394855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 msgr2=0x7f9a580d0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/3586394855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 0x7f9a580d0080 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9a580d2c80 tx=0x7f9a4801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 -- 172.21.15.122:0/3586394855 shutdown_connections 2024-09-17T05:30:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/3586394855 >> v2:172.21.15.173:3300/0 conn(0x7f9a5815ff80 0x7f9a58160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/3586394855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a580b2a20 0x7f9a5815fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/3586394855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 0x7f9a580d0080 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 -- 172.21.15.122:0/3586394855 >> 172.21.15.122:0/3586394855 conn(0x7f9a580d1c60 msgr2=0x7f9a580d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 -- 172.21.15.122:0/3586394855 shutdown_connections 2024-09-17T05:30:08.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 -- 172.21.15.122:0/3586394855 wait complete. 2024-09-17T05:30:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.034+0000 7f9a5dc00640 1 Processor -- start 2024-09-17T05:30:08.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 -- start start 2024-09-17T05:30:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a580b2a20 0x7f9a5814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 0x7f9a580b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a5815ff80 0x7f9a580b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a580b1890 con 0x7f9a580cfca0 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9a580b1710 con 0x7f9a5815ff80 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9a580b1a10 con 0x7f9a580b2a20 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 0x7f9a580b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a580b2a20 0x7f9a5814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 0x7f9a580b67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41520/0 (socket says 172.21.15.122:41520) 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 -- 172.21.15.122:0/1426076272 learned_addr learned my addr 172.21.15.122:0/1426076272 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5d200640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a5815ff80 0x7f9a580b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 -- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a580b2a20 msgr2=0x7f9a5814a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a580b2a20 0x7f9a5814a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 -- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a5815ff80 msgr2=0x7f9a580b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a5815ff80 0x7f9a580b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 -- 172.21.15.122:0/1426076272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5814acc0 con 0x7f9a580cfca0 2024-09-17T05:30:08.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5d200640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a5815ff80 0x7f9a580b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:08.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a57400640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 0x7f9a580b67b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e9c0 tx=0x7f9a4400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:08.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a4fe00640 1 -- 172.21.15.122:0/1426076272 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400ccc0 con 0x7f9a580cfca0 2024-09-17T05:30:08.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a4fe00640 1 -- 172.21.15.122:0/1426076272 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a44004030 con 0x7f9a580cfca0 2024-09-17T05:30:08.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a581658f0 con 0x7f9a580cfca0 2024-09-17T05:30:08.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a4fe00640 1 -- 172.21.15.122:0/1426076272 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44004480 con 0x7f9a580cfca0 2024-09-17T05:30:08.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a58165eb0 con 0x7f9a580cfca0 2024-09-17T05:30:08.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.038+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a580d0080 con 0x7f9a580cfca0 2024-09-17T05:30:08.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.042+0000 7f9a4fe00640 1 -- 172.21.15.122:0/1426076272 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f9a44003600 con 0x7f9a580cfca0 2024-09-17T05:30:08.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.042+0000 7f9a4fe00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a38073b30 0x7f9a38075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.042+0000 7f9a4fe00640 1 -- 172.21.15.122:0/1426076272 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f9a440934c0 con 0x7f9a580cfca0 2024-09-17T05:30:08.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.042+0000 7f9a57e00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a38073b30 0x7f9a38075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.042+0000 7f9a57e00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a38073b30 0x7f9a38075ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a480af000 tx=0x7f9a4801b100 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:08.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.046+0000 7f9a4fe00640 1 -- 172.21.15.122:0/1426076272 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9a5814acc0 con 0x7f9a580cfca0 2024-09-17T05:30:08.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.238+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone10202", "target": ["mon-mgr", ""]}) -- 0x7f9a580b2550 con 0x7f9a38073b30 2024-09-17T05:30:08.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a4fe00640 1 -- 172.21.15.122:0/1426076272 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9a580b2550 con 0x7f9a38073b30 2024-09-17T05:30:08.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a38073b30 msgr2=0x7f9a38075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a38073b30 0x7f9a38075ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a480af000 tx=0x7f9a4801b100 comp rx=0 tx=0).stop 2024-09-17T05:30:08.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 msgr2=0x7f9a580b67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 0x7f9a580b67b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e9c0 tx=0x7f9a4400ee80 comp rx=0 tx=0).stop 2024-09-17T05:30:08.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 shutdown_connections 2024-09-17T05:30:08.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a38073b30 0x7f9a38075ff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a5815ff80 0x7f9a580b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a580cfca0 0x7f9a580b67b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 --2- 172.21.15.122:0/1426076272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a580b2a20 0x7f9a5814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 >> 172.21.15.122:0/1426076272 conn(0x7f9a580d1c60 msgr2=0x7f9a5815f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:08.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 shutdown_connections 2024-09-17T05:30:08.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.258+0000 7f9a5dc00640 1 -- 172.21.15.122:0/1426076272 wait complete. 2024-09-17T05:30:08.280 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:30:08.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:30:08.334 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:30:08.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:08.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:30:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 -- 172.21.15.122:0/2276483640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ed4073e60 msgr2=0x7f0ed40742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 --2- 172.21.15.122:0/2276483640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ed4073e60 0x7f0ed40742d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009c80 tx=0x7f0ecc01cd10 comp rx=0 tx=0).stop 2024-09-17T05:30:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 -- 172.21.15.122:0/2276483640 shutdown_connections 2024-09-17T05:30:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 --2- 172.21.15.122:0/2276483640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ed4073e60 0x7f0ed40742d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 --2- 172.21.15.122:0/2276483640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ed415c160 0x7f0ed40b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 -- 172.21.15.122:0/2276483640 >> 172.21.15.122:0/2276483640 conn(0x7f0ed406cc20 msgr2=0x7f0ed406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 -- 172.21.15.122:0/2276483640 shutdown_connections 2024-09-17T05:30:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 -- 172.21.15.122:0/2276483640 wait complete. 2024-09-17T05:30:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.546+0000 7f0edb000640 1 Processor -- start 2024-09-17T05:30:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0edb000640 1 -- start start 2024-09-17T05:30:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0edb000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed4073e60 0x7f0ed4156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0edb000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ed4155130 0x7f0ed413bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0edb000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ed415c160 0x7f0ed413c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0edb000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed406fc80 con 0x7f0ed4073e60 2024-09-17T05:30:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0edb000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed406fb00 con 0x7f0ed4155130 2024-09-17T05:30:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0edb000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed406fe00 con 0x7f0ed415c160 2024-09-17T05:30:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0ed9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ed4155130 0x7f0ed413bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0eda600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ed415c160 0x7f0ed413c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0eda600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ed415c160 0x7f0ed413c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:38130/0 (socket says 172.21.15.122:38130) 2024-09-17T05:30:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0eda600640 1 -- 172.21.15.122:0/3369674139 learned_addr learned my addr 172.21.15.122:0/3369674139 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0eda600640 1 -- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ed4155130 msgr2=0x7f0ed413bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0eda600640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ed4155130 0x7f0ed413bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0eda600640 1 -- 172.21.15.122:0/3369674139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed4073e60 msgr2=0x7f0ed4156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0eda600640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed4073e60 0x7f0ed4156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.550+0000 7f0eda600640 1 -- 172.21.15.122:0/3369674139 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed4157330 con 0x7f0ed415c160 2024-09-17T05:30:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.554+0000 7f0eda600640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ed415c160 0x7f0ed413c400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009bd0 tx=0x7f0ecc0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.554+0000 7f0ed1600640 1 -- 172.21.15.122:0/3369674139 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc0bb070 con 0x7f0ed415c160 2024-09-17T05:30:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.554+0000 7f0ed1600640 1 -- 172.21.15.122:0/3369674139 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ecc004d40 con 0x7f0ed415c160 2024-09-17T05:30:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.554+0000 7f0ed1600640 1 -- 172.21.15.122:0/3369674139 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc0b2040 con 0x7f0ed415c160 2024-09-17T05:30:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.554+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed40cfe90 con 0x7f0ed415c160 2024-09-17T05:30:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.554+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed40b20e0 con 0x7f0ed415c160 2024-09-17T05:30:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.554+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed4070320 con 0x7f0ed415c160 2024-09-17T05:30:08.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.558+0000 7f0ed1600640 1 -- 172.21.15.122:0/3369674139 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f0ecc0a5b30 con 0x7f0ed415c160 2024-09-17T05:30:08.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.562+0000 7f0ed1600640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eb8073b00 0x7f0eb8075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:08.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.562+0000 7f0ed1600640 1 -- 172.21.15.122:0/3369674139 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f0ecc12cc20 con 0x7f0ed415c160 2024-09-17T05:30:08.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.562+0000 7f0ed9c00640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eb8073b00 0x7f0eb8075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:08.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.562+0000 7f0ed9c00640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eb8073b00 0x7f0eb8075fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80058d0 tx=0x7f0ec8005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:08.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.566+0000 7f0ed1600640 1 -- 172.21.15.122:0/3369674139 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0ecc0169b0 con 0x7f0ed415c160 2024-09-17T05:30:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f0ed4153e30 con 0x7f0ed415c160 2024-09-17T05:30:08.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0ed1600640 1 -- 172.21.15.122:0/3369674139 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7f0ecc0f90d0 con 0x7f0ed415c160 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eb8073b00 msgr2=0x7f0eb8075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eb8073b00 0x7f0eb8075fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80058d0 tx=0x7f0ec8005860 comp rx=0 tx=0).stop 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ed415c160 msgr2=0x7f0ed413c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ed415c160 0x7f0ed413c400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009bd0 tx=0x7f0ecc0092e0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 shutdown_connections 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eb8073b00 0x7f0eb8075fc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ed415c160 0x7f0ed413c400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ed4155130 0x7f0ed413bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 --2- 172.21.15.122:0/3369674139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ed4073e60 0x7f0ed4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.782+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 >> 172.21.15.122:0/3369674139 conn(0x7f0ed406cc20 msgr2=0x7f0ed415c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.786+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 shutdown_connections 2024-09-17T05:30:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:08.786+0000 7f0edb000640 1 -- 172.21.15.122:0/3369674139 wait complete. 2024-09-17T05:30:08.804 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:30:08.804 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:30:08.903 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:30:09.017 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:30:09.017 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:30:09.017 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:09.017 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:09.026 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:09.026 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:09.076 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:30:09.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.266+0000 7ff6b0e00640 1 -- 172.21.15.122:0/51990207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac0ce800 msgr2=0x7ff6ac06cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.266+0000 7ff6b0e00640 1 --2- 172.21.15.122:0/51990207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac0ce800 0x7ff6ac06cc20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009c80 tx=0x7ff69c01cda0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.266+0000 7ff6b0e00640 1 -- 172.21.15.122:0/51990207 shutdown_connections 2024-09-17T05:30:09.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.266+0000 7ff6b0e00640 1 --2- 172.21.15.122:0/51990207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac0ce800 0x7ff6ac06cc20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.266+0000 7ff6b0e00640 1 -- 172.21.15.122:0/51990207 >> 172.21.15.122:0/51990207 conn(0x7ff6ac0b2240 msgr2=0x7ff6ac0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.266+0000 7ff6b0e00640 1 -- 172.21.15.122:0/51990207 shutdown_connections 2024-09-17T05:30:09.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.266+0000 7ff6b0e00640 1 -- 172.21.15.122:0/51990207 wait complete. 2024-09-17T05:30:09.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.266+0000 7ff6b0e00640 1 Processor -- start 2024-09-17T05:30:09.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 -- start start 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6ac070530 0x7ff6ac158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac073170 0x7ff6ac1fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6ac0ce800 0x7ff6ac0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac1696a0 con 0x7ff6ac073170 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac169520 con 0x7ff6ac0ce800 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac169820 con 0x7ff6ac070530 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac073170 0x7ff6ac1fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac073170 0x7ff6ac1fc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41548/0 (socket says 172.21.15.122:41548) 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 -- 172.21.15.122:0/667381174 learned_addr learned my addr 172.21.15.122:0/667381174 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a9e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6ac070530 0x7ff6ac158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6aa800640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6ac0ce800 0x7ff6ac0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 -- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6ac070530 msgr2=0x7ff6ac158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6ac070530 0x7ff6ac158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 -- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6ac0ce800 msgr2=0x7ff6ac0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6ac0ce800 0x7ff6ac0afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 -- 172.21.15.122:0/667381174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac15eed0 con 0x7ff6ac073170 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a1400640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac073170 0x7ff6ac1fc080 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff69400c480 tx=0x7ff69400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a2a00640 1 -- 172.21.15.122:0/667381174 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694017070 con 0x7ff6ac073170 2024-09-17T05:30:09.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a2a00640 1 -- 172.21.15.122:0/667381174 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff694003600 con 0x7ff6ac073170 2024-09-17T05:30:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6a2a00640 1 -- 172.21.15.122:0/667381174 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694003a50 con 0x7ff6ac073170 2024-09-17T05:30:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac0b4e00 con 0x7ff6ac073170 2024-09-17T05:30:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac15f140 con 0x7ff6ac073170 2024-09-17T05:30:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.270+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ac06cc20 con 0x7ff6ac073170 2024-09-17T05:30:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.274+0000 7ff6a2a00640 1 -- 172.21.15.122:0/667381174 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7ff69401a070 con 0x7ff6ac073170 2024-09-17T05:30:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.274+0000 7ff6a2a00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff688073b00 0x7ff688075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.274+0000 7ff6a2a00640 1 -- 172.21.15.122:0/667381174 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7ff694091e10 con 0x7ff6ac073170 2024-09-17T05:30:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.274+0000 7ff6a9e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff688073b00 0x7ff688075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.274+0000 7ff6a9e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff688073b00 0x7ff688075fc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0af000 tx=0x7ff69c01b5e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.278+0000 7ff6a2a00640 1 -- 172.21.15.122:0/667381174 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff694007e20 con 0x7ff6ac073170 2024-09-17T05:30:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.454+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6ac13e790 con 0x7ff6ac073170 2024-09-17T05:30:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.454+0000 7ff6a2a00640 1 -- 172.21.15.122:0/667381174 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7ff69405e2c0 con 0x7ff6ac073170 2024-09-17T05:30:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:30:09.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff688073b00 msgr2=0x7ff688075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff688073b00 0x7ff688075fc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0af000 tx=0x7ff69c01b5e0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac073170 msgr2=0x7ff6ac1fc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac073170 0x7ff6ac1fc080 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff69400c480 tx=0x7ff69400c940 comp rx=0 tx=0).stop 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 shutdown_connections 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff688073b00 0x7ff688075fc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6ac0ce800 0x7ff6ac0afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6ac073170 0x7ff6ac1fc080 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 --2- 172.21.15.122:0/667381174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6ac070530 0x7ff6ac158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 >> 172.21.15.122:0/667381174 conn(0x7ff6ac0b2240 msgr2=0x7ff6ac06f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 shutdown_connections 2024-09-17T05:30:09.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.458+0000 7ff6b0e00640 1 -- 172.21.15.122:0/667381174 wait complete. 2024-09-17T05:30:09.478 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:30:09.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:09.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:09.525 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:30:09.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:09.526 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:30:09.543 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:30:09.543 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:30:09.543 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:09.559 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:30:09.559 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:30:09.559 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:09.559 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:09.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:09.597 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:30:09.597 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:30:09.597 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:09.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:30:09.620 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- 172.21.15.122:0/1816992537 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f247c072a50 msgr2=0x7f247c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 --2- 172.21.15.122:0/1816992537 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f247c072a50 0x7f247c15df30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2464009870 tx=0x7f246401c880 comp rx=0 tx=0).stop 2024-09-17T05:30:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- 172.21.15.122:0/1816992537 shutdown_connections 2024-09-17T05:30:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 --2- 172.21.15.122:0/1816992537 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f247c072a50 0x7f247c15df30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- 172.21.15.122:0/1816992537 >> 172.21.15.122:0/1816992537 conn(0x7f247c06cc20 msgr2=0x7f247c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- 172.21.15.122:0/1816992537 shutdown_connections 2024-09-17T05:30:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- 172.21.15.122:0/1816992537 wait complete. 2024-09-17T05:30:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 Processor -- start 2024-09-17T05:30:09.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- start start 2024-09-17T05:30:09.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f247c06aba0 0x7f247c166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f247c072a50 0x7f247c153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f247c15e470 0x7f247c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f247c163f20 con 0x7f247c072a50 2024-09-17T05:30:09.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f247c163da0 con 0x7f247c15e470 2024-09-17T05:30:09.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.874+0000 7f2480a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f247c1640a0 con 0x7f247c06aba0 2024-09-17T05:30:09.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f247c15e470 0x7f247c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f247c15e470 0x7f247c0b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:40932/0 (socket says 172.21.15.122:40932) 2024-09-17T05:30:09.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 -- 172.21.15.122:0/1584235106 learned_addr learned my addr 172.21.15.122:0/1584235106 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247aa00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f247c072a50 0x7f247c153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247b400640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f247c06aba0 0x7f247c166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 -- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f247c06aba0 msgr2=0x7f247c166dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f247c06aba0 0x7f247c166dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 -- 172.21.15.122:0/1584235106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f247c072a50 msgr2=0x7f247c153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f247c072a50 0x7f247c153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 -- 172.21.15.122:0/1584235106 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c1f8440 con 0x7f247c15e470 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f247be00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f247c15e470 0x7f247c0b02b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f247400c700 tx=0x7f247400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f2473400640 1 -- 172.21.15.122:0/1584235106 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474003b80 con 0x7f247c15e470 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f2473400640 1 -- 172.21.15.122:0/1584235106 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2474003d20 con 0x7f247c15e470 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f2473400640 1 -- 172.21.15.122:0/1584235106 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247400aab0 con 0x7f247c15e470 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c1f7920 con 0x7f247c15e470 2024-09-17T05:30:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c1f8160 con 0x7f247c15e470 2024-09-17T05:30:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.878+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247c15df30 con 0x7f247c15e470 2024-09-17T05:30:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.882+0000 7f2473400640 1 -- 172.21.15.122:0/1584235106 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f247400ac50 con 0x7f247c15e470 2024-09-17T05:30:09.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.882+0000 7f2473400640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2458073b00 0x7f2458075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:09.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.882+0000 7f2473400640 1 -- 172.21.15.122:0/1584235106 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f2474093310 con 0x7f247c15e470 2024-09-17T05:30:09.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.886+0000 7f247b400640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2458073b00 0x7f2458075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:09.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.886+0000 7f247b400640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2458073b00 0x7f2458075fc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f246c0046c0 tx=0x7f246c007d70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:09.890+0000 7f2473400640 1 -- 172.21.15.122:0/1584235106 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f247c1f7920 con 0x7f247c15e470 2024-09-17T05:30:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.258+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f247c0cc5c0 con 0x7f247c15e470 2024-09-17T05:30:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.258+0000 7f2473400640 1 -- 172.21.15.122:0/1584235106 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f247405f7c0 con 0x7f247c15e470 2024-09-17T05:30:10.260 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:10.260 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"btime":"2024-09-17T05:28:16:887773+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3520614210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3520614210},{"type":"v1","addr":"172.21.15.122:6834","nonce":3520614210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5308,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2442281734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2442281734},{"type":"v1","addr":"172.21.15.173:6836","nonce":2442281734}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/638961094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":638961094},{"type":"v1","addr":"172.21.15.122:6835","nonce":638961094}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:03.784287+0000","modified":"2024-09-17T05:28:06.804180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/4117982032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4117982032},{"type":"v1","addr":"172.21.15.173:6837","nonce":4117982032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5388,"qdb_cluster":[5388]},"id":3}]} 2024-09-17T05:30:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-09-17T05:30:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2458073b00 msgr2=0x7f2458075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2458073b00 0x7f2458075fc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f246c0046c0 tx=0x7f246c007d70 comp rx=0 tx=0).stop 2024-09-17T05:30:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f247c15e470 msgr2=0x7f247c0b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f247c15e470 0x7f247c0b02b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f247400c700 tx=0x7f247400cbc0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 shutdown_connections 2024-09-17T05:30:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2458073b00 0x7f2458075fc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f247c15e470 0x7f247c0b02b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f247c072a50 0x7f247c153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 --2- 172.21.15.122:0/1584235106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f247c06aba0 0x7f247c166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 >> 172.21.15.122:0/1584235106 conn(0x7f247c06cc20 msgr2=0x7f247c0b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 shutdown_connections 2024-09-17T05:30:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.262+0000 7f2480a00640 1 -- 172.21.15.122:0/1584235106 wait complete. 2024-09-17T05:30:10.294 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:10.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 -- 172.21.15.122:0/2952950348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a8070410 msgr2=0x7f32a80b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 --2- 172.21.15.122:0/2952950348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a8070410 0x7f32a80b3cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009c80 tx=0x7f32a001cda0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 -- 172.21.15.122:0/2952950348 shutdown_connections 2024-09-17T05:30:10.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 --2- 172.21.15.122:0/2952950348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a8070410 0x7f32a80b3cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 --2- 172.21.15.122:0/2952950348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a815a8c0 0x7f32a815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 -- 172.21.15.122:0/2952950348 >> 172.21.15.122:0/2952950348 conn(0x7f32a806cc20 msgr2=0x7f32a806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:10.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 -- 172.21.15.122:0/2952950348 shutdown_connections 2024-09-17T05:30:10.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 -- 172.21.15.122:0/2952950348 wait complete. 2024-09-17T05:30:10.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 Processor -- start 2024-09-17T05:30:10.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.526+0000 7f32ae000640 1 -- start start 2024-09-17T05:30:10.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ae000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a8070410 0x7f32a8151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:10.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ae000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a8073570 0x7f32a8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:10.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ae000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a815a8c0 0x7f32a8158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:10.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ae000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32a815b930 con 0x7f32a8073570 2024-09-17T05:30:10.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ae000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f32a815b7b0 con 0x7f32a815a8c0 2024-09-17T05:30:10.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ae000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f32a815bab0 con 0x7f32a8070410 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a8073570 0x7f32a8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a815a8c0 0x7f32a8158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a815a8c0 0x7f32a8158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:40956/0 (socket says 172.21.15.122:40956) 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 -- 172.21.15.122:0/3420605499 learned_addr learned my addr 172.21.15.122:0/3420605499 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32acc00640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a8070410 0x7f32a8151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 -- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a8070410 msgr2=0x7f32a8151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a8070410 0x7f32a8151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 -- 172.21.15.122:0/3420605499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a8073570 msgr2=0x7f32a8156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a8073570 0x7f32a8156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 -- 172.21.15.122:0/3420605499 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a815dbf0 con 0x7f32a815a8c0 2024-09-17T05:30:10.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.530+0000 7f32ad600640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a815a8c0 0x7f32a8158ae0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32a001cbd0 tx=0x7f32a0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:10.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.534+0000 7f32a4c00640 1 -- 172.21.15.122:0/3420605499 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00bb070 con 0x7f32a815a8c0 2024-09-17T05:30:10.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.534+0000 7f32a4c00640 1 -- 172.21.15.122:0/3420605499 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a0004b60 con 0x7f32a815a8c0 2024-09-17T05:30:10.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.534+0000 7f32a4c00640 1 -- 172.21.15.122:0/3420605499 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00b2040 con 0x7f32a815a8c0 2024-09-17T05:30:10.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.534+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a8155b90 con 0x7f32a815a8c0 2024-09-17T05:30:10.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.534+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a8155e30 con 0x7f32a815a8c0 2024-09-17T05:30:10.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.534+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a80b3cc0 con 0x7f32a815a8c0 2024-09-17T05:30:10.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.534+0000 7f32a4c00640 1 -- 172.21.15.122:0/3420605499 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f32a0004460 con 0x7f32a815a8c0 2024-09-17T05:30:10.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.538+0000 7f32a4c00640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3288073b30 0x7f3288075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:10.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.538+0000 7f32a4c00640 1 -- 172.21.15.122:0/3420605499 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f32a012cc20 con 0x7f32a815a8c0 2024-09-17T05:30:10.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.538+0000 7f32acc00640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3288073b30 0x7f3288075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:10.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.542+0000 7f32acc00640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3288073b30 0x7f3288075ff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:10.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.546+0000 7f32a4c00640 1 -- 172.21.15.122:0/3420605499 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f32a0016990 con 0x7f32a815a8c0 2024-09-17T05:30:10.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.914+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32a80b1d70 con 0x7f32a815a8c0 2024-09-17T05:30:10.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.914+0000 7f32a4c00640 1 -- 172.21.15.122:0/3420605499 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f32a00f90d0 con 0x7f32a815a8c0 2024-09-17T05:30:10.915 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:10.916 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"btime":"2024-09-17T05:28:16:887773+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3520614210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3520614210},{"type":"v1","addr":"172.21.15.122:6834","nonce":3520614210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5308,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2442281734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2442281734},{"type":"v1","addr":"172.21.15.173:6836","nonce":2442281734}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/638961094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":638961094},{"type":"v1","addr":"172.21.15.122:6835","nonce":638961094}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:03.784287+0000","modified":"2024-09-17T05:28:06.804180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/4117982032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4117982032},{"type":"v1","addr":"172.21.15.173:6837","nonce":4117982032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5388,"qdb_cluster":[5388]},"id":3}]} 2024-09-17T05:30:10.916 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-09-17T05:30:10.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3288073b30 msgr2=0x7f3288075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3288073b30 0x7f3288075ff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c008040 comp rx=0 tx=0).stop 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a815a8c0 msgr2=0x7f32a8158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a815a8c0 0x7f32a8158ae0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f32a001cbd0 tx=0x7f32a0004270 comp rx=0 tx=0).stop 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 shutdown_connections 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3288073b30 0x7f3288075ff0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32a815a8c0 0x7f32a8158ae0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a8073570 0x7f32a8156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 --2- 172.21.15.122:0/3420605499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32a8070410 0x7f32a8151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:10.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 >> 172.21.15.122:0/3420605499 conn(0x7f32a806cc20 msgr2=0x7f32a806fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:10.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 shutdown_connections 2024-09-17T05:30:10.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:10.918+0000 7f32ae000640 1 -- 172.21.15.122:0/3420605499 wait complete. 2024-09-17T05:30:10.953 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:30:11.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.174+0000 7f5e92400640 1 -- 172.21.15.122:0/2795511952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e8c0ced80 msgr2=0x7f5e8c0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.174+0000 7f5e92400640 1 --2- 172.21.15.122:0/2795511952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e8c0ced80 0x7f5e8c0ae9b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009840 tx=0x7f5e7801c820 comp rx=0 tx=0).stop 2024-09-17T05:30:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.174+0000 7f5e92400640 1 -- 172.21.15.122:0/2795511952 shutdown_connections 2024-09-17T05:30:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.174+0000 7f5e92400640 1 --2- 172.21.15.122:0/2795511952 >> v2:172.21.15.173:3300/0 conn(0x7f5e8c0cdb30 0x7f5e8c0aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.174+0000 7f5e92400640 1 --2- 172.21.15.122:0/2795511952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e8c0ced80 0x7f5e8c0ae9b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.174+0000 7f5e92400640 1 -- 172.21.15.122:0/2795511952 >> 172.21.15.122:0/2795511952 conn(0x7f5e8c0d1dc0 msgr2=0x7f5e8c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:11.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.174+0000 7f5e92400640 1 -- 172.21.15.122:0/2795511952 shutdown_connections 2024-09-17T05:30:11.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.174+0000 7f5e92400640 1 -- 172.21.15.122:0/2795511952 wait complete. 2024-09-17T05:30:11.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e92400640 1 Processor -- start 2024-09-17T05:30:11.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e92400640 1 -- start start 2024-09-17T05:30:11.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e92400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e8c0cdb30 0x7f5e8c1fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e92400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e8c0ced80 0x7f5e8c150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e92400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e8c15bea0 0x7f5e8c1fc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e92400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1692f0 con 0x7f5e8c15bea0 2024-09-17T05:30:11.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e92400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c169170 con 0x7f5e8c0ced80 2024-09-17T05:30:11.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e92400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8c169470 con 0x7f5e8c0cdb30 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e8c15bea0 0x7f5e8c1fc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e8b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e8c0cdb30 0x7f5e8c1fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e8c15bea0 0x7f5e8c1fc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41578/0 (socket says 172.21.15.122:41578) 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e8be00640 1 -- 172.21.15.122:0/2341551633 learned_addr learned my addr 172.21.15.122:0/2341551633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.178+0000 7f5e8aa00640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e8c0ced80 0x7f5e8c150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e8be00640 1 -- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e8c0cdb30 msgr2=0x7f5e8c1fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e8be00640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e8c0cdb30 0x7f5e8c1fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e8be00640 1 -- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e8c0ced80 msgr2=0x7f5e8c150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e8be00640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e8c0ced80 0x7f5e8c150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e8be00640 1 -- 172.21.15.122:0/2341551633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c1fcf30 con 0x7f5e8c15bea0 2024-09-17T05:30:11.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e8be00640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e8c15bea0 0x7f5e8c1fc9f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400ef00 tx=0x7f5e8400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e83400640 1 -- 172.21.15.122:0/2341551633 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84004b30 con 0x7f5e8c15bea0 2024-09-17T05:30:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e83400640 1 -- 172.21.15.122:0/2341551633 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e84004de0 con 0x7f5e8c15bea0 2024-09-17T05:30:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c0c5910 con 0x7f5e8c15bea0 2024-09-17T05:30:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e83400640 1 -- 172.21.15.122:0/2341551633 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400b7d0 con 0x7f5e8c15bea0 2024-09-17T05:30:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.182+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c0c7300 con 0x7f5e8c15bea0 2024-09-17T05:30:11.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.186+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c1599f0 con 0x7f5e8c15bea0 2024-09-17T05:30:11.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.186+0000 7f5e83400640 1 -- 172.21.15.122:0/2341551633 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f5e8400b970 con 0x7f5e8c15bea0 2024-09-17T05:30:11.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.186+0000 7f5e83400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5e6c073bd0 0x7f5e6c076090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.186+0000 7f5e83400640 1 -- 172.21.15.122:0/2341551633 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5910+0+0 (secure 0 0 0) 0x7f5e840932f0 con 0x7f5e8c15bea0 2024-09-17T05:30:11.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.190+0000 7f5e8b400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5e6c073bd0 0x7f5e6c076090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.190+0000 7f5e8b400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5e6c073bd0 0x7f5e6c076090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c007fd0 tx=0x7f5e7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:11.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.194+0000 7f5e83400640 1 -- 172.21.15.122:0/2341551633 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5e8405f7a0 con 0x7f5e8c15bea0 2024-09-17T05:30:11.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.382+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e8c0ce620 con 0x7f5e8c15bea0 2024-09-17T05:30:11.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e83400640 1 -- 172.21.15.122:0/2341551633 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+17072 (secure 0 0 0) 0x7f5e840641d0 con 0x7f5e8c15bea0 2024-09-17T05:30:11.386 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:11.386 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":49,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:30:10.539003+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:01.479885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:02.745102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":0}]},{"pgid":"7.28","mappings":[{"from":5,"to":7}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:30:11.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5e6c073bd0 msgr2=0x7f5e6c076090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5e6c073bd0 0x7f5e6c076090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c007fd0 tx=0x7f5e7c009040 comp rx=0 tx=0).stop 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e8c15bea0 msgr2=0x7f5e8c1fc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e8c15bea0 0x7f5e8c1fc9f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400ef00 tx=0x7f5e8400c740 comp rx=0 tx=0).stop 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 shutdown_connections 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5e6c073bd0 0x7f5e6c076090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e8c15bea0 0x7f5e8c1fc9f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e8c0ced80 0x7f5e8c150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 --2- 172.21.15.122:0/2341551633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e8c0cdb30 0x7f5e8c1fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 >> 172.21.15.122:0/2341551633 conn(0x7f5e8c0d1dc0 msgr2=0x7f5e8c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 shutdown_connections 2024-09-17T05:30:11.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.386+0000 7f5e92400640 1 -- 172.21.15.122:0/2341551633 wait complete. 2024-09-17T05:30:11.409 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:30:11.409 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:11.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 -- 172.21.15.122:0/1010393938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 msgr2=0x7fed6006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 --2- 172.21.15.122:0/1010393938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 0x7fed6006d160 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fed48005fd0 tx=0x7fed4801c820 comp rx=0 tx=0).stop 2024-09-17T05:30:11.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 -- 172.21.15.122:0/1010393938 shutdown_connections 2024-09-17T05:30:11.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 --2- 172.21.15.122:0/1010393938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 0x7fed6006d160 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 --2- 172.21.15.122:0/1010393938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed600ce800 0x7fed6006cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 -- 172.21.15.122:0/1010393938 >> 172.21.15.122:0/1010393938 conn(0x7fed600b2240 msgr2=0x7fed600b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:11.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 -- 172.21.15.122:0/1010393938 shutdown_connections 2024-09-17T05:30:11.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 -- 172.21.15.122:0/1010393938 wait complete. 2024-09-17T05:30:11.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 Processor -- start 2024-09-17T05:30:11.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.550+0000 7fed66c00640 1 -- start start 2024-09-17T05:30:11.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 0x7fed6013b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed600ce800 0x7fed6013a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed60143fe0 0x7fed601426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed601682b0 con 0x7fed60073170 2024-09-17T05:30:11.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fed60168130 con 0x7fed60143fe0 2024-09-17T05:30:11.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fed60168430 con 0x7fed600ce800 2024-09-17T05:30:11.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 0x7fed6013b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 0x7fed6013b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34732/0 (socket says 172.21.15.122:34732) 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 -- 172.21.15.122:0/3291590989 learned_addr learned my addr 172.21.15.122:0/3291590989 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed57400640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed600ce800 0x7fed6013a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed64c00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed60143fe0 0x7fed601426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 -- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed600ce800 msgr2=0x7fed6013a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed600ce800 0x7fed6013a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 -- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed60143fe0 msgr2=0x7fed601426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed60143fe0 0x7fed601426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 -- 172.21.15.122:0/3291590989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6013acd0 con 0x7fed60073170 2024-09-17T05:30:11.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5fe00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 0x7fed6013b7b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fed5000ec00 tx=0x7fed5000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:11.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5cc00640 1 -- 172.21.15.122:0/3291590989 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50010630 con 0x7fed60073170 2024-09-17T05:30:11.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5cc00640 1 -- 172.21.15.122:0/3291590989 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed500108c0 con 0x7fed60073170 2024-09-17T05:30:11.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed60143160 con 0x7fed60073170 2024-09-17T05:30:11.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5cc00640 1 -- 172.21.15.122:0/3291590989 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50010cf0 con 0x7fed60073170 2024-09-17T05:30:11.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed60143720 con 0x7fed60073170 2024-09-17T05:30:11.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6015e8b0 con 0x7fed60073170 2024-09-17T05:30:11.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.554+0000 7fed5cc00640 1 -- 172.21.15.122:0/3291590989 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fed50018420 con 0x7fed60073170 2024-09-17T05:30:11.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.558+0000 7fed5cc00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed3c073b30 0x7fed3c075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.558+0000 7fed5cc00640 1 -- 172.21.15.122:0/3291590989 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5910+0+0 (secure 0 0 0) 0x7fed500937a0 con 0x7fed60073170 2024-09-17T05:30:11.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.558+0000 7fed57400640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed3c073b30 0x7fed3c075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.558+0000 7fed57400640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed3c073b30 0x7fed3c075ff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fed480ae000 tx=0x7fed4801b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:11.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.562+0000 7fed5cc00640 1 -- 172.21.15.122:0/3291590989 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fed5005fc50 con 0x7fed60073170 2024-09-17T05:30:11.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed6015ad00 con 0x7fed60073170 2024-09-17T05:30:11.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed5cc00640 1 -- 172.21.15.122:0/3291590989 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fed50064680 con 0x7fed60073170 2024-09-17T05:30:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:11.763 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"btime":"2024-09-17T05:28:16:887773+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3520614210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3520614210},{"type":"v1","addr":"172.21.15.122:6834","nonce":3520614210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5308,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2442281734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2442281734},{"type":"v1","addr":"172.21.15.173:6836","nonce":2442281734}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/638961094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":638961094},{"type":"v1","addr":"172.21.15.122:6835","nonce":638961094}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:03.784287+0000","modified":"2024-09-17T05:28:06.804180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/4117982032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4117982032},{"type":"v1","addr":"172.21.15.173:6837","nonce":4117982032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5388,"qdb_cluster":[5388]},"id":3}]} 2024-09-17T05:30:11.763 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-09-17T05:30:11.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed3c073b30 msgr2=0x7fed3c075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed3c073b30 0x7fed3c075ff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fed480ae000 tx=0x7fed4801b060 comp rx=0 tx=0).stop 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 msgr2=0x7fed6013b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 0x7fed6013b7b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fed5000ec00 tx=0x7fed5000c6a0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 shutdown_connections 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fed3c073b30 0x7fed3c075ff0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed60143fe0 0x7fed601426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed600ce800 0x7fed6013a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 --2- 172.21.15.122:0/3291590989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60073170 0x7fed6013b7b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 >> 172.21.15.122:0/3291590989 conn(0x7fed600b2240 msgr2=0x7fed600b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 shutdown_connections 2024-09-17T05:30:11.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.762+0000 7fed66c00640 1 -- 172.21.15.122:0/3291590989 wait complete. 2024-09-17T05:30:11.786 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:11.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.926+0000 7f3a98400640 1 -- 172.21.15.122:0/421382425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a900b6400 msgr2=0x7f3a900ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.926+0000 7f3a98400640 1 --2- 172.21.15.122:0/421382425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a900b6400 0x7f3a900ccdf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009bc0 tx=0x7f3a8801cba0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.926+0000 7f3a98400640 1 -- 172.21.15.122:0/421382425 shutdown_connections 2024-09-17T05:30:11.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.926+0000 7f3a98400640 1 --2- 172.21.15.122:0/421382425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a900b6400 0x7f3a900ccdf0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.926+0000 7f3a98400640 1 --2- 172.21.15.122:0/421382425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a900cebf0 0x7f3a900cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.926+0000 7f3a98400640 1 -- 172.21.15.122:0/421382425 >> 172.21.15.122:0/421382425 conn(0x7f3a900d1da0 msgr2=0x7f3a900d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:11.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.926+0000 7f3a98400640 1 -- 172.21.15.122:0/421382425 shutdown_connections 2024-09-17T05:30:11.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.926+0000 7f3a98400640 1 -- 172.21.15.122:0/421382425 wait complete. 2024-09-17T05:30:11.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 Processor -- start 2024-09-17T05:30:11.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 -- start start 2024-09-17T05:30:11.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a900b6400 0x7f3a9014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a900cebf0 0x7f3a900c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9015be80 0x7f3a90152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a901692b0 con 0x7f3a9015be80 2024-09-17T05:30:11.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a90169130 con 0x7f3a900cebf0 2024-09-17T05:30:11.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3a90169430 con 0x7f3a900b6400 2024-09-17T05:30:11.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a95a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a900b6400 0x7f3a9014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a96400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9015be80 0x7f3a90152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a95000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a900cebf0 0x7f3a900c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a95a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a900b6400 0x7f3a9014de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33348/0 (socket says 172.21.15.122:33348) 2024-09-17T05:30:11.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a95000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a900cebf0 0x7f3a900c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:55358/0 (socket says 172.21.15.122:55358) 2024-09-17T05:30:11.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a96400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9015be80 0x7f3a90152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34758/0 (socket says 172.21.15.122:34758) 2024-09-17T05:30:11.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a95a00640 1 -- 172.21.15.122:0/496976721 learned_addr learned my addr 172.21.15.122:0/496976721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:11.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a96400640 1 -- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a900b6400 msgr2=0x7f3a9014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a96400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a900b6400 0x7f3a9014de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a96400640 1 -- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a900cebf0 msgr2=0x7f3a900c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a96400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a900cebf0 0x7f3a900c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a96400640 1 -- 172.21.15.122:0/496976721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a900c7fa0 con 0x7f3a9015be80 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a95000640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a900cebf0 0x7f3a900c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a96400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9015be80 0x7f3a90152060 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3a880ac000 tx=0x7f3a880a5980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a8d600640 1 -- 172.21.15.122:0/496976721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a880bb070 con 0x7f3a9015be80 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a8d600640 1 -- 172.21.15.122:0/496976721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a88004e00 con 0x7f3a9015be80 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a8d600640 1 -- 172.21.15.122:0/496976721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a880b2040 con 0x7f3a9015be80 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a900c8280 con 0x7f3a9015be80 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.930+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a90153650 con 0x7f3a9015be80 2024-09-17T05:30:11.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.934+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a900c84c0 con 0x7f3a9015be80 2024-09-17T05:30:11.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.934+0000 7f3a8d600640 1 -- 172.21.15.122:0/496976721 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f3a880be070 con 0x7f3a9015be80 2024-09-17T05:30:11.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.934+0000 7f3a8d600640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a70073b00 0x7f3a70075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:11.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.934+0000 7f3a8d600640 1 -- 172.21.15.122:0/496976721 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5910+0+0 (secure 0 0 0) 0x7f3a8812c7d0 con 0x7f3a9015be80 2024-09-17T05:30:11.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.934+0000 7f3a95a00640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a70073b00 0x7f3a70075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:11.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.934+0000 7f3a95a00640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a70073b00 0x7f3a70075fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3a84007fd0 tx=0x7f3a84007f60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:11.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:11.938+0000 7f3a8d600640 1 -- 172.21.15.122:0/496976721 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3a880a5e20 con 0x7f3a9015be80 2024-09-17T05:30:12.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a900ce4e0 con 0x7f3a9015be80 2024-09-17T05:30:12.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a8d600640 1 -- 172.21.15.122:0/496976721 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f3a88016790 con 0x7f3a9015be80 2024-09-17T05:30:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:12.142 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"btime":"2024-09-17T05:28:16:887773+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3520614210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3520614210},{"type":"v1","addr":"172.21.15.122:6834","nonce":3520614210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5308,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2442281734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2442281734},{"type":"v1","addr":"172.21.15.173:6836","nonce":2442281734}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/638961094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":638961094},{"type":"v1","addr":"172.21.15.122:6835","nonce":638961094}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:28:03.784287+0000","modified":"2024-09-17T05:28:06.804180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5388":{"gid":5388,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6837/4117982032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4117982032},{"type":"v1","addr":"172.21.15.173:6837","nonce":4117982032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5388,"qdb_cluster":[5388]},"id":3}]} 2024-09-17T05:30:12.142 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a70073b00 msgr2=0x7f3a70075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a70073b00 0x7f3a70075fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3a84007fd0 tx=0x7f3a84007f60 comp rx=0 tx=0).stop 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9015be80 msgr2=0x7f3a90152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9015be80 0x7f3a90152060 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3a880ac000 tx=0x7f3a880a5980 comp rx=0 tx=0).stop 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 shutdown_connections 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a70073b00 0x7f3a70075fc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9015be80 0x7f3a90152060 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a900cebf0 0x7f3a900c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 --2- 172.21.15.122:0/496976721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a900b6400 0x7f3a9014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 >> 172.21.15.122:0/496976721 conn(0x7f3a900d1da0 msgr2=0x7f3a900b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 shutdown_connections 2024-09-17T05:30:12.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.142+0000 7f3a98400640 1 -- 172.21.15.122:0/496976721 wait complete. 2024-09-17T05:30:12.163 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:30:12.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 -- 172.21.15.122:0/3170927228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680d28e0 msgr2=0x7f52680cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 --2- 172.21.15.122:0/3170927228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680d28e0 0x7f52680cc9d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f52680745d0 tx=0x7f526001cd80 comp rx=0 tx=0).stop 2024-09-17T05:30:12.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 -- 172.21.15.122:0/3170927228 shutdown_connections 2024-09-17T05:30:12.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 --2- 172.21.15.122:0/3170927228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680d28e0 0x7f52680cc9d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 --2- 172.21.15.122:0/3170927228 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52680cdb30 0x7f52680cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 --2- 172.21.15.122:0/3170927228 >> v2:172.21.15.173:3300/0 conn(0x7f526815bea0 0x7f52680ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 -- 172.21.15.122:0/3170927228 >> 172.21.15.122:0/3170927228 conn(0x7f52680d1dc0 msgr2=0x7f52680d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:12.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 -- 172.21.15.122:0/3170927228 shutdown_connections 2024-09-17T05:30:12.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.302+0000 7f526cc00640 1 -- 172.21.15.122:0/3170927228 wait complete. 2024-09-17T05:30:12.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 Processor -- start 2024-09-17T05:30:12.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 -- start start 2024-09-17T05:30:12.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680cdb30 0x7f526806f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52680d28e0 0x7f5268153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f526815bea0 0x7f52680b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52680af120 con 0x7f52680cdb30 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52680aefa0 con 0x7f52680d28e0 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f52680af2a0 con 0x7f526815bea0 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680cdb30 0x7f526806f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680cdb30 0x7f526806f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34782/0 (socket says 172.21.15.122:34782) 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 -- 172.21.15.122:0/459326065 learned_addr learned my addr 172.21.15.122:0/459326065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f525d400640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52680d28e0 0x7f5268153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5266800640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f526815bea0 0x7f52680b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 -- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f526815bea0 msgr2=0x7f52680b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f526815bea0 0x7f52680b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 -- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52680d28e0 msgr2=0x7f5268153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52680d28e0 0x7f5268153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 -- 172.21.15.122:0/459326065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526806fd70 con 0x7f52680cdb30 2024-09-17T05:30:12.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f525d400640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52680d28e0 0x7f5268153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:12.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5266800640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f526815bea0 0x7f52680b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:12.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f5265e00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680cdb30 0x7f526806f560 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f525800e7d0 tx=0x7f525800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:12.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f525ea00640 1 -- 172.21.15.122:0/459326065 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525800ccc0 con 0x7f52680cdb30 2024-09-17T05:30:12.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f525ea00640 1 -- 172.21.15.122:0/459326065 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5258010430 con 0x7f52680cdb30 2024-09-17T05:30:12.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f525ea00640 1 -- 172.21.15.122:0/459326065 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258010860 con 0x7f52680cdb30 2024-09-17T05:30:12.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526806faa0 con 0x7f52680cdb30 2024-09-17T05:30:12.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52680b1960 con 0x7f52680cdb30 2024-09-17T05:30:12.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.306+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52680af830 con 0x7f52680cdb30 2024-09-17T05:30:12.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.310+0000 7f525ea00640 1 -- 172.21.15.122:0/459326065 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f5258010a00 con 0x7f52680cdb30 2024-09-17T05:30:12.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.310+0000 7f525ea00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5244073b30 0x7f5244075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:12.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.310+0000 7f525ea00640 1 -- 172.21.15.122:0/459326065 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5910+0+0 (secure 0 0 0) 0x7f5258093490 con 0x7f52680cdb30 2024-09-17T05:30:12.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.310+0000 7f525d400640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5244073b30 0x7f5244075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:12.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.310+0000 7f525d400640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5244073b30 0x7f5244075ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5268154c50 tx=0x7f52500091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:12.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.314+0000 7f525ea00640 1 -- 172.21.15.122:0/459326065 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f525805fff0 con 0x7f52680cdb30 2024-09-17T05:30:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.486+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5268070a20 con 0x7f52680cdb30 2024-09-17T05:30:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.486+0000 7f525ea00640 1 -- 172.21.15.122:0/459326065 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+17072 (secure 0 0 0) 0x7f525805f8f0 con 0x7f52680cdb30 2024-09-17T05:30:12.489 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:12.489 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":49,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:30:10.539003+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:28:01.479885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-17T05:28:02.745102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":0}]},{"pgid":"7.28","mappings":[{"from":5,"to":7}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:30:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5244073b30 msgr2=0x7f5244075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5244073b30 0x7f5244075ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5268154c50 tx=0x7f52500091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680cdb30 msgr2=0x7f526806f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680cdb30 0x7f526806f560 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f525800e7d0 tx=0x7f525800ec90 comp rx=0 tx=0).stop 2024-09-17T05:30:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 shutdown_connections 2024-09-17T05:30:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5244073b30 0x7f5244075ff0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f526815bea0 0x7f52680b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52680d28e0 0x7f5268153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 --2- 172.21.15.122:0/459326065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52680cdb30 0x7f526806f560 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 >> 172.21.15.122:0/459326065 conn(0x7f52680d1dc0 msgr2=0x7f5268159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 shutdown_connections 2024-09-17T05:30:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.490+0000 7f526cc00640 1 -- 172.21.15.122:0/459326065 wait complete. 2024-09-17T05:30:12.511 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:30:12.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.654+0000 7fe5e5c00640 1 -- 172.21.15.122:0/22955744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 msgr2=0x7fe5e00acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.654+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/22955744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 0x7fe5e00acd40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009bc0 tx=0x7fe5d801cb10 comp rx=0 tx=0).stop 2024-09-17T05:30:12.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.654+0000 7fe5e5c00640 1 -- 172.21.15.122:0/22955744 shutdown_connections 2024-09-17T05:30:12.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.654+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/22955744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 0x7fe5e00acd40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.654+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/22955744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5e00cd100 0x7fe5e006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.654+0000 7fe5e5c00640 1 -- 172.21.15.122:0/22955744 >> 172.21.15.122:0/22955744 conn(0x7fe5e00b18f0 msgr2=0x7fe5e00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.654+0000 7fe5e5c00640 1 -- 172.21.15.122:0/22955744 shutdown_connections 2024-09-17T05:30:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.654+0000 7fe5e5c00640 1 -- 172.21.15.122:0/22955744 wait complete. 2024-09-17T05:30:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 Processor -- start 2024-09-17T05:30:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 -- start start 2024-09-17T05:30:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 0x7fe5e0159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e00cd100 0x7fe5e0159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5e015dae0 0x7fe5e00b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e00cf1a0 con 0x7fe5e00b49e0 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e00cf020 con 0x7fe5e00cd100 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e00cf320 con 0x7fe5e015dae0 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 0x7fe5e0159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 0x7fe5e0159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34806/0 (socket says 172.21.15.122:34806) 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 -- 172.21.15.122:0/1257870037 learned_addr learned my addr 172.21.15.122:0/1257870037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5d6000640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e00cd100 0x7fe5e0159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5df400640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5e015dae0 0x7fe5e00b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 -- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5e015dae0 msgr2=0x7fe5e00b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5e015dae0 0x7fe5e00b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 -- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e00cd100 msgr2=0x7fe5e0159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e00cd100 0x7fe5e0159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 -- 172.21.15.122:0/1257870037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e0164d70 con 0x7fe5e00b49e0 2024-09-17T05:30:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5df400640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5e015dae0 0x7fe5e00b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5dea00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 0x7fe5e0159520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c430 tx=0x7fe5d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5d7400640 1 -- 172.21.15.122:0/1257870037 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0011790 con 0x7fe5e00b49e0 2024-09-17T05:30:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5d7400640 1 -- 172.21.15.122:0/1257870037 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5d0017070 con 0x7fe5e00b49e0 2024-09-17T05:30:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5d7400640 1 -- 172.21.15.122:0/1257870037 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0011ce0 con 0x7fe5e00b49e0 2024-09-17T05:30:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e0156f20 con 0x7fe5e00b49e0 2024-09-17T05:30:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e01574e0 con 0x7fe5e00b49e0 2024-09-17T05:30:12.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.658+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e00acd40 con 0x7fe5e00b49e0 2024-09-17T05:30:12.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.662+0000 7fe5d7400640 1 -- 172.21.15.122:0/1257870037 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fe5d001c460 con 0x7fe5e00b49e0 2024-09-17T05:30:12.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.662+0000 7fe5d7400640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe5bc073b00 0x7fe5bc075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:12.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.662+0000 7fe5d7400640 1 -- 172.21.15.122:0/1257870037 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5910+0+0 (secure 0 0 0) 0x7fe5d00927e0 con 0x7fe5e00b49e0 2024-09-17T05:30:12.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.662+0000 7fe5d6000640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe5bc073b00 0x7fe5bc075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:12.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.662+0000 7fe5d6000640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe5bc073b00 0x7fe5bc075fc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0158e30 tx=0x7fe5c80091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:12.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.666+0000 7fe5d7400640 1 -- 172.21.15.122:0/1257870037 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe5d005f3a0 con 0x7fe5e00b49e0 2024-09-17T05:30:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.866+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe5e00cfdb0 con 0x7fe5e00b49e0 2024-09-17T05:30:12.867 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:30:12.866+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:30:12.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.870+0000 7fe5d7400640 1 -- 172.21.15.122:0/1257870037 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fe5d005eca0 con 0x7fe5e00b49e0 2024-09-17T05:30:12.869 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:30:12.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe5bc073b00 msgr2=0x7fe5bc075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe5bc073b00 0x7fe5bc075fc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0158e30 tx=0x7fe5c80091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 msgr2=0x7fe5e0159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:12.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 0x7fe5e0159520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000c430 tx=0x7fe5d000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 shutdown_connections 2024-09-17T05:30:12.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe5bc073b00 0x7fe5bc075fc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5e015dae0 0x7fe5e00b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5e00cd100 0x7fe5e0159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 --2- 172.21.15.122:0/1257870037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5e00b49e0 0x7fe5e0159520 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:12.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 >> 172.21.15.122:0/1257870037 conn(0x7fe5e00b18f0 msgr2=0x7fe5e0158410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:12.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 shutdown_connections 2024-09-17T05:30:12.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:12.874+0000 7fe5e5c00640 1 -- 172.21.15.122:0/1257870037 wait complete. 2024-09-17T05:30:12.894 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:30:12.938 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:30:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 -- 172.21.15.122:0/1167561604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015bc80 msgr2=0x7f6d400cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 --2- 172.21.15.122:0/1167561604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015bc80 0x7f6d400cf9d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009900 tx=0x7f6d3001c820 comp rx=0 tx=0).stop 2024-09-17T05:30:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 -- 172.21.15.122:0/1167561604 shutdown_connections 2024-09-17T05:30:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 --2- 172.21.15.122:0/1167561604 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d400d2150 0x7f6d4015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 --2- 172.21.15.122:0/1167561604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015bc80 0x7f6d400cf9d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 -- 172.21.15.122:0/1167561604 >> 172.21.15.122:0/1167561604 conn(0x7f6d4009aa10 msgr2=0x7f6d4009adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 -- 172.21.15.122:0/1167561604 shutdown_connections 2024-09-17T05:30:13.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 -- 172.21.15.122:0/1167561604 wait complete. 2024-09-17T05:30:13.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 Processor -- start 2024-09-17T05:30:13.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.038+0000 7f6d47400640 1 -- start start 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d400d2150 0x7f6d400c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4015bc80 0x7f6d40152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015c060 0x7f6d40071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d400cd180 con 0x7f6d4015c060 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6d400cd000 con 0x7f6d4015bc80 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6d400cd300 con 0x7f6d400d2150 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015c060 0x7f6d40071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015c060 0x7f6d40071060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34834/0 (socket says 172.21.15.122:34834) 2024-09-17T05:30:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 -- 172.21.15.122:0/1372005749 learned_addr learned my addr 172.21.15.122:0/1372005749 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:13.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d44a00640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d400d2150 0x7f6d400c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:13.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d37e00640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4015bc80 0x7f6d40152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:13.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 -- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d400d2150 msgr2=0x7f6d400c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:13.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d400d2150 0x7f6d400c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:13.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 -- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4015bc80 msgr2=0x7f6d40152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:13.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4015bc80 0x7f6d40152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:13.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 -- 172.21.15.122:0/1372005749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d40152940 con 0x7f6d4015c060 2024-09-17T05:30:13.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d45400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015c060 0x7f6d40071060 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800ea80 tx=0x7f6d3800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:13.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d3d600640 1 -- 172.21.15.122:0/1372005749 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38004b30 con 0x7f6d4015c060 2024-09-17T05:30:13.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d3d600640 1 -- 172.21.15.122:0/1372005749 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d38004cf0 con 0x7f6d4015c060 2024-09-17T05:30:13.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d400c70b0 con 0x7f6d4015c060 2024-09-17T05:30:13.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d3d600640 1 -- 172.21.15.122:0/1372005749 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38005280 con 0x7f6d4015c060 2024-09-17T05:30:13.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d400c7670 con 0x7f6d4015c060 2024-09-17T05:30:13.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.042+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4015c800 con 0x7f6d4015c060 2024-09-17T05:30:13.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.046+0000 7f6d3d600640 1 -- 172.21.15.122:0/1372005749 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f6d38005420 con 0x7f6d4015c060 2024-09-17T05:30:13.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.046+0000 7f6d3d600640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d1c073b00 0x7f6d1c075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:13.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.046+0000 7f6d3d600640 1 -- 172.21.15.122:0/1372005749 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5968+0+0 (secure 0 0 0) 0x7f6d38093550 con 0x7f6d4015c060 2024-09-17T05:30:13.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.046+0000 7f6d44a00640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d1c073b00 0x7f6d1c075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:13.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.046+0000 7f6d44a00640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d1c073b00 0x7f6d1c075fc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6d300af000 tx=0x7f6d3001b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:13.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.050+0000 7f6d3d600640 1 -- 172.21.15.122:0/1372005749 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6d3805fa20 con 0x7f6d4015c060 2024-09-17T05:30:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.250+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6d40070cb0 con 0x7f6d4015c060 2024-09-17T05:30:13.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.870+0000 7f6d3d600640 1 -- 172.21.15.122:0/1372005749 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f6d38064450 con 0x7f6d4015c060 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d1c073b00 msgr2=0x7f6d1c075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d1c073b00 0x7f6d1c075fc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6d300af000 tx=0x7f6d3001b060 comp rx=0 tx=0).stop 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015c060 msgr2=0x7f6d40071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015c060 0x7f6d40071060 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800ea80 tx=0x7f6d3800ef40 comp rx=0 tx=0).stop 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 shutdown_connections 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d1c073b00 0x7f6d1c075fc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d4015c060 0x7f6d40071060 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4015bc80 0x7f6d40152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 --2- 172.21.15.122:0/1372005749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d400d2150 0x7f6d400c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:13.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 >> 172.21.15.122:0/1372005749 conn(0x7f6d4009aa10 msgr2=0x7f6d400d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:13.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 shutdown_connections 2024-09-17T05:30:13.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:13.874+0000 7f6d47400640 1 -- 172.21.15.122:0/1372005749 wait complete. 2024-09-17T05:30:13.895 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:30:14.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.042+0000 7fd3d4a00640 1 -- 172.21.15.122:0/1783547319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d00cff00 msgr2=0x7fd3d00d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.042+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/1783547319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d00cff00 0x7fd3d00d28c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009840 tx=0x7fd3c001c710 comp rx=0 tx=0).stop 2024-09-17T05:30:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.042+0000 7fd3d4a00640 1 -- 172.21.15.122:0/1783547319 shutdown_connections 2024-09-17T05:30:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.042+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/1783547319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d015f700 0x7fd3d00d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.042+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/1783547319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d00cff00 0x7fd3d00d28c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.042+0000 7fd3d4a00640 1 -- 172.21.15.122:0/1783547319 >> 172.21.15.122:0/1783547319 conn(0x7fd3d00d1da0 msgr2=0x7fd3d00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.042+0000 7fd3d4a00640 1 -- 172.21.15.122:0/1783547319 shutdown_connections 2024-09-17T05:30:14.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.042+0000 7fd3d4a00640 1 -- 172.21.15.122:0/1783547319 wait complete. 2024-09-17T05:30:14.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 Processor -- start 2024-09-17T05:30:14.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 -- start start 2024-09-17T05:30:14.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00cff00 0x7fd3d015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:14.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d00d02e0 0x7fd3d0150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:14.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d015f700 0x7fd3d00b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:14.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d00b0ca0 con 0x7fd3d00d02e0 2024-09-17T05:30:14.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d00b0b20 con 0x7fd3d015f700 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d0168bf0 con 0x7fd3d00cff00 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d00d02e0 0x7fd3d0150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d00d02e0 0x7fd3d0150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34848/0 (socket says 172.21.15.122:34848) 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 -- 172.21.15.122:0/620628954 learned_addr learned my addr 172.21.15.122:0/620628954 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3ce800640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00cff00 0x7fd3d015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3cf200640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d015f700 0x7fd3d00b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 -- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00cff00 msgr2=0x7fd3d015bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00cff00 0x7fd3d015bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 -- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d015f700 msgr2=0x7fd3d00b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d015f700 0x7fd3d00b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 -- 172.21.15.122:0/620628954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d00b7a20 con 0x7fd3d00d02e0 2024-09-17T05:30:14.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c5e00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d00d02e0 0x7fd3d0150ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800c700 tx=0x7fd3b800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c7400640 1 -- 172.21.15.122:0/620628954 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8017070 con 0x7fd3d00d02e0 2024-09-17T05:30:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d00cd980 con 0x7fd3d00d02e0 2024-09-17T05:30:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c7400640 1 -- 172.21.15.122:0/620628954 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3b8003600 con 0x7fd3d00d02e0 2024-09-17T05:30:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3c7400640 1 -- 172.21.15.122:0/620628954 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8003a50 con 0x7fd3d00d02e0 2024-09-17T05:30:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d0151430 con 0x7fd3d00d02e0 2024-09-17T05:30:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.046+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d00d28c0 con 0x7fd3d00d02e0 2024-09-17T05:30:14.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.050+0000 7fd3c7400640 1 -- 172.21.15.122:0/620628954 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fd3b801a070 con 0x7fd3d00d02e0 2024-09-17T05:30:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.050+0000 7fd3c7400640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3ac073b00 0x7fd3ac075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:14.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.050+0000 7fd3c7400640 1 -- 172.21.15.122:0/620628954 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5968+0+0 (secure 0 0 0) 0x7fd3b8091ed0 con 0x7fd3d00d02e0 2024-09-17T05:30:14.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.050+0000 7fd3ce800640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3ac073b00 0x7fd3ac075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:14.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.050+0000 7fd3ce800640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3ac073b00 0x7fd3ac075fc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00af000 tx=0x7fd3c001af50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:14.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.054+0000 7fd3c7400640 1 -- 172.21.15.122:0/620628954 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd3b805e320 con 0x7fd3d00d02e0 2024-09-17T05:30:14.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.226+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd3d0155270 con 0x7fd3d00d02e0 2024-09-17T05:30:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.878+0000 7fd3c7400640 1 -- 172.21.15.122:0/620628954 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) ==== 181+0+0 (secure 0 0 0) 0x7fd3b8062d50 con 0x7fd3d00d02e0 2024-09-17T05:30:14.878 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:30:14.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3ac073b00 msgr2=0x7fd3ac075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:14.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3ac073b00 0x7fd3ac075fc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00af000 tx=0x7fd3c001af50 comp rx=0 tx=0).stop 2024-09-17T05:30:14.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d00d02e0 msgr2=0x7fd3d0150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:14.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d00d02e0 0x7fd3d0150ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800c700 tx=0x7fd3b800cb00 comp rx=0 tx=0).stop 2024-09-17T05:30:14.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 shutdown_connections 2024-09-17T05:30:14.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3ac073b00 0x7fd3ac075fc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:14.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d015f700 0x7fd3d00b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:14.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3d00d02e0 0x7fd3d0150ef0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:14.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 --2- 172.21.15.122:0/620628954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00cff00 0x7fd3d015bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:14.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 >> 172.21.15.122:0/620628954 conn(0x7fd3d00d1da0 msgr2=0x7fd3d01538e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:14.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 shutdown_connections 2024-09-17T05:30:14.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:14.882+0000 7fd3d4a00640 1 -- 172.21.15.122:0/620628954 wait complete. 2024-09-17T05:30:14.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:30:15.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 -- 172.21.15.122:0/761645573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600ce890 msgr2=0x7fdc6015df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 --2- 172.21.15.122:0/761645573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600ce890 0x7fdc6015df70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc60069630 tx=0x7fdc5801dc70 comp rx=0 tx=0).stop 2024-09-17T05:30:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 -- 172.21.15.122:0/761645573 shutdown_connections 2024-09-17T05:30:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 --2- 172.21.15.122:0/761645573 >> v2:172.21.15.173:3300/0 conn(0x7fdc600cf440 0x7fdc600ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 --2- 172.21.15.122:0/761645573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600ce890 0x7fdc6015df70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 --2- 172.21.15.122:0/761645573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc600734f0 0x7fdc6015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 -- 172.21.15.122:0/761645573 >> 172.21.15.122:0/761645573 conn(0x7fdc600cfbc0 msgr2=0x7fdc600cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 -- 172.21.15.122:0/761645573 shutdown_connections 2024-09-17T05:30:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 -- 172.21.15.122:0/761645573 wait complete. 2024-09-17T05:30:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 Processor -- start 2024-09-17T05:30:15.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.058+0000 7fdc65a00640 1 -- start start 2024-09-17T05:30:15.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc600734f0 0x7fdc60155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:15.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc600ce890 0x7fdc601f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:15.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600cf440 0x7fdc601f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6006eb40 con 0x7fdc600cf440 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6015ed20 con 0x7fdc600ce890 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6006ecc0 con 0x7fdc600734f0 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600cf440 0x7fdc601f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600cf440 0x7fdc601f8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34874/0 (socket says 172.21.15.122:34874) 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 -- 172.21.15.122:0/539030997 learned_addr learned my addr 172.21.15.122:0/539030997 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc56000640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc600ce890 0x7fdc601f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5ea00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc600734f0 0x7fdc60155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 -- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc600734f0 msgr2=0x7fdc60155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc600734f0 0x7fdc60155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 -- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc600ce890 msgr2=0x7fdc601f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc600ce890 0x7fdc601f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 -- 172.21.15.122:0/539030997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc601f66f0 con 0x7fdc600cf440 2024-09-17T05:30:15.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5ea00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc600734f0 0x7fdc60155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:15.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc5f400640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600cf440 0x7fdc601f8d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800ace0 tx=0x7fdc580a6ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:15.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc57400640 1 -- 172.21.15.122:0/539030997 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580b3040 con 0x7fdc600cf440 2024-09-17T05:30:15.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc57400640 1 -- 172.21.15.122:0/539030997 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc58004d40 con 0x7fdc600cf440 2024-09-17T05:30:15.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc57400640 1 -- 172.21.15.122:0/539030997 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004320 con 0x7fdc600cf440 2024-09-17T05:30:15.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc601f96e0 con 0x7fdc600cf440 2024-09-17T05:30:15.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601f59b0 con 0x7fdc600cf440 2024-09-17T05:30:15.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc60069410 con 0x7fdc600cf440 2024-09-17T05:30:15.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.062+0000 7fdc57400640 1 -- 172.21.15.122:0/539030997 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fdc580bf060 con 0x7fdc600cf440 2024-09-17T05:30:15.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.066+0000 7fdc57400640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc3c073b00 0x7fdc3c075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:15.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.066+0000 7fdc57400640 1 -- 172.21.15.122:0/539030997 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5530+0+0 (secure 0 0 0) 0x7fdc5812c710 con 0x7fdc600cf440 2024-09-17T05:30:15.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.066+0000 7fdc5ea00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc3c073b00 0x7fdc3c075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:15.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.066+0000 7fdc5ea00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc3c073b00 0x7fdc3c075fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdc50006fd0 tx=0x7fdc50008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:15.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.070+0000 7fdc57400640 1 -- 172.21.15.122:0/539030997 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdc601f66f0 con 0x7fdc600cf440 2024-09-17T05:30:15.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.242+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdc601f9c00 con 0x7fdc600cf440 2024-09-17T05:30:15.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.878+0000 7fdc57400640 1 -- 172.21.15.122:0/539030997 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) ==== 169+0+0 (secure 0 0 0) 0x7fdc601f9c00 con 0x7fdc600cf440 2024-09-17T05:30:15.880 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.882+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc3c073b00 msgr2=0x7fdc3c075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.882+0000 7fdc65a00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc3c073b00 0x7fdc3c075fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdc50006fd0 tx=0x7fdc50008040 comp rx=0 tx=0).stop 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.882+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600cf440 msgr2=0x7fdc601f8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.882+0000 7fdc65a00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600cf440 0x7fdc601f8d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800ace0 tx=0x7fdc580a6ae0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.886+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 shutdown_connections 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.886+0000 7fdc65a00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc3c073b00 0x7fdc3c075fc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.886+0000 7fdc65a00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc600cf440 0x7fdc601f8d90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.886+0000 7fdc65a00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc600ce890 0x7fdc601f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.886+0000 7fdc65a00640 1 --2- 172.21.15.122:0/539030997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc600734f0 0x7fdc60155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.886+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 >> 172.21.15.122:0/539030997 conn(0x7fdc600cfbc0 msgr2=0x7fdc601537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.886+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 shutdown_connections 2024-09-17T05:30:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:15.886+0000 7fdc65a00640 1 -- 172.21.15.122:0/539030997 wait complete. 2024-09-17T05:30:15.918 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:30:16.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.058+0000 7f7059e00640 1 -- 172.21.15.122:0/2605959104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7054164c60 msgr2=0x7f7054167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.058+0000 7f7059e00640 1 --2- 172.21.15.122:0/2605959104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7054164c60 0x7f7054167050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f704c009bc0 tx=0x7f704c01cae0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.058+0000 7f7059e00640 1 -- 172.21.15.122:0/2605959104 shutdown_connections 2024-09-17T05:30:16.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.058+0000 7f7059e00640 1 --2- 172.21.15.122:0/2605959104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7054164c60 0x7f7054167050 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.058+0000 7f7059e00640 1 -- 172.21.15.122:0/2605959104 >> 172.21.15.122:0/2605959104 conn(0x7f70540d1e30 msgr2=0x7f70540d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:16.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.058+0000 7f7059e00640 1 -- 172.21.15.122:0/2605959104 shutdown_connections 2024-09-17T05:30:16.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.058+0000 7f7059e00640 1 -- 172.21.15.122:0/2605959104 wait complete. 2024-09-17T05:30:16.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 Processor -- start 2024-09-17T05:30:16.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 -- start start 2024-09-17T05:30:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70540b2e80 0x7f7054155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705415f490 0x7f70540ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7054164c60 0x7f70541f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7054169f30 con 0x7f705415f490 2024-09-17T05:30:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7054169db0 con 0x7f7054164c60 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f705416a0b0 con 0x7f70540b2e80 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705415f490 0x7f70540ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705415f490 0x7f70540ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34890/0 (socket says 172.21.15.122:34890) 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 -- 172.21.15.122:0/3836978127 learned_addr learned my addr 172.21.15.122:0/3836978127 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052a00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70540b2e80 0x7f7054155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7053400640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7054164c60 0x7f70541f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 -- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70540b2e80 msgr2=0x7f7054155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70540b2e80 0x7f7054155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 -- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7054164c60 msgr2=0x7f70541f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7054164c60 0x7f70541f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 -- 172.21.15.122:0/3836978127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70540cdb40 con 0x7f705415f490 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7052000640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705415f490 0x7f70540ccac0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f704000c450 tx=0x7f704000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f704b400640 1 -- 172.21.15.122:0/3836978127 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040012830 con 0x7f705415f490 2024-09-17T05:30:16.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f704b400640 1 -- 172.21.15.122:0/3836978127 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7040017070 con 0x7f705415f490 2024-09-17T05:30:16.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f704b400640 1 -- 172.21.15.122:0/3836978127 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040012d80 con 0x7f705415f490 2024-09-17T05:30:16.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70540cd580 con 0x7f705415f490 2024-09-17T05:30:16.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.062+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70541f7400 con 0x7f705415f490 2024-09-17T05:30:16.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.066+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70540b14c0 con 0x7f705415f490 2024-09-17T05:30:16.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.066+0000 7f704b400640 1 -- 172.21.15.122:0/3836978127 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f704001c460 con 0x7f705415f490 2024-09-17T05:30:16.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.066+0000 7f704b400640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7034073b00 0x7f7034075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.066+0000 7f704b400640 1 -- 172.21.15.122:0/3836978127 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f7040092850 con 0x7f705415f490 2024-09-17T05:30:16.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.066+0000 7f7052a00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7034073b00 0x7f7034075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.066+0000 7f7052a00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7034073b00 0x7f7034075fc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7044000f70 tx=0x7f7044009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:16.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.070+0000 7f704b400640 1 -- 172.21.15.122:0/3836978127 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f704005f350 con 0x7f705415f490 2024-09-17T05:30:16.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.250+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f705415b430 con 0x7f705415f490 2024-09-17T05:30:16.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f704b400640 1 -- 172.21.15.122:0/3836978127 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f704005ec50 con 0x7f705415f490 2024-09-17T05:30:16.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7034073b00 msgr2=0x7f7034075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7034073b00 0x7f7034075fc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7044000f70 tx=0x7f7044009040 comp rx=0 tx=0).stop 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705415f490 msgr2=0x7f70540ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705415f490 0x7f70540ccac0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f704000c450 tx=0x7f704000c910 comp rx=0 tx=0).stop 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 shutdown_connections 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7034073b00 0x7f7034075fc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7054164c60 0x7f70541f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705415f490 0x7f70540ccac0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 --2- 172.21.15.122:0/3836978127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70540b2e80 0x7f7054155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 >> 172.21.15.122:0/3836978127 conn(0x7f70540d1e30 msgr2=0x7f70540719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 shutdown_connections 2024-09-17T05:30:16.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.254+0000 7f7059e00640 1 -- 172.21.15.122:0/3836978127 wait complete. 2024-09-17T05:30:16.275 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-17T05:30:16.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.414+0000 7fa0a1400640 1 -- 172.21.15.122:0/494896726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 msgr2=0x7fa09c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.414+0000 7fa0a1400640 1 --2- 172.21.15.122:0/494896726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 0x7fa09c0aeef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa094009c80 tx=0x7fa09401cda0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.414+0000 7fa0a1400640 1 -- 172.21.15.122:0/494896726 shutdown_connections 2024-09-17T05:30:16.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.414+0000 7fa0a1400640 1 --2- 172.21.15.122:0/494896726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 0x7fa09c0aeef0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.414+0000 7fa0a1400640 1 --2- 172.21.15.122:0/494896726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa09c0ced80 0x7fa09c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.414+0000 7fa0a1400640 1 -- 172.21.15.122:0/494896726 >> 172.21.15.122:0/494896726 conn(0x7fa09c0d1dc0 msgr2=0x7fa09c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:16.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.414+0000 7fa0a1400640 1 -- 172.21.15.122:0/494896726 shutdown_connections 2024-09-17T05:30:16.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.414+0000 7fa0a1400640 1 -- 172.21.15.122:0/494896726 wait complete. 2024-09-17T05:30:16.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 Processor -- start 2024-09-17T05:30:16.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 -- start start 2024-09-17T05:30:16.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 0x7fa09c1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa09c0ced80 0x7fa09c0b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa09c15bea0 0x7fa09c158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c169290 con 0x7fa09c0cdb30 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c169110 con 0x7fa09c0ced80 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa09c169410 con 0x7fa09c15bea0 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 0x7fa09c1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 0x7fa09c1562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34916/0 (socket says 172.21.15.122:34916) 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 -- 172.21.15.122:0/2269043971 learned_addr learned my addr 172.21.15.122:0/2269043971 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09b200640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa09c15bea0 0x7fa09c158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa091e00640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa09c0ced80 0x7fa09c0b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 -- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa09c15bea0 msgr2=0x7fa09c158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa09c15bea0 0x7fa09c158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 -- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa09c0ced80 msgr2=0x7fa09c0b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa09c0ced80 0x7fa09c0b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 -- 172.21.15.122:0/2269043971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09c156820 con 0x7fa09c0cdb30 2024-09-17T05:30:16.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa09a800640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 0x7fa09c1562e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa08c001ec0 tx=0x7fa08c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:16.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa093400640 1 -- 172.21.15.122:0/2269043971 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0052d0 con 0x7fa09c0cdb30 2024-09-17T05:30:16.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa093400640 1 -- 172.21.15.122:0/2269043971 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa08c007020 con 0x7fa09c0cdb30 2024-09-17T05:30:16.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa093400640 1 -- 172.21.15.122:0/2269043971 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c007470 con 0x7fa09c0cdb30 2024-09-17T05:30:16.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09c0c93e0 con 0x7fa09c0cdb30 2024-09-17T05:30:16.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.418+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09c0c99a0 con 0x7fa09c0cdb30 2024-09-17T05:30:16.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.422+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa09c0ce670 con 0x7fa09c0cdb30 2024-09-17T05:30:16.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.422+0000 7fa093400640 1 -- 172.21.15.122:0/2269043971 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fa08c0067e0 con 0x7fa09c0cdb30 2024-09-17T05:30:16.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.422+0000 7fa093400640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa078073b30 0x7fa078075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:16.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.422+0000 7fa093400640 1 -- 172.21.15.122:0/2269043971 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fa08c091f20 con 0x7fa09c0cdb30 2024-09-17T05:30:16.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.422+0000 7fa091e00640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa078073b30 0x7fa078075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:16.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.426+0000 7fa091e00640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa078073b30 0x7fa078075ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa09c159bc0 tx=0x7fa08400b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:16.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.426+0000 7fa093400640 1 -- 172.21.15.122:0/2269043971 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa08c05e3d0 con 0x7fa09c0cdb30 2024-09-17T05:30:16.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.654+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7fa09c0aed70 con 0x7fa09c0cdb30 2024-09-17T05:30:16.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa093400640 1 -- 172.21.15.122:0/2269043971 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v242) ==== 166+0+0 (secure 0 0 0) 0x7fa08c062e00 con 0x7fa09c0cdb30 2024-09-17T05:30:16.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa078073b30 msgr2=0x7fa078075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa078073b30 0x7fa078075ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa09c159bc0 tx=0x7fa08400b390 comp rx=0 tx=0).stop 2024-09-17T05:30:16.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 msgr2=0x7fa09c1562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:16.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 0x7fa09c1562e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa08c001ec0 tx=0x7fa08c00c890 comp rx=0 tx=0).stop 2024-09-17T05:30:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 shutdown_connections 2024-09-17T05:30:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa078073b30 0x7fa078075ff0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa09c15bea0 0x7fa09c158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa09c0ced80 0x7fa09c0b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 --2- 172.21.15.122:0/2269043971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa09c0cdb30 0x7fa09c1562e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 >> 172.21.15.122:0/2269043971 conn(0x7fa09c0d1dc0 msgr2=0x7fa09c0b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 shutdown_connections 2024-09-17T05:30:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:16.890+0000 7fa0a1400640 1 -- 172.21.15.122:0/2269043971 wait complete. 2024-09-17T05:30:16.912 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-17T05:30:16.912 INFO:tasks.cephfs_test_runner:Starting test: test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-17T05:30:16.912 INFO:tasks.cephfs_test_runner:test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:30:16.913 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:30:17.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.054+0000 7f08e4e00640 1 -- 172.21.15.122:0/3777527028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4004810 msgr2=0x7f08c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.054+0000 7f08e4e00640 1 --2- 172.21.15.122:0/3777527028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c4004810 0x7f08c4004c90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009b60 tx=0x7f08c801d210 comp rx=0 tx=0).stop 2024-09-17T05:30:17.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.054+0000 7f08de800640 1 -- 172.21.15.122:0/3777527028 reap_dead start 2024-09-17T05:30:17.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.054+0000 7f08e4e00640 1 -- 172.21.15.122:0/3777527028 shutdown_connections 2024-09-17T05:30:17.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.054+0000 7f08e4e00640 1 -- 172.21.15.122:0/3777527028 >> 172.21.15.122:0/3777527028 conn(0x7f08e00d1dc0 msgr2=0x7f08e00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:17.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.054+0000 7f08e4e00640 1 -- 172.21.15.122:0/3777527028 shutdown_connections 2024-09-17T05:30:17.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.054+0000 7f08e4e00640 1 -- 172.21.15.122:0/3777527028 wait complete. 2024-09-17T05:30:17.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 Processor -- start 2024-09-17T05:30:17.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 -- start start 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c4004810 0x7f08e0159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e00c8020 0x7f08e0071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08e0151c30 0x7f08e01520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08e015c470 con 0x7f08e0151c30 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08e00d2af0 con 0x7f08e00c8020 2024-09-17T05:30:17.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f08e00ae9b0 con 0x7f08c4004810 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08e0151c30 0x7f08e01520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08e0151c30 0x7f08e01520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34922/0 (socket says 172.21.15.122:34922) 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 -- 172.21.15.122:0/1492982404 learned_addr learned my addr 172.21.15.122:0/1492982404 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08d5400640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e00c8020 0x7f08e0071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08dde00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c4004810 0x7f08e0159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 -- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c4004810 msgr2=0x7f08e0159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c4004810 0x7f08e0159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 -- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e00c8020 msgr2=0x7f08e0071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e00c8020 0x7f08e0071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 -- 172.21.15.122:0/1492982404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e01f3170 con 0x7f08e0151c30 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08de800640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08e0151c30 0x7f08e01520c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f08e0074d10 tx=0x7f08d8010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08d6a00640 1 -- 172.21.15.122:0/1492982404 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8003980 con 0x7f08e0151c30 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08d6a00640 1 -- 172.21.15.122:0/1492982404 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08d8015070 con 0x7f08e0151c30 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08d6a00640 1 -- 172.21.15.122:0/1492982404 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8004030 con 0x7f08e0151c30 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e01f23b0 con 0x7f08e0151c30 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.058+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e01f2970 con 0x7f08e0151c30 2024-09-17T05:30:17.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.062+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e00c8400 con 0x7f08e0151c30 2024-09-17T05:30:17.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.062+0000 7f08d6a00640 1 -- 172.21.15.122:0/1492982404 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f08d80041d0 con 0x7f08e0151c30 2024-09-17T05:30:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.062+0000 7f08d6a00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08bc073b30 0x7f08bc075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.062+0000 7f08d6a00640 1 -- 172.21.15.122:0/1492982404 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f08d80934d0 con 0x7f08e0151c30 2024-09-17T05:30:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.062+0000 7f08dde00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08bc073b30 0x7f08bc075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.066+0000 7f08dde00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08bc073b30 0x7f08bc075ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f08d0013770 tx=0x7f08d00016f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:17.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.066+0000 7f08d6a00640 1 -- 172.21.15.122:0/1492982404 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f08e00c8400 con 0x7f08e0151c30 2024-09-17T05:30:17.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.234+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08e015ab20 con 0x7f08e0151c30 2024-09-17T05:30:17.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08d6a00640 1 -- 172.21.15.122:0/1492982404 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13137 (secure 0 0 0) 0x7f08d805f980 con 0x7f08e0151c30 2024-09-17T05:30:17.237 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:17.237 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":52,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:30:15.880895+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.173:6837/4117982032":"2024-09-18T05:30:12.870192+0000","172.21.15.173:6835/4117982032":"2024-09-18T05:30:12.870192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:30:17.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08bc073b30 msgr2=0x7f08bc075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08bc073b30 0x7f08bc075ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f08d0013770 tx=0x7f08d00016f0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08e0151c30 msgr2=0x7f08e01520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08e0151c30 0x7f08e01520c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f08e0074d10 tx=0x7f08d8010f40 comp rx=0 tx=0).stop 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 shutdown_connections 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08bc073b30 0x7f08bc075ff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08e0151c30 0x7f08e01520c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e00c8020 0x7f08e0071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 --2- 172.21.15.122:0/1492982404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c4004810 0x7f08e0159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 >> 172.21.15.122:0/1492982404 conn(0x7f08e00d1dc0 msgr2=0x7f08e0159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 shutdown_connections 2024-09-17T05:30:17.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.238+0000 7f08e4e00640 1 -- 172.21.15.122:0/1492982404 wait complete. 2024-09-17T05:30:17.260 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:30:17.260 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:30:17.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.402+0000 7f9fea400640 1 -- 172.21.15.122:0/3500372141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe415bd50 msgr2=0x7f9fe40ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.402+0000 7f9fea400640 1 --2- 172.21.15.122:0/3500372141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe415bd50 0x7f9fe40ce5e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9fe406d870 tx=0x7f9fd401ca60 comp rx=0 tx=0).stop 2024-09-17T05:30:17.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.402+0000 7f9fea400640 1 -- 172.21.15.122:0/3500372141 shutdown_connections 2024-09-17T05:30:17.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.402+0000 7f9fea400640 1 --2- 172.21.15.122:0/3500372141 >> v2:172.21.15.173:3300/0 conn(0x7f9fe415c130 0x7f9fe40ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.402+0000 7f9fea400640 1 --2- 172.21.15.122:0/3500372141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe415bd50 0x7f9fe40ce5e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.402+0000 7f9fea400640 1 -- 172.21.15.122:0/3500372141 >> 172.21.15.122:0/3500372141 conn(0x7f9fe40d1ea0 msgr2=0x7f9fe40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:17.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.402+0000 7f9fea400640 1 -- 172.21.15.122:0/3500372141 shutdown_connections 2024-09-17T05:30:17.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.402+0000 7f9fea400640 1 -- 172.21.15.122:0/3500372141 wait complete. 2024-09-17T05:30:17.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 Processor -- start 2024-09-17T05:30:17.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 -- start start 2024-09-17T05:30:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe40cf4e0 0x7f9fe414d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe415bd50 0x7f9fe4154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe415c130 0x7f9fe4071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4168e50 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4168cd0 con 0x7f9fe415bd50 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe4168fd0 con 0x7f9fe415c130 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe40cf4e0 0x7f9fe414d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe40cf4e0 0x7f9fe414d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34936/0 (socket says 172.21.15.122:34936) 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 -- 172.21.15.122:0/720608362 learned_addr learned my addr 172.21.15.122:0/720608362 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe2a00640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe415bd50 0x7f9fe4154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 -- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe415c130 msgr2=0x7f9fe4071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe415c130 0x7f9fe4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 -- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe415bd50 msgr2=0x7f9fe4154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe415bd50 0x7f9fe4154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 -- 172.21.15.122:0/720608362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe41f8850 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe2a00640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe415bd50 0x7f9fe4154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fe3400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe40cf4e0 0x7f9fe414d690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009b10 tx=0x7f9fd40a58c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fdb400640 1 -- 172.21.15.122:0/720608362 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40b2040 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fdb400640 1 -- 172.21.15.122:0/720608362 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fd40042c0 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fdb400640 1 -- 172.21.15.122:0/720608362 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40046d0 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe41f8b30 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.406+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe41f90f0 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.410+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe40c5dd0 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.410+0000 7f9fdb400640 1 -- 172.21.15.122:0/720608362 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f9fd40ad440 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.410+0000 7f9fdb400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc4073b30 0x7f9fc4075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.410+0000 7f9fdb400640 1 -- 172.21.15.122:0/720608362 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f9fd412d3e0 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.410+0000 7f9fe2a00640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc4073b30 0x7f9fc4075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.414+0000 7f9fe2a00640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc4073b30 0x7f9fc4075ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0005fd0 tx=0x7f9fd0005dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:17.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.414+0000 7f9fdb400640 1 -- 172.21.15.122:0/720608362 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9fd4016770 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.582+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9fe40ceb70 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.582+0000 7f9fdb400640 1 -- 172.21.15.122:0/720608362 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f9fd40f9e60 con 0x7f9fe40cf4e0 2024-09-17T05:30:17.583 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc4073b30 msgr2=0x7f9fc4075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc4073b30 0x7f9fc4075ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0005fd0 tx=0x7f9fd0005dc0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe40cf4e0 msgr2=0x7f9fe414d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe40cf4e0 0x7f9fe414d690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009b10 tx=0x7f9fd40a58c0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 shutdown_connections 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc4073b30 0x7f9fc4075ff0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe415c130 0x7f9fe4071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe415bd50 0x7f9fe4154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 --2- 172.21.15.122:0/720608362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe40cf4e0 0x7f9fe414d690 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 >> 172.21.15.122:0/720608362 conn(0x7f9fe40d1ea0 msgr2=0x7f9fe415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 shutdown_connections 2024-09-17T05:30:17.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.586+0000 7f9fea400640 1 -- 172.21.15.122:0/720608362 wait complete. 2024-09-17T05:30:17.605 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-17T05:30:17.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.746+0000 7f51c4800640 1 -- 172.21.15.122:0/3277711239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a4004d50 msgr2=0x7f51a40051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.746+0000 7f51c4800640 1 --2- 172.21.15.122:0/3277711239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a4004d50 0x7f51a40051b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f51b800c610 tx=0x7f51b801d2c0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.746+0000 7f51c2800640 1 -- 172.21.15.122:0/3277711239 reap_dead start 2024-09-17T05:30:17.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.746+0000 7f51c4800640 1 -- 172.21.15.122:0/3277711239 shutdown_connections 2024-09-17T05:30:17.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.746+0000 7f51c4800640 1 -- 172.21.15.122:0/3277711239 >> 172.21.15.122:0/3277711239 conn(0x7f51bc0d0c90 msgr2=0x7f51bc0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:17.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.746+0000 7f51c4800640 1 -- 172.21.15.122:0/3277711239 shutdown_connections 2024-09-17T05:30:17.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.746+0000 7f51c4800640 1 -- 172.21.15.122:0/3277711239 wait complete. 2024-09-17T05:30:17.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 Processor -- start 2024-09-17T05:30:17.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 -- start start 2024-09-17T05:30:17.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a4004d50 0x7f51bc0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51bc0ce860 0x7f51bc0cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc1f40c0 0x7f51bc1f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc15c340 con 0x7f51bc1f40c0 2024-09-17T05:30:17.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc15c1c0 con 0x7f51bc0ce860 2024-09-17T05:30:17.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f51bc15c4c0 con 0x7f51a4004d50 2024-09-17T05:30:17.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc1f40c0 0x7f51bc1f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc1f40c0 0x7f51bc1f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34960/0 (socket says 172.21.15.122:34960) 2024-09-17T05:30:17.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 -- 172.21.15.122:0/3100196486 learned_addr learned my addr 172.21.15.122:0/3100196486 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:17.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c1400640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51bc0ce860 0x7f51bc0cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c1e00640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a4004d50 0x7f51bc0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 -- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a4004d50 msgr2=0x7f51bc0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a4004d50 0x7f51bc0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 -- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51bc0ce860 msgr2=0x7f51bc0cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:17.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51bc0ce860 0x7f51bc0cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:17.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 -- 172.21.15.122:0/3100196486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc0cedb0 con 0x7f51bc1f40c0 2024-09-17T05:30:17.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c2800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc1f40c0 0x7f51bc1f4540 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f51b801cd80 tx=0x7f51b8002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:17.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51b6000640 1 -- 172.21.15.122:0/3100196486 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8003b80 con 0x7f51bc1f40c0 2024-09-17T05:30:17.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51b6000640 1 -- 172.21.15.122:0/3100196486 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51b8003d20 con 0x7f51bc1f40c0 2024-09-17T05:30:17.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51b6000640 1 -- 172.21.15.122:0/3100196486 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b80ae670 con 0x7f51bc1f40c0 2024-09-17T05:30:17.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51bc0cf090 con 0x7f51bc1f40c0 2024-09-17T05:30:17.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.750+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51bc0cf650 con 0x7f51bc1f40c0 2024-09-17T05:30:17.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.754+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51bc071e50 con 0x7f51bc1f40c0 2024-09-17T05:30:17.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.754+0000 7f51b6000640 1 -- 172.21.15.122:0/3100196486 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f51b80c1060 con 0x7f51bc1f40c0 2024-09-17T05:30:17.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.754+0000 7f51b6000640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a0073b30 0x7f51a0075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:17.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.754+0000 7f51b6000640 1 -- 172.21.15.122:0/3100196486 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f51b812cee0 con 0x7f51bc1f40c0 2024-09-17T05:30:17.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.754+0000 7f51c1e00640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a0073b30 0x7f51a0075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:17.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.758+0000 7f51c1e00640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a0073b30 0x7f51a0075ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000c10 tx=0x7f51b0006b80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:17.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.758+0000 7f51b6000640 1 -- 172.21.15.122:0/3100196486 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f51b8017070 con 0x7f51bc1f40c0 2024-09-17T05:30:17.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:17.998+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7f51bc159ac0 con 0x7f51bc1f40c0 2024-09-17T05:30:18.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.894+0000 7f51b6000640 1 -- 172.21.15.122:0/3100196486 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v244) ==== 170+0+0 (secure 0 0 0) 0x7f51b80f99e0 con 0x7f51bc1f40c0 2024-09-17T05:30:18.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a0073b30 msgr2=0x7f51a0075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:18.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a0073b30 0x7f51a0075ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51b0000c10 tx=0x7f51b0006b80 comp rx=0 tx=0).stop 2024-09-17T05:30:18.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc1f40c0 msgr2=0x7f51bc1f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:18.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc1f40c0 0x7f51bc1f4540 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f51b801cd80 tx=0x7f51b8002040 comp rx=0 tx=0).stop 2024-09-17T05:30:18.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 shutdown_connections 2024-09-17T05:30:18.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a0073b30 0x7f51a0075ff0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc1f40c0 0x7f51bc1f4540 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51bc0ce860 0x7f51bc0cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 --2- 172.21.15.122:0/3100196486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a4004d50 0x7f51bc0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:18.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 >> 172.21.15.122:0/3100196486 conn(0x7f51bc0d0c90 msgr2=0x7f51bc0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:18.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 shutdown_connections 2024-09-17T05:30:18.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:18.898+0000 7f51c4800640 1 -- 172.21.15.122:0/3100196486 wait complete. 2024-09-17T05:30:18.923 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:30:19.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 -- 172.21.15.122:0/3884815111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0ccaf0 msgr2=0x7f712c0b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 --2- 172.21.15.122:0/3884815111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0ccaf0 0x7f712c0b2590 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f712c09b750 tx=0x7f712401ccf0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 -- 172.21.15.122:0/3884815111 shutdown_connections 2024-09-17T05:30:19.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 --2- 172.21.15.122:0/3884815111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0ccaf0 0x7f712c0b2590 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 --2- 172.21.15.122:0/3884815111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f712c0cefb0 0x7f712c0cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 --2- 172.21.15.122:0/3884815111 >> v2:172.21.15.173:3300/0 conn(0x7f712c15f000 0x7f712c0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 -- 172.21.15.122:0/3884815111 >> 172.21.15.122:0/3884815111 conn(0x7f712c09aa10 msgr2=0x7f712c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 -- 172.21.15.122:0/3884815111 shutdown_connections 2024-09-17T05:30:19.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 -- 172.21.15.122:0/3884815111 wait complete. 2024-09-17T05:30:19.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 Processor -- start 2024-09-17T05:30:19.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.070+0000 7f7131c00640 1 -- start start 2024-09-17T05:30:19.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c0ccaf0 0x7f712c0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0cefb0 0x7f712c151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f712c15f000 0x7f712c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f712c15ffd0 con 0x7f712c0cefb0 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f712c15fe50 con 0x7f712c0ccaf0 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f712c160150 con 0x7f712c15f000 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0cefb0 0x7f712c151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0cefb0 0x7f712c151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34966/0 (socket says 172.21.15.122:34966) 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 -- 172.21.15.122:0/2753893570 learned_addr learned my addr 172.21.15.122:0/2753893570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f712be00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c0ccaf0 0x7f712c0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131200640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f712c15f000 0x7f712c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 -- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f712c15f000 msgr2=0x7f712c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f712c15f000 0x7f712c15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 -- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c0ccaf0 msgr2=0x7f712c0c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c0ccaf0 0x7f712c0c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 -- 172.21.15.122:0/2753893570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c15ae90 con 0x7f712c0cefb0 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f712be00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c0ccaf0 0x7f712c0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7123e00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0cefb0 0x7f712c151940 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f711400d780 tx=0x7f711400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7128c00640 1 -- 172.21.15.122:0/2753893570 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114017070 con 0x7f712c0cefb0 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7128c00640 1 -- 172.21.15.122:0/2753893570 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f711400de90 con 0x7f712c0cefb0 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7128c00640 1 -- 172.21.15.122:0/2753893570 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114005020 con 0x7f712c0cefb0 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c15b170 con 0x7f712c0cefb0 2024-09-17T05:30:19.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c156df0 con 0x7f712c0cefb0 2024-09-17T05:30:19.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.074+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c0cc5b0 con 0x7f712c0cefb0 2024-09-17T05:30:19.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.078+0000 7f7128c00640 1 -- 172.21.15.122:0/2753893570 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f71140051c0 con 0x7f712c0cefb0 2024-09-17T05:30:19.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.078+0000 7f7128c00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7108073b00 0x7f7108075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.078+0000 7f7128c00640 1 -- 172.21.15.122:0/2753893570 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f7114095360 con 0x7f712c0cefb0 2024-09-17T05:30:19.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.078+0000 7f712be00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7108073b00 0x7f7108075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.078+0000 7f712be00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7108073b00 0x7f7108075fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f712c1529f0 tx=0x7f711c005eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.082+0000 7f7128c00640 1 -- 172.21.15.122:0/2753893570 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f711405f070 con 0x7f712c0cefb0 2024-09-17T05:30:19.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.250+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f712c156fc0 con 0x7f712c0cefb0 2024-09-17T05:30:19.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.250+0000 7f7128c00640 1 -- 172.21.15.122:0/2753893570 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7f7114063aa0 con 0x7f712c0cefb0 2024-09-17T05:30:19.251 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:30:19.251 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:30:19.251 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:30:19.251 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:30:19.251 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:30:19.251 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:30:19.252 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:30:19.253 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:30:19.253 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:30:19.253 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:30:19.253 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:30:19.253 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:30:19.253 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:30:19.253 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:30:19.253 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:30:19.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7108073b00 msgr2=0x7f7108075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7108073b00 0x7f7108075fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f712c1529f0 tx=0x7f711c005eb0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0cefb0 msgr2=0x7f712c151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0cefb0 0x7f712c151940 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f711400d780 tx=0x7f711400dc40 comp rx=0 tx=0).stop 2024-09-17T05:30:19.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 shutdown_connections 2024-09-17T05:30:19.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7108073b00 0x7f7108075fc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f712c15f000 0x7f712c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f712c0cefb0 0x7f712c151940 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 --2- 172.21.15.122:0/2753893570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c0ccaf0 0x7f712c0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 >> 172.21.15.122:0/2753893570 conn(0x7f712c09aa10 msgr2=0x7f712c15dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 shutdown_connections 2024-09-17T05:30:19.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.254+0000 7f7131c00640 1 -- 172.21.15.122:0/2753893570 wait complete. 2024-09-17T05:30:19.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:30:19.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.418+0000 7fdcb7600640 1 -- 172.21.15.122:0/2945477142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc98003c20 msgr2=0x7fdc98004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.418+0000 7fdcb7600640 1 --2- 172.21.15.122:0/2945477142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc98003c20 0x7fdc98004020 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdca400c3f0 tx=0x7fdca401ced0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.418+0000 7fdcb5600640 1 -- 172.21.15.122:0/2945477142 reap_dead start 2024-09-17T05:30:19.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.418+0000 7fdcb7600640 1 -- 172.21.15.122:0/2945477142 shutdown_connections 2024-09-17T05:30:19.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.418+0000 7fdcb7600640 1 -- 172.21.15.122:0/2945477142 >> 172.21.15.122:0/2945477142 conn(0x7fdcb00d1e00 msgr2=0x7fdcb00d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.418+0000 7fdcb7600640 1 -- 172.21.15.122:0/2945477142 shutdown_connections 2024-09-17T05:30:19.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.418+0000 7fdcb7600640 1 -- 172.21.15.122:0/2945477142 wait complete. 2024-09-17T05:30:19.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 Processor -- start 2024-09-17T05:30:19.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 -- start start 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc98003c20 0x7fdcb01f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcb00cb580 0x7fdcb006df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcb006e480 0x7fdcb015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb0167ff0 con 0x7fdcb00cb580 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb0167e70 con 0x7fdcb006e480 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdcb0168170 con 0x7fdc98003c20 2024-09-17T05:30:19.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcb00cb580 0x7fdcb006df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcb00cb580 0x7fdcb006df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34986/0 (socket says 172.21.15.122:34986) 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 -- 172.21.15.122:0/3360674053 learned_addr learned my addr 172.21.15.122:0/3360674053 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 -- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc98003c20 msgr2=0x7fdcb01f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc98003c20 0x7fdcb01f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 -- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcb006e480 msgr2=0x7fdcb015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcb006e480 0x7fdcb015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 -- 172.21.15.122:0/3360674053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb01fb610 con 0x7fdcb00cb580 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcafe00640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcb00cb580 0x7fdcb006df40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c011b40 tx=0x7fdc9c009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcacc00640 1 -- 172.21.15.122:0/3360674053 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c01e070 con 0x7fdcb00cb580 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcacc00640 1 -- 172.21.15.122:0/3360674053 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc9c004460 con 0x7fdcb00cb580 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb01fb8f0 con 0x7fdcb00cb580 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcacc00640 1 -- 172.21.15.122:0/3360674053 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00fc40 con 0x7fdcb00cb580 2024-09-17T05:30:19.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb01fbeb0 con 0x7fdcb00cb580 2024-09-17T05:30:19.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.422+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb006e8a0 con 0x7fdcb00cb580 2024-09-17T05:30:19.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.426+0000 7fdcacc00640 1 -- 172.21.15.122:0/3360674053 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fdc9c021070 con 0x7fdcb00cb580 2024-09-17T05:30:19.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.426+0000 7fdcacc00640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc90073b30 0x7fdc90075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:19.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.426+0000 7fdcacc00640 1 -- 172.21.15.122:0/3360674053 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fdc9c093440 con 0x7fdcb00cb580 2024-09-17T05:30:19.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.426+0000 7fdcb4c00640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc90073b30 0x7fdc90075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:19.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.426+0000 7fdcb4c00640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc90073b30 0x7fdc90075ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdca400c3f0 tx=0x7fdc98005840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:19.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.430+0000 7fdcacc00640 1 -- 172.21.15.122:0/3360674053 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdc9c05f8f0 con 0x7fdcb00cb580 2024-09-17T05:30:19.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.598+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdcb00b1860 con 0x7fdcb00cb580 2024-09-17T05:30:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcacc00640 1 -- 172.21.15.122:0/3360674053 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fdc9c064320 con 0x7fdcb00cb580 2024-09-17T05:30:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc90073b30 msgr2=0x7fdc90075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc90073b30 0x7fdc90075ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdca400c3f0 tx=0x7fdc98005840 comp rx=0 tx=0).stop 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcb00cb580 msgr2=0x7fdcb006df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcb00cb580 0x7fdcb006df40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c011b40 tx=0x7fdc9c009e50 comp rx=0 tx=0).stop 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 shutdown_connections 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc90073b30 0x7fdc90075ff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcb006e480 0x7fdcb015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcb00cb580 0x7fdcb006df40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 --2- 172.21.15.122:0/3360674053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc98003c20 0x7fdcb01f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 >> 172.21.15.122:0/3360674053 conn(0x7fdcb00d1e00 msgr2=0x7fdcb015e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 shutdown_connections 2024-09-17T05:30:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:19.602+0000 7fdcb7600640 1 -- 172.21.15.122:0/3360674053 wait complete. 2024-09-17T05:30:19.624 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:30:19.624 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:30:19.727 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:30:19.813 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:30:19.813 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:19.813 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:19.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:19.822 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:30:19.822 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:19.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:30:19.832 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:19.832 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:19.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:30:19.873 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:30:19.873 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:19.882 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:30:19.882 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:30:19.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:20.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.058+0000 7fb8df400640 1 -- 172.21.15.122:0/2792030614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 msgr2=0x7fb8d80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.058+0000 7fb8df400640 1 --2- 172.21.15.122:0/2792030614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 0x7fb8d80cebb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009840 tx=0x7fb8c001c820 comp rx=0 tx=0).stop 2024-09-17T05:30:20.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.058+0000 7fb8df400640 1 -- 172.21.15.122:0/2792030614 shutdown_connections 2024-09-17T05:30:20.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.058+0000 7fb8df400640 1 --2- 172.21.15.122:0/2792030614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 0x7fb8d80cebb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.058+0000 7fb8df400640 1 --2- 172.21.15.122:0/2792030614 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d815bea0 0x7fb8d80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.058+0000 7fb8df400640 1 -- 172.21.15.122:0/2792030614 >> 172.21.15.122:0/2792030614 conn(0x7fb8d80d1dc0 msgr2=0x7fb8d80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:20.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.058+0000 7fb8df400640 1 -- 172.21.15.122:0/2792030614 shutdown_connections 2024-09-17T05:30:20.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.058+0000 7fb8df400640 1 -- 172.21.15.122:0/2792030614 wait complete. 2024-09-17T05:30:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8df400640 1 Processor -- start 2024-09-17T05:30:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8df400640 1 -- start start 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8df400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 0x7fb8d8159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8df400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d80d28e0 0x7fb8d8149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8df400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d815bea0 0x7fb8d814a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8df400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8168fc0 con 0x7fb8d815bea0 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8df400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d8168e40 con 0x7fb8d80cdb30 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8df400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d8169140 con 0x7fb8d80d28e0 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d815bea0 0x7fb8d814a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 0x7fb8d8159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d815bea0 0x7fb8d814a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34988/0 (socket says 172.21.15.122:34988) 2024-09-17T05:30:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dd400640 1 -- 172.21.15.122:0/196262706 learned_addr learned my addr 172.21.15.122:0/196262706 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8cfe00640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d80d28e0 0x7fb8d8149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dca00640 1 -- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d80d28e0 msgr2=0x7fb8d8149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dca00640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d80d28e0 0x7fb8d8149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dca00640 1 -- 172.21.15.122:0/196262706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d815bea0 msgr2=0x7fb8d814a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dca00640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d815bea0 0x7fb8d814a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dca00640 1 -- 172.21.15.122:0/196262706 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d80b4680 con 0x7fb8d80cdb30 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dd400640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d815bea0 0x7fb8d814a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.062+0000 7fb8dca00640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 0x7fb8d8159500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800e6c0 tx=0x7fb8c800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8d5600640 1 -- 172.21.15.122:0/196262706 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800cab0 con 0x7fb8d80cdb30 2024-09-17T05:30:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8d5600640 1 -- 172.21.15.122:0/196262706 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8c800cd40 con 0x7fb8d80cdb30 2024-09-17T05:30:20.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8d5600640 1 -- 172.21.15.122:0/196262706 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8010690 con 0x7fb8d80cdb30 2024-09-17T05:30:20.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d81fb050 con 0x7fb8d80cdb30 2024-09-17T05:30:20.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d81fb5d0 con 0x7fb8d80cdb30 2024-09-17T05:30:20.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d80b4400 con 0x7fb8d80cdb30 2024-09-17T05:30:20.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8d5600640 1 -- 172.21.15.122:0/196262706 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fb8c8010830 con 0x7fb8d80cdb30 2024-09-17T05:30:20.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8d5600640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b4073b00 0x7fb8b4075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8d5600640 1 -- 172.21.15.122:0/196262706 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fb8c80939b0 con 0x7fb8d80cdb30 2024-09-17T05:30:20.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.066+0000 7fb8cfe00640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b4073b00 0x7fb8b4075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.070+0000 7fb8cfe00640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b4073b00 0x7fb8b4075fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00af000 tx=0x7fb8c001b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:20.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.070+0000 7fb8d5600640 1 -- 172.21.15.122:0/196262706 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb8d80b4400 con 0x7fb8d80cdb30 2024-09-17T05:30:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.274+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8d80cc9d0 con 0x7fb8d80cdb30 2024-09-17T05:30:20.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.274+0000 7fb8d5600640 1 -- 172.21.15.122:0/196262706 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3917 (secure 0 0 0) 0x7fb8c805fe60 con 0x7fb8d80cdb30 2024-09-17T05:30:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:20.276 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":29,"btime":"2024-09-17T05:30:13:872412+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4988,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3520614210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3520614210},{"type":"v1","addr":"172.21.15.122:6834","nonce":3520614210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5308,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2442281734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2442281734},{"type":"v1","addr":"172.21.15.173:6836","nonce":2442281734}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5379,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/638961094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":638961094},{"type":"v1","addr":"172.21.15.122:6835","nonce":638961094}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/184788625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":184788625},{"type":"v1","addr":"172.21.15.173:6837","nonce":184788625}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-17T05:30:20.276 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 29 2024-09-17T05:30:20.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b4073b00 msgr2=0x7fb8b4075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b4073b00 0x7fb8b4075fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00af000 tx=0x7fb8c001b060 comp rx=0 tx=0).stop 2024-09-17T05:30:20.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 msgr2=0x7fb8d8159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 0x7fb8d8159500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800e6c0 tx=0x7fb8c800eb80 comp rx=0 tx=0).stop 2024-09-17T05:30:20.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 shutdown_connections 2024-09-17T05:30:20.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b4073b00 0x7fb8b4075fc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d815bea0 0x7fb8d814a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d80d28e0 0x7fb8d8149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 --2- 172.21.15.122:0/196262706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d80cdb30 0x7fb8d8159500 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 >> 172.21.15.122:0/196262706 conn(0x7fb8d80d1dc0 msgr2=0x7fb8d815a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:20.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 shutdown_connections 2024-09-17T05:30:20.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.278+0000 7fb8df400640 1 -- 172.21.15.122:0/196262706 wait complete. 2024-09-17T05:30:20.297 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:30:20.297 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:30:20.297 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:30:20.297 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:30:20.298 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:30:20.298 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:30:20.298 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:30:20.298 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:30:20.298 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:30:20.298 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:30:20.299 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:30:20.299 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:30:20.299 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:30:20.299 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:30:20.299 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:30:20.300 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:30:20.309 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:30:20.310 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:30:20.310 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:30:20.311 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:30:20.313 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:30:20.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:30:20.316 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:30:20.316 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:30:20.359 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:30:20.360 DEBUG:teuthology.parallel:result is None 2024-09-17T05:30:20.360 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:30:20.361 DEBUG:teuthology.parallel:result is None 2024-09-17T05:30:20.361 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:30:20.361 DEBUG:teuthology.parallel:result is None 2024-09-17T05:30:20.363 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:30:20.363 DEBUG:teuthology.parallel:result is None 2024-09-17T05:30:20.363 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:30:20.447 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:30:20.448 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:30:20.454 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:30:20.454 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:30:20.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.514+0000 7fb6cf000640 1 -- 172.21.15.122:0/2713283450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6c80cdb30 msgr2=0x7fb6c80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.514+0000 7fb6cf000640 1 --2- 172.21.15.122:0/2713283450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6c80cdb30 0x7fb6c80cebb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009840 tx=0x7fb6b001c820 comp rx=0 tx=0).stop 2024-09-17T05:30:20.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 -- 172.21.15.122:0/2713283450 shutdown_connections 2024-09-17T05:30:20.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 --2- 172.21.15.122:0/2713283450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6c80cdb30 0x7fb6c80cebb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 --2- 172.21.15.122:0/2713283450 >> v2:172.21.15.173:3300/0 conn(0x7fb6c815bea0 0x7fb6c80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 -- 172.21.15.122:0/2713283450 >> 172.21.15.122:0/2713283450 conn(0x7fb6c80d1dc0 msgr2=0x7fb6c80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:20.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 -- 172.21.15.122:0/2713283450 shutdown_connections 2024-09-17T05:30:20.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 -- 172.21.15.122:0/2713283450 wait complete. 2024-09-17T05:30:20.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 Processor -- start 2024-09-17T05:30:20.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 -- start start 2024-09-17T05:30:20.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c80cdb30 0x7fb6c80cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6c80d28e0 0x7fb6c81f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c815bea0 0x7fb6c81572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8168fc0 con 0x7fb6c815bea0 2024-09-17T05:30:20.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8168e40 con 0x7fb6c80cdb30 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cf000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c8169140 con 0x7fb6c80d28e0 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c815bea0 0x7fb6c81572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c80cdb30 0x7fb6c80cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c80cdb30 0x7fb6c80cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:55628/0 (socket says 172.21.15.122:55628) 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 -- 172.21.15.122:0/283559719 learned_addr learned my addr 172.21.15.122:0/283559719 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6bf400640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6c80d28e0 0x7fb6c81f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 -- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6c80d28e0 msgr2=0x7fb6c81f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6c80d28e0 0x7fb6c81f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 -- 172.21.15.122:0/283559719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c815bea0 msgr2=0x7fb6c81572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c815bea0 0x7fb6c81572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 -- 172.21.15.122:0/283559719 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c80cbd40 con 0x7fb6c80cdb30 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6cd000640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c815bea0 0x7fb6c81572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:30:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.518+0000 7fb6c7e00640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c80cdb30 0x7fb6c80cb710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800e7a0 tx=0x7fb6b800ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6c4c00640 1 -- 172.21.15.122:0/283559719 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b800ccc0 con 0x7fb6c80cdb30 2024-09-17T05:30:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6c4c00640 1 -- 172.21.15.122:0/283559719 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6b8010430 con 0x7fb6c80cdb30 2024-09-17T05:30:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c8165f20 con 0x7fb6c80cdb30 2024-09-17T05:30:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6c4c00640 1 -- 172.21.15.122:0/283559719 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8010880 con 0x7fb6c80cdb30 2024-09-17T05:30:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c81577e0 con 0x7fb6c80cdb30 2024-09-17T05:30:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c8165d10 con 0x7fb6c80cdb30 2024-09-17T05:30:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6c4c00640 1 -- 172.21.15.122:0/283559719 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fb6b8010a20 con 0x7fb6c80cdb30 2024-09-17T05:30:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6c4c00640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb6a4073b00 0x7fb6a4075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6c4c00640 1 -- 172.21.15.122:0/283559719 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fb6b80937b0 con 0x7fb6c80cdb30 2024-09-17T05:30:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.522+0000 7fb6bf400640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb6a4073b00 0x7fb6a4075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.526+0000 7fb6bf400640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb6a4073b00 0x7fb6a4075fc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00af000 tx=0x7fb6b001b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:20.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.526+0000 7fb6c4c00640 1 -- 172.21.15.122:0/283559719 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb6b805fc60 con 0x7fb6c80cdb30 2024-09-17T05:30:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.706+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb6c80cc9d0 con 0x7fb6c80cdb30 2024-09-17T05:30:20.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6c4c00640 1 -- 172.21.15.122:0/283559719 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7fb6b8064690 con 0x7fb6c80cdb30 2024-09-17T05:30:20.913 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:30:20.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb6a4073b00 msgr2=0x7fb6a4075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb6a4073b00 0x7fb6a4075fc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00af000 tx=0x7fb6b001b060 comp rx=0 tx=0).stop 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c80cdb30 msgr2=0x7fb6c80cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c80cdb30 0x7fb6c80cb710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800e7a0 tx=0x7fb6b800ec60 comp rx=0 tx=0).stop 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 shutdown_connections 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb6a4073b00 0x7fb6a4075fc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c815bea0 0x7fb6c81572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6c80d28e0 0x7fb6c81f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 --2- 172.21.15.122:0/283559719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6c80cdb30 0x7fb6c80cb710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 >> 172.21.15.122:0/283559719 conn(0x7fb6c80d1dc0 msgr2=0x7fb6c80d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 shutdown_connections 2024-09-17T05:30:20.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:20.914+0000 7fb6cf000640 1 -- 172.21.15.122:0/283559719 wait complete. 2024-09-17T05:30:20.936 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:30:21.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 -- 172.21.15.122:0/888068241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf4015bde0 msgr2=0x7fbf40068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 --2- 172.21.15.122:0/888068241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf4015bde0 0x7fbf40068c00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009870 tx=0x7fbf3401c780 comp rx=0 tx=0).stop 2024-09-17T05:30:21.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 -- 172.21.15.122:0/888068241 shutdown_connections 2024-09-17T05:30:21.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 --2- 172.21.15.122:0/888068241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf4015bde0 0x7fbf40068c00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 -- 172.21.15.122:0/888068241 >> 172.21.15.122:0/888068241 conn(0x7fbf400b2240 msgr2=0x7fbf400b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 -- 172.21.15.122:0/888068241 shutdown_connections 2024-09-17T05:30:21.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 -- 172.21.15.122:0/888068241 wait complete. 2024-09-17T05:30:21.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 Processor -- start 2024-09-17T05:30:21.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.078+0000 7fbf48000640 1 -- start start 2024-09-17T05:30:21.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf400b4270 0x7fbf4015f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf400d0af0 0x7fbf401442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4015bde0 0x7fbf40152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4016a6a0 con 0x7fbf400b4270 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4016a520 con 0x7fbf4015bde0 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4016a820 con 0x7fbf400d0af0 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf400b4270 0x7fbf4015f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf400b4270 0x7fbf4015f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35028/0 (socket says 172.21.15.122:35028) 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 -- 172.21.15.122:0/3947825465 learned_addr learned my addr 172.21.15.122:0/3947825465 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf46000640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4015bde0 0x7fbf40152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf44c00640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf400d0af0 0x7fbf401442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 -- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf400d0af0 msgr2=0x7fbf401442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf400d0af0 0x7fbf401442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 -- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4015bde0 msgr2=0x7fbf40152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4015bde0 0x7fbf40152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 -- 172.21.15.122:0/3947825465 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf401472a0 con 0x7fbf400b4270 2024-09-17T05:30:21.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf45600640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf400b4270 0x7fbf4015f010 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbf3401cc60 tx=0x7fbf340056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf3d600640 1 -- 172.21.15.122:0/3947825465 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340be070 con 0x7fbf400b4270 2024-09-17T05:30:21.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf3d600640 1 -- 172.21.15.122:0/3947825465 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf340a86d0 con 0x7fbf400b4270 2024-09-17T05:30:21.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4015c830 con 0x7fbf400b4270 2024-09-17T05:30:21.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf3d600640 1 -- 172.21.15.122:0/3947825465 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340a8ae0 con 0x7fbf400b4270 2024-09-17T05:30:21.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4015e9a0 con 0x7fbf400b4270 2024-09-17T05:30:21.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf40068c00 con 0x7fbf400b4270 2024-09-17T05:30:21.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.082+0000 7fbf3d600640 1 -- 172.21.15.122:0/3947825465 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fbf340c1070 con 0x7fbf400b4270 2024-09-17T05:30:21.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.086+0000 7fbf3d600640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf20073b00 0x7fbf20075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.086+0000 7fbf3d600640 1 -- 172.21.15.122:0/3947825465 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4885+0+0 (secure 0 0 0) 0x7fbf3412c7c0 con 0x7fbf400b4270 2024-09-17T05:30:21.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.086+0000 7fbf44c00640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf20073b00 0x7fbf20075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.086+0000 7fbf44c00640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf20073b00 0x7fbf20075fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbf4014add0 tx=0x7fbf2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.090+0000 7fbf3d600640 1 -- 172.21.15.122:0/3947825465 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbf340ac030 con 0x7fbf400b4270 2024-09-17T05:30:21.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbf4006f7e0 con 0x7fbf400b4270 2024-09-17T05:30:21.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf3d600640 1 -- 172.21.15.122:0/3947825465 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fbf34016370 con 0x7fbf400b4270 2024-09-17T05:30:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:21.318 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:30:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:30:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:30:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:30:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:30:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:30:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:30:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:30:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:30:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:30:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:30:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:30:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:30:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:30:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:30:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:30:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:30:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:30:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:30:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:30:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:30:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:30:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:30:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:30:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:30:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:30:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:30:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:30:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow r", 2024-09-17T05:30:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:30:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:30:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:30:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:30:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:30:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:30:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:30:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:30:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:30:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:30:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:30:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:30:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:30:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:30:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:30:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:30:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:30:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:30:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:30:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:30:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:30:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:30:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:30:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:30:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:30:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:30:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:30:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:30:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:30:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:30:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:30:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:30:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:30:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:30:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:30:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:30:21.344 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:30:21.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf20073b00 msgr2=0x7fbf20075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf20073b00 0x7fbf20075fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbf4014add0 tx=0x7fbf2c009040 comp rx=0 tx=0).stop 2024-09-17T05:30:21.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf400b4270 msgr2=0x7fbf4015f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf400b4270 0x7fbf4015f010 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbf3401cc60 tx=0x7fbf340056d0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 shutdown_connections 2024-09-17T05:30:21.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbf20073b00 0x7fbf20075fc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4015bde0 0x7fbf40152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf400d0af0 0x7fbf401442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 --2- 172.21.15.122:0/3947825465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf400b4270 0x7fbf4015f010 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 >> 172.21.15.122:0/3947825465 conn(0x7fbf400b2240 msgr2=0x7fbf400b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 shutdown_connections 2024-09-17T05:30:21.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.318+0000 7fbf48000640 1 -- 172.21.15.122:0/3947825465 wait complete. 2024-09-17T05:30:21.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:30:21.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.490+0000 7ff2a2600640 1 -- 172.21.15.122:0/378029098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0cde90 msgr2=0x7ff29c0af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.490+0000 7ff2a2600640 1 --2- 172.21.15.122:0/378029098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0cde90 0x7ff29c0af4e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff28400a950 tx=0x7ff28401db70 comp rx=0 tx=0).stop 2024-09-17T05:30:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.490+0000 7ff2a2600640 1 -- 172.21.15.122:0/378029098 shutdown_connections 2024-09-17T05:30:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.490+0000 7ff2a2600640 1 --2- 172.21.15.122:0/378029098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c0cde90 0x7ff29c0af4e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.490+0000 7ff2a2600640 1 -- 172.21.15.122:0/378029098 >> 172.21.15.122:0/378029098 conn(0x7ff29c073490 msgr2=0x7ff29c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.490+0000 7ff2a2600640 1 -- 172.21.15.122:0/378029098 shutdown_connections 2024-09-17T05:30:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.490+0000 7ff2a2600640 1 -- 172.21.15.122:0/378029098 wait complete. 2024-09-17T05:30:21.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 Processor -- start 2024-09-17T05:30:21.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 -- start start 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff29c0ccca0 0x7ff29c0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c0cde90 0x7ff29c1fb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c15a250 0x7ff29c1fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c168340 con 0x7ff29c0cde90 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c1681c0 con 0x7ff29c15a250 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff29c1684c0 con 0x7ff29c0ccca0 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c0cde90 0x7ff29c1fb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c0cde90 0x7ff29c1fb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60370/0 (socket says 172.21.15.122:60370) 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 -- 172.21.15.122:0/3945645953 learned_addr learned my addr 172.21.15.122:0/3945645953 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a1200640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff29c0ccca0 0x7ff29c0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a1c00640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c15a250 0x7ff29c1fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 -- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff29c0ccca0 msgr2=0x7ff29c0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff29c0ccca0 0x7ff29c0c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 -- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c15a250 msgr2=0x7ff29c1fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c15a250 0x7ff29c1fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 -- 172.21.15.122:0/3945645953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c0c6dc0 con 0x7ff29c0cde90 2024-09-17T05:30:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff293e00640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c0cde90 0x7ff29c1fb0b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff28401d5e0 tx=0x7ff284005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff299600640 1 -- 172.21.15.122:0/3945645953 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840bd070 con 0x7ff29c0cde90 2024-09-17T05:30:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff299600640 1 -- 172.21.15.122:0/3945645953 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff284005320 con 0x7ff29c0cde90 2024-09-17T05:30:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c0b41b0 con 0x7ff29c0cde90 2024-09-17T05:30:21.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff299600640 1 -- 172.21.15.122:0/3945645953 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2840b3040 con 0x7ff29c0cde90 2024-09-17T05:30:21.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c0c68b0 con 0x7ff29c0cde90 2024-09-17T05:30:21.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.494+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29c0af4e0 con 0x7ff29c0cde90 2024-09-17T05:30:21.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.498+0000 7ff299600640 1 -- 172.21.15.122:0/3945645953 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7ff2840c0070 con 0x7ff29c0cde90 2024-09-17T05:30:21.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.498+0000 7ff299600640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff278073b00 0x7ff278075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.498+0000 7ff299600640 1 -- 172.21.15.122:0/3945645953 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4885+0+0 (secure 0 0 0) 0x7ff28412cb70 con 0x7ff29c0cde90 2024-09-17T05:30:21.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.498+0000 7ff2a1200640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff278073b00 0x7ff278075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.498+0000 7ff2a1200640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff278073b00 0x7ff278075fc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff28c007fd0 tx=0x7ff28c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.502+0000 7ff299600640 1 -- 172.21.15.122:0/3945645953 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff284017760 con 0x7ff29c0cde90 2024-09-17T05:30:21.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.670+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff29c06df50 con 0x7ff29c0cde90 2024-09-17T05:30:21.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.670+0000 7ff299600640 1 -- 172.21.15.122:0/3945645953 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) ==== 84+0+65 (secure 0 0 0) 0x7ff2840f8fa0 con 0x7ff29c0cde90 2024-09-17T05:30:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:21.671 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:30:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:30:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:30:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:21.672 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:30:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff278073b00 msgr2=0x7ff278075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff278073b00 0x7ff278075fc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff28c007fd0 tx=0x7ff28c009040 comp rx=0 tx=0).stop 2024-09-17T05:30:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c0cde90 msgr2=0x7ff29c1fb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c0cde90 0x7ff29c1fb0b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff28401d5e0 tx=0x7ff284005cc0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 shutdown_connections 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff278073b00 0x7ff278075fc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff29c15a250 0x7ff29c1fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c0cde90 0x7ff29c1fb0b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 --2- 172.21.15.122:0/3945645953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff29c0ccca0 0x7ff29c0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 >> 172.21.15.122:0/3945645953 conn(0x7ff29c073490 msgr2=0x7ff29c0706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 shutdown_connections 2024-09-17T05:30:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.674+0000 7ff2a2600640 1 -- 172.21.15.122:0/3945645953 wait complete. 2024-09-17T05:30:21.693 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:30:21.693 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:30:21.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.838+0000 7f2b1d200640 1 -- 172.21.15.122:0/1016715168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180734f0 msgr2=0x7f2b1815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.838+0000 7f2b1d200640 1 --2- 172.21.15.122:0/1016715168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180734f0 0x7f2b1815a9d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800a950 tx=0x7f2b0801dab0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.838+0000 7f2b1d200640 1 -- 172.21.15.122:0/1016715168 shutdown_connections 2024-09-17T05:30:21.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.838+0000 7f2b1d200640 1 --2- 172.21.15.122:0/1016715168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b180cea60 0x7f2b1806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.838+0000 7f2b1d200640 1 --2- 172.21.15.122:0/1016715168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180734f0 0x7f2b1815a9d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.838+0000 7f2b1d200640 1 -- 172.21.15.122:0/1016715168 >> 172.21.15.122:0/1016715168 conn(0x7f2b180cfbc0 msgr2=0x7f2b180cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:21.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.838+0000 7f2b1d200640 1 -- 172.21.15.122:0/1016715168 shutdown_connections 2024-09-17T05:30:21.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.838+0000 7f2b1d200640 1 -- 172.21.15.122:0/1016715168 wait complete. 2024-09-17T05:30:21.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 Processor -- start 2024-09-17T05:30:21.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 -- start start 2024-09-17T05:30:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b180734f0 0x7f2b181f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180cea60 0x7f2b181f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b180cf440 0x7f2b180cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1806fa50 con 0x7f2b180cea60 2024-09-17T05:30:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1806f8d0 con 0x7f2b180734f0 2024-09-17T05:30:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1806fbd0 con 0x7f2b180cf440 2024-09-17T05:30:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180cea60 0x7f2b181f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180cea60 0x7f2b181f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60386/0 (socket says 172.21.15.122:60386) 2024-09-17T05:30:21.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 -- 172.21.15.122:0/637307514 learned_addr learned my addr 172.21.15.122:0/637307514 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:21.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15e00640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b180734f0 0x7f2b181f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b16800640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b180cf440 0x7f2b180cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 -- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b180cf440 msgr2=0x7f2b180cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b180cf440 0x7f2b180cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 -- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b180734f0 msgr2=0x7f2b181f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b180734f0 0x7f2b181f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 -- 172.21.15.122:0/637307514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b181f5f30 con 0x7f2b180cea60 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b15400640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180cea60 0x7f2b181f9900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c460 tx=0x7f2b0400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b0e000640 1 -- 172.21.15.122:0/637307514 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04016020 con 0x7f2b180cea60 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b0e000640 1 -- 172.21.15.122:0/637307514 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b04012940 con 0x7f2b180cea60 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b181f6210 con 0x7f2b180cea60 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b0e000640 1 -- 172.21.15.122:0/637307514 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04012d90 con 0x7f2b180cea60 2024-09-17T05:30:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181f67d0 con 0x7f2b180cea60 2024-09-17T05:30:21.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.842+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b18069410 con 0x7f2b180cea60 2024-09-17T05:30:21.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.846+0000 7f2b0e000640 1 -- 172.21.15.122:0/637307514 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2b04019020 con 0x7f2b180cea60 2024-09-17T05:30:21.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.846+0000 7f2b0e000640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2af8073b00 0x7f2af8075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:21.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.846+0000 7f2b0e000640 1 -- 172.21.15.122:0/637307514 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4885+0+0 (secure 0 0 0) 0x7f2b04092860 con 0x7f2b180cea60 2024-09-17T05:30:21.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.846+0000 7f2b15e00640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2af8073b00 0x7f2af8075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:21.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.846+0000 7f2b15e00640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2af8073b00 0x7f2af8075fc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b080b0000 tx=0x7f2b0801c2f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:21.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:21.850+0000 7f2b0e000640 1 -- 172.21.15.122:0/637307514 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2b04020020 con 0x7f2b180cea60 2024-09-17T05:30:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.018+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2b180d0850 con 0x7f2b180cea60 2024-09-17T05:30:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.918+0000 7f2b0e000640 1 -- 172.21.15.122:0/637307514 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) ==== 138+0+0 (secure 0 0 0) 0x7f2b0405edd0 con 0x7f2b180cea60 2024-09-17T05:30:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:30:22.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2af8073b00 msgr2=0x7f2af8075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:22.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2af8073b00 0x7f2af8075fc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b080b0000 tx=0x7f2b0801c2f0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180cea60 msgr2=0x7f2b181f9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180cea60 0x7f2b181f9900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c460 tx=0x7f2b0400c920 comp rx=0 tx=0).stop 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 shutdown_connections 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2af8073b00 0x7f2af8075fc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b180cf440 0x7f2b180cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b180cea60 0x7f2b181f9900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 --2- 172.21.15.122:0/637307514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b180734f0 0x7f2b181f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 >> 172.21.15.122:0/637307514 conn(0x7f2b180cfbc0 msgr2=0x7f2b1815bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 shutdown_connections 2024-09-17T05:30:22.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:22.922+0000 7f2b1d200640 1 -- 172.21.15.122:0/637307514 wait complete. 2024-09-17T05:30:22.951 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:30:23.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.102+0000 7f1540e00640 1 -- 172.21.15.122:0/822463108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c15c130 msgr2=0x7f153c0ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.102+0000 7f1540e00640 1 --2- 172.21.15.122:0/822463108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c15c130 0x7f153c0ceb20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f153c073a20 tx=0x7f152801c7a0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.102+0000 7f1540e00640 1 -- 172.21.15.122:0/822463108 shutdown_connections 2024-09-17T05:30:23.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.102+0000 7f1540e00640 1 --2- 172.21.15.122:0/822463108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c15c130 0x7f153c0ceb20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.102+0000 7f1540e00640 1 --2- 172.21.15.122:0/822463108 >> v2:172.21.15.173:3300/0 conn(0x7f153c15bd50 0x7f153c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.102+0000 7f1540e00640 1 -- 172.21.15.122:0/822463108 >> 172.21.15.122:0/822463108 conn(0x7f153c0d1ea0 msgr2=0x7f153c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:23.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.102+0000 7f1540e00640 1 -- 172.21.15.122:0/822463108 shutdown_connections 2024-09-17T05:30:23.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.102+0000 7f1540e00640 1 -- 172.21.15.122:0/822463108 wait complete. 2024-09-17T05:30:23.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 Processor -- start 2024-09-17T05:30:23.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 -- start start 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c0cf4e0 0x7f153c0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f153c15bd50 0x7f153c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f153c15c130 0x7f153c152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f153c168e50 con 0x7f153c0cf4e0 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f153c168cd0 con 0x7f153c15c130 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f153c168fd0 con 0x7f153c15bd50 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c0cf4e0 0x7f153c0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c0cf4e0 0x7f153c0b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60408/0 (socket says 172.21.15.122:60408) 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 -- 172.21.15.122:0/3425052174 learned_addr learned my addr 172.21.15.122:0/3425052174 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f153a800640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f153c15c130 0x7f153c152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539400640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f153c15bd50 0x7f153c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 -- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f153c15bd50 msgr2=0x7f153c0b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f153c15bd50 0x7f153c0b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 -- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f153c15c130 msgr2=0x7f153c152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f153c15c130 0x7f153c152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 -- 172.21.15.122:0/3425052174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c0713f0 con 0x7f153c0cf4e0 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1539e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c0cf4e0 0x7f153c0b75a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f152c009fd0 tx=0x7f152c00c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1532000640 1 -- 172.21.15.122:0/3425052174 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c00ce50 con 0x7f153c0cf4e0 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1532000640 1 -- 172.21.15.122:0/3425052174 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f152c011470 con 0x7f153c0cf4e0 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c0b51b0 con 0x7f153c0cf4e0 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1532000640 1 -- 172.21.15.122:0/3425052174 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c0118a0 con 0x7f153c0cf4e0 2024-09-17T05:30:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.106+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c0b5770 con 0x7f153c0cf4e0 2024-09-17T05:30:23.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.110+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c0c5dd0 con 0x7f153c0cf4e0 2024-09-17T05:30:23.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.110+0000 7f1532000640 1 -- 172.21.15.122:0/3425052174 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f152c011a40 con 0x7f153c0cf4e0 2024-09-17T05:30:23.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.110+0000 7f1532000640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f151c073ca0 0x7f151c076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:23.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.110+0000 7f1532000640 1 -- 172.21.15.122:0/3425052174 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5255+0+0 (secure 0 0 0) 0x7f152c093610 con 0x7f153c0cf4e0 2024-09-17T05:30:23.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.110+0000 7f1539400640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f151c073ca0 0x7f151c076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.110+0000 7f1539400640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f151c073ca0 0x7f151c076160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f15280af000 tx=0x7f15280a7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:23.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.114+0000 7f1532000640 1 -- 172.21.15.122:0/3425052174 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f152c016030 con 0x7f153c0cf4e0 2024-09-17T05:30:23.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.286+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f153c076360 con 0x7f153c0cf4e0 2024-09-17T05:30:23.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.922+0000 7f1532000640 1 -- 172.21.15.122:0/3425052174 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) ==== 170+0+0 (secure 0 0 0) 0x7f152c05fac0 con 0x7f153c0cf4e0 2024-09-17T05:30:23.923 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:30:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f151c073ca0 msgr2=0x7f151c076160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f151c073ca0 0x7f151c076160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f15280af000 tx=0x7f15280a7040 comp rx=0 tx=0).stop 2024-09-17T05:30:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c0cf4e0 msgr2=0x7f153c0b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c0cf4e0 0x7f153c0b75a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f152c009fd0 tx=0x7f152c00c270 comp rx=0 tx=0).stop 2024-09-17T05:30:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 shutdown_connections 2024-09-17T05:30:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f151c073ca0 0x7f151c076160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f153c15c130 0x7f153c152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f153c15bd50 0x7f153c0b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 --2- 172.21.15.122:0/3425052174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f153c0cf4e0 0x7f153c0b75a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 >> 172.21.15.122:0/3425052174 conn(0x7f153c0d1ea0 msgr2=0x7f153c15abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 shutdown_connections 2024-09-17T05:30:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:23.926+0000 7f1540e00640 1 -- 172.21.15.122:0/3425052174 wait complete. 2024-09-17T05:30:23.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:30:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 -- 172.21.15.122:0/1169957265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89680cdf20 msgr2=0x7f89680b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 --2- 172.21.15.122:0/1169957265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89680cdf20 0x7f89680b1a30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8960009bc0 tx=0x7f896001cb70 comp rx=0 tx=0).stop 2024-09-17T05:30:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 -- 172.21.15.122:0/1169957265 shutdown_connections 2024-09-17T05:30:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 --2- 172.21.15.122:0/1169957265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89680cdf20 0x7f89680b1a30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 -- 172.21.15.122:0/1169957265 >> 172.21.15.122:0/1169957265 conn(0x7f89680d1e00 msgr2=0x7f89680d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 -- 172.21.15.122:0/1169957265 shutdown_connections 2024-09-17T05:30:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 -- 172.21.15.122:0/1169957265 wait complete. 2024-09-17T05:30:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 Processor -- start 2024-09-17T05:30:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.114+0000 7f896da00640 1 -- start start 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89680b3bb0 0x7f896806f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89680cdf20 0x7f8968151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f896815f340 0x7f8968159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f896816a3e0 con 0x7f896815f340 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f896816a260 con 0x7f89680cdf20 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f896816a560 con 0x7f89680b3bb0 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f896815f340 0x7f8968159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f896815f340 0x7f8968159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60422/0 (socket says 172.21.15.122:60422) 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 -- 172.21.15.122:0/1424928907 learned_addr learned my addr 172.21.15.122:0/1424928907 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8966000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89680cdf20 0x7f8968151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8966a00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89680b3bb0 0x7f896806f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 -- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89680b3bb0 msgr2=0x7f896806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89680b3bb0 0x7f896806f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 -- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89680cdf20 msgr2=0x7f8968151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89680cdf20 0x7f8968151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 -- 172.21.15.122:0/1424928907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8968158de0 con 0x7f896815f340 2024-09-17T05:30:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f8967400640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f896815f340 0x7f8968159ab0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f896001a7a0 tx=0x7f8960009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f895ea00640 1 -- 172.21.15.122:0/1424928907 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89600b2040 con 0x7f896815f340 2024-09-17T05:30:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f895ea00640 1 -- 172.21.15.122:0/1424928907 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8960004d40 con 0x7f896815f340 2024-09-17T05:30:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f895ea00640 1 -- 172.21.15.122:0/1424928907 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8960004240 con 0x7f896815f340 2024-09-17T05:30:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89681590c0 con 0x7f896815f340 2024-09-17T05:30:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896806fc90 con 0x7f896815f340 2024-09-17T05:30:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.118+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89680b34a0 con 0x7f896815f340 2024-09-17T05:30:24.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.122+0000 7f895ea00640 1 -- 172.21.15.122:0/1424928907 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f89600be060 con 0x7f896815f340 2024-09-17T05:30:24.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.122+0000 7f895ea00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8948073b00 0x7f8948075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:24.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.122+0000 7f895ea00640 1 -- 172.21.15.122:0/1424928907 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5637+0+0 (secure 0 0 0) 0x7f896012ca40 con 0x7f896815f340 2024-09-17T05:30:24.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.122+0000 7f8966a00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8948073b00 0x7f8948075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:24.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.122+0000 7f8966a00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8948073b00 0x7f8948075fc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8958007d70 tx=0x7f8958007d00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:24.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.126+0000 7f895ea00640 1 -- 172.21.15.122:0/1424928907 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8960016790 con 0x7f896815f340 2024-09-17T05:30:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.326+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f896815ae50 con 0x7f896815f340 2024-09-17T05:30:24.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.946+0000 7f895ea00640 1 -- 172.21.15.122:0/1424928907 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f89600f8ef0 con 0x7f896815f340 2024-09-17T05:30:24.949 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 8 and data pool 9 2024-09-17T05:30:24.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8948073b00 msgr2=0x7f8948075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8948073b00 0x7f8948075fc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8958007d70 tx=0x7f8958007d00 comp rx=0 tx=0).stop 2024-09-17T05:30:24.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f896815f340 msgr2=0x7f8968159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:24.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f896815f340 0x7f8968159ab0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f896001a7a0 tx=0x7f8960009300 comp rx=0 tx=0).stop 2024-09-17T05:30:24.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 shutdown_connections 2024-09-17T05:30:24.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8948073b00 0x7f8948075fc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f896815f340 0x7f8968159ab0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89680cdf20 0x7f8968151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 --2- 172.21.15.122:0/1424928907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89680b3bb0 0x7f896806f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:24.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 >> 172.21.15.122:0/1424928907 conn(0x7f89680d1e00 msgr2=0x7f89680b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:24.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.954+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 shutdown_connections 2024-09-17T05:30:24.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:24.958+0000 7f896da00640 1 -- 172.21.15.122:0/1424928907 wait complete. 2024-09-17T05:30:24.994 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:30:25.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.134+0000 7f80f1e00640 1 -- 172.21.15.122:0/924434864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0cdb30 msgr2=0x7f80ec0af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.134+0000 7f80f1e00640 1 --2- 172.21.15.122:0/924434864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0cdb30 0x7f80ec0af0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009bc0 tx=0x7f80e401cba0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.134+0000 7f80f1e00640 1 -- 172.21.15.122:0/924434864 shutdown_connections 2024-09-17T05:30:25.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.134+0000 7f80f1e00640 1 --2- 172.21.15.122:0/924434864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0cdb30 0x7f80ec0af0c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.134+0000 7f80f1e00640 1 -- 172.21.15.122:0/924434864 >> 172.21.15.122:0/924434864 conn(0x7f80ec0d1dc0 msgr2=0x7f80ec0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.134+0000 7f80f1e00640 1 -- 172.21.15.122:0/924434864 shutdown_connections 2024-09-17T05:30:25.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.134+0000 7f80f1e00640 1 -- 172.21.15.122:0/924434864 wait complete. 2024-09-17T05:30:25.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 Processor -- start 2024-09-17T05:30:25.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 -- start start 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80ec0cdb30 0x7f80ec0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0ced80 0x7f80ec1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80ec15bea0 0x7f80ec159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec16a840 con 0x7f80ec0ced80 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80ec16a6c0 con 0x7f80ec0cdb30 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f80ec16a9c0 con 0x7f80ec15bea0 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0ced80 0x7f80ec1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0ced80 0x7f80ec1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60432/0 (socket says 172.21.15.122:60432) 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 -- 172.21.15.122:0/1513951279 learned_addr learned my addr 172.21.15.122:0/1513951279 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:25.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80eaa00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80ec0cdb30 0x7f80ec0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80eb400640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80ec15bea0 0x7f80ec159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 -- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80ec15bea0 msgr2=0x7f80ec159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80ec15bea0 0x7f80ec159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 -- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80ec0cdb30 msgr2=0x7f80ec0ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80ec0cdb30 0x7f80ec0ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 -- 172.21.15.122:0/1513951279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec09ebb0 con 0x7f80ec0ced80 2024-09-17T05:30:25.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80ea000640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0ced80 0x7f80ec1595b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c420 tx=0x7f80d800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80e3400640 1 -- 172.21.15.122:0/1513951279 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8017070 con 0x7f80ec0ced80 2024-09-17T05:30:25.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80e3400640 1 -- 172.21.15.122:0/1513951279 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80d8003600 con 0x7f80ec0ced80 2024-09-17T05:30:25.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.138+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec09ee90 con 0x7f80ec0ced80 2024-09-17T05:30:25.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.142+0000 7f80e3400640 1 -- 172.21.15.122:0/1513951279 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8003a90 con 0x7f80ec0ced80 2024-09-17T05:30:25.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.142+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80ec09f450 con 0x7f80ec0ced80 2024-09-17T05:30:25.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.142+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80ec0ce670 con 0x7f80ec0ced80 2024-09-17T05:30:25.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.142+0000 7f80e3400640 1 -- 172.21.15.122:0/1513951279 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f80d801a070 con 0x7f80ec0ced80 2024-09-17T05:30:25.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.142+0000 7f80e3400640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80cc073b00 0x7f80cc075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.142+0000 7f80e3400640 1 -- 172.21.15.122:0/1513951279 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f80d80946a0 con 0x7f80ec0ced80 2024-09-17T05:30:25.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.142+0000 7f80eaa00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80cc073b00 0x7f80cc075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.146+0000 7f80eaa00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80cc073b00 0x7f80cc075fc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80dc007fd0 tx=0x7f80dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.146+0000 7f80e3400640 1 -- 172.21.15.122:0/1513951279 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f80d805e1c0 con 0x7f80ec0ced80 2024-09-17T05:30:25.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.318+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80ec09f8d0 con 0x7f80ec0ced80 2024-09-17T05:30:25.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.318+0000 7f80e3400640 1 -- 172.21.15.122:0/1513951279 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16841 (secure 0 0 0) 0x7f80d8062bf0 con 0x7f80ec0ced80 2024-09-17T05:30:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:25.320 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":59,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:30:24.330713+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:30:22.033029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:30:23.299476+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80cc073b00 msgr2=0x7f80cc075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80cc073b00 0x7f80cc075fc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80dc007fd0 tx=0x7f80dc009040 comp rx=0 tx=0).stop 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0ced80 msgr2=0x7f80ec1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0ced80 0x7f80ec1595b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f80d800c420 tx=0x7f80d800c8e0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 shutdown_connections 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80cc073b00 0x7f80cc075fc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80ec15bea0 0x7f80ec159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80ec0ced80 0x7f80ec1595b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 --2- 172.21.15.122:0/1513951279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80ec0cdb30 0x7f80ec0ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 >> 172.21.15.122:0/1513951279 conn(0x7f80ec0d1dc0 msgr2=0x7f80ec0b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 shutdown_connections 2024-09-17T05:30:25.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.322+0000 7f80f1e00640 1 -- 172.21.15.122:0/1513951279 wait complete. 2024-09-17T05:30:25.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:30:25.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.482+0000 7fb64c000640 1 -- 172.21.15.122:0/3107694379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 msgr2=0x7fb64406f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.482+0000 7fb64c000640 1 --2- 172.21.15.122:0/3107694379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 0x7fb64406f520 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb630009830 tx=0x7fb644070980 comp rx=0 tx=0).stop 2024-09-17T05:30:25.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.482+0000 7fb64c000640 1 -- 172.21.15.122:0/3107694379 shutdown_connections 2024-09-17T05:30:25.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.482+0000 7fb64c000640 1 --2- 172.21.15.122:0/3107694379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 0x7fb64406f520 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.482+0000 7fb64c000640 1 -- 172.21.15.122:0/3107694379 >> 172.21.15.122:0/3107694379 conn(0x7fb6440b2240 msgr2=0x7fb6440b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.482+0000 7fb64c000640 1 -- 172.21.15.122:0/3107694379 shutdown_connections 2024-09-17T05:30:25.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.482+0000 7fb64c000640 1 -- 172.21.15.122:0/3107694379 wait complete. 2024-09-17T05:30:25.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 Processor -- start 2024-09-17T05:30:25.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 -- start start 2024-09-17T05:30:25.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb64406d330 0x7fb64415e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 0x7fb64413db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6440ce800 0x7fb644155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb644169440 con 0x7fb644073170 2024-09-17T05:30:25.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441692c0 con 0x7fb6440ce800 2024-09-17T05:30:25.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb6441695c0 con 0x7fb64406d330 2024-09-17T05:30:25.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 0x7fb64413db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 0x7fb64413db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60444/0 (socket says 172.21.15.122:60444) 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 -- 172.21.15.122:0/2039066200 learned_addr learned my addr 172.21.15.122:0/2039066200 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64a000640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6440ce800 0x7fb644155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb649600640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb64406d330 0x7fb64415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 -- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb64406d330 msgr2=0x7fb64415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb64406d330 0x7fb64415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 -- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6440ce800 msgr2=0x7fb644155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6440ce800 0x7fb644155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 -- 172.21.15.122:0/2039066200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64413e480 con 0x7fb644073170 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb648c00640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 0x7fb64413db10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb630009780 tx=0x7fb6300a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb641600640 1 -- 172.21.15.122:0/2039066200 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6300b2040 con 0x7fb644073170 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb641600640 1 -- 172.21.15.122:0/2039066200 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6300a5c50 con 0x7fb644073170 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64413e760 con 0x7fb644073170 2024-09-17T05:30:25.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb641600640 1 -- 172.21.15.122:0/2039066200 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630003860 con 0x7fb644073170 2024-09-17T05:30:25.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64413ed20 con 0x7fb644073170 2024-09-17T05:30:25.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.486+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64406cc20 con 0x7fb644073170 2024-09-17T05:30:25.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.490+0000 7fb641600640 1 -- 172.21.15.122:0/2039066200 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fb6300bd060 con 0x7fb644073170 2024-09-17T05:30:25.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.490+0000 7fb641600640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb624073b30 0x7fb624075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.490+0000 7fb641600640 1 -- 172.21.15.122:0/2039066200 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fb63012c2e0 con 0x7fb644073170 2024-09-17T05:30:25.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.490+0000 7fb649600640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb624073b30 0x7fb624075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.490+0000 7fb649600640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb624073b30 0x7fb624075ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6380012f0 tx=0x7fb638009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.494+0000 7fb641600640 1 -- 172.21.15.122:0/2039066200 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb630016410 con 0x7fb644073170 2024-09-17T05:30:25.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.662+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6440b2e00 con 0x7fb644073170 2024-09-17T05:30:25.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb641600640 1 -- 172.21.15.122:0/2039066200 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16841 (secure 0 0 0) 0x7fb6300f8de0 con 0x7fb644073170 2024-09-17T05:30:25.666 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:25.666 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":59,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:30:24.330713+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:30:22.033029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:30:23.299476+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb624073b30 msgr2=0x7fb624075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb624073b30 0x7fb624075ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6380012f0 tx=0x7fb638009040 comp rx=0 tx=0).stop 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 msgr2=0x7fb64413db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 0x7fb64413db10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb630009780 tx=0x7fb6300a5670 comp rx=0 tx=0).stop 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 shutdown_connections 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb624073b30 0x7fb624075ff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6440ce800 0x7fb644155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb644073170 0x7fb64413db10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 --2- 172.21.15.122:0/2039066200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb64406d330 0x7fb64415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 >> 172.21.15.122:0/2039066200 conn(0x7fb6440b2240 msgr2=0x7fb64415ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 shutdown_connections 2024-09-17T05:30:25.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.666+0000 7fb64c000640 1 -- 172.21.15.122:0/2039066200 wait complete. 2024-09-17T05:30:25.688 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:30:25.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- 172.21.15.122:0/3860892777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e281605d0 msgr2=0x7f2e280d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/3860892777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e281605d0 0x7f2e280d0690 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009900 tx=0x7f2e1801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- 172.21.15.122:0/3860892777 shutdown_connections 2024-09-17T05:30:25.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/3860892777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280afb10 0x7f2e280affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/3860892777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e281605d0 0x7f2e280d0690 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- 172.21.15.122:0/3860892777 >> 172.21.15.122:0/3860892777 conn(0x7f2e280d1dd0 msgr2=0x7f2e280d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:25.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- 172.21.15.122:0/3860892777 shutdown_connections 2024-09-17T05:30:25.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- 172.21.15.122:0/3860892777 wait complete. 2024-09-17T05:30:25.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 Processor -- start 2024-09-17T05:30:25.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- start start 2024-09-17T05:30:25.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e280afb10 0x7f2e28148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280b1650 0x7f2e2806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e281605d0 0x7f2e280b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e280b0e60 con 0x7f2e280afb10 2024-09-17T05:30:25.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2e280b1430 con 0x7f2e281605d0 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2fa00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2e28169540 con 0x7f2e280b1650 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e280afb10 0x7f2e28148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e280afb10 0x7f2e28148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60456/0 (socket says 172.21.15.122:60456) 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2d000640 1 -- 172.21.15.122:0/460315123 learned_addr learned my addr 172.21.15.122:0/460315123 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e1fe00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280b1650 0x7f2e2806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2da00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e281605d0 0x7f2e280b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2d000640 1 -- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280b1650 msgr2=0x7f2e2806e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2d000640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280b1650 0x7f2e2806e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2d000640 1 -- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e281605d0 msgr2=0x7f2e280b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2d000640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e281605d0 0x7f2e280b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.834+0000 7f2e2d000640 1 -- 172.21.15.122:0/460315123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2815dec0 con 0x7f2e280afb10 2024-09-17T05:30:25.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e2d000640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e280afb10 0x7f2e28148150 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009850 tx=0x7f2e18005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e25600640 1 -- 172.21.15.122:0/460315123 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e180b2040 con 0x7f2e280afb10 2024-09-17T05:30:25.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e25600640 1 -- 172.21.15.122:0/460315123 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e18004330 con 0x7f2e280afb10 2024-09-17T05:30:25.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2815e1a0 con 0x7f2e280afb10 2024-09-17T05:30:25.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e25600640 1 -- 172.21.15.122:0/460315123 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18005290 con 0x7f2e280afb10 2024-09-17T05:30:25.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e28148690 con 0x7f2e280afb10 2024-09-17T05:30:25.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e280affc0 con 0x7f2e280afb10 2024-09-17T05:30:25.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e25600640 1 -- 172.21.15.122:0/460315123 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2e180bd060 con 0x7f2e280afb10 2024-09-17T05:30:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e25600640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b30 0x7f2e04075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e25600640 1 -- 172.21.15.122:0/460315123 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7f2e1812e6a0 con 0x7f2e280afb10 2024-09-17T05:30:25.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.838+0000 7f2e1fe00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b30 0x7f2e04075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:25.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.842+0000 7f2e1fe00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b30 0x7f2e04075ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e280b4950 tx=0x7f2e10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:25.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:25.842+0000 7f2e25600640 1 -- 172.21.15.122:0/460315123 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2e180164d0 con 0x7f2e280afb10 2024-09-17T05:30:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.050+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2e280d1080 con 0x7f2e280afb10 2024-09-17T05:30:26.915 INFO:tasks.ceph.mds.d.smithi173.stderr:2024-09-17T05:30:26.918+0000 7f6038000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:30:26.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.974+0000 7f2e25600640 1 -- 172.21.15.122:0/460315123 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f2e180f8b10 con 0x7f2e280afb10 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b30 msgr2=0x7f2e04075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b30 0x7f2e04075ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e280b4950 tx=0x7f2e10009040 comp rx=0 tx=0).stop 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e280afb10 msgr2=0x7f2e28148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e280afb10 0x7f2e28148150 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009850 tx=0x7f2e18005760 comp rx=0 tx=0).stop 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 shutdown_connections 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b30 0x7f2e04075ff0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e281605d0 0x7f2e280b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280b1650 0x7f2e2806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 --2- 172.21.15.122:0/460315123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e280afb10 0x7f2e28148150 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 >> 172.21.15.122:0/460315123 conn(0x7f2e280d1dd0 msgr2=0x7f2e280ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:26.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 shutdown_connections 2024-09-17T05:30:26.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:26.978+0000 7f2e2fa00640 1 -- 172.21.15.122:0/460315123 wait complete. 2024-09-17T05:30:27.000 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:30:27.001 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:30:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.146+0000 7f5d26400640 1 -- 172.21.15.122:0/1937090338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 msgr2=0x7f5d2006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.146+0000 7f5d26400640 1 --2- 172.21.15.122:0/1937090338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 0x7f5d2006f7b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009840 tx=0x7f5d0c01c700 comp rx=0 tx=0).stop 2024-09-17T05:30:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 -- 172.21.15.122:0/1937090338 shutdown_connections 2024-09-17T05:30:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 --2- 172.21.15.122:0/1937090338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d200b49e0 0x7f5d200acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 --2- 172.21.15.122:0/1937090338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 0x7f5d2006f7b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 -- 172.21.15.122:0/1937090338 >> 172.21.15.122:0/1937090338 conn(0x7f5d200b18f0 msgr2=0x7f5d200b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 -- 172.21.15.122:0/1937090338 shutdown_connections 2024-09-17T05:30:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 -- 172.21.15.122:0/1937090338 wait complete. 2024-09-17T05:30:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 Processor -- start 2024-09-17T05:30:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 -- start start 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d200b49e0 0x7f5d20073170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 0x7f5d20158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d2015dae0 0x7f5d2014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d201674b0 con 0x7f5d200cd100 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20167330 con 0x7f5d200b49e0 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d26400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5d20167630 con 0x7f5d2015dae0 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 0x7f5d20158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 0x7f5d20158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60470/0 (socket says 172.21.15.122:60470) 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 -- 172.21.15.122:0/3551330813 learned_addr learned my addr 172.21.15.122:0/3551330813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1f400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d200b49e0 0x7f5d20073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1fe00640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d2015dae0 0x7f5d2014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 -- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d2015dae0 msgr2=0x7f5d2014f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d2015dae0 0x7f5d2014f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 -- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d200b49e0 msgr2=0x7f5d20073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d200b49e0 0x7f5d20073170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 -- 172.21.15.122:0/3551330813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d201f6430 con 0x7f5d200cd100 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1fe00640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d2015dae0 0x7f5d2014f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.150+0000 7f5d1ea00640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 0x7f5d20158dd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009790 tx=0x7f5d0c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d17400640 1 -- 172.21.15.122:0/3551330813 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c0bc020 con 0x7f5d200cd100 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d17400640 1 -- 172.21.15.122:0/3551330813 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d0c005030 con 0x7f5d200cd100 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d201f5820 con 0x7f5d200cd100 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d17400640 1 -- 172.21.15.122:0/3551330813 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c005460 con 0x7f5d200cd100 2024-09-17T05:30:27.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201f60c0 con 0x7f5d200cd100 2024-09-17T05:30:27.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d200acd40 con 0x7f5d200cd100 2024-09-17T05:30:27.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d17400640 1 -- 172.21.15.122:0/3551330813 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f5d0c0bf020 con 0x7f5d200cd100 2024-09-17T05:30:27.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d17400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d00073b30 0x7f5d00075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:27.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d17400640 1 -- 172.21.15.122:0/3551330813 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f5d0c12dad0 con 0x7f5d200cd100 2024-09-17T05:30:27.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.154+0000 7f5d1f400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d00073b30 0x7f5d00075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:27.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.158+0000 7f5d1f400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d00073b30 0x7f5d00075ff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d10004a40 tx=0x7f5d10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:27.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.158+0000 7f5d17400640 1 -- 172.21.15.122:0/3551330813 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5d0c0c6030 con 0x7f5d200cd100 2024-09-17T05:30:27.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.358+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5d200ad990 con 0x7f5d200cd100 2024-09-17T05:30:27.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.974+0000 7f5d17400640 1 -- 172.21.15.122:0/3551330813 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f5d0c016410 con 0x7f5d200cd100 2024-09-17T05:30:27.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d00073b30 msgr2=0x7f5d00075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d00073b30 0x7f5d00075ff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d10004a40 tx=0x7f5d10009040 comp rx=0 tx=0).stop 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 msgr2=0x7f5d20158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 0x7f5d20158dd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009790 tx=0x7f5d0c004040 comp rx=0 tx=0).stop 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 shutdown_connections 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d00073b30 0x7f5d00075ff0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d2015dae0 0x7f5d2014f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d200cd100 0x7f5d20158dd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 --2- 172.21.15.122:0/3551330813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d200b49e0 0x7f5d20073170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 >> 172.21.15.122:0/3551330813 conn(0x7f5d200b18f0 msgr2=0x7f5d200adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 shutdown_connections 2024-09-17T05:30:27.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:27.978+0000 7f5d26400640 1 -- 172.21.15.122:0/3551330813 wait complete. 2024-09-17T05:30:28.000 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:30:28.000 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- 172.21.15.122:0/532332373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd140cd100 msgr2=0x7fdd140acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 --2- 172.21.15.122:0/532332373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd140cd100 0x7fdd140acd40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009bc0 tx=0x7fdd0c01cb10 comp rx=0 tx=0).stop 2024-09-17T05:30:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- 172.21.15.122:0/532332373 shutdown_connections 2024-09-17T05:30:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 --2- 172.21.15.122:0/532332373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd140cd100 0x7fdd140acd40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- 172.21.15.122:0/532332373 >> 172.21.15.122:0/532332373 conn(0x7fdd140b18f0 msgr2=0x7fdd140b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- 172.21.15.122:0/532332373 shutdown_connections 2024-09-17T05:30:28.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- 172.21.15.122:0/532332373 wait complete. 2024-09-17T05:30:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 Processor -- start 2024-09-17T05:30:28.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- start start 2024-09-17T05:30:28.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd140b4a50 0x7fdd1415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd140cd100 0x7fdd1414b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd1415db50 0x7fdd1415c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdd14168940 con 0x7fdd1415db50 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd141687c0 con 0x7fdd140b4a50 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd19e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdd14168ac0 con 0x7fdd140cd100 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd13400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd1415db50 0x7fdd1415c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd13400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd1415db50 0x7fdd1415c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60496/0 (socket says 172.21.15.122:60496) 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.142+0000 7fdd13400640 1 -- 172.21.15.122:0/2769251248 learned_addr learned my addr 172.21.15.122:0/2769251248 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd12a00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd140b4a50 0x7fdd1415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd13400640 1 -- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd140cd100 msgr2=0x7fdd1414b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd13400640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd140cd100 0x7fdd1414b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd13400640 1 -- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd140b4a50 msgr2=0x7fdd1415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd13400640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd140b4a50 0x7fdd1415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd13400640 1 -- 172.21.15.122:0/2769251248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1414bce0 con 0x7fdd1415db50 2024-09-17T05:30:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd12a00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd140b4a50 0x7fdd1415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd13400640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd1415db50 0x7fdd1415c840 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009b10 tx=0x7fdd0c003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd0be00640 1 -- 172.21.15.122:0/2769251248 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0bb070 con 0x7fdd1415db50 2024-09-17T05:30:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd0be00640 1 -- 172.21.15.122:0/2769251248 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd0c004e00 con 0x7fdd1415db50 2024-09-17T05:30:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd0be00640 1 -- 172.21.15.122:0/2769251248 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0b2040 con 0x7fdd1415db50 2024-09-17T05:30:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1415b7a0 con 0x7fdd1415db50 2024-09-17T05:30:28.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1415bd60 con 0x7fdd1415db50 2024-09-17T05:30:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd140acd40 con 0x7fdd1415db50 2024-09-17T05:30:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.146+0000 7fdd0be00640 1 -- 172.21.15.122:0/2769251248 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fdd0c005250 con 0x7fdd1415db50 2024-09-17T05:30:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.150+0000 7fdd0be00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcf0073b00 0x7fdcf0075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.150+0000 7fdd0be00640 1 -- 172.21.15.122:0/2769251248 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fdd0c12c710 con 0x7fdd1415db50 2024-09-17T05:30:28.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.150+0000 7fdd12a00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcf0073b00 0x7fdcf0075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.150+0000 7fdd12a00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcf0073b00 0x7fdcf0075fc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdd04000f70 tx=0x7fdd04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.150+0000 7fdd0be00640 1 -- 172.21.15.122:0/2769251248 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdd0c016790 con 0x7fdd1415db50 2024-09-17T05:30:28.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd14158e60 con 0x7fdd1415db50 2024-09-17T05:30:28.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd0be00640 1 -- 172.21.15.122:0/2769251248 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5232 (secure 0 0 0) 0x7fdd0c0f9210 con 0x7fdd1415db50 2024-09-17T05:30:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:28.354 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":36,"btime":"2024-09-17T05:30:27:977715+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110418608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110418608},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110418608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1187468185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1187468185},{"type":"v1","addr":"172.21.15.122:6835","nonce":1187468185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3667127301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3667127301},{"type":"v1","addr":"172.21.15.122:6834","nonce":3667127301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:30:24.330680+0000","modified":"2024-09-17T05:30:27.362693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6836/204904815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":204904815},{"type":"v1","addr":"172.21.15.173:6836","nonce":204904815}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5931,"qdb_cluster":[5931]},"id":4}]} 2024-09-17T05:30:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 36 2024-09-17T05:30:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcf0073b00 msgr2=0x7fdcf0075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcf0073b00 0x7fdcf0075fc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdd04000f70 tx=0x7fdd04009040 comp rx=0 tx=0).stop 2024-09-17T05:30:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd1415db50 msgr2=0x7fdd1415c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd1415db50 0x7fdd1415c840 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009b10 tx=0x7fdd0c003b70 comp rx=0 tx=0).stop 2024-09-17T05:30:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 shutdown_connections 2024-09-17T05:30:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcf0073b00 0x7fdcf0075fc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd1415db50 0x7fdd1415c840 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd140cd100 0x7fdd1414b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 --2- 172.21.15.122:0/2769251248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd140b4a50 0x7fdd1415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 >> 172.21.15.122:0/2769251248 conn(0x7fdd140b18f0 msgr2=0x7fdd140b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 shutdown_connections 2024-09-17T05:30:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.354+0000 7fdd19e00640 1 -- 172.21.15.122:0/2769251248 wait complete. 2024-09-17T05:30:28.376 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:30:28.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 -- 172.21.15.122:0/2522996389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2c15f000 msgr2=0x7f6e2c0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 --2- 172.21.15.122:0/2522996389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2c15f000 0x7f6e2c0d01b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009c80 tx=0x7f6e1c01cd10 comp rx=0 tx=0).stop 2024-09-17T05:30:28.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 -- 172.21.15.122:0/2522996389 shutdown_connections 2024-09-17T05:30:28.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 --2- 172.21.15.122:0/2522996389 >> v2:172.21.15.173:3300/0 conn(0x7f6e2c0b2590 0x7f6e2c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 --2- 172.21.15.122:0/2522996389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2c15f000 0x7f6e2c0d01b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 -- 172.21.15.122:0/2522996389 >> 172.21.15.122:0/2522996389 conn(0x7f6e2c09aa10 msgr2=0x7f6e2c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 -- 172.21.15.122:0/2522996389 shutdown_connections 2024-09-17T05:30:28.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 -- 172.21.15.122:0/2522996389 wait complete. 2024-09-17T05:30:28.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 Processor -- start 2024-09-17T05:30:28.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.518+0000 7f6e2be00640 1 -- start start 2024-09-17T05:30:28.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e2c0b2590 0x7f6e2c0c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2c0cefb0 0x7f6e2c15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c15f000 0x7f6e2c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c168f00 con 0x7f6e2c15f000 2024-09-17T05:30:28.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c168d80 con 0x7f6e2c0b2590 2024-09-17T05:30:28.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2c169080 con 0x7f6e2c0cefb0 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c15f000 0x7f6e2c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e2c0b2590 0x7f6e2c0c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2a000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2c0cefb0 0x7f6e2c15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e2c0b2590 0x7f6e2c0c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:37872/0 (socket says 172.21.15.122:37872) 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2aa00640 1 -- 172.21.15.122:0/3371778972 learned_addr learned my addr 172.21.15.122:0/3371778972 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2b400640 1 -- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2c0cefb0 msgr2=0x7f6e2c15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2b400640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2c0cefb0 0x7f6e2c15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2b400640 1 -- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e2c0b2590 msgr2=0x7f6e2c0c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2b400640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e2c0b2590 0x7f6e2c0c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2b400640 1 -- 172.21.15.122:0/3371778972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c15da40 con 0x7f6e2c15f000 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2b400640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c15f000 0x7f6e2c0c6830 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400e970 tx=0x7f6e2400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e22a00640 1 -- 172.21.15.122:0/3371778972 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400ccc0 con 0x7f6e2c15f000 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e22a00640 1 -- 172.21.15.122:0/3371778972 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e24004020 con 0x7f6e2c15f000 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e22a00640 1 -- 172.21.15.122:0/3371778972 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24004470 con 0x7f6e2c15f000 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c1666b0 con 0x7f6e2c15f000 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1663d0 con 0x7f6e2c15f000 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.522+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c0b0920 con 0x7f6e2c15f000 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.526+0000 7f6e22a00640 1 -- 172.21.15.122:0/3371778972 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f6e24005020 con 0x7f6e2c15f000 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.526+0000 7f6e22a00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e0c073b00 0x7f6e0c075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.526+0000 7f6e22a00640 1 -- 172.21.15.122:0/3371778972 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f6e240934b0 con 0x7f6e2c15f000 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.526+0000 7f6e2aa00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e0c073b00 0x7f6e0c075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.526+0000 7f6e2aa00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e0c073b00 0x7f6e0c075fc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c01b3c0 tx=0x7f6e1c01b550 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.526+0000 7f6e22a00640 1 -- 172.21.15.122:0/3371778972 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6e2c1666b0 con 0x7f6e2c15f000 2024-09-17T05:30:28.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e2c0b29a0 con 0x7f6e2c15f000 2024-09-17T05:30:28.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e22a00640 1 -- 172.21.15.122:0/3371778972 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16828 (secure 0 0 0) 0x7f6e2c15da40 con 0x7f6e2c15f000 2024-09-17T05:30:28.698 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:28.699 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":60,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:30:25.949583+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:30:22.033029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:30:23.299476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e0c073b00 msgr2=0x7f6e0c075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e0c073b00 0x7f6e0c075fc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c01b3c0 tx=0x7f6e1c01b550 comp rx=0 tx=0).stop 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c15f000 msgr2=0x7f6e2c0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c15f000 0x7f6e2c0c6830 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400e970 tx=0x7f6e2400ee30 comp rx=0 tx=0).stop 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 shutdown_connections 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e0c073b00 0x7f6e0c075fc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2c15f000 0x7f6e2c0c6830 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2c0cefb0 0x7f6e2c15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 --2- 172.21.15.122:0/3371778972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e2c0b2590 0x7f6e2c0c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.698+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 >> 172.21.15.122:0/3371778972 conn(0x7f6e2c09aa10 msgr2=0x7f6e2c09adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.702+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 shutdown_connections 2024-09-17T05:30:28.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.702+0000 7f6e2be00640 1 -- 172.21.15.122:0/3371778972 wait complete. 2024-09-17T05:30:28.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:30:28.721 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:30:28.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f4800640 1 -- 172.21.15.122:0/4106071716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92d4004da0 msgr2=0x7f92d4005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f4800640 1 --2- 172.21.15.122:0/4106071716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92d4004da0 0x7f92d4005220 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f92e8001f50 tx=0x7f92e801cc40 comp rx=0 tx=0).stop 2024-09-17T05:30:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f2800640 1 -- 172.21.15.122:0/4106071716 reap_dead start 2024-09-17T05:30:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f4800640 1 -- 172.21.15.122:0/4106071716 shutdown_connections 2024-09-17T05:30:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f4800640 1 -- 172.21.15.122:0/4106071716 >> 172.21.15.122:0/4106071716 conn(0x7f92ec0d1da0 msgr2=0x7f92ec0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f4800640 1 -- 172.21.15.122:0/4106071716 shutdown_connections 2024-09-17T05:30:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f4800640 1 -- 172.21.15.122:0/4106071716 wait complete. 2024-09-17T05:30:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f4800640 1 Processor -- start 2024-09-17T05:30:28.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.862+0000 7f92f4800640 1 -- start start 2024-09-17T05:30:28.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d4004da0 0x7f92ec1f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92ec0cba70 0x7f92ec15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92ec15a690 0x7f92ec15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec0b65a0 con 0x7f92ec15a690 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec0b6420 con 0x7f92d4004da0 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec0b6720 con 0x7f92ec0cba70 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92ec15a690 0x7f92ec15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92ec15a690 0x7f92ec15e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60518/0 (socket says 172.21.15.122:60518) 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 -- 172.21.15.122:0/4035659493 learned_addr learned my addr 172.21.15.122:0/4035659493 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f1e00640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d4004da0 0x7f92ec1f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f1400640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92ec0cba70 0x7f92ec15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 -- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92ec0cba70 msgr2=0x7f92ec15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92ec0cba70 0x7f92ec15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 -- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d4004da0 msgr2=0x7f92ec1f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:28.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d4004da0 0x7f92ec1f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:28.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 -- 172.21.15.122:0/4035659493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec0d1210 con 0x7f92ec15a690 2024-09-17T05:30:28.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f2800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92ec15a690 0x7f92ec15e6d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f92e8001f50 tx=0x7f92e8001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92e6000640 1 -- 172.21.15.122:0/4035659493 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8001ba0 con 0x7f92ec15a690 2024-09-17T05:30:28.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92e6000640 1 -- 172.21.15.122:0/4035659493 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92e80b4030 con 0x7f92ec15a690 2024-09-17T05:30:28.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec0c9f60 con 0x7f92ec15a690 2024-09-17T05:30:28.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92e6000640 1 -- 172.21.15.122:0/4035659493 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8006c50 con 0x7f92ec15a690 2024-09-17T05:30:28.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec1f3040 con 0x7f92ec15a690 2024-09-17T05:30:28.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec0ca130 con 0x7f92ec15a690 2024-09-17T05:30:28.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.866+0000 7f92e6000640 1 -- 172.21.15.122:0/4035659493 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f92e80066e0 con 0x7f92ec15a690 2024-09-17T05:30:28.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.870+0000 7f92e6000640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f92d0073b00 0x7f92d0075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:28.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.870+0000 7f92e6000640 1 -- 172.21.15.122:0/4035659493 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f92e812f680 con 0x7f92ec15a690 2024-09-17T05:30:28.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.870+0000 7f92f1e00640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f92d0073b00 0x7f92d0075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:28.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.870+0000 7f92f1e00640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f92d0073b00 0x7f92d0075fc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009b60 tx=0x7f92e00091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:28.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:28.874+0000 7f92e6000640 1 -- 172.21.15.122:0/4035659493 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f92e80169f0 con 0x7f92ec15a690 2024-09-17T05:30:29.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.038+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92ec0ce890 con 0x7f92d0073b00 2024-09-17T05:30:29.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.046+0000 7f92e6000640 1 -- 172.21.15.122:0/4035659493 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247562 (secure 0 0 0) 0x7f92ec0ce890 con 0x7f92d0073b00 2024-09-17T05:30:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:29.049 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:30:29.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f92d0073b00 msgr2=0x7f92d0075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f92d0073b00 0x7f92d0075fc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009b60 tx=0x7f92e00091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92ec15a690 msgr2=0x7f92ec15e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92ec15a690 0x7f92ec15e6d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f92e8001f50 tx=0x7f92e8001940 comp rx=0 tx=0).stop 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 shutdown_connections 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f92d0073b00 0x7f92d0075fc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92ec15a690 0x7f92ec15e6d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92ec0cba70 0x7f92ec15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 --2- 172.21.15.122:0/4035659493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d4004da0 0x7f92ec1f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 >> 172.21.15.122:0/4035659493 conn(0x7f92ec0d1da0 msgr2=0x7f92ec071510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 shutdown_connections 2024-09-17T05:30:29.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.050+0000 7f92f4800640 1 -- 172.21.15.122:0/4035659493 wait complete. 2024-09-17T05:30:29.069 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-17T05:30:28.533444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590402,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351760,"kb_used_data":6480,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754622960,"statfs":{"total":773094113280,"available":772733911040,"internally_reserved":0,"allocated":6635520,"data_stored":3427636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":353357366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.601333"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946789+0000","last_change":"2024-09-17T05:30:23.946789+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946789+0000","last_undegraded":"2024-09-17T05:30:23.946789+0000","last_fullsized":"2024-09-17T05:30:23.946789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935503+0000","last_change":"2024-09-17T05:30:23.935503+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935503+0000","last_undegraded":"2024-09-17T05:30:23.935503+0000","last_fullsized":"2024-09-17T05:30:23.935503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937682+0000","last_change":"2024-09-17T05:30:23.937682+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937682+0000","last_undegraded":"2024-09-17T05:30:23.937682+0000","last_fullsized":"2024-09-17T05:30:23.937682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932092+0000","last_change":"2024-09-17T05:30:23.932092+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932092+0000","last_undegraded":"2024-09-17T05:30:23.932092+0000","last_fullsized":"2024-09-17T05:30:23.932092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937192+0000","last_change":"2024-09-17T05:30:23.937192+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937192+0000","last_undegraded":"2024-09-17T05:30:23.937192+0000","last_fullsized":"2024-09-17T05:30:23.937192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941621+0000","last_change":"2024-09-17T05:30:23.941621+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941621+0000","last_undegraded":"2024-09-17T05:30:23.941621+0000","last_fullsized":"2024-09-17T05:30:23.941621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932950+0000","last_change":"2024-09-17T05:30:23.932950+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932950+0000","last_undegraded":"2024-09-17T05:30:23.932950+0000","last_fullsized":"2024-09-17T05:30:23.932950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864357+0000","last_change":"2024-09-17T05:30:24.969090+0000","last_active":"2024-09-17T05:30:27.864357+0000","last_peered":"2024-09-17T05:30:27.864357+0000","last_clean":"2024-09-17T05:30:27.864357+0000","last_became_active":"2024-09-17T05:30:24.968736+0000","last_became_peered":"2024-09-17T05:30:24.968736+0000","last_unstale":"2024-09-17T05:30:27.864357+0000","last_undegraded":"2024-09-17T05:30:27.864357+0000","last_fullsized":"2024-09-17T05:30:27.864357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:13.357280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935531+0000","last_change":"2024-09-17T05:30:23.935531+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935531+0000","last_undegraded":"2024-09-17T05:30:23.935531+0000","last_fullsized":"2024-09-17T05:30:23.935531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946112+0000","last_change":"2024-09-17T05:30:23.946112+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946112+0000","last_undegraded":"2024-09-17T05:30:23.946112+0000","last_fullsized":"2024-09-17T05:30:23.946112+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939489+0000","last_change":"2024-09-17T05:30:23.939489+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939489+0000","last_undegraded":"2024-09-17T05:30:23.939489+0000","last_fullsized":"2024-09-17T05:30:23.939489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935459+0000","last_change":"2024-09-17T05:30:23.935459+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935459+0000","last_undegraded":"2024-09-17T05:30:23.935459+0000","last_fullsized":"2024-09-17T05:30:23.935459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.943180+0000","last_change":"2024-09-17T05:30:23.943180+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.943180+0000","last_undegraded":"2024-09-17T05:30:23.943180+0000","last_fullsized":"2024-09-17T05:30:23.943180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934423+0000","last_change":"2024-09-17T05:30:23.934423+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934423+0000","last_undegraded":"2024-09-17T05:30:23.934423+0000","last_fullsized":"2024-09-17T05:30:23.934423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941670+0000","last_change":"2024-09-17T05:30:23.941670+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941670+0000","last_undegraded":"2024-09-17T05:30:23.941670+0000","last_fullsized":"2024-09-17T05:30:23.941670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937557+0000","last_change":"2024-09-17T05:30:23.937557+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937557+0000","last_undegraded":"2024-09-17T05:30:23.937557+0000","last_fullsized":"2024-09-17T05:30:23.937557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946333+0000","last_change":"2024-09-17T05:30:23.946333+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946333+0000","last_undegraded":"2024-09-17T05:30:23.946333+0000","last_fullsized":"2024-09-17T05:30:23.946333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938746+0000","last_change":"2024-09-17T05:30:23.938746+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938746+0000","last_undegraded":"2024-09-17T05:30:23.938746+0000","last_fullsized":"2024-09-17T05:30:23.938746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936163+0000","last_change":"2024-09-17T05:30:23.936163+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936163+0000","last_undegraded":"2024-09-17T05:30:23.936163+0000","last_fullsized":"2024-09-17T05:30:23.936163+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939976+0000","last_change":"2024-09-17T05:30:23.939976+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939976+0000","last_undegraded":"2024-09-17T05:30:23.939976+0000","last_fullsized":"2024-09-17T05:30:23.939976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.942782+0000","last_change":"2024-09-17T05:30:23.942782+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.942782+0000","last_undegraded":"2024-09-17T05:30:23.942782+0000","last_fullsized":"2024-09-17T05:30:23.942782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938893+0000","last_change":"2024-09-17T05:30:23.938893+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938893+0000","last_undegraded":"2024-09-17T05:30:23.938893+0000","last_fullsized":"2024-09-17T05:30:23.938893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932460+0000","last_change":"2024-09-17T05:30:23.932460+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932460+0000","last_undegraded":"2024-09-17T05:30:23.932460+0000","last_fullsized":"2024-09-17T05:30:23.932460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936101+0000","last_change":"2024-09-17T05:30:23.936101+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936101+0000","last_undegraded":"2024-09-17T05:30:23.936101+0000","last_fullsized":"2024-09-17T05:30:23.936101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935649+0000","last_change":"2024-09-17T05:30:23.935649+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935649+0000","last_undegraded":"2024-09-17T05:30:23.935649+0000","last_fullsized":"2024-09-17T05:30:23.935649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941579+0000","last_change":"2024-09-17T05:30:23.941579+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941579+0000","last_undegraded":"2024-09-17T05:30:23.941579+0000","last_fullsized":"2024-09-17T05:30:23.941579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935721+0000","last_change":"2024-09-17T05:30:23.935721+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935721+0000","last_undegraded":"2024-09-17T05:30:23.935721+0000","last_fullsized":"2024-09-17T05:30:23.935721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864500+0000","last_change":"2024-09-17T05:30:24.976244+0000","last_active":"2024-09-17T05:30:27.864500+0000","last_peered":"2024-09-17T05:30:27.864500+0000","last_clean":"2024-09-17T05:30:27.864500+0000","last_became_active":"2024-09-17T05:30:24.975619+0000","last_became_peered":"2024-09-17T05:30:24.975619+0000","last_unstale":"2024-09-17T05:30:27.864500+0000","last_undegraded":"2024-09-17T05:30:27.864500+0000","last_fullsized":"2024-09-17T05:30:27.864500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:42.656824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864466+0000","last_change":"2024-09-17T05:30:24.972218+0000","last_active":"2024-09-17T05:30:27.864466+0000","last_peered":"2024-09-17T05:30:27.864466+0000","last_clean":"2024-09-17T05:30:27.864466+0000","last_became_active":"2024-09-17T05:30:24.969583+0000","last_became_peered":"2024-09-17T05:30:24.969583+0000","last_unstale":"2024-09-17T05:30:27.864466+0000","last_undegraded":"2024-09-17T05:30:27.864466+0000","last_fullsized":"2024-09-17T05:30:27.864466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:45.665750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931376+0000","last_change":"2024-09-17T05:30:23.931376+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931376+0000","last_undegraded":"2024-09-17T05:30:23.931376+0000","last_fullsized":"2024-09-17T05:30:23.931376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936553+0000","last_change":"2024-09-17T05:30:23.936553+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936553+0000","last_undegraded":"2024-09-17T05:30:23.936553+0000","last_fullsized":"2024-09-17T05:30:23.936553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.940234+0000","last_change":"2024-09-17T05:30:23.940234+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.940234+0000","last_undegraded":"2024-09-17T05:30:23.940234+0000","last_fullsized":"2024-09-17T05:30:23.940234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.945686+0000","last_change":"2024-09-17T05:30:23.945686+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.945686+0000","last_undegraded":"2024-09-17T05:30:23.945686+0000","last_fullsized":"2024-09-17T05:30:23.945686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.955314+0000","last_change":"2024-09-17T05:30:23.950637+0000","last_active":"2024-09-17T05:30:23.955314+0000","last_peered":"2024-09-17T05:30:23.955314+0000","last_clean":"2024-09-17T05:30:23.955314+0000","last_became_active":"2024-09-17T05:30:23.950363+0000","last_became_peered":"2024-09-17T05:30:23.950363+0000","last_unstale":"2024-09-17T05:30:23.955314+0000","last_undegraded":"2024-09-17T05:30:23.955314+0000","last_fullsized":"2024-09-17T05:30:23.955314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:01.009438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.945807+0000","last_change":"2024-09-17T05:30:23.945807+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.945807+0000","last_undegraded":"2024-09-17T05:30:23.945807+0000","last_fullsized":"2024-09-17T05:30:23.945807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956419+0000","last_change":"2024-09-17T05:30:23.952541+0000","last_active":"2024-09-17T05:30:23.956419+0000","last_peered":"2024-09-17T05:30:23.956419+0000","last_clean":"2024-09-17T05:30:23.956419+0000","last_became_active":"2024-09-17T05:30:23.952190+0000","last_became_peered":"2024-09-17T05:30:23.952190+0000","last_unstale":"2024-09-17T05:30:23.956419+0000","last_undegraded":"2024-09-17T05:30:23.956419+0000","last_fullsized":"2024-09-17T05:30:23.956419+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:49.775010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864584+0000","last_change":"2024-09-17T05:30:24.976969+0000","last_active":"2024-09-17T05:30:27.864584+0000","last_peered":"2024-09-17T05:30:27.864584+0000","last_clean":"2024-09-17T05:30:27.864584+0000","last_became_active":"2024-09-17T05:30:24.976702+0000","last_became_peered":"2024-09-17T05:30:24.976702+0000","last_unstale":"2024-09-17T05:30:27.864584+0000","last_undegraded":"2024-09-17T05:30:27.864584+0000","last_fullsized":"2024-09-17T05:30:27.864584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:29.194444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.959443+0000","last_change":"2024-09-17T05:30:23.958614+0000","last_active":"2024-09-17T05:30:23.959443+0000","last_peered":"2024-09-17T05:30:23.959443+0000","last_clean":"2024-09-17T05:30:23.959443+0000","last_became_active":"2024-09-17T05:30:23.958200+0000","last_became_peered":"2024-09-17T05:30:23.958200+0000","last_unstale":"2024-09-17T05:30:23.959443+0000","last_undegraded":"2024-09-17T05:30:23.959443+0000","last_fullsized":"2024-09-17T05:30:23.959443+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:07.976069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.943960+0000","last_change":"2024-09-17T05:30:23.943960+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.943960+0000","last_undegraded":"2024-09-17T05:30:23.943960+0000","last_fullsized":"2024-09-17T05:30:23.943960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949074+0000","last_change":"2024-09-17T05:30:23.940517+0000","last_active":"2024-09-17T05:30:23.949074+0000","last_peered":"2024-09-17T05:30:23.949074+0000","last_clean":"2024-09-17T05:30:23.949074+0000","last_became_active":"2024-09-17T05:30:23.940255+0000","last_became_peered":"2024-09-17T05:30:23.940255+0000","last_unstale":"2024-09-17T05:30:23.949074+0000","last_undegraded":"2024-09-17T05:30:23.949074+0000","last_fullsized":"2024-09-17T05:30:23.949074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:47.160401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864815+0000","last_change":"2024-09-17T05:30:24.974140+0000","last_active":"2024-09-17T05:30:27.864815+0000","last_peered":"2024-09-17T05:30:27.864815+0000","last_clean":"2024-09-17T05:30:27.864815+0000","last_became_active":"2024-09-17T05:30:24.973833+0000","last_became_peered":"2024-09-17T05:30:24.973833+0000","last_unstale":"2024-09-17T05:30:27.864815+0000","last_undegraded":"2024-09-17T05:30:27.864815+0000","last_fullsized":"2024-09-17T05:30:27.864815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:28.597753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952758+0000","last_change":"2024-09-17T05:30:23.946652+0000","last_active":"2024-09-17T05:30:23.952758+0000","last_peered":"2024-09-17T05:30:23.952758+0000","last_clean":"2024-09-17T05:30:23.952758+0000","last_became_active":"2024-09-17T05:30:23.946243+0000","last_became_peered":"2024-09-17T05:30:23.946243+0000","last_unstale":"2024-09-17T05:30:23.952758+0000","last_undegraded":"2024-09-17T05:30:23.952758+0000","last_fullsized":"2024-09-17T05:30:23.952758+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:55.869955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939243+0000","last_change":"2024-09-17T05:30:23.939243+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939243+0000","last_undegraded":"2024-09-17T05:30:23.939243+0000","last_fullsized":"2024-09-17T05:30:23.939243+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.958482+0000","last_change":"2024-09-17T05:30:23.953938+0000","last_active":"2024-09-17T05:30:23.958482+0000","last_peered":"2024-09-17T05:30:23.958482+0000","last_clean":"2024-09-17T05:30:23.958482+0000","last_became_active":"2024-09-17T05:30:23.953683+0000","last_became_peered":"2024-09-17T05:30:23.953683+0000","last_unstale":"2024-09-17T05:30:23.958482+0000","last_undegraded":"2024-09-17T05:30:23.958482+0000","last_fullsized":"2024-09-17T05:30:23.958482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:44.089365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934227+0000","last_change":"2024-09-17T05:30:23.934227+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934227+0000","last_undegraded":"2024-09-17T05:30:23.934227+0000","last_fullsized":"2024-09-17T05:30:23.934227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.951311+0000","last_change":"2024-09-17T05:30:23.948325+0000","last_active":"2024-09-17T05:30:23.951311+0000","last_peered":"2024-09-17T05:30:23.951311+0000","last_clean":"2024-09-17T05:30:23.951311+0000","last_became_active":"2024-09-17T05:30:23.948001+0000","last_became_peered":"2024-09-17T05:30:23.948001+0000","last_unstale":"2024-09-17T05:30:23.951311+0000","last_undegraded":"2024-09-17T05:30:23.951311+0000","last_fullsized":"2024-09-17T05:30:23.951311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:07.032087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937367+0000","last_change":"2024-09-17T05:30:23.937367+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937367+0000","last_undegraded":"2024-09-17T05:30:23.937367+0000","last_fullsized":"2024-09-17T05:30:23.937367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956908+0000","last_change":"2024-09-17T05:30:23.954282+0000","last_active":"2024-09-17T05:30:23.956908+0000","last_peered":"2024-09-17T05:30:23.956908+0000","last_clean":"2024-09-17T05:30:23.956908+0000","last_became_active":"2024-09-17T05:30:23.953881+0000","last_became_peered":"2024-09-17T05:30:23.953881+0000","last_unstale":"2024-09-17T05:30:23.956908+0000","last_undegraded":"2024-09-17T05:30:23.956908+0000","last_fullsized":"2024-09-17T05:30:23.956908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:55.869955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.940305+0000","last_change":"2024-09-17T05:30:23.940305+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.940305+0000","last_undegraded":"2024-09-17T05:30:23.940305+0000","last_fullsized":"2024-09-17T05:30:23.940305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.946897+0000","last_change":"2024-09-17T05:30:23.943335+0000","last_active":"2024-09-17T05:30:23.946897+0000","last_peered":"2024-09-17T05:30:23.946897+0000","last_clean":"2024-09-17T05:30:23.946897+0000","last_became_active":"2024-09-17T05:30:23.943018+0000","last_became_peered":"2024-09-17T05:30:23.943018+0000","last_unstale":"2024-09-17T05:30:23.946897+0000","last_undegraded":"2024-09-17T05:30:23.946897+0000","last_fullsized":"2024-09-17T05:30:23.946897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:45.720384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937983+0000","last_change":"2024-09-17T05:30:23.937983+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937983+0000","last_undegraded":"2024-09-17T05:30:23.937983+0000","last_fullsized":"2024-09-17T05:30:23.937983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954980+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954714+0000","last_became_peered":"2024-09-17T05:30:23.954714+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:24.175086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864850+0000","last_change":"2024-09-17T05:30:24.976889+0000","last_active":"2024-09-17T05:30:27.864850+0000","last_peered":"2024-09-17T05:30:27.864850+0000","last_clean":"2024-09-17T05:30:27.864850+0000","last_became_active":"2024-09-17T05:30:24.976636+0000","last_became_peered":"2024-09-17T05:30:24.976636+0000","last_unstale":"2024-09-17T05:30:27.864850+0000","last_undegraded":"2024-09-17T05:30:27.864850+0000","last_fullsized":"2024-09-17T05:30:27.864850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:54.276295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.948946+0000","last_change":"2024-09-17T05:30:23.942665+0000","last_active":"2024-09-17T05:30:23.948946+0000","last_peered":"2024-09-17T05:30:23.948946+0000","last_clean":"2024-09-17T05:30:23.948946+0000","last_became_active":"2024-09-17T05:30:23.942377+0000","last_became_peered":"2024-09-17T05:30:23.942377+0000","last_unstale":"2024-09-17T05:30:23.948946+0000","last_undegraded":"2024-09-17T05:30:23.948946+0000","last_fullsized":"2024-09-17T05:30:23.948946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:35.760533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932249+0000","last_change":"2024-09-17T05:30:23.932249+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932249+0000","last_undegraded":"2024-09-17T05:30:23.932249+0000","last_fullsized":"2024-09-17T05:30:23.932249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956867+0000","last_change":"2024-09-17T05:30:23.953324+0000","last_active":"2024-09-17T05:30:23.956867+0000","last_peered":"2024-09-17T05:30:23.956867+0000","last_clean":"2024-09-17T05:30:23.956867+0000","last_became_active":"2024-09-17T05:30:23.953049+0000","last_became_peered":"2024-09-17T05:30:23.953049+0000","last_unstale":"2024-09-17T05:30:23.956867+0000","last_undegraded":"2024-09-17T05:30:23.956867+0000","last_fullsized":"2024-09-17T05:30:23.956867+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:58.939352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935005+0000","last_change":"2024-09-17T05:30:23.935005+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935005+0000","last_undegraded":"2024-09-17T05:30:23.935005+0000","last_fullsized":"2024-09-17T05:30:23.935005+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954792+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954435+0000","last_became_peered":"2024-09-17T05:30:23.954435+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:30.663318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934544+0000","last_change":"2024-09-17T05:30:23.934544+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934544+0000","last_undegraded":"2024-09-17T05:30:23.934544+0000","last_fullsized":"2024-09-17T05:30:23.934544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949916+0000","last_change":"2024-09-17T05:30:23.948409+0000","last_active":"2024-09-17T05:30:23.949916+0000","last_peered":"2024-09-17T05:30:23.949916+0000","last_clean":"2024-09-17T05:30:23.949916+0000","last_became_active":"2024-09-17T05:30:23.948114+0000","last_became_peered":"2024-09-17T05:30:23.948114+0000","last_unstale":"2024-09-17T05:30:23.949916+0000","last_undegraded":"2024-09-17T05:30:23.949916+0000","last_fullsized":"2024-09-17T05:30:23.949916+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:53.478330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934022+0000","last_change":"2024-09-17T05:30:23.934022+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934022+0000","last_undegraded":"2024-09-17T05:30:23.934022+0000","last_fullsized":"2024-09-17T05:30:23.934022+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.948904+0000","last_change":"2024-09-17T05:30:23.943942+0000","last_active":"2024-09-17T05:30:23.948904+0000","last_peered":"2024-09-17T05:30:23.948904+0000","last_clean":"2024-09-17T05:30:23.948904+0000","last_became_active":"2024-09-17T05:30:23.943669+0000","last_became_peered":"2024-09-17T05:30:23.943669+0000","last_unstale":"2024-09-17T05:30:23.948904+0000","last_undegraded":"2024-09-17T05:30:23.948904+0000","last_fullsized":"2024-09-17T05:30:23.948904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:38.122369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931093+0000","last_change":"2024-09-17T05:30:23.931093+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931093+0000","last_undegraded":"2024-09-17T05:30:23.931093+0000","last_fullsized":"2024-09-17T05:30:23.931093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949792+0000","last_change":"2024-09-17T05:30:23.949103+0000","last_active":"2024-09-17T05:30:23.949792+0000","last_peered":"2024-09-17T05:30:23.949792+0000","last_clean":"2024-09-17T05:30:23.949792+0000","last_became_active":"2024-09-17T05:30:23.948810+0000","last_became_peered":"2024-09-17T05:30:23.948810+0000","last_unstale":"2024-09-17T05:30:23.949792+0000","last_undegraded":"2024-09-17T05:30:23.949792+0000","last_fullsized":"2024-09-17T05:30:23.949792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:45.173108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938331+0000","last_change":"2024-09-17T05:30:23.938331+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938331+0000","last_undegraded":"2024-09-17T05:30:23.938331+0000","last_fullsized":"2024-09-17T05:30:23.938331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.961778+0000","last_change":"2024-09-17T05:30:23.960591+0000","last_active":"2024-09-17T05:30:23.961778+0000","last_peered":"2024-09-17T05:30:23.961778+0000","last_clean":"2024-09-17T05:30:23.961778+0000","last_became_active":"2024-09-17T05:30:23.960068+0000","last_became_peered":"2024-09-17T05:30:23.960068+0000","last_unstale":"2024-09-17T05:30:23.961778+0000","last_undegraded":"2024-09-17T05:30:23.961778+0000","last_fullsized":"2024-09-17T05:30:23.961778+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:32.536820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937514+0000","last_change":"2024-09-17T05:30:23.937514+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937514+0000","last_undegraded":"2024-09-17T05:30:23.937514+0000","last_fullsized":"2024-09-17T05:30:23.937514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864645+0000","last_change":"2024-09-17T05:30:23.951223+0000","last_active":"2024-09-17T05:30:27.864645+0000","last_peered":"2024-09-17T05:30:27.864645+0000","last_clean":"2024-09-17T05:30:27.864645+0000","last_became_active":"2024-09-17T05:30:23.950514+0000","last_became_peered":"2024-09-17T05:30:23.950514+0000","last_unstale":"2024-09-17T05:30:27.864645+0000","last_undegraded":"2024-09-17T05:30:27.864645+0000","last_fullsized":"2024-09-17T05:30:27.864645+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:45.736482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936819+0000","last_change":"2024-09-17T05:30:23.936819+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936819+0000","last_undegraded":"2024-09-17T05:30:23.936819+0000","last_fullsized":"2024-09-17T05:30:23.936819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952033+0000","last_change":"2024-09-17T05:30:23.944667+0000","last_active":"2024-09-17T05:30:23.952033+0000","last_peered":"2024-09-17T05:30:23.952033+0000","last_clean":"2024-09-17T05:30:23.952033+0000","last_became_active":"2024-09-17T05:30:23.944381+0000","last_became_peered":"2024-09-17T05:30:23.944381+0000","last_unstale":"2024-09-17T05:30:23.952033+0000","last_undegraded":"2024-09-17T05:30:23.952033+0000","last_fullsized":"2024-09-17T05:30:23.952033+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:27.536358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931506+0000","last_change":"2024-09-17T05:30:23.931506+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931506+0000","last_undegraded":"2024-09-17T05:30:23.931506+0000","last_fullsized":"2024-09-17T05:30:23.931506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864732+0000","last_change":"2024-09-17T05:30:23.960543+0000","last_active":"2024-09-17T05:30:27.864732+0000","last_peered":"2024-09-17T05:30:27.864732+0000","last_clean":"2024-09-17T05:30:27.864732+0000","last_became_active":"2024-09-17T05:30:23.960241+0000","last_became_peered":"2024-09-17T05:30:23.960241+0000","last_unstale":"2024-09-17T05:30:27.864732+0000","last_undegraded":"2024-09-17T05:30:27.864732+0000","last_fullsized":"2024-09-17T05:30:27.864732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:25.924845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934516+0000","last_change":"2024-09-17T05:30:23.934516+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934516+0000","last_undegraded":"2024-09-17T05:30:23.934516+0000","last_fullsized":"2024-09-17T05:30:23.934516+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.955243+0000","last_change":"2024-09-17T05:30:23.951379+0000","last_active":"2024-09-17T05:30:23.955243+0000","last_peered":"2024-09-17T05:30:23.955243+0000","last_clean":"2024-09-17T05:30:23.955243+0000","last_became_active":"2024-09-17T05:30:23.950900+0000","last_became_peered":"2024-09-17T05:30:23.950900+0000","last_unstale":"2024-09-17T05:30:23.955243+0000","last_undegraded":"2024-09-17T05:30:23.955243+0000","last_fullsized":"2024-09-17T05:30:23.955243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:54.643188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864503+0000","last_change":"2024-09-17T05:30:24.973721+0000","last_active":"2024-09-17T05:30:27.864503+0000","last_peered":"2024-09-17T05:30:27.864503+0000","last_clean":"2024-09-17T05:30:27.864503+0000","last_became_active":"2024-09-17T05:30:24.973391+0000","last_became_peered":"2024-09-17T05:30:24.973391+0000","last_unstale":"2024-09-17T05:30:27.864503+0000","last_undegraded":"2024-09-17T05:30:27.864503+0000","last_fullsized":"2024-09-17T05:30:27.864503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:02.895238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864244+0000","last_change":"2024-09-17T05:30:23.950654+0000","last_active":"2024-09-17T05:30:27.864244+0000","last_peered":"2024-09-17T05:30:27.864244+0000","last_clean":"2024-09-17T05:30:27.864244+0000","last_became_active":"2024-09-17T05:30:23.950372+0000","last_became_peered":"2024-09-17T05:30:23.950372+0000","last_unstale":"2024-09-17T05:30:27.864244+0000","last_undegraded":"2024-09-17T05:30:27.864244+0000","last_fullsized":"2024-09-17T05:30:27.864244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:46.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864458+0000","last_change":"2024-09-17T05:30:24.964323+0000","last_active":"2024-09-17T05:30:27.864458+0000","last_peered":"2024-09-17T05:30:27.864458+0000","last_clean":"2024-09-17T05:30:27.864458+0000","last_became_active":"2024-09-17T05:30:24.964039+0000","last_became_peered":"2024-09-17T05:30:24.964039+0000","last_unstale":"2024-09-17T05:30:27.864458+0000","last_undegraded":"2024-09-17T05:30:27.864458+0000","last_fullsized":"2024-09-17T05:30:27.864458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:52.710209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.951246+0000","last_change":"2024-09-17T05:30:23.946216+0000","last_active":"2024-09-17T05:30:23.951246+0000","last_peered":"2024-09-17T05:30:23.951246+0000","last_clean":"2024-09-17T05:30:23.951246+0000","last_became_active":"2024-09-17T05:30:23.945797+0000","last_became_peered":"2024-09-17T05:30:23.945797+0000","last_unstale":"2024-09-17T05:30:23.951246+0000","last_undegraded":"2024-09-17T05:30:23.951246+0000","last_fullsized":"2024-09-17T05:30:23.951246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:16.565367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937877+0000","last_change":"2024-09-17T05:30:23.937877+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937877+0000","last_undegraded":"2024-09-17T05:30:23.937877+0000","last_fullsized":"2024-09-17T05:30:23.937877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.957639+0000","last_change":"2024-09-17T05:30:23.953693+0000","last_active":"2024-09-17T05:30:23.957639+0000","last_peered":"2024-09-17T05:30:23.957639+0000","last_clean":"2024-09-17T05:30:23.957639+0000","last_became_active":"2024-09-17T05:30:23.953424+0000","last_became_peered":"2024-09-17T05:30:23.953424+0000","last_unstale":"2024-09-17T05:30:23.957639+0000","last_undegraded":"2024-09-17T05:30:23.957639+0000","last_fullsized":"2024-09-17T05:30:23.957639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:19.341040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937107+0000","last_change":"2024-09-17T05:30:23.937107+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937107+0000","last_undegraded":"2024-09-17T05:30:23.937107+0000","last_fullsized":"2024-09-17T05:30:23.937107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.959458+0000","last_change":"2024-09-17T05:30:23.958649+0000","last_active":"2024-09-17T05:30:23.959458+0000","last_peered":"2024-09-17T05:30:23.959458+0000","last_clean":"2024-09-17T05:30:23.959458+0000","last_became_active":"2024-09-17T05:30:23.958329+0000","last_became_peered":"2024-09-17T05:30:23.958329+0000","last_unstale":"2024-09-17T05:30:23.959458+0000","last_undegraded":"2024-09-17T05:30:23.959458+0000","last_fullsized":"2024-09-17T05:30:23.959458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.772225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935536+0000","last_change":"2024-09-17T05:30:23.935536+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935536+0000","last_undegraded":"2024-09-17T05:30:23.935536+0000","last_fullsized":"2024-09-17T05:30:23.935536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.950069+0000","last_change":"2024-09-17T05:30:23.943837+0000","last_active":"2024-09-17T05:30:23.950069+0000","last_peered":"2024-09-17T05:30:23.950069+0000","last_clean":"2024-09-17T05:30:23.950069+0000","last_became_active":"2024-09-17T05:30:23.942677+0000","last_became_peered":"2024-09-17T05:30:23.942677+0000","last_unstale":"2024-09-17T05:30:23.950069+0000","last_undegraded":"2024-09-17T05:30:23.950069+0000","last_fullsized":"2024-09-17T05:30:23.950069+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:26.543985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864917+0000","last_change":"2024-09-17T05:30:24.974072+0000","last_active":"2024-09-17T05:30:27.864917+0000","last_peered":"2024-09-17T05:30:27.864917+0000","last_clean":"2024-09-17T05:30:27.864917+0000","last_became_active":"2024-09-17T05:30:24.973224+0000","last_became_peered":"2024-09-17T05:30:24.973224+0000","last_unstale":"2024-09-17T05:30:27.864917+0000","last_undegraded":"2024-09-17T05:30:27.864917+0000","last_fullsized":"2024-09-17T05:30:27.864917+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:35.858572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.953004+0000","last_change":"2024-09-17T05:30:23.947956+0000","last_active":"2024-09-17T05:30:23.953004+0000","last_peered":"2024-09-17T05:30:23.953004+0000","last_clean":"2024-09-17T05:30:23.953004+0000","last_became_active":"2024-09-17T05:30:23.947641+0000","last_became_peered":"2024-09-17T05:30:23.947641+0000","last_unstale":"2024-09-17T05:30:23.953004+0000","last_undegraded":"2024-09-17T05:30:23.953004+0000","last_fullsized":"2024-09-17T05:30:23.953004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:41.752305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937632+0000","last_change":"2024-09-17T05:30:23.937632+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937632+0000","last_undegraded":"2024-09-17T05:30:23.937632+0000","last_fullsized":"2024-09-17T05:30:23.937632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.957936+0000","last_change":"2024-09-17T05:30:23.957320+0000","last_active":"2024-09-17T05:30:23.957936+0000","last_peered":"2024-09-17T05:30:23.957936+0000","last_clean":"2024-09-17T05:30:23.957936+0000","last_became_active":"2024-09-17T05:30:23.956956+0000","last_became_peered":"2024-09-17T05:30:23.956956+0000","last_unstale":"2024-09-17T05:30:23.957936+0000","last_undegraded":"2024-09-17T05:30:23.957936+0000","last_fullsized":"2024-09-17T05:30:23.957936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:52.455342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.942385+0000","last_change":"2024-09-17T05:30:23.942385+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.942385+0000","last_undegraded":"2024-09-17T05:30:23.942385+0000","last_fullsized":"2024-09-17T05:30:23.942385+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952593+0000","last_change":"2024-09-17T05:30:23.945568+0000","last_active":"2024-09-17T05:30:23.952593+0000","last_peered":"2024-09-17T05:30:23.952593+0000","last_clean":"2024-09-17T05:30:23.952593+0000","last_became_active":"2024-09-17T05:30:23.945279+0000","last_became_peered":"2024-09-17T05:30:23.945279+0000","last_unstale":"2024-09-17T05:30:23.952593+0000","last_undegraded":"2024-09-17T05:30:23.952593+0000","last_fullsized":"2024-09-17T05:30:23.952593+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:58.939352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864556+0000","last_change":"2024-09-17T05:30:24.974524+0000","last_active":"2024-09-17T05:30:27.864556+0000","last_peered":"2024-09-17T05:30:27.864556+0000","last_clean":"2024-09-17T05:30:27.864556+0000","last_became_active":"2024-09-17T05:30:24.974227+0000","last_became_peered":"2024-09-17T05:30:24.974227+0000","last_unstale":"2024-09-17T05:30:27.864556+0000","last_undegraded":"2024-09-17T05:30:27.864556+0000","last_fullsized":"2024-09-17T05:30:27.864556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:47.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.961936+0000","last_change":"2024-09-17T05:30:23.960348+0000","last_active":"2024-09-17T05:30:23.961936+0000","last_peered":"2024-09-17T05:30:23.961936+0000","last_clean":"2024-09-17T05:30:23.961936+0000","last_became_active":"2024-09-17T05:30:23.960056+0000","last_became_peered":"2024-09-17T05:30:23.960056+0000","last_unstale":"2024-09-17T05:30:23.961936+0000","last_undegraded":"2024-09-17T05:30:23.961936+0000","last_fullsized":"2024-09-17T05:30:23.961936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:19.075142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934587+0000","last_change":"2024-09-17T05:30:23.934587+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934587+0000","last_undegraded":"2024-09-17T05:30:23.934587+0000","last_fullsized":"2024-09-17T05:30:23.934587+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864474+0000","last_change":"2024-09-17T05:30:23.956301+0000","last_active":"2024-09-17T05:30:27.864474+0000","last_peered":"2024-09-17T05:30:27.864474+0000","last_clean":"2024-09-17T05:30:27.864474+0000","last_became_active":"2024-09-17T05:30:23.955995+0000","last_became_peered":"2024-09-17T05:30:23.955995+0000","last_unstale":"2024-09-17T05:30:27.864474+0000","last_undegraded":"2024-09-17T05:30:27.864474+0000","last_fullsized":"2024-09-17T05:30:27.864474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:02.746869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864899+0000","last_change":"2024-09-17T05:30:24.973772+0000","last_active":"2024-09-17T05:30:27.864899+0000","last_peered":"2024-09-17T05:30:27.864899+0000","last_clean":"2024-09-17T05:30:27.864899+0000","last_became_active":"2024-09-17T05:30:24.972340+0000","last_became_peered":"2024-09-17T05:30:24.972340+0000","last_unstale":"2024-09-17T05:30:27.864899+0000","last_undegraded":"2024-09-17T05:30:27.864899+0000","last_fullsized":"2024-09-17T05:30:27.864899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:43.483847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956830+0000","last_change":"2024-09-17T05:30:23.952489+0000","last_active":"2024-09-17T05:30:23.956830+0000","last_peered":"2024-09-17T05:30:23.956830+0000","last_clean":"2024-09-17T05:30:23.956830+0000","last_became_active":"2024-09-17T05:30:23.952212+0000","last_became_peered":"2024-09-17T05:30:23.952212+0000","last_unstale":"2024-09-17T05:30:23.956830+0000","last_undegraded":"2024-09-17T05:30:23.956830+0000","last_fullsized":"2024-09-17T05:30:23.956830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:27.536358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":179,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.928112+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:30:23.928112+0000","last_peered":"2024-09-17T05:30:23.928112+0000","last_clean":"2024-09-17T05:30:23.928112+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:30:23.928112+0000","last_undegraded":"2024-09-17T05:30:23.928112+0000","last_fullsized":"2024-09-17T05:30:23.928112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40168,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331672,"statfs":{"total":96636764160,"available":96595632128,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40172,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331668,"statfs":{"total":96636764160,"available":96595628032,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509509,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45736,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326104,"statfs":{"total":96636764160,"available":96589930496,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509508,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40176,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331664,"statfs":{"total":96636764160,"available":96595623936,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509509,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58220,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313620,"statfs":{"total":96636764160,"available":96577146880,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509510,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40748,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331092,"statfs":{"total":96636764160,"available":96595038208,"internally_reserved":0,"allocated":1269760,"data_stored":868797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1310720,"data_stored":888265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:30:29.074 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:30:29.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- 172.21.15.122:0/1262338185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 msgr2=0x7fdd800cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1262338185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 0x7fdd800cd950 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009840 tx=0x7fdd7801c700 comp rx=0 tx=0).stop 2024-09-17T05:30:29.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- 172.21.15.122:0/1262338185 shutdown_connections 2024-09-17T05:30:29.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1262338185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 0x7fdd800cd950 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1262338185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd8015bd50 0x7fdd800ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- 172.21.15.122:0/1262338185 >> 172.21.15.122:0/1262338185 conn(0x7fdd800d1ea0 msgr2=0x7fdd800d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- 172.21.15.122:0/1262338185 shutdown_connections 2024-09-17T05:30:29.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- 172.21.15.122:0/1262338185 wait complete. 2024-09-17T05:30:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 Processor -- start 2024-09-17T05:30:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- start start 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 0x7fdd800b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd8015bd50 0x7fdd801fb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd8015c130 0x7fdd800c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80168ea0 con 0x7fdd800cf4e0 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80168d20 con 0x7fdd8015c130 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.254+0000 7fdd87c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdd80169020 con 0x7fdd8015bd50 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 0x7fdd800b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 0x7fdd800b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60530/0 (socket says 172.21.15.122:60530) 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 -- 172.21.15.122:0/1469651738 learned_addr learned my addr 172.21.15.122:0/1469651738 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85c00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd8015c130 0x7fdd800c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 -- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd8015bd50 msgr2=0x7fdd801fb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd7fe00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd8015bd50 0x7fdd801fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd8015bd50 0x7fdd801fb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 -- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd8015c130 msgr2=0x7fdd800c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd8015c130 0x7fdd800c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 -- 172.21.15.122:0/1469651738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd801fb580 con 0x7fdd800cf4e0 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85c00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd8015c130 0x7fdd800c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd7fe00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd8015bd50 0x7fdd801fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd85200640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 0x7fdd800b63d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000db00 tx=0x7fdd7000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd7cc00640 1 -- 172.21.15.122:0/1469651738 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70017070 con 0x7fdd800cf4e0 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd7cc00640 1 -- 172.21.15.122:0/1469651738 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd70004b30 con 0x7fdd800cf4e0 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd7cc00640 1 -- 172.21.15.122:0/1469651738 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70005020 con 0x7fdd800cf4e0 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd801f7440 con 0x7fdd800cf4e0 2024-09-17T05:30:29.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd801f7c80 con 0x7fdd800cf4e0 2024-09-17T05:30:29.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd800ce5e0 con 0x7fdd800cf4e0 2024-09-17T05:30:29.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.258+0000 7fdd7cc00640 1 -- 172.21.15.122:0/1469651738 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fdd700051c0 con 0x7fdd800cf4e0 2024-09-17T05:30:29.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.262+0000 7fdd7cc00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd60073b00 0x7fdd60075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.262+0000 7fdd7cc00640 1 -- 172.21.15.122:0/1469651738 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fdd70092ff0 con 0x7fdd800cf4e0 2024-09-17T05:30:29.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.262+0000 7fdd7fe00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd60073b00 0x7fdd60075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.262+0000 7fdd7fe00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd60073b00 0x7fdd60075fc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdd80072330 tx=0x7fdd6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:29.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.266+0000 7fdd7cc00640 1 -- 172.21.15.122:0/1469651738 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdd70020070 con 0x7fdd800cf4e0 2024-09-17T05:30:29.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.430+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd80076360 con 0x7fdd60073b00 2024-09-17T05:30:29.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.438+0000 7fdd7cc00640 1 -- 172.21.15.122:0/1469651738 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247562 (secure 0 0 0) 0x7fdd80076360 con 0x7fdd60073b00 2024-09-17T05:30:29.440 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:29.441 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:30:29.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd60073b00 msgr2=0x7fdd60075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd60073b00 0x7fdd60075fc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdd80072330 tx=0x7fdd6c009040 comp rx=0 tx=0).stop 2024-09-17T05:30:29.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 msgr2=0x7fdd800b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 0x7fdd800b63d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000db00 tx=0x7fdd7000b260 comp rx=0 tx=0).stop 2024-09-17T05:30:29.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 shutdown_connections 2024-09-17T05:30:29.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdd60073b00 0x7fdd60075fc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd8015c130 0x7fdd800c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd8015bd50 0x7fdd801fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 --2- 172.21.15.122:0/1469651738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd800cf4e0 0x7fdd800b63d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 >> 172.21.15.122:0/1469651738 conn(0x7fdd800d1ea0 msgr2=0x7fdd800d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 shutdown_connections 2024-09-17T05:30:29.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.442+0000 7fdd87c00640 1 -- 172.21.15.122:0/1469651738 wait complete. 2024-09-17T05:30:29.460 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-17T05:30:28.533444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590402,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351760,"kb_used_data":6480,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754622960,"statfs":{"total":773094113280,"available":772733911040,"internally_reserved":0,"allocated":6635520,"data_stored":3427636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":353357366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.601333"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946789+0000","last_change":"2024-09-17T05:30:23.946789+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946789+0000","last_undegraded":"2024-09-17T05:30:23.946789+0000","last_fullsized":"2024-09-17T05:30:23.946789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935503+0000","last_change":"2024-09-17T05:30:23.935503+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935503+0000","last_undegraded":"2024-09-17T05:30:23.935503+0000","last_fullsized":"2024-09-17T05:30:23.935503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937682+0000","last_change":"2024-09-17T05:30:23.937682+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937682+0000","last_undegraded":"2024-09-17T05:30:23.937682+0000","last_fullsized":"2024-09-17T05:30:23.937682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932092+0000","last_change":"2024-09-17T05:30:23.932092+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932092+0000","last_undegraded":"2024-09-17T05:30:23.932092+0000","last_fullsized":"2024-09-17T05:30:23.932092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937192+0000","last_change":"2024-09-17T05:30:23.937192+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937192+0000","last_undegraded":"2024-09-17T05:30:23.937192+0000","last_fullsized":"2024-09-17T05:30:23.937192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941621+0000","last_change":"2024-09-17T05:30:23.941621+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941621+0000","last_undegraded":"2024-09-17T05:30:23.941621+0000","last_fullsized":"2024-09-17T05:30:23.941621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932950+0000","last_change":"2024-09-17T05:30:23.932950+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932950+0000","last_undegraded":"2024-09-17T05:30:23.932950+0000","last_fullsized":"2024-09-17T05:30:23.932950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864357+0000","last_change":"2024-09-17T05:30:24.969090+0000","last_active":"2024-09-17T05:30:27.864357+0000","last_peered":"2024-09-17T05:30:27.864357+0000","last_clean":"2024-09-17T05:30:27.864357+0000","last_became_active":"2024-09-17T05:30:24.968736+0000","last_became_peered":"2024-09-17T05:30:24.968736+0000","last_unstale":"2024-09-17T05:30:27.864357+0000","last_undegraded":"2024-09-17T05:30:27.864357+0000","last_fullsized":"2024-09-17T05:30:27.864357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:13.357280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935531+0000","last_change":"2024-09-17T05:30:23.935531+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935531+0000","last_undegraded":"2024-09-17T05:30:23.935531+0000","last_fullsized":"2024-09-17T05:30:23.935531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946112+0000","last_change":"2024-09-17T05:30:23.946112+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946112+0000","last_undegraded":"2024-09-17T05:30:23.946112+0000","last_fullsized":"2024-09-17T05:30:23.946112+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939489+0000","last_change":"2024-09-17T05:30:23.939489+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939489+0000","last_undegraded":"2024-09-17T05:30:23.939489+0000","last_fullsized":"2024-09-17T05:30:23.939489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935459+0000","last_change":"2024-09-17T05:30:23.935459+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935459+0000","last_undegraded":"2024-09-17T05:30:23.935459+0000","last_fullsized":"2024-09-17T05:30:23.935459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.943180+0000","last_change":"2024-09-17T05:30:23.943180+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.943180+0000","last_undegraded":"2024-09-17T05:30:23.943180+0000","last_fullsized":"2024-09-17T05:30:23.943180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934423+0000","last_change":"2024-09-17T05:30:23.934423+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934423+0000","last_undegraded":"2024-09-17T05:30:23.934423+0000","last_fullsized":"2024-09-17T05:30:23.934423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941670+0000","last_change":"2024-09-17T05:30:23.941670+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941670+0000","last_undegraded":"2024-09-17T05:30:23.941670+0000","last_fullsized":"2024-09-17T05:30:23.941670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937557+0000","last_change":"2024-09-17T05:30:23.937557+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937557+0000","last_undegraded":"2024-09-17T05:30:23.937557+0000","last_fullsized":"2024-09-17T05:30:23.937557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946333+0000","last_change":"2024-09-17T05:30:23.946333+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946333+0000","last_undegraded":"2024-09-17T05:30:23.946333+0000","last_fullsized":"2024-09-17T05:30:23.946333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938746+0000","last_change":"2024-09-17T05:30:23.938746+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938746+0000","last_undegraded":"2024-09-17T05:30:23.938746+0000","last_fullsized":"2024-09-17T05:30:23.938746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936163+0000","last_change":"2024-09-17T05:30:23.936163+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936163+0000","last_undegraded":"2024-09-17T05:30:23.936163+0000","last_fullsized":"2024-09-17T05:30:23.936163+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939976+0000","last_change":"2024-09-17T05:30:23.939976+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939976+0000","last_undegraded":"2024-09-17T05:30:23.939976+0000","last_fullsized":"2024-09-17T05:30:23.939976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.942782+0000","last_change":"2024-09-17T05:30:23.942782+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.942782+0000","last_undegraded":"2024-09-17T05:30:23.942782+0000","last_fullsized":"2024-09-17T05:30:23.942782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938893+0000","last_change":"2024-09-17T05:30:23.938893+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938893+0000","last_undegraded":"2024-09-17T05:30:23.938893+0000","last_fullsized":"2024-09-17T05:30:23.938893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932460+0000","last_change":"2024-09-17T05:30:23.932460+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932460+0000","last_undegraded":"2024-09-17T05:30:23.932460+0000","last_fullsized":"2024-09-17T05:30:23.932460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936101+0000","last_change":"2024-09-17T05:30:23.936101+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936101+0000","last_undegraded":"2024-09-17T05:30:23.936101+0000","last_fullsized":"2024-09-17T05:30:23.936101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935649+0000","last_change":"2024-09-17T05:30:23.935649+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935649+0000","last_undegraded":"2024-09-17T05:30:23.935649+0000","last_fullsized":"2024-09-17T05:30:23.935649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941579+0000","last_change":"2024-09-17T05:30:23.941579+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941579+0000","last_undegraded":"2024-09-17T05:30:23.941579+0000","last_fullsized":"2024-09-17T05:30:23.941579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935721+0000","last_change":"2024-09-17T05:30:23.935721+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935721+0000","last_undegraded":"2024-09-17T05:30:23.935721+0000","last_fullsized":"2024-09-17T05:30:23.935721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864500+0000","last_change":"2024-09-17T05:30:24.976244+0000","last_active":"2024-09-17T05:30:27.864500+0000","last_peered":"2024-09-17T05:30:27.864500+0000","last_clean":"2024-09-17T05:30:27.864500+0000","last_became_active":"2024-09-17T05:30:24.975619+0000","last_became_peered":"2024-09-17T05:30:24.975619+0000","last_unstale":"2024-09-17T05:30:27.864500+0000","last_undegraded":"2024-09-17T05:30:27.864500+0000","last_fullsized":"2024-09-17T05:30:27.864500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:42.656824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864466+0000","last_change":"2024-09-17T05:30:24.972218+0000","last_active":"2024-09-17T05:30:27.864466+0000","last_peered":"2024-09-17T05:30:27.864466+0000","last_clean":"2024-09-17T05:30:27.864466+0000","last_became_active":"2024-09-17T05:30:24.969583+0000","last_became_peered":"2024-09-17T05:30:24.969583+0000","last_unstale":"2024-09-17T05:30:27.864466+0000","last_undegraded":"2024-09-17T05:30:27.864466+0000","last_fullsized":"2024-09-17T05:30:27.864466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:45.665750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931376+0000","last_change":"2024-09-17T05:30:23.931376+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931376+0000","last_undegraded":"2024-09-17T05:30:23.931376+0000","last_fullsized":"2024-09-17T05:30:23.931376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936553+0000","last_change":"2024-09-17T05:30:23.936553+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936553+0000","last_undegraded":"2024-09-17T05:30:23.936553+0000","last_fullsized":"2024-09-17T05:30:23.936553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.940234+0000","last_change":"2024-09-17T05:30:23.940234+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.940234+0000","last_undegraded":"2024-09-17T05:30:23.940234+0000","last_fullsized":"2024-09-17T05:30:23.940234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.945686+0000","last_change":"2024-09-17T05:30:23.945686+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.945686+0000","last_undegraded":"2024-09-17T05:30:23.945686+0000","last_fullsized":"2024-09-17T05:30:23.945686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.955314+0000","last_change":"2024-09-17T05:30:23.950637+0000","last_active":"2024-09-17T05:30:23.955314+0000","last_peered":"2024-09-17T05:30:23.955314+0000","last_clean":"2024-09-17T05:30:23.955314+0000","last_became_active":"2024-09-17T05:30:23.950363+0000","last_became_peered":"2024-09-17T05:30:23.950363+0000","last_unstale":"2024-09-17T05:30:23.955314+0000","last_undegraded":"2024-09-17T05:30:23.955314+0000","last_fullsized":"2024-09-17T05:30:23.955314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:01.009438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.945807+0000","last_change":"2024-09-17T05:30:23.945807+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.945807+0000","last_undegraded":"2024-09-17T05:30:23.945807+0000","last_fullsized":"2024-09-17T05:30:23.945807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956419+0000","last_change":"2024-09-17T05:30:23.952541+0000","last_active":"2024-09-17T05:30:23.956419+0000","last_peered":"2024-09-17T05:30:23.956419+0000","last_clean":"2024-09-17T05:30:23.956419+0000","last_became_active":"2024-09-17T05:30:23.952190+0000","last_became_peered":"2024-09-17T05:30:23.952190+0000","last_unstale":"2024-09-17T05:30:23.956419+0000","last_undegraded":"2024-09-17T05:30:23.956419+0000","last_fullsized":"2024-09-17T05:30:23.956419+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:49.775010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864584+0000","last_change":"2024-09-17T05:30:24.976969+0000","last_active":"2024-09-17T05:30:27.864584+0000","last_peered":"2024-09-17T05:30:27.864584+0000","last_clean":"2024-09-17T05:30:27.864584+0000","last_became_active":"2024-09-17T05:30:24.976702+0000","last_became_peered":"2024-09-17T05:30:24.976702+0000","last_unstale":"2024-09-17T05:30:27.864584+0000","last_undegraded":"2024-09-17T05:30:27.864584+0000","last_fullsized":"2024-09-17T05:30:27.864584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:29.194444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.959443+0000","last_change":"2024-09-17T05:30:23.958614+0000","last_active":"2024-09-17T05:30:23.959443+0000","last_peered":"2024-09-17T05:30:23.959443+0000","last_clean":"2024-09-17T05:30:23.959443+0000","last_became_active":"2024-09-17T05:30:23.958200+0000","last_became_peered":"2024-09-17T05:30:23.958200+0000","last_unstale":"2024-09-17T05:30:23.959443+0000","last_undegraded":"2024-09-17T05:30:23.959443+0000","last_fullsized":"2024-09-17T05:30:23.959443+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:07.976069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.943960+0000","last_change":"2024-09-17T05:30:23.943960+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.943960+0000","last_undegraded":"2024-09-17T05:30:23.943960+0000","last_fullsized":"2024-09-17T05:30:23.943960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949074+0000","last_change":"2024-09-17T05:30:23.940517+0000","last_active":"2024-09-17T05:30:23.949074+0000","last_peered":"2024-09-17T05:30:23.949074+0000","last_clean":"2024-09-17T05:30:23.949074+0000","last_became_active":"2024-09-17T05:30:23.940255+0000","last_became_peered":"2024-09-17T05:30:23.940255+0000","last_unstale":"2024-09-17T05:30:23.949074+0000","last_undegraded":"2024-09-17T05:30:23.949074+0000","last_fullsized":"2024-09-17T05:30:23.949074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:47.160401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864815+0000","last_change":"2024-09-17T05:30:24.974140+0000","last_active":"2024-09-17T05:30:27.864815+0000","last_peered":"2024-09-17T05:30:27.864815+0000","last_clean":"2024-09-17T05:30:27.864815+0000","last_became_active":"2024-09-17T05:30:24.973833+0000","last_became_peered":"2024-09-17T05:30:24.973833+0000","last_unstale":"2024-09-17T05:30:27.864815+0000","last_undegraded":"2024-09-17T05:30:27.864815+0000","last_fullsized":"2024-09-17T05:30:27.864815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:28.597753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952758+0000","last_change":"2024-09-17T05:30:23.946652+0000","last_active":"2024-09-17T05:30:23.952758+0000","last_peered":"2024-09-17T05:30:23.952758+0000","last_clean":"2024-09-17T05:30:23.952758+0000","last_became_active":"2024-09-17T05:30:23.946243+0000","last_became_peered":"2024-09-17T05:30:23.946243+0000","last_unstale":"2024-09-17T05:30:23.952758+0000","last_undegraded":"2024-09-17T05:30:23.952758+0000","last_fullsized":"2024-09-17T05:30:23.952758+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:55.869955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939243+0000","last_change":"2024-09-17T05:30:23.939243+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939243+0000","last_undegraded":"2024-09-17T05:30:23.939243+0000","last_fullsized":"2024-09-17T05:30:23.939243+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.958482+0000","last_change":"2024-09-17T05:30:23.953938+0000","last_active":"2024-09-17T05:30:23.958482+0000","last_peered":"2024-09-17T05:30:23.958482+0000","last_clean":"2024-09-17T05:30:23.958482+0000","last_became_active":"2024-09-17T05:30:23.953683+0000","last_became_peered":"2024-09-17T05:30:23.953683+0000","last_unstale":"2024-09-17T05:30:23.958482+0000","last_undegraded":"2024-09-17T05:30:23.958482+0000","last_fullsized":"2024-09-17T05:30:23.958482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:44.089365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934227+0000","last_change":"2024-09-17T05:30:23.934227+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934227+0000","last_undegraded":"2024-09-17T05:30:23.934227+0000","last_fullsized":"2024-09-17T05:30:23.934227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.951311+0000","last_change":"2024-09-17T05:30:23.948325+0000","last_active":"2024-09-17T05:30:23.951311+0000","last_peered":"2024-09-17T05:30:23.951311+0000","last_clean":"2024-09-17T05:30:23.951311+0000","last_became_active":"2024-09-17T05:30:23.948001+0000","last_became_peered":"2024-09-17T05:30:23.948001+0000","last_unstale":"2024-09-17T05:30:23.951311+0000","last_undegraded":"2024-09-17T05:30:23.951311+0000","last_fullsized":"2024-09-17T05:30:23.951311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:07.032087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937367+0000","last_change":"2024-09-17T05:30:23.937367+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937367+0000","last_undegraded":"2024-09-17T05:30:23.937367+0000","last_fullsized":"2024-09-17T05:30:23.937367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956908+0000","last_change":"2024-09-17T05:30:23.954282+0000","last_active":"2024-09-17T05:30:23.956908+0000","last_peered":"2024-09-17T05:30:23.956908+0000","last_clean":"2024-09-17T05:30:23.956908+0000","last_became_active":"2024-09-17T05:30:23.953881+0000","last_became_peered":"2024-09-17T05:30:23.953881+0000","last_unstale":"2024-09-17T05:30:23.956908+0000","last_undegraded":"2024-09-17T05:30:23.956908+0000","last_fullsized":"2024-09-17T05:30:23.956908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:55.869955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.940305+0000","last_change":"2024-09-17T05:30:23.940305+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.940305+0000","last_undegraded":"2024-09-17T05:30:23.940305+0000","last_fullsized":"2024-09-17T05:30:23.940305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.946897+0000","last_change":"2024-09-17T05:30:23.943335+0000","last_active":"2024-09-17T05:30:23.946897+0000","last_peered":"2024-09-17T05:30:23.946897+0000","last_clean":"2024-09-17T05:30:23.946897+0000","last_became_active":"2024-09-17T05:30:23.943018+0000","last_became_peered":"2024-09-17T05:30:23.943018+0000","last_unstale":"2024-09-17T05:30:23.946897+0000","last_undegraded":"2024-09-17T05:30:23.946897+0000","last_fullsized":"2024-09-17T05:30:23.946897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:45.720384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937983+0000","last_change":"2024-09-17T05:30:23.937983+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937983+0000","last_undegraded":"2024-09-17T05:30:23.937983+0000","last_fullsized":"2024-09-17T05:30:23.937983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954980+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954714+0000","last_became_peered":"2024-09-17T05:30:23.954714+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:24.175086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864850+0000","last_change":"2024-09-17T05:30:24.976889+0000","last_active":"2024-09-17T05:30:27.864850+0000","last_peered":"2024-09-17T05:30:27.864850+0000","last_clean":"2024-09-17T05:30:27.864850+0000","last_became_active":"2024-09-17T05:30:24.976636+0000","last_became_peered":"2024-09-17T05:30:24.976636+0000","last_unstale":"2024-09-17T05:30:27.864850+0000","last_undegraded":"2024-09-17T05:30:27.864850+0000","last_fullsized":"2024-09-17T05:30:27.864850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:54.276295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.948946+0000","last_change":"2024-09-17T05:30:23.942665+0000","last_active":"2024-09-17T05:30:23.948946+0000","last_peered":"2024-09-17T05:30:23.948946+0000","last_clean":"2024-09-17T05:30:23.948946+0000","last_became_active":"2024-09-17T05:30:23.942377+0000","last_became_peered":"2024-09-17T05:30:23.942377+0000","last_unstale":"2024-09-17T05:30:23.948946+0000","last_undegraded":"2024-09-17T05:30:23.948946+0000","last_fullsized":"2024-09-17T05:30:23.948946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:35.760533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932249+0000","last_change":"2024-09-17T05:30:23.932249+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932249+0000","last_undegraded":"2024-09-17T05:30:23.932249+0000","last_fullsized":"2024-09-17T05:30:23.932249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956867+0000","last_change":"2024-09-17T05:30:23.953324+0000","last_active":"2024-09-17T05:30:23.956867+0000","last_peered":"2024-09-17T05:30:23.956867+0000","last_clean":"2024-09-17T05:30:23.956867+0000","last_became_active":"2024-09-17T05:30:23.953049+0000","last_became_peered":"2024-09-17T05:30:23.953049+0000","last_unstale":"2024-09-17T05:30:23.956867+0000","last_undegraded":"2024-09-17T05:30:23.956867+0000","last_fullsized":"2024-09-17T05:30:23.956867+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:58.939352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935005+0000","last_change":"2024-09-17T05:30:23.935005+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935005+0000","last_undegraded":"2024-09-17T05:30:23.935005+0000","last_fullsized":"2024-09-17T05:30:23.935005+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954792+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954435+0000","last_became_peered":"2024-09-17T05:30:23.954435+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:30.663318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934544+0000","last_change":"2024-09-17T05:30:23.934544+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934544+0000","last_undegraded":"2024-09-17T05:30:23.934544+0000","last_fullsized":"2024-09-17T05:30:23.934544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949916+0000","last_change":"2024-09-17T05:30:23.948409+0000","last_active":"2024-09-17T05:30:23.949916+0000","last_peered":"2024-09-17T05:30:23.949916+0000","last_clean":"2024-09-17T05:30:23.949916+0000","last_became_active":"2024-09-17T05:30:23.948114+0000","last_became_peered":"2024-09-17T05:30:23.948114+0000","last_unstale":"2024-09-17T05:30:23.949916+0000","last_undegraded":"2024-09-17T05:30:23.949916+0000","last_fullsized":"2024-09-17T05:30:23.949916+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:53.478330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934022+0000","last_change":"2024-09-17T05:30:23.934022+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934022+0000","last_undegraded":"2024-09-17T05:30:23.934022+0000","last_fullsized":"2024-09-17T05:30:23.934022+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.948904+0000","last_change":"2024-09-17T05:30:23.943942+0000","last_active":"2024-09-17T05:30:23.948904+0000","last_peered":"2024-09-17T05:30:23.948904+0000","last_clean":"2024-09-17T05:30:23.948904+0000","last_became_active":"2024-09-17T05:30:23.943669+0000","last_became_peered":"2024-09-17T05:30:23.943669+0000","last_unstale":"2024-09-17T05:30:23.948904+0000","last_undegraded":"2024-09-17T05:30:23.948904+0000","last_fullsized":"2024-09-17T05:30:23.948904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:38.122369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931093+0000","last_change":"2024-09-17T05:30:23.931093+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931093+0000","last_undegraded":"2024-09-17T05:30:23.931093+0000","last_fullsized":"2024-09-17T05:30:23.931093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949792+0000","last_change":"2024-09-17T05:30:23.949103+0000","last_active":"2024-09-17T05:30:23.949792+0000","last_peered":"2024-09-17T05:30:23.949792+0000","last_clean":"2024-09-17T05:30:23.949792+0000","last_became_active":"2024-09-17T05:30:23.948810+0000","last_became_peered":"2024-09-17T05:30:23.948810+0000","last_unstale":"2024-09-17T05:30:23.949792+0000","last_undegraded":"2024-09-17T05:30:23.949792+0000","last_fullsized":"2024-09-17T05:30:23.949792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:45.173108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938331+0000","last_change":"2024-09-17T05:30:23.938331+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938331+0000","last_undegraded":"2024-09-17T05:30:23.938331+0000","last_fullsized":"2024-09-17T05:30:23.938331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.961778+0000","last_change":"2024-09-17T05:30:23.960591+0000","last_active":"2024-09-17T05:30:23.961778+0000","last_peered":"2024-09-17T05:30:23.961778+0000","last_clean":"2024-09-17T05:30:23.961778+0000","last_became_active":"2024-09-17T05:30:23.960068+0000","last_became_peered":"2024-09-17T05:30:23.960068+0000","last_unstale":"2024-09-17T05:30:23.961778+0000","last_undegraded":"2024-09-17T05:30:23.961778+0000","last_fullsized":"2024-09-17T05:30:23.961778+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:32.536820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937514+0000","last_change":"2024-09-17T05:30:23.937514+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937514+0000","last_undegraded":"2024-09-17T05:30:23.937514+0000","last_fullsized":"2024-09-17T05:30:23.937514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864645+0000","last_change":"2024-09-17T05:30:23.951223+0000","last_active":"2024-09-17T05:30:27.864645+0000","last_peered":"2024-09-17T05:30:27.864645+0000","last_clean":"2024-09-17T05:30:27.864645+0000","last_became_active":"2024-09-17T05:30:23.950514+0000","last_became_peered":"2024-09-17T05:30:23.950514+0000","last_unstale":"2024-09-17T05:30:27.864645+0000","last_undegraded":"2024-09-17T05:30:27.864645+0000","last_fullsized":"2024-09-17T05:30:27.864645+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:45.736482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936819+0000","last_change":"2024-09-17T05:30:23.936819+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936819+0000","last_undegraded":"2024-09-17T05:30:23.936819+0000","last_fullsized":"2024-09-17T05:30:23.936819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952033+0000","last_change":"2024-09-17T05:30:23.944667+0000","last_active":"2024-09-17T05:30:23.952033+0000","last_peered":"2024-09-17T05:30:23.952033+0000","last_clean":"2024-09-17T05:30:23.952033+0000","last_became_active":"2024-09-17T05:30:23.944381+0000","last_became_peered":"2024-09-17T05:30:23.944381+0000","last_unstale":"2024-09-17T05:30:23.952033+0000","last_undegraded":"2024-09-17T05:30:23.952033+0000","last_fullsized":"2024-09-17T05:30:23.952033+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:27.536358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931506+0000","last_change":"2024-09-17T05:30:23.931506+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931506+0000","last_undegraded":"2024-09-17T05:30:23.931506+0000","last_fullsized":"2024-09-17T05:30:23.931506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864732+0000","last_change":"2024-09-17T05:30:23.960543+0000","last_active":"2024-09-17T05:30:27.864732+0000","last_peered":"2024-09-17T05:30:27.864732+0000","last_clean":"2024-09-17T05:30:27.864732+0000","last_became_active":"2024-09-17T05:30:23.960241+0000","last_became_peered":"2024-09-17T05:30:23.960241+0000","last_unstale":"2024-09-17T05:30:27.864732+0000","last_undegraded":"2024-09-17T05:30:27.864732+0000","last_fullsized":"2024-09-17T05:30:27.864732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:25.924845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934516+0000","last_change":"2024-09-17T05:30:23.934516+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934516+0000","last_undegraded":"2024-09-17T05:30:23.934516+0000","last_fullsized":"2024-09-17T05:30:23.934516+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.955243+0000","last_change":"2024-09-17T05:30:23.951379+0000","last_active":"2024-09-17T05:30:23.955243+0000","last_peered":"2024-09-17T05:30:23.955243+0000","last_clean":"2024-09-17T05:30:23.955243+0000","last_became_active":"2024-09-17T05:30:23.950900+0000","last_became_peered":"2024-09-17T05:30:23.950900+0000","last_unstale":"2024-09-17T05:30:23.955243+0000","last_undegraded":"2024-09-17T05:30:23.955243+0000","last_fullsized":"2024-09-17T05:30:23.955243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:54.643188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864503+0000","last_change":"2024-09-17T05:30:24.973721+0000","last_active":"2024-09-17T05:30:27.864503+0000","last_peered":"2024-09-17T05:30:27.864503+0000","last_clean":"2024-09-17T05:30:27.864503+0000","last_became_active":"2024-09-17T05:30:24.973391+0000","last_became_peered":"2024-09-17T05:30:24.973391+0000","last_unstale":"2024-09-17T05:30:27.864503+0000","last_undegraded":"2024-09-17T05:30:27.864503+0000","last_fullsized":"2024-09-17T05:30:27.864503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:02.895238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864244+0000","last_change":"2024-09-17T05:30:23.950654+0000","last_active":"2024-09-17T05:30:27.864244+0000","last_peered":"2024-09-17T05:30:27.864244+0000","last_clean":"2024-09-17T05:30:27.864244+0000","last_became_active":"2024-09-17T05:30:23.950372+0000","last_became_peered":"2024-09-17T05:30:23.950372+0000","last_unstale":"2024-09-17T05:30:27.864244+0000","last_undegraded":"2024-09-17T05:30:27.864244+0000","last_fullsized":"2024-09-17T05:30:27.864244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:46.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864458+0000","last_change":"2024-09-17T05:30:24.964323+0000","last_active":"2024-09-17T05:30:27.864458+0000","last_peered":"2024-09-17T05:30:27.864458+0000","last_clean":"2024-09-17T05:30:27.864458+0000","last_became_active":"2024-09-17T05:30:24.964039+0000","last_became_peered":"2024-09-17T05:30:24.964039+0000","last_unstale":"2024-09-17T05:30:27.864458+0000","last_undegraded":"2024-09-17T05:30:27.864458+0000","last_fullsized":"2024-09-17T05:30:27.864458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:52.710209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.951246+0000","last_change":"2024-09-17T05:30:23.946216+0000","last_active":"2024-09-17T05:30:23.951246+0000","last_peered":"2024-09-17T05:30:23.951246+0000","last_clean":"2024-09-17T05:30:23.951246+0000","last_became_active":"2024-09-17T05:30:23.945797+0000","last_became_peered":"2024-09-17T05:30:23.945797+0000","last_unstale":"2024-09-17T05:30:23.951246+0000","last_undegraded":"2024-09-17T05:30:23.951246+0000","last_fullsized":"2024-09-17T05:30:23.951246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:16.565367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937877+0000","last_change":"2024-09-17T05:30:23.937877+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937877+0000","last_undegraded":"2024-09-17T05:30:23.937877+0000","last_fullsized":"2024-09-17T05:30:23.937877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.957639+0000","last_change":"2024-09-17T05:30:23.953693+0000","last_active":"2024-09-17T05:30:23.957639+0000","last_peered":"2024-09-17T05:30:23.957639+0000","last_clean":"2024-09-17T05:30:23.957639+0000","last_became_active":"2024-09-17T05:30:23.953424+0000","last_became_peered":"2024-09-17T05:30:23.953424+0000","last_unstale":"2024-09-17T05:30:23.957639+0000","last_undegraded":"2024-09-17T05:30:23.957639+0000","last_fullsized":"2024-09-17T05:30:23.957639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:19.341040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937107+0000","last_change":"2024-09-17T05:30:23.937107+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937107+0000","last_undegraded":"2024-09-17T05:30:23.937107+0000","last_fullsized":"2024-09-17T05:30:23.937107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.959458+0000","last_change":"2024-09-17T05:30:23.958649+0000","last_active":"2024-09-17T05:30:23.959458+0000","last_peered":"2024-09-17T05:30:23.959458+0000","last_clean":"2024-09-17T05:30:23.959458+0000","last_became_active":"2024-09-17T05:30:23.958329+0000","last_became_peered":"2024-09-17T05:30:23.958329+0000","last_unstale":"2024-09-17T05:30:23.959458+0000","last_undegraded":"2024-09-17T05:30:23.959458+0000","last_fullsized":"2024-09-17T05:30:23.959458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.772225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935536+0000","last_change":"2024-09-17T05:30:23.935536+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935536+0000","last_undegraded":"2024-09-17T05:30:23.935536+0000","last_fullsized":"2024-09-17T05:30:23.935536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.950069+0000","last_change":"2024-09-17T05:30:23.943837+0000","last_active":"2024-09-17T05:30:23.950069+0000","last_peered":"2024-09-17T05:30:23.950069+0000","last_clean":"2024-09-17T05:30:23.950069+0000","last_became_active":"2024-09-17T05:30:23.942677+0000","last_became_peered":"2024-09-17T05:30:23.942677+0000","last_unstale":"2024-09-17T05:30:23.950069+0000","last_undegraded":"2024-09-17T05:30:23.950069+0000","last_fullsized":"2024-09-17T05:30:23.950069+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:26.543985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864917+0000","last_change":"2024-09-17T05:30:24.974072+0000","last_active":"2024-09-17T05:30:27.864917+0000","last_peered":"2024-09-17T05:30:27.864917+0000","last_clean":"2024-09-17T05:30:27.864917+0000","last_became_active":"2024-09-17T05:30:24.973224+0000","last_became_peered":"2024-09-17T05:30:24.973224+0000","last_unstale":"2024-09-17T05:30:27.864917+0000","last_undegraded":"2024-09-17T05:30:27.864917+0000","last_fullsized":"2024-09-17T05:30:27.864917+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:35.858572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.953004+0000","last_change":"2024-09-17T05:30:23.947956+0000","last_active":"2024-09-17T05:30:23.953004+0000","last_peered":"2024-09-17T05:30:23.953004+0000","last_clean":"2024-09-17T05:30:23.953004+0000","last_became_active":"2024-09-17T05:30:23.947641+0000","last_became_peered":"2024-09-17T05:30:23.947641+0000","last_unstale":"2024-09-17T05:30:23.953004+0000","last_undegraded":"2024-09-17T05:30:23.953004+0000","last_fullsized":"2024-09-17T05:30:23.953004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:41.752305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937632+0000","last_change":"2024-09-17T05:30:23.937632+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937632+0000","last_undegraded":"2024-09-17T05:30:23.937632+0000","last_fullsized":"2024-09-17T05:30:23.937632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.957936+0000","last_change":"2024-09-17T05:30:23.957320+0000","last_active":"2024-09-17T05:30:23.957936+0000","last_peered":"2024-09-17T05:30:23.957936+0000","last_clean":"2024-09-17T05:30:23.957936+0000","last_became_active":"2024-09-17T05:30:23.956956+0000","last_became_peered":"2024-09-17T05:30:23.956956+0000","last_unstale":"2024-09-17T05:30:23.957936+0000","last_undegraded":"2024-09-17T05:30:23.957936+0000","last_fullsized":"2024-09-17T05:30:23.957936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:52.455342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.942385+0000","last_change":"2024-09-17T05:30:23.942385+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.942385+0000","last_undegraded":"2024-09-17T05:30:23.942385+0000","last_fullsized":"2024-09-17T05:30:23.942385+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952593+0000","last_change":"2024-09-17T05:30:23.945568+0000","last_active":"2024-09-17T05:30:23.952593+0000","last_peered":"2024-09-17T05:30:23.952593+0000","last_clean":"2024-09-17T05:30:23.952593+0000","last_became_active":"2024-09-17T05:30:23.945279+0000","last_became_peered":"2024-09-17T05:30:23.945279+0000","last_unstale":"2024-09-17T05:30:23.952593+0000","last_undegraded":"2024-09-17T05:30:23.952593+0000","last_fullsized":"2024-09-17T05:30:23.952593+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:58.939352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864556+0000","last_change":"2024-09-17T05:30:24.974524+0000","last_active":"2024-09-17T05:30:27.864556+0000","last_peered":"2024-09-17T05:30:27.864556+0000","last_clean":"2024-09-17T05:30:27.864556+0000","last_became_active":"2024-09-17T05:30:24.974227+0000","last_became_peered":"2024-09-17T05:30:24.974227+0000","last_unstale":"2024-09-17T05:30:27.864556+0000","last_undegraded":"2024-09-17T05:30:27.864556+0000","last_fullsized":"2024-09-17T05:30:27.864556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:47.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.961936+0000","last_change":"2024-09-17T05:30:23.960348+0000","last_active":"2024-09-17T05:30:23.961936+0000","last_peered":"2024-09-17T05:30:23.961936+0000","last_clean":"2024-09-17T05:30:23.961936+0000","last_became_active":"2024-09-17T05:30:23.960056+0000","last_became_peered":"2024-09-17T05:30:23.960056+0000","last_unstale":"2024-09-17T05:30:23.961936+0000","last_undegraded":"2024-09-17T05:30:23.961936+0000","last_fullsized":"2024-09-17T05:30:23.961936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:19.075142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934587+0000","last_change":"2024-09-17T05:30:23.934587+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934587+0000","last_undegraded":"2024-09-17T05:30:23.934587+0000","last_fullsized":"2024-09-17T05:30:23.934587+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864474+0000","last_change":"2024-09-17T05:30:23.956301+0000","last_active":"2024-09-17T05:30:27.864474+0000","last_peered":"2024-09-17T05:30:27.864474+0000","last_clean":"2024-09-17T05:30:27.864474+0000","last_became_active":"2024-09-17T05:30:23.955995+0000","last_became_peered":"2024-09-17T05:30:23.955995+0000","last_unstale":"2024-09-17T05:30:27.864474+0000","last_undegraded":"2024-09-17T05:30:27.864474+0000","last_fullsized":"2024-09-17T05:30:27.864474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:02.746869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864899+0000","last_change":"2024-09-17T05:30:24.973772+0000","last_active":"2024-09-17T05:30:27.864899+0000","last_peered":"2024-09-17T05:30:27.864899+0000","last_clean":"2024-09-17T05:30:27.864899+0000","last_became_active":"2024-09-17T05:30:24.972340+0000","last_became_peered":"2024-09-17T05:30:24.972340+0000","last_unstale":"2024-09-17T05:30:27.864899+0000","last_undegraded":"2024-09-17T05:30:27.864899+0000","last_fullsized":"2024-09-17T05:30:27.864899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:43.483847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956830+0000","last_change":"2024-09-17T05:30:23.952489+0000","last_active":"2024-09-17T05:30:23.956830+0000","last_peered":"2024-09-17T05:30:23.956830+0000","last_clean":"2024-09-17T05:30:23.956830+0000","last_became_active":"2024-09-17T05:30:23.952212+0000","last_became_peered":"2024-09-17T05:30:23.952212+0000","last_unstale":"2024-09-17T05:30:23.956830+0000","last_undegraded":"2024-09-17T05:30:23.956830+0000","last_fullsized":"2024-09-17T05:30:23.956830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:27.536358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":179,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.928112+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:30:23.928112+0000","last_peered":"2024-09-17T05:30:23.928112+0000","last_clean":"2024-09-17T05:30:23.928112+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:30:23.928112+0000","last_undegraded":"2024-09-17T05:30:23.928112+0000","last_fullsized":"2024-09-17T05:30:23.928112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40168,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331672,"statfs":{"total":96636764160,"available":96595632128,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40172,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331668,"statfs":{"total":96636764160,"available":96595628032,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509509,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45736,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326104,"statfs":{"total":96636764160,"available":96589930496,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509508,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40176,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331664,"statfs":{"total":96636764160,"available":96595623936,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509509,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58220,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313620,"statfs":{"total":96636764160,"available":96577146880,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509510,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40748,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331092,"statfs":{"total":96636764160,"available":96595038208,"internally_reserved":0,"allocated":1269760,"data_stored":868797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1310720,"data_stored":888265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:30:29.464 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:30:29.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.642+0000 7f52e8200640 1 -- 172.21.15.122:0/1100398726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e015dae0 msgr2=0x7f52e00acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.642+0000 7f52e8200640 1 --2- 172.21.15.122:0/1100398726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e015dae0 0x7f52e00acd40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f52d400ad90 tx=0x7f52d401dc20 comp rx=0 tx=0).stop 2024-09-17T05:30:29.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- 172.21.15.122:0/1100398726 shutdown_connections 2024-09-17T05:30:29.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 --2- 172.21.15.122:0/1100398726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52e00b49e0 0x7f52e006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 --2- 172.21.15.122:0/1100398726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e015dae0 0x7f52e00acd40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- 172.21.15.122:0/1100398726 >> 172.21.15.122:0/1100398726 conn(0x7f52e00b18f0 msgr2=0x7f52e00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- 172.21.15.122:0/1100398726 shutdown_connections 2024-09-17T05:30:29.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- 172.21.15.122:0/1100398726 wait complete. 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 Processor -- start 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- start start 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e00b49e0 0x7f52e014acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52e00cd100 0x7f52e01562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52e015dae0 0x7f52e00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52e00cf230 con 0x7f52e00b49e0 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52e00cf0b0 con 0x7f52e00cd100 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f52e00cf3b0 con 0x7f52e015dae0 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e00b49e0 0x7f52e014acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e00b49e0 0x7f52e014acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60558/0 (socket says 172.21.15.122:60558) 2024-09-17T05:30:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 -- 172.21.15.122:0/1513093011 learned_addr learned my addr 172.21.15.122:0/1513093011 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e4e00640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52e00cd100 0x7f52e01562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e6200640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52e015dae0 0x7f52e00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 -- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52e015dae0 msgr2=0x7f52e00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52e015dae0 0x7f52e00cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 -- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52e00cd100 msgr2=0x7f52e01562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52e00cd100 0x7f52e01562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 -- 172.21.15.122:0/1513093011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e00cc320 con 0x7f52e00b49e0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e5800640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e00b49e0 0x7f52e014acc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000f90 tx=0x7f52d400a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52dd600640 1 -- 172.21.15.122:0/1513093011 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d40b3040 con 0x7f52e00b49e0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e0158e20 con 0x7f52e00b49e0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52dd600640 1 -- 172.21.15.122:0/1513093011 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52d4004d40 con 0x7f52e00b49e0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.646+0000 7f52dd600640 1 -- 172.21.15.122:0/1513093011 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4004200 con 0x7f52e00b49e0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.650+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e01593c0 con 0x7f52e00b49e0 2024-09-17T05:30:29.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.650+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e0069410 con 0x7f52e00b49e0 2024-09-17T05:30:29.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.650+0000 7f52dd600640 1 -- 172.21.15.122:0/1513093011 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f52d40043a0 con 0x7f52e00b49e0 2024-09-17T05:30:29.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.650+0000 7f52dd600640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52c0073b00 0x7f52c0075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:29.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.650+0000 7f52dd600640 1 -- 172.21.15.122:0/1513093011 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f52d412cf80 con 0x7f52e00b49e0 2024-09-17T05:30:29.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.650+0000 7f52e4e00640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52c0073b00 0x7f52c0075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:29.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.654+0000 7f52e4e00640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52c0073b00 0x7f52c0075fc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f52e009de80 tx=0x7f52cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:29.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.654+0000 7f52dd600640 1 -- 172.21.15.122:0/1513093011 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f52d40bf060 con 0x7f52e00b49e0 2024-09-17T05:30:29.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.818+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52e00af5e0 con 0x7f52c0073b00 2024-09-17T05:30:29.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.822+0000 7f52dd600640 1 -- 172.21.15.122:0/1513093011 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247562 (secure 0 0 0) 0x7f52e00af5e0 con 0x7f52c0073b00 2024-09-17T05:30:29.825 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:29.826 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52c0073b00 msgr2=0x7f52c0075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52c0073b00 0x7f52c0075fc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f52e009de80 tx=0x7f52cc009040 comp rx=0 tx=0).stop 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e00b49e0 msgr2=0x7f52e014acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e00b49e0 0x7f52e014acc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000f90 tx=0x7f52d400a470 comp rx=0 tx=0).stop 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 shutdown_connections 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52c0073b00 0x7f52c0075fc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52e015dae0 0x7f52e00cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52e00cd100 0x7f52e01562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 --2- 172.21.15.122:0/1513093011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52e00b49e0 0x7f52e014acc0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 >> 172.21.15.122:0/1513093011 conn(0x7f52e00b18f0 msgr2=0x7f52e0157bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 shutdown_connections 2024-09-17T05:30:29.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:29.826+0000 7f52e8200640 1 -- 172.21.15.122:0/1513093011 wait complete. 2024-09-17T05:30:29.845 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-17T05:30:28.533444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590402,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351760,"kb_used_data":6480,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754622960,"statfs":{"total":773094113280,"available":772733911040,"internally_reserved":0,"allocated":6635520,"data_stored":3427636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":353357366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.601333"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946789+0000","last_change":"2024-09-17T05:30:23.946789+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946789+0000","last_undegraded":"2024-09-17T05:30:23.946789+0000","last_fullsized":"2024-09-17T05:30:23.946789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935503+0000","last_change":"2024-09-17T05:30:23.935503+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935503+0000","last_undegraded":"2024-09-17T05:30:23.935503+0000","last_fullsized":"2024-09-17T05:30:23.935503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937682+0000","last_change":"2024-09-17T05:30:23.937682+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937682+0000","last_undegraded":"2024-09-17T05:30:23.937682+0000","last_fullsized":"2024-09-17T05:30:23.937682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932092+0000","last_change":"2024-09-17T05:30:23.932092+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932092+0000","last_undegraded":"2024-09-17T05:30:23.932092+0000","last_fullsized":"2024-09-17T05:30:23.932092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937192+0000","last_change":"2024-09-17T05:30:23.937192+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937192+0000","last_undegraded":"2024-09-17T05:30:23.937192+0000","last_fullsized":"2024-09-17T05:30:23.937192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941621+0000","last_change":"2024-09-17T05:30:23.941621+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941621+0000","last_undegraded":"2024-09-17T05:30:23.941621+0000","last_fullsized":"2024-09-17T05:30:23.941621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932950+0000","last_change":"2024-09-17T05:30:23.932950+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932950+0000","last_undegraded":"2024-09-17T05:30:23.932950+0000","last_fullsized":"2024-09-17T05:30:23.932950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864357+0000","last_change":"2024-09-17T05:30:24.969090+0000","last_active":"2024-09-17T05:30:27.864357+0000","last_peered":"2024-09-17T05:30:27.864357+0000","last_clean":"2024-09-17T05:30:27.864357+0000","last_became_active":"2024-09-17T05:30:24.968736+0000","last_became_peered":"2024-09-17T05:30:24.968736+0000","last_unstale":"2024-09-17T05:30:27.864357+0000","last_undegraded":"2024-09-17T05:30:27.864357+0000","last_fullsized":"2024-09-17T05:30:27.864357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:13.357280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935531+0000","last_change":"2024-09-17T05:30:23.935531+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935531+0000","last_undegraded":"2024-09-17T05:30:23.935531+0000","last_fullsized":"2024-09-17T05:30:23.935531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946112+0000","last_change":"2024-09-17T05:30:23.946112+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946112+0000","last_undegraded":"2024-09-17T05:30:23.946112+0000","last_fullsized":"2024-09-17T05:30:23.946112+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939489+0000","last_change":"2024-09-17T05:30:23.939489+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939489+0000","last_undegraded":"2024-09-17T05:30:23.939489+0000","last_fullsized":"2024-09-17T05:30:23.939489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935459+0000","last_change":"2024-09-17T05:30:23.935459+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935459+0000","last_undegraded":"2024-09-17T05:30:23.935459+0000","last_fullsized":"2024-09-17T05:30:23.935459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.943180+0000","last_change":"2024-09-17T05:30:23.943180+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.943180+0000","last_undegraded":"2024-09-17T05:30:23.943180+0000","last_fullsized":"2024-09-17T05:30:23.943180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934423+0000","last_change":"2024-09-17T05:30:23.934423+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934423+0000","last_undegraded":"2024-09-17T05:30:23.934423+0000","last_fullsized":"2024-09-17T05:30:23.934423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941670+0000","last_change":"2024-09-17T05:30:23.941670+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941670+0000","last_undegraded":"2024-09-17T05:30:23.941670+0000","last_fullsized":"2024-09-17T05:30:23.941670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937557+0000","last_change":"2024-09-17T05:30:23.937557+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937557+0000","last_undegraded":"2024-09-17T05:30:23.937557+0000","last_fullsized":"2024-09-17T05:30:23.937557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.946333+0000","last_change":"2024-09-17T05:30:23.946333+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.946333+0000","last_undegraded":"2024-09-17T05:30:23.946333+0000","last_fullsized":"2024-09-17T05:30:23.946333+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938746+0000","last_change":"2024-09-17T05:30:23.938746+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938746+0000","last_undegraded":"2024-09-17T05:30:23.938746+0000","last_fullsized":"2024-09-17T05:30:23.938746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936163+0000","last_change":"2024-09-17T05:30:23.936163+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936163+0000","last_undegraded":"2024-09-17T05:30:23.936163+0000","last_fullsized":"2024-09-17T05:30:23.936163+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939976+0000","last_change":"2024-09-17T05:30:23.939976+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939976+0000","last_undegraded":"2024-09-17T05:30:23.939976+0000","last_fullsized":"2024-09-17T05:30:23.939976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.942782+0000","last_change":"2024-09-17T05:30:23.942782+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.942782+0000","last_undegraded":"2024-09-17T05:30:23.942782+0000","last_fullsized":"2024-09-17T05:30:23.942782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938893+0000","last_change":"2024-09-17T05:30:23.938893+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938893+0000","last_undegraded":"2024-09-17T05:30:23.938893+0000","last_fullsized":"2024-09-17T05:30:23.938893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932460+0000","last_change":"2024-09-17T05:30:23.932460+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932460+0000","last_undegraded":"2024-09-17T05:30:23.932460+0000","last_fullsized":"2024-09-17T05:30:23.932460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936101+0000","last_change":"2024-09-17T05:30:23.936101+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936101+0000","last_undegraded":"2024-09-17T05:30:23.936101+0000","last_fullsized":"2024-09-17T05:30:23.936101+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935649+0000","last_change":"2024-09-17T05:30:23.935649+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935649+0000","last_undegraded":"2024-09-17T05:30:23.935649+0000","last_fullsized":"2024-09-17T05:30:23.935649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.941579+0000","last_change":"2024-09-17T05:30:23.941579+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.941579+0000","last_undegraded":"2024-09-17T05:30:23.941579+0000","last_fullsized":"2024-09-17T05:30:23.941579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935721+0000","last_change":"2024-09-17T05:30:23.935721+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935721+0000","last_undegraded":"2024-09-17T05:30:23.935721+0000","last_fullsized":"2024-09-17T05:30:23.935721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864500+0000","last_change":"2024-09-17T05:30:24.976244+0000","last_active":"2024-09-17T05:30:27.864500+0000","last_peered":"2024-09-17T05:30:27.864500+0000","last_clean":"2024-09-17T05:30:27.864500+0000","last_became_active":"2024-09-17T05:30:24.975619+0000","last_became_peered":"2024-09-17T05:30:24.975619+0000","last_unstale":"2024-09-17T05:30:27.864500+0000","last_undegraded":"2024-09-17T05:30:27.864500+0000","last_fullsized":"2024-09-17T05:30:27.864500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:42.656824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864466+0000","last_change":"2024-09-17T05:30:24.972218+0000","last_active":"2024-09-17T05:30:27.864466+0000","last_peered":"2024-09-17T05:30:27.864466+0000","last_clean":"2024-09-17T05:30:27.864466+0000","last_became_active":"2024-09-17T05:30:24.969583+0000","last_became_peered":"2024-09-17T05:30:24.969583+0000","last_unstale":"2024-09-17T05:30:27.864466+0000","last_undegraded":"2024-09-17T05:30:27.864466+0000","last_fullsized":"2024-09-17T05:30:27.864466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:45.665750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931376+0000","last_change":"2024-09-17T05:30:23.931376+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931376+0000","last_undegraded":"2024-09-17T05:30:23.931376+0000","last_fullsized":"2024-09-17T05:30:23.931376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936553+0000","last_change":"2024-09-17T05:30:23.936553+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936553+0000","last_undegraded":"2024-09-17T05:30:23.936553+0000","last_fullsized":"2024-09-17T05:30:23.936553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.940234+0000","last_change":"2024-09-17T05:30:23.940234+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.940234+0000","last_undegraded":"2024-09-17T05:30:23.940234+0000","last_fullsized":"2024-09-17T05:30:23.940234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.945686+0000","last_change":"2024-09-17T05:30:23.945686+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.945686+0000","last_undegraded":"2024-09-17T05:30:23.945686+0000","last_fullsized":"2024-09-17T05:30:23.945686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.955314+0000","last_change":"2024-09-17T05:30:23.950637+0000","last_active":"2024-09-17T05:30:23.955314+0000","last_peered":"2024-09-17T05:30:23.955314+0000","last_clean":"2024-09-17T05:30:23.955314+0000","last_became_active":"2024-09-17T05:30:23.950363+0000","last_became_peered":"2024-09-17T05:30:23.950363+0000","last_unstale":"2024-09-17T05:30:23.955314+0000","last_undegraded":"2024-09-17T05:30:23.955314+0000","last_fullsized":"2024-09-17T05:30:23.955314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:01.009438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.945807+0000","last_change":"2024-09-17T05:30:23.945807+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.945807+0000","last_undegraded":"2024-09-17T05:30:23.945807+0000","last_fullsized":"2024-09-17T05:30:23.945807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956419+0000","last_change":"2024-09-17T05:30:23.952541+0000","last_active":"2024-09-17T05:30:23.956419+0000","last_peered":"2024-09-17T05:30:23.956419+0000","last_clean":"2024-09-17T05:30:23.956419+0000","last_became_active":"2024-09-17T05:30:23.952190+0000","last_became_peered":"2024-09-17T05:30:23.952190+0000","last_unstale":"2024-09-17T05:30:23.956419+0000","last_undegraded":"2024-09-17T05:30:23.956419+0000","last_fullsized":"2024-09-17T05:30:23.956419+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:49.775010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864584+0000","last_change":"2024-09-17T05:30:24.976969+0000","last_active":"2024-09-17T05:30:27.864584+0000","last_peered":"2024-09-17T05:30:27.864584+0000","last_clean":"2024-09-17T05:30:27.864584+0000","last_became_active":"2024-09-17T05:30:24.976702+0000","last_became_peered":"2024-09-17T05:30:24.976702+0000","last_unstale":"2024-09-17T05:30:27.864584+0000","last_undegraded":"2024-09-17T05:30:27.864584+0000","last_fullsized":"2024-09-17T05:30:27.864584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:29.194444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.959443+0000","last_change":"2024-09-17T05:30:23.958614+0000","last_active":"2024-09-17T05:30:23.959443+0000","last_peered":"2024-09-17T05:30:23.959443+0000","last_clean":"2024-09-17T05:30:23.959443+0000","last_became_active":"2024-09-17T05:30:23.958200+0000","last_became_peered":"2024-09-17T05:30:23.958200+0000","last_unstale":"2024-09-17T05:30:23.959443+0000","last_undegraded":"2024-09-17T05:30:23.959443+0000","last_fullsized":"2024-09-17T05:30:23.959443+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:07.976069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.943960+0000","last_change":"2024-09-17T05:30:23.943960+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.943960+0000","last_undegraded":"2024-09-17T05:30:23.943960+0000","last_fullsized":"2024-09-17T05:30:23.943960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949074+0000","last_change":"2024-09-17T05:30:23.940517+0000","last_active":"2024-09-17T05:30:23.949074+0000","last_peered":"2024-09-17T05:30:23.949074+0000","last_clean":"2024-09-17T05:30:23.949074+0000","last_became_active":"2024-09-17T05:30:23.940255+0000","last_became_peered":"2024-09-17T05:30:23.940255+0000","last_unstale":"2024-09-17T05:30:23.949074+0000","last_undegraded":"2024-09-17T05:30:23.949074+0000","last_fullsized":"2024-09-17T05:30:23.949074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:47.160401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864815+0000","last_change":"2024-09-17T05:30:24.974140+0000","last_active":"2024-09-17T05:30:27.864815+0000","last_peered":"2024-09-17T05:30:27.864815+0000","last_clean":"2024-09-17T05:30:27.864815+0000","last_became_active":"2024-09-17T05:30:24.973833+0000","last_became_peered":"2024-09-17T05:30:24.973833+0000","last_unstale":"2024-09-17T05:30:27.864815+0000","last_undegraded":"2024-09-17T05:30:27.864815+0000","last_fullsized":"2024-09-17T05:30:27.864815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:28.597753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952758+0000","last_change":"2024-09-17T05:30:23.946652+0000","last_active":"2024-09-17T05:30:23.952758+0000","last_peered":"2024-09-17T05:30:23.952758+0000","last_clean":"2024-09-17T05:30:23.952758+0000","last_became_active":"2024-09-17T05:30:23.946243+0000","last_became_peered":"2024-09-17T05:30:23.946243+0000","last_unstale":"2024-09-17T05:30:23.952758+0000","last_undegraded":"2024-09-17T05:30:23.952758+0000","last_fullsized":"2024-09-17T05:30:23.952758+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:55.869955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.939243+0000","last_change":"2024-09-17T05:30:23.939243+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.939243+0000","last_undegraded":"2024-09-17T05:30:23.939243+0000","last_fullsized":"2024-09-17T05:30:23.939243+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.958482+0000","last_change":"2024-09-17T05:30:23.953938+0000","last_active":"2024-09-17T05:30:23.958482+0000","last_peered":"2024-09-17T05:30:23.958482+0000","last_clean":"2024-09-17T05:30:23.958482+0000","last_became_active":"2024-09-17T05:30:23.953683+0000","last_became_peered":"2024-09-17T05:30:23.953683+0000","last_unstale":"2024-09-17T05:30:23.958482+0000","last_undegraded":"2024-09-17T05:30:23.958482+0000","last_fullsized":"2024-09-17T05:30:23.958482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:44.089365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934227+0000","last_change":"2024-09-17T05:30:23.934227+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934227+0000","last_undegraded":"2024-09-17T05:30:23.934227+0000","last_fullsized":"2024-09-17T05:30:23.934227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.951311+0000","last_change":"2024-09-17T05:30:23.948325+0000","last_active":"2024-09-17T05:30:23.951311+0000","last_peered":"2024-09-17T05:30:23.951311+0000","last_clean":"2024-09-17T05:30:23.951311+0000","last_became_active":"2024-09-17T05:30:23.948001+0000","last_became_peered":"2024-09-17T05:30:23.948001+0000","last_unstale":"2024-09-17T05:30:23.951311+0000","last_undegraded":"2024-09-17T05:30:23.951311+0000","last_fullsized":"2024-09-17T05:30:23.951311+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:07.032087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937367+0000","last_change":"2024-09-17T05:30:23.937367+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937367+0000","last_undegraded":"2024-09-17T05:30:23.937367+0000","last_fullsized":"2024-09-17T05:30:23.937367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956908+0000","last_change":"2024-09-17T05:30:23.954282+0000","last_active":"2024-09-17T05:30:23.956908+0000","last_peered":"2024-09-17T05:30:23.956908+0000","last_clean":"2024-09-17T05:30:23.956908+0000","last_became_active":"2024-09-17T05:30:23.953881+0000","last_became_peered":"2024-09-17T05:30:23.953881+0000","last_unstale":"2024-09-17T05:30:23.956908+0000","last_undegraded":"2024-09-17T05:30:23.956908+0000","last_fullsized":"2024-09-17T05:30:23.956908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:55.869955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.940305+0000","last_change":"2024-09-17T05:30:23.940305+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.940305+0000","last_undegraded":"2024-09-17T05:30:23.940305+0000","last_fullsized":"2024-09-17T05:30:23.940305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.946897+0000","last_change":"2024-09-17T05:30:23.943335+0000","last_active":"2024-09-17T05:30:23.946897+0000","last_peered":"2024-09-17T05:30:23.946897+0000","last_clean":"2024-09-17T05:30:23.946897+0000","last_became_active":"2024-09-17T05:30:23.943018+0000","last_became_peered":"2024-09-17T05:30:23.943018+0000","last_unstale":"2024-09-17T05:30:23.946897+0000","last_undegraded":"2024-09-17T05:30:23.946897+0000","last_fullsized":"2024-09-17T05:30:23.946897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:45.720384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937983+0000","last_change":"2024-09-17T05:30:23.937983+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937983+0000","last_undegraded":"2024-09-17T05:30:23.937983+0000","last_fullsized":"2024-09-17T05:30:23.937983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954980+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954714+0000","last_became_peered":"2024-09-17T05:30:23.954714+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:24.175086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864850+0000","last_change":"2024-09-17T05:30:24.976889+0000","last_active":"2024-09-17T05:30:27.864850+0000","last_peered":"2024-09-17T05:30:27.864850+0000","last_clean":"2024-09-17T05:30:27.864850+0000","last_became_active":"2024-09-17T05:30:24.976636+0000","last_became_peered":"2024-09-17T05:30:24.976636+0000","last_unstale":"2024-09-17T05:30:27.864850+0000","last_undegraded":"2024-09-17T05:30:27.864850+0000","last_fullsized":"2024-09-17T05:30:27.864850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:54.276295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.948946+0000","last_change":"2024-09-17T05:30:23.942665+0000","last_active":"2024-09-17T05:30:23.948946+0000","last_peered":"2024-09-17T05:30:23.948946+0000","last_clean":"2024-09-17T05:30:23.948946+0000","last_became_active":"2024-09-17T05:30:23.942377+0000","last_became_peered":"2024-09-17T05:30:23.942377+0000","last_unstale":"2024-09-17T05:30:23.948946+0000","last_undegraded":"2024-09-17T05:30:23.948946+0000","last_fullsized":"2024-09-17T05:30:23.948946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:35.760533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.932249+0000","last_change":"2024-09-17T05:30:23.932249+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.932249+0000","last_undegraded":"2024-09-17T05:30:23.932249+0000","last_fullsized":"2024-09-17T05:30:23.932249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956867+0000","last_change":"2024-09-17T05:30:23.953324+0000","last_active":"2024-09-17T05:30:23.956867+0000","last_peered":"2024-09-17T05:30:23.956867+0000","last_clean":"2024-09-17T05:30:23.956867+0000","last_became_active":"2024-09-17T05:30:23.953049+0000","last_became_peered":"2024-09-17T05:30:23.953049+0000","last_unstale":"2024-09-17T05:30:23.956867+0000","last_undegraded":"2024-09-17T05:30:23.956867+0000","last_fullsized":"2024-09-17T05:30:23.956867+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:58.939352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935005+0000","last_change":"2024-09-17T05:30:23.935005+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935005+0000","last_undegraded":"2024-09-17T05:30:23.935005+0000","last_fullsized":"2024-09-17T05:30:23.935005+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954792+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954435+0000","last_became_peered":"2024-09-17T05:30:23.954435+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:30.663318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934544+0000","last_change":"2024-09-17T05:30:23.934544+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934544+0000","last_undegraded":"2024-09-17T05:30:23.934544+0000","last_fullsized":"2024-09-17T05:30:23.934544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949916+0000","last_change":"2024-09-17T05:30:23.948409+0000","last_active":"2024-09-17T05:30:23.949916+0000","last_peered":"2024-09-17T05:30:23.949916+0000","last_clean":"2024-09-17T05:30:23.949916+0000","last_became_active":"2024-09-17T05:30:23.948114+0000","last_became_peered":"2024-09-17T05:30:23.948114+0000","last_unstale":"2024-09-17T05:30:23.949916+0000","last_undegraded":"2024-09-17T05:30:23.949916+0000","last_fullsized":"2024-09-17T05:30:23.949916+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:53.478330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934022+0000","last_change":"2024-09-17T05:30:23.934022+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934022+0000","last_undegraded":"2024-09-17T05:30:23.934022+0000","last_fullsized":"2024-09-17T05:30:23.934022+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.948904+0000","last_change":"2024-09-17T05:30:23.943942+0000","last_active":"2024-09-17T05:30:23.948904+0000","last_peered":"2024-09-17T05:30:23.948904+0000","last_clean":"2024-09-17T05:30:23.948904+0000","last_became_active":"2024-09-17T05:30:23.943669+0000","last_became_peered":"2024-09-17T05:30:23.943669+0000","last_unstale":"2024-09-17T05:30:23.948904+0000","last_undegraded":"2024-09-17T05:30:23.948904+0000","last_fullsized":"2024-09-17T05:30:23.948904+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:38.122369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931093+0000","last_change":"2024-09-17T05:30:23.931093+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931093+0000","last_undegraded":"2024-09-17T05:30:23.931093+0000","last_fullsized":"2024-09-17T05:30:23.931093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.949792+0000","last_change":"2024-09-17T05:30:23.949103+0000","last_active":"2024-09-17T05:30:23.949792+0000","last_peered":"2024-09-17T05:30:23.949792+0000","last_clean":"2024-09-17T05:30:23.949792+0000","last_became_active":"2024-09-17T05:30:23.948810+0000","last_became_peered":"2024-09-17T05:30:23.948810+0000","last_unstale":"2024-09-17T05:30:23.949792+0000","last_undegraded":"2024-09-17T05:30:23.949792+0000","last_fullsized":"2024-09-17T05:30:23.949792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:45.173108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.938331+0000","last_change":"2024-09-17T05:30:23.938331+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.938331+0000","last_undegraded":"2024-09-17T05:30:23.938331+0000","last_fullsized":"2024-09-17T05:30:23.938331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.961778+0000","last_change":"2024-09-17T05:30:23.960591+0000","last_active":"2024-09-17T05:30:23.961778+0000","last_peered":"2024-09-17T05:30:23.961778+0000","last_clean":"2024-09-17T05:30:23.961778+0000","last_became_active":"2024-09-17T05:30:23.960068+0000","last_became_peered":"2024-09-17T05:30:23.960068+0000","last_unstale":"2024-09-17T05:30:23.961778+0000","last_undegraded":"2024-09-17T05:30:23.961778+0000","last_fullsized":"2024-09-17T05:30:23.961778+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:32.536820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937514+0000","last_change":"2024-09-17T05:30:23.937514+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937514+0000","last_undegraded":"2024-09-17T05:30:23.937514+0000","last_fullsized":"2024-09-17T05:30:23.937514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864645+0000","last_change":"2024-09-17T05:30:23.951223+0000","last_active":"2024-09-17T05:30:27.864645+0000","last_peered":"2024-09-17T05:30:27.864645+0000","last_clean":"2024-09-17T05:30:27.864645+0000","last_became_active":"2024-09-17T05:30:23.950514+0000","last_became_peered":"2024-09-17T05:30:23.950514+0000","last_unstale":"2024-09-17T05:30:27.864645+0000","last_undegraded":"2024-09-17T05:30:27.864645+0000","last_fullsized":"2024-09-17T05:30:27.864645+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:45.736482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.936819+0000","last_change":"2024-09-17T05:30:23.936819+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.936819+0000","last_undegraded":"2024-09-17T05:30:23.936819+0000","last_fullsized":"2024-09-17T05:30:23.936819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952033+0000","last_change":"2024-09-17T05:30:23.944667+0000","last_active":"2024-09-17T05:30:23.952033+0000","last_peered":"2024-09-17T05:30:23.952033+0000","last_clean":"2024-09-17T05:30:23.952033+0000","last_became_active":"2024-09-17T05:30:23.944381+0000","last_became_peered":"2024-09-17T05:30:23.944381+0000","last_unstale":"2024-09-17T05:30:23.952033+0000","last_undegraded":"2024-09-17T05:30:23.952033+0000","last_fullsized":"2024-09-17T05:30:23.952033+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:27.536358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.931506+0000","last_change":"2024-09-17T05:30:23.931506+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.931506+0000","last_undegraded":"2024-09-17T05:30:23.931506+0000","last_fullsized":"2024-09-17T05:30:23.931506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864732+0000","last_change":"2024-09-17T05:30:23.960543+0000","last_active":"2024-09-17T05:30:27.864732+0000","last_peered":"2024-09-17T05:30:27.864732+0000","last_clean":"2024-09-17T05:30:27.864732+0000","last_became_active":"2024-09-17T05:30:23.960241+0000","last_became_peered":"2024-09-17T05:30:23.960241+0000","last_unstale":"2024-09-17T05:30:27.864732+0000","last_undegraded":"2024-09-17T05:30:27.864732+0000","last_fullsized":"2024-09-17T05:30:27.864732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:25.924845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934516+0000","last_change":"2024-09-17T05:30:23.934516+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934516+0000","last_undegraded":"2024-09-17T05:30:23.934516+0000","last_fullsized":"2024-09-17T05:30:23.934516+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.955243+0000","last_change":"2024-09-17T05:30:23.951379+0000","last_active":"2024-09-17T05:30:23.955243+0000","last_peered":"2024-09-17T05:30:23.955243+0000","last_clean":"2024-09-17T05:30:23.955243+0000","last_became_active":"2024-09-17T05:30:23.950900+0000","last_became_peered":"2024-09-17T05:30:23.950900+0000","last_unstale":"2024-09-17T05:30:23.955243+0000","last_undegraded":"2024-09-17T05:30:23.955243+0000","last_fullsized":"2024-09-17T05:30:23.955243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:54.643188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864503+0000","last_change":"2024-09-17T05:30:24.973721+0000","last_active":"2024-09-17T05:30:27.864503+0000","last_peered":"2024-09-17T05:30:27.864503+0000","last_clean":"2024-09-17T05:30:27.864503+0000","last_became_active":"2024-09-17T05:30:24.973391+0000","last_became_peered":"2024-09-17T05:30:24.973391+0000","last_unstale":"2024-09-17T05:30:27.864503+0000","last_undegraded":"2024-09-17T05:30:27.864503+0000","last_fullsized":"2024-09-17T05:30:27.864503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:02.895238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864244+0000","last_change":"2024-09-17T05:30:23.950654+0000","last_active":"2024-09-17T05:30:27.864244+0000","last_peered":"2024-09-17T05:30:27.864244+0000","last_clean":"2024-09-17T05:30:27.864244+0000","last_became_active":"2024-09-17T05:30:23.950372+0000","last_became_peered":"2024-09-17T05:30:23.950372+0000","last_unstale":"2024-09-17T05:30:27.864244+0000","last_undegraded":"2024-09-17T05:30:27.864244+0000","last_fullsized":"2024-09-17T05:30:27.864244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:46.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864458+0000","last_change":"2024-09-17T05:30:24.964323+0000","last_active":"2024-09-17T05:30:27.864458+0000","last_peered":"2024-09-17T05:30:27.864458+0000","last_clean":"2024-09-17T05:30:27.864458+0000","last_became_active":"2024-09-17T05:30:24.964039+0000","last_became_peered":"2024-09-17T05:30:24.964039+0000","last_unstale":"2024-09-17T05:30:27.864458+0000","last_undegraded":"2024-09-17T05:30:27.864458+0000","last_fullsized":"2024-09-17T05:30:27.864458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:52.710209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.951246+0000","last_change":"2024-09-17T05:30:23.946216+0000","last_active":"2024-09-17T05:30:23.951246+0000","last_peered":"2024-09-17T05:30:23.951246+0000","last_clean":"2024-09-17T05:30:23.951246+0000","last_became_active":"2024-09-17T05:30:23.945797+0000","last_became_peered":"2024-09-17T05:30:23.945797+0000","last_unstale":"2024-09-17T05:30:23.951246+0000","last_undegraded":"2024-09-17T05:30:23.951246+0000","last_fullsized":"2024-09-17T05:30:23.951246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:16.565367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937877+0000","last_change":"2024-09-17T05:30:23.937877+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937877+0000","last_undegraded":"2024-09-17T05:30:23.937877+0000","last_fullsized":"2024-09-17T05:30:23.937877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.957639+0000","last_change":"2024-09-17T05:30:23.953693+0000","last_active":"2024-09-17T05:30:23.957639+0000","last_peered":"2024-09-17T05:30:23.957639+0000","last_clean":"2024-09-17T05:30:23.957639+0000","last_became_active":"2024-09-17T05:30:23.953424+0000","last_became_peered":"2024-09-17T05:30:23.953424+0000","last_unstale":"2024-09-17T05:30:23.957639+0000","last_undegraded":"2024-09-17T05:30:23.957639+0000","last_fullsized":"2024-09-17T05:30:23.957639+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:19.341040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937107+0000","last_change":"2024-09-17T05:30:23.937107+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937107+0000","last_undegraded":"2024-09-17T05:30:23.937107+0000","last_fullsized":"2024-09-17T05:30:23.937107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.959458+0000","last_change":"2024-09-17T05:30:23.958649+0000","last_active":"2024-09-17T05:30:23.959458+0000","last_peered":"2024-09-17T05:30:23.959458+0000","last_clean":"2024-09-17T05:30:23.959458+0000","last_became_active":"2024-09-17T05:30:23.958329+0000","last_became_peered":"2024-09-17T05:30:23.958329+0000","last_unstale":"2024-09-17T05:30:23.959458+0000","last_undegraded":"2024-09-17T05:30:23.959458+0000","last_fullsized":"2024-09-17T05:30:23.959458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.772225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.935536+0000","last_change":"2024-09-17T05:30:23.935536+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.935536+0000","last_undegraded":"2024-09-17T05:30:23.935536+0000","last_fullsized":"2024-09-17T05:30:23.935536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.950069+0000","last_change":"2024-09-17T05:30:23.943837+0000","last_active":"2024-09-17T05:30:23.950069+0000","last_peered":"2024-09-17T05:30:23.950069+0000","last_clean":"2024-09-17T05:30:23.950069+0000","last_became_active":"2024-09-17T05:30:23.942677+0000","last_became_peered":"2024-09-17T05:30:23.942677+0000","last_unstale":"2024-09-17T05:30:23.950069+0000","last_undegraded":"2024-09-17T05:30:23.950069+0000","last_fullsized":"2024-09-17T05:30:23.950069+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:26.543985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864917+0000","last_change":"2024-09-17T05:30:24.974072+0000","last_active":"2024-09-17T05:30:27.864917+0000","last_peered":"2024-09-17T05:30:27.864917+0000","last_clean":"2024-09-17T05:30:27.864917+0000","last_became_active":"2024-09-17T05:30:24.973224+0000","last_became_peered":"2024-09-17T05:30:24.973224+0000","last_unstale":"2024-09-17T05:30:27.864917+0000","last_undegraded":"2024-09-17T05:30:27.864917+0000","last_fullsized":"2024-09-17T05:30:27.864917+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:35.858572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.953004+0000","last_change":"2024-09-17T05:30:23.947956+0000","last_active":"2024-09-17T05:30:23.953004+0000","last_peered":"2024-09-17T05:30:23.953004+0000","last_clean":"2024-09-17T05:30:23.953004+0000","last_became_active":"2024-09-17T05:30:23.947641+0000","last_became_peered":"2024-09-17T05:30:23.947641+0000","last_unstale":"2024-09-17T05:30:23.953004+0000","last_undegraded":"2024-09-17T05:30:23.953004+0000","last_fullsized":"2024-09-17T05:30:23.953004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:41.752305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.937632+0000","last_change":"2024-09-17T05:30:23.937632+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.937632+0000","last_undegraded":"2024-09-17T05:30:23.937632+0000","last_fullsized":"2024-09-17T05:30:23.937632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.957936+0000","last_change":"2024-09-17T05:30:23.957320+0000","last_active":"2024-09-17T05:30:23.957936+0000","last_peered":"2024-09-17T05:30:23.957936+0000","last_clean":"2024-09-17T05:30:23.957936+0000","last_became_active":"2024-09-17T05:30:23.956956+0000","last_became_peered":"2024-09-17T05:30:23.956956+0000","last_unstale":"2024-09-17T05:30:23.957936+0000","last_undegraded":"2024-09-17T05:30:23.957936+0000","last_fullsized":"2024-09-17T05:30:23.957936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:52.455342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.942385+0000","last_change":"2024-09-17T05:30:23.942385+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.942385+0000","last_undegraded":"2024-09-17T05:30:23.942385+0000","last_fullsized":"2024-09-17T05:30:23.942385+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.952593+0000","last_change":"2024-09-17T05:30:23.945568+0000","last_active":"2024-09-17T05:30:23.952593+0000","last_peered":"2024-09-17T05:30:23.952593+0000","last_clean":"2024-09-17T05:30:23.952593+0000","last_became_active":"2024-09-17T05:30:23.945279+0000","last_became_peered":"2024-09-17T05:30:23.945279+0000","last_unstale":"2024-09-17T05:30:23.952593+0000","last_undegraded":"2024-09-17T05:30:23.952593+0000","last_fullsized":"2024-09-17T05:30:23.952593+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:58.939352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864556+0000","last_change":"2024-09-17T05:30:24.974524+0000","last_active":"2024-09-17T05:30:27.864556+0000","last_peered":"2024-09-17T05:30:27.864556+0000","last_clean":"2024-09-17T05:30:27.864556+0000","last_became_active":"2024-09-17T05:30:24.974227+0000","last_became_peered":"2024-09-17T05:30:24.974227+0000","last_unstale":"2024-09-17T05:30:27.864556+0000","last_undegraded":"2024-09-17T05:30:27.864556+0000","last_fullsized":"2024-09-17T05:30:27.864556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:47.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.961936+0000","last_change":"2024-09-17T05:30:23.960348+0000","last_active":"2024-09-17T05:30:23.961936+0000","last_peered":"2024-09-17T05:30:23.961936+0000","last_clean":"2024-09-17T05:30:23.961936+0000","last_became_active":"2024-09-17T05:30:23.960056+0000","last_became_peered":"2024-09-17T05:30:23.960056+0000","last_unstale":"2024-09-17T05:30:23.961936+0000","last_undegraded":"2024-09-17T05:30:23.961936+0000","last_fullsized":"2024-09-17T05:30:23.961936+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:19.075142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-17T05:30:23.934587+0000","last_change":"2024-09-17T05:30:23.934587+0000","last_active":"2024-09-17T05:30:23.922126+0000","last_peered":"2024-09-17T05:30:23.922126+0000","last_clean":"2024-09-17T05:30:23.922126+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:30:23.934587+0000","last_undegraded":"2024-09-17T05:30:23.934587+0000","last_fullsized":"2024-09-17T05:30:23.934587+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864474+0000","last_change":"2024-09-17T05:30:23.956301+0000","last_active":"2024-09-17T05:30:27.864474+0000","last_peered":"2024-09-17T05:30:27.864474+0000","last_clean":"2024-09-17T05:30:27.864474+0000","last_became_active":"2024-09-17T05:30:23.955995+0000","last_became_peered":"2024-09-17T05:30:23.955995+0000","last_unstale":"2024-09-17T05:30:27.864474+0000","last_undegraded":"2024-09-17T05:30:27.864474+0000","last_fullsized":"2024-09-17T05:30:27.864474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:02.746869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864899+0000","last_change":"2024-09-17T05:30:24.973772+0000","last_active":"2024-09-17T05:30:27.864899+0000","last_peered":"2024-09-17T05:30:27.864899+0000","last_clean":"2024-09-17T05:30:27.864899+0000","last_became_active":"2024-09-17T05:30:24.972340+0000","last_became_peered":"2024-09-17T05:30:24.972340+0000","last_unstale":"2024-09-17T05:30:27.864899+0000","last_undegraded":"2024-09-17T05:30:27.864899+0000","last_fullsized":"2024-09-17T05:30:27.864899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:43.483847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.956830+0000","last_change":"2024-09-17T05:30:23.952489+0000","last_active":"2024-09-17T05:30:23.956830+0000","last_peered":"2024-09-17T05:30:23.956830+0000","last_clean":"2024-09-17T05:30:23.956830+0000","last_became_active":"2024-09-17T05:30:23.952212+0000","last_became_peered":"2024-09-17T05:30:23.952212+0000","last_unstale":"2024-09-17T05:30:23.956830+0000","last_undegraded":"2024-09-17T05:30:23.956830+0000","last_fullsized":"2024-09-17T05:30:23.956830+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:27.536358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":179,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T05:30:23.928112+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:30:23.928112+0000","last_peered":"2024-09-17T05:30:23.928112+0000","last_clean":"2024-09-17T05:30:23.928112+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:30:23.928112+0000","last_undegraded":"2024-09-17T05:30:23.928112+0000","last_fullsized":"2024-09-17T05:30:23.928112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40168,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331672,"statfs":{"total":96636764160,"available":96595632128,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40172,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331668,"statfs":{"total":96636764160,"available":96595628032,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509509,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45736,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326104,"statfs":{"total":96636764160,"available":96589930496,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509508,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40176,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331664,"statfs":{"total":96636764160,"available":96595623936,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509509,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58220,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313620,"statfs":{"total":96636764160,"available":96577146880,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509510,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40748,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331092,"statfs":{"total":96636764160,"available":96595038208,"internally_reserved":0,"allocated":1269760,"data_stored":868797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40184,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331656,"statfs":{"total":96636764160,"available":96595615744,"internally_reserved":0,"allocated":675840,"data_stored":278429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1310720,"data_stored":888265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:30:29.849 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3f is not active+clean 2024-09-17T05:30:29.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.946789+0000', 'last_change': '2024-09-17T05:30:23.946789+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.946789+0000', 'last_undegraded': '2024-09-17T05:30:23.946789+0000', 'last_fullsized': '2024-09-17T05:30:23.946789+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.849 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3e is not active+clean 2024-09-17T05:30:29.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.935503+0000', 'last_change': '2024-09-17T05:30:23.935503+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.935503+0000', 'last_undegraded': '2024-09-17T05:30:23.935503+0000', 'last_fullsized': '2024-09-17T05:30:23.935503+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:30:29.849 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3d is not active+clean 2024-09-17T05:30:29.849 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937682+0000', 'last_change': '2024-09-17T05:30:23.937682+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937682+0000', 'last_undegraded': '2024-09-17T05:30:23.937682+0000', 'last_fullsized': '2024-09-17T05:30:23.937682+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3c is not active+clean 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.932092+0000', 'last_change': '2024-09-17T05:30:23.932092+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.932092+0000', 'last_undegraded': '2024-09-17T05:30:23.932092+0000', 'last_fullsized': '2024-09-17T05:30:23.932092+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3b is not active+clean 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937192+0000', 'last_change': '2024-09-17T05:30:23.937192+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937192+0000', 'last_undegraded': '2024-09-17T05:30:23.937192+0000', 'last_fullsized': '2024-09-17T05:30:23.937192+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3a is not active+clean 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.941621+0000', 'last_change': '2024-09-17T05:30:23.941621+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.941621+0000', 'last_undegraded': '2024-09-17T05:30:23.941621+0000', 'last_fullsized': '2024-09-17T05:30:23.941621+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:PG 9.39 is not active+clean 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.932950+0000', 'last_change': '2024-09-17T05:30:23.932950+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.932950+0000', 'last_undegraded': '2024-09-17T05:30:23.932950+0000', 'last_fullsized': '2024-09-17T05:30:23.932950+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:PG 9.37 is not active+clean 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.935531+0000', 'last_change': '2024-09-17T05:30:23.935531+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.935531+0000', 'last_undegraded': '2024-09-17T05:30:23.935531+0000', 'last_fullsized': '2024-09-17T05:30:23.935531+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:PG 9.36 is not active+clean 2024-09-17T05:30:29.850 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.946112+0000', 'last_change': '2024-09-17T05:30:23.946112+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.946112+0000', 'last_undegraded': '2024-09-17T05:30:23.946112+0000', 'last_fullsized': '2024-09-17T05:30:23.946112+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:PG 9.35 is not active+clean 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.939489+0000', 'last_change': '2024-09-17T05:30:23.939489+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.939489+0000', 'last_undegraded': '2024-09-17T05:30:23.939489+0000', 'last_fullsized': '2024-09-17T05:30:23.939489+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:PG 9.34 is not active+clean 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.935459+0000', 'last_change': '2024-09-17T05:30:23.935459+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.935459+0000', 'last_undegraded': '2024-09-17T05:30:23.935459+0000', 'last_fullsized': '2024-09-17T05:30:23.935459+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:PG 9.33 is not active+clean 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.943180+0000', 'last_change': '2024-09-17T05:30:23.943180+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.943180+0000', 'last_undegraded': '2024-09-17T05:30:23.943180+0000', 'last_fullsized': '2024-09-17T05:30:23.943180+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:PG 9.32 is not active+clean 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.934423+0000', 'last_change': '2024-09-17T05:30:23.934423+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.934423+0000', 'last_undegraded': '2024-09-17T05:30:23.934423+0000', 'last_fullsized': '2024-09-17T05:30:23.934423+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:PG 9.31 is not active+clean 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.941670+0000', 'last_change': '2024-09-17T05:30:23.941670+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.941670+0000', 'last_undegraded': '2024-09-17T05:30:23.941670+0000', 'last_fullsized': '2024-09-17T05:30:23.941670+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:30:29.851 INFO:tasks.ceph.ceph_manager.ceph:PG 9.30 is not active+clean 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937557+0000', 'last_change': '2024-09-17T05:30:23.937557+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937557+0000', 'last_undegraded': '2024-09-17T05:30:23.937557+0000', 'last_fullsized': '2024-09-17T05:30:23.937557+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2f is not active+clean 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.946333+0000', 'last_change': '2024-09-17T05:30:23.946333+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.946333+0000', 'last_undegraded': '2024-09-17T05:30:23.946333+0000', 'last_fullsized': '2024-09-17T05:30:23.946333+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2e is not active+clean 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.938746+0000', 'last_change': '2024-09-17T05:30:23.938746+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.938746+0000', 'last_undegraded': '2024-09-17T05:30:23.938746+0000', 'last_fullsized': '2024-09-17T05:30:23.938746+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2d is not active+clean 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.936163+0000', 'last_change': '2024-09-17T05:30:23.936163+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.936163+0000', 'last_undegraded': '2024-09-17T05:30:23.936163+0000', 'last_fullsized': '2024-09-17T05:30:23.936163+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2c is not active+clean 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.939976+0000', 'last_change': '2024-09-17T05:30:23.939976+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.939976+0000', 'last_undegraded': '2024-09-17T05:30:23.939976+0000', 'last_fullsized': '2024-09-17T05:30:23.939976+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2b is not active+clean 2024-09-17T05:30:29.852 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.942782+0000', 'last_change': '2024-09-17T05:30:23.942782+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.942782+0000', 'last_undegraded': '2024-09-17T05:30:23.942782+0000', 'last_fullsized': '2024-09-17T05:30:23.942782+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2a is not active+clean 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.938893+0000', 'last_change': '2024-09-17T05:30:23.938893+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.938893+0000', 'last_undegraded': '2024-09-17T05:30:23.938893+0000', 'last_fullsized': '2024-09-17T05:30:23.938893+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:PG 9.29 is not active+clean 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.932460+0000', 'last_change': '2024-09-17T05:30:23.932460+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.932460+0000', 'last_undegraded': '2024-09-17T05:30:23.932460+0000', 'last_fullsized': '2024-09-17T05:30:23.932460+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:PG 9.28 is not active+clean 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.936101+0000', 'last_change': '2024-09-17T05:30:23.936101+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.936101+0000', 'last_undegraded': '2024-09-17T05:30:23.936101+0000', 'last_fullsized': '2024-09-17T05:30:23.936101+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:PG 9.27 is not active+clean 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.935649+0000', 'last_change': '2024-09-17T05:30:23.935649+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.935649+0000', 'last_undegraded': '2024-09-17T05:30:23.935649+0000', 'last_fullsized': '2024-09-17T05:30:23.935649+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:PG 9.26 is not active+clean 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.941579+0000', 'last_change': '2024-09-17T05:30:23.941579+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.941579+0000', 'last_undegraded': '2024-09-17T05:30:23.941579+0000', 'last_fullsized': '2024-09-17T05:30:23.941579+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:PG 9.25 is not active+clean 2024-09-17T05:30:29.853 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.935721+0000', 'last_change': '2024-09-17T05:30:23.935721+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.935721+0000', 'last_undegraded': '2024-09-17T05:30:23.935721+0000', 'last_fullsized': '2024-09-17T05:30:23.935721+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:PG 9.22 is not active+clean 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.931376+0000', 'last_change': '2024-09-17T05:30:23.931376+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.931376+0000', 'last_undegraded': '2024-09-17T05:30:23.931376+0000', 'last_fullsized': '2024-09-17T05:30:23.931376+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:PG 9.21 is not active+clean 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.936553+0000', 'last_change': '2024-09-17T05:30:23.936553+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.936553+0000', 'last_undegraded': '2024-09-17T05:30:23.936553+0000', 'last_fullsized': '2024-09-17T05:30:23.936553+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:PG 9.20 is not active+clean 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.940234+0000', 'last_change': '2024-09-17T05:30:23.940234+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.940234+0000', 'last_undegraded': '2024-09-17T05:30:23.940234+0000', 'last_fullsized': '2024-09-17T05:30:23.940234+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1e is not active+clean 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.945686+0000', 'last_change': '2024-09-17T05:30:23.945686+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.945686+0000', 'last_undegraded': '2024-09-17T05:30:23.945686+0000', 'last_fullsized': '2024-09-17T05:30:23.945686+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1f is not active+clean 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.945807+0000', 'last_change': '2024-09-17T05:30:23.945807+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.945807+0000', 'last_undegraded': '2024-09-17T05:30:23.945807+0000', 'last_fullsized': '2024-09-17T05:30:23.945807+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.854 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1d is not active+clean 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.943960+0000', 'last_change': '2024-09-17T05:30:23.943960+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.943960+0000', 'last_undegraded': '2024-09-17T05:30:23.943960+0000', 'last_fullsized': '2024-09-17T05:30:23.943960+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1b is not active+clean 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.939243+0000', 'last_change': '2024-09-17T05:30:23.939243+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.939243+0000', 'last_undegraded': '2024-09-17T05:30:23.939243+0000', 'last_fullsized': '2024-09-17T05:30:23.939243+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:PG 9.18 is not active+clean 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.934227+0000', 'last_change': '2024-09-17T05:30:23.934227+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.934227+0000', 'last_undegraded': '2024-09-17T05:30:23.934227+0000', 'last_fullsized': '2024-09-17T05:30:23.934227+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:PG 9.19 is not active+clean 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937367+0000', 'last_change': '2024-09-17T05:30:23.937367+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937367+0000', 'last_undegraded': '2024-09-17T05:30:23.937367+0000', 'last_fullsized': '2024-09-17T05:30:23.937367+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:PG 9.16 is not active+clean 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.940305+0000', 'last_change': '2024-09-17T05:30:23.940305+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.940305+0000', 'last_undegraded': '2024-09-17T05:30:23.940305+0000', 'last_fullsized': '2024-09-17T05:30:23.940305+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:PG 9.17 is not active+clean 2024-09-17T05:30:29.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937983+0000', 'last_change': '2024-09-17T05:30:23.937983+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937983+0000', 'last_undegraded': '2024-09-17T05:30:23.937983+0000', 'last_fullsized': '2024-09-17T05:30:23.937983+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:PG 9.15 is not active+clean 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.932249+0000', 'last_change': '2024-09-17T05:30:23.932249+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.932249+0000', 'last_undegraded': '2024-09-17T05:30:23.932249+0000', 'last_fullsized': '2024-09-17T05:30:23.932249+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.935005+0000', 'last_change': '2024-09-17T05:30:23.935005+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.935005+0000', 'last_undegraded': '2024-09-17T05:30:23.935005+0000', 'last_fullsized': '2024-09-17T05:30:23.935005+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:PG 9.13 is not active+clean 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.934544+0000', 'last_change': '2024-09-17T05:30:23.934544+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.934544+0000', 'last_undegraded': '2024-09-17T05:30:23.934544+0000', 'last_fullsized': '2024-09-17T05:30:23.934544+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.934022+0000', 'last_change': '2024-09-17T05:30:23.934022+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.934022+0000', 'last_undegraded': '2024-09-17T05:30:23.934022+0000', 'last_fullsized': '2024-09-17T05:30:23.934022+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:PG 9.11 is not active+clean 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.931093+0000', 'last_change': '2024-09-17T05:30:23.931093+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.931093+0000', 'last_undegraded': '2024-09-17T05:30:23.931093+0000', 'last_fullsized': '2024-09-17T05:30:23.931093+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:30:29.856 INFO:tasks.ceph.ceph_manager.ceph:PG 9.e is not active+clean 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.938331+0000', 'last_change': '2024-09-17T05:30:23.938331+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.938331+0000', 'last_undegraded': '2024-09-17T05:30:23.938331+0000', 'last_fullsized': '2024-09-17T05:30:23.938331+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:PG 9.f is not active+clean 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937514+0000', 'last_change': '2024-09-17T05:30:23.937514+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937514+0000', 'last_undegraded': '2024-09-17T05:30:23.937514+0000', 'last_fullsized': '2024-09-17T05:30:23.937514+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.936819+0000', 'last_change': '2024-09-17T05:30:23.936819+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.936819+0000', 'last_undegraded': '2024-09-17T05:30:23.936819+0000', 'last_fullsized': '2024-09-17T05:30:23.936819+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:PG 9.d is not active+clean 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.931506+0000', 'last_change': '2024-09-17T05:30:23.931506+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.931506+0000', 'last_undegraded': '2024-09-17T05:30:23.931506+0000', 'last_fullsized': '2024-09-17T05:30:23.931506+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.934516+0000', 'last_change': '2024-09-17T05:30:23.934516+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.934516+0000', 'last_undegraded': '2024-09-17T05:30:23.934516+0000', 'last_fullsized': '2024-09-17T05:30:23.934516+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:PG 9.6 is not active+clean 2024-09-17T05:30:29.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937877+0000', 'last_change': '2024-09-17T05:30:23.937877+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937877+0000', 'last_undegraded': '2024-09-17T05:30:23.937877+0000', 'last_fullsized': '2024-09-17T05:30:23.937877+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:PG 9.7 is not active+clean 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937107+0000', 'last_change': '2024-09-17T05:30:23.937107+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937107+0000', 'last_undegraded': '2024-09-17T05:30:23.937107+0000', 'last_fullsized': '2024-09-17T05:30:23.937107+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.935536+0000', 'last_change': '2024-09-17T05:30:23.935536+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.935536+0000', 'last_undegraded': '2024-09-17T05:30:23.935536+0000', 'last_fullsized': '2024-09-17T05:30:23.935536+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2 is not active+clean 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.937632+0000', 'last_change': '2024-09-17T05:30:23.937632+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.937632+0000', 'last_undegraded': '2024-09-17T05:30:23.937632+0000', 'last_fullsized': '2024-09-17T05:30:23.937632+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3 is not active+clean 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.942385+0000', 'last_change': '2024-09-17T05:30:23.942385+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.942385+0000', 'last_undegraded': '2024-09-17T05:30:23.942385+0000', 'last_fullsized': '2024-09-17T05:30:23.942385+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-09-17T05:30:29.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:30:23.934587+0000', 'last_change': '2024-09-17T05:30:23.934587+0000', 'last_active': '2024-09-17T05:30:23.922126+0000', 'last_peered': '2024-09-17T05:30:23.922126+0000', 'last_clean': '2024-09-17T05:30:23.922126+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:30:23.934587+0000', 'last_undegraded': '2024-09-17T05:30:23.934587+0000', 'last_fullsized': '2024-09-17T05:30:23.934587+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'last_clean_scrub_stamp': '2024-09-17T05:30:23.922126+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:30:29.859 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:30:30.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.030+0000 7f13f7e00640 1 -- 172.21.15.122:0/548142659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 msgr2=0x7f13d8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.030+0000 7f13f7e00640 1 --2- 172.21.15.122:0/548142659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 0x7f13d8003e30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009b60 tx=0x7f13e401d240 comp rx=0 tx=0).stop 2024-09-17T05:30:30.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.030+0000 7f13f5e00640 1 -- 172.21.15.122:0/548142659 reap_dead start 2024-09-17T05:30:30.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.030+0000 7f13f7e00640 1 -- 172.21.15.122:0/548142659 shutdown_connections 2024-09-17T05:30:30.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.030+0000 7f13f7e00640 1 -- 172.21.15.122:0/548142659 >> 172.21.15.122:0/548142659 conn(0x7f13f00d0cd0 msgr2=0x7f13f00d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:30.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.030+0000 7f13f7e00640 1 -- 172.21.15.122:0/548142659 shutdown_connections 2024-09-17T05:30:30.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.030+0000 7f13f7e00640 1 -- 172.21.15.122:0/548142659 wait complete. 2024-09-17T05:30:30.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 Processor -- start 2024-09-17T05:30:30.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 -- start start 2024-09-17T05:30:30.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 0x7f13f00cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f01fac40 0x7f13f00b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f015ede0 0x7f13f015f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f13f00cfda0 con 0x7f13d8003a10 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f13f00cfc20 con 0x7f13f015ede0 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f13f00cb530 con 0x7f13f01fac40 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 0x7f13f00cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 0x7f13f00cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60566/0 (socket says 172.21.15.122:60566) 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 -- 172.21.15.122:0/372158871 learned_addr learned my addr 172.21.15.122:0/372158871 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 -- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f01fac40 msgr2=0x7f13f00b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f01fac40 0x7f13f00b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 -- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f015ede0 msgr2=0x7f13f015f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5e00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f015ede0 0x7f13f015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f015ede0 0x7f13f015f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 -- 172.21.15.122:0/372158871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f00b5670 con 0x7f13d8003a10 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5e00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f015ede0 0x7f13f015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f5400640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 0x7f13f00cabc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f13e401d070 tx=0x7f13e4005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13ed600640 1 -- 172.21.15.122:0/372158871 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e40b1040 con 0x7f13d8003a10 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13ed600640 1 -- 172.21.15.122:0/372158871 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13e40ac400 con 0x7f13d8003a10 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f01fb430 con 0x7f13d8003a10 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13ed600640 1 -- 172.21.15.122:0/372158871 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e4004290 con 0x7f13d8003a10 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.034+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f01fb9f0 con 0x7f13d8003a10 2024-09-17T05:30:30.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.038+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f006cc20 con 0x7f13d8003a10 2024-09-17T05:30:30.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.038+0000 7f13ed600640 1 -- 172.21.15.122:0/372158871 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f13e40bd020 con 0x7f13d8003a10 2024-09-17T05:30:30.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.038+0000 7f13ed600640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d0073b00 0x7f13d0075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.038+0000 7f13ed600640 1 -- 172.21.15.122:0/372158871 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f13e412dc80 con 0x7f13d8003a10 2024-09-17T05:30:30.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.038+0000 7f13f4a00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d0073b00 0x7f13d0075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.038+0000 7f13f4a00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d0073b00 0x7f13d0075fc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f13f00cc340 tx=0x7f13dc000ee0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:30.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.042+0000 7f13ed600640 1 -- 172.21.15.122:0/372158871 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f13e40c4020 con 0x7f13d8003a10 2024-09-17T05:30:30.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f13f015d720 con 0x7f13d8003a10 2024-09-17T05:30:30.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13ed600640 1 -- 172.21.15.122:0/372158871 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7f13e4016f80 con 0x7f13d8003a10 2024-09-17T05:30:30.270 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:30.270 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":351,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550693,"num_in_osds":8,"osd_in_since":1726550682,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":53},{"state_name":"active+clean","count":44}],"num_pgs":97,"num_pools":3,"num_objects":5,"data_bytes":590402,"bytes_used":360202240,"bytes_avail":772733911040,"bytes_total":773094113280,"inactive_pgs_ratio":0.54639172554016113,"write_bytes_sec":1279,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":36,"btime":"2024-09-17T05:30:27:977715+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":5931}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-17T05:29:12.508973+0000","services":{}},"progress_events":{}} 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d0073b00 msgr2=0x7f13d0075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d0073b00 0x7f13d0075fc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f13f00cc340 tx=0x7f13dc000ee0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 msgr2=0x7f13f00cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 0x7f13f00cabc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f13e401d070 tx=0x7f13e4005590 comp rx=0 tx=0).stop 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 shutdown_connections 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d0073b00 0x7f13d0075fc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f015ede0 0x7f13f015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f01fac40 0x7f13f00b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 --2- 172.21.15.122:0/372158871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13d8003a10 0x7f13f00cabc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 >> 172.21.15.122:0/372158871 conn(0x7f13f00d0cd0 msgr2=0x7f13f00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:30.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 shutdown_connections 2024-09-17T05:30:30.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.270+0000 7f13f7e00640 1 -- 172.21.15.122:0/372158871 wait complete. 2024-09-17T05:30:30.291 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:30:30.291 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:30:30.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- 172.21.15.122:0/1555053059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdac1615d0 msgr2=0x7fcdac0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1555053059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdac1615d0 0x7fcdac0aea90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009900 tx=0x7fcd9c01c820 comp rx=0 tx=0).stop 2024-09-17T05:30:30.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- 172.21.15.122:0/1555053059 shutdown_connections 2024-09-17T05:30:30.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1555053059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcdac0b0d40 0x7fcdac06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1555053059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdac1615d0 0x7fcdac0aea90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- 172.21.15.122:0/1555053059 >> 172.21.15.122:0/1555053059 conn(0x7fcdac0d1e30 msgr2=0x7fcdac0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:30.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- 172.21.15.122:0/1555053059 shutdown_connections 2024-09-17T05:30:30.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- 172.21.15.122:0/1555053059 wait complete. 2024-09-17T05:30:30.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 Processor -- start 2024-09-17T05:30:30.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- start start 2024-09-17T05:30:30.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdac0b0d40 0x7fcdac1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdac1615d0 0x7fcdac1fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcdac1619b0 0x7fcdac0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac0d1530 con 0x7fcdac0b0d40 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac0d13b0 con 0x7fcdac1615d0 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdb1600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcdac16b6c0 con 0x7fcdac1619b0 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdaaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdac0b0d40 0x7fcdac1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdaaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdac0b0d40 0x7fcdac1513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60568/0 (socket says 172.21.15.122:60568) 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.434+0000 7fcdaaa00640 1 -- 172.21.15.122:0/1832228381 learned_addr learned my addr 172.21.15.122:0/1832228381 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcda3e00640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdac1615d0 0x7fcdac1fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdab400640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcdac1619b0 0x7fcdac0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdaaa00640 1 -- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcdac1619b0 msgr2=0x7fcdac0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdaaa00640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcdac1619b0 0x7fcdac0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdaaa00640 1 -- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdac1615d0 msgr2=0x7fcdac1fbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdaaa00640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdac1615d0 0x7fcdac1fbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdaaa00640 1 -- 172.21.15.122:0/1832228381 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac1fc120 con 0x7fcdac0b0d40 2024-09-17T05:30:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdab400640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcdac1619b0 0x7fcdac0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdaaa00640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdac0b0d40 0x7fcdac1513b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009850 tx=0x7fcd9c0a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcda2a00640 1 -- 172.21.15.122:0/1832228381 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0b2040 con 0x7fcdac0b0d40 2024-09-17T05:30:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdac0b50b0 con 0x7fcdac0b0d40 2024-09-17T05:30:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcda2a00640 1 -- 172.21.15.122:0/1832228381 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd9c005030 con 0x7fcdac0b0d40 2024-09-17T05:30:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcda2a00640 1 -- 172.21.15.122:0/1832228381 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0044f0 con 0x7fcdac0b0d40 2024-09-17T05:30:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdac0b5670 con 0x7fcdac0b0d40 2024-09-17T05:30:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdac15e5f0 con 0x7fcdac0b0d40 2024-09-17T05:30:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.438+0000 7fcda2a00640 1 -- 172.21.15.122:0/1832228381 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fcd9c005460 con 0x7fcdac0b0d40 2024-09-17T05:30:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.442+0000 7fcda2a00640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd88073b00 0x7fcd88075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.442+0000 7fcda2a00640 1 -- 172.21.15.122:0/1832228381 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fcd9c12cc00 con 0x7fcdac0b0d40 2024-09-17T05:30:30.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.442+0000 7fcda3e00640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd88073b00 0x7fcd88075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:30.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.442+0000 7fcda3e00640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd88073b00 0x7fcd88075fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009600 tx=0x7fcd940091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:30.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.446+0000 7fcda2a00640 1 -- 172.21.15.122:0/1832228381 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcdac1fc120 con 0x7fcdac0b0d40 2024-09-17T05:30:30.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.606+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdac15e310 con 0x7fcd88073b00 2024-09-17T05:30:30.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.614+0000 7fcda2a00640 1 -- 172.21.15.122:0/1832228381 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fcdac15e310 con 0x7fcd88073b00 2024-09-17T05:30:30.615 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:30.617 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd88073b00 msgr2=0x7fcd88075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd88073b00 0x7fcd88075fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009600 tx=0x7fcd940091e0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdac0b0d40 msgr2=0x7fcdac1513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdac0b0d40 0x7fcdac1513b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009850 tx=0x7fcd9c0a57d0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 shutdown_connections 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd88073b00 0x7fcd88075fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcdac1619b0 0x7fcdac0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcdac1615d0 0x7fcdac1fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 --2- 172.21.15.122:0/1832228381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdac0b0d40 0x7fcdac1513b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 >> 172.21.15.122:0/1832228381 conn(0x7fcdac0d1e30 msgr2=0x7fcdac071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:30.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 shutdown_connections 2024-09-17T05:30:30.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:30.618+0000 7fcdb1600640 1 -- 172.21.15.122:0/1832228381 wait complete. 2024-09-17T05:30:30.637 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-17T05:30:30.533890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374280,"kb_used_data":6792,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754600440,"statfs":{"total":773094113280,"available":772710850560,"internally_reserved":0,"allocated":6955008,"data_stored":3568460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":376098358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.602895"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983368+0000","last_change":"2024-09-17T05:30:24.971099+0000","last_active":"2024-09-17T05:30:25.983368+0000","last_peered":"2024-09-17T05:30:25.983368+0000","last_clean":"2024-09-17T05:30:25.983368+0000","last_became_active":"2024-09-17T05:30:24.970644+0000","last_became_peered":"2024-09-17T05:30:24.970644+0000","last_unstale":"2024-09-17T05:30:25.983368+0000","last_undegraded":"2024-09-17T05:30:25.983368+0000","last_fullsized":"2024-09-17T05:30:25.983368+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:29.353770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959472+0000","last_change":"2024-09-17T05:30:24.969658+0000","last_active":"2024-09-17T05:30:25.959472+0000","last_peered":"2024-09-17T05:30:25.959472+0000","last_clean":"2024-09-17T05:30:25.959472+0000","last_became_active":"2024-09-17T05:30:24.968921+0000","last_became_peered":"2024-09-17T05:30:24.968921+0000","last_unstale":"2024-09-17T05:30:25.959472+0000","last_undegraded":"2024-09-17T05:30:25.959472+0000","last_fullsized":"2024-09-17T05:30:25.959472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:59.946299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954499+0000","last_change":"2024-09-17T05:30:24.956943+0000","last_active":"2024-09-17T05:30:25.954499+0000","last_peered":"2024-09-17T05:30:25.954499+0000","last_clean":"2024-09-17T05:30:25.954499+0000","last_became_active":"2024-09-17T05:30:24.956657+0000","last_became_peered":"2024-09-17T05:30:24.956657+0000","last_unstale":"2024-09-17T05:30:25.954499+0000","last_undegraded":"2024-09-17T05:30:25.954499+0000","last_fullsized":"2024-09-17T05:30:25.954499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:13.604844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956851+0000","last_change":"2024-09-17T05:30:24.962723+0000","last_active":"2024-09-17T05:30:25.956851+0000","last_peered":"2024-09-17T05:30:25.956851+0000","last_clean":"2024-09-17T05:30:25.956851+0000","last_became_active":"2024-09-17T05:30:24.962385+0000","last_became_peered":"2024-09-17T05:30:24.962385+0000","last_unstale":"2024-09-17T05:30:25.956851+0000","last_undegraded":"2024-09-17T05:30:25.956851+0000","last_fullsized":"2024-09-17T05:30:25.956851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:05.579221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982994+0000","last_change":"2024-09-17T05:30:24.963755+0000","last_active":"2024-09-17T05:30:25.982994+0000","last_peered":"2024-09-17T05:30:25.982994+0000","last_clean":"2024-09-17T05:30:25.982994+0000","last_became_active":"2024-09-17T05:30:24.963461+0000","last_became_peered":"2024-09-17T05:30:24.963461+0000","last_unstale":"2024-09-17T05:30:25.982994+0000","last_undegraded":"2024-09-17T05:30:25.982994+0000","last_fullsized":"2024-09-17T05:30:25.982994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:30.129620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982887+0000","last_change":"2024-09-17T05:30:24.970340+0000","last_active":"2024-09-17T05:30:25.982887+0000","last_peered":"2024-09-17T05:30:25.982887+0000","last_clean":"2024-09-17T05:30:25.982887+0000","last_became_active":"2024-09-17T05:30:24.969744+0000","last_became_peered":"2024-09-17T05:30:24.969744+0000","last_unstale":"2024-09-17T05:30:25.982887+0000","last_undegraded":"2024-09-17T05:30:25.982887+0000","last_fullsized":"2024-09-17T05:30:25.982887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:53.474035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956571+0000","last_change":"2024-09-17T05:30:24.961865+0000","last_active":"2024-09-17T05:30:25.956571+0000","last_peered":"2024-09-17T05:30:25.956571+0000","last_clean":"2024-09-17T05:30:25.956571+0000","last_became_active":"2024-09-17T05:30:24.961570+0000","last_became_peered":"2024-09-17T05:30:24.961570+0000","last_unstale":"2024-09-17T05:30:25.956571+0000","last_undegraded":"2024-09-17T05:30:25.956571+0000","last_fullsized":"2024-09-17T05:30:25.956571+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:42.626536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864357+0000","last_change":"2024-09-17T05:30:24.969090+0000","last_active":"2024-09-17T05:30:27.864357+0000","last_peered":"2024-09-17T05:30:27.864357+0000","last_clean":"2024-09-17T05:30:27.864357+0000","last_became_active":"2024-09-17T05:30:24.968736+0000","last_became_peered":"2024-09-17T05:30:24.968736+0000","last_unstale":"2024-09-17T05:30:27.864357+0000","last_undegraded":"2024-09-17T05:30:27.864357+0000","last_fullsized":"2024-09-17T05:30:27.864357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:13.357280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994100+0000","last_change":"2024-09-17T05:30:24.967509+0000","last_active":"2024-09-17T05:30:25.994100+0000","last_peered":"2024-09-17T05:30:25.994100+0000","last_clean":"2024-09-17T05:30:25.994100+0000","last_became_active":"2024-09-17T05:30:24.967197+0000","last_became_peered":"2024-09-17T05:30:24.967197+0000","last_unstale":"2024-09-17T05:30:25.994100+0000","last_undegraded":"2024-09-17T05:30:25.994100+0000","last_fullsized":"2024-09-17T05:30:25.994100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:13.435390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983331+0000","last_change":"2024-09-17T05:30:24.966805+0000","last_active":"2024-09-17T05:30:25.983331+0000","last_peered":"2024-09-17T05:30:25.983331+0000","last_clean":"2024-09-17T05:30:25.983331+0000","last_became_active":"2024-09-17T05:30:24.966550+0000","last_became_peered":"2024-09-17T05:30:24.966550+0000","last_unstale":"2024-09-17T05:30:25.983331+0000","last_undegraded":"2024-09-17T05:30:25.983331+0000","last_fullsized":"2024-09-17T05:30:25.983331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:40.647222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974503+0000","last_change":"2024-09-17T05:30:24.975269+0000","last_active":"2024-09-17T05:30:25.974503+0000","last_peered":"2024-09-17T05:30:25.974503+0000","last_clean":"2024-09-17T05:30:25.974503+0000","last_became_active":"2024-09-17T05:30:24.973279+0000","last_became_peered":"2024-09-17T05:30:24.973279+0000","last_unstale":"2024-09-17T05:30:25.974503+0000","last_undegraded":"2024-09-17T05:30:25.974503+0000","last_fullsized":"2024-09-17T05:30:25.974503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:13.463207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994024+0000","last_change":"2024-09-17T05:30:24.960568+0000","last_active":"2024-09-17T05:30:25.994024+0000","last_peered":"2024-09-17T05:30:25.994024+0000","last_clean":"2024-09-17T05:30:25.994024+0000","last_became_active":"2024-09-17T05:30:24.960307+0000","last_became_peered":"2024-09-17T05:30:24.960307+0000","last_unstale":"2024-09-17T05:30:25.994024+0000","last_undegraded":"2024-09-17T05:30:25.994024+0000","last_fullsized":"2024-09-17T05:30:25.994024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:19.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426598+0000","last_change":"2024-09-17T05:30:24.960733+0000","last_active":"2024-09-17T05:30:26.426598+0000","last_peered":"2024-09-17T05:30:26.426598+0000","last_clean":"2024-09-17T05:30:26.426598+0000","last_became_active":"2024-09-17T05:30:24.960477+0000","last_became_peered":"2024-09-17T05:30:24.960477+0000","last_unstale":"2024-09-17T05:30:26.426598+0000","last_undegraded":"2024-09-17T05:30:26.426598+0000","last_fullsized":"2024-09-17T05:30:26.426598+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:06.403444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.953915+0000","last_change":"2024-09-17T05:30:24.965125+0000","last_active":"2024-09-17T05:30:25.953915+0000","last_peered":"2024-09-17T05:30:25.953915+0000","last_clean":"2024-09-17T05:30:25.953915+0000","last_became_active":"2024-09-17T05:30:24.964872+0000","last_became_peered":"2024-09-17T05:30:24.964872+0000","last_unstale":"2024-09-17T05:30:25.953915+0000","last_undegraded":"2024-09-17T05:30:25.953915+0000","last_fullsized":"2024-09-17T05:30:25.953915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:12.841910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426287+0000","last_change":"2024-09-17T05:30:24.963095+0000","last_active":"2024-09-17T05:30:26.426287+0000","last_peered":"2024-09-17T05:30:26.426287+0000","last_clean":"2024-09-17T05:30:26.426287+0000","last_became_active":"2024-09-17T05:30:24.962824+0000","last_became_peered":"2024-09-17T05:30:24.962824+0000","last_unstale":"2024-09-17T05:30:26.426287+0000","last_undegraded":"2024-09-17T05:30:26.426287+0000","last_fullsized":"2024-09-17T05:30:26.426287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:53.586966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982781+0000","last_change":"2024-09-17T05:30:24.958007+0000","last_active":"2024-09-17T05:30:25.982781+0000","last_peered":"2024-09-17T05:30:25.982781+0000","last_clean":"2024-09-17T05:30:25.982781+0000","last_became_active":"2024-09-17T05:30:24.957745+0000","last_became_peered":"2024-09-17T05:30:24.957745+0000","last_unstale":"2024-09-17T05:30:25.982781+0000","last_undegraded":"2024-09-17T05:30:25.982781+0000","last_fullsized":"2024-09-17T05:30:25.982781+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:21.238127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983434+0000","last_change":"2024-09-17T05:30:24.963597+0000","last_active":"2024-09-17T05:30:25.983434+0000","last_peered":"2024-09-17T05:30:25.983434+0000","last_clean":"2024-09-17T05:30:25.983434+0000","last_became_active":"2024-09-17T05:30:24.962945+0000","last_became_peered":"2024-09-17T05:30:24.962945+0000","last_unstale":"2024-09-17T05:30:25.983434+0000","last_undegraded":"2024-09-17T05:30:25.983434+0000","last_fullsized":"2024-09-17T05:30:25.983434+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:45.758152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954245+0000","last_change":"2024-09-17T05:30:24.958526+0000","last_active":"2024-09-17T05:30:25.954245+0000","last_peered":"2024-09-17T05:30:25.954245+0000","last_clean":"2024-09-17T05:30:25.954245+0000","last_became_active":"2024-09-17T05:30:24.958238+0000","last_became_peered":"2024-09-17T05:30:24.958238+0000","last_unstale":"2024-09-17T05:30:25.954245+0000","last_undegraded":"2024-09-17T05:30:25.954245+0000","last_fullsized":"2024-09-17T05:30:25.954245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:16.257183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959481+0000","last_change":"2024-09-17T05:30:24.970050+0000","last_active":"2024-09-17T05:30:25.959481+0000","last_peered":"2024-09-17T05:30:25.959481+0000","last_clean":"2024-09-17T05:30:25.959481+0000","last_became_active":"2024-09-17T05:30:24.969733+0000","last_became_peered":"2024-09-17T05:30:24.969733+0000","last_unstale":"2024-09-17T05:30:25.959481+0000","last_undegraded":"2024-09-17T05:30:25.959481+0000","last_fullsized":"2024-09-17T05:30:25.959481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:42.759252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974460+0000","last_change":"2024-09-17T05:30:24.973029+0000","last_active":"2024-09-17T05:30:25.974460+0000","last_peered":"2024-09-17T05:30:25.974460+0000","last_clean":"2024-09-17T05:30:25.974460+0000","last_became_active":"2024-09-17T05:30:24.972776+0000","last_became_peered":"2024-09-17T05:30:24.972776+0000","last_unstale":"2024-09-17T05:30:25.974460+0000","last_undegraded":"2024-09-17T05:30:25.974460+0000","last_fullsized":"2024-09-17T05:30:25.974460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:41.320291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426443+0000","last_change":"2024-09-17T05:30:24.959367+0000","last_active":"2024-09-17T05:30:26.426443+0000","last_peered":"2024-09-17T05:30:26.426443+0000","last_clean":"2024-09-17T05:30:26.426443+0000","last_became_active":"2024-09-17T05:30:24.958664+0000","last_became_peered":"2024-09-17T05:30:24.958664+0000","last_unstale":"2024-09-17T05:30:26.426443+0000","last_undegraded":"2024-09-17T05:30:26.426443+0000","last_fullsized":"2024-09-17T05:30:26.426443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:22.869759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974472+0000","last_change":"2024-09-17T05:30:24.962135+0000","last_active":"2024-09-17T05:30:25.974472+0000","last_peered":"2024-09-17T05:30:25.974472+0000","last_clean":"2024-09-17T05:30:25.974472+0000","last_became_active":"2024-09-17T05:30:24.960058+0000","last_became_peered":"2024-09-17T05:30:24.960058+0000","last_unstale":"2024-09-17T05:30:25.974472+0000","last_undegraded":"2024-09-17T05:30:25.974472+0000","last_fullsized":"2024-09-17T05:30:25.974472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:16.257183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956616+0000","last_change":"2024-09-17T05:30:24.970499+0000","last_active":"2024-09-17T05:30:25.956616+0000","last_peered":"2024-09-17T05:30:25.956616+0000","last_clean":"2024-09-17T05:30:25.956616+0000","last_became_active":"2024-09-17T05:30:24.970051+0000","last_became_peered":"2024-09-17T05:30:24.970051+0000","last_unstale":"2024-09-17T05:30:25.956616+0000","last_undegraded":"2024-09-17T05:30:25.956616+0000","last_fullsized":"2024-09-17T05:30:25.956616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:37.784128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994259+0000","last_change":"2024-09-17T05:30:24.967415+0000","last_active":"2024-09-17T05:30:25.994259+0000","last_peered":"2024-09-17T05:30:25.994259+0000","last_clean":"2024-09-17T05:30:25.994259+0000","last_became_active":"2024-09-17T05:30:24.967121+0000","last_became_peered":"2024-09-17T05:30:24.967121+0000","last_unstale":"2024-09-17T05:30:25.994259+0000","last_undegraded":"2024-09-17T05:30:25.994259+0000","last_fullsized":"2024-09-17T05:30:25.994259+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:06.243934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959497+0000","last_change":"2024-09-17T05:30:24.969217+0000","last_active":"2024-09-17T05:30:25.959497+0000","last_peered":"2024-09-17T05:30:25.959497+0000","last_clean":"2024-09-17T05:30:25.959497+0000","last_became_active":"2024-09-17T05:30:24.968970+0000","last_became_peered":"2024-09-17T05:30:24.968970+0000","last_unstale":"2024-09-17T05:30:25.959497+0000","last_undegraded":"2024-09-17T05:30:25.959497+0000","last_fullsized":"2024-09-17T05:30:25.959497+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:24.928889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426617+0000","last_change":"2024-09-17T05:30:24.962929+0000","last_active":"2024-09-17T05:30:26.426617+0000","last_peered":"2024-09-17T05:30:26.426617+0000","last_clean":"2024-09-17T05:30:26.426617+0000","last_became_active":"2024-09-17T05:30:24.962618+0000","last_became_peered":"2024-09-17T05:30:24.962618+0000","last_unstale":"2024-09-17T05:30:26.426617+0000","last_undegraded":"2024-09-17T05:30:26.426617+0000","last_fullsized":"2024-09-17T05:30:26.426617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:22.425017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.960023+0000","last_change":"2024-09-17T05:30:24.959497+0000","last_active":"2024-09-17T05:30:25.960023+0000","last_peered":"2024-09-17T05:30:25.960023+0000","last_clean":"2024-09-17T05:30:25.960023+0000","last_became_active":"2024-09-17T05:30:24.959125+0000","last_became_peered":"2024-09-17T05:30:24.959125+0000","last_unstale":"2024-09-17T05:30:25.960023+0000","last_undegraded":"2024-09-17T05:30:25.960023+0000","last_fullsized":"2024-09-17T05:30:25.960023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:21.238127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864500+0000","last_change":"2024-09-17T05:30:24.976244+0000","last_active":"2024-09-17T05:30:27.864500+0000","last_peered":"2024-09-17T05:30:27.864500+0000","last_clean":"2024-09-17T05:30:27.864500+0000","last_became_active":"2024-09-17T05:30:24.975619+0000","last_became_peered":"2024-09-17T05:30:24.975619+0000","last_unstale":"2024-09-17T05:30:27.864500+0000","last_undegraded":"2024-09-17T05:30:27.864500+0000","last_fullsized":"2024-09-17T05:30:27.864500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:42.656824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864466+0000","last_change":"2024-09-17T05:30:24.972218+0000","last_active":"2024-09-17T05:30:27.864466+0000","last_peered":"2024-09-17T05:30:27.864466+0000","last_clean":"2024-09-17T05:30:27.864466+0000","last_became_active":"2024-09-17T05:30:24.969583+0000","last_became_peered":"2024-09-17T05:30:24.969583+0000","last_unstale":"2024-09-17T05:30:27.864466+0000","last_undegraded":"2024-09-17T05:30:27.864466+0000","last_fullsized":"2024-09-17T05:30:27.864466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:45.665750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956579+0000","last_change":"2024-09-17T05:30:24.963425+0000","last_active":"2024-09-17T05:30:25.956579+0000","last_peered":"2024-09-17T05:30:25.956579+0000","last_clean":"2024-09-17T05:30:25.956579+0000","last_became_active":"2024-09-17T05:30:24.961501+0000","last_became_peered":"2024-09-17T05:30:24.961501+0000","last_unstale":"2024-09-17T05:30:25.956579+0000","last_undegraded":"2024-09-17T05:30:25.956579+0000","last_fullsized":"2024-09-17T05:30:25.956579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:00.843710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959738+0000","last_change":"2024-09-17T05:30:24.955853+0000","last_active":"2024-09-17T05:30:25.959738+0000","last_peered":"2024-09-17T05:30:25.959738+0000","last_clean":"2024-09-17T05:30:25.959738+0000","last_became_active":"2024-09-17T05:30:24.955275+0000","last_became_peered":"2024-09-17T05:30:24.955275+0000","last_unstale":"2024-09-17T05:30:25.959738+0000","last_undegraded":"2024-09-17T05:30:25.959738+0000","last_fullsized":"2024-09-17T05:30:25.959738+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:52.400527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974127+0000","last_change":"2024-09-17T05:30:24.959364+0000","last_active":"2024-09-17T05:30:25.974127+0000","last_peered":"2024-09-17T05:30:25.974127+0000","last_clean":"2024-09-17T05:30:25.974127+0000","last_became_active":"2024-09-17T05:30:24.959037+0000","last_became_peered":"2024-09-17T05:30:24.959037+0000","last_unstale":"2024-09-17T05:30:25.974127+0000","last_undegraded":"2024-09-17T05:30:25.974127+0000","last_fullsized":"2024-09-17T05:30:25.974127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:12.841910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983352+0000","last_change":"2024-09-17T05:30:24.962003+0000","last_active":"2024-09-17T05:30:25.983352+0000","last_peered":"2024-09-17T05:30:25.983352+0000","last_clean":"2024-09-17T05:30:25.983352+0000","last_became_active":"2024-09-17T05:30:24.961160+0000","last_became_peered":"2024-09-17T05:30:24.961160+0000","last_unstale":"2024-09-17T05:30:25.983352+0000","last_undegraded":"2024-09-17T05:30:25.983352+0000","last_fullsized":"2024-09-17T05:30:25.983352+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:22.869759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974708+0000","last_change":"2024-09-17T05:30:23.950637+0000","last_active":"2024-09-17T05:30:25.974708+0000","last_peered":"2024-09-17T05:30:25.974708+0000","last_clean":"2024-09-17T05:30:25.974708+0000","last_became_active":"2024-09-17T05:30:23.950363+0000","last_became_peered":"2024-09-17T05:30:23.950363+0000","last_unstale":"2024-09-17T05:30:25.974708+0000","last_undegraded":"2024-09-17T05:30:25.974708+0000","last_fullsized":"2024-09-17T05:30:25.974708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:52.580019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983643+0000","last_change":"2024-09-17T05:30:24.968089+0000","last_active":"2024-09-17T05:30:25.983643+0000","last_peered":"2024-09-17T05:30:25.983643+0000","last_clean":"2024-09-17T05:30:25.983643+0000","last_became_active":"2024-09-17T05:30:24.967469+0000","last_became_peered":"2024-09-17T05:30:24.967469+0000","last_unstale":"2024-09-17T05:30:25.983643+0000","last_undegraded":"2024-09-17T05:30:25.983643+0000","last_fullsized":"2024-09-17T05:30:25.983643+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:07.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959477+0000","last_change":"2024-09-17T05:30:23.952541+0000","last_active":"2024-09-17T05:30:25.959477+0000","last_peered":"2024-09-17T05:30:25.959477+0000","last_clean":"2024-09-17T05:30:25.959477+0000","last_became_active":"2024-09-17T05:30:23.952190+0000","last_became_peered":"2024-09-17T05:30:23.952190+0000","last_unstale":"2024-09-17T05:30:25.959477+0000","last_undegraded":"2024-09-17T05:30:25.959477+0000","last_fullsized":"2024-09-17T05:30:25.959477+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:07.976069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864584+0000","last_change":"2024-09-17T05:30:24.976969+0000","last_active":"2024-09-17T05:30:27.864584+0000","last_peered":"2024-09-17T05:30:27.864584+0000","last_clean":"2024-09-17T05:30:27.864584+0000","last_became_active":"2024-09-17T05:30:24.976702+0000","last_became_peered":"2024-09-17T05:30:24.976702+0000","last_unstale":"2024-09-17T05:30:27.864584+0000","last_undegraded":"2024-09-17T05:30:27.864584+0000","last_fullsized":"2024-09-17T05:30:27.864584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:29.194444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983310+0000","last_change":"2024-09-17T05:30:23.958614+0000","last_active":"2024-09-17T05:30:25.983310+0000","last_peered":"2024-09-17T05:30:25.983310+0000","last_clean":"2024-09-17T05:30:25.983310+0000","last_became_active":"2024-09-17T05:30:23.958200+0000","last_became_peered":"2024-09-17T05:30:23.958200+0000","last_unstale":"2024-09-17T05:30:25.983310+0000","last_undegraded":"2024-09-17T05:30:25.983310+0000","last_fullsized":"2024-09-17T05:30:25.983310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:20.323922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983069+0000","last_change":"2024-09-17T05:30:24.969992+0000","last_active":"2024-09-17T05:30:25.983069+0000","last_peered":"2024-09-17T05:30:25.983069+0000","last_clean":"2024-09-17T05:30:25.983069+0000","last_became_active":"2024-09-17T05:30:24.969679+0000","last_became_peered":"2024-09-17T05:30:24.969679+0000","last_unstale":"2024-09-17T05:30:25.983069+0000","last_undegraded":"2024-09-17T05:30:25.983069+0000","last_fullsized":"2024-09-17T05:30:25.983069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:05.420612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956851+0000","last_change":"2024-09-17T05:30:23.940517+0000","last_active":"2024-09-17T05:30:25.956851+0000","last_peered":"2024-09-17T05:30:25.956851+0000","last_clean":"2024-09-17T05:30:25.956851+0000","last_became_active":"2024-09-17T05:30:23.940255+0000","last_became_peered":"2024-09-17T05:30:23.940255+0000","last_unstale":"2024-09-17T05:30:25.956851+0000","last_undegraded":"2024-09-17T05:30:25.956851+0000","last_fullsized":"2024-09-17T05:30:25.956851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:55.067522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864815+0000","last_change":"2024-09-17T05:30:24.974140+0000","last_active":"2024-09-17T05:30:27.864815+0000","last_peered":"2024-09-17T05:30:27.864815+0000","last_clean":"2024-09-17T05:30:27.864815+0000","last_became_active":"2024-09-17T05:30:24.973833+0000","last_became_peered":"2024-09-17T05:30:24.973833+0000","last_unstale":"2024-09-17T05:30:27.864815+0000","last_undegraded":"2024-09-17T05:30:27.864815+0000","last_fullsized":"2024-09-17T05:30:27.864815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:28.597753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954167+0000","last_change":"2024-09-17T05:30:23.946652+0000","last_active":"2024-09-17T05:30:25.954167+0000","last_peered":"2024-09-17T05:30:25.954167+0000","last_clean":"2024-09-17T05:30:25.954167+0000","last_became_active":"2024-09-17T05:30:23.946243+0000","last_became_peered":"2024-09-17T05:30:23.946243+0000","last_unstale":"2024-09-17T05:30:25.954167+0000","last_undegraded":"2024-09-17T05:30:25.954167+0000","last_fullsized":"2024-09-17T05:30:25.954167+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:37.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974481+0000","last_change":"2024-09-17T05:30:24.961529+0000","last_active":"2024-09-17T05:30:25.974481+0000","last_peered":"2024-09-17T05:30:25.974481+0000","last_clean":"2024-09-17T05:30:25.974481+0000","last_became_active":"2024-09-17T05:30:24.961275+0000","last_became_peered":"2024-09-17T05:30:24.961275+0000","last_unstale":"2024-09-17T05:30:25.974481+0000","last_undegraded":"2024-09-17T05:30:25.974481+0000","last_fullsized":"2024-09-17T05:30:25.974481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:30.145289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983082+0000","last_change":"2024-09-17T05:30:23.953938+0000","last_active":"2024-09-17T05:30:25.983082+0000","last_peered":"2024-09-17T05:30:25.983082+0000","last_clean":"2024-09-17T05:30:25.983082+0000","last_became_active":"2024-09-17T05:30:23.953683+0000","last_became_peered":"2024-09-17T05:30:23.953683+0000","last_unstale":"2024-09-17T05:30:25.983082+0000","last_undegraded":"2024-09-17T05:30:25.983082+0000","last_fullsized":"2024-09-17T05:30:25.983082+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:12.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.953751+0000","last_change":"2024-09-17T05:30:24.955868+0000","last_active":"2024-09-17T05:30:25.953751+0000","last_peered":"2024-09-17T05:30:25.953751+0000","last_clean":"2024-09-17T05:30:25.953751+0000","last_became_active":"2024-09-17T05:30:24.955453+0000","last_became_peered":"2024-09-17T05:30:24.955453+0000","last_unstale":"2024-09-17T05:30:25.953751+0000","last_undegraded":"2024-09-17T05:30:25.953751+0000","last_fullsized":"2024-09-17T05:30:25.953751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:38.689325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994012+0000","last_change":"2024-09-17T05:30:23.948325+0000","last_active":"2024-09-17T05:30:25.994012+0000","last_peered":"2024-09-17T05:30:25.994012+0000","last_clean":"2024-09-17T05:30:25.994012+0000","last_became_active":"2024-09-17T05:30:23.948001+0000","last_became_peered":"2024-09-17T05:30:23.948001+0000","last_unstale":"2024-09-17T05:30:25.994012+0000","last_undegraded":"2024-09-17T05:30:25.994012+0000","last_fullsized":"2024-09-17T05:30:25.994012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:30.593870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982787+0000","last_change":"2024-09-17T05:30:24.969386+0000","last_active":"2024-09-17T05:30:25.982787+0000","last_peered":"2024-09-17T05:30:25.982787+0000","last_clean":"2024-09-17T05:30:25.982787+0000","last_became_active":"2024-09-17T05:30:24.968937+0000","last_became_peered":"2024-09-17T05:30:24.968937+0000","last_unstale":"2024-09-17T05:30:25.982787+0000","last_undegraded":"2024-09-17T05:30:25.982787+0000","last_fullsized":"2024-09-17T05:30:25.982787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:06.136871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426207+0000","last_change":"2024-09-17T05:30:23.954282+0000","last_active":"2024-09-17T05:30:26.426207+0000","last_peered":"2024-09-17T05:30:26.426207+0000","last_clean":"2024-09-17T05:30:26.426207+0000","last_became_active":"2024-09-17T05:30:23.953881+0000","last_became_peered":"2024-09-17T05:30:23.953881+0000","last_unstale":"2024-09-17T05:30:26.426207+0000","last_undegraded":"2024-09-17T05:30:26.426207+0000","last_fullsized":"2024-09-17T05:30:26.426207+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:41.752305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426661+0000","last_change":"2024-09-17T05:30:24.960242+0000","last_active":"2024-09-17T05:30:26.426661+0000","last_peered":"2024-09-17T05:30:26.426661+0000","last_clean":"2024-09-17T05:30:26.426661+0000","last_became_active":"2024-09-17T05:30:24.959958+0000","last_became_peered":"2024-09-17T05:30:24.959958+0000","last_unstale":"2024-09-17T05:30:26.426661+0000","last_undegraded":"2024-09-17T05:30:26.426661+0000","last_fullsized":"2024-09-17T05:30:26.426661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:05.377943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994222+0000","last_change":"2024-09-17T05:30:23.943335+0000","last_active":"2024-09-17T05:30:25.994222+0000","last_peered":"2024-09-17T05:30:25.994222+0000","last_clean":"2024-09-17T05:30:25.994222+0000","last_became_active":"2024-09-17T05:30:23.943018+0000","last_became_peered":"2024-09-17T05:30:23.943018+0000","last_unstale":"2024-09-17T05:30:25.994222+0000","last_undegraded":"2024-09-17T05:30:25.994222+0000","last_fullsized":"2024-09-17T05:30:25.994222+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:19.341040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954591+0000","last_change":"2024-09-17T05:30:24.972619+0000","last_active":"2024-09-17T05:30:25.954591+0000","last_peered":"2024-09-17T05:30:25.954591+0000","last_clean":"2024-09-17T05:30:25.954591+0000","last_became_active":"2024-09-17T05:30:24.971682+0000","last_became_peered":"2024-09-17T05:30:24.971682+0000","last_unstale":"2024-09-17T05:30:25.954591+0000","last_undegraded":"2024-09-17T05:30:25.954591+0000","last_fullsized":"2024-09-17T05:30:25.954591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:16.312534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954980+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954714+0000","last_became_peered":"2024-09-17T05:30:23.954714+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:24.175086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864850+0000","last_change":"2024-09-17T05:30:24.976889+0000","last_active":"2024-09-17T05:30:27.864850+0000","last_peered":"2024-09-17T05:30:27.864850+0000","last_clean":"2024-09-17T05:30:27.864850+0000","last_became_active":"2024-09-17T05:30:24.976636+0000","last_became_peered":"2024-09-17T05:30:24.976636+0000","last_unstale":"2024-09-17T05:30:27.864850+0000","last_undegraded":"2024-09-17T05:30:27.864850+0000","last_fullsized":"2024-09-17T05:30:27.864850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:54.276295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.957049+0000","last_change":"2024-09-17T05:30:23.942665+0000","last_active":"2024-09-17T05:30:25.957049+0000","last_peered":"2024-09-17T05:30:25.957049+0000","last_clean":"2024-09-17T05:30:25.957049+0000","last_became_active":"2024-09-17T05:30:23.942377+0000","last_became_peered":"2024-09-17T05:30:23.942377+0000","last_unstale":"2024-09-17T05:30:25.957049+0000","last_undegraded":"2024-09-17T05:30:25.957049+0000","last_fullsized":"2024-09-17T05:30:25.957049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:20.108443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956927+0000","last_change":"2024-09-17T05:30:24.972003+0000","last_active":"2024-09-17T05:30:25.956927+0000","last_peered":"2024-09-17T05:30:25.956927+0000","last_clean":"2024-09-17T05:30:25.956927+0000","last_became_active":"2024-09-17T05:30:24.971652+0000","last_became_peered":"2024-09-17T05:30:24.971652+0000","last_unstale":"2024-09-17T05:30:25.956927+0000","last_undegraded":"2024-09-17T05:30:25.956927+0000","last_fullsized":"2024-09-17T05:30:25.956927+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:19.300631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":21,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426555+0000","last_change":"2024-09-17T05:30:23.953324+0000","last_active":"2024-09-17T05:30:26.426555+0000","last_peered":"2024-09-17T05:30:26.426555+0000","last_clean":"2024-09-17T05:30:26.426555+0000","last_became_active":"2024-09-17T05:30:23.953049+0000","last_became_peered":"2024-09-17T05:30:23.953049+0000","last_unstale":"2024-09-17T05:30:26.426555+0000","last_undegraded":"2024-09-17T05:30:26.426555+0000","last_fullsized":"2024-09-17T05:30:26.426555+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:51.393580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.993941+0000","last_change":"2024-09-17T05:30:24.957930+0000","last_active":"2024-09-17T05:30:25.993941+0000","last_peered":"2024-09-17T05:30:25.993941+0000","last_clean":"2024-09-17T05:30:25.993941+0000","last_became_active":"2024-09-17T05:30:24.957640+0000","last_became_peered":"2024-09-17T05:30:24.957640+0000","last_unstale":"2024-09-17T05:30:25.993941+0000","last_undegraded":"2024-09-17T05:30:25.993941+0000","last_fullsized":"2024-09-17T05:30:25.993941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:13.307547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954792+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954435+0000","last_became_peered":"2024-09-17T05:30:23.954435+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:30.663318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954015+0000","last_change":"2024-09-17T05:30:24.952788+0000","last_active":"2024-09-17T05:30:25.954015+0000","last_peered":"2024-09-17T05:30:25.954015+0000","last_clean":"2024-09-17T05:30:25.954015+0000","last_became_active":"2024-09-17T05:30:24.952549+0000","last_became_peered":"2024-09-17T05:30:24.952549+0000","last_unstale":"2024-09-17T05:30:25.954015+0000","last_undegraded":"2024-09-17T05:30:25.954015+0000","last_fullsized":"2024-09-17T05:30:25.954015+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:30.145289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956561+0000","last_change":"2024-09-17T05:30:23.948409+0000","last_active":"2024-09-17T05:30:25.956561+0000","last_peered":"2024-09-17T05:30:25.956561+0000","last_clean":"2024-09-17T05:30:25.956561+0000","last_became_active":"2024-09-17T05:30:23.948114+0000","last_became_peered":"2024-09-17T05:30:23.948114+0000","last_unstale":"2024-09-17T05:30:25.956561+0000","last_undegraded":"2024-09-17T05:30:25.956561+0000","last_fullsized":"2024-09-17T05:30:25.956561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:06.470015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994014+0000","last_change":"2024-09-17T05:30:24.962071+0000","last_active":"2024-09-17T05:30:25.994014+0000","last_peered":"2024-09-17T05:30:25.994014+0000","last_clean":"2024-09-17T05:30:25.994014+0000","last_became_active":"2024-09-17T05:30:24.960113+0000","last_became_peered":"2024-09-17T05:30:24.960113+0000","last_unstale":"2024-09-17T05:30:25.994014+0000","last_undegraded":"2024-09-17T05:30:25.994014+0000","last_fullsized":"2024-09-17T05:30:25.994014+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:21.330869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956782+0000","last_change":"2024-09-17T05:30:23.943942+0000","last_active":"2024-09-17T05:30:25.956782+0000","last_peered":"2024-09-17T05:30:25.956782+0000","last_clean":"2024-09-17T05:30:25.956782+0000","last_became_active":"2024-09-17T05:30:23.943669+0000","last_became_peered":"2024-09-17T05:30:23.943669+0000","last_unstale":"2024-09-17T05:30:25.956782+0000","last_undegraded":"2024-09-17T05:30:25.956782+0000","last_fullsized":"2024-09-17T05:30:25.956782+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:13.811827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956846+0000","last_change":"2024-09-17T05:30:24.960276+0000","last_active":"2024-09-17T05:30:25.956846+0000","last_peered":"2024-09-17T05:30:25.956846+0000","last_clean":"2024-09-17T05:30:25.956846+0000","last_became_active":"2024-09-17T05:30:24.959383+0000","last_became_peered":"2024-09-17T05:30:24.959383+0000","last_unstale":"2024-09-17T05:30:25.956846+0000","last_undegraded":"2024-09-17T05:30:25.956846+0000","last_fullsized":"2024-09-17T05:30:25.956846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:49.583118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":23,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956508+0000","last_change":"2024-09-17T05:30:23.949103+0000","last_active":"2024-09-17T05:30:25.956508+0000","last_peered":"2024-09-17T05:30:25.956508+0000","last_clean":"2024-09-17T05:30:25.956508+0000","last_became_active":"2024-09-17T05:30:23.948810+0000","last_became_peered":"2024-09-17T05:30:23.948810+0000","last_unstale":"2024-09-17T05:30:25.956508+0000","last_undegraded":"2024-09-17T05:30:25.956508+0000","last_fullsized":"2024-09-17T05:30:25.956508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.103339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954276+0000","last_change":"2024-09-17T05:30:24.958134+0000","last_active":"2024-09-17T05:30:25.954276+0000","last_peered":"2024-09-17T05:30:25.954276+0000","last_clean":"2024-09-17T05:30:25.954276+0000","last_became_active":"2024-09-17T05:30:24.957609+0000","last_became_peered":"2024-09-17T05:30:24.957609+0000","last_unstale":"2024-09-17T05:30:25.954276+0000","last_undegraded":"2024-09-17T05:30:25.954276+0000","last_fullsized":"2024-09-17T05:30:25.954276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:17:13.573184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959488+0000","last_change":"2024-09-17T05:30:23.960591+0000","last_active":"2024-09-17T05:30:25.959488+0000","last_peered":"2024-09-17T05:30:25.959488+0000","last_clean":"2024-09-17T05:30:25.959488+0000","last_became_active":"2024-09-17T05:30:23.960068+0000","last_became_peered":"2024-09-17T05:30:23.960068+0000","last_unstale":"2024-09-17T05:30:25.959488+0000","last_undegraded":"2024-09-17T05:30:25.959488+0000","last_fullsized":"2024-09-17T05:30:25.959488+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:19.075142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954256+0000","last_change":"2024-09-17T05:30:24.965156+0000","last_active":"2024-09-17T05:30:25.954256+0000","last_peered":"2024-09-17T05:30:25.954256+0000","last_clean":"2024-09-17T05:30:25.954256+0000","last_became_active":"2024-09-17T05:30:24.964861+0000","last_became_peered":"2024-09-17T05:30:24.964861+0000","last_unstale":"2024-09-17T05:30:25.954256+0000","last_undegraded":"2024-09-17T05:30:25.954256+0000","last_fullsized":"2024-09-17T05:30:25.954256+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:33:05.958676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864645+0000","last_change":"2024-09-17T05:30:23.951223+0000","last_active":"2024-09-17T05:30:27.864645+0000","last_peered":"2024-09-17T05:30:27.864645+0000","last_clean":"2024-09-17T05:30:27.864645+0000","last_became_active":"2024-09-17T05:30:23.950514+0000","last_became_peered":"2024-09-17T05:30:23.950514+0000","last_unstale":"2024-09-17T05:30:27.864645+0000","last_undegraded":"2024-09-17T05:30:27.864645+0000","last_fullsized":"2024-09-17T05:30:27.864645+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:45.736482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959277+0000","last_change":"2024-09-17T05:30:24.966104+0000","last_active":"2024-09-17T05:30:25.959277+0000","last_peered":"2024-09-17T05:30:25.959277+0000","last_clean":"2024-09-17T05:30:25.959277+0000","last_became_active":"2024-09-17T05:30:24.964848+0000","last_became_peered":"2024-09-17T05:30:24.964848+0000","last_unstale":"2024-09-17T05:30:25.959277+0000","last_undegraded":"2024-09-17T05:30:25.959277+0000","last_fullsized":"2024-09-17T05:30:25.959277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:28.543306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954296+0000","last_change":"2024-09-17T05:30:23.944667+0000","last_active":"2024-09-17T05:30:25.954296+0000","last_peered":"2024-09-17T05:30:25.954296+0000","last_clean":"2024-09-17T05:30:25.954296+0000","last_became_active":"2024-09-17T05:30:23.944381+0000","last_became_peered":"2024-09-17T05:30:23.944381+0000","last_unstale":"2024-09-17T05:30:25.954296+0000","last_undegraded":"2024-09-17T05:30:25.954296+0000","last_fullsized":"2024-09-17T05:30:25.954296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:20.231180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956948+0000","last_change":"2024-09-17T05:30:24.966889+0000","last_active":"2024-09-17T05:30:25.956948+0000","last_peered":"2024-09-17T05:30:25.956948+0000","last_clean":"2024-09-17T05:30:25.956948+0000","last_became_active":"2024-09-17T05:30:24.966626+0000","last_became_peered":"2024-09-17T05:30:24.966626+0000","last_unstale":"2024-09-17T05:30:25.956948+0000","last_undegraded":"2024-09-17T05:30:25.956948+0000","last_fullsized":"2024-09-17T05:30:25.956948+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:33.137934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864732+0000","last_change":"2024-09-17T05:30:23.960543+0000","last_active":"2024-09-17T05:30:27.864732+0000","last_peered":"2024-09-17T05:30:27.864732+0000","last_clean":"2024-09-17T05:30:27.864732+0000","last_became_active":"2024-09-17T05:30:23.960241+0000","last_became_peered":"2024-09-17T05:30:23.960241+0000","last_unstale":"2024-09-17T05:30:27.864732+0000","last_undegraded":"2024-09-17T05:30:27.864732+0000","last_fullsized":"2024-09-17T05:30:27.864732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:25.924845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994184+0000","last_change":"2024-09-17T05:30:24.961787+0000","last_active":"2024-09-17T05:30:25.994184+0000","last_peered":"2024-09-17T05:30:25.994184+0000","last_clean":"2024-09-17T05:30:25.994184+0000","last_became_active":"2024-09-17T05:30:24.961480+0000","last_became_peered":"2024-09-17T05:30:24.961480+0000","last_unstale":"2024-09-17T05:30:25.994184+0000","last_undegraded":"2024-09-17T05:30:25.994184+0000","last_fullsized":"2024-09-17T05:30:25.994184+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:19.521784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974371+0000","last_change":"2024-09-17T05:30:23.951379+0000","last_active":"2024-09-17T05:30:25.974371+0000","last_peered":"2024-09-17T05:30:25.974371+0000","last_clean":"2024-09-17T05:30:25.974371+0000","last_became_active":"2024-09-17T05:30:23.950900+0000","last_became_peered":"2024-09-17T05:30:23.950900+0000","last_unstale":"2024-09-17T05:30:25.974371+0000","last_undegraded":"2024-09-17T05:30:25.974371+0000","last_fullsized":"2024-09-17T05:30:25.974371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:24.722311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864503+0000","last_change":"2024-09-17T05:30:24.973721+0000","last_active":"2024-09-17T05:30:27.864503+0000","last_peered":"2024-09-17T05:30:27.864503+0000","last_clean":"2024-09-17T05:30:27.864503+0000","last_became_active":"2024-09-17T05:30:24.973391+0000","last_became_peered":"2024-09-17T05:30:24.973391+0000","last_unstale":"2024-09-17T05:30:27.864503+0000","last_undegraded":"2024-09-17T05:30:27.864503+0000","last_fullsized":"2024-09-17T05:30:27.864503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:02.895238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864244+0000","last_change":"2024-09-17T05:30:23.950654+0000","last_active":"2024-09-17T05:30:27.864244+0000","last_peered":"2024-09-17T05:30:27.864244+0000","last_clean":"2024-09-17T05:30:27.864244+0000","last_became_active":"2024-09-17T05:30:23.950372+0000","last_became_peered":"2024-09-17T05:30:23.950372+0000","last_unstale":"2024-09-17T05:30:27.864244+0000","last_undegraded":"2024-09-17T05:30:27.864244+0000","last_fullsized":"2024-09-17T05:30:27.864244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:46.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864458+0000","last_change":"2024-09-17T05:30:24.964323+0000","last_active":"2024-09-17T05:30:27.864458+0000","last_peered":"2024-09-17T05:30:27.864458+0000","last_clean":"2024-09-17T05:30:27.864458+0000","last_became_active":"2024-09-17T05:30:24.964039+0000","last_became_peered":"2024-09-17T05:30:24.964039+0000","last_unstale":"2024-09-17T05:30:27.864458+0000","last_undegraded":"2024-09-17T05:30:27.864458+0000","last_fullsized":"2024-09-17T05:30:27.864458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:52.710209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994152+0000","last_change":"2024-09-17T05:30:23.946216+0000","last_active":"2024-09-17T05:30:25.994152+0000","last_peered":"2024-09-17T05:30:25.994152+0000","last_clean":"2024-09-17T05:30:25.994152+0000","last_became_active":"2024-09-17T05:30:23.945797+0000","last_became_peered":"2024-09-17T05:30:23.945797+0000","last_unstale":"2024-09-17T05:30:25.994152+0000","last_undegraded":"2024-09-17T05:30:25.994152+0000","last_fullsized":"2024-09-17T05:30:25.994152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:55.988234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954490+0000","last_change":"2024-09-17T05:30:24.957869+0000","last_active":"2024-09-17T05:30:25.954490+0000","last_peered":"2024-09-17T05:30:25.954490+0000","last_clean":"2024-09-17T05:30:25.954490+0000","last_became_active":"2024-09-17T05:30:24.957469+0000","last_became_peered":"2024-09-17T05:30:24.957469+0000","last_unstale":"2024-09-17T05:30:25.954490+0000","last_undegraded":"2024-09-17T05:30:25.954490+0000","last_fullsized":"2024-09-17T05:30:25.954490+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:08.297928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959647+0000","last_change":"2024-09-17T05:30:23.953693+0000","last_active":"2024-09-17T05:30:25.959647+0000","last_peered":"2024-09-17T05:30:25.959647+0000","last_clean":"2024-09-17T05:30:25.959647+0000","last_became_active":"2024-09-17T05:30:23.953424+0000","last_became_peered":"2024-09-17T05:30:23.953424+0000","last_unstale":"2024-09-17T05:30:25.959647+0000","last_undegraded":"2024-09-17T05:30:25.959647+0000","last_fullsized":"2024-09-17T05:30:25.959647+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.772225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954494+0000","last_change":"2024-09-17T05:30:24.963268+0000","last_active":"2024-09-17T05:30:25.954494+0000","last_peered":"2024-09-17T05:30:25.954494+0000","last_clean":"2024-09-17T05:30:25.954494+0000","last_became_active":"2024-09-17T05:30:24.962823+0000","last_became_peered":"2024-09-17T05:30:24.962823+0000","last_unstale":"2024-09-17T05:30:25.954494+0000","last_undegraded":"2024-09-17T05:30:25.954494+0000","last_fullsized":"2024-09-17T05:30:25.954494+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:55.677952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983525+0000","last_change":"2024-09-17T05:30:23.958649+0000","last_active":"2024-09-17T05:30:25.983525+0000","last_peered":"2024-09-17T05:30:25.983525+0000","last_clean":"2024-09-17T05:30:25.983525+0000","last_became_active":"2024-09-17T05:30:23.958329+0000","last_became_peered":"2024-09-17T05:30:23.958329+0000","last_unstale":"2024-09-17T05:30:25.983525+0000","last_undegraded":"2024-09-17T05:30:25.983525+0000","last_fullsized":"2024-09-17T05:30:25.983525+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:18.514837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959686+0000","last_change":"2024-09-17T05:30:24.966794+0000","last_active":"2024-09-17T05:30:25.959686+0000","last_peered":"2024-09-17T05:30:25.959686+0000","last_clean":"2024-09-17T05:30:25.959686+0000","last_became_active":"2024-09-17T05:30:24.966503+0000","last_became_peered":"2024-09-17T05:30:24.966503+0000","last_unstale":"2024-09-17T05:30:25.959686+0000","last_undegraded":"2024-09-17T05:30:25.959686+0000","last_fullsized":"2024-09-17T05:30:25.959686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:38.777542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994067+0000","last_change":"2024-09-17T05:30:23.943837+0000","last_active":"2024-09-17T05:30:25.994067+0000","last_peered":"2024-09-17T05:30:25.994067+0000","last_clean":"2024-09-17T05:30:25.994067+0000","last_became_active":"2024-09-17T05:30:23.942677+0000","last_became_peered":"2024-09-17T05:30:23.942677+0000","last_unstale":"2024-09-17T05:30:25.994067+0000","last_undegraded":"2024-09-17T05:30:25.994067+0000","last_fullsized":"2024-09-17T05:30:25.994067+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:49.775010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864917+0000","last_change":"2024-09-17T05:30:24.974072+0000","last_active":"2024-09-17T05:30:27.864917+0000","last_peered":"2024-09-17T05:30:27.864917+0000","last_clean":"2024-09-17T05:30:27.864917+0000","last_became_active":"2024-09-17T05:30:24.973224+0000","last_became_peered":"2024-09-17T05:30:24.973224+0000","last_unstale":"2024-09-17T05:30:27.864917+0000","last_undegraded":"2024-09-17T05:30:27.864917+0000","last_fullsized":"2024-09-17T05:30:27.864917+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:35.858572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954059+0000","last_change":"2024-09-17T05:30:23.947956+0000","last_active":"2024-09-17T05:30:25.954059+0000","last_peered":"2024-09-17T05:30:25.954059+0000","last_clean":"2024-09-17T05:30:25.954059+0000","last_became_active":"2024-09-17T05:30:23.947641+0000","last_became_peered":"2024-09-17T05:30:23.947641+0000","last_unstale":"2024-09-17T05:30:25.954059+0000","last_undegraded":"2024-09-17T05:30:25.954059+0000","last_fullsized":"2024-09-17T05:30:25.954059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:19.881427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983045+0000","last_change":"2024-09-17T05:30:24.967750+0000","last_active":"2024-09-17T05:30:25.983045+0000","last_peered":"2024-09-17T05:30:25.983045+0000","last_clean":"2024-09-17T05:30:25.983045+0000","last_became_active":"2024-09-17T05:30:24.967460+0000","last_became_peered":"2024-09-17T05:30:24.967460+0000","last_unstale":"2024-09-17T05:30:25.983045+0000","last_undegraded":"2024-09-17T05:30:25.983045+0000","last_fullsized":"2024-09-17T05:30:25.983045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:09:11.265178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959229+0000","last_change":"2024-09-17T05:30:23.957320+0000","last_active":"2024-09-17T05:30:25.959229+0000","last_peered":"2024-09-17T05:30:25.959229+0000","last_clean":"2024-09-17T05:30:25.959229+0000","last_became_active":"2024-09-17T05:30:23.956956+0000","last_became_peered":"2024-09-17T05:30:23.956956+0000","last_unstale":"2024-09-17T05:30:25.959229+0000","last_undegraded":"2024-09-17T05:30:25.959229+0000","last_fullsized":"2024-09-17T05:30:25.959229+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:44.089365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426661+0000","last_change":"2024-09-17T05:30:24.965327+0000","last_active":"2024-09-17T05:30:26.426661+0000","last_peered":"2024-09-17T05:30:26.426661+0000","last_clean":"2024-09-17T05:30:26.426661+0000","last_became_active":"2024-09-17T05:30:24.965023+0000","last_became_peered":"2024-09-17T05:30:24.965023+0000","last_unstale":"2024-09-17T05:30:26.426661+0000","last_undegraded":"2024-09-17T05:30:26.426661+0000","last_fullsized":"2024-09-17T05:30:26.426661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:18.712947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.953856+0000","last_change":"2024-09-17T05:30:23.945568+0000","last_active":"2024-09-17T05:30:25.953856+0000","last_peered":"2024-09-17T05:30:25.953856+0000","last_clean":"2024-09-17T05:30:25.953856+0000","last_became_active":"2024-09-17T05:30:23.945279+0000","last_became_peered":"2024-09-17T05:30:23.945279+0000","last_unstale":"2024-09-17T05:30:25.953856+0000","last_undegraded":"2024-09-17T05:30:25.953856+0000","last_fullsized":"2024-09-17T05:30:25.953856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:51.393580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864556+0000","last_change":"2024-09-17T05:30:24.974524+0000","last_active":"2024-09-17T05:30:27.864556+0000","last_peered":"2024-09-17T05:30:27.864556+0000","last_clean":"2024-09-17T05:30:27.864556+0000","last_became_active":"2024-09-17T05:30:24.974227+0000","last_became_peered":"2024-09-17T05:30:24.974227+0000","last_unstale":"2024-09-17T05:30:27.864556+0000","last_undegraded":"2024-09-17T05:30:27.864556+0000","last_fullsized":"2024-09-17T05:30:27.864556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:47.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982856+0000","last_change":"2024-09-17T05:30:23.960348+0000","last_active":"2024-09-17T05:30:25.982856+0000","last_peered":"2024-09-17T05:30:25.982856+0000","last_clean":"2024-09-17T05:30:25.982856+0000","last_became_active":"2024-09-17T05:30:23.960056+0000","last_became_peered":"2024-09-17T05:30:23.960056+0000","last_unstale":"2024-09-17T05:30:25.982856+0000","last_undegraded":"2024-09-17T05:30:25.982856+0000","last_fullsized":"2024-09-17T05:30:25.982856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:31.457549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959807+0000","last_change":"2024-09-17T05:30:24.955548+0000","last_active":"2024-09-17T05:30:25.959807+0000","last_peered":"2024-09-17T05:30:25.959807+0000","last_clean":"2024-09-17T05:30:25.959807+0000","last_became_active":"2024-09-17T05:30:24.954911+0000","last_became_peered":"2024-09-17T05:30:24.954911+0000","last_unstale":"2024-09-17T05:30:25.959807+0000","last_undegraded":"2024-09-17T05:30:25.959807+0000","last_fullsized":"2024-09-17T05:30:25.959807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:20.888374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864474+0000","last_change":"2024-09-17T05:30:23.956301+0000","last_active":"2024-09-17T05:30:27.864474+0000","last_peered":"2024-09-17T05:30:27.864474+0000","last_clean":"2024-09-17T05:30:27.864474+0000","last_became_active":"2024-09-17T05:30:23.955995+0000","last_became_peered":"2024-09-17T05:30:23.955995+0000","last_unstale":"2024-09-17T05:30:27.864474+0000","last_undegraded":"2024-09-17T05:30:27.864474+0000","last_fullsized":"2024-09-17T05:30:27.864474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:02.746869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864899+0000","last_change":"2024-09-17T05:30:24.973772+0000","last_active":"2024-09-17T05:30:27.864899+0000","last_peered":"2024-09-17T05:30:27.864899+0000","last_clean":"2024-09-17T05:30:27.864899+0000","last_became_active":"2024-09-17T05:30:24.972340+0000","last_became_peered":"2024-09-17T05:30:24.972340+0000","last_unstale":"2024-09-17T05:30:27.864899+0000","last_undegraded":"2024-09-17T05:30:27.864899+0000","last_fullsized":"2024-09-17T05:30:27.864899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:43.483847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426374+0000","last_change":"2024-09-17T05:30:23.952489+0000","last_active":"2024-09-17T05:30:26.426374+0000","last_peered":"2024-09-17T05:30:26.426374+0000","last_clean":"2024-09-17T05:30:26.426374+0000","last_became_active":"2024-09-17T05:30:23.952212+0000","last_became_peered":"2024-09-17T05:30:23.952212+0000","last_unstale":"2024-09-17T05:30:26.426374+0000","last_undegraded":"2024-09-17T05:30:26.426374+0000","last_fullsized":"2024-09-17T05:30:26.426374+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:37.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":183,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994221+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:30:25.994221+0000","last_peered":"2024-09-17T05:30:25.994221+0000","last_clean":"2024-09-17T05:30:25.994221+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:30:25.994221+0000","last_undegraded":"2024-09-17T05:30:25.994221+0000","last_fullsized":"2024-09-17T05:30:25.994221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509510,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40212,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331628,"statfs":{"total":96636764160,"available":96595587072,"internally_reserved":0,"allocated":720896,"data_stored":297955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58264,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313576,"statfs":{"total":96636764160,"available":96577101824,"internally_reserved":0,"allocated":716800,"data_stored":297885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45780,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326060,"statfs":{"total":96636764160,"available":96589885440,"internally_reserved":0,"allocated":720896,"data_stored":298479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40216,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331624,"statfs":{"total":96636764160,"available":96595582976,"internally_reserved":0,"allocated":716800,"data_stored":297953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62440,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309400,"statfs":{"total":96636764160,"available":96572825600,"internally_reserved":0,"allocated":733184,"data_stored":299635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509511,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1310720,"data_stored":888813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":724992,"data_stored":299475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1310720,"data_stored":888265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:30:32.240 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:30:32.245 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:30:33.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:30:33.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- 172.21.15.122:0/3963109034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 msgr2=0x7f15e00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:33.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 --2- 172.21.15.122:0/3963109034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 0x7f15e00ce670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009bc0 tx=0x7f15d001cb10 comp rx=0 tx=0).stop 2024-09-17T05:30:33.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- 172.21.15.122:0/3963109034 shutdown_connections 2024-09-17T05:30:33.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 --2- 172.21.15.122:0/3963109034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 0x7f15e00ce670 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- 172.21.15.122:0/3963109034 >> 172.21.15.122:0/3963109034 conn(0x7f15e00d1dc0 msgr2=0x7f15e00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:33.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- 172.21.15.122:0/3963109034 shutdown_connections 2024-09-17T05:30:33.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- 172.21.15.122:0/3963109034 wait complete. 2024-09-17T05:30:33.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 Processor -- start 2024-09-17T05:30:33.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- start start 2024-09-17T05:30:33.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e00cdb30 0x7f15e01552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:33.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e00d28e0 0x7f15e0157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:33.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 0x7f15e0158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15e016a930 con 0x7f15e015bea0 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15e016a7b0 con 0x7f15e00d28e0 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e7000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f15e016aab0 con 0x7f15e00cdb30 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 0x7f15e0158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 0x7f15e0158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45612/0 (socket says 172.21.15.122:45612) 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.794+0000 7f15e5000640 1 -- 172.21.15.122:0/4286345075 learned_addr learned my addr 172.21.15.122:0/4286345075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15dfe00640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e00cdb30 0x7f15e01552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15d7400640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e00d28e0 0x7f15e0157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e5000640 1 -- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e00cdb30 msgr2=0x7f15e01552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e5000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e00cdb30 0x7f15e01552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e5000640 1 -- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e00d28e0 msgr2=0x7f15e0157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e5000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e00d28e0 0x7f15e0157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e5000640 1 -- 172.21.15.122:0/4286345075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e00b5d30 con 0x7f15e015bea0 2024-09-17T05:30:33.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e5000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 0x7f15e0158170 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f15d800c3f0 tx=0x7f15d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:33.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15dcc00640 1 -- 172.21.15.122:0/4286345075 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8011790 con 0x7f15e015bea0 2024-09-17T05:30:33.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15dcc00640 1 -- 172.21.15.122:0/4286345075 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15d8017070 con 0x7f15e015bea0 2024-09-17T05:30:33.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e0158ea0 con 0x7f15e015bea0 2024-09-17T05:30:33.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15dcc00640 1 -- 172.21.15.122:0/4286345075 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8011cc0 con 0x7f15e015bea0 2024-09-17T05:30:33.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e0159460 con 0x7f15e015bea0 2024-09-17T05:30:33.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e015a8c0 con 0x7f15e015bea0 2024-09-17T05:30:33.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.798+0000 7f15dcc00640 1 -- 172.21.15.122:0/4286345075 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f15d801c460 con 0x7f15e015bea0 2024-09-17T05:30:33.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.802+0000 7f15dcc00640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f15bc073b00 0x7f15bc075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:33.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.802+0000 7f15dcc00640 1 -- 172.21.15.122:0/4286345075 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f15d80928f0 con 0x7f15e015bea0 2024-09-17T05:30:33.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.802+0000 7f15dfe00640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f15bc073b00 0x7f15bc075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:33.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.802+0000 7f15dfe00640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f15bc073b00 0x7f15bc075fc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15d00af000 tx=0x7f15d001b350 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:33.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.802+0000 7f15dcc00640 1 -- 172.21.15.122:0/4286345075 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f15e015a8c0 con 0x7f15e015bea0 2024-09-17T05:30:33.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.970+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15e0072440 con 0x7f15bc073b00 2024-09-17T05:30:33.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.974+0000 7f15dcc00640 1 -- 172.21.15.122:0/4286345075 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f15e0072440 con 0x7f15bc073b00 2024-09-17T05:30:33.978 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:33.980 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f15bc073b00 msgr2=0x7f15bc075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f15bc073b00 0x7f15bc075fc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f15d00af000 tx=0x7f15d001b350 comp rx=0 tx=0).stop 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 msgr2=0x7f15e0158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 0x7f15e0158170 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f15d800c3f0 tx=0x7f15d800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 shutdown_connections 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f15bc073b00 0x7f15bc075fc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15e015bea0 0x7f15e0158170 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e00d28e0 0x7f15e0157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 --2- 172.21.15.122:0/4286345075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e00cdb30 0x7f15e01552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 >> 172.21.15.122:0/4286345075 conn(0x7f15e00d1dc0 msgr2=0x7f15e00b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:33.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 shutdown_connections 2024-09-17T05:30:33.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:33.978+0000 7f15e7000640 1 -- 172.21.15.122:0/4286345075 wait complete. 2024-09-17T05:30:33.999 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-17T05:30:32.534464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374280,"kb_used_data":6792,"kb_used_omap":12,"kb_used_meta":367283,"kb_avail":754600440,"statfs":{"total":773094113280,"available":772710850560,"internally_reserved":0,"allocated":6955008,"data_stored":3568460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":376098358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002063"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983368+0000","last_change":"2024-09-17T05:30:24.971099+0000","last_active":"2024-09-17T05:30:25.983368+0000","last_peered":"2024-09-17T05:30:25.983368+0000","last_clean":"2024-09-17T05:30:25.983368+0000","last_became_active":"2024-09-17T05:30:24.970644+0000","last_became_peered":"2024-09-17T05:30:24.970644+0000","last_unstale":"2024-09-17T05:30:25.983368+0000","last_undegraded":"2024-09-17T05:30:25.983368+0000","last_fullsized":"2024-09-17T05:30:25.983368+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:29.353770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959472+0000","last_change":"2024-09-17T05:30:24.969658+0000","last_active":"2024-09-17T05:30:25.959472+0000","last_peered":"2024-09-17T05:30:25.959472+0000","last_clean":"2024-09-17T05:30:25.959472+0000","last_became_active":"2024-09-17T05:30:24.968921+0000","last_became_peered":"2024-09-17T05:30:24.968921+0000","last_unstale":"2024-09-17T05:30:25.959472+0000","last_undegraded":"2024-09-17T05:30:25.959472+0000","last_fullsized":"2024-09-17T05:30:25.959472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:59.946299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954499+0000","last_change":"2024-09-17T05:30:24.956943+0000","last_active":"2024-09-17T05:30:25.954499+0000","last_peered":"2024-09-17T05:30:25.954499+0000","last_clean":"2024-09-17T05:30:25.954499+0000","last_became_active":"2024-09-17T05:30:24.956657+0000","last_became_peered":"2024-09-17T05:30:24.956657+0000","last_unstale":"2024-09-17T05:30:25.954499+0000","last_undegraded":"2024-09-17T05:30:25.954499+0000","last_fullsized":"2024-09-17T05:30:25.954499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:13.604844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956851+0000","last_change":"2024-09-17T05:30:24.962723+0000","last_active":"2024-09-17T05:30:25.956851+0000","last_peered":"2024-09-17T05:30:25.956851+0000","last_clean":"2024-09-17T05:30:25.956851+0000","last_became_active":"2024-09-17T05:30:24.962385+0000","last_became_peered":"2024-09-17T05:30:24.962385+0000","last_unstale":"2024-09-17T05:30:25.956851+0000","last_undegraded":"2024-09-17T05:30:25.956851+0000","last_fullsized":"2024-09-17T05:30:25.956851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:05.579221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982994+0000","last_change":"2024-09-17T05:30:24.963755+0000","last_active":"2024-09-17T05:30:25.982994+0000","last_peered":"2024-09-17T05:30:25.982994+0000","last_clean":"2024-09-17T05:30:25.982994+0000","last_became_active":"2024-09-17T05:30:24.963461+0000","last_became_peered":"2024-09-17T05:30:24.963461+0000","last_unstale":"2024-09-17T05:30:25.982994+0000","last_undegraded":"2024-09-17T05:30:25.982994+0000","last_fullsized":"2024-09-17T05:30:25.982994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:30.129620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982887+0000","last_change":"2024-09-17T05:30:24.970340+0000","last_active":"2024-09-17T05:30:25.982887+0000","last_peered":"2024-09-17T05:30:25.982887+0000","last_clean":"2024-09-17T05:30:25.982887+0000","last_became_active":"2024-09-17T05:30:24.969744+0000","last_became_peered":"2024-09-17T05:30:24.969744+0000","last_unstale":"2024-09-17T05:30:25.982887+0000","last_undegraded":"2024-09-17T05:30:25.982887+0000","last_fullsized":"2024-09-17T05:30:25.982887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:53.474035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956571+0000","last_change":"2024-09-17T05:30:24.961865+0000","last_active":"2024-09-17T05:30:25.956571+0000","last_peered":"2024-09-17T05:30:25.956571+0000","last_clean":"2024-09-17T05:30:25.956571+0000","last_became_active":"2024-09-17T05:30:24.961570+0000","last_became_peered":"2024-09-17T05:30:24.961570+0000","last_unstale":"2024-09-17T05:30:25.956571+0000","last_undegraded":"2024-09-17T05:30:25.956571+0000","last_fullsized":"2024-09-17T05:30:25.956571+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:42.626536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864357+0000","last_change":"2024-09-17T05:30:24.969090+0000","last_active":"2024-09-17T05:30:27.864357+0000","last_peered":"2024-09-17T05:30:27.864357+0000","last_clean":"2024-09-17T05:30:27.864357+0000","last_became_active":"2024-09-17T05:30:24.968736+0000","last_became_peered":"2024-09-17T05:30:24.968736+0000","last_unstale":"2024-09-17T05:30:27.864357+0000","last_undegraded":"2024-09-17T05:30:27.864357+0000","last_fullsized":"2024-09-17T05:30:27.864357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:13.357280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994100+0000","last_change":"2024-09-17T05:30:24.967509+0000","last_active":"2024-09-17T05:30:25.994100+0000","last_peered":"2024-09-17T05:30:25.994100+0000","last_clean":"2024-09-17T05:30:25.994100+0000","last_became_active":"2024-09-17T05:30:24.967197+0000","last_became_peered":"2024-09-17T05:30:24.967197+0000","last_unstale":"2024-09-17T05:30:25.994100+0000","last_undegraded":"2024-09-17T05:30:25.994100+0000","last_fullsized":"2024-09-17T05:30:25.994100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:13.435390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983331+0000","last_change":"2024-09-17T05:30:24.966805+0000","last_active":"2024-09-17T05:30:25.983331+0000","last_peered":"2024-09-17T05:30:25.983331+0000","last_clean":"2024-09-17T05:30:25.983331+0000","last_became_active":"2024-09-17T05:30:24.966550+0000","last_became_peered":"2024-09-17T05:30:24.966550+0000","last_unstale":"2024-09-17T05:30:25.983331+0000","last_undegraded":"2024-09-17T05:30:25.983331+0000","last_fullsized":"2024-09-17T05:30:25.983331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:40.647222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974503+0000","last_change":"2024-09-17T05:30:24.975269+0000","last_active":"2024-09-17T05:30:25.974503+0000","last_peered":"2024-09-17T05:30:25.974503+0000","last_clean":"2024-09-17T05:30:25.974503+0000","last_became_active":"2024-09-17T05:30:24.973279+0000","last_became_peered":"2024-09-17T05:30:24.973279+0000","last_unstale":"2024-09-17T05:30:25.974503+0000","last_undegraded":"2024-09-17T05:30:25.974503+0000","last_fullsized":"2024-09-17T05:30:25.974503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:13.463207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994024+0000","last_change":"2024-09-17T05:30:24.960568+0000","last_active":"2024-09-17T05:30:25.994024+0000","last_peered":"2024-09-17T05:30:25.994024+0000","last_clean":"2024-09-17T05:30:25.994024+0000","last_became_active":"2024-09-17T05:30:24.960307+0000","last_became_peered":"2024-09-17T05:30:24.960307+0000","last_unstale":"2024-09-17T05:30:25.994024+0000","last_undegraded":"2024-09-17T05:30:25.994024+0000","last_fullsized":"2024-09-17T05:30:25.994024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:19.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426598+0000","last_change":"2024-09-17T05:30:24.960733+0000","last_active":"2024-09-17T05:30:26.426598+0000","last_peered":"2024-09-17T05:30:26.426598+0000","last_clean":"2024-09-17T05:30:26.426598+0000","last_became_active":"2024-09-17T05:30:24.960477+0000","last_became_peered":"2024-09-17T05:30:24.960477+0000","last_unstale":"2024-09-17T05:30:26.426598+0000","last_undegraded":"2024-09-17T05:30:26.426598+0000","last_fullsized":"2024-09-17T05:30:26.426598+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:06.403444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.953915+0000","last_change":"2024-09-17T05:30:24.965125+0000","last_active":"2024-09-17T05:30:25.953915+0000","last_peered":"2024-09-17T05:30:25.953915+0000","last_clean":"2024-09-17T05:30:25.953915+0000","last_became_active":"2024-09-17T05:30:24.964872+0000","last_became_peered":"2024-09-17T05:30:24.964872+0000","last_unstale":"2024-09-17T05:30:25.953915+0000","last_undegraded":"2024-09-17T05:30:25.953915+0000","last_fullsized":"2024-09-17T05:30:25.953915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:12.841910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426287+0000","last_change":"2024-09-17T05:30:24.963095+0000","last_active":"2024-09-17T05:30:26.426287+0000","last_peered":"2024-09-17T05:30:26.426287+0000","last_clean":"2024-09-17T05:30:26.426287+0000","last_became_active":"2024-09-17T05:30:24.962824+0000","last_became_peered":"2024-09-17T05:30:24.962824+0000","last_unstale":"2024-09-17T05:30:26.426287+0000","last_undegraded":"2024-09-17T05:30:26.426287+0000","last_fullsized":"2024-09-17T05:30:26.426287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:53.586966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982781+0000","last_change":"2024-09-17T05:30:24.958007+0000","last_active":"2024-09-17T05:30:25.982781+0000","last_peered":"2024-09-17T05:30:25.982781+0000","last_clean":"2024-09-17T05:30:25.982781+0000","last_became_active":"2024-09-17T05:30:24.957745+0000","last_became_peered":"2024-09-17T05:30:24.957745+0000","last_unstale":"2024-09-17T05:30:25.982781+0000","last_undegraded":"2024-09-17T05:30:25.982781+0000","last_fullsized":"2024-09-17T05:30:25.982781+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:21.238127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983434+0000","last_change":"2024-09-17T05:30:24.963597+0000","last_active":"2024-09-17T05:30:25.983434+0000","last_peered":"2024-09-17T05:30:25.983434+0000","last_clean":"2024-09-17T05:30:25.983434+0000","last_became_active":"2024-09-17T05:30:24.962945+0000","last_became_peered":"2024-09-17T05:30:24.962945+0000","last_unstale":"2024-09-17T05:30:25.983434+0000","last_undegraded":"2024-09-17T05:30:25.983434+0000","last_fullsized":"2024-09-17T05:30:25.983434+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:45.758152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954245+0000","last_change":"2024-09-17T05:30:24.958526+0000","last_active":"2024-09-17T05:30:25.954245+0000","last_peered":"2024-09-17T05:30:25.954245+0000","last_clean":"2024-09-17T05:30:25.954245+0000","last_became_active":"2024-09-17T05:30:24.958238+0000","last_became_peered":"2024-09-17T05:30:24.958238+0000","last_unstale":"2024-09-17T05:30:25.954245+0000","last_undegraded":"2024-09-17T05:30:25.954245+0000","last_fullsized":"2024-09-17T05:30:25.954245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:16.257183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959481+0000","last_change":"2024-09-17T05:30:24.970050+0000","last_active":"2024-09-17T05:30:25.959481+0000","last_peered":"2024-09-17T05:30:25.959481+0000","last_clean":"2024-09-17T05:30:25.959481+0000","last_became_active":"2024-09-17T05:30:24.969733+0000","last_became_peered":"2024-09-17T05:30:24.969733+0000","last_unstale":"2024-09-17T05:30:25.959481+0000","last_undegraded":"2024-09-17T05:30:25.959481+0000","last_fullsized":"2024-09-17T05:30:25.959481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:42.759252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974460+0000","last_change":"2024-09-17T05:30:24.973029+0000","last_active":"2024-09-17T05:30:25.974460+0000","last_peered":"2024-09-17T05:30:25.974460+0000","last_clean":"2024-09-17T05:30:25.974460+0000","last_became_active":"2024-09-17T05:30:24.972776+0000","last_became_peered":"2024-09-17T05:30:24.972776+0000","last_unstale":"2024-09-17T05:30:25.974460+0000","last_undegraded":"2024-09-17T05:30:25.974460+0000","last_fullsized":"2024-09-17T05:30:25.974460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:41.320291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426443+0000","last_change":"2024-09-17T05:30:24.959367+0000","last_active":"2024-09-17T05:30:26.426443+0000","last_peered":"2024-09-17T05:30:26.426443+0000","last_clean":"2024-09-17T05:30:26.426443+0000","last_became_active":"2024-09-17T05:30:24.958664+0000","last_became_peered":"2024-09-17T05:30:24.958664+0000","last_unstale":"2024-09-17T05:30:26.426443+0000","last_undegraded":"2024-09-17T05:30:26.426443+0000","last_fullsized":"2024-09-17T05:30:26.426443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:22.869759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974472+0000","last_change":"2024-09-17T05:30:24.962135+0000","last_active":"2024-09-17T05:30:25.974472+0000","last_peered":"2024-09-17T05:30:25.974472+0000","last_clean":"2024-09-17T05:30:25.974472+0000","last_became_active":"2024-09-17T05:30:24.960058+0000","last_became_peered":"2024-09-17T05:30:24.960058+0000","last_unstale":"2024-09-17T05:30:25.974472+0000","last_undegraded":"2024-09-17T05:30:25.974472+0000","last_fullsized":"2024-09-17T05:30:25.974472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:16.257183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956616+0000","last_change":"2024-09-17T05:30:24.970499+0000","last_active":"2024-09-17T05:30:25.956616+0000","last_peered":"2024-09-17T05:30:25.956616+0000","last_clean":"2024-09-17T05:30:25.956616+0000","last_became_active":"2024-09-17T05:30:24.970051+0000","last_became_peered":"2024-09-17T05:30:24.970051+0000","last_unstale":"2024-09-17T05:30:25.956616+0000","last_undegraded":"2024-09-17T05:30:25.956616+0000","last_fullsized":"2024-09-17T05:30:25.956616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:37.784128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994259+0000","last_change":"2024-09-17T05:30:24.967415+0000","last_active":"2024-09-17T05:30:25.994259+0000","last_peered":"2024-09-17T05:30:25.994259+0000","last_clean":"2024-09-17T05:30:25.994259+0000","last_became_active":"2024-09-17T05:30:24.967121+0000","last_became_peered":"2024-09-17T05:30:24.967121+0000","last_unstale":"2024-09-17T05:30:25.994259+0000","last_undegraded":"2024-09-17T05:30:25.994259+0000","last_fullsized":"2024-09-17T05:30:25.994259+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:06.243934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959497+0000","last_change":"2024-09-17T05:30:24.969217+0000","last_active":"2024-09-17T05:30:25.959497+0000","last_peered":"2024-09-17T05:30:25.959497+0000","last_clean":"2024-09-17T05:30:25.959497+0000","last_became_active":"2024-09-17T05:30:24.968970+0000","last_became_peered":"2024-09-17T05:30:24.968970+0000","last_unstale":"2024-09-17T05:30:25.959497+0000","last_undegraded":"2024-09-17T05:30:25.959497+0000","last_fullsized":"2024-09-17T05:30:25.959497+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:24.928889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426617+0000","last_change":"2024-09-17T05:30:24.962929+0000","last_active":"2024-09-17T05:30:26.426617+0000","last_peered":"2024-09-17T05:30:26.426617+0000","last_clean":"2024-09-17T05:30:26.426617+0000","last_became_active":"2024-09-17T05:30:24.962618+0000","last_became_peered":"2024-09-17T05:30:24.962618+0000","last_unstale":"2024-09-17T05:30:26.426617+0000","last_undegraded":"2024-09-17T05:30:26.426617+0000","last_fullsized":"2024-09-17T05:30:26.426617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:22.425017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.960023+0000","last_change":"2024-09-17T05:30:24.959497+0000","last_active":"2024-09-17T05:30:25.960023+0000","last_peered":"2024-09-17T05:30:25.960023+0000","last_clean":"2024-09-17T05:30:25.960023+0000","last_became_active":"2024-09-17T05:30:24.959125+0000","last_became_peered":"2024-09-17T05:30:24.959125+0000","last_unstale":"2024-09-17T05:30:25.960023+0000","last_undegraded":"2024-09-17T05:30:25.960023+0000","last_fullsized":"2024-09-17T05:30:25.960023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:21.238127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864500+0000","last_change":"2024-09-17T05:30:24.976244+0000","last_active":"2024-09-17T05:30:27.864500+0000","last_peered":"2024-09-17T05:30:27.864500+0000","last_clean":"2024-09-17T05:30:27.864500+0000","last_became_active":"2024-09-17T05:30:24.975619+0000","last_became_peered":"2024-09-17T05:30:24.975619+0000","last_unstale":"2024-09-17T05:30:27.864500+0000","last_undegraded":"2024-09-17T05:30:27.864500+0000","last_fullsized":"2024-09-17T05:30:27.864500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:42.656824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864466+0000","last_change":"2024-09-17T05:30:24.972218+0000","last_active":"2024-09-17T05:30:27.864466+0000","last_peered":"2024-09-17T05:30:27.864466+0000","last_clean":"2024-09-17T05:30:27.864466+0000","last_became_active":"2024-09-17T05:30:24.969583+0000","last_became_peered":"2024-09-17T05:30:24.969583+0000","last_unstale":"2024-09-17T05:30:27.864466+0000","last_undegraded":"2024-09-17T05:30:27.864466+0000","last_fullsized":"2024-09-17T05:30:27.864466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:45.665750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956579+0000","last_change":"2024-09-17T05:30:24.963425+0000","last_active":"2024-09-17T05:30:25.956579+0000","last_peered":"2024-09-17T05:30:25.956579+0000","last_clean":"2024-09-17T05:30:25.956579+0000","last_became_active":"2024-09-17T05:30:24.961501+0000","last_became_peered":"2024-09-17T05:30:24.961501+0000","last_unstale":"2024-09-17T05:30:25.956579+0000","last_undegraded":"2024-09-17T05:30:25.956579+0000","last_fullsized":"2024-09-17T05:30:25.956579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:00.843710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959738+0000","last_change":"2024-09-17T05:30:24.955853+0000","last_active":"2024-09-17T05:30:25.959738+0000","last_peered":"2024-09-17T05:30:25.959738+0000","last_clean":"2024-09-17T05:30:25.959738+0000","last_became_active":"2024-09-17T05:30:24.955275+0000","last_became_peered":"2024-09-17T05:30:24.955275+0000","last_unstale":"2024-09-17T05:30:25.959738+0000","last_undegraded":"2024-09-17T05:30:25.959738+0000","last_fullsized":"2024-09-17T05:30:25.959738+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:52.400527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974127+0000","last_change":"2024-09-17T05:30:24.959364+0000","last_active":"2024-09-17T05:30:25.974127+0000","last_peered":"2024-09-17T05:30:25.974127+0000","last_clean":"2024-09-17T05:30:25.974127+0000","last_became_active":"2024-09-17T05:30:24.959037+0000","last_became_peered":"2024-09-17T05:30:24.959037+0000","last_unstale":"2024-09-17T05:30:25.974127+0000","last_undegraded":"2024-09-17T05:30:25.974127+0000","last_fullsized":"2024-09-17T05:30:25.974127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:12.841910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983352+0000","last_change":"2024-09-17T05:30:24.962003+0000","last_active":"2024-09-17T05:30:25.983352+0000","last_peered":"2024-09-17T05:30:25.983352+0000","last_clean":"2024-09-17T05:30:25.983352+0000","last_became_active":"2024-09-17T05:30:24.961160+0000","last_became_peered":"2024-09-17T05:30:24.961160+0000","last_unstale":"2024-09-17T05:30:25.983352+0000","last_undegraded":"2024-09-17T05:30:25.983352+0000","last_fullsized":"2024-09-17T05:30:25.983352+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:22.869759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974708+0000","last_change":"2024-09-17T05:30:23.950637+0000","last_active":"2024-09-17T05:30:25.974708+0000","last_peered":"2024-09-17T05:30:25.974708+0000","last_clean":"2024-09-17T05:30:25.974708+0000","last_became_active":"2024-09-17T05:30:23.950363+0000","last_became_peered":"2024-09-17T05:30:23.950363+0000","last_unstale":"2024-09-17T05:30:25.974708+0000","last_undegraded":"2024-09-17T05:30:25.974708+0000","last_fullsized":"2024-09-17T05:30:25.974708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:52.580019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983643+0000","last_change":"2024-09-17T05:30:24.968089+0000","last_active":"2024-09-17T05:30:25.983643+0000","last_peered":"2024-09-17T05:30:25.983643+0000","last_clean":"2024-09-17T05:30:25.983643+0000","last_became_active":"2024-09-17T05:30:24.967469+0000","last_became_peered":"2024-09-17T05:30:24.967469+0000","last_unstale":"2024-09-17T05:30:25.983643+0000","last_undegraded":"2024-09-17T05:30:25.983643+0000","last_fullsized":"2024-09-17T05:30:25.983643+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:07.410207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959477+0000","last_change":"2024-09-17T05:30:23.952541+0000","last_active":"2024-09-17T05:30:25.959477+0000","last_peered":"2024-09-17T05:30:25.959477+0000","last_clean":"2024-09-17T05:30:25.959477+0000","last_became_active":"2024-09-17T05:30:23.952190+0000","last_became_peered":"2024-09-17T05:30:23.952190+0000","last_unstale":"2024-09-17T05:30:25.959477+0000","last_undegraded":"2024-09-17T05:30:25.959477+0000","last_fullsized":"2024-09-17T05:30:25.959477+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:07.976069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864584+0000","last_change":"2024-09-17T05:30:24.976969+0000","last_active":"2024-09-17T05:30:27.864584+0000","last_peered":"2024-09-17T05:30:27.864584+0000","last_clean":"2024-09-17T05:30:27.864584+0000","last_became_active":"2024-09-17T05:30:24.976702+0000","last_became_peered":"2024-09-17T05:30:24.976702+0000","last_unstale":"2024-09-17T05:30:27.864584+0000","last_undegraded":"2024-09-17T05:30:27.864584+0000","last_fullsized":"2024-09-17T05:30:27.864584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:29.194444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983310+0000","last_change":"2024-09-17T05:30:23.958614+0000","last_active":"2024-09-17T05:30:25.983310+0000","last_peered":"2024-09-17T05:30:25.983310+0000","last_clean":"2024-09-17T05:30:25.983310+0000","last_became_active":"2024-09-17T05:30:23.958200+0000","last_became_peered":"2024-09-17T05:30:23.958200+0000","last_unstale":"2024-09-17T05:30:25.983310+0000","last_undegraded":"2024-09-17T05:30:25.983310+0000","last_fullsized":"2024-09-17T05:30:25.983310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:20.323922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983069+0000","last_change":"2024-09-17T05:30:24.969992+0000","last_active":"2024-09-17T05:30:25.983069+0000","last_peered":"2024-09-17T05:30:25.983069+0000","last_clean":"2024-09-17T05:30:25.983069+0000","last_became_active":"2024-09-17T05:30:24.969679+0000","last_became_peered":"2024-09-17T05:30:24.969679+0000","last_unstale":"2024-09-17T05:30:25.983069+0000","last_undegraded":"2024-09-17T05:30:25.983069+0000","last_fullsized":"2024-09-17T05:30:25.983069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:05.420612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956851+0000","last_change":"2024-09-17T05:30:23.940517+0000","last_active":"2024-09-17T05:30:25.956851+0000","last_peered":"2024-09-17T05:30:25.956851+0000","last_clean":"2024-09-17T05:30:25.956851+0000","last_became_active":"2024-09-17T05:30:23.940255+0000","last_became_peered":"2024-09-17T05:30:23.940255+0000","last_unstale":"2024-09-17T05:30:25.956851+0000","last_undegraded":"2024-09-17T05:30:25.956851+0000","last_fullsized":"2024-09-17T05:30:25.956851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:55.067522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864815+0000","last_change":"2024-09-17T05:30:24.974140+0000","last_active":"2024-09-17T05:30:27.864815+0000","last_peered":"2024-09-17T05:30:27.864815+0000","last_clean":"2024-09-17T05:30:27.864815+0000","last_became_active":"2024-09-17T05:30:24.973833+0000","last_became_peered":"2024-09-17T05:30:24.973833+0000","last_unstale":"2024-09-17T05:30:27.864815+0000","last_undegraded":"2024-09-17T05:30:27.864815+0000","last_fullsized":"2024-09-17T05:30:27.864815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:03:28.597753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954167+0000","last_change":"2024-09-17T05:30:23.946652+0000","last_active":"2024-09-17T05:30:25.954167+0000","last_peered":"2024-09-17T05:30:25.954167+0000","last_clean":"2024-09-17T05:30:25.954167+0000","last_became_active":"2024-09-17T05:30:23.946243+0000","last_became_peered":"2024-09-17T05:30:23.946243+0000","last_unstale":"2024-09-17T05:30:25.954167+0000","last_undegraded":"2024-09-17T05:30:25.954167+0000","last_fullsized":"2024-09-17T05:30:25.954167+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:37.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974481+0000","last_change":"2024-09-17T05:30:24.961529+0000","last_active":"2024-09-17T05:30:25.974481+0000","last_peered":"2024-09-17T05:30:25.974481+0000","last_clean":"2024-09-17T05:30:25.974481+0000","last_became_active":"2024-09-17T05:30:24.961275+0000","last_became_peered":"2024-09-17T05:30:24.961275+0000","last_unstale":"2024-09-17T05:30:25.974481+0000","last_undegraded":"2024-09-17T05:30:25.974481+0000","last_fullsized":"2024-09-17T05:30:25.974481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:30.145289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983082+0000","last_change":"2024-09-17T05:30:23.953938+0000","last_active":"2024-09-17T05:30:25.983082+0000","last_peered":"2024-09-17T05:30:25.983082+0000","last_clean":"2024-09-17T05:30:25.983082+0000","last_became_active":"2024-09-17T05:30:23.953683+0000","last_became_peered":"2024-09-17T05:30:23.953683+0000","last_unstale":"2024-09-17T05:30:25.983082+0000","last_undegraded":"2024-09-17T05:30:25.983082+0000","last_fullsized":"2024-09-17T05:30:25.983082+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:12.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.953751+0000","last_change":"2024-09-17T05:30:24.955868+0000","last_active":"2024-09-17T05:30:25.953751+0000","last_peered":"2024-09-17T05:30:25.953751+0000","last_clean":"2024-09-17T05:30:25.953751+0000","last_became_active":"2024-09-17T05:30:24.955453+0000","last_became_peered":"2024-09-17T05:30:24.955453+0000","last_unstale":"2024-09-17T05:30:25.953751+0000","last_undegraded":"2024-09-17T05:30:25.953751+0000","last_fullsized":"2024-09-17T05:30:25.953751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:38.689325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994012+0000","last_change":"2024-09-17T05:30:23.948325+0000","last_active":"2024-09-17T05:30:25.994012+0000","last_peered":"2024-09-17T05:30:25.994012+0000","last_clean":"2024-09-17T05:30:25.994012+0000","last_became_active":"2024-09-17T05:30:23.948001+0000","last_became_peered":"2024-09-17T05:30:23.948001+0000","last_unstale":"2024-09-17T05:30:25.994012+0000","last_undegraded":"2024-09-17T05:30:25.994012+0000","last_fullsized":"2024-09-17T05:30:25.994012+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:30.593870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982787+0000","last_change":"2024-09-17T05:30:24.969386+0000","last_active":"2024-09-17T05:30:25.982787+0000","last_peered":"2024-09-17T05:30:25.982787+0000","last_clean":"2024-09-17T05:30:25.982787+0000","last_became_active":"2024-09-17T05:30:24.968937+0000","last_became_peered":"2024-09-17T05:30:24.968937+0000","last_unstale":"2024-09-17T05:30:25.982787+0000","last_undegraded":"2024-09-17T05:30:25.982787+0000","last_fullsized":"2024-09-17T05:30:25.982787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:06.136871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426207+0000","last_change":"2024-09-17T05:30:23.954282+0000","last_active":"2024-09-17T05:30:26.426207+0000","last_peered":"2024-09-17T05:30:26.426207+0000","last_clean":"2024-09-17T05:30:26.426207+0000","last_became_active":"2024-09-17T05:30:23.953881+0000","last_became_peered":"2024-09-17T05:30:23.953881+0000","last_unstale":"2024-09-17T05:30:26.426207+0000","last_undegraded":"2024-09-17T05:30:26.426207+0000","last_fullsized":"2024-09-17T05:30:26.426207+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:41.752305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426661+0000","last_change":"2024-09-17T05:30:24.960242+0000","last_active":"2024-09-17T05:30:26.426661+0000","last_peered":"2024-09-17T05:30:26.426661+0000","last_clean":"2024-09-17T05:30:26.426661+0000","last_became_active":"2024-09-17T05:30:24.959958+0000","last_became_peered":"2024-09-17T05:30:24.959958+0000","last_unstale":"2024-09-17T05:30:26.426661+0000","last_undegraded":"2024-09-17T05:30:26.426661+0000","last_fullsized":"2024-09-17T05:30:26.426661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:05.377943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994222+0000","last_change":"2024-09-17T05:30:23.943335+0000","last_active":"2024-09-17T05:30:25.994222+0000","last_peered":"2024-09-17T05:30:25.994222+0000","last_clean":"2024-09-17T05:30:25.994222+0000","last_became_active":"2024-09-17T05:30:23.943018+0000","last_became_peered":"2024-09-17T05:30:23.943018+0000","last_unstale":"2024-09-17T05:30:25.994222+0000","last_undegraded":"2024-09-17T05:30:25.994222+0000","last_fullsized":"2024-09-17T05:30:25.994222+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:19.341040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954591+0000","last_change":"2024-09-17T05:30:24.972619+0000","last_active":"2024-09-17T05:30:25.954591+0000","last_peered":"2024-09-17T05:30:25.954591+0000","last_clean":"2024-09-17T05:30:25.954591+0000","last_became_active":"2024-09-17T05:30:24.971682+0000","last_became_peered":"2024-09-17T05:30:24.971682+0000","last_unstale":"2024-09-17T05:30:25.954591+0000","last_undegraded":"2024-09-17T05:30:25.954591+0000","last_fullsized":"2024-09-17T05:30:25.954591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:16.312534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954980+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954714+0000","last_became_peered":"2024-09-17T05:30:23.954714+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:24.175086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864850+0000","last_change":"2024-09-17T05:30:24.976889+0000","last_active":"2024-09-17T05:30:27.864850+0000","last_peered":"2024-09-17T05:30:27.864850+0000","last_clean":"2024-09-17T05:30:27.864850+0000","last_became_active":"2024-09-17T05:30:24.976636+0000","last_became_peered":"2024-09-17T05:30:24.976636+0000","last_unstale":"2024-09-17T05:30:27.864850+0000","last_undegraded":"2024-09-17T05:30:27.864850+0000","last_fullsized":"2024-09-17T05:30:27.864850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:54.276295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.957049+0000","last_change":"2024-09-17T05:30:23.942665+0000","last_active":"2024-09-17T05:30:25.957049+0000","last_peered":"2024-09-17T05:30:25.957049+0000","last_clean":"2024-09-17T05:30:25.957049+0000","last_became_active":"2024-09-17T05:30:23.942377+0000","last_became_peered":"2024-09-17T05:30:23.942377+0000","last_unstale":"2024-09-17T05:30:25.957049+0000","last_undegraded":"2024-09-17T05:30:25.957049+0000","last_fullsized":"2024-09-17T05:30:25.957049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:20.108443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956927+0000","last_change":"2024-09-17T05:30:24.972003+0000","last_active":"2024-09-17T05:30:25.956927+0000","last_peered":"2024-09-17T05:30:25.956927+0000","last_clean":"2024-09-17T05:30:25.956927+0000","last_became_active":"2024-09-17T05:30:24.971652+0000","last_became_peered":"2024-09-17T05:30:24.971652+0000","last_unstale":"2024-09-17T05:30:25.956927+0000","last_undegraded":"2024-09-17T05:30:25.956927+0000","last_fullsized":"2024-09-17T05:30:25.956927+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:24:19.300631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":21,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426555+0000","last_change":"2024-09-17T05:30:23.953324+0000","last_active":"2024-09-17T05:30:26.426555+0000","last_peered":"2024-09-17T05:30:26.426555+0000","last_clean":"2024-09-17T05:30:26.426555+0000","last_became_active":"2024-09-17T05:30:23.953049+0000","last_became_peered":"2024-09-17T05:30:23.953049+0000","last_unstale":"2024-09-17T05:30:26.426555+0000","last_undegraded":"2024-09-17T05:30:26.426555+0000","last_fullsized":"2024-09-17T05:30:26.426555+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:51.393580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.993941+0000","last_change":"2024-09-17T05:30:24.957930+0000","last_active":"2024-09-17T05:30:25.993941+0000","last_peered":"2024-09-17T05:30:25.993941+0000","last_clean":"2024-09-17T05:30:25.993941+0000","last_became_active":"2024-09-17T05:30:24.957640+0000","last_became_peered":"2024-09-17T05:30:24.957640+0000","last_unstale":"2024-09-17T05:30:25.993941+0000","last_undegraded":"2024-09-17T05:30:25.993941+0000","last_fullsized":"2024-09-17T05:30:25.993941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:13.307547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864589+0000","last_change":"2024-09-17T05:30:23.954792+0000","last_active":"2024-09-17T05:30:27.864589+0000","last_peered":"2024-09-17T05:30:27.864589+0000","last_clean":"2024-09-17T05:30:27.864589+0000","last_became_active":"2024-09-17T05:30:23.954435+0000","last_became_peered":"2024-09-17T05:30:23.954435+0000","last_unstale":"2024-09-17T05:30:27.864589+0000","last_undegraded":"2024-09-17T05:30:27.864589+0000","last_fullsized":"2024-09-17T05:30:27.864589+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:30.663318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954015+0000","last_change":"2024-09-17T05:30:24.952788+0000","last_active":"2024-09-17T05:30:25.954015+0000","last_peered":"2024-09-17T05:30:25.954015+0000","last_clean":"2024-09-17T05:30:25.954015+0000","last_became_active":"2024-09-17T05:30:24.952549+0000","last_became_peered":"2024-09-17T05:30:24.952549+0000","last_unstale":"2024-09-17T05:30:25.954015+0000","last_undegraded":"2024-09-17T05:30:25.954015+0000","last_fullsized":"2024-09-17T05:30:25.954015+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:30.145289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956561+0000","last_change":"2024-09-17T05:30:23.948409+0000","last_active":"2024-09-17T05:30:25.956561+0000","last_peered":"2024-09-17T05:30:25.956561+0000","last_clean":"2024-09-17T05:30:25.956561+0000","last_became_active":"2024-09-17T05:30:23.948114+0000","last_became_peered":"2024-09-17T05:30:23.948114+0000","last_unstale":"2024-09-17T05:30:25.956561+0000","last_undegraded":"2024-09-17T05:30:25.956561+0000","last_fullsized":"2024-09-17T05:30:25.956561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:06.470015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994014+0000","last_change":"2024-09-17T05:30:24.962071+0000","last_active":"2024-09-17T05:30:25.994014+0000","last_peered":"2024-09-17T05:30:25.994014+0000","last_clean":"2024-09-17T05:30:25.994014+0000","last_became_active":"2024-09-17T05:30:24.960113+0000","last_became_peered":"2024-09-17T05:30:24.960113+0000","last_unstale":"2024-09-17T05:30:25.994014+0000","last_undegraded":"2024-09-17T05:30:25.994014+0000","last_fullsized":"2024-09-17T05:30:25.994014+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:21.330869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956782+0000","last_change":"2024-09-17T05:30:23.943942+0000","last_active":"2024-09-17T05:30:25.956782+0000","last_peered":"2024-09-17T05:30:25.956782+0000","last_clean":"2024-09-17T05:30:25.956782+0000","last_became_active":"2024-09-17T05:30:23.943669+0000","last_became_peered":"2024-09-17T05:30:23.943669+0000","last_unstale":"2024-09-17T05:30:25.956782+0000","last_undegraded":"2024-09-17T05:30:25.956782+0000","last_fullsized":"2024-09-17T05:30:25.956782+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:13.811827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956846+0000","last_change":"2024-09-17T05:30:24.960276+0000","last_active":"2024-09-17T05:30:25.956846+0000","last_peered":"2024-09-17T05:30:25.956846+0000","last_clean":"2024-09-17T05:30:25.956846+0000","last_became_active":"2024-09-17T05:30:24.959383+0000","last_became_peered":"2024-09-17T05:30:24.959383+0000","last_unstale":"2024-09-17T05:30:25.956846+0000","last_undegraded":"2024-09-17T05:30:25.956846+0000","last_fullsized":"2024-09-17T05:30:25.956846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:49.583118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":23,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956508+0000","last_change":"2024-09-17T05:30:23.949103+0000","last_active":"2024-09-17T05:30:25.956508+0000","last_peered":"2024-09-17T05:30:25.956508+0000","last_clean":"2024-09-17T05:30:25.956508+0000","last_became_active":"2024-09-17T05:30:23.948810+0000","last_became_peered":"2024-09-17T05:30:23.948810+0000","last_unstale":"2024-09-17T05:30:25.956508+0000","last_undegraded":"2024-09-17T05:30:25.956508+0000","last_fullsized":"2024-09-17T05:30:25.956508+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:47.103339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954276+0000","last_change":"2024-09-17T05:30:24.958134+0000","last_active":"2024-09-17T05:30:25.954276+0000","last_peered":"2024-09-17T05:30:25.954276+0000","last_clean":"2024-09-17T05:30:25.954276+0000","last_became_active":"2024-09-17T05:30:24.957609+0000","last_became_peered":"2024-09-17T05:30:24.957609+0000","last_unstale":"2024-09-17T05:30:25.954276+0000","last_undegraded":"2024-09-17T05:30:25.954276+0000","last_fullsized":"2024-09-17T05:30:25.954276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:17:13.573184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959488+0000","last_change":"2024-09-17T05:30:23.960591+0000","last_active":"2024-09-17T05:30:25.959488+0000","last_peered":"2024-09-17T05:30:25.959488+0000","last_clean":"2024-09-17T05:30:25.959488+0000","last_became_active":"2024-09-17T05:30:23.960068+0000","last_became_peered":"2024-09-17T05:30:23.960068+0000","last_unstale":"2024-09-17T05:30:25.959488+0000","last_undegraded":"2024-09-17T05:30:25.959488+0000","last_fullsized":"2024-09-17T05:30:25.959488+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:19.075142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954256+0000","last_change":"2024-09-17T05:30:24.965156+0000","last_active":"2024-09-17T05:30:25.954256+0000","last_peered":"2024-09-17T05:30:25.954256+0000","last_clean":"2024-09-17T05:30:25.954256+0000","last_became_active":"2024-09-17T05:30:24.964861+0000","last_became_peered":"2024-09-17T05:30:24.964861+0000","last_unstale":"2024-09-17T05:30:25.954256+0000","last_undegraded":"2024-09-17T05:30:25.954256+0000","last_fullsized":"2024-09-17T05:30:25.954256+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:33:05.958676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864645+0000","last_change":"2024-09-17T05:30:23.951223+0000","last_active":"2024-09-17T05:30:27.864645+0000","last_peered":"2024-09-17T05:30:27.864645+0000","last_clean":"2024-09-17T05:30:27.864645+0000","last_became_active":"2024-09-17T05:30:23.950514+0000","last_became_peered":"2024-09-17T05:30:23.950514+0000","last_unstale":"2024-09-17T05:30:27.864645+0000","last_undegraded":"2024-09-17T05:30:27.864645+0000","last_fullsized":"2024-09-17T05:30:27.864645+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:45.736482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959277+0000","last_change":"2024-09-17T05:30:24.966104+0000","last_active":"2024-09-17T05:30:25.959277+0000","last_peered":"2024-09-17T05:30:25.959277+0000","last_clean":"2024-09-17T05:30:25.959277+0000","last_became_active":"2024-09-17T05:30:24.964848+0000","last_became_peered":"2024-09-17T05:30:24.964848+0000","last_unstale":"2024-09-17T05:30:25.959277+0000","last_undegraded":"2024-09-17T05:30:25.959277+0000","last_fullsized":"2024-09-17T05:30:25.959277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:28.543306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954296+0000","last_change":"2024-09-17T05:30:23.944667+0000","last_active":"2024-09-17T05:30:25.954296+0000","last_peered":"2024-09-17T05:30:25.954296+0000","last_clean":"2024-09-17T05:30:25.954296+0000","last_became_active":"2024-09-17T05:30:23.944381+0000","last_became_peered":"2024-09-17T05:30:23.944381+0000","last_unstale":"2024-09-17T05:30:25.954296+0000","last_undegraded":"2024-09-17T05:30:25.954296+0000","last_fullsized":"2024-09-17T05:30:25.954296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:20.231180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.956948+0000","last_change":"2024-09-17T05:30:24.966889+0000","last_active":"2024-09-17T05:30:25.956948+0000","last_peered":"2024-09-17T05:30:25.956948+0000","last_clean":"2024-09-17T05:30:25.956948+0000","last_became_active":"2024-09-17T05:30:24.966626+0000","last_became_peered":"2024-09-17T05:30:24.966626+0000","last_unstale":"2024-09-17T05:30:25.956948+0000","last_undegraded":"2024-09-17T05:30:25.956948+0000","last_fullsized":"2024-09-17T05:30:25.956948+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:33.137934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864732+0000","last_change":"2024-09-17T05:30:23.960543+0000","last_active":"2024-09-17T05:30:27.864732+0000","last_peered":"2024-09-17T05:30:27.864732+0000","last_clean":"2024-09-17T05:30:27.864732+0000","last_became_active":"2024-09-17T05:30:23.960241+0000","last_became_peered":"2024-09-17T05:30:23.960241+0000","last_unstale":"2024-09-17T05:30:27.864732+0000","last_undegraded":"2024-09-17T05:30:27.864732+0000","last_fullsized":"2024-09-17T05:30:27.864732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:25.924845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994184+0000","last_change":"2024-09-17T05:30:24.961787+0000","last_active":"2024-09-17T05:30:25.994184+0000","last_peered":"2024-09-17T05:30:25.994184+0000","last_clean":"2024-09-17T05:30:25.994184+0000","last_became_active":"2024-09-17T05:30:24.961480+0000","last_became_peered":"2024-09-17T05:30:24.961480+0000","last_unstale":"2024-09-17T05:30:25.994184+0000","last_undegraded":"2024-09-17T05:30:25.994184+0000","last_fullsized":"2024-09-17T05:30:25.994184+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:19.521784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.974371+0000","last_change":"2024-09-17T05:30:23.951379+0000","last_active":"2024-09-17T05:30:25.974371+0000","last_peered":"2024-09-17T05:30:25.974371+0000","last_clean":"2024-09-17T05:30:25.974371+0000","last_became_active":"2024-09-17T05:30:23.950900+0000","last_became_peered":"2024-09-17T05:30:23.950900+0000","last_unstale":"2024-09-17T05:30:25.974371+0000","last_undegraded":"2024-09-17T05:30:25.974371+0000","last_fullsized":"2024-09-17T05:30:25.974371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:24.722311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864503+0000","last_change":"2024-09-17T05:30:24.973721+0000","last_active":"2024-09-17T05:30:27.864503+0000","last_peered":"2024-09-17T05:30:27.864503+0000","last_clean":"2024-09-17T05:30:27.864503+0000","last_became_active":"2024-09-17T05:30:24.973391+0000","last_became_peered":"2024-09-17T05:30:24.973391+0000","last_unstale":"2024-09-17T05:30:27.864503+0000","last_undegraded":"2024-09-17T05:30:27.864503+0000","last_fullsized":"2024-09-17T05:30:27.864503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:43:02.895238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864244+0000","last_change":"2024-09-17T05:30:23.950654+0000","last_active":"2024-09-17T05:30:27.864244+0000","last_peered":"2024-09-17T05:30:27.864244+0000","last_clean":"2024-09-17T05:30:27.864244+0000","last_became_active":"2024-09-17T05:30:23.950372+0000","last_became_peered":"2024-09-17T05:30:23.950372+0000","last_unstale":"2024-09-17T05:30:27.864244+0000","last_undegraded":"2024-09-17T05:30:27.864244+0000","last_fullsized":"2024-09-17T05:30:27.864244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:46.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864458+0000","last_change":"2024-09-17T05:30:24.964323+0000","last_active":"2024-09-17T05:30:27.864458+0000","last_peered":"2024-09-17T05:30:27.864458+0000","last_clean":"2024-09-17T05:30:27.864458+0000","last_became_active":"2024-09-17T05:30:24.964039+0000","last_became_peered":"2024-09-17T05:30:24.964039+0000","last_unstale":"2024-09-17T05:30:27.864458+0000","last_undegraded":"2024-09-17T05:30:27.864458+0000","last_fullsized":"2024-09-17T05:30:27.864458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:52.710209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994152+0000","last_change":"2024-09-17T05:30:23.946216+0000","last_active":"2024-09-17T05:30:25.994152+0000","last_peered":"2024-09-17T05:30:25.994152+0000","last_clean":"2024-09-17T05:30:25.994152+0000","last_became_active":"2024-09-17T05:30:23.945797+0000","last_became_peered":"2024-09-17T05:30:23.945797+0000","last_unstale":"2024-09-17T05:30:25.994152+0000","last_undegraded":"2024-09-17T05:30:25.994152+0000","last_fullsized":"2024-09-17T05:30:25.994152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:55.988234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954490+0000","last_change":"2024-09-17T05:30:24.957869+0000","last_active":"2024-09-17T05:30:25.954490+0000","last_peered":"2024-09-17T05:30:25.954490+0000","last_clean":"2024-09-17T05:30:25.954490+0000","last_became_active":"2024-09-17T05:30:24.957469+0000","last_became_peered":"2024-09-17T05:30:24.957469+0000","last_unstale":"2024-09-17T05:30:25.954490+0000","last_undegraded":"2024-09-17T05:30:25.954490+0000","last_fullsized":"2024-09-17T05:30:25.954490+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:08.297928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959647+0000","last_change":"2024-09-17T05:30:23.953693+0000","last_active":"2024-09-17T05:30:25.959647+0000","last_peered":"2024-09-17T05:30:25.959647+0000","last_clean":"2024-09-17T05:30:25.959647+0000","last_became_active":"2024-09-17T05:30:23.953424+0000","last_became_peered":"2024-09-17T05:30:23.953424+0000","last_unstale":"2024-09-17T05:30:25.959647+0000","last_undegraded":"2024-09-17T05:30:25.959647+0000","last_fullsized":"2024-09-17T05:30:25.959647+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:08:30.772225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954494+0000","last_change":"2024-09-17T05:30:24.963268+0000","last_active":"2024-09-17T05:30:25.954494+0000","last_peered":"2024-09-17T05:30:25.954494+0000","last_clean":"2024-09-17T05:30:25.954494+0000","last_became_active":"2024-09-17T05:30:24.962823+0000","last_became_peered":"2024-09-17T05:30:24.962823+0000","last_unstale":"2024-09-17T05:30:25.954494+0000","last_undegraded":"2024-09-17T05:30:25.954494+0000","last_fullsized":"2024-09-17T05:30:25.954494+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:55.677952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983525+0000","last_change":"2024-09-17T05:30:23.958649+0000","last_active":"2024-09-17T05:30:25.983525+0000","last_peered":"2024-09-17T05:30:25.983525+0000","last_clean":"2024-09-17T05:30:25.983525+0000","last_became_active":"2024-09-17T05:30:23.958329+0000","last_became_peered":"2024-09-17T05:30:23.958329+0000","last_unstale":"2024-09-17T05:30:25.983525+0000","last_undegraded":"2024-09-17T05:30:25.983525+0000","last_fullsized":"2024-09-17T05:30:25.983525+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:18.514837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959686+0000","last_change":"2024-09-17T05:30:24.966794+0000","last_active":"2024-09-17T05:30:25.959686+0000","last_peered":"2024-09-17T05:30:25.959686+0000","last_clean":"2024-09-17T05:30:25.959686+0000","last_became_active":"2024-09-17T05:30:24.966503+0000","last_became_peered":"2024-09-17T05:30:24.966503+0000","last_unstale":"2024-09-17T05:30:25.959686+0000","last_undegraded":"2024-09-17T05:30:25.959686+0000","last_fullsized":"2024-09-17T05:30:25.959686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:38.777542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994067+0000","last_change":"2024-09-17T05:30:23.943837+0000","last_active":"2024-09-17T05:30:25.994067+0000","last_peered":"2024-09-17T05:30:25.994067+0000","last_clean":"2024-09-17T05:30:25.994067+0000","last_became_active":"2024-09-17T05:30:23.942677+0000","last_became_peered":"2024-09-17T05:30:23.942677+0000","last_unstale":"2024-09-17T05:30:25.994067+0000","last_undegraded":"2024-09-17T05:30:25.994067+0000","last_fullsized":"2024-09-17T05:30:25.994067+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:49.775010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864917+0000","last_change":"2024-09-17T05:30:24.974072+0000","last_active":"2024-09-17T05:30:27.864917+0000","last_peered":"2024-09-17T05:30:27.864917+0000","last_clean":"2024-09-17T05:30:27.864917+0000","last_became_active":"2024-09-17T05:30:24.973224+0000","last_became_peered":"2024-09-17T05:30:24.973224+0000","last_unstale":"2024-09-17T05:30:27.864917+0000","last_undegraded":"2024-09-17T05:30:27.864917+0000","last_fullsized":"2024-09-17T05:30:27.864917+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:35.858572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.954059+0000","last_change":"2024-09-17T05:30:23.947956+0000","last_active":"2024-09-17T05:30:25.954059+0000","last_peered":"2024-09-17T05:30:25.954059+0000","last_clean":"2024-09-17T05:30:25.954059+0000","last_became_active":"2024-09-17T05:30:23.947641+0000","last_became_peered":"2024-09-17T05:30:23.947641+0000","last_unstale":"2024-09-17T05:30:25.954059+0000","last_undegraded":"2024-09-17T05:30:25.954059+0000","last_fullsized":"2024-09-17T05:30:25.954059+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:19.881427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.983045+0000","last_change":"2024-09-17T05:30:24.967750+0000","last_active":"2024-09-17T05:30:25.983045+0000","last_peered":"2024-09-17T05:30:25.983045+0000","last_clean":"2024-09-17T05:30:25.983045+0000","last_became_active":"2024-09-17T05:30:24.967460+0000","last_became_peered":"2024-09-17T05:30:24.967460+0000","last_unstale":"2024-09-17T05:30:25.983045+0000","last_undegraded":"2024-09-17T05:30:25.983045+0000","last_fullsized":"2024-09-17T05:30:25.983045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:09:11.265178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959229+0000","last_change":"2024-09-17T05:30:23.957320+0000","last_active":"2024-09-17T05:30:25.959229+0000","last_peered":"2024-09-17T05:30:25.959229+0000","last_clean":"2024-09-17T05:30:25.959229+0000","last_became_active":"2024-09-17T05:30:23.956956+0000","last_became_peered":"2024-09-17T05:30:23.956956+0000","last_unstale":"2024-09-17T05:30:25.959229+0000","last_undegraded":"2024-09-17T05:30:25.959229+0000","last_fullsized":"2024-09-17T05:30:25.959229+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:44.089365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426661+0000","last_change":"2024-09-17T05:30:24.965327+0000","last_active":"2024-09-17T05:30:26.426661+0000","last_peered":"2024-09-17T05:30:26.426661+0000","last_clean":"2024-09-17T05:30:26.426661+0000","last_became_active":"2024-09-17T05:30:24.965023+0000","last_became_peered":"2024-09-17T05:30:24.965023+0000","last_unstale":"2024-09-17T05:30:26.426661+0000","last_undegraded":"2024-09-17T05:30:26.426661+0000","last_fullsized":"2024-09-17T05:30:26.426661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:18.712947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.953856+0000","last_change":"2024-09-17T05:30:23.945568+0000","last_active":"2024-09-17T05:30:25.953856+0000","last_peered":"2024-09-17T05:30:25.953856+0000","last_clean":"2024-09-17T05:30:25.953856+0000","last_became_active":"2024-09-17T05:30:23.945279+0000","last_became_peered":"2024-09-17T05:30:23.945279+0000","last_unstale":"2024-09-17T05:30:25.953856+0000","last_undegraded":"2024-09-17T05:30:25.953856+0000","last_fullsized":"2024-09-17T05:30:25.953856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:51.393580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864556+0000","last_change":"2024-09-17T05:30:24.974524+0000","last_active":"2024-09-17T05:30:27.864556+0000","last_peered":"2024-09-17T05:30:27.864556+0000","last_clean":"2024-09-17T05:30:27.864556+0000","last_became_active":"2024-09-17T05:30:24.974227+0000","last_became_peered":"2024-09-17T05:30:24.974227+0000","last_unstale":"2024-09-17T05:30:27.864556+0000","last_undegraded":"2024-09-17T05:30:27.864556+0000","last_fullsized":"2024-09-17T05:30:27.864556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:47.585236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.982856+0000","last_change":"2024-09-17T05:30:23.960348+0000","last_active":"2024-09-17T05:30:25.982856+0000","last_peered":"2024-09-17T05:30:25.982856+0000","last_clean":"2024-09-17T05:30:25.982856+0000","last_became_active":"2024-09-17T05:30:23.960056+0000","last_became_peered":"2024-09-17T05:30:23.960056+0000","last_unstale":"2024-09-17T05:30:25.982856+0000","last_undegraded":"2024-09-17T05:30:25.982856+0000","last_fullsized":"2024-09-17T05:30:25.982856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:31.457549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.959807+0000","last_change":"2024-09-17T05:30:24.955548+0000","last_active":"2024-09-17T05:30:25.959807+0000","last_peered":"2024-09-17T05:30:25.959807+0000","last_clean":"2024-09-17T05:30:25.959807+0000","last_became_active":"2024-09-17T05:30:24.954911+0000","last_became_peered":"2024-09-17T05:30:24.954911+0000","last_unstale":"2024-09-17T05:30:25.959807+0000","last_undegraded":"2024-09-17T05:30:25.959807+0000","last_fullsized":"2024-09-17T05:30:25.959807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:20.888374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864474+0000","last_change":"2024-09-17T05:30:23.956301+0000","last_active":"2024-09-17T05:30:27.864474+0000","last_peered":"2024-09-17T05:30:27.864474+0000","last_clean":"2024-09-17T05:30:27.864474+0000","last_became_active":"2024-09-17T05:30:23.955995+0000","last_became_peered":"2024-09-17T05:30:23.955995+0000","last_unstale":"2024-09-17T05:30:27.864474+0000","last_undegraded":"2024-09-17T05:30:27.864474+0000","last_fullsized":"2024-09-17T05:30:27.864474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:02.746869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:27.864899+0000","last_change":"2024-09-17T05:30:24.973772+0000","last_active":"2024-09-17T05:30:27.864899+0000","last_peered":"2024-09-17T05:30:27.864899+0000","last_clean":"2024-09-17T05:30:27.864899+0000","last_became_active":"2024-09-17T05:30:24.972340+0000","last_became_peered":"2024-09-17T05:30:24.972340+0000","last_unstale":"2024-09-17T05:30:27.864899+0000","last_undegraded":"2024-09-17T05:30:27.864899+0000","last_fullsized":"2024-09-17T05:30:27.864899+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:23.922126+0000","last_clean_scrub_stamp":"2024-09-17T05:30:23.922126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:43.483847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:26.426374+0000","last_change":"2024-09-17T05:30:23.952489+0000","last_active":"2024-09-17T05:30:26.426374+0000","last_peered":"2024-09-17T05:30:26.426374+0000","last_clean":"2024-09-17T05:30:26.426374+0000","last_became_active":"2024-09-17T05:30:23.952212+0000","last_became_peered":"2024-09-17T05:30:23.952212+0000","last_unstale":"2024-09-17T05:30:26.426374+0000","last_undegraded":"2024-09-17T05:30:26.426374+0000","last_fullsized":"2024-09-17T05:30:26.426374+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:30:22.915179+0000","last_clean_scrub_stamp":"2024-09-17T05:30:22.915179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:37.770595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":183,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-17T05:30:25.994221+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:30:25.994221+0000","last_peered":"2024-09-17T05:30:25.994221+0000","last_clean":"2024-09-17T05:30:25.994221+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:30:25.994221+0000","last_undegraded":"2024-09-17T05:30:25.994221+0000","last_fullsized":"2024-09-17T05:30:25.994221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509510,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40212,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331628,"statfs":{"total":96636764160,"available":96595587072,"internally_reserved":0,"allocated":720896,"data_stored":297955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58264,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313576,"statfs":{"total":96636764160,"available":96577101824,"internally_reserved":0,"allocated":716800,"data_stored":297885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45780,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326060,"statfs":{"total":96636764160,"available":96589885440,"internally_reserved":0,"allocated":720896,"data_stored":298479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40216,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331624,"statfs":{"total":96636764160,"available":96595582976,"internally_reserved":0,"allocated":716800,"data_stored":297953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62440,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309400,"statfs":{"total":96636764160,"available":96572825600,"internally_reserved":0,"allocated":733184,"data_stored":299635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509511,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1310720,"data_stored":888813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":724992,"data_stored":299475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1310720,"data_stored":888265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:30:34.003 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:30:34.004 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:30:34.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- 172.21.15.122:0/3366471976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815c3e0 msgr2=0x7fd74815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 --2- 172.21.15.122:0/3366471976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815c3e0 0x7fd74815c7c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd738009c80 tx=0x7fd73801f560 comp rx=0 tx=0).stop 2024-09-17T05:30:34.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- 172.21.15.122:0/3366471976 shutdown_connections 2024-09-17T05:30:34.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 --2- 172.21.15.122:0/3366471976 >> v2:172.21.15.173:3300/0 conn(0x7fd74815bfa0 0x7fd7480cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 --2- 172.21.15.122:0/3366471976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815c3e0 0x7fd74815c7c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- 172.21.15.122:0/3366471976 >> 172.21.15.122:0/3366471976 conn(0x7fd7480b18f0 msgr2=0x7fd7480b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:34.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- 172.21.15.122:0/3366471976 shutdown_connections 2024-09-17T05:30:34.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- 172.21.15.122:0/3366471976 wait complete. 2024-09-17T05:30:34.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 Processor -- start 2024-09-17T05:30:34.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- start start 2024-09-17T05:30:34.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7480cd540 0x7fd7481f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:34.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815bfa0 0x7fd748158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:34.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd74815c3e0 0x7fd7480cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:34.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd748160020 con 0x7fd74815bfa0 2024-09-17T05:30:34.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd74806f890 con 0x7fd74815c3e0 2024-09-17T05:30:34.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd74ca00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd7481601a0 con 0x7fd7480cd540 2024-09-17T05:30:34.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd73d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815bfa0 0x7fd748158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:34.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd73d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815bfa0 0x7fd748158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45622/0 (socket says 172.21.15.122:45622) 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd73d400640 1 -- 172.21.15.122:0/40300334 learned_addr learned my addr 172.21.15.122:0/40300334 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd745e00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7480cd540 0x7fd7481f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd746800640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd74815c3e0 0x7fd7480cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd73d400640 1 -- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7480cd540 msgr2=0x7fd7481f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.194+0000 7fd73d400640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7480cd540 0x7fd7481f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73d400640 1 -- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd74815c3e0 msgr2=0x7fd7480cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73d400640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd74815c3e0 0x7fd7480cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73d400640 1 -- 172.21.15.122:0/40300334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7481f4e70 con 0x7fd74815bfa0 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73d400640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815bfa0 0x7fd748158770 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd73000d980 tx=0x7fd73000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73ea00640 1 -- 172.21.15.122:0/40300334 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730017070 con 0x7fd74815bfa0 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73ea00640 1 -- 172.21.15.122:0/40300334 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd730009d20 con 0x7fd74815bfa0 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7481f5150 con 0x7fd74815bfa0 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73ea00640 1 -- 172.21.15.122:0/40300334 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7300106c0 con 0x7fd74815bfa0 2024-09-17T05:30:34.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7480b2e20 con 0x7fd74815bfa0 2024-09-17T05:30:34.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd748069410 con 0x7fd74815bfa0 2024-09-17T05:30:34.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73ea00640 1 -- 172.21.15.122:0/40300334 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fd73001a070 con 0x7fd74815bfa0 2024-09-17T05:30:34.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73ea00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd724073b00 0x7fd724075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:34.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.198+0000 7fd73ea00640 1 -- 172.21.15.122:0/40300334 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fd7300956b0 con 0x7fd74815bfa0 2024-09-17T05:30:34.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.202+0000 7fd745e00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd724073b00 0x7fd724075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.202+0000 7fd745e00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd724073b00 0x7fd724075fc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd7380b2000 tx=0x7fd73801dda0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:34.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.202+0000 7fd73ea00640 1 -- 172.21.15.122:0/40300334 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd73005f0d0 con 0x7fd74815bfa0 2024-09-17T05:30:34.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.434+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd748160660 con 0x7fd74815bfa0 2024-09-17T05:30:34.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.434+0000 7fd73ea00640 1 -- 172.21.15.122:0/40300334 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fd730063b00 con 0x7fd74815bfa0 2024-09-17T05:30:34.435 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:30:34.435 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:30:34.435 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:30:34.435 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:30:34.435 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:30:34.436 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:30:34.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd724073b00 msgr2=0x7fd724075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd724073b00 0x7fd724075fc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd7380b2000 tx=0x7fd73801dda0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815bfa0 msgr2=0x7fd748158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815bfa0 0x7fd748158770 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd73000d980 tx=0x7fd73000de40 comp rx=0 tx=0).stop 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 shutdown_connections 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd724073b00 0x7fd724075fc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd74815c3e0 0x7fd7480cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd74815bfa0 0x7fd748158770 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 --2- 172.21.15.122:0/40300334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7480cd540 0x7fd7481f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 >> 172.21.15.122:0/40300334 conn(0x7fd7480b18f0 msgr2=0x7fd7480b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 shutdown_connections 2024-09-17T05:30:34.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.438+0000 7fd74ca00640 1 -- 172.21.15.122:0/40300334 wait complete. 2024-09-17T05:30:34.458 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:30:34.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.598+0000 7ffaee000640 1 -- 172.21.15.122:0/2134851197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae80cf4e0 msgr2=0x7ffae80cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.598+0000 7ffaee000640 1 --2- 172.21.15.122:0/2134851197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae80cf4e0 0x7ffae80cd950 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009c80 tx=0x7ffae001cc80 comp rx=0 tx=0).stop 2024-09-17T05:30:34.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.598+0000 7ffaee000640 1 -- 172.21.15.122:0/2134851197 shutdown_connections 2024-09-17T05:30:34.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.598+0000 7ffaee000640 1 --2- 172.21.15.122:0/2134851197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae80cf4e0 0x7ffae80cd950 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.598+0000 7ffaee000640 1 --2- 172.21.15.122:0/2134851197 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffae815bd50 0x7ffae80ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.598+0000 7ffaee000640 1 -- 172.21.15.122:0/2134851197 >> 172.21.15.122:0/2134851197 conn(0x7ffae80d1ea0 msgr2=0x7ffae80d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:34.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.598+0000 7ffaee000640 1 -- 172.21.15.122:0/2134851197 shutdown_connections 2024-09-17T05:30:34.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.598+0000 7ffaee000640 1 -- 172.21.15.122:0/2134851197 wait complete. 2024-09-17T05:30:34.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 Processor -- start 2024-09-17T05:30:34.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 -- start start 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae80cf4e0 0x7ffae80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae815bd50 0x7ffae80cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffae815c130 0x7ffae81506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8168f00 con 0x7ffae815bd50 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8168d80 con 0x7ffae80cf4e0 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ffae8169080 con 0x7ffae815c130 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae815bd50 0x7ffae80cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae815bd50 0x7ffae80cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45630/0 (socket says 172.21.15.122:45630) 2024-09-17T05:30:34.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 -- 172.21.15.122:0/2081261357 learned_addr learned my addr 172.21.15.122:0/2081261357 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:34.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 -- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffae815c130 msgr2=0x7ffae81506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffae7e00640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffae815c130 0x7ffae81506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffae7400640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae80cf4e0 0x7ffae80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffae815c130 0x7ffae81506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 -- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae80cf4e0 msgr2=0x7ffae80c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae80cf4e0 0x7ffae80c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 -- 172.21.15.122:0/2081261357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae80cbbf0 con 0x7ffae815bd50 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffae7e00640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffae815c130 0x7ffae81506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffae7400640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae80cf4e0 0x7ffae80c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadea00640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae815bd50 0x7ffae80cb350 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ffad000b410 tx=0x7ffad000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadfe00640 1 -- 172.21.15.122:0/2081261357 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0015030 con 0x7ffae815bd50 2024-09-17T05:30:34.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadfe00640 1 -- 172.21.15.122:0/2081261357 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffad0004020 con 0x7ffae815bd50 2024-09-17T05:30:34.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffadfe00640 1 -- 172.21.15.122:0/2081261357 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad00044b0 con 0x7ffae815bd50 2024-09-17T05:30:34.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae80cbf30 con 0x7ffae815bd50 2024-09-17T05:30:34.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.602+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae80b5fc0 con 0x7ffae815bd50 2024-09-17T05:30:34.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.606+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae80ce5e0 con 0x7ffae815bd50 2024-09-17T05:30:34.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.606+0000 7ffadfe00640 1 -- 172.21.15.122:0/2081261357 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7ffad0018070 con 0x7ffae815bd50 2024-09-17T05:30:34.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.606+0000 7ffadfe00640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffac4073b00 0x7ffac4075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:34.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.606+0000 7ffadfe00640 1 -- 172.21.15.122:0/2081261357 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7ffad0092710 con 0x7ffae815bd50 2024-09-17T05:30:34.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.606+0000 7ffae7400640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffac4073b00 0x7ffac4075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.606+0000 7ffae7400640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffac4073b00 0x7ffac4075fc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffad8005fd0 tx=0x7ffad8005da0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:34.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.610+0000 7ffadfe00640 1 -- 172.21.15.122:0/2081261357 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ffae80cbf30 con 0x7ffae815bd50 2024-09-17T05:30:34.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.810+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffae8072180 con 0x7ffae815bd50 2024-09-17T05:30:34.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffadfe00640 1 -- 172.21.15.122:0/2081261357 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5232 (secure 0 0 0) 0x7ffad005f210 con 0x7ffae815bd50 2024-09-17T05:30:34.813 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:30:34.813 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":36,"btime":"2024-09-17T05:30:27:977715+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110418608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110418608},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110418608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1187468185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1187468185},{"type":"v1","addr":"172.21.15.122:6835","nonce":1187468185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3667127301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3667127301},{"type":"v1","addr":"172.21.15.122:6834","nonce":3667127301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:30:24.330680+0000","modified":"2024-09-17T05:30:27.362693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6836/204904815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":204904815},{"type":"v1","addr":"172.21.15.173:6836","nonce":204904815}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5931,"qdb_cluster":[5931]},"id":4}]} 2024-09-17T05:30:34.813 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 36 2024-09-17T05:30:34.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffac4073b00 msgr2=0x7ffac4075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffac4073b00 0x7ffac4075fc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffad8005fd0 tx=0x7ffad8005da0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae815bd50 msgr2=0x7ffae80cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:34.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae815bd50 0x7ffae80cb350 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ffad000b410 tx=0x7ffad000b8d0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 shutdown_connections 2024-09-17T05:30:34.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffac4073b00 0x7ffac4075fc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffae815c130 0x7ffae81506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffae815bd50 0x7ffae80cb350 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 --2- 172.21.15.122:0/2081261357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffae80cf4e0 0x7ffae80c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:34.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 >> 172.21.15.122:0/2081261357 conn(0x7ffae80d1ea0 msgr2=0x7ffae80d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:34.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 shutdown_connections 2024-09-17T05:30:34.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:34.814+0000 7ffaee000640 1 -- 172.21.15.122:0/2081261357 wait complete. 2024-09-17T05:30:35.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:30:24.330680+0000', 'modified': '2024-09-17T05:30:27.362693+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5931}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5931': {'gid': 5931, 'name': 'd', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6836/204904815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 204904815}, {'type': 'v1', 'addr': '172.21.15.173:6836', 'nonce': 204904815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5931, 'qdb_cluster': [5931]} 2024-09-17T05:30:35.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:30:35.237 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:30:35.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.378+0000 7f0e83e00640 1 -- 172.21.15.122:0/3722282713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840b2ca0 msgr2=0x7f0e840cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.378+0000 7f0e83e00640 1 --2- 172.21.15.122:0/3722282713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840b2ca0 0x7f0e840cefb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009c80 tx=0x7f0e7c01cd10 comp rx=0 tx=0).stop 2024-09-17T05:30:35.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.378+0000 7f0e83e00640 1 -- 172.21.15.122:0/3722282713 shutdown_connections 2024-09-17T05:30:35.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.378+0000 7f0e83e00640 1 --2- 172.21.15.122:0/3722282713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840b2ca0 0x7f0e840cefb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.378+0000 7f0e83e00640 1 -- 172.21.15.122:0/3722282713 >> 172.21.15.122:0/3722282713 conn(0x7f0e8409aa10 msgr2=0x7f0e8409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:35.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.378+0000 7f0e83e00640 1 -- 172.21.15.122:0/3722282713 shutdown_connections 2024-09-17T05:30:35.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.378+0000 7f0e83e00640 1 -- 172.21.15.122:0/3722282713 wait complete. 2024-09-17T05:30:35.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 Processor -- start 2024-09-17T05:30:35.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 -- start start 2024-09-17T05:30:35.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e840b2ca0 0x7f0e8414dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840d01b0 0x7f0e8415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8415f000 0x7f0e841f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8416a050 con 0x7f0e840d01b0 2024-09-17T05:30:35.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84169ed0 con 0x7f0e840b2ca0 2024-09-17T05:30:35.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8416a1d0 con 0x7f0e8415f000 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840d01b0 0x7f0e8415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840d01b0 0x7f0e8415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45642/0 (socket says 172.21.15.122:45642) 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 -- 172.21.15.122:0/790553463 learned_addr learned my addr 172.21.15.122:0/790553463 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e82a00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e840b2ca0 0x7f0e8414dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83400640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8415f000 0x7f0e841f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 -- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8415f000 msgr2=0x7f0e841f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8415f000 0x7f0e841f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 -- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e840b2ca0 msgr2=0x7f0e8414dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e840b2ca0 0x7f0e8414dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 -- 172.21.15.122:0/790553463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8415e3a0 con 0x7f0e840d01b0 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83400640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8415f000 0x7f0e841f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7be00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840d01b0 0x7f0e8415de60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c500 tx=0x7f0e6c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:35.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7b400640 1 -- 172.21.15.122:0/790553463 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c017070 con 0x7f0e840d01b0 2024-09-17T05:30:35.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7b400640 1 -- 172.21.15.122:0/790553463 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e6c0053c0 con 0x7f0e840d01b0 2024-09-17T05:30:35.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e7b400640 1 -- 172.21.15.122:0/790553463 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c012890 con 0x7f0e840d01b0 2024-09-17T05:30:35.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 -- 172.21.15.122:0/790553463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8415cdc0 con 0x7f0e840d01b0 2024-09-17T05:30:35.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.382+0000 7f0e83e00640 1 -- 172.21.15.122:0/790553463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8415d380 con 0x7f0e840d01b0 2024-09-17T05:30:35.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.386+0000 7f0e7b400640 1 -- 172.21.15.122:0/790553463 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f0e6c01a070 con 0x7f0e840d01b0 2024-09-17T05:30:35.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.386+0000 7f0e7b400640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e60073b00 0x7f0e60075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.386+0000 7f0e8d24b000 1 Processor -- start 2024-09-17T05:30:35.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.386+0000 7f0e7b400640 1 -- 172.21.15.122:0/790553463 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f0e6c091da0 con 0x7f0e840d01b0 2024-09-17T05:30:35.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- start start 2024-09-17T05:30:35.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e60073b00 0x7f0e60075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e60073b00 0x7f0e60075fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e74007fd0 tx=0x7f0e74009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:35.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 0x5602ad303ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 0x5602ad304600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5602ad2203f0 con 0x5602ad34fdc0 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602ad15db20 con 0x5602ad3506e0 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5602ad14cf50 con 0x5602ad304220 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45654/0 (socket says 172.21.15.122:45654) 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 -- 172.21.15.122:0/3689006785 learned_addr learned my addr 172.21.15.122:0/3689006785 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e7be00640 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 0x5602ad303ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e83400640 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 0x5602ad304600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 -- 172.21.15.122:0/3689006785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 msgr2=0x5602ad304600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 0x5602ad304600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 -- 172.21.15.122:0/3689006785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 msgr2=0x5602ad303ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 0x5602ad303ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 -- 172.21.15.122:0/3689006785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ad114590 con 0x5602ad34fdc0 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e82a00640 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3501a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400faf0 tx=0x7f0e74012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288b0ae0dec2f4ed server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e73e00640 1 -- 172.21.15.122:0/3689006785 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74025070 con 0x5602ad34fdc0 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e73e00640 1 -- 172.21.15.122:0/3689006785 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e7401b050 con 0x5602ad34fdc0 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e73e00640 1 -- 172.21.15.122:0/3689006785 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74020410 con 0x5602ad34fdc0 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- 172.21.15.122:0/3689006785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 msgr2=0x5602ad3501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3501a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400faf0 tx=0x7f0e74012000 comp rx=0 tx=0).stop 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- 172.21.15.122:0/3689006785 shutdown_connections 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 0x5602ad304600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 0x5602ad303ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/3689006785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3501a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- 172.21.15.122:0/3689006785 >> 172.21.15.122:0/3689006785 conn(0x5602ad1e9130 msgr2=0x5602ad32ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:35.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- 172.21.15.122:0/3689006785 shutdown_connections 2024-09-17T05:30:35.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- 172.21.15.122:0/3689006785 wait complete. 2024-09-17T05:30:35.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 -- start start 2024-09-17T05:30:35.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.390+0000 7f0e8d24b000 1 Processor -- start 2024-09-17T05:30:35.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 0x5602ad33a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad33a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 0x5602ad33ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5602ad0e75d0 con 0x5602ad34fdc0 2024-09-17T05:30:35.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602ad150d60 con 0x5602ad3506e0 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5602ad0f3500 con 0x5602ad304220 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 10 client.0 _ll_register_callbacks cb 0x5602ad0f7810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad33a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e82a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 0x5602ad33a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e83400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 0x5602ad33ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad33a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45670/0 (socket says 172.21.15.122:45670) 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 -- 172.21.15.122:0/2202263461 learned_addr learned my addr 172.21.15.122:0/2202263461 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 -- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 msgr2=0x5602ad33a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 0x5602ad33a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 -- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 msgr2=0x5602ad33ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 0x5602ad33ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 -- 172.21.15.122:0/2202263461 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ad0eb170 con 0x5602ad34fdc0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e7be00640 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad33a8a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c05ded0 tx=0x7f0e6c05e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12bdea809a3145 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e70c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e70c00640 1 -- 172.21.15.122:0/2202263461 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c062c00 con 0x5602ad34fdc0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e70c00640 1 -- 172.21.15.122:0/2202263461 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f0e6c059b20 con 0x5602ad34fdc0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2202263461 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602ad0e50f0 con 0x5602ad34fdc0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e70c00640 1 -- 172.21.15.122:0/2202263461 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0e6c067ab0 con 0x5602ad34fdc0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 10 client.6087 fetch_fsmap learned FSMap version 36 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2202263461 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5602ad1d1ef0 con 0x5602ad34fdc0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e70c00640 1 -- 172.21.15.122:0/2202263461 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5602ad0e50f0 con 0x5602ad34fdc0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e70c00640 1 -- 172.21.15.122:0/2202263461 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f0e6c059d20 con 0x5602ad34fdc0 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 10 client.6087 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 10 client.6087 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:30:35.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 20 client.6087 populate_metadata read hostname 'smithi122' 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] conn(0x5602ad33b7f0 0x5602ad361070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 4 client.6087 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2202263461 --> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602ad15fa70 con 0x5602ad33b7f0 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e82a00640 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] conn(0x5602ad33b7f0 0x5602ad361070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e70c00640 10 client.6087 ms_handle_connect on v2:172.21.15.173:6834/204904815 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.394+0000 7f0e82a00640 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] conn(0x5602ad33b7f0 0x5602ad361070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6118abb5b1378c5 server_cookie=c0605627e7f273bc in_seq=0 out_seq=0 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e70c00640 1 -- 172.21.15.122:0/2202263461 <== mds.0 v2:172.21.15.173:6834/204904815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602ad15fa70 con 0x5602ad33b7f0 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e70c00640 10 client.6087 handle_command_reply: tid=0 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 client.6087 shutdown 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2202263461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 msgr2=0x5602ad33a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad33a8a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c05ded0 tx=0x7f0e6c05e0d0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2202263461 shutdown_connections 2024-09-17T05:30:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] conn(0x5602ad33b7f0 0x5602ad361070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad3506e0 0x5602ad33ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad33a8a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2202263461 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad304220 0x5602ad33a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e70c00640 0 client.6087 ms_handle_reset on v2:172.21.15.173:6834/204904815 2024-09-17T05:30:35.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2202263461 >> 172.21.15.122:0/2202263461 conn(0x5602ad1e9130 msgr2=0x5602ad2fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:35.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2202263461 shutdown_connections 2024-09-17T05:30:35.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2202263461 wait complete. 2024-09-17T05:30:35.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.398+0000 7f0e8d24b000 20 client.6087 trim_cache size 0 max 16384 2024-09-17T05:30:35.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.414+0000 7f0e8d24b000 1 Processor -- start 2024-09-17T05:30:35.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.414+0000 7f0e8d24b000 1 -- start start 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 0x5602ad32bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad304220 0x5602ad32c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8406f900 con 0x5602ad34fdc0 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602ad118ec0 con 0x5602ad304220 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5602ad20c730 con 0x5602ad1e9130 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45680/0 (socket says 172.21.15.122:45680) 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 -- 172.21.15.122:0/1541929203 learned_addr learned my addr 172.21.15.122:0/1541929203 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 -- 172.21.15.122:0/1541929203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 msgr2=0x5602ad32bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 --2- 172.21.15.122:0/1541929203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 0x5602ad32bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 -- 172.21.15.122:0/1541929203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad304220 msgr2=0x5602ad32c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:35.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 --2- 172.21.15.122:0/1541929203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad304220 0x5602ad32c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 -- 172.21.15.122:0/1541929203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ad1a8d90 con 0x5602ad34fdc0 2024-09-17T05:30:35.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e83400640 1 --2- 172.21.15.122:0/1541929203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3a79f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009bd0 tx=0x7f0e7c019ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b045fa84c6e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:35.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e73400640 1 -- 172.21.15.122:0/1541929203 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0b2040 con 0x5602ad34fdc0 2024-09-17T05:30:35.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e73400640 1 -- 172.21.15.122:0/1541929203 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e7c004c70 con 0x5602ad34fdc0 2024-09-17T05:30:35.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e73400640 1 -- 172.21.15.122:0/1541929203 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c004e60 con 0x5602ad34fdc0 2024-09-17T05:30:35.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- 172.21.15.122:0/1541929203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 msgr2=0x5602ad3a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/1541929203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3a79f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009bd0 tx=0x7f0e7c019ab0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- 172.21.15.122:0/1541929203 shutdown_connections 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/1541929203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad34fdc0 0x5602ad3a79f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/1541929203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad304220 0x5602ad32c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/1541929203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 0x5602ad32bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- 172.21.15.122:0/1541929203 >> 172.21.15.122:0/1541929203 conn(0x5602ad306f60 msgr2=0x5602ad30d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- 172.21.15.122:0/1541929203 shutdown_connections 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- 172.21.15.122:0/1541929203 wait complete. 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- start start 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 Processor -- start 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 0x5602ad3c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad304220 0x5602ad3c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad34fdc0 0x5602ad3c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5602ad15c8f0 con 0x5602ad304220 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5602ad1d1a60 con 0x5602ad34fdc0 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5602ad150070 con 0x5602ad1e9130 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e8d24b000 10 client.0 _ll_register_callbacks cb 0x7f0e6c05a8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.418+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad304220 0x5602ad3c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad304220 0x5602ad3c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45696/0 (socket says 172.21.15.122:45696) 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e7be00640 1 -- 172.21.15.122:0/2078613491 learned_addr learned my addr 172.21.15.122:0/2078613491 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e7be00640 1 -- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 msgr2=0x5602ad3c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83400640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad34fdc0 0x5602ad3c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e82a00640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 0x5602ad3c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e7be00640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 0x5602ad3c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e7be00640 1 -- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad34fdc0 msgr2=0x5602ad3c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:30:35.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e7be00640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad34fdc0 0x5602ad3c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e7be00640 1 -- 172.21.15.122:0/2078613491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ad113a90 con 0x5602ad304220 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e82a00640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 0x5602ad3c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83400640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad34fdc0 0x5602ad3c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e7be00640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad304220 0x5602ad3c6f80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c067c70 tx=0x7f0e6c059f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e152138421d4d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 1 -- 172.21.15.122:0/2078613491 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c05db20 con 0x5602ad304220 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 1 -- 172.21.15.122:0/2078613491 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f0e6c05cea0 con 0x5602ad304220 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2078613491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7c004c70 con 0x5602ad304220 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 1 -- 172.21.15.122:0/2078613491 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0e6c05a070 con 0x5602ad304220 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 10 client.6093 fetch_fsmap learned FSMap version 36 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2078613491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5602ad0f2050 con 0x5602ad304220 2024-09-17T05:30:35.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 1 -- 172.21.15.122:0/2078613491 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e7c004c70 con 0x5602ad304220 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 1 -- 172.21.15.122:0/2078613491 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f0e6c05d0a0 con 0x5602ad304220 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 10 client.6093 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 10 client.6093 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 20 client.6093 populate_metadata read hostname 'smithi122' 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] conn(0x5602ad3506e0 0x5602ad364800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 4 client.6093 mds_command: new command op to 5931 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2078613491 --> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] -- command(tid 0: {"prefix": "status"}) -- 0x5602ad15b520 con 0x5602ad3506e0 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e82a00640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] conn(0x5602ad3506e0 0x5602ad364800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e82a00640 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] conn(0x5602ad3506e0 0x5602ad364800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf73c95f1786e5ae server_cookie=66babfdac229f647 in_seq=0 out_seq=0 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 10 client.6093 ms_handle_connect on v2:172.21.15.173:6834/204904815 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 1 -- 172.21.15.122:0/2078613491 <== mds.0 v2:172.21.15.173:6834/204904815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5602ad15b520 con 0x5602ad3506e0 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 10 client.6093 handle_command_reply: tid=0 2024-09-17T05:30:35.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 client.6093 shutdown 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2078613491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad304220 msgr2=0x5602ad3c6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad304220 0x5602ad3c6f80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c067c70 tx=0x7f0e6c059f20 comp rx=0 tx=0).stop 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2078613491 shutdown_connections 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:6834/204904815,v1:172.21.15.173:6836/204904815] conn(0x5602ad3506e0 0x5602ad364800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5602ad34fdc0 0x5602ad3c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602ad304220 0x5602ad3c6f80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e70c00640 0 client.6093 ms_handle_reset on v2:172.21.15.173:6834/204904815 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 --2- 172.21.15.122:0/2078613491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5602ad1e9130 0x5602ad3c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2078613491 >> 172.21.15.122:0/2078613491 conn(0x5602ad306f60 msgr2=0x5602ad32af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2078613491 shutdown_connections 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 1 -- 172.21.15.122:0/2078613491 wait complete. 2024-09-17T05:30:35.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e8d24b000 20 client.6093 trim_cache size 0 max 16384 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 5931, 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 10.467775210999999, 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 36, 2024-09-17T05:30:35.429 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 60, 2024-09-17T05:30:35.430 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 60, 2024-09-17T05:30:35.430 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 14.96574431 2024-09-17T05:30:35.430 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:30:35.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 -- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e60073b00 msgr2=0x7f0e60075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e60073b00 0x7f0e60075fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e74007fd0 tx=0x7f0e74009040 comp rx=0 tx=0).stop 2024-09-17T05:30:35.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 -- 172.21.15.122:0/790553463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840d01b0 msgr2=0x7f0e8415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840d01b0 0x7f0e8415de60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c500 tx=0x7f0e6c00c9c0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 -- 172.21.15.122:0/790553463 shutdown_connections 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e60073b00 0x7f0e60075fc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8415f000 0x7f0e841f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e840d01b0 0x7f0e8415de60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 --2- 172.21.15.122:0/790553463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e840b2ca0 0x7f0e8414dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.422+0000 7f0e83e00640 1 -- 172.21.15.122:0/790553463 >> 172.21.15.122:0/790553463 conn(0x7f0e8409aa10 msgr2=0x7f0e840ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.426+0000 7f0e83e00640 1 -- 172.21.15.122:0/790553463 shutdown_connections 2024-09-17T05:30:35.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.426+0000 7f0e83e00640 1 -- 172.21.15.122:0/790553463 wait complete. 2024-09-17T05:30:35.443 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:30:35.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- 172.21.15.122:0/3280204828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e0c003ab0 msgr2=0x7f2e0c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 --2- 172.21.15.122:0/3280204828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e0c003ab0 0x7f2e0c003ed0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fb0 tx=0x7f2e1801ceb0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- 172.21.15.122:0/3280204828 shutdown_connections 2024-09-17T05:30:35.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 --2- 172.21.15.122:0/3280204828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e0c003ab0 0x7f2e0c003ed0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- 172.21.15.122:0/3280204828 >> 172.21.15.122:0/3280204828 conn(0x7f2e280d1e30 msgr2=0x7f2e280d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:35.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- 172.21.15.122:0/3280204828 shutdown_connections 2024-09-17T05:30:35.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- 172.21.15.122:0/3280204828 wait complete. 2024-09-17T05:30:35.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 Processor -- start 2024-09-17T05:30:35.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- start start 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e0c003ab0 0x7f2e280cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e281558b0 0x7f2e281fab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280c8f00 0x7f2e280b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e280b0c20 con 0x7f2e281558b0 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2e280b0aa0 con 0x7f2e0c003ab0 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2f600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2e280b0da0 con 0x7f2e280c8f00 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e281558b0 0x7f2e281fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e281558b0 0x7f2e281fab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45722/0 (socket says 172.21.15.122:45722) 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 -- 172.21.15.122:0/1416968949 learned_addr learned my addr 172.21.15.122:0/1416968949 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2cc00640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e0c003ab0 0x7f2e280cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2d600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280c8f00 0x7f2e280b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 -- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280c8f00 msgr2=0x7f2e280b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280c8f00 0x7f2e280b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 -- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e0c003ab0 msgr2=0x7f2e280cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e0c003ab0 0x7f2e280cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 -- 172.21.15.122:0/1416968949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e281fb4a0 con 0x7f2e281558b0 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2d600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280c8f00 0x7f2e280b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e2cc00640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e0c003ab0 0x7f2e280cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.586+0000 7f2e1fe00640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e281558b0 0x7f2e281fab30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e10011780 tx=0x7f2e10011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e25600640 1 -- 172.21.15.122:0/1416968949 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1001f070 con 0x7f2e281558b0 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e25600640 1 -- 172.21.15.122:0/1416968949 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e10011e90 con 0x7f2e281558b0 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e25600640 1 -- 172.21.15.122:0/1416968949 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e100126b0 con 0x7f2e281558b0 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e281fb780 con 0x7f2e281558b0 2024-09-17T05:30:35.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e280b5070 con 0x7f2e281558b0 2024-09-17T05:30:35.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e280b1850 con 0x7f2e281558b0 2024-09-17T05:30:35.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e25600640 1 -- 172.21.15.122:0/1416968949 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2e10022070 con 0x7f2e281558b0 2024-09-17T05:30:35.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e25600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b00 0x7f2e04075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:35.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e25600640 1 -- 172.21.15.122:0/1416968949 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f2e100933c0 con 0x7f2e281558b0 2024-09-17T05:30:35.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.590+0000 7f2e2cc00640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b00 0x7f2e04075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:35.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.594+0000 7f2e2cc00640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b00 0x7f2e04075fc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2e180ac000 tx=0x7f2e18019cf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:35.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.594+0000 7f2e25600640 1 -- 172.21.15.122:0/1416968949 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2e1005f870 con 0x7f2e281558b0 2024-09-17T05:30:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.793+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2e2806e430 con 0x7f2e281558b0 2024-09-17T05:30:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e25600640 1 -- 172.21.15.122:0/1416968949 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f2e100642a0 con 0x7f2e281558b0 2024-09-17T05:30:35.797 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b00 msgr2=0x7f2e04075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b00 0x7f2e04075fc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2e180ac000 tx=0x7f2e18019cf0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e281558b0 msgr2=0x7f2e281fab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e281558b0 0x7f2e281fab30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e10011780 tx=0x7f2e10011c40 comp rx=0 tx=0).stop 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 shutdown_connections 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e04073b00 0x7f2e04075fc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e280c8f00 0x7f2e280b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e281558b0 0x7f2e281fab30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 --2- 172.21.15.122:0/1416968949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e0c003ab0 0x7f2e280cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 >> 172.21.15.122:0/1416968949 conn(0x7f2e280d1e30 msgr2=0x7f2e280b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 shutdown_connections 2024-09-17T05:30:35.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:35.797+0000 7f2e2f600640 1 -- 172.21.15.122:0/1416968949 wait complete. 2024-09-17T05:30:35.818 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:30:35.818 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:30:35.818 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:30:35.818 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:30:35.818 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:35.819 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:30:35.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:30:35.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:30:35.819 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:35.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:35.819 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:30:35.828 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:30:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:30:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:30:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:30:35.828 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42131sec preferred_lft 42131sec 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:30:35.830 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:30:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:30:35.831 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:30:35.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:35.832 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:30:35.889 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:30:35.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:35.891 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:30:35.943 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:30:35.943 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:30:35.943 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:30:35.943 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:30:35.944 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:30:35.944 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:30:35.944 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:30:35.944 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:30:35.944 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:30:35.944 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:30:35.944 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:30:35.944 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:30:35.944 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:30:35.944 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:30:35.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:30:35.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:30:35.945 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:30:36.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:30:36.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:36.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:36.083 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:30:36.093 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:30:36.093 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:30:36.093 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:36.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:36.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:30:36.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:30:36.156 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:36.208 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:30:36.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:30:36.208 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:36.259 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:30:36.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:36.260 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:36.348 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:36.349 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:36.349 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:30:36.364 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:36.364 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:36.364 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:30:36.364 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:30:36.364 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:30:36.373 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:30:36.374 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:30:36.374 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:30:36.374 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:30:36.374 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:30:36.374 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:30:36.374 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:30:36.374 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:30:36.377 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:30:36.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:36.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:30:36.398 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:30:36.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:30:36.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:30:36.510 INFO:teuthology.orchestra.run.smithi122.stdout:client5471 2024-09-17T05:30:36.510 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:36.511 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:36.520 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:30:36.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- 172.21.15.122:0/1249479516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c0d28e0 msgr2=0x7f232c0d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:36.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 --2- 172.21.15.122:0/1249479516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c0d28e0 0x7f232c0d0340 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f232c0b3850 tx=0x7f231801c890 comp rx=0 tx=0).stop 2024-09-17T05:30:36.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- 172.21.15.122:0/1249479516 shutdown_connections 2024-09-17T05:30:36.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 --2- 172.21.15.122:0/1249479516 >> v2:172.21.15.173:3300/0 conn(0x7f232c0af850 0x7f232c0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:36.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 --2- 172.21.15.122:0/1249479516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c0d28e0 0x7f232c0d0340 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:36.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- 172.21.15.122:0/1249479516 >> 172.21.15.122:0/1249479516 conn(0x7f232c0d1dc0 msgr2=0x7f232c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:36.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- 172.21.15.122:0/1249479516 shutdown_connections 2024-09-17T05:30:36.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- 172.21.15.122:0/1249479516 wait complete. 2024-09-17T05:30:36.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 Processor -- start 2024-09-17T05:30:36.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- start start 2024-09-17T05:30:36.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f232c0af850 0x7f232c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:36.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f232c0d28e0 0x7f232c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:36.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c15f390 0x7f232c147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:36.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f232c168ea0 con 0x7f232c15f390 2024-09-17T05:30:36.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f232c168d20 con 0x7f232c0d28e0 2024-09-17T05:30:36.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.713+0000 7f2334400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f232c169020 con 0x7f232c0af850 2024-09-17T05:30:36.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c15f390 0x7f232c147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:36.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c15f390 0x7f232c147130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45744/0 (socket says 172.21.15.122:45744) 2024-09-17T05:30:36.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 -- 172.21.15.122:0/1615044142 learned_addr learned my addr 172.21.15.122:0/1615044142 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:36.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2331a00640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f232c0af850 0x7f232c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:36.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2331000640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f232c0d28e0 0x7f232c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:36.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 -- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f232c0af850 msgr2=0x7f232c14ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:36.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f232c0af850 0x7f232c14ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:36.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 -- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f232c0d28e0 msgr2=0x7f232c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:36.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f232c0d28e0 0x7f232c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:36.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 -- 172.21.15.122:0/1615044142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c151e00 con 0x7f232c15f390 2024-09-17T05:30:36.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2331000640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f232c0d28e0 0x7f232c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:30:36.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2332400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c15f390 0x7f232c147130 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f232400d750 tx=0x7f232400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2329600640 1 -- 172.21.15.122:0/1615044142 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324017070 con 0x7f232c15f390 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2329600640 1 -- 172.21.15.122:0/1615044142 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2324009d20 con 0x7f232c15f390 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2329600640 1 -- 172.21.15.122:0/1615044142 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232400de60 con 0x7f232c15f390 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1520e0 con 0x7f232c15f390 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1526a0 con 0x7f232c15f390 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c0afc30 con 0x7f232c15f390 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.717+0000 7f2329600640 1 -- 172.21.15.122:0/1615044142 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2324005280 con 0x7f232c15f390 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.721+0000 7f2329600640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f230c073b00 0x7f230c075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:36.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.721+0000 7f2329600640 1 -- 172.21.15.122:0/1615044142 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f2324092b30 con 0x7f232c15f390 2024-09-17T05:30:36.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.721+0000 7f2331a00640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f230c073b00 0x7f230c075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:36.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.721+0000 7f2331a00640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f230c073b00 0x7f230c075fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f232000b450 tx=0x7f232000d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:36.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.725+0000 7f2329600640 1 -- 172.21.15.122:0/1615044142 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f232405efe0 con 0x7f232c15f390 2024-09-17T05:30:36.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:36.925+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f232c0b1e20 con 0x7f232c15f390 2024-09-17T05:30:37.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.005+0000 7f2329600640 1 -- 172.21.15.122:0/1615044142 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f2324063a10 con 0x7f232c15f390 2024-09-17T05:30:37.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f230c073b00 msgr2=0x7f230c075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f230c073b00 0x7f230c075fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f232000b450 tx=0x7f232000d040 comp rx=0 tx=0).stop 2024-09-17T05:30:37.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c15f390 msgr2=0x7f232c147130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c15f390 0x7f232c147130 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f232400d750 tx=0x7f232400dc10 comp rx=0 tx=0).stop 2024-09-17T05:30:37.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 shutdown_connections 2024-09-17T05:30:37.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f230c073b00 0x7f230c075fc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c15f390 0x7f232c147130 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f232c0d28e0 0x7f232c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 --2- 172.21.15.122:0/1615044142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f232c0af850 0x7f232c14ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 >> 172.21.15.122:0/1615044142 conn(0x7f232c0d1dc0 msgr2=0x7f232c070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:37.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 shutdown_connections 2024-09-17T05:30:37.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.009+0000 7f2334400640 1 -- 172.21.15.122:0/1615044142 wait complete. 2024-09-17T05:30:37.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:30:37.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 -- 172.21.15.122:0/1596155454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d3c004810 msgr2=0x7f1d3c004c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 --2- 172.21.15.122:0/1596155454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d3c004810 0x7f1d3c004c70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000ae10 tx=0x7f1d4001e1f0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 -- 172.21.15.122:0/1596155454 shutdown_connections 2024-09-17T05:30:37.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 --2- 172.21.15.122:0/1596155454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d3c004810 0x7f1d3c004c70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 -- 172.21.15.122:0/1596155454 >> 172.21.15.122:0/1596155454 conn(0x7f1d580b18f0 msgr2=0x7f1d580b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:37.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 -- 172.21.15.122:0/1596155454 shutdown_connections 2024-09-17T05:30:37.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 -- 172.21.15.122:0/1596155454 wait complete. 2024-09-17T05:30:37.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 Processor -- start 2024-09-17T05:30:37.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.173+0000 7f1d5e200640 1 -- start start 2024-09-17T05:30:37.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d3c004810 0x7f1d581492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d581fbb60 0x7f1d581f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d580cb120 0x7f1d580cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d580ceba0 con 0x7f1d580cb120 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d580cea20 con 0x7f1d3c004810 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1d580ced20 con 0x7f1d581fbb60 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d580cb120 0x7f1d580cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d580cb120 0x7f1d580cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45748/0 (socket says 172.21.15.122:45748) 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 -- 172.21.15.122:0/2682648233 learned_addr learned my addr 172.21.15.122:0/2682648233 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57400640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d3c004810 0x7f1d581492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d4ea00640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d581fbb60 0x7f1d581f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 -- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d581fbb60 msgr2=0x7f1d581f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d581fbb60 0x7f1d581f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 -- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d3c004810 msgr2=0x7f1d581492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d3c004810 0x7f1d581492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 -- 172.21.15.122:0/2682648233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d581f8e70 con 0x7f1d580cb120 2024-09-17T05:30:37.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d57e00640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d580cb120 0x7f1d580cb5d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0057f0 tx=0x7f1d5000e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:37.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d4fe00640 1 -- 172.21.15.122:0/2682648233 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d500107d0 con 0x7f1d580cb120 2024-09-17T05:30:37.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d4fe00640 1 -- 172.21.15.122:0/2682648233 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d50016070 con 0x7f1d580cb120 2024-09-17T05:30:37.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d581f9150 con 0x7f1d580cb120 2024-09-17T05:30:37.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d4fe00640 1 -- 172.21.15.122:0/2682648233 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50010d20 con 0x7f1d580cb120 2024-09-17T05:30:37.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5809aab0 con 0x7f1d580cb120 2024-09-17T05:30:37.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d58069410 con 0x7f1d580cb120 2024-09-17T05:30:37.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.177+0000 7f1d4fe00640 1 -- 172.21.15.122:0/2682648233 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f1d5001b460 con 0x7f1d580cb120 2024-09-17T05:30:37.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.181+0000 7f1d4fe00640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d34073b30 0x7f1d34075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.181+0000 7f1d4fe00640 1 -- 172.21.15.122:0/2682648233 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f1d50092d50 con 0x7f1d580cb120 2024-09-17T05:30:37.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.181+0000 7f1d57400640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d34073b30 0x7f1d34075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.181+0000 7f1d57400640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d34073b30 0x7f1d34075ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1d48000f90 tx=0x7f1d48006ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:37.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.185+0000 7f1d4fe00640 1 -- 172.21.15.122:0/2682648233 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1d5005f850 con 0x7f1d580cb120 2024-09-17T05:30:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.377+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1d5806ef30 con 0x7f1d34073b30 2024-09-17T05:30:37.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.377+0000 7f1d4fe00640 1 -- 172.21.15.122:0/2682648233 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1d5806ef30 con 0x7f1d34073b30 2024-09-17T05:30:37.380 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:30:37.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:30:37.380 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:30:37.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:30:37.380 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:30:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d34073b30 msgr2=0x7f1d34075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d34073b30 0x7f1d34075ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1d48000f90 tx=0x7f1d48006ea0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d580cb120 msgr2=0x7f1d580cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d580cb120 0x7f1d580cb5d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0057f0 tx=0x7f1d5000e770 comp rx=0 tx=0).stop 2024-09-17T05:30:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 shutdown_connections 2024-09-17T05:30:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1d34073b30 0x7f1d34075ff0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d580cb120 0x7f1d580cb5d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d581fbb60 0x7f1d581f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 --2- 172.21.15.122:0/2682648233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d3c004810 0x7f1d581492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 >> 172.21.15.122:0/2682648233 conn(0x7f1d580b18f0 msgr2=0x7f1d58073db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 shutdown_connections 2024-09-17T05:30:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.381+0000 7f1d5e200640 1 -- 172.21.15.122:0/2682648233 wait complete. 2024-09-17T05:30:37.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:30:37.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.545+0000 7f231f400640 1 -- 172.21.15.122:0/2075585610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23180af0a0 msgr2=0x7f23180cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.545+0000 7f231f400640 1 --2- 172.21.15.122:0/2075585610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23180af0a0 0x7f23180cffe0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2310009bc0 tx=0x7f231001cb50 comp rx=0 tx=0).stop 2024-09-17T05:30:37.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.545+0000 7f231f400640 1 -- 172.21.15.122:0/2075585610 shutdown_connections 2024-09-17T05:30:37.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.545+0000 7f231f400640 1 --2- 172.21.15.122:0/2075585610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23180af0a0 0x7f23180cffe0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.545+0000 7f231f400640 1 --2- 172.21.15.122:0/2075585610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f231815dea0 0x7f23180b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.545+0000 7f231f400640 1 -- 172.21.15.122:0/2075585610 >> 172.21.15.122:0/2075585610 conn(0x7f23180d1da0 msgr2=0x7f23180d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:37.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.545+0000 7f231f400640 1 -- 172.21.15.122:0/2075585610 shutdown_connections 2024-09-17T05:30:37.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.545+0000 7f231f400640 1 -- 172.21.15.122:0/2075585610 wait complete. 2024-09-17T05:30:37.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 Processor -- start 2024-09-17T05:30:37.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 -- start start 2024-09-17T05:30:37.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23180af0a0 0x7f2318145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f231815dea0 0x7f23180b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231815e280 0x7f231809bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f231816ad70 con 0x7f231815e280 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f231816abf0 con 0x7f23180af0a0 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f231816aef0 con 0x7f231815dea0 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231815e280 0x7f231809bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231815e280 0x7f231809bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45766/0 (socket says 172.21.15.122:45766) 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 -- 172.21.15.122:0/640352405 learned_addr learned my addr 172.21.15.122:0/640352405 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231ca00640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23180af0a0 0x7f2318145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f230fe00640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f231815dea0 0x7f23180b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 -- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f231815dea0 msgr2=0x7f23180b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f231815dea0 0x7f23180b65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 -- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23180af0a0 msgr2=0x7f2318145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23180af0a0 0x7f2318145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 -- 172.21.15.122:0/640352405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2318161520 con 0x7f231815e280 2024-09-17T05:30:37.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231d400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231815e280 0x7f231809bb10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f231001a780 tx=0x7f23100a5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:37.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f2315600640 1 -- 172.21.15.122:0/640352405 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23100b2040 con 0x7f231815e280 2024-09-17T05:30:37.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f2315600640 1 -- 172.21.15.122:0/640352405 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2310004d40 con 0x7f231815e280 2024-09-17T05:30:37.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f2315600640 1 -- 172.21.15.122:0/640352405 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310004300 con 0x7f231815e280 2024-09-17T05:30:37.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2318161240 con 0x7f231815e280 2024-09-17T05:30:37.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23180b5dd0 con 0x7f231815e280 2024-09-17T05:30:37.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.549+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23180cffe0 con 0x7f231815e280 2024-09-17T05:30:37.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.553+0000 7f2315600640 1 -- 172.21.15.122:0/640352405 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f23100be060 con 0x7f231815e280 2024-09-17T05:30:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.553+0000 7f2315600640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f4073b00 0x7f22f4075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.553+0000 7f2315600640 1 -- 172.21.15.122:0/640352405 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f231012c6e0 con 0x7f231815e280 2024-09-17T05:30:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.553+0000 7f231ca00640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f4073b00 0x7f22f4075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.553+0000 7f231ca00640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f4073b00 0x7f22f4075fc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f230800bc00 tx=0x7f230800b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:37.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.557+0000 7f2315600640 1 -- 172.21.15.122:0/640352405 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2310016770 con 0x7f231815e280 2024-09-17T05:30:37.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.725+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f23180b3280 con 0x7f231815e280 2024-09-17T05:30:37.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.725+0000 7f2315600640 1 -- 172.21.15.122:0/640352405 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f23100f8bf0 con 0x7f231815e280 2024-09-17T05:30:37.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f4073b00 msgr2=0x7f22f4075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f4073b00 0x7f22f4075fc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f230800bc00 tx=0x7f230800b390 comp rx=0 tx=0).stop 2024-09-17T05:30:37.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231815e280 msgr2=0x7f231809bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231815e280 0x7f231809bb10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f231001a780 tx=0x7f23100a5a40 comp rx=0 tx=0).stop 2024-09-17T05:30:37.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 shutdown_connections 2024-09-17T05:30:37.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f4073b00 0x7f22f4075fc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f231815e280 0x7f231809bb10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f231815dea0 0x7f23180b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 --2- 172.21.15.122:0/640352405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23180af0a0 0x7f2318145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 >> 172.21.15.122:0/640352405 conn(0x7f23180d1da0 msgr2=0x7f2318071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:37.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 shutdown_connections 2024-09-17T05:30:37.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.729+0000 7f231f400640 1 -- 172.21.15.122:0/640352405 wait complete. 2024-09-17T05:30:37.750 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:30:37.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- 172.21.15.122:0/1490885104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c06d330 msgr2=0x7f632c073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 --2- 172.21.15.122:0/1490885104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c06d330 0x7f632c073980 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6318009840 tx=0x7f631801c820 comp rx=0 tx=0).stop 2024-09-17T05:30:37.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- 172.21.15.122:0/1490885104 shutdown_connections 2024-09-17T05:30:37.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 --2- 172.21.15.122:0/1490885104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c06d330 0x7f632c073980 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- 172.21.15.122:0/1490885104 >> 172.21.15.122:0/1490885104 conn(0x7f632c0d0d30 msgr2=0x7f632c0d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:37.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- 172.21.15.122:0/1490885104 shutdown_connections 2024-09-17T05:30:37.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- 172.21.15.122:0/1490885104 wait complete. 2024-09-17T05:30:37.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 Processor -- start 2024-09-17T05:30:37.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- start start 2024-09-17T05:30:37.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f632c06d330 0x7f632c149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c074650 0x7f632c1f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f632c0cb410 0x7f632c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f632c16a680 con 0x7f632c074650 2024-09-17T05:30:37.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f632c16a500 con 0x7f632c06d330 2024-09-17T05:30:37.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6334600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f632c16a800 con 0x7f632c0cb410 2024-09-17T05:30:37.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6331200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c074650 0x7f632c1f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6331200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c074650 0x7f632c1f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45778/0 (socket says 172.21.15.122:45778) 2024-09-17T05:30:37.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.893+0000 7f6331200640 1 -- 172.21.15.122:0/4209993893 learned_addr learned my addr 172.21.15.122:0/4209993893 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:37.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6331200640 1 -- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f632c0cb410 msgr2=0x7f632c15d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6332600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f632c0cb410 0x7f632c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6331c00640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f632c06d330 0x7f632c149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6331200640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f632c0cb410 0x7f632c15d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6331200640 1 -- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f632c06d330 msgr2=0x7f632c149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6331200640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f632c06d330 0x7f632c149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6331200640 1 -- 172.21.15.122:0/4209993893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c06d130 con 0x7f632c074650 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6331c00640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f632c06d330 0x7f632c149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6332600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f632c0cb410 0x7f632c15d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6331200640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c074650 0x7f632c1f6810 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6318009790 tx=0x7f6318004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6329600640 1 -- 172.21.15.122:0/4209993893 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63180042f0 con 0x7f632c074650 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6329600640 1 -- 172.21.15.122:0/4209993893 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6318005030 con 0x7f632c074650 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6329600640 1 -- 172.21.15.122:0/4209993893 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318005460 con 0x7f632c074650 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c149f70 con 0x7f632c074650 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c14a530 con 0x7f632c074650 2024-09-17T05:30:37.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632c073980 con 0x7f632c074650 2024-09-17T05:30:37.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.897+0000 7f6329600640 1 -- 172.21.15.122:0/4209993893 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f63180c0020 con 0x7f632c074650 2024-09-17T05:30:37.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.901+0000 7f6329600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f630c073b00 0x7f630c075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:37.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.901+0000 7f6329600640 1 -- 172.21.15.122:0/4209993893 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f631812d2b0 con 0x7f632c074650 2024-09-17T05:30:37.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.901+0000 7f6331c00640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f630c073b00 0x7f630c075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:37.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.901+0000 7f6331c00640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f630c073b00 0x7f630c075fc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6320000f70 tx=0x7f6320009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:37.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:37.905+0000 7f6329600640 1 -- 172.21.15.122:0/4209993893 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f63180c7020 con 0x7f632c074650 2024-09-17T05:30:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.073+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f632c0af100 con 0x7f632c074650 2024-09-17T05:30:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.073+0000 7f6329600640 1 -- 172.21.15.122:0/4209993893 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v34) ==== 116+0+2 (secure 0 0 0) 0x7f6318016410 con 0x7f632c074650 2024-09-17T05:30:38.075 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f630c073b00 msgr2=0x7f630c075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f630c073b00 0x7f630c075fc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6320000f70 tx=0x7f6320009040 comp rx=0 tx=0).stop 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c074650 msgr2=0x7f632c1f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c074650 0x7f632c1f6810 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6318009790 tx=0x7f6318004040 comp rx=0 tx=0).stop 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 shutdown_connections 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f630c073b00 0x7f630c075fc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f632c0cb410 0x7f632c15d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f632c074650 0x7f632c1f6810 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 --2- 172.21.15.122:0/4209993893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f632c06d330 0x7f632c149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 >> 172.21.15.122:0/4209993893 conn(0x7f632c0d0d30 msgr2=0x7f632c15fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 shutdown_connections 2024-09-17T05:30:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.077+0000 7f6334600640 1 -- 172.21.15.122:0/4209993893 wait complete. 2024-09-17T05:30:38.097 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:30:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- 172.21.15.122:0/180725880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 msgr2=0x7fdeac074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 --2- 172.21.15.122:0/180725880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 0x7fdeac074310 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fde9c009900 tx=0x7fde9c01c850 comp rx=0 tx=0).stop 2024-09-17T05:30:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- 172.21.15.122:0/180725880 shutdown_connections 2024-09-17T05:30:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 --2- 172.21.15.122:0/180725880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 0x7fdeac074310 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- 172.21.15.122:0/180725880 >> 172.21.15.122:0/180725880 conn(0x7fdeac0d0c90 msgr2=0x7fdeac0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- 172.21.15.122:0/180725880 shutdown_connections 2024-09-17T05:30:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- 172.21.15.122:0/180725880 wait complete. 2024-09-17T05:30:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 Processor -- start 2024-09-17T05:30:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- start start 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac074850 0x7fdeac154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 0x7fdeac14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15efb0 0x7fdeac14d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac169460 con 0x7fdeac151ce0 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac1692e0 con 0x7fdeac074850 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb3a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdeac1695e0 con 0x7fdeac15efb0 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdea3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 0x7fdeac14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdea3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 0x7fdeac14d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45806/0 (socket says 172.21.15.122:45806) 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdea3e00640 1 -- 172.21.15.122:0/4216609331 learned_addr learned my addr 172.21.15.122:0/4216609331 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb1000640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac074850 0x7fdeac154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.241+0000 7fdeb1a00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15efb0 0x7fdeac14d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea3e00640 1 -- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15efb0 msgr2=0x7fdeac14d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea3e00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15efb0 0x7fdeac14d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea3e00640 1 -- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac074850 msgr2=0x7fdeac154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea3e00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac074850 0x7fdeac154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea3e00640 1 -- 172.21.15.122:0/4216609331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac15b9c0 con 0x7fdeac151ce0 2024-09-17T05:30:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea3e00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 0x7fdeac14d0a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fde9400c480 tx=0x7fde9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:38.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea9600640 1 -- 172.21.15.122:0/4216609331 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde940052d0 con 0x7fdeac151ce0 2024-09-17T05:30:38.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea9600640 1 -- 172.21.15.122:0/4216609331 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde94005470 con 0x7fdeac151ce0 2024-09-17T05:30:38.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea9600640 1 -- 172.21.15.122:0/4216609331 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94012870 con 0x7fdeac151ce0 2024-09-17T05:30:38.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeac15b040 con 0x7fdeac151ce0 2024-09-17T05:30:38.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeac0b6620 con 0x7fdeac151ce0 2024-09-17T05:30:38.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeac074310 con 0x7fdeac151ce0 2024-09-17T05:30:38.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea9600640 1 -- 172.21.15.122:0/4216609331 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7fde94003600 con 0x7fdeac151ce0 2024-09-17T05:30:38.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.245+0000 7fdea9600640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde88073b00 0x7fde88075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.249+0000 7fdea9600640 1 -- 172.21.15.122:0/4216609331 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fde940925e0 con 0x7fdeac151ce0 2024-09-17T05:30:38.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.249+0000 7fdeb1000640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde88073b00 0x7fde88075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.249+0000 7fdeb1000640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde88073b00 0x7fde88075fc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0af000 tx=0x7fde9c01b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:38.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.249+0000 7fdea9600640 1 -- 172.21.15.122:0/4216609331 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fde9405ea90 con 0x7fdeac151ce0 2024-09-17T05:30:38.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.417+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fdeac075c10 con 0x7fdeac151ce0 2024-09-17T05:30:38.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.421+0000 7fdea9600640 1 -- 172.21.15.122:0/4216609331 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7fde940634c0 con 0x7fdeac151ce0 2024-09-17T05:30:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde88073b00 msgr2=0x7fde88075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde88073b00 0x7fde88075fc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0af000 tx=0x7fde9c01b090 comp rx=0 tx=0).stop 2024-09-17T05:30:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 msgr2=0x7fdeac14d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 0x7fdeac14d0a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fde9400c480 tx=0x7fde9400c940 comp rx=0 tx=0).stop 2024-09-17T05:30:38.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 shutdown_connections 2024-09-17T05:30:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde88073b00 0x7fde88075fc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15efb0 0x7fdeac14d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdeac151ce0 0x7fdeac14d0a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 --2- 172.21.15.122:0/4216609331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac074850 0x7fdeac154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 >> 172.21.15.122:0/4216609331 conn(0x7fdeac0d0c90 msgr2=0x7fdeac06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 shutdown_connections 2024-09-17T05:30:38.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.425+0000 7fdeb3a00640 1 -- 172.21.15.122:0/4216609331 wait complete. 2024-09-17T05:30:38.439 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.442+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.439 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.442+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.440 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.442+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.440 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.442+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.440 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.442+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.444 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T05:30:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- 172.21.15.122:0/4034905142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 msgr2=0x7f7b200d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 --2- 172.21.15.122:0/4034905142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 0x7f7b200d28c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009840 tx=0x7f7b1001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- 172.21.15.122:0/4034905142 shutdown_connections 2024-09-17T05:30:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 --2- 172.21.15.122:0/4034905142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 0x7f7b200d28c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- 172.21.15.122:0/4034905142 >> 172.21.15.122:0/4034905142 conn(0x7f7b200d1da0 msgr2=0x7f7b200d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- 172.21.15.122:0/4034905142 shutdown_connections 2024-09-17T05:30:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- 172.21.15.122:0/4034905142 wait complete. 2024-09-17T05:30:38.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 Processor -- start 2024-09-17T05:30:38.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- start start 2024-09-17T05:30:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 0x7f7b20145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b200d02e0 0x7f7b20153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b2015f700 0x7f7b20154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20169f90 con 0x7f7b200cff00 2024-09-17T05:30:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20169e10 con 0x7f7b200d02e0 2024-09-17T05:30:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.589+0000 7f7b24e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2016a110 con 0x7f7b2015f700 2024-09-17T05:30:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 0x7f7b20145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 0x7f7b20145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45816/0 (socket says 172.21.15.122:45816) 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 -- 172.21.15.122:0/1723277722 learned_addr learned my addr 172.21.15.122:0/1723277722 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b16a00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b200d02e0 0x7f7b20153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1fe00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b2015f700 0x7f7b20154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 -- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b2015f700 msgr2=0x7f7b20154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b2015f700 0x7f7b20154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 -- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b200d02e0 msgr2=0x7f7b20153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b200d02e0 0x7f7b20153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 -- 172.21.15.122:0/1723277722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b200c5a00 con 0x7f7b200cff00 2024-09-17T05:30:38.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b1f400640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 0x7f7b20145000 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7b100b0040 tx=0x7f7b10004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b17e00640 1 -- 172.21.15.122:0/1723277722 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b100b2040 con 0x7f7b200cff00 2024-09-17T05:30:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b17e00640 1 -- 172.21.15.122:0/1723277722 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b100036f0 con 0x7f7b200cff00 2024-09-17T05:30:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b200c5ce0 con 0x7f7b200cff00 2024-09-17T05:30:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b17e00640 1 -- 172.21.15.122:0/1723277722 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10003b20 con 0x7f7b200cff00 2024-09-17T05:30:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b200c62a0 con 0x7f7b200cff00 2024-09-17T05:30:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b200d28c0 con 0x7f7b200cff00 2024-09-17T05:30:38.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.593+0000 7f7b17e00640 1 -- 172.21.15.122:0/1723277722 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f7b100bd060 con 0x7f7b200cff00 2024-09-17T05:30:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.597+0000 7f7b17e00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7afc073b00 0x7f7afc075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.597+0000 7f7b17e00640 1 -- 172.21.15.122:0/1723277722 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f7b1012c540 con 0x7f7b200cff00 2024-09-17T05:30:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.597+0000 7f7b16a00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7afc073b00 0x7f7afc075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:38.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.597+0000 7f7b16a00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7afc073b00 0x7f7afc075fc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b200ccca0 tx=0x7f7b08009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:38.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.601+0000 7f7b17e00640 1 -- 172.21.15.122:0/1723277722 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7b10016410 con 0x7f7b200cff00 2024-09-17T05:30:38.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.793+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f7b2015e740 con 0x7f7afc073b00 2024-09-17T05:30:38.805 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.806+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.805 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.806+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.805 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.806+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.805 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.806+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.805 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:30:38.806+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:30:38.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b17e00640 1 -- 172.21.15.122:0/1723277722 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7b2015e740 con 0x7f7afc073b00 2024-09-17T05:30:38.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7afc073b00 msgr2=0x7f7afc075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7afc073b00 0x7f7afc075fc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b200ccca0 tx=0x7f7b08009040 comp rx=0 tx=0).stop 2024-09-17T05:30:38.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 msgr2=0x7f7b20145000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 0x7f7b20145000 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7b100b0040 tx=0x7f7b10004220 comp rx=0 tx=0).stop 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 shutdown_connections 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7afc073b00 0x7f7afc075fc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b2015f700 0x7f7b20154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b200d02e0 0x7f7b20153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 --2- 172.21.15.122:0/1723277722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b200cff00 0x7f7b20145000 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 >> 172.21.15.122:0/1723277722 conn(0x7f7b200d1da0 msgr2=0x7f7b200d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 shutdown_connections 2024-09-17T05:30:38.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:38.841+0000 7f7b24e00640 1 -- 172.21.15.122:0/1723277722 wait complete. 2024-09-17T05:30:38.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:30:39.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.009+0000 7f2dcb000640 1 -- 172.21.15.122:0/2187694344 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc41605d0 msgr2=0x7f2dc40d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.009+0000 7f2dcb000640 1 --2- 172.21.15.122:0/2187694344 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc41605d0 0x7f2dc40d0690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009900 tx=0x7f2db401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.009+0000 7f2dcb000640 1 -- 172.21.15.122:0/2187694344 shutdown_connections 2024-09-17T05:30:39.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.009+0000 7f2dcb000640 1 --2- 172.21.15.122:0/2187694344 >> v2:172.21.15.173:3300/0 conn(0x7f2dc40b1650 0x7f2dc40b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.009+0000 7f2dcb000640 1 --2- 172.21.15.122:0/2187694344 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc41605d0 0x7f2dc40d0690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.009+0000 7f2dcb000640 1 -- 172.21.15.122:0/2187694344 >> 172.21.15.122:0/2187694344 conn(0x7f2dc40d1dd0 msgr2=0x7f2dc40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.009+0000 7f2dcb000640 1 -- 172.21.15.122:0/2187694344 shutdown_connections 2024-09-17T05:30:39.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.009+0000 7f2dcb000640 1 -- 172.21.15.122:0/2187694344 wait complete. 2024-09-17T05:30:39.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 Processor -- start 2024-09-17T05:30:39.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 -- start start 2024-09-17T05:30:39.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc40aeb00 0x7f2dc41f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dc40b1650 0x7f2dc41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc41605d0 0x7f2dc41659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc40b1220 con 0x7f2dc40b1650 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc40b0ee0 con 0x7f2dc41605d0 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc40b13a0 con 0x7f2dc40aeb00 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dc40b1650 0x7f2dc41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dc40b1650 0x7f2dc41579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45822/0 (socket says 172.21.15.122:45822) 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 -- 172.21.15.122:0/324616244 learned_addr learned my addr 172.21.15.122:0/324616244 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dc3e00640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc40aeb00 0x7f2dc41f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dc9000640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc41605d0 0x7f2dc41659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 -- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc40aeb00 msgr2=0x7f2dc41f8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc40aeb00 0x7f2dc41f8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 -- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc41605d0 msgr2=0x7f2dc41659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc41605d0 0x7f2dc41659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 -- 172.21.15.122:0/324616244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc40722b0 con 0x7f2dc40b1650 2024-09-17T05:30:39.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dbb400640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dc40b1650 0x7f2dc41579e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00ea20 tx=0x7f2dac00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dc0c00640 1 -- 172.21.15.122:0/324616244 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00cd20 con 0x7f2dc40b1650 2024-09-17T05:30:39.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dc0c00640 1 -- 172.21.15.122:0/324616244 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dac010430 con 0x7f2dc40b1650 2024-09-17T05:30:39.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dc0c00640 1 -- 172.21.15.122:0/324616244 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac010860 con 0x7f2dc40b1650 2024-09-17T05:30:39.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc41f8a50 con 0x7f2dc40b1650 2024-09-17T05:30:39.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc406e170 con 0x7f2dc40b1650 2024-09-17T05:30:39.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.013+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc40b1a30 con 0x7f2dc40b1650 2024-09-17T05:30:39.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.017+0000 7f2dc0c00640 1 -- 172.21.15.122:0/324616244 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 98050+0+0 (secure 0 0 0) 0x7f2dac010a00 con 0x7f2dc40b1650 2024-09-17T05:30:39.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.017+0000 7f2dc0c00640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2da0073b00 0x7f2da0075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:30:39.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.017+0000 7f2dc0c00640 1 -- 172.21.15.122:0/324616244 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f2dac093590 con 0x7f2dc40b1650 2024-09-17T05:30:39.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.017+0000 7f2dc3e00640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2da0073b00 0x7f2da0075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:30:39.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.017+0000 7f2dc3e00640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2da0073b00 0x7f2da0075fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2db40af000 tx=0x7f2db401b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:30:39.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.021+0000 7f2dc0c00640 1 -- 172.21.15.122:0/324616244 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2dc41f8a50 con 0x7f2dc40b1650 2024-09-17T05:30:39.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.213+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2dc40d10e0 con 0x7f2da0073b00 2024-09-17T05:30:39.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.221+0000 7f2dc0c00640 1 -- 172.21.15.122:0/324616244 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f2db40042a0 con 0x7f2da0073b00 2024-09-17T05:30:39.223 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:30:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2da0073b00 msgr2=0x7f2da0075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2da0073b00 0x7f2da0075fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2db40af000 tx=0x7f2db401b120 comp rx=0 tx=0).stop 2024-09-17T05:30:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dc40b1650 msgr2=0x7f2dc41579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:30:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dc40b1650 0x7f2dc41579e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00ea20 tx=0x7f2dac00eee0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 shutdown_connections 2024-09-17T05:30:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2da0073b00 0x7f2da0075fc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc41605d0 0x7f2dc41659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dc40b1650 0x7f2dc41579e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 --2- 172.21.15.122:0/324616244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc40aeb00 0x7f2dc41f8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:30:39.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 >> 172.21.15.122:0/324616244 conn(0x7f2dc40d1dd0 msgr2=0x7f2dc4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:30:39.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 shutdown_connections 2024-09-17T05:30:39.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:30:39.225+0000 7f2dcb000640 1 -- 172.21.15.122:0/324616244 wait complete. 2024-09-17T05:30:39.244 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:30:39.244 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:39.244 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:39.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:39.252 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:30:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:30:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:30:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.81611 s, 122 MB/s 2024-09-17T05:30:48.130 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:48.130 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:48.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:48.139 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:30:56.827 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:30:56.827 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:30:56.827 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.63733 s, 124 MB/s 2024-09-17T05:30:56.828 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:30:56.829 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:30:56.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:30:56.837 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:31:02.573 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:31:02.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:31:05.959 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:31:05.959 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:31:05.959 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.06987 s, 118 MB/s 2024-09-17T05:31:05.960 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:31:05.960 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:05.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:05.969 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:31:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:31:14.626 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:31:14.626 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.60451 s, 125 MB/s 2024-09-17T05:31:14.627 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:31:14.627 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:14.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:14.636 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:31:23.148 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:31:23.148 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:31:23.148 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.45453 s, 127 MB/s 2024-09-17T05:31:23.150 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:31:23.150 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:23.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:23.159 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:31:31.972 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:31:31.972 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:31:31.972 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.76257 s, 123 MB/s 2024-09-17T05:31:31.973 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:31:31.973 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:31.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:31.982 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:31:32.638 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:31:32.641 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:31:40.926 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:31:40.927 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:31:40.927 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.89432 s, 121 MB/s 2024-09-17T05:31:40.928 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:31:40.928 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:40.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:40.937 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:31:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:31:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:31:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.71476 s, 123 MB/s 2024-09-17T05:31:49.704 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:31:49.705 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:49.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:49.717 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:31:58.607 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:31:58.607 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:31:58.607 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.83872 s, 121 MB/s 2024-09-17T05:31:58.608 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:31:58.608 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:31:58.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:31:58.617 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:32:02.691 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:32:02.696 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:32:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:32:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:32:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.92969 s, 120 MB/s 2024-09-17T05:32:07.603 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T05:32:07.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.748+0000 7f5d0a000640 1 -- 172.21.15.122:0/93957874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0415bf60 msgr2=0x7f5d040733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.748+0000 7f5d0a000640 1 --2- 172.21.15.122:0/93957874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0415bf60 0x7f5d040733a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009840 tx=0x7f5cf401c820 comp rx=0 tx=0).stop 2024-09-17T05:32:07.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.748+0000 7f5d0a000640 1 -- 172.21.15.122:0/93957874 shutdown_connections 2024-09-17T05:32:07.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.748+0000 7f5d0a000640 1 --2- 172.21.15.122:0/93957874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d040cdac0 0x7f5d040cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.748+0000 7f5d0a000640 1 --2- 172.21.15.122:0/93957874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0415bf60 0x7f5d040733a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.748+0000 7f5d0a000640 1 -- 172.21.15.122:0/93957874 >> 172.21.15.122:0/93957874 conn(0x7f5d040cfbc0 msgr2=0x7f5d040cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:07.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.748+0000 7f5d0a000640 1 -- 172.21.15.122:0/93957874 shutdown_connections 2024-09-17T05:32:07.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.748+0000 7f5d0a000640 1 -- 172.21.15.122:0/93957874 wait complete. 2024-09-17T05:32:07.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 Processor -- start 2024-09-17T05:32:07.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 -- start start 2024-09-17T05:32:07.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d040b5170 0x7f5d041574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d040cdac0 0x7f5d0406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0415bf60 0x7f5d0406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04169020 con 0x7f5d0415bf60 2024-09-17T05:32:07.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5d04168ea0 con 0x7f5d040b5170 2024-09-17T05:32:07.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5d041691a0 con 0x7f5d040cdac0 2024-09-17T05:32:07.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0415bf60 0x7f5d0406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0415bf60 0x7f5d0406fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39692/0 (socket says 172.21.15.122:39692) 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 -- 172.21.15.122:0/1790273682 learned_addr learned my addr 172.21.15.122:0/1790273682 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5cfaa00640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d040cdac0 0x7f5d0406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03400640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d040b5170 0x7f5d041574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 -- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d040cdac0 msgr2=0x7f5d0406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d040cdac0 0x7f5d0406e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 -- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d040b5170 msgr2=0x7f5d041574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d040b5170 0x7f5d041574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 -- 172.21.15.122:0/1790273682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d04160850 con 0x7f5d0415bf60 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d03e00640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0415bf60 0x7f5d0406fef0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00eb20 tx=0x7f5cfc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5cfbe00640 1 -- 172.21.15.122:0/1790273682 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc004b30 con 0x7f5d0415bf60 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5cfbe00640 1 -- 172.21.15.122:0/1790273682 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cfc004cf0 con 0x7f5d0415bf60 2024-09-17T05:32:07.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0406ecd0 con 0x7f5d0415bf60 2024-09-17T05:32:07.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5cfbe00640 1 -- 172.21.15.122:0/1790273682 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc005280 con 0x7f5d0415bf60 2024-09-17T05:32:07.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.752+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0406f290 con 0x7f5d0415bf60 2024-09-17T05:32:07.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.756+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d04069410 con 0x7f5d0415bf60 2024-09-17T05:32:07.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.756+0000 7f5cfbe00640 1 -- 172.21.15.122:0/1790273682 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 98050+0+0 (secure 0 0 0) 0x7f5cfc005420 con 0x7f5d0415bf60 2024-09-17T05:32:07.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.756+0000 7f5cfbe00640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ce0073b00 0x7f5ce0075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:07.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.756+0000 7f5cfbe00640 1 -- 172.21.15.122:0/1790273682 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f5cfc0936a0 con 0x7f5d0415bf60 2024-09-17T05:32:07.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.756+0000 7f5d03400640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ce0073b00 0x7f5ce0075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:07.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.760+0000 7f5d03400640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ce0073b00 0x7f5ce0075fc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40af000 tx=0x7f5cf401b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:07.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.760+0000 7f5cfbe00640 1 -- 172.21.15.122:0/1790273682 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5cfc05fb50 con 0x7f5d0415bf60 2024-09-17T05:32:07.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.956+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f5d040cdf50 con 0x7f5ce0073b00 2024-09-17T05:32:07.964 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:07.965+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:07.964 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:07.965+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:07.964 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:07.965+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:07.964 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:07.965+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:07.965 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:07.965+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:07.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.980+0000 7f5cfbe00640 1 -- 172.21.15.122:0/1790273682 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5d040cdf50 con 0x7f5ce0073b00 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ce0073b00 msgr2=0x7f5ce0075fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ce0073b00 0x7f5ce0075fc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40af000 tx=0x7f5cf401b060 comp rx=0 tx=0).stop 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0415bf60 msgr2=0x7f5d0406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0415bf60 0x7f5d0406fef0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00eb20 tx=0x7f5cfc00c2d0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 shutdown_connections 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ce0073b00 0x7f5ce0075fc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0415bf60 0x7f5d0406fef0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d040cdac0 0x7f5d0406e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 --2- 172.21.15.122:0/1790273682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d040b5170 0x7f5d041574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:07.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 >> 172.21.15.122:0/1790273682 conn(0x7f5d040cfbc0 msgr2=0x7f5d04072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:07.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 shutdown_connections 2024-09-17T05:32:07.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:07.984+0000 7f5d0a000640 1 -- 172.21.15.122:0/1790273682 wait complete. 2024-09-17T05:32:08.004 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:32:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.148+0000 7f51d1800640 1 -- 172.21.15.122:0/3778130307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 msgr2=0x7f51cc0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.148+0000 7f51d1800640 1 --2- 172.21.15.122:0/3778130307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 0x7f51cc0d0240 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009900 tx=0x7f51bc01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.148+0000 7f51d1800640 1 -- 172.21.15.122:0/3778130307 shutdown_connections 2024-09-17T05:32:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.148+0000 7f51d1800640 1 --2- 172.21.15.122:0/3778130307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51cc0b0d10 0x7f51cc0d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.148+0000 7f51d1800640 1 --2- 172.21.15.122:0/3778130307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 0x7f51cc0d0240 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.148+0000 7f51d1800640 1 -- 172.21.15.122:0/3778130307 >> 172.21.15.122:0/3778130307 conn(0x7f51cc09aa10 msgr2=0x7f51cc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.148+0000 7f51d1800640 1 -- 172.21.15.122:0/3778130307 shutdown_connections 2024-09-17T05:32:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.148+0000 7f51d1800640 1 -- 172.21.15.122:0/3778130307 wait complete. 2024-09-17T05:32:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 Processor -- start 2024-09-17T05:32:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 -- start start 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51cc0b0d10 0x7f51cc0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51cc0d20f0 0x7f51cc15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 0x7f51cc156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc168cb0 con 0x7f51cc15bd40 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc168b30 con 0x7f51cc0b0d10 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f51cc168e30 con 0x7f51cc0d20f0 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51cbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51cc0b0d10 0x7f51cc0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d0e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 0x7f51cc156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51cbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51cc0b0d10 0x7f51cc0b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:53538/0 (socket says 172.21.15.122:53538) 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51cbe00640 1 -- 172.21.15.122:0/1917989626 learned_addr learned my addr 172.21.15.122:0/1917989626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51c3400640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51cc0d20f0 0x7f51cc15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d0e00640 1 -- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51cc0d20f0 msgr2=0x7f51cc15dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d0e00640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51cc0d20f0 0x7f51cc15dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d0e00640 1 -- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51cc0b0d10 msgr2=0x7f51cc0b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d0e00640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51cc0b0d10 0x7f51cc0b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d0e00640 1 -- 172.21.15.122:0/1917989626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc15e230 con 0x7f51cc15bd40 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51cbe00640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51cc0b0d10 0x7f51cc0b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51c3400640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51cc0d20f0 0x7f51cc15dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d0e00640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 0x7f51cc156ad0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c700 tx=0x7f51c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51c8c00640 1 -- 172.21.15.122:0/1917989626 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4003b80 con 0x7f51cc15bd40 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51c8c00640 1 -- 172.21.15.122:0/1917989626 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51c4003d20 con 0x7f51cc15bd40 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51cc15e500 con 0x7f51cc15bd40 2024-09-17T05:32:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51c8c00640 1 -- 172.21.15.122:0/1917989626 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c400aab0 con 0x7f51cc15bd40 2024-09-17T05:32:08.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51cc0b4d00 con 0x7f51cc15bd40 2024-09-17T05:32:08.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.152+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51cc0d1bb0 con 0x7f51cc15bd40 2024-09-17T05:32:08.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.156+0000 7f51c8c00640 1 -- 172.21.15.122:0/1917989626 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 98050+0+0 (secure 0 0 0) 0x7f51c400ac50 con 0x7f51cc15bd40 2024-09-17T05:32:08.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.156+0000 7f51c8c00640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a8073b30 0x7f51a8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:08.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.156+0000 7f51c8c00640 1 -- 172.21.15.122:0/1917989626 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f51c40926d0 con 0x7f51cc15bd40 2024-09-17T05:32:08.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.156+0000 7f51cbe00640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a8073b30 0x7f51a8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:08.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.156+0000 7f51cbe00640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a8073b30 0x7f51a8075ff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f51bc01a510 tx=0x7f51bc01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:08.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.160+0000 7f51c8c00640 1 -- 172.21.15.122:0/1917989626 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f51c405eb80 con 0x7f51cc15bd40 2024-09-17T05:32:08.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.356+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f51cc15f0d0 con 0x7f51a8073b30 2024-09-17T05:32:08.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.360+0000 7f51c8c00640 1 -- 172.21.15.122:0/1917989626 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f51cc15f0d0 con 0x7f51a8073b30 2024-09-17T05:32:08.363 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:32:08.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a8073b30 msgr2=0x7f51a8075ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a8073b30 0x7f51a8075ff0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f51bc01a510 tx=0x7f51bc01b120 comp rx=0 tx=0).stop 2024-09-17T05:32:08.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 msgr2=0x7f51cc156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:08.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 0x7f51cc156ad0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c700 tx=0x7f51c400cbc0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 shutdown_connections 2024-09-17T05:32:08.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51a8073b30 0x7f51a8075ff0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51cc15bd40 0x7f51cc156ad0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51cc0d20f0 0x7f51cc15dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 --2- 172.21.15.122:0/1917989626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51cc0b0d10 0x7f51cc0b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:08.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 >> 172.21.15.122:0/1917989626 conn(0x7f51cc09aa10 msgr2=0x7f51cc15a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:08.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 shutdown_connections 2024-09-17T05:32:08.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:08.364+0000 7f51d1800640 1 -- 172.21.15.122:0/1917989626 wait complete. 2024-09-17T05:32:08.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:08.384 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:32:08.384 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:32:08.393 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:32:08.393 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="10405347328" 2024-09-17T05:32:08.393 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:08.394 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-17T05:32:10.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:10.395 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:32:10.395 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:32:10.407 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:32:10.407 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="10405347328" 2024-09-17T05:32:10.407 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:10.408 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-17T05:32:12.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:12.409 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:32:12.409 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:32:12.420 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/a4140c04-4d6b-4f9c-809f-6ab84a420271 2024-09-17T05:32:12.420 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="10737418240" 2024-09-17T05:32:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:12.421 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-17T05:32:12.421 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-17T05:32:12.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- 172.21.15.122:0/1007174152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40d2810 msgr2=0x7f8bc4160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 --2- 172.21.15.122:0/1007174152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40d2810 0x7f8bc4160350 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009840 tx=0x7f8bb001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- 172.21.15.122:0/1007174152 shutdown_connections 2024-09-17T05:32:12.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 --2- 172.21.15.122:0/1007174152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40d2810 0x7f8bc4160350 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- 172.21.15.122:0/1007174152 >> 172.21.15.122:0/1007174152 conn(0x7f8bc40d1cf0 msgr2=0x7f8bc40d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:12.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- 172.21.15.122:0/1007174152 shutdown_connections 2024-09-17T05:32:12.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- 172.21.15.122:0/1007174152 wait complete. 2024-09-17T05:32:12.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 Processor -- start 2024-09-17T05:32:12.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- start start 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc40b25e0 0x7f8bc414acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40cfd30 0x7f8bc4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc40d2810 0x7f8bc41f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc416a0a0 con 0x7f8bc40cfd30 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4169f20 con 0x7f8bc40d2810 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.616+0000 7f8bca000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc416a220 con 0x7f8bc40b25e0 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40cfd30 0x7f8bc4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40cfd30 0x7f8bc4156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46962/0 (socket says 172.21.15.122:46962) 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 -- 172.21.15.122:0/1874461278 learned_addr learned my addr 172.21.15.122:0/1874461278 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:12.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc8c00640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc40b25e0 0x7f8bc414acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:12.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc9600640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc40d2810 0x7f8bc41f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:12.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 -- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc40b25e0 msgr2=0x7f8bc414acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc40b25e0 0x7f8bc414acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 -- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc40d2810 msgr2=0x7f8bc41f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc40d2810 0x7f8bc41f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 -- 172.21.15.122:0/1874461278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc406e4a0 con 0x7f8bc40cfd30 2024-09-17T05:32:12.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc3e00640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40cfd30 0x7f8bc4156300 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009790 tx=0x7f8bb0004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:12.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc0c00640 1 -- 172.21.15.122:0/1874461278 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00b2040 con 0x7f8bc40cfd30 2024-09-17T05:32:12.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc0c00640 1 -- 172.21.15.122:0/1874461278 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bb0005310 con 0x7f8bc40cfd30 2024-09-17T05:32:12.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc0c00640 1 -- 172.21.15.122:0/1874461278 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00044d0 con 0x7f8bc40cfd30 2024-09-17T05:32:12.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc406e740 con 0x7f8bc40cfd30 2024-09-17T05:32:12.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc406ed00 con 0x7f8bc40cfd30 2024-09-17T05:32:12.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc40af540 con 0x7f8bc40cfd30 2024-09-17T05:32:12.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.620+0000 7f8bc0c00640 1 -- 172.21.15.122:0/1874461278 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 98101+0+0 (secure 0 0 0) 0x7f8bb00a5830 con 0x7f8bc40cfd30 2024-09-17T05:32:12.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.624+0000 7f8bc0c00640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba4073bc0 0x7f8ba4076080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:12.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.624+0000 7f8bc0c00640 1 -- 172.21.15.122:0/1874461278 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f8bb012c560 con 0x7f8bc40cfd30 2024-09-17T05:32:12.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.624+0000 7f8bc8c00640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba4073bc0 0x7f8ba4076080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:12.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.624+0000 7f8bc8c00640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba4073bc0 0x7f8ba4076080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80046c0 tx=0x7f8bb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:12.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.628+0000 7f8bc0c00640 1 -- 172.21.15.122:0/1874461278 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8bb00bc060 con 0x7f8bc40cfd30 2024-09-17T05:32:12.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.824+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f8bc406eed0 con 0x7f8ba4073bc0 2024-09-17T05:32:12.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.892+0000 7f8bc0c00640 1 -- 172.21.15.122:0/1874461278 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8bc406eed0 con 0x7f8ba4073bc0 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba4073bc0 msgr2=0x7f8ba4076080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba4073bc0 0x7f8ba4076080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80046c0 tx=0x7f8bb8009040 comp rx=0 tx=0).stop 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40cfd30 msgr2=0x7f8bc4156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40cfd30 0x7f8bc4156300 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009790 tx=0x7f8bb0004220 comp rx=0 tx=0).stop 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 shutdown_connections 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba4073bc0 0x7f8ba4076080 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc40d2810 0x7f8bc41f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc40cfd30 0x7f8bc4156300 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 --2- 172.21.15.122:0/1874461278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc40b25e0 0x7f8bc414acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:12.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 >> 172.21.15.122:0/1874461278 conn(0x7f8bc40d1cf0 msgr2=0x7f8bc40d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:12.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 shutdown_connections 2024-09-17T05:32:12.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:12.896+0000 7f8bca000640 1 -- 172.21.15.122:0/1874461278 wait complete. 2024-09-17T05:32:12.898 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.901+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.899 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.901+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.899 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.901+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.899 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.901+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.899 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.901+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.915 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.917+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.916 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.917+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.916 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.917+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.916 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.917+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.916 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:12.917+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:12.917 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:32:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb551200640 1 -- 172.21.15.122:0/433053790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb530005350 msgr2=0x7fb53000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb551200640 1 --2- 172.21.15.122:0/433053790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb530005350 0x7fb53000b9d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0753f0 tx=0x7fb54401cc20 comp rx=0 tx=0).stop 2024-09-17T05:32:13.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb54be00640 1 -- 172.21.15.122:0/433053790 reap_dead start 2024-09-17T05:32:13.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb551200640 1 -- 172.21.15.122:0/433053790 shutdown_connections 2024-09-17T05:32:13.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb551200640 1 -- 172.21.15.122:0/433053790 >> 172.21.15.122:0/433053790 conn(0x7fb54c0d1c30 msgr2=0x7fb54c0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:13.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb551200640 1 -- 172.21.15.122:0/433053790 shutdown_connections 2024-09-17T05:32:13.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb551200640 1 -- 172.21.15.122:0/433053790 wait complete. 2024-09-17T05:32:13.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb551200640 1 Processor -- start 2024-09-17T05:32:13.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.060+0000 7fb551200640 1 -- start start 2024-09-17T05:32:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb530005350 0x7fb54c0b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54c15a1b0 0x7fb54c1f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c1f5c90 0x7fb54c1f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c167b70 con 0x7fb54c1f5c90 2024-09-17T05:32:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1679f0 con 0x7fb530005350 2024-09-17T05:32:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c167cf0 con 0x7fb54c15a1b0 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c1f5c90 0x7fb54c1f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c1f5c90 0x7fb54c1f6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46990/0 (socket says 172.21.15.122:46990) 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 -- 172.21.15.122:0/3790576942 learned_addr learned my addr 172.21.15.122:0/3790576942 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54b400640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb530005350 0x7fb54c0b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb542a00640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54c15a1b0 0x7fb54c1f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 -- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54c15a1b0 msgr2=0x7fb54c1f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54c15a1b0 0x7fb54c1f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 -- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb530005350 msgr2=0x7fb54c0b73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb530005350 0x7fb54c0b73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 -- 172.21.15.122:0/3790576942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c0b6480 con 0x7fb54c1f5c90 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb54be00640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c1f5c90 0x7fb54c1f6110 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb5440985d0 tx=0x7fb5440a5e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb543e00640 1 -- 172.21.15.122:0/3790576942 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544001ba0 con 0x7fb54c1f5c90 2024-09-17T05:32:13.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb543e00640 1 -- 172.21.15.122:0/3790576942 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5440b4020 con 0x7fb54c1f5c90 2024-09-17T05:32:13.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb543e00640 1 -- 172.21.15.122:0/3790576942 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544006c50 con 0x7fb54c1f5c90 2024-09-17T05:32:13.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c15a610 con 0x7fb54c1f5c90 2024-09-17T05:32:13.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c0b56c0 con 0x7fb54c1f5c90 2024-09-17T05:32:13.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.064+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c0cfb00 con 0x7fb54c1f5c90 2024-09-17T05:32:13.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.068+0000 7fb543e00640 1 -- 172.21.15.122:0/3790576942 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 98101+0+0 (secure 0 0 0) 0x7fb544003b80 con 0x7fb54c1f5c90 2024-09-17T05:32:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.068+0000 7fb543e00640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb528073bc0 0x7fb528076080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.068+0000 7fb543e00640 1 -- 172.21.15.122:0/3790576942 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fb54412c530 con 0x7fb54c1f5c90 2024-09-17T05:32:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.068+0000 7fb54b400640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb528073bc0 0x7fb528076080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.068+0000 7fb54b400640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb528073bc0 0x7fb528076080 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb54c073f10 tx=0x7fb53c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:13.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.072+0000 7fb543e00640 1 -- 172.21.15.122:0/3790576942 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb5440be030 con 0x7fb54c1f5c90 2024-09-17T05:32:13.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.300+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb54c071cd0 con 0x7fb54c1f5c90 2024-09-17T05:32:13.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.300+0000 7fb543e00640 1 -- 172.21.15.122:0/3790576942 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1912 (secure 0 0 0) 0x7fb5440169d0 con 0x7fb54c1f5c90 2024-09-17T05:32:13.302 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:13.302 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:32:13.302 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:32:13.302 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:32:13.303 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 454, 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 62, 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:32:13.304 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 2585, 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 10738547433, 2024-09-17T05:32:13.305 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 21873598464, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 751220514816, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 85, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 90852715, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 0, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 22 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 37, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:30:37:004420+0000", 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:32:13.306 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 4, 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "d", 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 5931 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:32:13.307 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 12, 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:31:32.550945+0000", 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": {} 2024-09-17T05:32:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:32:13.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb528073bc0 msgr2=0x7fb528076080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb528073bc0 0x7fb528076080 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb54c073f10 tx=0x7fb53c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c1f5c90 msgr2=0x7fb54c1f6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c1f5c90 0x7fb54c1f6110 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb5440985d0 tx=0x7fb5440a5e90 comp rx=0 tx=0).stop 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 shutdown_connections 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb528073bc0 0x7fb528076080 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb54c1f5c90 0x7fb54c1f6110 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54c15a1b0 0x7fb54c1f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 --2- 172.21.15.122:0/3790576942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb530005350 0x7fb54c0b73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 >> 172.21.15.122:0/3790576942 conn(0x7fb54c0d1c30 msgr2=0x7fb54c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 shutdown_connections 2024-09-17T05:32:13.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:13.304+0000 7fb551200640 1 -- 172.21.15.122:0/3790576942 wait complete. 2024-09-17T05:32:13.903 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:13.905+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:13.904 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:13.905+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:13.904 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:13.905+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:13.904 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:13.905+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:13.904 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:13.905+0000 7fc08d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:14.326 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:32:14.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e62000640 1 -- 172.21.15.122:0/408201066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 msgr2=0x7f3e40004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e62000640 1 --2- 172.21.15.122:0/408201066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 0x7f3e40004c90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c310 tx=0x7f3e4401cfc0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e5be00640 1 -- 172.21.15.122:0/408201066 reap_dead start 2024-09-17T05:32:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e62000640 1 -- 172.21.15.122:0/408201066 shutdown_connections 2024-09-17T05:32:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e62000640 1 -- 172.21.15.122:0/408201066 >> 172.21.15.122:0/408201066 conn(0x7f3e5c0cffd0 msgr2=0x7f3e5c0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e62000640 1 -- 172.21.15.122:0/408201066 shutdown_connections 2024-09-17T05:32:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e62000640 1 -- 172.21.15.122:0/408201066 wait complete. 2024-09-17T05:32:14.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e62000640 1 Processor -- start 2024-09-17T05:32:14.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.548+0000 7f3e62000640 1 -- start start 2024-09-17T05:32:14.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 0x7f3e5c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c15bd20 0x7f3e5c0b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c0b58f0 0x7f3e5c1f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c0ce740 con 0x7f3e40004810 2024-09-17T05:32:14.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c0ce5c0 con 0x7f3e5c15bd20 2024-09-17T05:32:14.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5c0ce8c0 con 0x7f3e5c0b58f0 2024-09-17T05:32:14.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 0x7f3e5c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 0x7f3e5c15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47004/0 (socket says 172.21.15.122:47004) 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 -- 172.21.15.122:0/1962478326 learned_addr learned my addr 172.21.15.122:0/1962478326 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e53e00640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c15bd20 0x7f3e5c0b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 -- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c0b58f0 msgr2=0x7f3e5c1f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c0b58f0 0x7f3e5c1f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 -- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c15bd20 msgr2=0x7f3e5c0b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c15bd20 0x7f3e5c0b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 -- 172.21.15.122:0/1962478326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c15c270 con 0x7f3e40004810 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e5b400640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 0x7f3e5c15d310 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c8d0 tx=0x7f3e4c00cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e53400640 1 -- 172.21.15.122:0/1962478326 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c004220 con 0x7f3e40004810 2024-09-17T05:32:14.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e53400640 1 -- 172.21.15.122:0/1962478326 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e4c0079b0 con 0x7f3e40004810 2024-09-17T05:32:14.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e53400640 1 -- 172.21.15.122:0/1962478326 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0044b0 con 0x7f3e40004810 2024-09-17T05:32:14.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c15c550 con 0x7f3e40004810 2024-09-17T05:32:14.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c15cb10 con 0x7f3e40004810 2024-09-17T05:32:14.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.552+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c0cb3d0 con 0x7f3e40004810 2024-09-17T05:32:14.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.556+0000 7f3e53400640 1 -- 172.21.15.122:0/1962478326 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 98101+0+0 (secure 0 0 0) 0x7f3e4c003600 con 0x7f3e40004810 2024-09-17T05:32:14.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.556+0000 7f3e53400640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e38073bc0 0x7f3e38076080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:14.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.556+0000 7f3e53400640 1 -- 172.21.15.122:0/1962478326 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f3e4c092f20 con 0x7f3e40004810 2024-09-17T05:32:14.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.556+0000 7f3e53e00640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e38073bc0 0x7f3e38076080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:14.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.560+0000 7f3e53e00640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e38073bc0 0x7f3e38076080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3e440ac000 tx=0x7f3e440af040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:14.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.560+0000 7f3e53400640 1 -- 172.21.15.122:0/1962478326 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3e4c014030 con 0x7f3e40004810 2024-09-17T05:32:14.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.896+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3e5c15e610 con 0x7f3e40004810 2024-09-17T05:32:14.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.896+0000 7f3e53400640 1 -- 172.21.15.122:0/1962478326 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2143 (secure 0 0 0) 0x7f3e4c05fbd0 con 0x7f3e40004810 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:32:14.898 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 455, 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:32:14.899 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 62, 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:14.900 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 2588, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 10738645797, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 21873778688, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 751220334592, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 2446300, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 90862720, 2024-09-17T05:32:14.901 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 0, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 24 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 37, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:30:37:004420+0000", 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 4, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:32:14.902 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "d", 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 5931 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:32:14.903 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 12, 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:31:32.550945+0000", 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0, 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:32:14.904 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:14.905 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:14.905 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e38073bc0 msgr2=0x7f3e38076080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e38073bc0 0x7f3e38076080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3e440ac000 tx=0x7f3e440af040 comp rx=0 tx=0).stop 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 msgr2=0x7f3e5c15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 0x7f3e5c15d310 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c8d0 tx=0x7f3e4c00cd90 comp rx=0 tx=0).stop 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 shutdown_connections 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e38073bc0 0x7f3e38076080 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c0b58f0 0x7f3e5c1f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c15bd20 0x7f3e5c0b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 --2- 172.21.15.122:0/1962478326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e40004810 0x7f3e5c15d310 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 >> 172.21.15.122:0/1962478326 conn(0x7f3e5c0cffd0 msgr2=0x7f3e5c0ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 shutdown_connections 2024-09-17T05:32:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:14.900+0000 7f3e62000640 1 -- 172.21.15.122:0/1962478326 wait complete. 2024-09-17T05:32:14.935 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-17T05:32:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.164+0000 7f590b600640 1 -- 172.21.15.122:0/975628916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ec003a10 msgr2=0x7f58ec003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.164+0000 7f590b600640 1 --2- 172.21.15.122:0/975628916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ec003a10 0x7f58ec003e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009b30 tx=0x7f58f801d1b0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.164+0000 7f5909600640 1 -- 172.21.15.122:0/975628916 reap_dead start 2024-09-17T05:32:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.164+0000 7f590b600640 1 -- 172.21.15.122:0/975628916 shutdown_connections 2024-09-17T05:32:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.164+0000 7f590b600640 1 -- 172.21.15.122:0/975628916 >> 172.21.15.122:0/975628916 conn(0x7f59040d1dd0 msgr2=0x7f59040d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:15.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.164+0000 7f590b600640 1 -- 172.21.15.122:0/975628916 shutdown_connections 2024-09-17T05:32:15.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.164+0000 7f590b600640 1 -- 172.21.15.122:0/975628916 wait complete. 2024-09-17T05:32:15.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f590b600640 1 Processor -- start 2024-09-17T05:32:15.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f590b600640 1 -- start start 2024-09-17T05:32:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f590b600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59041f5e40 0x7f59040cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f590b600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59040c8f10 0x7f59041652b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f590b600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59041657f0 0x7f59041f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f590b600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59040b13a0 con 0x7f59041f5e40 2024-09-17T05:32:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f590b600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59040b1220 con 0x7f59040c8f10 2024-09-17T05:32:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f590b600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5904167530 con 0x7f59041657f0 2024-09-17T05:32:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5908c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59041f5e40 0x7f59040cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5908c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59041f5e40 0x7f59040cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47020/0 (socket says 172.21.15.122:47020) 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5908c00640 1 -- 172.21.15.122:0/3471456956 learned_addr learned my addr 172.21.15.122:0/3471456956 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5909600640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59041657f0 0x7f59041f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5903e00640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59040c8f10 0x7f59041652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5908c00640 1 -- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59041657f0 msgr2=0x7f59041f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5908c00640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59041657f0 0x7f59041f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5908c00640 1 -- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59040c8f10 msgr2=0x7f59041652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5908c00640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59040c8f10 0x7f59041652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5908c00640 1 -- 172.21.15.122:0/3471456956 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59040cbfe0 con 0x7f59041f5e40 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.168+0000 7f5909600640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59041657f0 0x7f59041f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.172+0000 7f5908c00640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59041f5e40 0x7f59040cb6b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f58f801cc20 tx=0x7f58f8005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.172+0000 7f5900c00640 1 -- 172.21.15.122:0/3471456956 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80b1040 con 0x7f59041f5e40 2024-09-17T05:32:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.172+0000 7f5900c00640 1 -- 172.21.15.122:0/3471456956 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58f80bb070 con 0x7f59041f5e40 2024-09-17T05:32:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.172+0000 7f5900c00640 1 -- 172.21.15.122:0/3471456956 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8004290 con 0x7f59041f5e40 2024-09-17T05:32:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.172+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590415a880 con 0x7f59041f5e40 2024-09-17T05:32:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.172+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59040cbbf0 con 0x7f59041f5e40 2024-09-17T05:32:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.172+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59040d0b90 con 0x7f59041f5e40 2024-09-17T05:32:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.176+0000 7f5900c00640 1 -- 172.21.15.122:0/3471456956 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 98101+0+0 (secure 0 0 0) 0x7f58f80046d0 con 0x7f59041f5e40 2024-09-17T05:32:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.180+0000 7f5900c00640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58e4078000 0x7f58e407a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.180+0000 7f5900c00640 1 -- 172.21.15.122:0/3471456956 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f58f812d810 con 0x7f59041f5e40 2024-09-17T05:32:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.180+0000 7f5903e00640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58e4078000 0x7f58e407a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.180+0000 7f5903e00640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58e4078000 0x7f58e407a4c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f59040ce130 tx=0x7f58f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:15.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.184+0000 7f5900c00640 1 -- 172.21.15.122:0/3471456956 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f58f8016ec0 con 0x7f59041f5e40 2024-09-17T05:32:15.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.476+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f59041668f0 con 0x7f58e4078000 2024-09-17T05:32:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f5900c00640 1 -- 172.21.15.122:0/3471456956 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59041668f0 con 0x7f58e4078000 2024-09-17T05:32:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58e4078000 msgr2=0x7f58e407a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58e4078000 0x7f58e407a4c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f59040ce130 tx=0x7f58f0009040 comp rx=0 tx=0).stop 2024-09-17T05:32:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59041f5e40 msgr2=0x7f59040cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59041f5e40 0x7f59040cb6b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f58f801cc20 tx=0x7f58f8005600 comp rx=0 tx=0).stop 2024-09-17T05:32:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 shutdown_connections 2024-09-17T05:32:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58e4078000 0x7f58e407a4c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59041657f0 0x7f59041f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59040c8f10 0x7f59041652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 --2- 172.21.15.122:0/3471456956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59041f5e40 0x7f59040cb6b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 >> 172.21.15.122:0/3471456956 conn(0x7f59040d1dd0 msgr2=0x7f5904071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:15.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 shutdown_connections 2024-09-17T05:32:15.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.624+0000 7f590b600640 1 -- 172.21.15.122:0/3471456956 wait complete. 2024-09-17T05:32:15.650 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:32:15.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 -- 172.21.15.122:0/3480852250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d015c5e0 msgr2=0x7f09d006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 --2- 172.21.15.122:0/3480852250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d015c5e0 0x7f09d006ec50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009840 tx=0x7f09c401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 -- 172.21.15.122:0/3480852250 shutdown_connections 2024-09-17T05:32:15.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 --2- 172.21.15.122:0/3480852250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d0072c90 0x7f09d015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 --2- 172.21.15.122:0/3480852250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d015c5e0 0x7f09d006ec50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 -- 172.21.15.122:0/3480852250 >> 172.21.15.122:0/3480852250 conn(0x7f09d00b1c20 msgr2=0x7f09d00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:15.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 -- 172.21.15.122:0/3480852250 shutdown_connections 2024-09-17T05:32:15.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 -- 172.21.15.122:0/3480852250 wait complete. 2024-09-17T05:32:15.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 Processor -- start 2024-09-17T05:32:15.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.792+0000 7f09d8600640 1 -- start start 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09d0072c90 0x7f09d0143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d00744c0 0x7f09d01f9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d015c5e0 0x7f09d00b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0152bf0 con 0x7f09d0072c90 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0152a70 con 0x7f09d00744c0 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f09d0152d70 con 0x7f09d015c5e0 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09d0072c90 0x7f09d0143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09d0072c90 0x7f09d0143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47022/0 (socket says 172.21.15.122:47022) 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 -- 172.21.15.122:0/1781796764 learned_addr learned my addr 172.21.15.122:0/1781796764 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5200640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d00744c0 0x7f09d01f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d6600640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d015c5e0 0x7f09d00b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 -- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d015c5e0 msgr2=0x7f09d00b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d015c5e0 0x7f09d00b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 -- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d00744c0 msgr2=0x7f09d01f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d00744c0 0x7f09d01f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 -- 172.21.15.122:0/1781796764 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d01f9c00 con 0x7f09d0072c90 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d5c00640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09d0072c90 0x7f09d0143d00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009790 tx=0x7f09c40a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09cd600640 1 -- 172.21.15.122:0/1781796764 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c40b2040 con 0x7f09d0072c90 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09cd600640 1 -- 172.21.15.122:0/1781796764 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09c4005030 con 0x7f09d0072c90 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d01f9ee0 con 0x7f09d0072c90 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09cd600640 1 -- 172.21.15.122:0/1781796764 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c40044f0 con 0x7f09d0072c90 2024-09-17T05:32:15.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d00cce40 con 0x7f09d0072c90 2024-09-17T05:32:15.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d01fa120 con 0x7f09d0072c90 2024-09-17T05:32:15.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.796+0000 7f09cd600640 1 -- 172.21.15.122:0/1781796764 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 98254+0+0 (secure 0 0 0) 0x7f09c4005440 con 0x7f09d0072c90 2024-09-17T05:32:15.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.800+0000 7f09cd600640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f09b0073d40 0x7f09b0076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:15.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.800+0000 7f09cd600640 1 -- 172.21.15.122:0/1781796764 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f09c412cbb0 con 0x7f09d0072c90 2024-09-17T05:32:15.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.800+0000 7f09d5200640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f09b0073d40 0x7f09b0076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:15.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.800+0000 7f09d5200640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f09b0073d40 0x7f09b0076200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09d006ade0 tx=0x7f09bc0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:15.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:15.804+0000 7f09cd600640 1 -- 172.21.15.122:0/1781796764 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f09d01fa120 con 0x7f09d0072c90 2024-09-17T05:32:16.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.000+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f09d00cc970 con 0x7f09b0073d40 2024-09-17T05:32:16.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.000+0000 7f09cd600640 1 -- 172.21.15.122:0/1781796764 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f09d00cc970 con 0x7f09b0073d40 2024-09-17T05:32:16.002 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:32:16.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:16.002 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:32:16.002 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:16.002 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f09b0073d40 msgr2=0x7f09b0076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f09b0073d40 0x7f09b0076200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09d006ade0 tx=0x7f09bc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09d0072c90 msgr2=0x7f09d0143d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09d0072c90 0x7f09d0143d00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009790 tx=0x7f09c40a57a0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 shutdown_connections 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f09b0073d40 0x7f09b0076200 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d015c5e0 0x7f09d00b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d00744c0 0x7f09d01f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 --2- 172.21.15.122:0/1781796764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09d0072c90 0x7f09d0143d00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 >> 172.21.15.122:0/1781796764 conn(0x7f09d00b1c20 msgr2=0x7f09d009b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:16.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 shutdown_connections 2024-09-17T05:32:16.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.004+0000 7f09d8600640 1 -- 172.21.15.122:0/1781796764 wait complete. 2024-09-17T05:32:16.023 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:32:16.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.164+0000 7fc2da200640 1 -- 172.21.15.122:0/578122302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 msgr2=0x7fc2d40d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.164+0000 7fc2da200640 1 --2- 172.21.15.122:0/578122302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 0x7fc2d40d1a40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009900 tx=0x7fc2bc01c820 comp rx=0 tx=0).stop 2024-09-17T05:32:16.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.164+0000 7fc2da200640 1 -- 172.21.15.122:0/578122302 shutdown_connections 2024-09-17T05:32:16.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.164+0000 7fc2da200640 1 --2- 172.21.15.122:0/578122302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 0x7fc2d40d1a40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.164+0000 7fc2da200640 1 -- 172.21.15.122:0/578122302 >> 172.21.15.122:0/578122302 conn(0x7fc2d409aa10 msgr2=0x7fc2d409adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:16.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.164+0000 7fc2da200640 1 -- 172.21.15.122:0/578122302 shutdown_connections 2024-09-17T05:32:16.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.164+0000 7fc2da200640 1 -- 172.21.15.122:0/578122302 wait complete. 2024-09-17T05:32:16.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 Processor -- start 2024-09-17T05:32:16.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 -- start start 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2d40d1f80 0x7fc2d41f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2d415bc80 0x7fc2d4148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 0x7fc2d4153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d416aa30 con 0x7fc2d415c060 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d416a8b0 con 0x7fc2d40d1f80 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d416abb0 con 0x7fc2d415bc80 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 0x7fc2d4153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2d40d1f80 0x7fc2d41f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2d40d1f80 0x7fc2d41f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:45110/0 (socket says 172.21.15.122:45110) 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3400640 1 -- 172.21.15.122:0/2398644751 learned_addr learned my addr 172.21.15.122:0/2398644751 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:16.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2caa00640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2d415bc80 0x7fc2d4148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3e00640 1 -- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2d415bc80 msgr2=0x7fc2d4148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3e00640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2d415bc80 0x7fc2d4148b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3e00640 1 -- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2d40d1f80 msgr2=0x7fc2d41f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3e00640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2d40d1f80 0x7fc2d41f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3e00640 1 -- 172.21.15.122:0/2398644751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d4158d30 con 0x7fc2d415c060 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2caa00640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2d415bc80 0x7fc2d4148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2d3e00640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 0x7fc2d4153d80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c3c0 tx=0x7fc2cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2cbe00640 1 -- 172.21.15.122:0/2398644751 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc017070 con 0x7fc2d415c060 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2cbe00640 1 -- 172.21.15.122:0/2398644751 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2cc0053c0 con 0x7fc2d415c060 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d41fa310 con 0x7fc2d415c060 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2cbe00640 1 -- 172.21.15.122:0/2398644751 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc003860 con 0x7fc2d415c060 2024-09-17T05:32:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.168+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d41fa8d0 con 0x7fc2d415c060 2024-09-17T05:32:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.172+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d40d1a40 con 0x7fc2d415c060 2024-09-17T05:32:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.172+0000 7fc2cbe00640 1 -- 172.21.15.122:0/2398644751 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 98254+0+0 (secure 0 0 0) 0x7fc2cc01a070 con 0x7fc2d415c060 2024-09-17T05:32:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.172+0000 7fc2cbe00640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc2b0073d40 0x7fc2b0076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.172+0000 7fc2cbe00640 1 -- 172.21.15.122:0/2398644751 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fc2cc091ff0 con 0x7fc2d415c060 2024-09-17T05:32:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.172+0000 7fc2d3400640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc2b0073d40 0x7fc2b0076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.172+0000 7fc2d3400640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc2b0073d40 0x7fc2b0076200 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40012f0 tx=0x7fc2c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.176+0000 7fc2cbe00640 1 -- 172.21.15.122:0/2398644751 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc2cc05e290 con 0x7fc2d415c060 2024-09-17T05:32:16.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.368+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2d4070cb0 con 0x7fc2b0073d40 2024-09-17T05:32:16.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2cbe00640 1 -- 172.21.15.122:0/2398644751 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7fc2d4070cb0 con 0x7fc2b0073d40 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sv1" 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1clone1" 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:16.376 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:32:16.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc2b0073d40 msgr2=0x7fc2b0076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc2b0073d40 0x7fc2b0076200 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40012f0 tx=0x7fc2c4009040 comp rx=0 tx=0).stop 2024-09-17T05:32:16.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 msgr2=0x7fc2d4153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 0x7fc2d4153d80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c3c0 tx=0x7fc2cc00c880 comp rx=0 tx=0).stop 2024-09-17T05:32:16.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 shutdown_connections 2024-09-17T05:32:16.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc2b0073d40 0x7fc2b0076200 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2d415c060 0x7fc2d4153d80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2d415bc80 0x7fc2d4148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 --2- 172.21.15.122:0/2398644751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2d40d1f80 0x7fc2d41f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 >> 172.21.15.122:0/2398644751 conn(0x7fc2d409aa10 msgr2=0x7fc2d41545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:16.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 shutdown_connections 2024-09-17T05:32:16.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.376+0000 7fc2da200640 1 -- 172.21.15.122:0/2398644751 wait complete. 2024-09-17T05:32:16.397 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T05:32:16.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.540+0000 7f99e3800640 1 -- 172.21.15.122:0/2708142231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc0b5bf0 msgr2=0x7f99dc06f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.540+0000 7f99e3800640 1 --2- 172.21.15.122:0/2708142231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc0b5bf0 0x7f99dc06f690 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009bc0 tx=0x7f99d401cb10 comp rx=0 tx=0).stop 2024-09-17T05:32:16.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.540+0000 7f99e3800640 1 -- 172.21.15.122:0/2708142231 shutdown_connections 2024-09-17T05:32:16.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.540+0000 7f99e3800640 1 --2- 172.21.15.122:0/2708142231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc0b5bf0 0x7f99dc06f690 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.540+0000 7f99e3800640 1 -- 172.21.15.122:0/2708142231 >> 172.21.15.122:0/2708142231 conn(0x7f99dc06dd30 msgr2=0x7f99dc06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:16.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.540+0000 7f99e3800640 1 -- 172.21.15.122:0/2708142231 shutdown_connections 2024-09-17T05:32:16.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.540+0000 7f99e3800640 1 -- 172.21.15.122:0/2708142231 wait complete. 2024-09-17T05:32:16.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 Processor -- start 2024-09-17T05:32:16.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 -- start start 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99dc0b5bf0 0x7f99dc0add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc15b960 0x7f99dc15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99dc15e2f0 0x7f99dc15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc1472e0 con 0x7f99dc0b5bf0 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc147160 con 0x7f99dc15b960 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc147460 con 0x7f99dc15e2f0 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e0e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99dc0b5bf0 0x7f99dc0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99dbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc15b960 0x7f99dc15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e1800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99dc15e2f0 0x7f99dc15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99dbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc15b960 0x7f99dc15a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:45126/0 (socket says 172.21.15.122:45126) 2024-09-17T05:32:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e0e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99dc0b5bf0 0x7f99dc0add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47036/0 (socket says 172.21.15.122:47036) 2024-09-17T05:32:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99dbe00640 1 -- 172.21.15.122:0/1755670970 learned_addr learned my addr 172.21.15.122:0/1755670970 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e1800640 1 -- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc15b960 msgr2=0x7f99dc15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e1800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc15b960 0x7f99dc15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e1800640 1 -- 172.21.15.122:0/1755670970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99dc0b5bf0 msgr2=0x7f99dc0add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e1800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99dc0b5bf0 0x7f99dc0add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e1800640 1 -- 172.21.15.122:0/1755670970 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc15aba0 con 0x7f99dc15e2f0 2024-09-17T05:32:16.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e1800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99dc15e2f0 0x7f99dc15f950 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0b6140 tx=0x7f99d4009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99d8c00640 1 -- 172.21.15.122:0/1755670970 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40bb070 con 0x7f99dc15e2f0 2024-09-17T05:32:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99d8c00640 1 -- 172.21.15.122:0/1755670970 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99d4004d40 con 0x7f99dc15e2f0 2024-09-17T05:32:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99d8c00640 1 -- 172.21.15.122:0/1755670970 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40b2040 con 0x7f99dc15e2f0 2024-09-17T05:32:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc160be0 con 0x7f99dc15e2f0 2024-09-17T05:32:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.544+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc160620 con 0x7f99dc15e2f0 2024-09-17T05:32:16.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.548+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc147950 con 0x7f99dc15e2f0 2024-09-17T05:32:16.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.548+0000 7f99d8c00640 1 -- 172.21.15.122:0/1755670970 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 25) ==== 98254+0+0 (secure 0 0 0) 0x7f99d40be070 con 0x7f99dc15e2f0 2024-09-17T05:32:16.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.548+0000 7f99d8c00640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99bc073d40 0x7f99bc076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.548+0000 7f99d8c00640 1 -- 172.21.15.122:0/1755670970 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f99d412cfa0 con 0x7f99dc15e2f0 2024-09-17T05:32:16.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.548+0000 7f99e0e00640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99bc073d40 0x7f99bc076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.548+0000 7f99e0e00640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99bc073d40 0x7f99bc076200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f99d0001820 tx=0x7f99d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:16.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.552+0000 7f99d8c00640 1 -- 172.21.15.122:0/1755670970 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f99d4016790 con 0x7f99dc15e2f0 2024-09-17T05:32:16.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.748+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99dc13f4c0 con 0x7f99bc073d40 2024-09-17T05:32:16.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.760+0000 7f99d8c00640 1 -- 172.21.15.122:0/1755670970 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f99d000f730 con 0x7f99bc073d40 2024-09-17T05:32:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:16.763 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:32:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1" 2024-09-17T05:32:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:16.763 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99bc073d40 msgr2=0x7f99bc076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99bc073d40 0x7f99bc076200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f99d0001820 tx=0x7f99d0009040 comp rx=0 tx=0).stop 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99dc15e2f0 msgr2=0x7f99dc15f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99dc15e2f0 0x7f99dc15f950 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0b6140 tx=0x7f99d4009280 comp rx=0 tx=0).stop 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 shutdown_connections 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99bc073d40 0x7f99bc076200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99dc15e2f0 0x7f99dc15f950 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99dc15b960 0x7f99dc15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 --2- 172.21.15.122:0/1755670970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99dc0b5bf0 0x7f99dc0add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 >> 172.21.15.122:0/1755670970 conn(0x7f99dc06dd30 msgr2=0x7f99dc070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 shutdown_connections 2024-09-17T05:32:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.764+0000 7f99e3800640 1 -- 172.21.15.122:0/1755670970 wait complete. 2024-09-17T05:32:16.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T05:32:16.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.928+0000 7f9a81800640 1 -- 172.21.15.122:0/344779551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 msgr2=0x7f9a7c0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.928+0000 7f9a81800640 1 --2- 172.21.15.122:0/344779551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 0x7f9a7c0affc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009c80 tx=0x7f9a7401cda0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.928+0000 7f9a81800640 1 -- 172.21.15.122:0/344779551 shutdown_connections 2024-09-17T05:32:16.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.928+0000 7f9a81800640 1 --2- 172.21.15.122:0/344779551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 0x7f9a7c0affc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.928+0000 7f9a81800640 1 --2- 172.21.15.122:0/344779551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a7c0b1650 0x7f9a7c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.928+0000 7f9a81800640 1 -- 172.21.15.122:0/344779551 >> 172.21.15.122:0/344779551 conn(0x7f9a7c0d1dd0 msgr2=0x7f9a7c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:16.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.928+0000 7f9a81800640 1 -- 172.21.15.122:0/344779551 shutdown_connections 2024-09-17T05:32:16.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.928+0000 7f9a81800640 1 -- 172.21.15.122:0/344779551 wait complete. 2024-09-17T05:32:16.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 Processor -- start 2024-09-17T05:32:16.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 -- start start 2024-09-17T05:32:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 0x7f9a7c165970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a7c0b1650 0x7f9a7c0b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a7c1605d0 0x7f9a7c15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c169540 con 0x7f9a7c0afb10 2024-09-17T05:32:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7c0b1430 con 0x7f9a7c1605d0 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7c1696c0 con 0x7f9a7c0b1650 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 0x7f9a7c165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 0x7f9a7c165970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47056/0 (socket says 172.21.15.122:47056) 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 -- 172.21.15.122:0/2595026225 learned_addr learned my addr 172.21.15.122:0/2595026225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7b400640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a7c1605d0 0x7f9a7c15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7a000640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a7c0b1650 0x7f9a7c0b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 -- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a7c0b1650 msgr2=0x7f9a7c0b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a7c0b1650 0x7f9a7c0b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 -- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a7c1605d0 msgr2=0x7f9a7c15bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a7c1605d0 0x7f9a7c15bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 -- 172.21.15.122:0/2595026225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7c0b4c20 con 0x7f9a7c0afb10 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a7aa00640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 0x7f9a7c165970 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00c3c0 tx=0x7f9a6c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:16.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a72a00640 1 -- 172.21.15.122:0/2595026225 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c017070 con 0x7f9a7c0afb10 2024-09-17T05:32:16.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a72a00640 1 -- 172.21.15.122:0/2595026225 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a6c0053c0 con 0x7f9a7c0afb10 2024-09-17T05:32:16.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a72a00640 1 -- 172.21.15.122:0/2595026225 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c003860 con 0x7f9a7c0afb10 2024-09-17T05:32:16.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7c164ed0 con 0x7f9a7c0afb10 2024-09-17T05:32:16.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7c0b51a0 con 0x7f9a7c0afb10 2024-09-17T05:32:16.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.932+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7c0affc0 con 0x7f9a7c0afb10 2024-09-17T05:32:16.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.936+0000 7f9a72a00640 1 -- 172.21.15.122:0/2595026225 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 98254+0+0 (secure 0 0 0) 0x7f9a6c01a070 con 0x7f9a7c0afb10 2024-09-17T05:32:16.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.936+0000 7f9a72a00640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a5c073d40 0x7f9a5c076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:16.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.936+0000 7f9a72a00640 1 -- 172.21.15.122:0/2595026225 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f9a6c092090 con 0x7f9a7c0afb10 2024-09-17T05:32:16.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.936+0000 7f9a7a000640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a5c073d40 0x7f9a5c076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:16.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.936+0000 7f9a7a000640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a5c073d40 0x7f9a5c076200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c166cd0 tx=0x7f9a6800b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:16.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:16.940+0000 7f9a72a00640 1 -- 172.21.15.122:0/2595026225 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9a6c05e2f0 con 0x7f9a7c0afb10 2024-09-17T05:32:17.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.144+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a7c0d1080 con 0x7f9a5c073d40 2024-09-17T05:32:17.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.168+0000 7f9a72a00640 1 -- 172.21.15.122:0/2595026225 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9a7c0d1080 con 0x7f9a5c073d40 2024-09-17T05:32:17.170 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:17.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a5c073d40 msgr2=0x7f9a5c076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a5c073d40 0x7f9a5c076200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c166cd0 tx=0x7f9a6800b390 comp rx=0 tx=0).stop 2024-09-17T05:32:17.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 msgr2=0x7f9a7c165970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 0x7f9a7c165970 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00c3c0 tx=0x7f9a6c00c880 comp rx=0 tx=0).stop 2024-09-17T05:32:17.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 shutdown_connections 2024-09-17T05:32:17.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a5c073d40 0x7f9a5c076200 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a7c1605d0 0x7f9a7c15bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a7c0b1650 0x7f9a7c0b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 --2- 172.21.15.122:0/2595026225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a7c0afb10 0x7f9a7c165970 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 >> 172.21.15.122:0/2595026225 conn(0x7f9a7c0d1dd0 msgr2=0x7f9a7c1590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 shutdown_connections 2024-09-17T05:32:17.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.172+0000 7f9a81800640 1 -- 172.21.15.122:0/2595026225 wait complete. 2024-09-17T05:32:17.191 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 -- 172.21.15.122:0/3109994755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cf80b2a00 msgr2=0x7f7cf80b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 --2- 172.21.15.122:0/3109994755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cf80b2a00 0x7f7cf80b39e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009840 tx=0x7f7ce001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 -- 172.21.15.122:0/3109994755 shutdown_connections 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 --2- 172.21.15.122:0/3109994755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cf80b2a00 0x7f7cf80b39e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 --2- 172.21.15.122:0/3109994755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cf815f340 0x7f7cf80b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 -- 172.21.15.122:0/3109994755 >> 172.21.15.122:0/3109994755 conn(0x7f7cf80d1e00 msgr2=0x7f7cf80d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 -- 172.21.15.122:0/3109994755 shutdown_connections 2024-09-17T05:32:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 -- 172.21.15.122:0/3109994755 wait complete. 2024-09-17T05:32:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 Processor -- start 2024-09-17T05:32:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.336+0000 7f7cfec00640 1 -- start start 2024-09-17T05:32:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cf80b2a00 0x7f7cf814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cf80ccf10 0x7f7cf80c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cf815f340 0x7f7cf815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8168f10 con 0x7f7cf80b2a00 2024-09-17T05:32:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8168d90 con 0x7f7cf80ccf10 2024-09-17T05:32:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf8169090 con 0x7f7cf815f340 2024-09-17T05:32:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cf80b2a00 0x7f7cf814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cf80b2a00 0x7f7cf814a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47068/0 (socket says 172.21.15.122:47068) 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 -- 172.21.15.122:0/2787785789 learned_addr learned my addr 172.21.15.122:0/2787785789 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 -- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cf815f340 msgr2=0x7f7cf815cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cf815f340 0x7f7cf815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 -- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cf80ccf10 msgr2=0x7f7cf80c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cf80ccf10 0x7f7cf80c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 -- 172.21.15.122:0/2787785789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf80c5f00 con 0x7f7cf80b2a00 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf7e00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cf80b2a00 0x7f7cf814a3a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800ebf0 tx=0x7f7ce800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf4c00640 1 -- 172.21.15.122:0/2787785789 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8010430 con 0x7f7cf80b2a00 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf4c00640 1 -- 172.21.15.122:0/2787785789 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ce80106c0 con 0x7f7cf80b2a00 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf80b4e60 con 0x7f7cf80b2a00 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf4c00640 1 -- 172.21.15.122:0/2787785789 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8010af0 con 0x7f7cf80b2a00 2024-09-17T05:32:17.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf80b5420 con 0x7f7cf80b2a00 2024-09-17T05:32:17.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf80cd3a0 con 0x7f7cf80b2a00 2024-09-17T05:32:17.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.340+0000 7f7cf4c00640 1 -- 172.21.15.122:0/2787785789 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 98254+0+0 (secure 0 0 0) 0x7f7ce8010c90 con 0x7f7cf80b2a00 2024-09-17T05:32:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.344+0000 7f7cf4c00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cd4073d40 0x7f7cd4076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.344+0000 7f7cf4c00640 1 -- 172.21.15.122:0/2787785789 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f7ce8093b80 con 0x7f7cf80b2a00 2024-09-17T05:32:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.344+0000 7f7cef400640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cd4073d40 0x7f7cd4076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.344+0000 7f7cef400640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cd4073d40 0x7f7cd4076200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00af000 tx=0x7f7ce0019590 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.348+0000 7f7cf4c00640 1 -- 172.21.15.122:0/2787785789 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7ce80603e0 con 0x7f7cf80b2a00 2024-09-17T05:32:17.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.536+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f7cf815a8e0 con 0x7f7cd4073d40 2024-09-17T05:32:17.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cf4c00640 1 -- 172.21.15.122:0/2787785789 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7ce0009460 con 0x7f7cd4073d40 2024-09-17T05:32:17.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cd4073d40 msgr2=0x7f7cd4076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cd4073d40 0x7f7cd4076200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00af000 tx=0x7f7ce0019590 comp rx=0 tx=0).stop 2024-09-17T05:32:17.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cf80b2a00 msgr2=0x7f7cf814a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cf80b2a00 0x7f7cf814a3a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800ebf0 tx=0x7f7ce800c6a0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 shutdown_connections 2024-09-17T05:32:17.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cd4073d40 0x7f7cd4076200 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cf815f340 0x7f7cf815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cf80ccf10 0x7f7cf80c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 --2- 172.21.15.122:0/2787785789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cf80b2a00 0x7f7cf814a3a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 >> 172.21.15.122:0/2787785789 conn(0x7f7cf80d1e00 msgr2=0x7f7cf8072070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 shutdown_connections 2024-09-17T05:32:17.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.556+0000 7f7cfec00640 1 -- 172.21.15.122:0/2787785789 wait complete. 2024-09-17T05:32:17.560 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.561+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.560 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.561+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.560 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.561+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.560 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.561+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.560 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.561+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.574 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.577+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.575 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.577+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.575 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.577+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.575 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.577+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.575 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:17.577+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:17.578 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-17T05:32:17.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 -- 172.21.15.122:0/3134044044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 msgr2=0x7fbdc40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 --2- 172.21.15.122:0/3134044044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 0x7fbdc40ce700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009900 tx=0x7fbdb801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 -- 172.21.15.122:0/3134044044 shutdown_connections 2024-09-17T05:32:17.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 --2- 172.21.15.122:0/3134044044 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdc40b0b60 0x7fbdc40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 --2- 172.21.15.122:0/3134044044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 0x7fbdc40ce700 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 -- 172.21.15.122:0/3134044044 >> 172.21.15.122:0/3134044044 conn(0x7fbdc40d1da0 msgr2=0x7fbdc40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:17.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 -- 172.21.15.122:0/3134044044 shutdown_connections 2024-09-17T05:32:17.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 -- 172.21.15.122:0/3134044044 wait complete. 2024-09-17T05:32:17.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 Processor -- start 2024-09-17T05:32:17.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.724+0000 7fbdcc600640 1 -- start start 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdc40b0b60 0x7fbdc4158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 0x7fbdc41542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc415c1f0 0x7fbdc41547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4169040 con 0x7fbdc415c1f0 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc4168ec0 con 0x7fbdc415be10 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbdc41691c0 con 0x7fbdc40b0b60 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdca600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc415c1f0 0x7fbdc41547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 0x7fbdc41542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdc40b0b60 0x7fbdc4158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 0x7fbdc41542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:45192/0 (socket says 172.21.15.122:45192) 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 -- 172.21.15.122:0/228520091 learned_addr learned my addr 172.21.15.122:0/228520091 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdca600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc415c1f0 0x7fbdc41547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47072/0 (socket says 172.21.15.122:47072) 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 -- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdc40b0b60 msgr2=0x7fbdc4158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdc40b0b60 0x7fbdc4158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 -- 172.21.15.122:0/228520091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc415c1f0 msgr2=0x7fbdc41547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc415c1f0 0x7fbdc41547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 -- 172.21.15.122:0/228520091 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc4158880 con 0x7fbdc415be10 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9c00640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdc40b0b60 0x7fbdc4158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc9200640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 0x7fbdc41542b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c3f0 tx=0x7fbdb000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc1600640 1 -- 172.21.15.122:0/228520091 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0017070 con 0x7fbdc415be10 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc1600640 1 -- 172.21.15.122:0/228520091 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdb00053c0 con 0x7fbdc415be10 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc1600640 1 -- 172.21.15.122:0/228520091 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0003860 con 0x7fbdc415be10 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc414ff90 con 0x7fbdc415be10 2024-09-17T05:32:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc4150230 con 0x7fbdc415be10 2024-09-17T05:32:17.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdc40ccbd0 con 0x7fbdc415be10 2024-09-17T05:32:17.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.728+0000 7fbdc1600640 1 -- 172.21.15.122:0/228520091 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 98254+0+0 (secure 0 0 0) 0x7fbdb001a070 con 0x7fbdc415be10 2024-09-17T05:32:17.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.732+0000 7fbdc1600640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbda4073d40 0x7fbda4076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:17.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.732+0000 7fbdc1600640 1 -- 172.21.15.122:0/228520091 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fbdb00920c0 con 0x7fbdc415be10 2024-09-17T05:32:17.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.732+0000 7fbdc9c00640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbda4073d40 0x7fbda4076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:17.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.732+0000 7fbdc9c00640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbda4073d40 0x7fbda4076200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80af000 tx=0x7fbdb801b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:17.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.736+0000 7fbdc1600640 1 -- 172.21.15.122:0/228520091 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbdb005e320 con 0x7fbdc415be10 2024-09-17T05:32:17.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:17.928+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fbdc415b400 con 0x7fbda4073d40 2024-09-17T05:32:18.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:18.668+0000 7fbdc1600640 1 -- 172.21.15.122:0/228520091 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fbdb005a590 con 0x7fbdc415be10 2024-09-17T05:32:19.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdc1600640 1 -- 172.21.15.122:0/228520091 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbdc415b400 con 0x7fbda4073d40 2024-09-17T05:32:19.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbda4073d40 msgr2=0x7fbda4076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbda4073d40 0x7fbda4076200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80af000 tx=0x7fbdb801b120 comp rx=0 tx=0).stop 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 msgr2=0x7fbdc41542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 0x7fbdc41542b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c3f0 tx=0x7fbdb000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 shutdown_connections 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbda4073d40 0x7fbda4076200 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbdc415c1f0 0x7fbdc41547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdc415be10 0x7fbdc41542b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 --2- 172.21.15.122:0/228520091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdc40b0b60 0x7fbdc4158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 >> 172.21.15.122:0/228520091 conn(0x7fbdc40d1da0 msgr2=0x7fbdc41579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 shutdown_connections 2024-09-17T05:32:19.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.392+0000 7fbdcc600640 1 -- 172.21.15.122:0/228520091 wait complete. 2024-09-17T05:32:19.415 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:32:19.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:32:19.470 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:32:19.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:32:19.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:32:19.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.660+0000 7f78fec00640 1 -- 172.21.15.122:0/2019600741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc0051d0 msgr2=0x7f78dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.660+0000 7f78fec00640 1 --2- 172.21.15.122:0/2019600741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78dc0051d0 0x7f78dc00b9d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c640 tx=0x7f78f001d260 comp rx=0 tx=0).stop 2024-09-17T05:32:19.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.660+0000 7f78fcc00640 1 -- 172.21.15.122:0/2019600741 reap_dead start 2024-09-17T05:32:19.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- 172.21.15.122:0/2019600741 shutdown_connections 2024-09-17T05:32:19.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- 172.21.15.122:0/2019600741 >> 172.21.15.122:0/2019600741 conn(0x7f78f80d1dd0 msgr2=0x7f78f80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:19.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- 172.21.15.122:0/2019600741 shutdown_connections 2024-09-17T05:32:19.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- 172.21.15.122:0/2019600741 wait complete. 2024-09-17T05:32:19.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 Processor -- start 2024-09-17T05:32:19.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- start start 2024-09-17T05:32:19.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78dc0051d0 0x7f78f80b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f81f8b30 0x7f78f806df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78f8153df0 0x7f78f8154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78f80b0cd0 con 0x7f78f81f8b30 2024-09-17T05:32:19.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f78f80b06d0 con 0x7f78f8153df0 2024-09-17T05:32:19.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f78f80b0e50 con 0x7f78dc0051d0 2024-09-17T05:32:19.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f81f8b30 0x7f78f806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f81f8b30 0x7f78f806df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47092/0 (socket says 172.21.15.122:47092) 2024-09-17T05:32:19.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 -- 172.21.15.122:0/3993972059 learned_addr learned my addr 172.21.15.122:0/3993972059 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:19.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78f7e00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78dc0051d0 0x7f78f80b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fcc00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78f8153df0 0x7f78f8154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 -- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78dc0051d0 msgr2=0x7f78f80b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78dc0051d0 0x7f78f80b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 -- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78f8153df0 msgr2=0x7f78f8154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78f8153df0 0x7f78f8154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 -- 172.21.15.122:0/3993972059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f81547b0 con 0x7f78f81f8b30 2024-09-17T05:32:19.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fcc00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78f8153df0 0x7f78f8154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:19.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78ef400640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f81f8b30 0x7f78f806df90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f78e0003f30 tx=0x7f78e0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:19.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78f4c00640 1 -- 172.21.15.122:0/3993972059 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0011640 con 0x7f78f81f8b30 2024-09-17T05:32:19.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78f4c00640 1 -- 172.21.15.122:0/3993972059 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78e0016020 con 0x7f78f81f8b30 2024-09-17T05:32:19.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f81f5a50 con 0x7f78f81f8b30 2024-09-17T05:32:19.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.664+0000 7f78f4c00640 1 -- 172.21.15.122:0/3993972059 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0011a60 con 0x7f78f81f8b30 2024-09-17T05:32:19.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.668+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f81f51b0 con 0x7f78f81f8b30 2024-09-17T05:32:19.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.668+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f80affc0 con 0x7f78f81f8b30 2024-09-17T05:32:19.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.668+0000 7f78f4c00640 1 -- 172.21.15.122:0/3993972059 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f78e0011c00 con 0x7f78f81f8b30 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.668+0000 7f78f4c00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78d4073e60 0x7f78d4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.668+0000 7f78f4c00640 1 -- 172.21.15.122:0/3993972059 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f78e0096700 con 0x7f78f81f8b30 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.668+0000 7f78f7e00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78d4073e60 0x7f78d4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.672+0000 7f78f7e00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78d4073e60 0x7f78d4076320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f78f8073810 tx=0x7f78e80091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:19.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.676+0000 7f78f4c00640 1 -- 172.21.15.122:0/3993972059 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f78e0020020 con 0x7f78f81f8b30 2024-09-17T05:32:19.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f78f80d1080 con 0x7f78f81f8b30 2024-09-17T05:32:19.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78f4c00640 1 -- 172.21.15.122:0/3993972059 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7f78e00603b0 con 0x7f78f81f8b30 2024-09-17T05:32:19.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78d4073e60 msgr2=0x7f78d4076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78d4073e60 0x7f78d4076320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f78f8073810 tx=0x7f78e80091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f81f8b30 msgr2=0x7f78f806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f81f8b30 0x7f78f806df90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f78e0003f30 tx=0x7f78e0001640 comp rx=0 tx=0).stop 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 shutdown_connections 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78d4073e60 0x7f78d4076320 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78f8153df0 0x7f78f8154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78f81f8b30 0x7f78f806df90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 --2- 172.21.15.122:0/3993972059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78dc0051d0 0x7f78f80b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.840+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 >> 172.21.15.122:0/3993972059 conn(0x7f78f80d1dd0 msgr2=0x7f78f8070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.844+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 shutdown_connections 2024-09-17T05:32:19.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:19.844+0000 7f78fec00640 1 -- 172.21.15.122:0/3993972059 wait complete. 2024-09-17T05:32:19.863 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:32:19.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:32:19.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:32:20.045 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:32:20.045 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:32:20.045 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:32:20.045 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:20.054 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:32:20.054 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:20.112 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:32:20.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.384+0000 7ff172800640 1 -- 172.21.15.122:0/84579525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff16c0cce50 msgr2=0x7ff16c165fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.384+0000 7ff172800640 1 --2- 172.21.15.122:0/84579525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff16c0cce50 0x7ff16c165fe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff1540098d0 tx=0x7ff15401e9e0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.384+0000 7ff172800640 1 -- 172.21.15.122:0/84579525 shutdown_connections 2024-09-17T05:32:20.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.384+0000 7ff172800640 1 --2- 172.21.15.122:0/84579525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff16c0cce50 0x7ff16c165fe0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.384+0000 7ff172800640 1 -- 172.21.15.122:0/84579525 >> 172.21.15.122:0/84579525 conn(0x7ff16c0b18f0 msgr2=0x7ff16c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:20.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.384+0000 7ff172800640 1 -- 172.21.15.122:0/84579525 shutdown_connections 2024-09-17T05:32:20.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.384+0000 7ff172800640 1 -- 172.21.15.122:0/84579525 wait complete. 2024-09-17T05:32:20.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.388+0000 7ff172800640 1 Processor -- start 2024-09-17T05:32:20.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.388+0000 7ff172800640 1 -- start start 2024-09-17T05:32:20.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff16c06ffd0 0x7ff16c09d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:20.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff16c070960 0x7ff16c149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:20.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff16c0cce50 0x7ff16c09c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:20.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c0b4b10 con 0x7ff16c070960 2024-09-17T05:32:20.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c0b4990 con 0x7ff16c0cce50 2024-09-17T05:32:20.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff16c168530 con 0x7ff16c06ffd0 2024-09-17T05:32:20.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff16c070960 0x7ff16c149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:20.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff16c070960 0x7ff16c149ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47094/0 (socket says 172.21.15.122:47094) 2024-09-17T05:32:20.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 -- 172.21.15.122:0/1177274445 learned_addr learned my addr 172.21.15.122:0/1177274445 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:20.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 -- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff16c06ffd0 msgr2=0x7ff16c09d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff16c06ffd0 0x7ff16c09d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 -- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff16c0cce50 msgr2=0x7ff16c09c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff16c0cce50 0x7ff16c09c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 -- 172.21.15.122:0/1177274445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c0cebe0 con 0x7ff16c070960 2024-09-17T05:32:20.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff16aa00640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff16c070960 0x7ff16c149ae0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00cc10 tx=0x7ff15c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:20.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff163400640 1 -- 172.21.15.122:0/1177274445 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c016020 con 0x7ff16c070960 2024-09-17T05:32:20.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff163400640 1 -- 172.21.15.122:0/1177274445 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff15c00ab60 con 0x7ff16c070960 2024-09-17T05:32:20.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff163400640 1 -- 172.21.15.122:0/1177274445 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c012420 con 0x7ff16c070960 2024-09-17T05:32:20.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16c0af310 con 0x7ff16c070960 2024-09-17T05:32:20.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16c0b1330 con 0x7ff16c070960 2024-09-17T05:32:20.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.392+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16c069410 con 0x7ff16c070960 2024-09-17T05:32:20.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.396+0000 7ff163400640 1 -- 172.21.15.122:0/1177274445 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7ff15c019020 con 0x7ff16c070960 2024-09-17T05:32:20.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.396+0000 7ff163400640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff148073e60 0x7ff148076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:20.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.396+0000 7ff163400640 1 -- 172.21.15.122:0/1177274445 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7ff15c0931e0 con 0x7ff16c070960 2024-09-17T05:32:20.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.396+0000 7ff16b400640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff148073e60 0x7ff148076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:20.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.400+0000 7ff16b400640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff148073e60 0x7ff148076320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff164007fd0 tx=0x7ff1640078c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:20.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.404+0000 7ff163400640 1 -- 172.21.15.122:0/1177274445 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff15c020020 con 0x7ff16c070960 2024-09-17T05:32:20.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.748+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff16c0cc740 con 0x7ff16c070960 2024-09-17T05:32:20.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.748+0000 7ff163400640 1 -- 172.21.15.122:0/1177274445 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7ff15c05fa60 con 0x7ff16c070960 2024-09-17T05:32:20.750 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:32:20.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff148073e60 msgr2=0x7ff148076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff148073e60 0x7ff148076320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff164007fd0 tx=0x7ff1640078c0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff16c070960 msgr2=0x7ff16c149ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff16c070960 0x7ff16c149ae0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00cc10 tx=0x7ff15c00cd00 comp rx=0 tx=0).stop 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 shutdown_connections 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff148073e60 0x7ff148076320 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff16c0cce50 0x7ff16c09c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff16c070960 0x7ff16c149ae0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 --2- 172.21.15.122:0/1177274445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff16c06ffd0 0x7ff16c09d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 >> 172.21.15.122:0/1177274445 conn(0x7ff16c0b18f0 msgr2=0x7ff16c0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 shutdown_connections 2024-09-17T05:32:20.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:20.752+0000 7ff172800640 1 -- 172.21.15.122:0/1177274445 wait complete. 2024-09-17T05:32:20.784 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:32:20.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:20.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:20.852 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:32:20.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:20.852 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:32:20.868 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:32:20.868 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:32:20.868 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:20.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:32:20.891 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:32:20.891 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:32:20.891 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:20.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:20.923 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:32:20.923 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:32:20.924 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:20.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:32:20.954 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:21.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.192+0000 7fcc12e00640 1 -- 172.21.15.122:0/2052359388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc0c06f930 msgr2=0x7fcc0c06fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.192+0000 7fcc12e00640 1 --2- 172.21.15.122:0/2052359388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc0c06f930 0x7fcc0c06fd70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8005fd0 tx=0x7fcbf801c820 comp rx=0 tx=0).stop 2024-09-17T05:32:21.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.192+0000 7fcc12e00640 1 -- 172.21.15.122:0/2052359388 shutdown_connections 2024-09-17T05:32:21.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.192+0000 7fcc12e00640 1 --2- 172.21.15.122:0/2052359388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc0c06f930 0x7fcc0c06fd70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.192+0000 7fcc12e00640 1 -- 172.21.15.122:0/2052359388 >> 172.21.15.122:0/2052359388 conn(0x7fcc0c0b2240 msgr2=0x7fcc0c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.192+0000 7fcc12e00640 1 -- 172.21.15.122:0/2052359388 shutdown_connections 2024-09-17T05:32:21.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.192+0000 7fcc12e00640 1 -- 172.21.15.122:0/2052359388 wait complete. 2024-09-17T05:32:21.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc12e00640 1 Processor -- start 2024-09-17T05:32:21.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc12e00640 1 -- start start 2024-09-17T05:32:21.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc12e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc0c06cc20 0x7fcc0c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc12e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc0c06f930 0x7fcc0c14e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc12e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc0c15f160 0x7fcc0c14e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc12e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c16ca50 con 0x7fcc0c15f160 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc12e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c16c8d0 con 0x7fcc0c06f930 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc12e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c16cbd0 con 0x7fcc0c06cc20 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc0c15f160 0x7fcc0c14e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc0c15f160 0x7fcc0c14e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47108/0 (socket says 172.21.15.122:47108) 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 -- 172.21.15.122:0/2402026102 learned_addr learned my addr 172.21.15.122:0/2402026102 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc0be00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc0c06cc20 0x7fcc0c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc0b400640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc0c06f930 0x7fcc0c14e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 -- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc0c06cc20 msgr2=0x7fcc0c154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc0c06cc20 0x7fcc0c154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 -- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc0c06f930 msgr2=0x7fcc0c14e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc0c06f930 0x7fcc0c14e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 -- 172.21.15.122:0/2402026102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c14eb50 con 0x7fcc0c15f160 2024-09-17T05:32:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.196+0000 7fcc10e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc0c15f160 0x7fcc0c14e560 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400cbe0 tx=0x7fcc0400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.200+0000 7fcc03e00640 1 -- 172.21.15.122:0/2402026102 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04016020 con 0x7fcc0c15f160 2024-09-17T05:32:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.200+0000 7fcc03e00640 1 -- 172.21.15.122:0/2402026102 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc0400ab40 con 0x7fcc0c15f160 2024-09-17T05:32:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.200+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c1524a0 con 0x7fcc0c15f160 2024-09-17T05:32:21.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.200+0000 7fcc03e00640 1 -- 172.21.15.122:0/2402026102 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04012420 con 0x7fcc0c15f160 2024-09-17T05:32:21.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.200+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c152a60 con 0x7fcc0c15f160 2024-09-17T05:32:21.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.200+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c0b0030 con 0x7fcc0c15f160 2024-09-17T05:32:21.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.200+0000 7fcc03e00640 1 -- 172.21.15.122:0/2402026102 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fcc04019020 con 0x7fcc0c15f160 2024-09-17T05:32:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.204+0000 7fcc03e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcbec073e60 0x7fcbec076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.204+0000 7fcc03e00640 1 -- 172.21.15.122:0/2402026102 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7fcc04092c80 con 0x7fcc0c15f160 2024-09-17T05:32:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.204+0000 7fcc0be00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcbec073e60 0x7fcbec076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.204+0000 7fcc0be00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcbec073e60 0x7fcbec076320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc007fd0 tx=0x7fcbfc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.208+0000 7fcc03e00640 1 -- 172.21.15.122:0/2402026102 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcc04020030 con 0x7fcc0c15f160 2024-09-17T05:32:21.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc0c06fd70 con 0x7fcc0c15f160 2024-09-17T05:32:21.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc03e00640 1 -- 172.21.15.122:0/2402026102 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5232 (secure 0 0 0) 0x7fcc0405ef10 con 0x7fcc0c15f160 2024-09-17T05:32:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:21.509 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":37,"btime":"2024-09-17T05:30:37:004420+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110418608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110418608},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110418608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1187468185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1187468185},{"type":"v1","addr":"172.21.15.122:6835","nonce":1187468185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3667127301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3667127301},{"type":"v1","addr":"172.21.15.122:6834","nonce":3667127301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:30:24.330680+0000","modified":"2024-09-17T05:30:27.362693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6836/204904815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":204904815},{"type":"v1","addr":"172.21.15.173:6836","nonce":204904815}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5931,"qdb_cluster":[5931]},"id":4}]} 2024-09-17T05:32:21.509 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 37 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcbec073e60 msgr2=0x7fcbec076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcbec073e60 0x7fcbec076320 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc007fd0 tx=0x7fcbfc009040 comp rx=0 tx=0).stop 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc0c15f160 msgr2=0x7fcc0c14e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc0c15f160 0x7fcc0c14e560 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400cbe0 tx=0x7fcc0400ccd0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 shutdown_connections 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcbec073e60 0x7fcbec076320 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc0c15f160 0x7fcc0c14e560 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc0c06f930 0x7fcc0c14e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 --2- 172.21.15.122:0/2402026102 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc0c06cc20 0x7fcc0c154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 >> 172.21.15.122:0/2402026102 conn(0x7fcc0c0b2240 msgr2=0x7fcc0c161100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 shutdown_connections 2024-09-17T05:32:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.508+0000 7fcc12e00640 1 -- 172.21.15.122:0/2402026102 wait complete. 2024-09-17T05:32:21.530 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- 172.21.15.122:0/3955126693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 msgr2=0x7fb3140cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 --2- 172.21.15.122:0/3955126693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 0x7fb3140cefb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb300009900 tx=0x7fb30001c850 comp rx=0 tx=0).stop 2024-09-17T05:32:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- 172.21.15.122:0/3955126693 shutdown_connections 2024-09-17T05:32:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 --2- 172.21.15.122:0/3955126693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3140b2590 0x7fb3140b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 --2- 172.21.15.122:0/3955126693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 0x7fb3140cefb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- 172.21.15.122:0/3955126693 >> 172.21.15.122:0/3955126693 conn(0x7fb31409aa10 msgr2=0x7fb31409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- 172.21.15.122:0/3955126693 shutdown_connections 2024-09-17T05:32:21.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- 172.21.15.122:0/3955126693 wait complete. 2024-09-17T05:32:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 Processor -- start 2024-09-17T05:32:21.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- start start 2024-09-17T05:32:21.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3140b2590 0x7fb314151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 0x7fb3140b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb31415f000 0x7fb3140c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb314168f00 con 0x7fb3140d01b0 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb314168d80 con 0x7fb3140b2590 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb31a800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb314169080 con 0x7fb31415f000 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb318a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 0x7fb3140b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb318a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 0x7fb3140b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52106/0 (socket says 172.21.15.122:52106) 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb318a00640 1 -- 172.21.15.122:0/255931525 learned_addr learned my addr 172.21.15.122:0/255931525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb319400640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3140b2590 0x7fb314151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.672+0000 7fb319e00640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb31415f000 0x7fb3140c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb318a00640 1 -- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb31415f000 msgr2=0x7fb3140c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb318a00640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb31415f000 0x7fb3140c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb318a00640 1 -- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3140b2590 msgr2=0x7fb314151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb318a00640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3140b2590 0x7fb314151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb318a00640 1 -- 172.21.15.122:0/255931525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3140c6f20 con 0x7fb3140d01b0 2024-09-17T05:32:21.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb319400640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3140b2590 0x7fb314151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:21.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb318a00640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 0x7fb3140b5460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb3000b0040 tx=0x7fb300005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb311600640 1 -- 172.21.15.122:0/255931525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3000b2040 con 0x7fb3140d01b0 2024-09-17T05:32:21.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb311600640 1 -- 172.21.15.122:0/255931525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3000042f0 con 0x7fb3140d01b0 2024-09-17T05:32:21.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3140b5b40 con 0x7fb3140d01b0 2024-09-17T05:32:21.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb311600640 1 -- 172.21.15.122:0/255931525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3000052d0 con 0x7fb3140d01b0 2024-09-17T05:32:21.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3140b6100 con 0x7fb3140d01b0 2024-09-17T05:32:21.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3140b29e0 con 0x7fb3140d01b0 2024-09-17T05:32:21.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb311600640 1 -- 172.21.15.122:0/255931525 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fb300005470 con 0x7fb3140d01b0 2024-09-17T05:32:21.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb311600640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:21.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.676+0000 7fb311600640 1 -- 172.21.15.122:0/255931525 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7fb30012c740 con 0x7fb3140d01b0 2024-09-17T05:32:21.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.680+0000 7fb319400640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:21.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.680+0000 7fb319400640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb3080016b0 tx=0x7fb308009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:21.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.680+0000 7fb311600640 1 -- 172.21.15.122:0/255931525 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb3000164d0 con 0x7fb3140d01b0 2024-09-17T05:32:21.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.884+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3140cc190 con 0x7fb3140d01b0 2024-09-17T05:32:21.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.884+0000 7fb311600640 1 -- 172.21.15.122:0/255931525 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5232 (secure 0 0 0) 0x7fb3000f8fc0 con 0x7fb3140d01b0 2024-09-17T05:32:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:21.886 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":37,"btime":"2024-09-17T05:30:37:004420+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110418608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110418608},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110418608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1187468185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1187468185},{"type":"v1","addr":"172.21.15.122:6835","nonce":1187468185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3667127301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3667127301},{"type":"v1","addr":"172.21.15.122:6834","nonce":3667127301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:30:24.330680+0000","modified":"2024-09-17T05:30:27.362693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6836/204904815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":204904815},{"type":"v1","addr":"172.21.15.173:6836","nonce":204904815}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5931,"qdb_cluster":[5931]},"id":4}]} 2024-09-17T05:32:21.886 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 37 2024-09-17T05:32:21.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 msgr2=0x7fb2f4076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb3080016b0 tx=0x7fb308009040 comp rx=0 tx=0).stop 2024-09-17T05:32:21.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 msgr2=0x7fb3140b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 0x7fb3140b5460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb3000b0040 tx=0x7fb300005e70 comp rx=0 tx=0).stop 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 shutdown_connections 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb31415f000 0x7fb3140c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3140d01b0 0x7fb3140b5460 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 --2- 172.21.15.122:0/255931525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3140b2590 0x7fb314151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 >> 172.21.15.122:0/255931525 conn(0x7fb31409aa10 msgr2=0x7fb3140c8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 shutdown_connections 2024-09-17T05:32:21.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:21.888+0000 7fb31a800640 1 -- 172.21.15.122:0/255931525 wait complete. 2024-09-17T05:32:21.907 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:32:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.048+0000 7fa72a600640 1 -- 172.21.15.122:0/3074519131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 msgr2=0x7fa708004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.048+0000 7fa72a600640 1 --2- 172.21.15.122:0/3074519131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 0x7fa708004c90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009b60 tx=0x7fa70c01d2b0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.048+0000 7fa729c00640 1 -- 172.21.15.122:0/3074519131 reap_dead start 2024-09-17T05:32:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.048+0000 7fa72a600640 1 -- 172.21.15.122:0/3074519131 shutdown_connections 2024-09-17T05:32:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.048+0000 7fa72a600640 1 -- 172.21.15.122:0/3074519131 >> 172.21.15.122:0/3074519131 conn(0x7fa72409aa10 msgr2=0x7fa72409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.048+0000 7fa72a600640 1 -- 172.21.15.122:0/3074519131 shutdown_connections 2024-09-17T05:32:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.048+0000 7fa72a600640 1 -- 172.21.15.122:0/3074519131 wait complete. 2024-09-17T05:32:22.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.048+0000 7fa72a600640 1 Processor -- start 2024-09-17T05:32:22.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 -- start start 2024-09-17T05:32:22.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 0x7fa7240cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72415a870 0x7fa7241f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240cba60 0x7fa7241f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa724166cd0 con 0x7fa708004810 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa724166b50 con 0x7fa7240cba60 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa724166e50 con 0x7fa72415a870 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 0x7fa7240cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 0x7fa7240cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52118/0 (socket says 172.21.15.122:52118) 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 -- 172.21.15.122:0/2662812588 learned_addr learned my addr 172.21.15.122:0/2662812588 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa71be00640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72415a870 0x7fa7241f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729c00640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240cba60 0x7fa7241f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 -- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72415a870 msgr2=0x7fa7241f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72415a870 0x7fa7241f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 -- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240cba60 msgr2=0x7fa7241f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240cba60 0x7fa7241f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 -- 172.21.15.122:0/2662812588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7240c9010 con 0x7fa708004810 2024-09-17T05:32:22.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa71be00640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72415a870 0x7fa7241f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:22.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa729200640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 0x7fa7240cb520 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa71400e520 tx=0x7fa71400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa721600640 1 -- 172.21.15.122:0/2662812588 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714015070 con 0x7fa708004810 2024-09-17T05:32:22.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa721600640 1 -- 172.21.15.122:0/2662812588 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7140109a0 con 0x7fa708004810 2024-09-17T05:32:22.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7240c92f0 con 0x7fa708004810 2024-09-17T05:32:22.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa721600640 1 -- 172.21.15.122:0/2662812588 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714010df0 con 0x7fa708004810 2024-09-17T05:32:22.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7240cad20 con 0x7fa708004810 2024-09-17T05:32:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.052+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7240c8340 con 0x7fa708004810 2024-09-17T05:32:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.056+0000 7fa721600640 1 -- 172.21.15.122:0/2662812588 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fa714018070 con 0x7fa708004810 2024-09-17T05:32:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.056+0000 7fa721600640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa700073e90 0x7fa700076350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.056+0000 7fa721600640 1 -- 172.21.15.122:0/2662812588 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7fa714093670 con 0x7fa708004810 2024-09-17T05:32:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.056+0000 7fa71be00640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa700073e90 0x7fa700076350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.056+0000 7fa71be00640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa700073e90 0x7fa700076350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0ae000 tx=0x7fa70c01a050 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.060+0000 7fa721600640 1 -- 172.21.15.122:0/2662812588 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa71405fec0 con 0x7fa708004810 2024-09-17T05:32:22.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.228+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa724076210 con 0x7fa708004810 2024-09-17T05:32:22.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.228+0000 7fa721600640 1 -- 172.21.15.122:0/2662812588 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17033 (secure 0 0 0) 0x7fa71405f7c0 con 0x7fa708004810 2024-09-17T05:32:22.230 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:22.230 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":63,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:32:20.663744+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:30:22.033029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:30:23.299476+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1d","mappings":[{"from":6,"to":4}]},{"pgid":"9.3b","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa700073e90 msgr2=0x7fa700076350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa700073e90 0x7fa700076350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0ae000 tx=0x7fa70c01a050 comp rx=0 tx=0).stop 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 msgr2=0x7fa7240cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 0x7fa7240cb520 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa71400e520 tx=0x7fa71400e9e0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 shutdown_connections 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa700073e90 0x7fa700076350 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240cba60 0x7fa7241f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72415a870 0x7fa7241f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 --2- 172.21.15.122:0/2662812588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa708004810 0x7fa7240cb520 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 >> 172.21.15.122:0/2662812588 conn(0x7fa72409aa10 msgr2=0x7fa72415d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 shutdown_connections 2024-09-17T05:32:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.232+0000 7fa72a600640 1 -- 172.21.15.122:0/2662812588 wait complete. 2024-09-17T05:32:22.251 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:32:22.251 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:22.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- 172.21.15.122:0/1915784643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480afca0 msgr2=0x7f62480b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 --2- 172.21.15.122:0/1915784643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480afca0 0x7f62480b00d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f62480cfb60 tx=0x7f624001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- 172.21.15.122:0/1915784643 shutdown_connections 2024-09-17T05:32:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 --2- 172.21.15.122:0/1915784643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480afca0 0x7f62480b00d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 --2- 172.21.15.122:0/1915784643 >> v2:172.21.15.173:3300/0 conn(0x7f624815f490 0x7f62480b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- 172.21.15.122:0/1915784643 >> 172.21.15.122:0/1915784643 conn(0x7f62480d1e30 msgr2=0x7f62480d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- 172.21.15.122:0/1915784643 shutdown_connections 2024-09-17T05:32:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- 172.21.15.122:0/1915784643 wait complete. 2024-09-17T05:32:22.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 Processor -- start 2024-09-17T05:32:22.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- start start 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f62480afca0 0x7f624815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480b2e80 0x7f624814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f624815f490 0x7f62480cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6248168e00 con 0x7f62480b2e80 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6248168c80 con 0x7f624815f490 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.392+0000 7f624e200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6248168f80 con 0x7f62480afca0 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480b2e80 0x7f624814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480b2e80 0x7f624814fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52134/0 (socket says 172.21.15.122:52134) 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 -- 172.21.15.122:0/2354140035 learned_addr learned my addr 172.21.15.122:0/2354140035 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f6247e00640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f624815f490 0x7f62480cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f6247400640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f62480afca0 0x7f624815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 -- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f62480afca0 msgr2=0x7f624815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f62480afca0 0x7f624815ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 -- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f624815f490 msgr2=0x7f62480cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f624815f490 0x7f62480cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 -- 172.21.15.122:0/2354140035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62480cbdf0 con 0x7f62480b2e80 2024-09-17T05:32:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623ea00640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480b2e80 0x7f624814fe50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f623000c480 tx=0x7f623000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623fe00640 1 -- 172.21.15.122:0/2354140035 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230012830 con 0x7f62480b2e80 2024-09-17T05:32:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623fe00640 1 -- 172.21.15.122:0/2354140035 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6230017070 con 0x7f62480b2e80 2024-09-17T05:32:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62480b5950 con 0x7f62480b2e80 2024-09-17T05:32:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623fe00640 1 -- 172.21.15.122:0/2354140035 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230012d80 con 0x7f62480b2e80 2024-09-17T05:32:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62480b5f10 con 0x7f62480b2e80 2024-09-17T05:32:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62480b00d0 con 0x7f62480b2e80 2024-09-17T05:32:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.396+0000 7f623fe00640 1 -- 172.21.15.122:0/2354140035 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f623001c460 con 0x7f62480b2e80 2024-09-17T05:32:22.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.400+0000 7f623fe00640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6224073e60 0x7f6224076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.400+0000 7f623fe00640 1 -- 172.21.15.122:0/2354140035 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7f6230092a70 con 0x7f62480b2e80 2024-09-17T05:32:22.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.400+0000 7f6247400640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6224073e60 0x7f6224076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.400+0000 7f6247400640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6224073e60 0x7f6224076320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f62380058d0 tx=0x7f6238005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.400+0000 7f623fe00640 1 -- 172.21.15.122:0/2354140035 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f623005f570 con 0x7f62480b2e80 2024-09-17T05:32:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.604+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62480d00d0 con 0x7f62480b2e80 2024-09-17T05:32:22.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.604+0000 7f623fe00640 1 -- 172.21.15.122:0/2354140035 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5232 (secure 0 0 0) 0x7f623005ee70 con 0x7f62480b2e80 2024-09-17T05:32:22.607 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:22.607 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":37,"btime":"2024-09-17T05:30:37:004420+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110418608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110418608},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110418608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1187468185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1187468185},{"type":"v1","addr":"172.21.15.122:6835","nonce":1187468185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3667127301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3667127301},{"type":"v1","addr":"172.21.15.122:6834","nonce":3667127301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:30:24.330680+0000","modified":"2024-09-17T05:30:27.362693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6836/204904815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":204904815},{"type":"v1","addr":"172.21.15.173:6836","nonce":204904815}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5931,"qdb_cluster":[5931]},"id":4}]} 2024-09-17T05:32:22.608 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 37 2024-09-17T05:32:22.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6224073e60 msgr2=0x7f6224076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6224073e60 0x7f6224076320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f62380058d0 tx=0x7f6238005860 comp rx=0 tx=0).stop 2024-09-17T05:32:22.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480b2e80 msgr2=0x7f624814fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480b2e80 0x7f624814fe50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f623000c480 tx=0x7f623000c940 comp rx=0 tx=0).stop 2024-09-17T05:32:22.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 shutdown_connections 2024-09-17T05:32:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6224073e60 0x7f6224076320 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f624815f490 0x7f62480cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62480b2e80 0x7f624814fe50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 --2- 172.21.15.122:0/2354140035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f62480afca0 0x7f624815ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 >> 172.21.15.122:0/2354140035 conn(0x7f62480d1e30 msgr2=0x7f62480b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 shutdown_connections 2024-09-17T05:32:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.608+0000 7f624e200640 1 -- 172.21.15.122:0/2354140035 wait complete. 2024-09-17T05:32:22.628 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:22.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.772+0000 7f52c3800640 1 -- 172.21.15.122:0/1465716312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 msgr2=0x7f52bc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.772+0000 7f52c3800640 1 --2- 172.21.15.122:0/1465716312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 0x7f52bc0cebb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009bc0 tx=0x7f52b401cba0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.772+0000 7f52c3800640 1 -- 172.21.15.122:0/1465716312 shutdown_connections 2024-09-17T05:32:22.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.772+0000 7f52c3800640 1 --2- 172.21.15.122:0/1465716312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 0x7f52bc0cebb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.772+0000 7f52c3800640 1 --2- 172.21.15.122:0/1465716312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc15bea0 0x7f52bc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.772+0000 7f52c3800640 1 -- 172.21.15.122:0/1465716312 >> 172.21.15.122:0/1465716312 conn(0x7f52bc0d1dc0 msgr2=0x7f52bc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:22.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.772+0000 7f52c3800640 1 -- 172.21.15.122:0/1465716312 shutdown_connections 2024-09-17T05:32:22.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.772+0000 7f52c3800640 1 -- 172.21.15.122:0/1465716312 wait complete. 2024-09-17T05:32:22.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 Processor -- start 2024-09-17T05:32:22.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 -- start start 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc0cdb30 0x7f52bc1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 0x7f52bc153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc15bea0 0x7f52bc09f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc169020 con 0x7f52bc0d28e0 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc168ea0 con 0x7f52bc15bea0 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc1691a0 con 0x7f52bc0cdb30 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 0x7f52bc153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 0x7f52bc153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52156/0 (socket says 172.21.15.122:52156) 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 -- 172.21.15.122:0/1426069165 learned_addr learned my addr 172.21.15.122:0/1426069165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:22.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c0e00640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc0cdb30 0x7f52bc1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c1800640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc15bea0 0x7f52bc09f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 -- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc0cdb30 msgr2=0x7f52bc1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc0cdb30 0x7f52bc1545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 -- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc15bea0 msgr2=0x7f52bc09f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc15bea0 0x7f52bc09f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 -- 172.21.15.122:0/1426069165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc1f9070 con 0x7f52bc0d28e0 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52bbe00640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 0x7f52bc153e20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c480 tx=0x7f52a800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52b8c00640 1 -- 172.21.15.122:0/1426069165 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8016020 con 0x7f52bc0d28e0 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52b8c00640 1 -- 172.21.15.122:0/1426069165 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a8012920 con 0x7f52bc0d28e0 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52b8c00640 1 -- 172.21.15.122:0/1426069165 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8012d70 con 0x7f52bc0d28e0 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc1f9350 con 0x7f52bc0d28e0 2024-09-17T05:32:22.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc1f9910 con 0x7f52bc0d28e0 2024-09-17T05:32:22.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.776+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52bc0cc9d0 con 0x7f52bc0d28e0 2024-09-17T05:32:22.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.780+0000 7f52b8c00640 1 -- 172.21.15.122:0/1426069165 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f52a8019020 con 0x7f52bc0d28e0 2024-09-17T05:32:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.780+0000 7f52b8c00640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f529c073e60 0x7f529c076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.780+0000 7f52b8c00640 1 -- 172.21.15.122:0/1426069165 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7f52a8092d50 con 0x7f52bc0d28e0 2024-09-17T05:32:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.780+0000 7f52c0e00640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f529c073e60 0x7f529c076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:22.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.780+0000 7f52c0e00640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f529c073e60 0x7f529c076320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0b3d10 tx=0x7f52ac00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.784+0000 7f52b8c00640 1 -- 172.21.15.122:0/1426069165 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f52a8020020 con 0x7f52bc0d28e0 2024-09-17T05:32:22.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52bc0cce70 con 0x7f52bc0d28e0 2024-09-17T05:32:22.988 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:22.988 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":37,"btime":"2024-09-17T05:30:37:004420+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110418608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110418608},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110418608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1187468185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1187468185},{"type":"v1","addr":"172.21.15.122:6835","nonce":1187468185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3667127301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3667127301},{"type":"v1","addr":"172.21.15.122:6834","nonce":3667127301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:30:24.330680+0000","modified":"2024-09-17T05:30:27.362693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5931},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5931":{"gid":5931,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6836/204904815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":204904815},{"type":"v1","addr":"172.21.15.173:6836","nonce":204904815}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5931,"qdb_cluster":[5931]},"id":4}]} 2024-09-17T05:32:22.988 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 37 2024-09-17T05:32:23.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52b8c00640 1 -- 172.21.15.122:0/1426069165 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5232 (secure 0 0 0) 0x7f52a805ef80 con 0x7f52bc0d28e0 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f529c073e60 msgr2=0x7f529c076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f529c073e60 0x7f529c076320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0b3d10 tx=0x7f52ac00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 msgr2=0x7f52bc153e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 0x7f52bc153e20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c480 tx=0x7f52a800c940 comp rx=0 tx=0).stop 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 shutdown_connections 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f529c073e60 0x7f529c076320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc15bea0 0x7f52bc09f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0d28e0 0x7f52bc153e20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 --2- 172.21.15.122:0/1426069165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc0cdb30 0x7f52bc1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 >> 172.21.15.122:0/1426069165 conn(0x7f52bc0d1dc0 msgr2=0x7f52bc0b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 shutdown_connections 2024-09-17T05:32:23.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:22.988+0000 7f52c3800640 1 -- 172.21.15.122:0/1426069165 wait complete. 2024-09-17T05:32:23.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:32:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.172+0000 7f2200e00640 1 -- 172.21.15.122:0/197345653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21fc15b890 msgr2=0x7f21fc0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.172+0000 7f2200e00640 1 --2- 172.21.15.122:0/197345653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21fc15b890 0x7f21fc0ce010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009840 tx=0x7f21ec01c820 comp rx=0 tx=0).stop 2024-09-17T05:32:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.172+0000 7f2200e00640 1 -- 172.21.15.122:0/197345653 shutdown_connections 2024-09-17T05:32:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.172+0000 7f2200e00640 1 --2- 172.21.15.122:0/197345653 >> v2:172.21.15.173:3300/0 conn(0x7f21fc15bc70 0x7f21fc0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.172+0000 7f2200e00640 1 --2- 172.21.15.122:0/197345653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21fc15b890 0x7f21fc0ce010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.172+0000 7f2200e00640 1 -- 172.21.15.122:0/197345653 >> 172.21.15.122:0/197345653 conn(0x7f21fc06dd30 msgr2=0x7f21fc06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.172+0000 7f2200e00640 1 -- 172.21.15.122:0/197345653 shutdown_connections 2024-09-17T05:32:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.172+0000 7f2200e00640 1 -- 172.21.15.122:0/197345653 wait complete. 2024-09-17T05:32:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 Processor -- start 2024-09-17T05:32:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 -- start start 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21fc06cc20 0x7f21fc1fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21fc15b890 0x7f21fc0d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21fc15bc70 0x7f21fc0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc168c90 con 0x7f21fc06cc20 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc168b10 con 0x7f21fc15b890 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc168e10 con 0x7f21fc15bc70 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21fc06cc20 0x7f21fc1fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21fc06cc20 0x7f21fc1fbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52172/0 (socket says 172.21.15.122:52172) 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 -- 172.21.15.122:0/4071596865 learned_addr learned my addr 172.21.15.122:0/4071596865 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:23.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fbe00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21fc15bc70 0x7f21fc0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21faa00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21fc15b890 0x7f21fc0d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 -- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21fc15bc70 msgr2=0x7f21fc0b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21fc15bc70 0x7f21fc0b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 -- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21fc15b890 msgr2=0x7f21fc0d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21fc15b890 0x7f21fc0d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 -- 172.21.15.122:0/4071596865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc0d06c0 con 0x7f21fc06cc20 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21fb400640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21fc06cc20 0x7f21fc1fbb30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f21ec01cd00 tx=0x7f21ec005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21f3400640 1 -- 172.21.15.122:0/4071596865 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0ba020 con 0x7f21fc06cc20 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21f3400640 1 -- 172.21.15.122:0/4071596865 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21ec004330 con 0x7f21fc06cc20 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc0b4440 con 0x7f21fc06cc20 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f21f3400640 1 -- 172.21.15.122:0/4071596865 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0b2040 con 0x7f21fc06cc20 2024-09-17T05:32:23.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1fb330 con 0x7f21fc06cc20 2024-09-17T05:32:23.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.176+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc0ce010 con 0x7f21fc06cc20 2024-09-17T05:32:23.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.180+0000 7f21f3400640 1 -- 172.21.15.122:0/4071596865 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f21ec0bd070 con 0x7f21fc06cc20 2024-09-17T05:32:23.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.180+0000 7f21f3400640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21dc073e60 0x7f21dc076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.180+0000 7f21f3400640 1 -- 172.21.15.122:0/4071596865 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7f21ec12bf50 con 0x7f21fc06cc20 2024-09-17T05:32:23.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.180+0000 7f21faa00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21dc073e60 0x7f21dc076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.180+0000 7f21faa00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21dc073e60 0x7f21dc076320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f21fc15e7a0 tx=0x7f21e8005f40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.184+0000 7f21f3400640 1 -- 172.21.15.122:0/4071596865 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f21ec016410 con 0x7f21fc06cc20 2024-09-17T05:32:23.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.356+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21fc071830 con 0x7f21fc06cc20 2024-09-17T05:32:23.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.356+0000 7f21f3400640 1 -- 172.21.15.122:0/4071596865 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17033 (secure 0 0 0) 0x7f21ec0f8a50 con 0x7f21fc06cc20 2024-09-17T05:32:23.358 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":63,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:32:20.663744+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:30:22.033029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-17T05:30:23.299476+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.1d","mappings":[{"from":6,"to":4}]},{"pgid":"9.3b","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:32:23.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21dc073e60 msgr2=0x7f21dc076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21dc073e60 0x7f21dc076320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f21fc15e7a0 tx=0x7f21e8005f40 comp rx=0 tx=0).stop 2024-09-17T05:32:23.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21fc06cc20 msgr2=0x7f21fc1fbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21fc06cc20 0x7f21fc1fbb30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f21ec01cd00 tx=0x7f21ec005d20 comp rx=0 tx=0).stop 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 shutdown_connections 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f21dc073e60 0x7f21dc076320 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21fc15bc70 0x7f21fc0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21fc15b890 0x7f21fc0d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 --2- 172.21.15.122:0/4071596865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21fc06cc20 0x7f21fc1fbb30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 >> 172.21.15.122:0/4071596865 conn(0x7f21fc06dd30 msgr2=0x7f21fc0ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 shutdown_connections 2024-09-17T05:32:23.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.360+0000 7f2200e00640 1 -- 172.21.15.122:0/4071596865 wait complete. 2024-09-17T05:32:23.380 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:32:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.520+0000 7f1fd1600640 1 -- 172.21.15.122:0/3802503383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc15bc70 msgr2=0x7f1fcc0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.520+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3802503383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc15bc70 0x7f1fcc0ce010 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0c7b40 tx=0x7f1fb401c870 comp rx=0 tx=0).stop 2024-09-17T05:32:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.520+0000 7f1fd1600640 1 -- 172.21.15.122:0/3802503383 shutdown_connections 2024-09-17T05:32:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.520+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3802503383 >> v2:172.21.15.173:3300/0 conn(0x7f1fcc06cc20 0x7f1fcc06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.520+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3802503383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc15bc70 0x7f1fcc0ce010 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.520+0000 7f1fd1600640 1 -- 172.21.15.122:0/3802503383 >> 172.21.15.122:0/3802503383 conn(0x7f1fcc06dd30 msgr2=0x7f1fcc06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.520+0000 7f1fd1600640 1 -- 172.21.15.122:0/3802503383 shutdown_connections 2024-09-17T05:32:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.520+0000 7f1fd1600640 1 -- 172.21.15.122:0/3802503383 wait complete. 2024-09-17T05:32:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 Processor -- start 2024-09-17T05:32:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 -- start start 2024-09-17T05:32:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc06cc20 0x7f1fcc14f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fcc15b890 0x7f1fcc165a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fcc15bc70 0x7f1fcc1f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc1687a0 con 0x7f1fcc06cc20 2024-09-17T05:32:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1fcc0746f0 con 0x7f1fcc15b890 2024-09-17T05:32:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1fcc168920 con 0x7f1fcc15bc70 2024-09-17T05:32:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc06cc20 0x7f1fcc14f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc06cc20 0x7f1fcc14f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52196/0 (socket says 172.21.15.122:52196) 2024-09-17T05:32:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 -- 172.21.15.122:0/3944922191 learned_addr learned my addr 172.21.15.122:0/3944922191 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd0c00640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fcc15bc70 0x7f1fcc1f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fc3e00640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fcc15b890 0x7f1fcc165a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 -- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fcc15bc70 msgr2=0x7f1fcc1f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fcc15bc70 0x7f1fcc1f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 -- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fcc15b890 msgr2=0x7f1fcc165a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fcc15b890 0x7f1fcc165a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 -- 172.21.15.122:0/3944922191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fcc0ce8e0 con 0x7f1fcc06cc20 2024-09-17T05:32:23.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fcbe00640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc06cc20 0x7f1fcc14f240 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c480 tx=0x7f1fbc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fc8c00640 1 -- 172.21.15.122:0/3944922191 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc012830 con 0x7f1fcc06cc20 2024-09-17T05:32:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fc8c00640 1 -- 172.21.15.122:0/3944922191 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fbc017070 con 0x7f1fcc06cc20 2024-09-17T05:32:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fc8c00640 1 -- 172.21.15.122:0/3944922191 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc012d80 con 0x7f1fcc06cc20 2024-09-17T05:32:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fcc0cebc0 con 0x7f1fcc06cc20 2024-09-17T05:32:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fcc166530 con 0x7f1fcc06cc20 2024-09-17T05:32:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.524+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fcc06d050 con 0x7f1fcc06cc20 2024-09-17T05:32:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.528+0000 7f1fc8c00640 1 -- 172.21.15.122:0/3944922191 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f1fbc01c460 con 0x7f1fcc06cc20 2024-09-17T05:32:23.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.528+0000 7f1fc8c00640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa8073e60 0x7f1fa8076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.528+0000 7f1fc8c00640 1 -- 172.21.15.122:0/3944922191 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7f1fbc092930 con 0x7f1fcc06cc20 2024-09-17T05:32:23.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.528+0000 7f1fc3e00640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa8073e60 0x7f1fa8076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.528+0000 7f1fc3e00640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa8073e60 0x7f1fa8076320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40af000 tx=0x7f1fb401b0b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.532+0000 7f1fc8c00640 1 -- 172.21.15.122:0/3944922191 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1fbc05f430 con 0x7f1fcc06cc20 2024-09-17T05:32:23.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.736+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1fcc0cbfd0 con 0x7f1fcc06cc20 2024-09-17T05:32:23.736 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:32:23.736+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:32:23.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.736+0000 7f1fc8c00640 1 -- 172.21.15.122:0/3944922191 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f1fbc05ed30 con 0x7f1fcc06cc20 2024-09-17T05:32:23.738 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:32:23.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa8073e60 msgr2=0x7f1fa8076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa8073e60 0x7f1fa8076320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40af000 tx=0x7f1fb401b0b0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc06cc20 msgr2=0x7f1fcc14f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc06cc20 0x7f1fcc14f240 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00c480 tx=0x7f1fbc00c940 comp rx=0 tx=0).stop 2024-09-17T05:32:23.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 shutdown_connections 2024-09-17T05:32:23.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa8073e60 0x7f1fa8076320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fcc15bc70 0x7f1fcc1f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fcc15b890 0x7f1fcc165a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 --2- 172.21.15.122:0/3944922191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fcc06cc20 0x7f1fcc14f240 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 >> 172.21.15.122:0/3944922191 conn(0x7f1fcc06dd30 msgr2=0x7f1fcc0b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 shutdown_connections 2024-09-17T05:32:23.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.740+0000 7f1fd1600640 1 -- 172.21.15.122:0/3944922191 wait complete. 2024-09-17T05:32:23.761 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:32:23.811 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:32:23.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 -- 172.21.15.122:0/2530897257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 msgr2=0x7f3c340d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/2530897257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 0x7f3c340d28f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009900 tx=0x7f3c2001c850 comp rx=0 tx=0).stop 2024-09-17T05:32:23.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 -- 172.21.15.122:0/2530897257 shutdown_connections 2024-09-17T05:32:23.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/2530897257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c340d0950 0x7f3c340aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/2530897257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 0x7f3c340d28f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 -- 172.21.15.122:0/2530897257 >> 172.21.15.122:0/2530897257 conn(0x7f3c340d1e00 msgr2=0x7f3c340d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:23.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 -- 172.21.15.122:0/2530897257 shutdown_connections 2024-09-17T05:32:23.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 -- 172.21.15.122:0/2530897257 wait complete. 2024-09-17T05:32:23.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 Processor -- start 2024-09-17T05:32:23.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.904+0000 7f3c3ac00640 1 -- start start 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 0x7f3c341f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c340d0950 0x7f3c341581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3415bf70 0x7f3c34155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34168d40 con 0x7f3c340b0f50 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34168bc0 con 0x7f3c340d0950 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3c34168ec0 con 0x7f3c3415bf70 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 0x7f3c341f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 0x7f3c341f8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52206/0 (socket says 172.21.15.122:52206) 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 -- 172.21.15.122:0/1636291469 learned_addr learned my addr 172.21.15.122:0/1636291469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33400640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c340d0950 0x7f3c341581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c38c00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3415bf70 0x7f3c34155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 -- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3415bf70 msgr2=0x7f3c34155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3415bf70 0x7f3c34155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 -- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c340d0950 msgr2=0x7f3c341581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:23.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c340d0950 0x7f3c341581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:23.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 -- 172.21.15.122:0/1636291469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c340b6200 con 0x7f3c340b0f50 2024-09-17T05:32:23.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c33e00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 0x7f3c341f8950 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c3c0 tx=0x7f3c2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c2be00640 1 -- 172.21.15.122:0/1636291469 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24017070 con 0x7f3c340b0f50 2024-09-17T05:32:23.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c2be00640 1 -- 172.21.15.122:0/1636291469 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c240053c0 con 0x7f3c340b0f50 2024-09-17T05:32:23.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c2be00640 1 -- 172.21.15.122:0/1636291469 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24003860 con 0x7f3c340b0f50 2024-09-17T05:32:23.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c340b0550 con 0x7f3c340b0f50 2024-09-17T05:32:23.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c340cae50 con 0x7f3c340b0f50 2024-09-17T05:32:23.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.908+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c340d28f0 con 0x7f3c340b0f50 2024-09-17T05:32:23.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.912+0000 7f3c2be00640 1 -- 172.21.15.122:0/1636291469 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f3c2401a070 con 0x7f3c340b0f50 2024-09-17T05:32:23.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.912+0000 7f3c2be00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c14073e60 0x7f3c14076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:23.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.912+0000 7f3c2be00640 1 -- 172.21.15.122:0/1636291469 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5909+0+0 (secure 0 0 0) 0x7f3c24092150 con 0x7f3c340b0f50 2024-09-17T05:32:23.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.912+0000 7f3c33400640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c14073e60 0x7f3c14076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:23.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.912+0000 7f3c33400640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c14073e60 0x7f3c14076320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3c200af000 tx=0x7f3c2001b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:23.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:23.916+0000 7f3c2be00640 1 -- 172.21.15.122:0/1636291469 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3c2405e380 con 0x7f3c340b0f50 2024-09-17T05:32:24.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.120+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3c34076480 con 0x7f3c340b0f50 2024-09-17T05:32:24.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c2be00640 1 -- 172.21.15.122:0/1636291469 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f3c24062db0 con 0x7f3c340b0f50 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c14073e60 msgr2=0x7f3c14076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c14073e60 0x7f3c14076320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3c200af000 tx=0x7f3c2001b090 comp rx=0 tx=0).stop 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 msgr2=0x7f3c341f8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 0x7f3c341f8950 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c3c0 tx=0x7f3c2400c880 comp rx=0 tx=0).stop 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 shutdown_connections 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c14073e60 0x7f3c14076320 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c3415bf70 0x7f3c34155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c340d0950 0x7f3c341581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 --2- 172.21.15.122:0/1636291469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c340b0f50 0x7f3c341f8950 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 >> 172.21.15.122:0/1636291469 conn(0x7f3c340d1e00 msgr2=0x7f3c340d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:24.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 shutdown_connections 2024-09-17T05:32:24.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.744+0000 7f3c3ac00640 1 -- 172.21.15.122:0/1636291469 wait complete. 2024-09-17T05:32:24.767 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:32:24.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 -- 172.21.15.122:0/2763917170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f015e360 msgr2=0x7fa3f00adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 --2- 172.21.15.122:0/2763917170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f015e360 0x7fa3f00adf50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009900 tx=0x7fa3e001c850 comp rx=0 tx=0).stop 2024-09-17T05:32:24.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 -- 172.21.15.122:0/2763917170 shutdown_connections 2024-09-17T05:32:24.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 --2- 172.21.15.122:0/2763917170 >> v2:172.21.15.173:3300/0 conn(0x7fa3f015e740 0x7fa3f015ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 --2- 172.21.15.122:0/2763917170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f015e360 0x7fa3f00adf50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 -- 172.21.15.122:0/2763917170 >> 172.21.15.122:0/2763917170 conn(0x7fa3f00cfbc0 msgr2=0x7fa3f00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:24.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 -- 172.21.15.122:0/2763917170 shutdown_connections 2024-09-17T05:32:24.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 -- 172.21.15.122:0/2763917170 wait complete. 2024-09-17T05:32:24.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 Processor -- start 2024-09-17T05:32:24.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.912+0000 7fa3f5200640 1 -- start start 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f00aef60 0x7fa3f0157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f015e360 0x7fa3f0157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3f015e740 0x7fa3f014c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01680b0 con 0x7fa3f015e740 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0167f30 con 0x7fa3f015e360 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f0168230 con 0x7fa3f00aef60 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f00aef60 0x7fa3f0157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f00aef60 0x7fa3f0157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:40306/0 (socket says 172.21.15.122:40306) 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 -- 172.21.15.122:0/3058348713 learned_addr learned my addr 172.21.15.122:0/3058348713 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:24.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ee800640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3f015e740 0x7fa3f014c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:24.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ed400640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f015e360 0x7fa3f0157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:24.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 -- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f015e360 msgr2=0x7fa3f0157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f015e360 0x7fa3f0157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 -- 172.21.15.122:0/3058348713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3f015e740 msgr2=0x7fa3f014c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:24.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3f015e740 0x7fa3f014c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:24.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 -- 172.21.15.122:0/3058348713 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f00b35b0 con 0x7fa3f00aef60 2024-09-17T05:32:24.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3ede00640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f00aef60 0x7fa3f0157a60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00b0040 tx=0x7fa3e0005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:24.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3e6000640 1 -- 172.21.15.122:0/3058348713 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00b2040 con 0x7fa3f00aef60 2024-09-17T05:32:24.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3e6000640 1 -- 172.21.15.122:0/3058348713 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e00043e0 con 0x7fa3f00aef60 2024-09-17T05:32:24.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f00b3850 con 0x7fa3f00aef60 2024-09-17T05:32:24.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3e6000640 1 -- 172.21.15.122:0/3058348713 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00052d0 con 0x7fa3f00aef60 2024-09-17T05:32:24.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f00b3e10 con 0x7fa3f00aef60 2024-09-17T05:32:24.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.916+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f015ebf0 con 0x7fa3f00aef60 2024-09-17T05:32:24.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.920+0000 7fa3e6000640 1 -- 172.21.15.122:0/3058348713 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fa3e00bd060 con 0x7fa3f00aef60 2024-09-17T05:32:24.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.920+0000 7fa3e6000640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3d0073e60 0x7fa3d0076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:24.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.920+0000 7fa3e6000640 1 -- 172.21.15.122:0/3058348713 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 5909+0+0 (secure 0 0 0) 0x7fa3e012c170 con 0x7fa3f00aef60 2024-09-17T05:32:24.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.920+0000 7fa3ed400640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3d0073e60 0x7fa3d0076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:24.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.920+0000 7fa3ed400640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3d0073e60 0x7fa3d0076320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa3f01586b0 tx=0x7fa3dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:24.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:24.924+0000 7fa3e6000640 1 -- 172.21.15.122:0/3058348713 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa3e00164d0 con 0x7fa3f00aef60 2024-09-17T05:32:25.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.096+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa3f006ff10 con 0x7fa3f00aef60 2024-09-17T05:32:25.750 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:32:25.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.748+0000 7fa3e6000640 1 -- 172.21.15.122:0/3058348713 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) ==== 181+0+0 (secure 0 0 0) 0x7fa3e00f83a0 con 0x7fa3f00aef60 2024-09-17T05:32:25.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3d0073e60 msgr2=0x7fa3d0076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3d0073e60 0x7fa3d0076320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa3f01586b0 tx=0x7fa3dc009040 comp rx=0 tx=0).stop 2024-09-17T05:32:25.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f00aef60 msgr2=0x7fa3f0157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f00aef60 0x7fa3f0157a60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00b0040 tx=0x7fa3e0005e90 comp rx=0 tx=0).stop 2024-09-17T05:32:25.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 shutdown_connections 2024-09-17T05:32:25.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3d0073e60 0x7fa3d0076320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3f015e740 0x7fa3f014c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f015e360 0x7fa3f0157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 --2- 172.21.15.122:0/3058348713 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3f00aef60 0x7fa3f0157a60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 >> 172.21.15.122:0/3058348713 conn(0x7fa3f00cfbc0 msgr2=0x7fa3f006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:25.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 shutdown_connections 2024-09-17T05:32:25.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.752+0000 7fa3f5200640 1 -- 172.21.15.122:0/3058348713 wait complete. 2024-09-17T05:32:25.774 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:32:25.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- 172.21.15.122:0/1418000826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3200004810 msgr2=0x7f3200004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 --2- 172.21.15.122:0/1418000826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3200004810 0x7f3200004c90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f320800aca0 tx=0x7f320801e090 comp rx=0 tx=0).stop 2024-09-17T05:32:25.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 -- 172.21.15.122:0/1418000826 reap_dead start 2024-09-17T05:32:25.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- 172.21.15.122:0/1418000826 shutdown_connections 2024-09-17T05:32:25.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- 172.21.15.122:0/1418000826 >> 172.21.15.122:0/1418000826 conn(0x7f321c0cfbc0 msgr2=0x7f321c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:25.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- 172.21.15.122:0/1418000826 shutdown_connections 2024-09-17T05:32:25.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- 172.21.15.122:0/1418000826 wait complete. 2024-09-17T05:32:25.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 Processor -- start 2024-09-17T05:32:25.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- start start 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200004810 0x7f321c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f321c14d860 0x7f321c06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c14dcc0 0x7f321c070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cdf60 con 0x7f321c14dcc0 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cdde0 con 0x7f3200004810 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3222c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0ce0e0 con 0x7f321c14d860 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c14dcc0 0x7f321c070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c14dcc0 0x7f321c070040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52212/0 (socket says 172.21.15.122:52212) 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 -- 172.21.15.122:0/2677800292 learned_addr learned my addr 172.21.15.122:0/2677800292 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f321b400640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f321c14d860 0x7f321c06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f321be00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200004810 0x7f321c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 -- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f321c14d860 msgr2=0x7f321c06fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f321c14d860 0x7f321c06fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 -- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200004810 msgr2=0x7f321c151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200004810 0x7f321c151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.920+0000 7f3220c00640 1 -- 172.21.15.122:0/2677800292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c14eed0 con 0x7f321c14dcc0 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3220c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c14dcc0 0x7f321c070040 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f321c0768a0 tx=0x7f3214011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3213e00640 1 -- 172.21.15.122:0/2677800292 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214004440 con 0x7f321c14dcc0 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3213e00640 1 -- 172.21.15.122:0/2677800292 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3214016070 con 0x7f321c14dcc0 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3213e00640 1 -- 172.21.15.122:0/2677800292 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321400faf0 con 0x7f321c14dcc0 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c14f1b0 con 0x7f321c14dcc0 2024-09-17T05:32:25.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c14f770 con 0x7f321c14dcc0 2024-09-17T05:32:25.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c069410 con 0x7f321c14dcc0 2024-09-17T05:32:25.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3213e00640 1 -- 172.21.15.122:0/2677800292 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f321400fc90 con 0x7f321c14dcc0 2024-09-17T05:32:25.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3213e00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31fc073e60 0x7f31fc076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:25.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.924+0000 7f3213e00640 1 -- 172.21.15.122:0/2677800292 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5471+0+0 (secure 0 0 0) 0x7f3214093610 con 0x7f321c14dcc0 2024-09-17T05:32:25.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.928+0000 7f321be00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31fc073e60 0x7f31fc076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:25.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.928+0000 7f321be00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31fc073e60 0x7f31fc076320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f320c001bf0 tx=0x7f320c001f00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:25.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:25.928+0000 7f3213e00640 1 -- 172.21.15.122:0/2677800292 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3214022020 con 0x7f321c14dcc0 2024-09-17T05:32:26.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.096+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f321c06a400 con 0x7f321c14dcc0 2024-09-17T05:32:26.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.752+0000 7f3213e00640 1 -- 172.21.15.122:0/2677800292 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) ==== 169+0+0 (secure 0 0 0) 0x7f3214060110 con 0x7f321c14dcc0 2024-09-17T05:32:26.752 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:32:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.752+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31fc073e60 msgr2=0x7f31fc076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.752+0000 7f3222c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31fc073e60 0x7f31fc076320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f320c001bf0 tx=0x7f320c001f00 comp rx=0 tx=0).stop 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.752+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c14dcc0 msgr2=0x7f321c070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.752+0000 7f3222c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c14dcc0 0x7f321c070040 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f321c0768a0 tx=0x7f3214011be0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.756+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 shutdown_connections 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.756+0000 7f3222c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f31fc073e60 0x7f31fc076320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.756+0000 7f3222c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f321c14dcc0 0x7f321c070040 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.756+0000 7f3222c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f321c14d860 0x7f321c06fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.756+0000 7f3222c00640 1 --2- 172.21.15.122:0/2677800292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3200004810 0x7f321c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.756+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 >> 172.21.15.122:0/2677800292 conn(0x7f321c0cfbc0 msgr2=0x7f321c0af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.756+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 shutdown_connections 2024-09-17T05:32:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.756+0000 7f3222c00640 1 -- 172.21.15.122:0/2677800292 wait complete. 2024-09-17T05:32:26.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:32:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.924+0000 7fbb06800640 1 -- 172.21.15.122:0/2533704104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbae4004810 msgr2=0x7fbae4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.924+0000 7fbb06800640 1 --2- 172.21.15.122:0/2533704104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbae4004810 0x7fbae4004c90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0016f0 tx=0x7fbaec01c7b0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.924+0000 7fbaffe00640 1 -- 172.21.15.122:0/2533704104 reap_dead start 2024-09-17T05:32:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.924+0000 7fbb06800640 1 -- 172.21.15.122:0/2533704104 shutdown_connections 2024-09-17T05:32:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.924+0000 7fbb06800640 1 -- 172.21.15.122:0/2533704104 >> 172.21.15.122:0/2533704104 conn(0x7fbb000b18f0 msgr2=0x7fbb000b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.924+0000 7fbb06800640 1 -- 172.21.15.122:0/2533704104 shutdown_connections 2024-09-17T05:32:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.924+0000 7fbb06800640 1 -- 172.21.15.122:0/2533704104 wait complete. 2024-09-17T05:32:26.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 Processor -- start 2024-09-17T05:32:26.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 -- start start 2024-09-17T05:32:26.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbae4004810 0x7fbb001f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb001f8a90 0x7fbb0015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb001f7e00 0x7fbb0015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb000cca10 con 0x7fbb001f8a90 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb000b4e90 con 0x7fbae4004810 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbb000ccb90 con 0x7fbb001f7e00 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb001f8a90 0x7fbb0015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb001f8a90 0x7fbb0015d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52222/0 (socket says 172.21.15.122:52222) 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 -- 172.21.15.122:0/1155645637 learned_addr learned my addr 172.21.15.122:0/1155645637 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbaffe00640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb001f7e00 0x7fbb0015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbaff400640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbae4004810 0x7fbb001f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 -- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb001f7e00 msgr2=0x7fbb0015d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb001f7e00 0x7fbb0015d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 -- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbae4004810 msgr2=0x7fbb001f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbae4004810 0x7fbb001f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 -- 172.21.15.122:0/1155645637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb001f59d0 con 0x7fbb001f8a90 2024-09-17T05:32:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbafea00640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb001f8a90 0x7fbb0015d460 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbaec01cc90 tx=0x7fbaec0013a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbaf7400640 1 -- 172.21.15.122:0/1155645637 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0a5820 con 0x7fbb001f8a90 2024-09-17T05:32:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbaf7400640 1 -- 172.21.15.122:0/1155645637 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaec0a5ab0 con 0x7fbb001f8a90 2024-09-17T05:32:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbaf7400640 1 -- 172.21.15.122:0/1155645637 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec003600 con 0x7fbb001f8a90 2024-09-17T05:32:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb001f4df0 con 0x7fbb001f8a90 2024-09-17T05:32:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.928+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb001f53b0 con 0x7fbb001f8a90 2024-09-17T05:32:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.932+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb00069410 con 0x7fbb001f8a90 2024-09-17T05:32:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.932+0000 7fbaf7400640 1 -- 172.21.15.122:0/1155645637 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fbaec0037a0 con 0x7fbb001f8a90 2024-09-17T05:32:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.932+0000 7fbaf7400640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbae0073e90 0x7fbae0076350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:26.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.932+0000 7fbaf7400640 1 -- 172.21.15.122:0/1155645637 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7fbaec12c4b0 con 0x7fbb001f8a90 2024-09-17T05:32:26.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.932+0000 7fbaff400640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbae0073e90 0x7fbae0076350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:26.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.932+0000 7fbaff400640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbae0073e90 0x7fbae0076350 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0001ec0 tx=0x7fbaf0001bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:26.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:26.936+0000 7fbaf7400640 1 -- 172.21.15.122:0/1155645637 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbaec130e20 con 0x7fbb001f8a90 2024-09-17T05:32:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.108+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fbb000b2bf0 con 0x7fbb001f8a90 2024-09-17T05:32:27.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbaf7400640 1 -- 172.21.15.122:0/1155645637 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v36) ==== 71+0+0 (secure 0 0 0) 0x7fbaec0b4070 con 0x7fbb001f8a90 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbae0073e90 msgr2=0x7fbae0076350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbae0073e90 0x7fbae0076350 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0001ec0 tx=0x7fbaf0001bc0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb001f8a90 msgr2=0x7fbb0015d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb001f8a90 0x7fbb0015d460 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbaec01cc90 tx=0x7fbaec0013a0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 shutdown_connections 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbae0073e90 0x7fbae0076350 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb001f7e00 0x7fbb0015d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb001f8a90 0x7fbb0015d460 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 --2- 172.21.15.122:0/1155645637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbae4004810 0x7fbb001f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 >> 172.21.15.122:0/1155645637 conn(0x7fbb000b18f0 msgr2=0x7fbb0006fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 shutdown_connections 2024-09-17T05:32:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.112+0000 7fbb06800640 1 -- 172.21.15.122:0/1155645637 wait complete. 2024-09-17T05:32:27.132 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-17T05:32:27.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.272+0000 7fac31a00640 1 -- 172.21.15.122:0/3083833845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15b890 msgr2=0x7fac2c0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.272+0000 7fac31a00640 1 --2- 172.21.15.122:0/3083833845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15b890 0x7fac2c0ce010 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0cdd00 tx=0x7fac1c01c7e0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.272+0000 7fac31a00640 1 -- 172.21.15.122:0/3083833845 shutdown_connections 2024-09-17T05:32:27.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.272+0000 7fac31a00640 1 --2- 172.21.15.122:0/3083833845 >> v2:172.21.15.173:3300/0 conn(0x7fac2c15bc70 0x7fac2c0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.272+0000 7fac31a00640 1 --2- 172.21.15.122:0/3083833845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15b890 0x7fac2c0ce010 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.272+0000 7fac31a00640 1 -- 172.21.15.122:0/3083833845 >> 172.21.15.122:0/3083833845 conn(0x7fac2c06dd30 msgr2=0x7fac2c06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:27.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.272+0000 7fac31a00640 1 -- 172.21.15.122:0/3083833845 shutdown_connections 2024-09-17T05:32:27.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.272+0000 7fac31a00640 1 -- 172.21.15.122:0/3083833845 wait complete. 2024-09-17T05:32:27.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31a00640 1 Processor -- start 2024-09-17T05:32:27.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31a00640 1 -- start start 2024-09-17T05:32:27.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac2c06cc20 0x7fac2c0b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac2c15b890 0x7fac2c0c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15bc70 0x7fac2c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c168cd0 con 0x7fac2c15bc70 2024-09-17T05:32:27.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c168b50 con 0x7fac2c06cc20 2024-09-17T05:32:27.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fac2c168e50 con 0x7fac2c15b890 2024-09-17T05:32:27.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15bc70 0x7fac2c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15bc70 0x7fac2c0c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52246/0 (socket says 172.21.15.122:52246) 2024-09-17T05:32:27.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31000640 1 -- 172.21.15.122:0/1675804319 learned_addr learned my addr 172.21.15.122:0/1675804319 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:27.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac2be00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac2c06cc20 0x7fac2c0b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31000640 1 -- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac2c15b890 msgr2=0x7fac2c0c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31000640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac2c15b890 0x7fac2c0c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31000640 1 -- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac2c06cc20 msgr2=0x7fac2c0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31000640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac2c06cc20 0x7fac2c0b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.276+0000 7fac31000640 1 -- 172.21.15.122:0/1675804319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c0c4760 con 0x7fac2c15bc70 2024-09-17T05:32:27.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.280+0000 7fac31000640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15bc70 0x7fac2c0c6060 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fac2400c770 tx=0x7fac2400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:27.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.280+0000 7fac28c00640 1 -- 172.21.15.122:0/1675804319 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24017070 con 0x7fac2c15bc70 2024-09-17T05:32:27.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.280+0000 7fac28c00640 1 -- 172.21.15.122:0/1675804319 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac24005070 con 0x7fac2c15bc70 2024-09-17T05:32:27.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.280+0000 7fac28c00640 1 -- 172.21.15.122:0/1675804319 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240054c0 con 0x7fac2c15bc70 2024-09-17T05:32:27.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.280+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c0c4a40 con 0x7fac2c15bc70 2024-09-17T05:32:27.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.280+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c0c5000 con 0x7fac2c15bc70 2024-09-17T05:32:27.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.284+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c0cdff0 con 0x7fac2c15bc70 2024-09-17T05:32:27.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.284+0000 7fac28c00640 1 -- 172.21.15.122:0/1675804319 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fac2401a070 con 0x7fac2c15bc70 2024-09-17T05:32:27.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.284+0000 7fac28c00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fac08073eb0 0x7fac08076370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.284+0000 7fac28c00640 1 -- 172.21.15.122:0/1675804319 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7fac24092560 con 0x7fac2c15bc70 2024-09-17T05:32:27.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.284+0000 7fac2be00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fac08073eb0 0x7fac08076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.288+0000 7fac2be00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fac08073eb0 0x7fac08076370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0af000 tx=0x7fac1c01b020 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:27.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.292+0000 7fac28c00640 1 -- 172.21.15.122:0/1675804319 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fac2405ede0 con 0x7fac2c15bc70 2024-09-17T05:32:27.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.516+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7fac2c0713a0 con 0x7fac2c15bc70 2024-09-17T05:32:27.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.752+0000 7fac28c00640 1 -- 172.21.15.122:0/1675804319 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v325) ==== 167+0+0 (secure 0 0 0) 0x7fac2405e6e0 con 0x7fac2c15bc70 2024-09-17T05:32:27.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fac08073eb0 msgr2=0x7fac08076370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fac08073eb0 0x7fac08076370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0af000 tx=0x7fac1c01b020 comp rx=0 tx=0).stop 2024-09-17T05:32:27.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15bc70 msgr2=0x7fac2c0c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15bc70 0x7fac2c0c6060 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fac2400c770 tx=0x7fac2400cc30 comp rx=0 tx=0).stop 2024-09-17T05:32:27.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 shutdown_connections 2024-09-17T05:32:27.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fac08073eb0 0x7fac08076370 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac2c15bc70 0x7fac2c0c6060 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac2c15b890 0x7fac2c0c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 --2- 172.21.15.122:0/1675804319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac2c06cc20 0x7fac2c0b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 >> 172.21.15.122:0/1675804319 conn(0x7fac2c06dd30 msgr2=0x7fac2c0739a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:27.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 shutdown_connections 2024-09-17T05:32:27.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.756+0000 7fac31a00640 1 -- 172.21.15.122:0/1675804319 wait complete. 2024-09-17T05:32:27.777 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-17T05:32:27.778 INFO:tasks.cephfs_test_runner:Starting test: test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed in output of "ceph status" 2024-09-17T05:32:27.778 INFO:tasks.cephfs_test_runner:test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:32:27.778 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:32:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- 172.21.15.122:0/1960932872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 msgr2=0x7f4064003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 --2- 172.21.15.122:0/1960932872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 0x7f4064003eb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4070009b60 tx=0x7f407001d270 comp rx=0 tx=0).stop 2024-09-17T05:32:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4085400640 1 -- 172.21.15.122:0/1960932872 reap_dead start 2024-09-17T05:32:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- 172.21.15.122:0/1960932872 shutdown_connections 2024-09-17T05:32:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- 172.21.15.122:0/1960932872 >> 172.21.15.122:0/1960932872 conn(0x7f40800d1e30 msgr2=0x7f40800d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- 172.21.15.122:0/1960932872 shutdown_connections 2024-09-17T05:32:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- 172.21.15.122:0/1960932872 wait complete. 2024-09-17T05:32:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 Processor -- start 2024-09-17T05:32:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- start start 2024-09-17T05:32:27.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 0x7f40800c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40800c7e30 0x7f40801f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40801f4f00 0x7f408015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40800d0030 con 0x7f4064003a90 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f40800cfeb0 con 0x7f40800c7e30 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4087400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f40800afe20 con 0x7f40801f4f00 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4084a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 0x7f40800c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4084a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 0x7f40800c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52272/0 (socket says 172.21.15.122:52272) 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4084a00640 1 -- 172.21.15.122:0/84141008 learned_addr learned my addr 172.21.15.122:0/84141008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.920+0000 7f4077e00640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40800c7e30 0x7f40801f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4085400640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40801f4f00 0x7f408015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4084a00640 1 -- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40801f4f00 msgr2=0x7f408015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4084a00640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40801f4f00 0x7f408015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4084a00640 1 -- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40800c7e30 msgr2=0x7f40801f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4084a00640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40800c7e30 0x7f40801f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4084a00640 1 -- 172.21.15.122:0/84141008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40801f5410 con 0x7f4064003a90 2024-09-17T05:32:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4084a00640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 0x7f40800c5de0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f40700af040 tx=0x7f4070005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:27.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f407d600640 1 -- 172.21.15.122:0/84141008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700b1040 con 0x7f4064003a90 2024-09-17T05:32:27.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f407d600640 1 -- 172.21.15.122:0/84141008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40700bb070 con 0x7f4064003a90 2024-09-17T05:32:27.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40801f56f0 con 0x7f4064003a90 2024-09-17T05:32:27.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f407d600640 1 -- 172.21.15.122:0/84141008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070009730 con 0x7f4064003a90 2024-09-17T05:32:27.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40800c6320 con 0x7f4064003a90 2024-09-17T05:32:27.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40800af710 con 0x7f4064003a90 2024-09-17T05:32:27.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f407d600640 1 -- 172.21.15.122:0/84141008 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f40700046d0 con 0x7f4064003a90 2024-09-17T05:32:27.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.924+0000 7f407d600640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f405c073e60 0x7f405c076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:27.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.928+0000 7f407d600640 1 -- 172.21.15.122:0/84141008 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f407012d850 con 0x7f4064003a90 2024-09-17T05:32:27.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.928+0000 7f4077e00640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f405c073e60 0x7f405c076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:27.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.928+0000 7f4077e00640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f405c073e60 0x7f405c076320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4080072910 tx=0x7f4068001500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:27.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:27.928+0000 7f407d600640 1 -- 172.21.15.122:0/84141008 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4070016f80 con 0x7f4064003a90 2024-09-17T05:32:28.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.100+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40801f58c0 con 0x7f4064003a90 2024-09-17T05:32:28.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.100+0000 7f407d600640 1 -- 172.21.15.122:0/84141008 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13179 (secure 0 0 0) 0x7f40700f9d00 con 0x7f4064003a90 2024-09-17T05:32:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:28.102 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":66,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:32:26.751154+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.173:6836/204904815":"2024-09-18T05:32:23.740485+0000","172.21.15.173:6834/204904815":"2024-09-18T05:32:23.740485+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:32:28.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f405c073e60 msgr2=0x7f405c076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f405c073e60 0x7f405c076320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4080072910 tx=0x7f4068001500 comp rx=0 tx=0).stop 2024-09-17T05:32:28.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 msgr2=0x7f40800c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 0x7f40800c5de0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f40700af040 tx=0x7f4070005590 comp rx=0 tx=0).stop 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 shutdown_connections 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f405c073e60 0x7f405c076320 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40801f4f00 0x7f408015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40800c7e30 0x7f40801f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 --2- 172.21.15.122:0/84141008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4064003a90 0x7f40800c5de0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 >> 172.21.15.122:0/84141008 conn(0x7f40800d1e30 msgr2=0x7f40800b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 shutdown_connections 2024-09-17T05:32:28.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.104+0000 7f4087400640 1 -- 172.21.15.122:0/84141008 wait complete. 2024-09-17T05:32:28.125 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:32:28.125 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:32:28.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.268+0000 7f5113c00640 1 -- 172.21.15.122:0/3649100416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f0004010 msgr2=0x7f50f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.268+0000 7f5113c00640 1 --2- 172.21.15.122:0/3649100416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50f0004010 0x7f50f0004810 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f401f6a0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.268+0000 7f5111c00640 1 -- 172.21.15.122:0/3649100416 reap_dead start 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.268+0000 7f5113c00640 1 -- 172.21.15.122:0/3649100416 shutdown_connections 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.268+0000 7f5113c00640 1 -- 172.21.15.122:0/3649100416 >> 172.21.15.122:0/3649100416 conn(0x7f510c0cd740 msgr2=0x7f510c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.268+0000 7f5113c00640 1 -- 172.21.15.122:0/3649100416 shutdown_connections 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 -- 172.21.15.122:0/3649100416 wait complete. 2024-09-17T05:32:28.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 Processor -- start 2024-09-17T05:32:28.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 -- start start 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f0004010 0x7f510c0afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f510c0c4b10 0x7f510c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c157d20 0x7f510c0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f510c06d250 con 0x7f510c157d20 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f510c06d0d0 con 0x7f510c0c4b10 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f510c06d3d0 con 0x7f50f0004010 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c157d20 0x7f510c0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c157d20 0x7f510c0b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52286/0 (socket says 172.21.15.122:52286) 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 -- 172.21.15.122:0/1092784006 learned_addr learned my addr 172.21.15.122:0/1092784006 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:28.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5103e00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f510c0c4b10 0x7f510c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111200640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f0004010 0x7f510c0afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 -- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f0004010 msgr2=0x7f510c0afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f0004010 0x7f510c0afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 -- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f510c0c4b10 msgr2=0x7f510c0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f510c0c4b10 0x7f510c0b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 -- 172.21.15.122:0/1092784006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1539a0 con 0x7f510c157d20 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5103e00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f510c0c4b10 0x7f510c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5111c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c157d20 0x7f510c0b65e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f510c0b0350 tx=0x7f5104006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5109600640 1 -- 172.21.15.122:0/1092784006 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104015020 con 0x7f510c157d20 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5109600640 1 -- 172.21.15.122:0/1092784006 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5104011900 con 0x7f510c157d20 2024-09-17T05:32:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5109600640 1 -- 172.21.15.122:0/1092784006 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104011d50 con 0x7f510c157d20 2024-09-17T05:32:28.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c15a630 con 0x7f510c157d20 2024-09-17T05:32:28.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.272+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c153dc0 con 0x7f510c157d20 2024-09-17T05:32:28.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.276+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c0cccd0 con 0x7f510c157d20 2024-09-17T05:32:28.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.276+0000 7f5109600640 1 -- 172.21.15.122:0/1092784006 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f5104018020 con 0x7f510c157d20 2024-09-17T05:32:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.276+0000 7f5109600640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50e8073e60 0x7f50e8076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.276+0000 7f5109600640 1 -- 172.21.15.122:0/1092784006 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f51040933b0 con 0x7f510c157d20 2024-09-17T05:32:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.276+0000 7f5111200640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50e8073e60 0x7f50e8076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.280+0000 7f5111200640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50e8073e60 0x7f50e8076320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0099f0 tx=0x7f50fc0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:28.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.280+0000 7f5109600640 1 -- 172.21.15.122:0/1092784006 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f510401f030 con 0x7f510c157d20 2024-09-17T05:32:28.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.448+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f510c073a40 con 0x7f510c157d20 2024-09-17T05:32:28.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.448+0000 7f5109600640 1 -- 172.21.15.122:0/1092784006 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f510405feb0 con 0x7f510c157d20 2024-09-17T05:32:28.450 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50e8073e60 msgr2=0x7f50e8076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50e8073e60 0x7f50e8076320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0099f0 tx=0x7f50fc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c157d20 msgr2=0x7f510c0b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c157d20 0x7f510c0b65e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f510c0b0350 tx=0x7f5104006430 comp rx=0 tx=0).stop 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 shutdown_connections 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f50e8073e60 0x7f50e8076320 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f510c157d20 0x7f510c0b65e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f510c0c4b10 0x7f510c0b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 --2- 172.21.15.122:0/1092784006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50f0004010 0x7f510c0afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 >> 172.21.15.122:0/1092784006 conn(0x7f510c0cd740 msgr2=0x7f510c0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 shutdown_connections 2024-09-17T05:32:28.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.452+0000 7f5113c00640 1 -- 172.21.15.122:0/1092784006 wait complete. 2024-09-17T05:32:28.472 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-17T05:32:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 -- 172.21.15.122:0/1257182560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180b2590 msgr2=0x7fb3180b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 --2- 172.21.15.122:0/1257182560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180b2590 0x7fb3180b2a00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb310009bc0 tx=0x7fb31001cb10 comp rx=0 tx=0).stop 2024-09-17T05:32:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 -- 172.21.15.122:0/1257182560 shutdown_connections 2024-09-17T05:32:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 --2- 172.21.15.122:0/1257182560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180b2590 0x7fb3180b2a00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 --2- 172.21.15.122:0/1257182560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3180d01b0 0x7fb3180cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 -- 172.21.15.122:0/1257182560 >> 172.21.15.122:0/1257182560 conn(0x7fb31809aa10 msgr2=0x7fb31809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 -- 172.21.15.122:0/1257182560 shutdown_connections 2024-09-17T05:32:28.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 -- 172.21.15.122:0/1257182560 wait complete. 2024-09-17T05:32:28.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.612+0000 7fb317200640 1 Processor -- start 2024-09-17T05:32:28.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 -- start start 2024-09-17T05:32:28.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3180b2590 0x7fb3180b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180d01b0 0x7fb31815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb31815f000 0x7fb3180ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb318168ea0 con 0x7fb3180d01b0 2024-09-17T05:32:28.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb318168d20 con 0x7fb31815f000 2024-09-17T05:32:28.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb318169020 con 0x7fb3180b2590 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180d01b0 0x7fb31815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180d01b0 0x7fb31815d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52304/0 (socket says 172.21.15.122:52304) 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 -- 172.21.15.122:0/4284483600 learned_addr learned my addr 172.21.15.122:0/4284483600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb316800640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb31815f000 0x7fb3180ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 -- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3180b2590 msgr2=0x7fb3180b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3180b2590 0x7fb3180b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 -- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb31815f000 msgr2=0x7fb3180ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb31815f000 0x7fb3180ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 -- 172.21.15.122:0/4284483600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3180b6960 con 0x7fb3180d01b0 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb316800640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb31815f000 0x7fb3180ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30fe00640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180d01b0 0x7fb31815d080 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb30000e9f0 tx=0x7fb30000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:28.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30e000640 1 -- 172.21.15.122:0/4284483600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30000cd80 con 0x7fb3180d01b0 2024-09-17T05:32:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30e000640 1 -- 172.21.15.122:0/4284483600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb300010430 con 0x7fb3180d01b0 2024-09-17T05:32:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb30e000640 1 -- 172.21.15.122:0/4284483600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300010860 con 0x7fb3180d01b0 2024-09-17T05:32:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3180c9a70 con 0x7fb3180d01b0 2024-09-17T05:32:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3180b4aa0 con 0x7fb3180d01b0 2024-09-17T05:32:28.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.616+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3180c9e60 con 0x7fb3180d01b0 2024-09-17T05:32:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.620+0000 7fb30e000640 1 -- 172.21.15.122:0/4284483600 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fb300010a00 con 0x7fb3180d01b0 2024-09-17T05:32:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.620+0000 7fb30e000640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:28.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.620+0000 7fb30e000640 1 -- 172.21.15.122:0/4284483600 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7fb3000937d0 con 0x7fb3180d01b0 2024-09-17T05:32:28.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.620+0000 7fb315e00640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:28.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.620+0000 7fb315e00640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb308007fd0 tx=0x7fb308007940 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:28.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.624+0000 7fb30e000640 1 -- 172.21.15.122:0/4284483600 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb300060350 con 0x7fb3180d01b0 2024-09-17T05:32:28.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:28.852+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7fb3180cefb0 con 0x7fb3180d01b0 2024-09-17T05:32:29.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.760+0000 7fb30e000640 1 -- 172.21.15.122:0/4284483600 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v327) ==== 170+0+0 (secure 0 0 0) 0x7fb30005fc50 con 0x7fb3180d01b0 2024-09-17T05:32:29.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 msgr2=0x7fb2f4076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb308007fd0 tx=0x7fb308007940 comp rx=0 tx=0).stop 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180d01b0 msgr2=0x7fb31815d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180d01b0 0x7fb31815d080 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb30000e9f0 tx=0x7fb30000eeb0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 shutdown_connections 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2f4073e60 0x7fb2f4076320 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb31815f000 0x7fb3180ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3180d01b0 0x7fb31815d080 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 --2- 172.21.15.122:0/4284483600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3180b2590 0x7fb3180b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 >> 172.21.15.122:0/4284483600 conn(0x7fb31809aa10 msgr2=0x7fb31815dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 shutdown_connections 2024-09-17T05:32:29.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.764+0000 7fb317200640 1 -- 172.21.15.122:0/4284483600 wait complete. 2024-09-17T05:32:29.785 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:32:29.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.928+0000 7f73c0600640 1 -- 172.21.15.122:0/2773357401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 msgr2=0x7f73a0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.928+0000 7f73c0600640 1 --2- 172.21.15.122:0/2773357401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 0x7f73a0003e60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009b60 tx=0x7f73ac01d240 comp rx=0 tx=0).stop 2024-09-17T05:32:29.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.928+0000 7f73be600640 1 -- 172.21.15.122:0/2773357401 reap_dead start 2024-09-17T05:32:29.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.928+0000 7f73c0600640 1 -- 172.21.15.122:0/2773357401 shutdown_connections 2024-09-17T05:32:29.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.928+0000 7f73c0600640 1 -- 172.21.15.122:0/2773357401 >> 172.21.15.122:0/2773357401 conn(0x7f73b80d1e30 msgr2=0x7f73b80d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:29.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.928+0000 7f73c0600640 1 -- 172.21.15.122:0/2773357401 shutdown_connections 2024-09-17T05:32:29.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.928+0000 7f73c0600640 1 -- 172.21.15.122:0/2773357401 wait complete. 2024-09-17T05:32:29.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 Processor -- start 2024-09-17T05:32:29.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 -- start start 2024-09-17T05:32:29.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 0x7f73b8148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73b8147940 0x7f73b80cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73b80cb8b0 0x7f73b80cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80b2290 con 0x7f73a0003a40 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f73b80b2110 con 0x7f73b8147940 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f73b8168490 con 0x7f73b80cb8b0 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 0x7f73b8148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 0x7f73b8148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52310/0 (socket says 172.21.15.122:52310) 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 -- 172.21.15.122:0/452056340 learned_addr learned my addr 172.21.15.122:0/452056340 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73be600640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73b80cb8b0 0x7f73b80cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bd200640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73b8147940 0x7f73b80cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 -- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73b80cb8b0 msgr2=0x7f73b80cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73b80cb8b0 0x7f73b80cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 -- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73b8147940 msgr2=0x7f73b80cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73b8147940 0x7f73b80cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 -- 172.21.15.122:0/452056340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b80ca8b0 con 0x7f73a0003a40 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73bdc00640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 0x7f73b8148250 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0af040 tx=0x7f73ac005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73b5600640 1 -- 172.21.15.122:0/452056340 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0b1040 con 0x7f73a0003a40 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73b5600640 1 -- 172.21.15.122:0/452056340 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73ac0bb070 con 0x7f73a0003a40 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73b5600640 1 -- 172.21.15.122:0/452056340 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac004290 con 0x7f73a0003a40 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b80ce5e0 con 0x7f73a0003a40 2024-09-17T05:32:29.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b80cae50 con 0x7f73a0003a40 2024-09-17T05:32:29.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.932+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b80b0130 con 0x7f73a0003a40 2024-09-17T05:32:29.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.936+0000 7f73b5600640 1 -- 172.21.15.122:0/452056340 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f73ac0046d0 con 0x7f73a0003a40 2024-09-17T05:32:29.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.936+0000 7f73b5600640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7398073e60 0x7f7398076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:29.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.936+0000 7f73b5600640 1 -- 172.21.15.122:0/452056340 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f73ac12dab0 con 0x7f73a0003a40 2024-09-17T05:32:29.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.936+0000 7f73bd200640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7398073e60 0x7f7398076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:29.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.936+0000 7f73bd200640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7398073e60 0x7f7398076320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f73b81f9110 tx=0x7f73a4006b30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:29.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:29.940+0000 7f73b5600640 1 -- 172.21.15.122:0/452056340 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f73ac016f80 con 0x7f73a0003a40 2024-09-17T05:32:30.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.112+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f73b80b1b80 con 0x7f73a0003a40 2024-09-17T05:32:30.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.112+0000 7f73b5600640 1 -- 172.21.15.122:0/452056340 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) ==== 59+0+1998 (secure 0 0 0) 0x7f73ac0fa5b0 con 0x7f73a0003a40 2024-09-17T05:32:30.113 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:32:30.114 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:32:30.115 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:32:30.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7398073e60 msgr2=0x7f7398076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7398073e60 0x7f7398076320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f73b81f9110 tx=0x7f73a4006b30 comp rx=0 tx=0).stop 2024-09-17T05:32:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 msgr2=0x7f73b8148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 0x7f73b8148250 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0af040 tx=0x7f73ac005590 comp rx=0 tx=0).stop 2024-09-17T05:32:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 shutdown_connections 2024-09-17T05:32:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7398073e60 0x7f7398076320 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73b80cb8b0 0x7f73b80cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73b8147940 0x7f73b80cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 --2- 172.21.15.122:0/452056340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73a0003a40 0x7f73b8148250 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 >> 172.21.15.122:0/452056340 conn(0x7f73b80d1e30 msgr2=0x7f73b80cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 shutdown_connections 2024-09-17T05:32:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.116+0000 7f73c0600640 1 -- 172.21.15.122:0/452056340 wait complete. 2024-09-17T05:32:30.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:32:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 -- 172.21.15.122:0/321676375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f0c0cd100 msgr2=0x7f0f0c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 --2- 172.21.15.122:0/321676375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f0c0cd100 0x7f0f0c06e7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009840 tx=0x7f0ef801c7f0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 -- 172.21.15.122:0/321676375 shutdown_connections 2024-09-17T05:32:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 --2- 172.21.15.122:0/321676375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f0c0cd100 0x7f0f0c06e7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 --2- 172.21.15.122:0/321676375 >> v2:172.21.15.173:3300/0 conn(0x7f0f0c0b43d0 0x7f0f0c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 -- 172.21.15.122:0/321676375 >> 172.21.15.122:0/321676375 conn(0x7f0f0c0b18f0 msgr2=0x7f0f0c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 -- 172.21.15.122:0/321676375 shutdown_connections 2024-09-17T05:32:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 -- 172.21.15.122:0/321676375 wait complete. 2024-09-17T05:32:30.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 Processor -- start 2024-09-17T05:32:30.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 -- start start 2024-09-17T05:32:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f0c0b2bf0 0x7f0f0c0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.280+0000 7f0f11c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f0c0b43d0 0x7f0f0c152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f11c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f0c0cd100 0x7f0f0c152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f11c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c167440 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f11c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1672c0 con 0x7f0f0c0b2bf0 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f11c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0f0c1675c0 con 0x7f0f0c0cd100 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f0c0b43d0 0x7f0f0c152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f0c0b43d0 0x7f0f0c152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52324/0 (socket says 172.21.15.122:52324) 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 -- 172.21.15.122:0/1519536605 learned_addr learned my addr 172.21.15.122:0/1519536605 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0aa00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f0c0b2bf0 0x7f0f0c0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0b400640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f0c0cd100 0x7f0f0c152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 -- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f0c0cd100 msgr2=0x7f0f0c152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f0c0cd100 0x7f0f0c152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 -- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f0c0b2bf0 msgr2=0x7f0f0c0adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f0c0b2bf0 0x7f0f0c0adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 -- 172.21.15.122:0/1519536605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c155650 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f0a000640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f0c0b43d0 0x7f0f0c152400 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009790 tx=0x7f0ef80a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f02a00640 1 -- 172.21.15.122:0/1519536605 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80b2040 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f02a00640 1 -- 172.21.15.122:0/1519536605 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ef8004470 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f02a00640 1 -- 172.21.15.122:0/1519536605 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8005280 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c155930 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c155ef0 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0c069410 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.284+0000 7f0f02a00640 1 -- 172.21.15.122:0/1519536605 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f0ef8005420 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.288+0000 7f0f02a00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eec073e60 0x7f0eec076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.288+0000 7f0f02a00640 1 -- 172.21.15.122:0/1519536605 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f0ef812c360 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.288+0000 7f0f0aa00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eec073e60 0x7f0eec076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.288+0000 7f0f0aa00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eec073e60 0x7f0eec076320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00a710 tx=0x7f0efc00a2f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:30.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.292+0000 7f0f02a00640 1 -- 172.21.15.122:0/1519536605 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0ef8016410 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.456+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0f0c0b47b0 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f02a00640 1 -- 172.21.15.122:0/1519536605 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f0ef80f8de0 con 0x7f0f0c0b43d0 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eec073e60 msgr2=0x7f0eec076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eec073e60 0x7f0eec076320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00a710 tx=0x7f0efc00a2f0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f0c0b43d0 msgr2=0x7f0f0c152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f0c0b43d0 0x7f0f0c152400 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009790 tx=0x7f0ef80a5730 comp rx=0 tx=0).stop 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 shutdown_connections 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0eec073e60 0x7f0eec076320 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f0c0cd100 0x7f0f0c152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f0c0b43d0 0x7f0f0c152400 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 --2- 172.21.15.122:0/1519536605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f0c0b2bf0 0x7f0f0c0adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 >> 172.21.15.122:0/1519536605 conn(0x7f0f0c0b18f0 msgr2=0x7f0f0c0cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:30.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 shutdown_connections 2024-09-17T05:32:30.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.460+0000 7f0f11c00640 1 -- 172.21.15.122:0/1519536605 wait complete. 2024-09-17T05:32:30.482 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:32:30.482 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:32:30.554 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:32:30.660 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:32:30.660 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:32:30.660 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:30.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:30.669 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:32:30.669 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:30.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:32:30.678 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:32:30.678 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:30.718 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:32:30.718 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:32:30.719 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:30.730 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:32:30.730 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:32:30.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:30.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.900+0000 7f0984c00640 1 -- 172.21.15.122:0/2537417921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0964004da0 msgr2=0x7f0964005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.900+0000 7f0984c00640 1 --2- 172.21.15.122:0/2537417921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0964004da0 0x7f0964005220 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0978001270 tx=0x7f097801e990 comp rx=0 tx=0).stop 2024-09-17T05:32:30.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.900+0000 7f097e800640 1 -- 172.21.15.122:0/2537417921 reap_dead start 2024-09-17T05:32:30.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.900+0000 7f0984c00640 1 -- 172.21.15.122:0/2537417921 shutdown_connections 2024-09-17T05:32:30.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.900+0000 7f0984c00640 1 -- 172.21.15.122:0/2537417921 >> 172.21.15.122:0/2537417921 conn(0x7f09800b18f0 msgr2=0x7f09800b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:30.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.900+0000 7f0984c00640 1 -- 172.21.15.122:0/2537417921 shutdown_connections 2024-09-17T05:32:30.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.900+0000 7f0984c00640 1 -- 172.21.15.122:0/2537417921 wait complete. 2024-09-17T05:32:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 Processor -- start 2024-09-17T05:32:30.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 -- start start 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0964004da0 0x7f09801567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09800cae50 0x7f0980158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09800cb2d0 0x7f09801591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f09800acec0 con 0x7f09800cb2d0 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f09800acd40 con 0x7f0964004da0 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f09800ad040 con 0x7f09800cae50 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09800cb2d0 0x7f09801591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09800cb2d0 0x7f09801591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52348/0 (socket says 172.21.15.122:52348) 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 -- 172.21.15.122:0/2543915888 learned_addr learned my addr 172.21.15.122:0/2543915888 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:30.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097d400640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09800cae50 0x7f0980158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097de00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0964004da0 0x7f09801567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 -- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09800cae50 msgr2=0x7f0980158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09800cae50 0x7f0980158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 -- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0964004da0 msgr2=0x7f09801567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0964004da0 0x7f09801567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:30.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 -- 172.21.15.122:0/2543915888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09801f4630 con 0x7f09800cb2d0 2024-09-17T05:32:30.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f097e800640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09800cb2d0 0x7f09801591b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f097801f230 tx=0x7f097801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:30.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0976000640 1 -- 172.21.15.122:0/2543915888 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09780012b0 con 0x7f09800cb2d0 2024-09-17T05:32:30.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0976000640 1 -- 172.21.15.122:0/2543915888 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09780b4030 con 0x7f09800cb2d0 2024-09-17T05:32:30.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0976000640 1 -- 172.21.15.122:0/2543915888 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978006930 con 0x7f09800cb2d0 2024-09-17T05:32:30.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098015be90 con 0x7f09800cb2d0 2024-09-17T05:32:30.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098015b3b0 con 0x7f09800cb2d0 2024-09-17T05:32:30.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.904+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0980069410 con 0x7f09800cb2d0 2024-09-17T05:32:30.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.908+0000 7f0976000640 1 -- 172.21.15.122:0/2543915888 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f09780a79c0 con 0x7f09800cb2d0 2024-09-17T05:32:30.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.908+0000 7f0976000640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0960073e60 0x7f0960076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:30.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.908+0000 7f0976000640 1 -- 172.21.15.122:0/2543915888 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f097812c710 con 0x7f09800cb2d0 2024-09-17T05:32:30.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.908+0000 7f097de00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0960073e60 0x7f0960076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:30.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.908+0000 7f097de00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0960073e60 0x7f0960076320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0970004690 tx=0x7f0970001550 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:30.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:30.912+0000 7f0976000640 1 -- 172.21.15.122:0/2543915888 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f09780186a0 con 0x7f09800cb2d0 2024-09-17T05:32:31.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.112+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09800b1f10 con 0x7f09800cb2d0 2024-09-17T05:32:31.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.112+0000 7f0976000640 1 -- 172.21.15.122:0/2543915888 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3923 (secure 0 0 0) 0x7f09780f8cd0 con 0x7f09800cb2d0 2024-09-17T05:32:31.114 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:31.115 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":39,"btime":"2024-09-17T05:32:24:742816+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110418608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110418608},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110418608}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1707103346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1707103346},{"type":"v1","addr":"172.21.15.173:6836","nonce":1707103346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1187468185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1187468185},{"type":"v1","addr":"172.21.15.122:6835","nonce":1187468185}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5925,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/3667127301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3667127301},{"type":"v1","addr":"172.21.15.122:6834","nonce":3667127301}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-09-17T05:32:31.115 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 39 2024-09-17T05:32:31.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0960073e60 msgr2=0x7f0960076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0960073e60 0x7f0960076320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0970004690 tx=0x7f0970001550 comp rx=0 tx=0).stop 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09800cb2d0 msgr2=0x7f09801591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09800cb2d0 0x7f09801591b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f097801f230 tx=0x7f097801ebe0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 shutdown_connections 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0960073e60 0x7f0960076320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09800cb2d0 0x7f09801591b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09800cae50 0x7f0980158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 --2- 172.21.15.122:0/2543915888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0964004da0 0x7f09801567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 >> 172.21.15.122:0/2543915888 conn(0x7f09800b18f0 msgr2=0x7f09800aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 shutdown_connections 2024-09-17T05:32:31.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:31.116+0000 7f0984c00640 1 -- 172.21.15.122:0/2543915888 wait complete. 2024-09-17T05:32:31.137 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:32:31.137 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:32:31.137 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:32:31.138 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:32:31.138 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:32:31.138 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:32:31.138 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:32:31.138 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:32:31.138 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:32:31.138 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:32:31.139 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:32:31.139 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:32:31.139 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:32:31.139 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:32:31.139 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:32:31.139 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:32:31.151 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:32:31.151 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:32:31.154 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:32:31.154 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:32:31.155 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:32:31.155 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:32:31.197 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:32:31.197 DEBUG:teuthology.parallel:result is None 2024-09-17T05:32:31.203 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:32:31.203 DEBUG:teuthology.parallel:result is None 2024-09-17T05:32:31.205 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:32:31.206 DEBUG:teuthology.parallel:result is None 2024-09-17T05:32:31.279 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:32:31.297 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:32:31.298 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:32:32.749 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:32:32.753 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:32:37.151 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:32:37.152 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:32:37.156 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:32:37.156 DEBUG:teuthology.parallel:result is None 2024-09-17T05:32:37.156 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:32:37.251 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:32:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 -- 172.21.15.122:0/2471235236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 msgr2=0x7fe488073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 --2- 172.21.15.122:0/2471235236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 0x7fe488073d10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe488074660 tx=0x7fe48001ccf0 comp rx=0 tx=0).stop 2024-09-17T05:32:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 -- 172.21.15.122:0/2471235236 shutdown_connections 2024-09-17T05:32:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 --2- 172.21.15.122:0/2471235236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 0x7fe488073d10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 --2- 172.21.15.122:0/2471235236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe48815f2d0 0x7fe48815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 --2- 172.21.15.122:0/2471235236 >> v2:172.21.15.173:3300/0 conn(0x7fe4880affd0 0x7fe4880cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 -- 172.21.15.122:0/2471235236 >> 172.21.15.122:0/2471235236 conn(0x7fe4880d0d50 msgr2=0x7fe4880d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 -- 172.21.15.122:0/2471235236 shutdown_connections 2024-09-17T05:32:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.312+0000 7fe48d600640 1 -- 172.21.15.122:0/2471235236 wait complete. 2024-09-17T05:32:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 Processor -- start 2024-09-17T05:32:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 -- start start 2024-09-17T05:32:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 0x7fe488141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4880affd0 0x7fe48814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe48815f2d0 0x7fe48814c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4880cc3b0 con 0x7fe488073880 2024-09-17T05:32:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4880af840 con 0x7fe4880affd0 2024-09-17T05:32:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe4880cc530 con 0x7fe48815f2d0 2024-09-17T05:32:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 0x7fe488141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 0x7fe488141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42156/0 (socket says 172.21.15.122:42156) 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 -- 172.21.15.122:0/871535694 learned_addr learned my addr 172.21.15.122:0/871535694 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe47e000640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4880affd0 0x7fe48814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe487400640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe48815f2d0 0x7fe48814c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 -- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe48815f2d0 msgr2=0x7fe48814c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe48815f2d0 0x7fe48814c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 -- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4880affd0 msgr2=0x7fe48814c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4880affd0 0x7fe48814c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 -- 172.21.15.122:0/871535694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48815c750 con 0x7fe488073880 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe486a00640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 0x7fe488141f60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe47800d770 tx=0x7fe47800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe47f400640 1 -- 172.21.15.122:0/871535694 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478017070 con 0x7fe488073880 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe47f400640 1 -- 172.21.15.122:0/871535694 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe478009cc0 con 0x7fe488073880 2024-09-17T05:32:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe47f400640 1 -- 172.21.15.122:0/871535694 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4780106c0 con 0x7fe488073880 2024-09-17T05:32:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe488142750 con 0x7fe488073880 2024-09-17T05:32:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.316+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4880cabc0 con 0x7fe488073880 2024-09-17T05:32:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.320+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48806ccc0 con 0x7fe488073880 2024-09-17T05:32:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.320+0000 7fe47f400640 1 -- 172.21.15.122:0/871535694 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fe47801a070 con 0x7fe488073880 2024-09-17T05:32:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.320+0000 7fe47f400640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe464073e60 0x7fe464076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.320+0000 7fe47f400640 1 -- 172.21.15.122:0/871535694 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4999+0+0 (secure 0 0 0) 0x7fe478092ea0 con 0x7fe488073880 2024-09-17T05:32:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.320+0000 7fe47e000640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe464073e60 0x7fe464076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.320+0000 7fe47e000640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe464073e60 0x7fe464076320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe4881faef0 tx=0x7fe47000b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.324+0000 7fe47f400640 1 -- 172.21.15.122:0/871535694 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe47805f0d0 con 0x7fe488073880 2024-09-17T05:32:37.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:37.500+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe488150360 con 0x7fe488073880 2024-09-17T05:32:38.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe47f400640 1 -- 172.21.15.122:0/871535694 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v71) ==== 97+0+0 (secure 0 0 0) 0x7fe478063b00 con 0x7fe488073880 2024-09-17T05:32:38.788 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:32:38.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe464073e60 msgr2=0x7fe464076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:38.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe464073e60 0x7fe464076320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe4881faef0 tx=0x7fe47000b390 comp rx=0 tx=0).stop 2024-09-17T05:32:38.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 msgr2=0x7fe488141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:38.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 0x7fe488141f60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe47800d770 tx=0x7fe47800dc30 comp rx=0 tx=0).stop 2024-09-17T05:32:38.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 shutdown_connections 2024-09-17T05:32:38.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe464073e60 0x7fe464076320 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:38.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe48815f2d0 0x7fe48814c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:38.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4880affd0 0x7fe48814c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:38.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 --2- 172.21.15.122:0/871535694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe488073880 0x7fe488141f60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:38.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.788+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 >> 172.21.15.122:0/871535694 conn(0x7fe4880d0d50 msgr2=0x7fe48814f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:38.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.792+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 shutdown_connections 2024-09-17T05:32:38.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.792+0000 7fe48d600640 1 -- 172.21.15.122:0/871535694 wait complete. 2024-09-17T05:32:38.815 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:32:38.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4cdc00640 1 -- 172.21.15.122:0/2066669772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 msgr2=0x7fe4ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:38.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4cdc00640 1 --2- 172.21.15.122:0/2066669772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 0x7fe4ac00b9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ec0 tx=0x7fe4c001d230 comp rx=0 tx=0).stop 2024-09-17T05:32:38.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4c7400640 1 -- 172.21.15.122:0/2066669772 reap_dead start 2024-09-17T05:32:38.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4cdc00640 1 -- 172.21.15.122:0/2066669772 shutdown_connections 2024-09-17T05:32:38.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4cdc00640 1 -- 172.21.15.122:0/2066669772 >> 172.21.15.122:0/2066669772 conn(0x7fe4c806dd30 msgr2=0x7fe4c806e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:38.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4cdc00640 1 -- 172.21.15.122:0/2066669772 shutdown_connections 2024-09-17T05:32:38.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4cdc00640 1 -- 172.21.15.122:0/2066669772 wait complete. 2024-09-17T05:32:38.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4cdc00640 1 Processor -- start 2024-09-17T05:32:38.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.960+0000 7fe4cdc00640 1 -- start start 2024-09-17T05:32:38.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 0x7fe4c81f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:38.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4c8071090 0x7fe4c80ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:38.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4c80cf240 0x7fe4c815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:38.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c815d550 con 0x7fe4ac0051d0 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c815d3d0 con 0x7fe4c80cf240 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c815d6d0 con 0x7fe4c8071090 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 0x7fe4c81f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 0x7fe4c81f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42174/0 (socket says 172.21.15.122:42174) 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 -- 172.21.15.122:0/3599139428 learned_addr learned my addr 172.21.15.122:0/3599139428 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c7400640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4c80cf240 0x7fe4c815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4be000640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4c8071090 0x7fe4c80ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 -- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4c8071090 msgr2=0x7fe4c80ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4c8071090 0x7fe4c80ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 -- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4c80cf240 msgr2=0x7fe4c815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4c80cf240 0x7fe4c815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 -- 172.21.15.122:0/3599139428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c80d0d30 con 0x7fe4ac0051d0 2024-09-17T05:32:38.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c7400640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4c80cf240 0x7fe4c815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:38.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4c6a00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 0x7fe4c81f4360 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800f660 tx=0x7fe4b800fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:38.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4bf400640 1 -- 172.21.15.122:0/3599139428 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b800baa0 con 0x7fe4ac0051d0 2024-09-17T05:32:38.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4bf400640 1 -- 172.21.15.122:0/3599139428 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4b800bd30 con 0x7fe4ac0051d0 2024-09-17T05:32:38.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c80d0fd0 con 0x7fe4ac0051d0 2024-09-17T05:32:38.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4bf400640 1 -- 172.21.15.122:0/3599139428 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80116a0 con 0x7fe4ac0051d0 2024-09-17T05:32:38.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c80d1590 con 0x7fe4ac0051d0 2024-09-17T05:32:38.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.964+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c80b4710 con 0x7fe4ac0051d0 2024-09-17T05:32:38.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.968+0000 7fe4bf400640 1 -- 172.21.15.122:0/3599139428 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fe4b8019020 con 0x7fe4ac0051d0 2024-09-17T05:32:38.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.968+0000 7fe4bf400640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4a4073e60 0x7fe4a4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:38.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.968+0000 7fe4bf400640 1 -- 172.21.15.122:0/3599139428 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4913+0+0 (secure 0 0 0) 0x7fe4b8093360 con 0x7fe4ac0051d0 2024-09-17T05:32:38.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.968+0000 7fe4be000640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4a4073e60 0x7fe4a4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:38.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.968+0000 7fe4be000640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4a4073e60 0x7fe4a4076320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe4c81540e0 tx=0x7fe4b0001cc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:38.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:38.972+0000 7fe4bf400640 1 -- 172.21.15.122:0/3599139428 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe4b805f810 con 0x7fe4ac0051d0 2024-09-17T05:32:39.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe4c80cce00 con 0x7fe4ac0051d0 2024-09-17T05:32:39.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4bf400640 1 -- 172.21.15.122:0/3599139428 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fe4b8064240 con 0x7fe4ac0051d0 2024-09-17T05:32:39.200 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:32:39.202 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.205 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:32:39.205 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:32:39.205 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.205 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:32:39.205 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:32:39.205 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.207 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:32:39.207 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:32:39.207 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.207 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:32:39.207 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:32:39.207 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.208 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:32:39.208 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:32:39.209 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.209 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:32:39.209 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:32:39.209 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.209 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:32:39.211 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow r", 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:32:39.213 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:32:39.214 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.215 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:32:39.216 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:32:39.217 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:32:39.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4a4073e60 msgr2=0x7fe4a4076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4a4073e60 0x7fe4a4076320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe4c81540e0 tx=0x7fe4b0001cc0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 msgr2=0x7fe4c81f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 0x7fe4c81f4360 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800f660 tx=0x7fe4b800fb20 comp rx=0 tx=0).stop 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 shutdown_connections 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4a4073e60 0x7fe4a4076320 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4c80cf240 0x7fe4c815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4c8071090 0x7fe4c80ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 --2- 172.21.15.122:0/3599139428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4ac0051d0 0x7fe4c81f4360 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 >> 172.21.15.122:0/3599139428 conn(0x7fe4c806dd30 msgr2=0x7fe4c80b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 shutdown_connections 2024-09-17T05:32:39.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.200+0000 7fe4cdc00640 1 -- 172.21.15.122:0/3599139428 wait complete. 2024-09-17T05:32:39.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:32:39.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.364+0000 7fa78c600640 1 -- 172.21.15.122:0/3306296307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7840add50 msgr2=0x7fa78415cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.364+0000 7fa78c600640 1 --2- 172.21.15.122:0/3306296307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7840add50 0x7fa78415cbd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa78415c150 tx=0x7fa77001c800 comp rx=0 tx=0).stop 2024-09-17T05:32:39.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.364+0000 7fa78c600640 1 -- 172.21.15.122:0/3306296307 shutdown_connections 2024-09-17T05:32:39.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.364+0000 7fa78c600640 1 --2- 172.21.15.122:0/3306296307 >> v2:172.21.15.173:3300/0 conn(0x7fa78406cc20 0x7fa78406d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.364+0000 7fa78c600640 1 --2- 172.21.15.122:0/3306296307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7840add50 0x7fa78415cbd0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.364+0000 7fa78c600640 1 -- 172.21.15.122:0/3306296307 >> 172.21.15.122:0/3306296307 conn(0x7fa7840b2890 msgr2=0x7fa7840b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:39.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.364+0000 7fa78c600640 1 -- 172.21.15.122:0/3306296307 shutdown_connections 2024-09-17T05:32:39.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.364+0000 7fa78c600640 1 -- 172.21.15.122:0/3306296307 wait complete. 2024-09-17T05:32:39.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 Processor -- start 2024-09-17T05:32:39.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 -- start start 2024-09-17T05:32:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa78406cc20 0x7fa7841f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7840add50 0x7fa784150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7841561c0 0x7fa78415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa784168c60 con 0x7fa7841561c0 2024-09-17T05:32:39.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa784168ae0 con 0x7fa7840add50 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa784168de0 con 0x7fa78406cc20 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7841561c0 0x7fa78415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7841561c0 0x7fa78415ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42192/0 (socket says 172.21.15.122:42192) 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 -- 172.21.15.122:0/1646970226 learned_addr learned my addr 172.21.15.122:0/1646970226 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa789200640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7840add50 0x7fa784150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa789c00640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa78406cc20 0x7fa7841f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 -- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa78406cc20 msgr2=0x7fa7841f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa78406cc20 0x7fa7841f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 -- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7840add50 msgr2=0x7fa784150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7840add50 0x7fa784150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 -- 172.21.15.122:0/1646970226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa784071130 con 0x7fa7841561c0 2024-09-17T05:32:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78a600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7841561c0 0x7fa78415ab70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00d750 tx=0x7fa77c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa781600640 1 -- 172.21.15.122:0/1646970226 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c017070 con 0x7fa7841561c0 2024-09-17T05:32:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa781600640 1 -- 172.21.15.122:0/1646970226 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa77c009d20 con 0x7fa7841561c0 2024-09-17T05:32:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa784071410 con 0x7fa7841561c0 2024-09-17T05:32:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa781600640 1 -- 172.21.15.122:0/1646970226 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00de60 con 0x7fa7841561c0 2024-09-17T05:32:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa784150840 con 0x7fa7841561c0 2024-09-17T05:32:39.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.368+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa784071650 con 0x7fa7841561c0 2024-09-17T05:32:39.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.372+0000 7fa781600640 1 -- 172.21.15.122:0/1646970226 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fa77c005280 con 0x7fa7841561c0 2024-09-17T05:32:39.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.372+0000 7fa781600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa764073e60 0x7fa764076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:39.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.372+0000 7fa781600640 1 -- 172.21.15.122:0/1646970226 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4913+0+0 (secure 0 0 0) 0x7fa77c093020 con 0x7fa7841561c0 2024-09-17T05:32:39.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.372+0000 7fa789c00640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa764073e60 0x7fa764076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:39.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.372+0000 7fa789c00640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa764073e60 0x7fa764076320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa778001780 tx=0x7fa778009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:39.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.376+0000 7fa781600640 1 -- 172.21.15.122:0/1646970226 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa784071650 con 0x7fa7841561c0 2024-09-17T05:32:39.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa7840708d0 con 0x7fa7841561c0 2024-09-17T05:32:39.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa781600640 1 -- 172.21.15.122:0/1646970226 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7fa7840708d0 con 0x7fa7841561c0 2024-09-17T05:32:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:39.550 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:32:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:32:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:32:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:39.550 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:32:39.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa764073e60 msgr2=0x7fa764076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa764073e60 0x7fa764076320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa778001780 tx=0x7fa778009040 comp rx=0 tx=0).stop 2024-09-17T05:32:39.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7841561c0 msgr2=0x7fa78415ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7841561c0 0x7fa78415ab70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00d750 tx=0x7fa77c00dc10 comp rx=0 tx=0).stop 2024-09-17T05:32:39.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 shutdown_connections 2024-09-17T05:32:39.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa764073e60 0x7fa764076320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7841561c0 0x7fa78415ab70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7840add50 0x7fa784150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 --2- 172.21.15.122:0/1646970226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa78406cc20 0x7fa7841f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.548+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 >> 172.21.15.122:0/1646970226 conn(0x7fa7840b2890 msgr2=0x7fa784155630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:39.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.552+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 shutdown_connections 2024-09-17T05:32:39.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.552+0000 7fa78c600640 1 -- 172.21.15.122:0/1646970226 wait complete. 2024-09-17T05:32:39.570 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:32:39.570 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:32:39.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.712+0000 7f83d3600640 1 -- 172.21.15.122:0/2504790401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0d28e0 msgr2=0x7f83cc0d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.712+0000 7f83d3600640 1 --2- 172.21.15.122:0/2504790401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0d28e0 0x7f83cc0d0340 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009900 tx=0x7f83b801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.712+0000 7f83d3600640 1 -- 172.21.15.122:0/2504790401 shutdown_connections 2024-09-17T05:32:39.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 --2- 172.21.15.122:0/2504790401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83cc0af850 0x7f83cc0afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 --2- 172.21.15.122:0/2504790401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0d28e0 0x7f83cc0d0340 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 -- 172.21.15.122:0/2504790401 >> 172.21.15.122:0/2504790401 conn(0x7f83cc0d1dc0 msgr2=0x7f83cc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:39.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 -- 172.21.15.122:0/2504790401 shutdown_connections 2024-09-17T05:32:39.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 -- 172.21.15.122:0/2504790401 wait complete. 2024-09-17T05:32:39.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 Processor -- start 2024-09-17T05:32:39.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 -- start start 2024-09-17T05:32:39.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0af850 0x7f83cc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:39.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83cc0d28e0 0x7f83cc0c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:39.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83cc15f390 0x7f83cc14f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:39.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc168ef0 con 0x7f83cc0af850 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc168d70 con 0x7f83cc15f390 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d3600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f83cc169070 con 0x7f83cc0d28e0 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0af850 0x7f83cc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0af850 0x7f83cc0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42204/0 (socket says 172.21.15.122:42204) 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 -- 172.21.15.122:0/1480462820 learned_addr learned my addr 172.21.15.122:0/1480462820 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83cbe00640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83cc0d28e0 0x7f83cc0c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d1600640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83cc15f390 0x7f83cc14f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 -- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83cc0d28e0 msgr2=0x7f83cc0c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83cc0d28e0 0x7f83cc0c64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 -- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83cc15f390 msgr2=0x7f83cc14f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83cc15f390 0x7f83cc14f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 -- 172.21.15.122:0/1480462820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc14fcf0 con 0x7f83cc0af850 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83cbe00640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83cc0d28e0 0x7f83cc0c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83d0c00640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0af850 0x7f83cc0b1230 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00c3f0 tx=0x7f83bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83c8c00640 1 -- 172.21.15.122:0/1480462820 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0052d0 con 0x7f83cc0af850 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83c8c00640 1 -- 172.21.15.122:0/1480462820 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83bc007020 con 0x7f83cc0af850 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.716+0000 7f83c8c00640 1 -- 172.21.15.122:0/1480462820 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc007470 con 0x7f83cc0af850 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.720+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc14ffd0 con 0x7f83cc0af850 2024-09-17T05:32:39.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.720+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1f91d0 con 0x7f83cc0af850 2024-09-17T05:32:39.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.720+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83cc0c6cd0 con 0x7f83cc0af850 2024-09-17T05:32:39.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.720+0000 7f83c8c00640 1 -- 172.21.15.122:0/1480462820 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f83bc0067e0 con 0x7f83cc0af850 2024-09-17T05:32:39.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.720+0000 7f83c8c00640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83ac073e60 0x7f83ac076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:39.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.720+0000 7f83c8c00640 1 -- 172.21.15.122:0/1480462820 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4913+0+0 (secure 0 0 0) 0x7f83bc092290 con 0x7f83cc0af850 2024-09-17T05:32:39.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.720+0000 7f83cbe00640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83ac073e60 0x7f83ac076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:39.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.724+0000 7f83cbe00640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83ac073e60 0x7f83ac076320 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f83b80af000 tx=0x7f83b801b0f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:39.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.724+0000 7f83c8c00640 1 -- 172.21.15.122:0/1480462820 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f83cc0c6cd0 con 0x7f83cc0af850 2024-09-17T05:32:39.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:39.892+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f83cc150480 con 0x7f83cc0af850 2024-09-17T05:32:40.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.800+0000 7f83c8c00640 1 -- 172.21.15.122:0/1480462820 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f83bc05eb10 con 0x7f83cc0af850 2024-09-17T05:32:40.802 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83ac073e60 msgr2=0x7f83ac076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83ac073e60 0x7f83ac076320 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f83b80af000 tx=0x7f83b801b0f0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0af850 msgr2=0x7f83cc0b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0af850 0x7f83cc0b1230 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00c3f0 tx=0x7f83bc00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 shutdown_connections 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83ac073e60 0x7f83ac076320 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83cc15f390 0x7f83cc14f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83cc0d28e0 0x7f83cc0c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 --2- 172.21.15.122:0/1480462820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83cc0af850 0x7f83cc0b1230 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 >> 172.21.15.122:0/1480462820 conn(0x7f83cc0d1dc0 msgr2=0x7f83cc1508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 shutdown_connections 2024-09-17T05:32:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.804+0000 7f83d3600640 1 -- 172.21.15.122:0/1480462820 wait complete. 2024-09-17T05:32:40.831 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.980+0000 7f765d200640 1 -- 172.21.15.122:0/3803076664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76580afb10 msgr2=0x7f76580affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.980+0000 7f765d200640 1 --2- 172.21.15.122:0/3803076664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76580afb10 0x7f76580affc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7650009bc0 tx=0x7f765001cba0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.980+0000 7f765d200640 1 -- 172.21.15.122:0/3803076664 shutdown_connections 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.980+0000 7f765d200640 1 --2- 172.21.15.122:0/3803076664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76580afb10 0x7f76580affc0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.980+0000 7f765d200640 1 -- 172.21.15.122:0/3803076664 >> 172.21.15.122:0/3803076664 conn(0x7f76580d1dd0 msgr2=0x7f76580d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:40.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.980+0000 7f765d200640 1 -- 172.21.15.122:0/3803076664 shutdown_connections 2024-09-17T05:32:40.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.980+0000 7f765d200640 1 -- 172.21.15.122:0/3803076664 wait complete. 2024-09-17T05:32:40.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 Processor -- start 2024-09-17T05:32:40.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 -- start start 2024-09-17T05:32:40.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76580afb10 0x7f765814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:40.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76580b1650 0x7f765815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:40.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76581605d0 0x7f7658155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:40.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f765816a6d0 con 0x7f76581605d0 2024-09-17T05:32:40.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f765816a550 con 0x7f76580b1650 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f765816a850 con 0x7f76580afb10 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76581605d0 0x7f7658155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76581605d0 0x7f7658155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42226/0 (socket says 172.21.15.122:42226) 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 -- 172.21.15.122:0/4271966784 learned_addr learned my addr 172.21.15.122:0/4271966784 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 -- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76580afb10 msgr2=0x7f765814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76580afb10 0x7f765814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 -- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76580b1650 msgr2=0x7f765815e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76580b1650 0x7f765815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 -- 172.21.15.122:0/4271966784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76581f9d50 con 0x7f76581605d0 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f7656800640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76581605d0 0x7f7658155de0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7650009b10 tx=0x7f7650009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f764e000640 1 -- 172.21.15.122:0/4271966784 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76500b2040 con 0x7f76581605d0 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f764e000640 1 -- 172.21.15.122:0/4271966784 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7650004d40 con 0x7f76581605d0 2024-09-17T05:32:40.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f764e000640 1 -- 172.21.15.122:0/4271966784 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650004260 con 0x7f76581605d0 2024-09-17T05:32:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76581f9a70 con 0x7f76581605d0 2024-09-17T05:32:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765815dfe0 con 0x7f76581605d0 2024-09-17T05:32:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.984+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76580affc0 con 0x7f76581605d0 2024-09-17T05:32:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.988+0000 7f764e000640 1 -- 172.21.15.122:0/4271966784 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f76500be060 con 0x7f76581605d0 2024-09-17T05:32:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.988+0000 7f764e000640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7634073e60 0x7f7634076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.988+0000 7f764e000640 1 -- 172.21.15.122:0/4271966784 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5283+0+0 (secure 0 0 0) 0x7f765012cd00 con 0x7f76581605d0 2024-09-17T05:32:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.988+0000 7f7655e00640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7634073e60 0x7f7634076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.992+0000 7f7655e00640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7634073e60 0x7f7634076320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7648009670 tx=0x7f7648006bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:40.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:40.992+0000 7f764e000640 1 -- 172.21.15.122:0/4271966784 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7650016790 con 0x7f76581605d0 2024-09-17T05:32:41.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.160+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f76580d1080 con 0x7f76581605d0 2024-09-17T05:32:41.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.804+0000 7f764e000640 1 -- 172.21.15.122:0/4271966784 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f76500f9800 con 0x7f76581605d0 2024-09-17T05:32:41.806 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:32:41.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7634073e60 msgr2=0x7f7634076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7634073e60 0x7f7634076320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7648009670 tx=0x7f7648006bd0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76581605d0 msgr2=0x7f7658155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76581605d0 0x7f7658155de0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7650009b10 tx=0x7f7650009300 comp rx=0 tx=0).stop 2024-09-17T05:32:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 shutdown_connections 2024-09-17T05:32:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7634073e60 0x7f7634076320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76581605d0 0x7f7658155de0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76580b1650 0x7f765815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 --2- 172.21.15.122:0/4271966784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76580afb10 0x7f765814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:41.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 >> 172.21.15.122:0/4271966784 conn(0x7f76580d1dd0 msgr2=0x7f76580b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.808+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 shutdown_connections 2024-09-17T05:32:41.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.812+0000 7f765d200640 1 -- 172.21.15.122:0/4271966784 wait complete. 2024-09-17T05:32:41.850 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:32:41.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- 172.21.15.122:0/3877648708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 msgr2=0x7fe574004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:41.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 --2- 172.21.15.122:0/3877648708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 0x7fe574004c90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe578009d40 tx=0x7fe57801d450 comp rx=0 tx=0).stop 2024-09-17T05:32:41.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe58fe00640 1 -- 172.21.15.122:0/3877648708 reap_dead start 2024-09-17T05:32:41.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- 172.21.15.122:0/3877648708 shutdown_connections 2024-09-17T05:32:41.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- 172.21.15.122:0/3877648708 >> 172.21.15.122:0/3877648708 conn(0x7fe59009aa10 msgr2=0x7fe59009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:41.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- 172.21.15.122:0/3877648708 shutdown_connections 2024-09-17T05:32:41.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- 172.21.15.122:0/3877648708 wait complete. 2024-09-17T05:32:41.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 Processor -- start 2024-09-17T05:32:41.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- start start 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 0x7fe5900c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5901f8f50 0x7fe5900c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5900cba70 0x7fe5901f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900d0450 con 0x7fe574004810 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900cd820 con 0x7fe5900cba70 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe594e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe5900d05d0 con 0x7fe5901f8f50 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe58f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 0x7fe5900c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe58f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 0x7fe5900c4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35298/0 (socket says 172.21.15.122:35298) 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:41.996+0000 7fe58f400640 1 -- 172.21.15.122:0/741038696 learned_addr learned my addr 172.21.15.122:0/741038696 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe58fe00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5900cba70 0x7fe5901f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:42.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe58f400640 1 -- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5901f8f50 msgr2=0x7fe5900c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe586a00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5901f8f50 0x7fe5900c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe58f400640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5901f8f50 0x7fe5900c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe58f400640 1 -- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5900cba70 msgr2=0x7fe5901f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe58f400640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5900cba70 0x7fe5901f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe58f400640 1 -- 172.21.15.122:0/741038696 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900caa10 con 0x7fe574004810 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe58fe00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5900cba70 0x7fe5901f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe586a00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5901f8f50 0x7fe5900c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe58f400640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 0x7fe5900c4910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe580005f70 tx=0x7fe58000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe587e00640 1 -- 172.21.15.122:0/741038696 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58000fe00 con 0x7fe574004810 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe587e00640 1 -- 172.21.15.122:0/741038696 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe580019410 con 0x7fe574004810 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe587e00640 1 -- 172.21.15.122:0/741038696 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580019860 con 0x7fe574004810 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5900cacb0 con 0x7fe574004810 2024-09-17T05:32:42.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5900cb270 con 0x7fe574004810 2024-09-17T05:32:42.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.000+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5900ce0d0 con 0x7fe574004810 2024-09-17T05:32:42.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.004+0000 7fe587e00640 1 -- 172.21.15.122:0/741038696 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fe580019a00 con 0x7fe574004810 2024-09-17T05:32:42.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.004+0000 7fe587e00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe56c073e60 0x7fe56c076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:42.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.004+0000 7fe587e00640 1 -- 172.21.15.122:0/741038696 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5665+0+0 (secure 0 0 0) 0x7fe580093ed0 con 0x7fe574004810 2024-09-17T05:32:42.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.004+0000 7fe586a00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe56c073e60 0x7fe56c076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:42.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.004+0000 7fe586a00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe56c073e60 0x7fe56c076320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe5780ae000 tx=0x7fe574005840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:42.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.008+0000 7fe587e00640 1 -- 172.21.15.122:0/741038696 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe580060a10 con 0x7fe574004810 2024-09-17T05:32:42.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.208+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe5900cc190 con 0x7fe574004810 2024-09-17T05:32:42.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.840+0000 7fe587e00640 1 -- 172.21.15.122:0/741038696 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7fe580060310 con 0x7fe574004810 2024-09-17T05:32:42.842 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 10 and data pool 11 2024-09-17T05:32:42.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.848+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe56c073e60 msgr2=0x7fe56c076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.848+0000 7fe594e00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe56c073e60 0x7fe56c076320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe5780ae000 tx=0x7fe574005840 comp rx=0 tx=0).stop 2024-09-17T05:32:42.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.848+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 msgr2=0x7fe5900c4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:42.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.848+0000 7fe594e00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 0x7fe5900c4910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe580005f70 tx=0x7fe58000f3c0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.852+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 shutdown_connections 2024-09-17T05:32:42.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.852+0000 7fe594e00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe56c073e60 0x7fe56c076320 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.852+0000 7fe594e00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5900cba70 0x7fe5901f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.852+0000 7fe594e00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5901f8f50 0x7fe5900c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.852+0000 7fe594e00640 1 --2- 172.21.15.122:0/741038696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe574004810 0x7fe5900c4910 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:42.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.852+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 >> 172.21.15.122:0/741038696 conn(0x7fe59009aa10 msgr2=0x7fe5900b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:42.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.852+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 shutdown_connections 2024-09-17T05:32:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:42.852+0000 7fe594e00640 1 -- 172.21.15.122:0/741038696 wait complete. 2024-09-17T05:32:42.879 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:32:43.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 -- 172.21.15.122:0/2213746538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 msgr2=0x7fc0e00add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 --2- 172.21.15.122:0/2213746538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 0x7fc0e00add50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009c80 tx=0x7fc0d401cda0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 -- 172.21.15.122:0/2213746538 shutdown_connections 2024-09-17T05:32:43.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 --2- 172.21.15.122:0/2213746538 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0e00744c0 0x7fc0e00b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 --2- 172.21.15.122:0/2213746538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 0x7fc0e00add50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 -- 172.21.15.122:0/2213746538 >> 172.21.15.122:0/2213746538 conn(0x7fc0e006dd30 msgr2=0x7fc0e006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:43.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 -- 172.21.15.122:0/2213746538 shutdown_connections 2024-09-17T05:32:43.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 -- 172.21.15.122:0/2213746538 wait complete. 2024-09-17T05:32:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 Processor -- start 2024-09-17T05:32:43.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.020+0000 7fc0e8400640 1 -- start start 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0e00744c0 0x7fc0e01604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 0x7fc0e0160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0e015df80 0x7fc0e014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e016b0d0 con 0x7fc0e015dba0 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e016af50 con 0x7fc0e015df80 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e016b250 con 0x7fc0e00744c0 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 0x7fc0e0160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 0x7fc0e0160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35328/0 (socket says 172.21.15.122:35328) 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 -- 172.21.15.122:0/1652595047 learned_addr learned my addr 172.21.15.122:0/1652595047 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e6400640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0e015df80 0x7fc0e014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5a00640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0e00744c0 0x7fc0e01604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 -- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0e00744c0 msgr2=0x7fc0e01604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0e00744c0 0x7fc0e01604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 -- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0e015df80 msgr2=0x7fc0e014d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0e015df80 0x7fc0e014d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 -- 172.21.15.122:0/1652595047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e0161020 con 0x7fc0e015dba0 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e5000640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 0x7fc0e0160ae0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00c430 tx=0x7fc0cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0dd600640 1 -- 172.21.15.122:0/1652595047 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc017070 con 0x7fc0e015dba0 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0dd600640 1 -- 172.21.15.122:0/1652595047 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0cc0053c0 con 0x7fc0e015dba0 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0dd600640 1 -- 172.21.15.122:0/1652595047 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc003860 con 0x7fc0e015dba0 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e01455a0 con 0x7fc0e015dba0 2024-09-17T05:32:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e0145b60 con 0x7fc0e015dba0 2024-09-17T05:32:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e00b54c0 con 0x7fc0e015dba0 2024-09-17T05:32:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.024+0000 7fc0dd600640 1 -- 172.21.15.122:0/1652595047 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fc0cc01a070 con 0x7fc0e015dba0 2024-09-17T05:32:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.028+0000 7fc0dd600640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0c0073e60 0x7fc0c0076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.028+0000 7fc0dd600640 1 -- 172.21.15.122:0/1652595047 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fc0cc092180 con 0x7fc0e015dba0 2024-09-17T05:32:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.028+0000 7fc0e5a00640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0c0073e60 0x7fc0c0076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.028+0000 7fc0e5a00640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0c0073e60 0x7fc0c0076320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40af000 tx=0x7fc0d401b5e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.032+0000 7fc0dd600640 1 -- 172.21.15.122:0/1652595047 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc0e0161020 con 0x7fc0e015dba0 2024-09-17T05:32:43.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.200+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0e015c130 con 0x7fc0e015dba0 2024-09-17T05:32:43.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.200+0000 7fc0dd600640 1 -- 172.21.15.122:0/1652595047 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16828 (secure 0 0 0) 0x7fc0cc05e3b0 con 0x7fc0e015dba0 2024-09-17T05:32:43.202 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:43.202 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":74,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:32:42.215089+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:32:39.907333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:32:41.177668+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:32:43.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0c0073e60 msgr2=0x7fc0c0076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0c0073e60 0x7fc0c0076320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40af000 tx=0x7fc0d401b5e0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 msgr2=0x7fc0e0160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 0x7fc0e0160ae0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00c430 tx=0x7fc0cc00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 shutdown_connections 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0c0073e60 0x7fc0c0076320 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0e015df80 0x7fc0e014d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0e015dba0 0x7fc0e0160ae0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 --2- 172.21.15.122:0/1652595047 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0e00744c0 0x7fc0e01604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 >> 172.21.15.122:0/1652595047 conn(0x7fc0e006dd30 msgr2=0x7fc0e015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 shutdown_connections 2024-09-17T05:32:43.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.204+0000 7fc0e8400640 1 -- 172.21.15.122:0/1652595047 wait complete. 2024-09-17T05:32:43.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:32:43.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 -- 172.21.15.122:0/4139213412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc0d29e0 msgr2=0x7fd8fc0b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 --2- 172.21.15.122:0/4139213412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc0d29e0 0x7fd8fc0b6f80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009900 tx=0x7fd8f401c870 comp rx=0 tx=0).stop 2024-09-17T05:32:43.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 -- 172.21.15.122:0/4139213412 shutdown_connections 2024-09-17T05:32:43.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 --2- 172.21.15.122:0/4139213412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc0d29e0 0x7fd8fc0b6f80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 --2- 172.21.15.122:0/4139213412 >> v2:172.21.15.173:3300/0 conn(0x7fd8fc1613d0 0x7fd8fc0d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 -- 172.21.15.122:0/4139213412 >> 172.21.15.122:0/4139213412 conn(0x7fd8fc0d3e00 msgr2=0x7fd8fc0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:43.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 -- 172.21.15.122:0/4139213412 shutdown_connections 2024-09-17T05:32:43.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 -- 172.21.15.122:0/4139213412 wait complete. 2024-09-17T05:32:43.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 Processor -- start 2024-09-17T05:32:43.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.364+0000 7fd901c00640 1 -- start start 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8fc0d29e0 0x7fd8fc152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8fc1613d0 0x7fd8fc0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc1617b0 0x7fd8fc1faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc16acc0 con 0x7fd8fc1617b0 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0b8240 con 0x7fd8fc0d29e0 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc16ae40 con 0x7fd8fc1613d0 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc1617b0 0x7fd8fc1faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc1617b0 0x7fd8fc1faf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35342/0 (socket says 172.21.15.122:35342) 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 -- 172.21.15.122:0/3690617422 learned_addr learned my addr 172.21.15.122:0/3690617422 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd8fbe00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8fc0d29e0 0x7fd8fc152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd8fb400640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8fc1613d0 0x7fd8fc0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 -- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8fc1613d0 msgr2=0x7fd8fc0c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8fc1613d0 0x7fd8fc0c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 -- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8fc0d29e0 msgr2=0x7fd8fc152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8fc0d29e0 0x7fd8fc152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 -- 172.21.15.122:0/3690617422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc1f8cf0 con 0x7fd8fc1617b0 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd8fb400640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8fc1613d0 0x7fd8fc0c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901200640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc1617b0 0x7fd8fc1faf10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40b0040 tx=0x7fd8f4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd8f3e00640 1 -- 172.21.15.122:0/3690617422 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40b2040 con 0x7fd8fc1617b0 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd8f3e00640 1 -- 172.21.15.122:0/3690617422 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8f4005230 con 0x7fd8fc1617b0 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd8f3e00640 1 -- 172.21.15.122:0/3690617422 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40046d0 con 0x7fd8fc1617b0 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc1f8fd0 con 0x7fd8fc1617b0 2024-09-17T05:32:43.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc1f9590 con 0x7fd8fc1617b0 2024-09-17T05:32:43.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.368+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8fc0b6f80 con 0x7fd8fc1617b0 2024-09-17T05:32:43.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.372+0000 7fd8f3e00640 1 -- 172.21.15.122:0/3690617422 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fd8f4004870 con 0x7fd8fc1617b0 2024-09-17T05:32:43.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.372+0000 7fd8f3e00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8dc073e60 0x7fd8dc076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.372+0000 7fd8f3e00640 1 -- 172.21.15.122:0/3690617422 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fd8f412d0c0 con 0x7fd8fc1617b0 2024-09-17T05:32:43.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.372+0000 7fd8fbe00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8dc073e60 0x7fd8dc076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.372+0000 7fd8fbe00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8dc073e60 0x7fd8dc076320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc15a4f0 tx=0x7fd8ec009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.376+0000 7fd8f3e00640 1 -- 172.21.15.122:0/3690617422 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd8f40c4060 con 0x7fd8fc1617b0 2024-09-17T05:32:43.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8fc15b320 con 0x7fd8fc1617b0 2024-09-17T05:32:43.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd8f3e00640 1 -- 172.21.15.122:0/3690617422 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16828 (secure 0 0 0) 0x7fd8f4016490 con 0x7fd8fc1617b0 2024-09-17T05:32:43.545 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:43.545 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":74,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:32:42.215089+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:32:39.907333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:32:41.177668+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:32:43.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8dc073e60 msgr2=0x7fd8dc076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8dc073e60 0x7fd8dc076320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc15a4f0 tx=0x7fd8ec009500 comp rx=0 tx=0).stop 2024-09-17T05:32:43.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc1617b0 msgr2=0x7fd8fc1faf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc1617b0 0x7fd8fc1faf10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40b0040 tx=0x7fd8f4004620 comp rx=0 tx=0).stop 2024-09-17T05:32:43.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 shutdown_connections 2024-09-17T05:32:43.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8dc073e60 0x7fd8dc076320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8fc1617b0 0x7fd8fc1faf10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8fc1613d0 0x7fd8fc0c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 --2- 172.21.15.122:0/3690617422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8fc0d29e0 0x7fd8fc152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 >> 172.21.15.122:0/3690617422 conn(0x7fd8fc0d3e00 msgr2=0x7fd8fc06e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:43.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 shutdown_connections 2024-09-17T05:32:43.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.544+0000 7fd901c00640 1 -- 172.21.15.122:0/3690617422 wait complete. 2024-09-17T05:32:43.566 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:32:43.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- 172.21.15.122:0/3313641224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb8415db50 msgr2=0x7fdb840acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 --2- 172.21.15.122:0/3313641224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb8415db50 0x7fdb840acd40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400acd0 tx=0x7fdb7401de60 comp rx=0 tx=0).stop 2024-09-17T05:32:43.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- 172.21.15.122:0/3313641224 shutdown_connections 2024-09-17T05:32:43.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 --2- 172.21.15.122:0/3313641224 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb840b4a50 0x7fdb840ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 --2- 172.21.15.122:0/3313641224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb8415db50 0x7fdb840acd40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- 172.21.15.122:0/3313641224 >> 172.21.15.122:0/3313641224 conn(0x7fdb840b18f0 msgr2=0x7fdb840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:43.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- 172.21.15.122:0/3313641224 shutdown_connections 2024-09-17T05:32:43.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- 172.21.15.122:0/3313641224 wait complete. 2024-09-17T05:32:43.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 Processor -- start 2024-09-17T05:32:43.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- start start 2024-09-17T05:32:43.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb840b4a50 0x7fdb841faca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb840cd100 0x7fdb841f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8415db50 0x7fdb84162450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb84167120 con 0x7fdb8415db50 2024-09-17T05:32:43.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840cf4b0 con 0x7fdb840b4a50 2024-09-17T05:32:43.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb89a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdb841672a0 con 0x7fdb840cd100 2024-09-17T05:32:43.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb83400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8415db50 0x7fdb84162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb83400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8415db50 0x7fdb84162450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35348/0 (socket says 172.21.15.122:35348) 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb83400640 1 -- 172.21.15.122:0/832376971 learned_addr learned my addr 172.21.15.122:0/832376971 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb7a000640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb840cd100 0x7fdb841f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.712+0000 7fdb82a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb840b4a50 0x7fdb841faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb83400640 1 -- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb840cd100 msgr2=0x7fdb841f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb83400640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb840cd100 0x7fdb841f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb83400640 1 -- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb840b4a50 msgr2=0x7fdb841faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb83400640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb840b4a50 0x7fdb841faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb83400640 1 -- 172.21.15.122:0/832376971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb84164b20 con 0x7fdb8415db50 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb82a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb840b4a50 0x7fdb841faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:43.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb83400640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8415db50 0x7fdb84162450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00e8c0 tx=0x7fdb7c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb7b400640 1 -- 172.21.15.122:0/832376971 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00ccc0 con 0x7fdb8415db50 2024-09-17T05:32:43.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb7b400640 1 -- 172.21.15.122:0/832376971 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb7c00ce60 con 0x7fdb8415db50 2024-09-17T05:32:43.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb7b400640 1 -- 172.21.15.122:0/832376971 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c005280 con 0x7fdb8415db50 2024-09-17T05:32:43.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb84164e00 con 0x7fdb8415db50 2024-09-17T05:32:43.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8415b310 con 0x7fdb8415db50 2024-09-17T05:32:43.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb84069410 con 0x7fdb8415db50 2024-09-17T05:32:43.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb7b400640 1 -- 172.21.15.122:0/832376971 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fdb7c005420 con 0x7fdb8415db50 2024-09-17T05:32:43.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb7b400640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60073e60 0x7fdb60076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:43.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.716+0000 7fdb7b400640 1 -- 172.21.15.122:0/832376971 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fdb7c093680 con 0x7fdb8415db50 2024-09-17T05:32:43.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.720+0000 7fdb82a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60073e60 0x7fdb60076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:43.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.720+0000 7fdb82a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60073e60 0x7fdb60076320 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdb740b0000 tx=0x7fdb7401c6a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:43.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.720+0000 7fdb7b400640 1 -- 172.21.15.122:0/832376971 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdb7c0601e0 con 0x7fdb8415db50 2024-09-17T05:32:43.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:43.928+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdb84073cd0 con 0x7fdb8415db50 2024-09-17T05:32:44.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb7b400640 1 -- 172.21.15.122:0/832376971 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7fdb7c05fae0 con 0x7fdb8415db50 2024-09-17T05:32:44.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60073e60 msgr2=0x7fdb60076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60073e60 0x7fdb60076320 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdb740b0000 tx=0x7fdb7401c6a0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8415db50 msgr2=0x7fdb84162450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:44.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8415db50 0x7fdb84162450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00e8c0 tx=0x7fdb7c00ed80 comp rx=0 tx=0).stop 2024-09-17T05:32:44.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 shutdown_connections 2024-09-17T05:32:44.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60073e60 0x7fdb60076320 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8415db50 0x7fdb84162450 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb840cd100 0x7fdb841f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 --2- 172.21.15.122:0/832376971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb840b4a50 0x7fdb841faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:44.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 >> 172.21.15.122:0/832376971 conn(0x7fdb840b18f0 msgr2=0x7fdb84073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:44.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 shutdown_connections 2024-09-17T05:32:44.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:44.848+0000 7fdb89a00640 1 -- 172.21.15.122:0/832376971 wait complete. 2024-09-17T05:32:44.873 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:32:44.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 -- 172.21.15.122:0/1228585238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40cfe90 msgr2=0x7f80d40d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 --2- 172.21.15.122:0/1228585238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40cfe90 0x7f80d40d0270 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f80d4073f40 tx=0x7f80c401f0b0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 -- 172.21.15.122:0/1228585238 shutdown_connections 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 --2- 172.21.15.122:0/1228585238 >> v2:172.21.15.173:3300/0 conn(0x7f80d40b2cf0 0x7f80d4160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 --2- 172.21.15.122:0/1228585238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80d40d07b0 0x7f80d415fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 --2- 172.21.15.122:0/1228585238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40cfe90 0x7f80d40d0270 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 -- 172.21.15.122:0/1228585238 >> 172.21.15.122:0/1228585238 conn(0x7f80d40d1dc0 msgr2=0x7f80d40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 -- 172.21.15.122:0/1228585238 shutdown_connections 2024-09-17T05:32:45.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.020+0000 7f80da800640 1 -- 172.21.15.122:0/1228585238 wait complete. 2024-09-17T05:32:45.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 Processor -- start 2024-09-17T05:32:45.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 -- start start 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80d40b2cf0 0x7f80d40c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80d40cfe90 0x7f80d414cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40d07b0 0x7f80d40cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80d40b18f0 con 0x7f80d40d07b0 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80d40b1770 con 0x7f80d40b2cf0 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f80d40b1a70 con 0x7f80d40cfe90 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40d07b0 0x7f80d40cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40d07b0 0x7f80d40cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35356/0 (socket says 172.21.15.122:35356) 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 -- 172.21.15.122:0/2773245441 learned_addr learned my addr 172.21.15.122:0/2773245441 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:45.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3400640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80d40b2cf0 0x7f80d40c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80caa00640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80d40cfe90 0x7f80d414cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 -- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80d40cfe90 msgr2=0x7f80d414cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80d40cfe90 0x7f80d414cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 -- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80d40b2cf0 msgr2=0x7f80d40c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80d40b2cf0 0x7f80d40c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 -- 172.21.15.122:0/2773245441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d40ca6e0 con 0x7f80d40d07b0 2024-09-17T05:32:45.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80d3e00640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40d07b0 0x7f80d40cf040 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00e990 tx=0x7f80cc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80cbe00640 1 -- 172.21.15.122:0/2773245441 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc004d30 con 0x7f80d40d07b0 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80cbe00640 1 -- 172.21.15.122:0/2773245441 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80cc00cb00 con 0x7f80d40d07b0 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d40ca960 con 0x7f80d40d07b0 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80cbe00640 1 -- 172.21.15.122:0/2773245441 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc005020 con 0x7f80d40d07b0 2024-09-17T05:32:45.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d40cac00 con 0x7f80d40d07b0 2024-09-17T05:32:45.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.024+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d40cf7a0 con 0x7f80d40d07b0 2024-09-17T05:32:45.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.028+0000 7f80cbe00640 1 -- 172.21.15.122:0/2773245441 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f80cc0051c0 con 0x7f80d40d07b0 2024-09-17T05:32:45.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.028+0000 7f80cbe00640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80b0073e60 0x7f80b0076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:45.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.028+0000 7f80cbe00640 1 -- 172.21.15.122:0/2773245441 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f80cc0934d0 con 0x7f80d40d07b0 2024-09-17T05:32:45.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.028+0000 7f80d3400640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80b0073e60 0x7f80b0076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:45.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.028+0000 7f80d3400640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80b0073e60 0x7f80b0076320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f80c40b2000 tx=0x7f80c401d8f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:45.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.032+0000 7f80cbe00640 1 -- 172.21.15.122:0/2773245441 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f80d40ca6e0 con 0x7f80d40d07b0 2024-09-17T05:32:45.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.232+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f80d40d0270 con 0x7f80d40d07b0 2024-09-17T05:32:45.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.988+0000 7f80cbe00640 1 -- 172.21.15.122:0/2773245441 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f80cc016030 con 0x7f80d40d07b0 2024-09-17T05:32:45.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80b0073e60 msgr2=0x7f80b0076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80b0073e60 0x7f80b0076320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f80c40b2000 tx=0x7f80c401d8f0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40d07b0 msgr2=0x7f80d40cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:45.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40d07b0 0x7f80d40cf040 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00e990 tx=0x7f80cc00ee50 comp rx=0 tx=0).stop 2024-09-17T05:32:45.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 shutdown_connections 2024-09-17T05:32:45.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f80b0073e60 0x7f80b0076320 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d40d07b0 0x7f80d40cf040 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80d40cfe90 0x7f80d414cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 --2- 172.21.15.122:0/2773245441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80d40b2cf0 0x7f80d40c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:45.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 >> 172.21.15.122:0/2773245441 conn(0x7f80d40d1dc0 msgr2=0x7f80d4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:45.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 shutdown_connections 2024-09-17T05:32:45.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:45.992+0000 7f80da800640 1 -- 172.21.15.122:0/2773245441 wait complete. 2024-09-17T05:32:46.014 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:32:46.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:46.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 -- 172.21.15.122:0/3350324590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c0cf4e0 msgr2=0x7f5b8c0c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 --2- 172.21.15.122:0/3350324590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c0cf4e0 0x7f5b8c0c5dd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0d0a60 tx=0x7f5b8401cc40 comp rx=0 tx=0).stop 2024-09-17T05:32:46.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 -- 172.21.15.122:0/3350324590 shutdown_connections 2024-09-17T05:32:46.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 --2- 172.21.15.122:0/3350324590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c0cf4e0 0x7f5b8c0c5dd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 --2- 172.21.15.122:0/3350324590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b8c15c130 0x7f5b8c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 --2- 172.21.15.122:0/3350324590 >> v2:172.21.15.173:3300/0 conn(0x7f5b8c15bd50 0x7f5b8c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 -- 172.21.15.122:0/3350324590 >> 172.21.15.122:0/3350324590 conn(0x7f5b8c0d1ea0 msgr2=0x7f5b8c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 -- 172.21.15.122:0/3350324590 shutdown_connections 2024-09-17T05:32:46.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.156+0000 7f5b93600640 1 -- 172.21.15.122:0/3350324590 wait complete. 2024-09-17T05:32:46.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 Processor -- start 2024-09-17T05:32:46.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 -- start start 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b8c0cf4e0 0x7f5b8c165f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8c15bd50 0x7f5b8c155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c15c130 0x7f5b8c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0cccd0 con 0x7f5b8c15c130 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0ccb50 con 0x7f5b8c15bd50 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c0cce50 con 0x7f5b8c0cf4e0 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c15c130 0x7f5b8c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c15c130 0x7f5b8c155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35380/0 (socket says 172.21.15.122:35380) 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 -- 172.21.15.122:0/3808953003 learned_addr learned my addr 172.21.15.122:0/3808953003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b8be00640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8c15bd50 0x7f5b8c155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b90c00640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b8c0cf4e0 0x7f5b8c165f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 -- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b8c0cf4e0 msgr2=0x7f5b8c165f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b8c0cf4e0 0x7f5b8c165f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 -- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8c15bd50 msgr2=0x7f5b8c155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8c15bd50 0x7f5b8c155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 -- 172.21.15.122:0/3808953003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c156700 con 0x7f5b8c15c130 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b91600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c15c130 0x7f5b8c155dc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5b8401b2f0 tx=0x7f5b84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b88c00640 1 -- 172.21.15.122:0/3808953003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b840bb070 con 0x7f5b8c15c130 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b88c00640 1 -- 172.21.15.122:0/3808953003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b84004b40 con 0x7f5b8c15c130 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b88c00640 1 -- 172.21.15.122:0/3808953003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b840b2040 con 0x7f5b8c15c130 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c06e8d0 con 0x7f5b8c15c130 2024-09-17T05:32:46.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.160+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c1569e0 con 0x7f5b8c15c130 2024-09-17T05:32:46.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.164+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c0c5dd0 con 0x7f5b8c15c130 2024-09-17T05:32:46.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.164+0000 7f5b88c00640 1 -- 172.21.15.122:0/3808953003 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f5b84004460 con 0x7f5b8c15c130 2024-09-17T05:32:46.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.164+0000 7f5b88c00640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b6c073e60 0x7f5b6c076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.164+0000 7f5b88c00640 1 -- 172.21.15.122:0/3808953003 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f5b8412ccf0 con 0x7f5b8c15c130 2024-09-17T05:32:46.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.164+0000 7f5b90c00640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b6c073e60 0x7f5b6c076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.164+0000 7f5b90c00640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b6c073e60 0x7f5b6c076320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00bb70 tx=0x7f5b7c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.168+0000 7f5b88c00640 1 -- 172.21.15.122:0/3808953003 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5b8c06e8d0 con 0x7f5b8c15c130 2024-09-17T05:32:46.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.368+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b8c0b4400 con 0x7f5b8c15c130 2024-09-17T05:32:46.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.368+0000 7f5b88c00640 1 -- 172.21.15.122:0/3808953003 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5234 (secure 0 0 0) 0x7f5b84016950 con 0x7f5b8c15c130 2024-09-17T05:32:46.370 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:46.371 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":46,"btime":"2024-09-17T05:32:45:992259+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3900952275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3900952275},{"type":"v1","addr":"172.21.15.173:6836","nonce":3900952275}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5711,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/645422945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":645422945},{"type":"v1","addr":"172.21.15.173:6837","nonce":645422945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6408,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1968037162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1968037162},{"type":"v1","addr":"172.21.15.122:6833","nonce":1968037162}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:32:42.215056+0000","modified":"2024-09-17T05:32:45.239781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6426},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6426":{"gid":6426,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2505208774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2505208774},{"type":"v1","addr":"172.21.15.122:6835","nonce":2505208774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6426,"qdb_cluster":[6426]},"id":5}]} 2024-09-17T05:32:46.371 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 46 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b6c073e60 msgr2=0x7f5b6c076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b6c073e60 0x7f5b6c076320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00bb70 tx=0x7f5b7c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c15c130 msgr2=0x7f5b8c155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c15c130 0x7f5b8c155dc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5b8401b2f0 tx=0x7f5b84004270 comp rx=0 tx=0).stop 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 shutdown_connections 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b6c073e60 0x7f5b6c076320 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c15c130 0x7f5b8c155dc0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8c15bd50 0x7f5b8c155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 --2- 172.21.15.122:0/3808953003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b8c0cf4e0 0x7f5b8c165f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 >> 172.21.15.122:0/3808953003 conn(0x7f5b8c0d1ea0 msgr2=0x7f5b8c071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 shutdown_connections 2024-09-17T05:32:46.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.372+0000 7f5b93600640 1 -- 172.21.15.122:0/3808953003 wait complete. 2024-09-17T05:32:46.392 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:32:46.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 -- 172.21.15.122:0/2906791653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe188074a50 msgr2=0x7fe1880ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 --2- 172.21.15.122:0/2906791653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe188074a50 0x7fe1880ad000 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe1880cf860 tx=0x7fe18001cd80 comp rx=0 tx=0).stop 2024-09-17T05:32:46.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 -- 172.21.15.122:0/2906791653 shutdown_connections 2024-09-17T05:32:46.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 --2- 172.21.15.122:0/2906791653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe188074a50 0x7fe1880ad000 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 --2- 172.21.15.122:0/2906791653 >> v2:172.21.15.173:3300/0 conn(0x7fe18815f4f0 0x7fe1880af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 -- 172.21.15.122:0/2906791653 >> 172.21.15.122:0/2906791653 conn(0x7fe188073490 msgr2=0x7fe1880d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 -- 172.21.15.122:0/2906791653 shutdown_connections 2024-09-17T05:32:46.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 -- 172.21.15.122:0/2906791653 wait complete. 2024-09-17T05:32:46.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 Processor -- start 2024-09-17T05:32:46.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.532+0000 7fe187e00640 1 -- start start 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe188074a50 0x7fe1881fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18815f110 0x7fe1880af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18815f4f0 0x7fe18815c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1881683f0 con 0x7fe18815f110 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe188168270 con 0x7fe188074a50 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe188168570 con 0x7fe18815f4f0 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18815f110 0x7fe1880af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18815f110 0x7fe1880af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35408/0 (socket says 172.21.15.122:35408) 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 -- 172.21.15.122:0/2115843252 learned_addr learned my addr 172.21.15.122:0/2115843252 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe186a00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe188074a50 0x7fe1881fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187400640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18815f4f0 0x7fe18815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 -- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18815f4f0 msgr2=0x7fe18815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18815f4f0 0x7fe18815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 -- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe188074a50 msgr2=0x7fe1881fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe188074a50 0x7fe1881fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 -- 172.21.15.122:0/2115843252 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe188151a60 con 0x7fe18815f110 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17fe00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18815f110 0x7fe1880af720 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe17000d720 tx=0x7fe17000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17ea00640 1 -- 172.21.15.122:0/2115843252 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170017070 con 0x7fe18815f110 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17ea00640 1 -- 172.21.15.122:0/2115843252 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe170009d20 con 0x7fe18815f110 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe17ea00640 1 -- 172.21.15.122:0/2115843252 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170005280 con 0x7fe18815f110 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe188151d40 con 0x7fe18815f110 2024-09-17T05:32:46.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1880998e0 con 0x7fe18815f110 2024-09-17T05:32:46.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.536+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1880ad000 con 0x7fe18815f110 2024-09-17T05:32:46.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.540+0000 7fe17ea00640 1 -- 172.21.15.122:0/2115843252 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fe170005420 con 0x7fe18815f110 2024-09-17T05:32:46.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.540+0000 7fe17ea00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe164073e60 0x7fe164076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.540+0000 7fe17ea00640 1 -- 172.21.15.122:0/2115843252 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7fe170092ea0 con 0x7fe18815f110 2024-09-17T05:32:46.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.540+0000 7fe186a00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe164073e60 0x7fe164076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.540+0000 7fe186a00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe164073e60 0x7fe164076320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe178007fd0 tx=0x7fe178009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.544+0000 7fe17ea00640 1 -- 172.21.15.122:0/2115843252 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe17005f0d0 con 0x7fe18815f110 2024-09-17T05:32:46.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.712+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe188151f10 con 0x7fe18815f110 2024-09-17T05:32:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.712+0000 7fe17ea00640 1 -- 172.21.15.122:0/2115843252 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16815 (secure 0 0 0) 0x7fe170063b00 con 0x7fe18815f110 2024-09-17T05:32:46.715 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:46.715 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":75,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:32:43.826120+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:32:39.907333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:32:41.177668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe164073e60 msgr2=0x7fe164076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe164073e60 0x7fe164076320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe178007fd0 tx=0x7fe178009040 comp rx=0 tx=0).stop 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18815f110 msgr2=0x7fe1880af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18815f110 0x7fe1880af720 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe17000d720 tx=0x7fe17000dbe0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 shutdown_connections 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe164073e60 0x7fe164076320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18815f4f0 0x7fe18815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18815f110 0x7fe1880af720 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 --2- 172.21.15.122:0/2115843252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe188074a50 0x7fe1881fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 >> 172.21.15.122:0/2115843252 conn(0x7fe188073490 msgr2=0x7fe1880cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 shutdown_connections 2024-09-17T05:32:46.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.716+0000 7fe187e00640 1 -- 172.21.15.122:0/2115843252 wait complete. 2024-09-17T05:32:46.737 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:32:46.737 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:32:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- 172.21.15.122:0/4115218153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f0004d50 msgr2=0x7f75f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 --2- 172.21.15.122:0/4115218153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f0004d50 0x7f75f00051d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7604009ee0 tx=0x7f760401d5f0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760b400640 1 -- 172.21.15.122:0/4115218153 reap_dead start 2024-09-17T05:32:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- 172.21.15.122:0/4115218153 shutdown_connections 2024-09-17T05:32:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- 172.21.15.122:0/4115218153 >> 172.21.15.122:0/4115218153 conn(0x7f760c073490 msgr2=0x7f760c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- 172.21.15.122:0/4115218153 shutdown_connections 2024-09-17T05:32:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- 172.21.15.122:0/4115218153 wait complete. 2024-09-17T05:32:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 Processor -- start 2024-09-17T05:32:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- start start 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f760c0cd310 0x7f760c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f760c0cd6f0 0x7f760c1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f760c0ad000 0x7f760c0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f760c163b20 con 0x7f760c0cd6f0 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f760c070bb0 con 0x7f760c0ad000 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f760c163ca0 con 0x7f760c0cd310 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f760c0cd6f0 0x7f760c1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f760c0cd6f0 0x7f760c1f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35430/0 (socket says 172.21.15.122:35430) 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.880+0000 7f760a000640 1 -- 172.21.15.122:0/763584991 learned_addr learned my addr 172.21.15.122:0/763584991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760b400640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f760c0ad000 0x7f760c0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760aa00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f760c0cd310 0x7f760c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760a000640 1 -- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f760c0cd310 msgr2=0x7f760c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760a000640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f760c0cd310 0x7f760c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760a000640 1 -- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f760c0ad000 msgr2=0x7f760c0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760a000640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f760c0ad000 0x7f760c0c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760a000640 1 -- 172.21.15.122:0/763584991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c0afad0 con 0x7f760c0cd6f0 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760aa00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f760c0cd310 0x7f760c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760a000640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f760c0cd6f0 0x7f760c1f84d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f75f8004e60 tx=0x7f75f8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f7602a00640 1 -- 172.21.15.122:0/763584991 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8015020 con 0x7f760c0cd6f0 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f7602a00640 1 -- 172.21.15.122:0/763584991 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f8011960 con 0x7f760c0cd6f0 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c0afdb0 con 0x7f760c0cd6f0 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f7602a00640 1 -- 172.21.15.122:0/763584991 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8011d90 con 0x7f760c0cd6f0 2024-09-17T05:32:46.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c0b0370 con 0x7f760c0cd6f0 2024-09-17T05:32:46.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c0d0be0 con 0x7f760c0cd6f0 2024-09-17T05:32:46.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.884+0000 7f7602a00640 1 -- 172.21.15.122:0/763584991 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f75f8018020 con 0x7f760c0cd6f0 2024-09-17T05:32:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.888+0000 7f7602a00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f75ec073e60 0x7f75ec076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.888+0000 7f7602a00640 1 -- 172.21.15.122:0/763584991 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f75f8093620 con 0x7f760c0cd6f0 2024-09-17T05:32:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.888+0000 7f760aa00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f75ec073e60 0x7f75ec076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:46.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.888+0000 7f760aa00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f75ec073e60 0x7f75ec076320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f75fc001870 tx=0x7f75fc001960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:46.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:46.888+0000 7f7602a00640 1 -- 172.21.15.122:0/763584991 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f75f801f020 con 0x7f760c0cd6f0 2024-09-17T05:32:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.060+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f760c0af3f0 con 0x7f75ec073e60 2024-09-17T05:32:47.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.068+0000 7f7602a00640 1 -- 172.21.15.122:0/763584991 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252058 (secure 0 0 0) 0x7f760c0af3f0 con 0x7f75ec073e60 2024-09-17T05:32:47.070 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:47.072 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:32:47.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f75ec073e60 msgr2=0x7f75ec076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f75ec073e60 0x7f75ec076320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f75fc001870 tx=0x7f75fc001960 comp rx=0 tx=0).stop 2024-09-17T05:32:47.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f760c0cd6f0 msgr2=0x7f760c1f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f760c0cd6f0 0x7f760c1f84d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f75f8004e60 tx=0x7f75f8001640 comp rx=0 tx=0).stop 2024-09-17T05:32:47.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 shutdown_connections 2024-09-17T05:32:47.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f75ec073e60 0x7f75ec076320 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f760c0ad000 0x7f760c0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f760c0cd6f0 0x7f760c1f84d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 --2- 172.21.15.122:0/763584991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f760c0cd310 0x7f760c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 >> 172.21.15.122:0/763584991 conn(0x7f760c073490 msgr2=0x7f760c15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:47.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 shutdown_connections 2024-09-17T05:32:47.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.072+0000 7f760be00640 1 -- 172.21.15.122:0/763584991 wait complete. 2024-09-17T05:32:47.091 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-17T05:32:47.004125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406268,"kb_used_data":8200,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754568452,"statfs":{"total":773094113280,"available":772678094848,"internally_reserved":0,"allocated":8396800,"data_stored":4220516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":407424566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190566"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859812+0000","last_change":"2024-09-17T05:32:42.850446+0000","last_active":"2024-09-17T05:32:43.859812+0000","last_peered":"2024-09-17T05:32:43.859812+0000","last_clean":"2024-09-17T05:32:43.859812+0000","last_became_active":"2024-09-17T05:32:42.850158+0000","last_became_peered":"2024-09-17T05:32:42.850158+0000","last_unstale":"2024-09-17T05:32:43.859812+0000","last_undegraded":"2024-09-17T05:32:43.859812+0000","last_fullsized":"2024-09-17T05:32:43.859812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:41.844548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.837778+0000","last_change":"2024-09-17T05:32:42.836753+0000","last_active":"2024-09-17T05:32:42.837778+0000","last_peered":"2024-09-17T05:32:42.837778+0000","last_clean":"2024-09-17T05:32:42.837778+0000","last_became_active":"2024-09-17T05:32:42.836490+0000","last_became_peered":"2024-09-17T05:32:42.836490+0000","last_unstale":"2024-09-17T05:32:42.837778+0000","last_undegraded":"2024-09-17T05:32:42.837778+0000","last_fullsized":"2024-09-17T05:32:42.837778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:21.791821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833439+0000","last_change":"2024-09-17T05:32:42.841990+0000","last_active":"2024-09-17T05:32:43.833439+0000","last_peered":"2024-09-17T05:32:43.833439+0000","last_clean":"2024-09-17T05:32:43.833439+0000","last_became_active":"2024-09-17T05:32:42.841706+0000","last_became_peered":"2024-09-17T05:32:42.841706+0000","last_unstale":"2024-09-17T05:32:43.833439+0000","last_undegraded":"2024-09-17T05:32:43.833439+0000","last_fullsized":"2024-09-17T05:32:43.833439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:23.083468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834035+0000","last_change":"2024-09-17T05:32:42.842588+0000","last_active":"2024-09-17T05:32:43.834035+0000","last_peered":"2024-09-17T05:32:43.834035+0000","last_clean":"2024-09-17T05:32:43.834035+0000","last_became_active":"2024-09-17T05:32:42.842309+0000","last_became_peered":"2024-09-17T05:32:42.842309+0000","last_unstale":"2024-09-17T05:32:43.834035+0000","last_undegraded":"2024-09-17T05:32:43.834035+0000","last_fullsized":"2024-09-17T05:32:43.834035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:14.342414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.851405+0000","last_change":"2024-09-17T05:32:42.849299+0000","last_active":"2024-09-17T05:32:42.851405+0000","last_peered":"2024-09-17T05:32:42.851405+0000","last_clean":"2024-09-17T05:32:42.851405+0000","last_became_active":"2024-09-17T05:32:42.849065+0000","last_became_peered":"2024-09-17T05:32:42.849065+0000","last_unstale":"2024-09-17T05:32:42.851405+0000","last_undegraded":"2024-09-17T05:32:42.851405+0000","last_fullsized":"2024-09-17T05:32:42.851405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:01:23.217270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830086+0000","last_change":"2024-09-17T05:32:42.843322+0000","last_active":"2024-09-17T05:32:43.830086+0000","last_peered":"2024-09-17T05:32:43.830086+0000","last_clean":"2024-09-17T05:32:43.830086+0000","last_became_active":"2024-09-17T05:32:42.843074+0000","last_became_peered":"2024-09-17T05:32:42.843074+0000","last_unstale":"2024-09-17T05:32:43.830086+0000","last_undegraded":"2024-09-17T05:32:43.830086+0000","last_fullsized":"2024-09-17T05:32:43.830086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:24.780866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861994+0000","last_change":"2024-09-17T05:32:42.847172+0000","last_active":"2024-09-17T05:32:43.861994+0000","last_peered":"2024-09-17T05:32:43.861994+0000","last_clean":"2024-09-17T05:32:43.861994+0000","last_became_active":"2024-09-17T05:32:42.846910+0000","last_became_peered":"2024-09-17T05:32:42.846910+0000","last_unstale":"2024-09-17T05:32:43.861994+0000","last_undegraded":"2024-09-17T05:32:43.861994+0000","last_fullsized":"2024-09-17T05:32:43.861994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:23.944825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859324+0000","last_change":"2024-09-17T05:32:42.851761+0000","last_active":"2024-09-17T05:32:43.859324+0000","last_peered":"2024-09-17T05:32:43.859324+0000","last_clean":"2024-09-17T05:32:43.859324+0000","last_became_active":"2024-09-17T05:32:42.850931+0000","last_became_peered":"2024-09-17T05:32:42.850931+0000","last_unstale":"2024-09-17T05:32:43.859324+0000","last_undegraded":"2024-09-17T05:32:43.859324+0000","last_fullsized":"2024-09-17T05:32:43.859324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:07.602801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834573+0000","last_change":"2024-09-17T05:32:42.851748+0000","last_active":"2024-09-17T05:32:43.834573+0000","last_peered":"2024-09-17T05:32:43.834573+0000","last_clean":"2024-09-17T05:32:43.834573+0000","last_became_active":"2024-09-17T05:32:42.851447+0000","last_became_peered":"2024-09-17T05:32:42.851447+0000","last_unstale":"2024-09-17T05:32:43.834573+0000","last_undegraded":"2024-09-17T05:32:43.834573+0000","last_fullsized":"2024-09-17T05:32:43.834573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:55.668841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862570+0000","last_change":"2024-09-17T05:32:42.848133+0000","last_active":"2024-09-17T05:32:43.862570+0000","last_peered":"2024-09-17T05:32:43.862570+0000","last_clean":"2024-09-17T05:32:43.862570+0000","last_became_active":"2024-09-17T05:32:42.845525+0000","last_became_peered":"2024-09-17T05:32:42.845525+0000","last_unstale":"2024-09-17T05:32:43.862570+0000","last_undegraded":"2024-09-17T05:32:43.862570+0000","last_fullsized":"2024-09-17T05:32:43.862570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:26.270649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859933+0000","last_change":"2024-09-17T05:32:42.857839+0000","last_active":"2024-09-17T05:32:43.859933+0000","last_peered":"2024-09-17T05:32:43.859933+0000","last_clean":"2024-09-17T05:32:43.859933+0000","last_became_active":"2024-09-17T05:32:42.857439+0000","last_became_peered":"2024-09-17T05:32:42.857439+0000","last_unstale":"2024-09-17T05:32:43.859933+0000","last_undegraded":"2024-09-17T05:32:43.859933+0000","last_fullsized":"2024-09-17T05:32:43.859933+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:26.339928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833718+0000","last_change":"2024-09-17T05:32:42.851174+0000","last_active":"2024-09-17T05:32:43.833718+0000","last_peered":"2024-09-17T05:32:43.833718+0000","last_clean":"2024-09-17T05:32:43.833718+0000","last_became_active":"2024-09-17T05:32:42.850696+0000","last_became_peered":"2024-09-17T05:32:42.850696+0000","last_unstale":"2024-09-17T05:32:43.833718+0000","last_undegraded":"2024-09-17T05:32:43.833718+0000","last_fullsized":"2024-09-17T05:32:43.833718+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:59.785005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834233+0000","last_change":"2024-09-17T05:32:42.842338+0000","last_active":"2024-09-17T05:32:43.834233+0000","last_peered":"2024-09-17T05:32:43.834233+0000","last_clean":"2024-09-17T05:32:43.834233+0000","last_became_active":"2024-09-17T05:32:42.841976+0000","last_became_peered":"2024-09-17T05:32:42.841976+0000","last_unstale":"2024-09-17T05:32:43.834233+0000","last_undegraded":"2024-09-17T05:32:43.834233+0000","last_fullsized":"2024-09-17T05:32:43.834233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:18.728423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833859+0000","last_change":"2024-09-17T05:32:42.852925+0000","last_active":"2024-09-17T05:32:43.833859+0000","last_peered":"2024-09-17T05:32:43.833859+0000","last_clean":"2024-09-17T05:32:43.833859+0000","last_became_active":"2024-09-17T05:32:42.852588+0000","last_became_peered":"2024-09-17T05:32:42.852588+0000","last_unstale":"2024-09-17T05:32:43.833859+0000","last_undegraded":"2024-09-17T05:32:43.833859+0000","last_fullsized":"2024-09-17T05:32:43.833859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:10.758147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834093+0000","last_change":"2024-09-17T05:32:42.855395+0000","last_active":"2024-09-17T05:32:43.834093+0000","last_peered":"2024-09-17T05:32:43.834093+0000","last_clean":"2024-09-17T05:32:43.834093+0000","last_became_active":"2024-09-17T05:32:42.855175+0000","last_became_peered":"2024-09-17T05:32:42.855175+0000","last_unstale":"2024-09-17T05:32:43.834093+0000","last_undegraded":"2024-09-17T05:32:43.834093+0000","last_fullsized":"2024-09-17T05:32:43.834093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:43.482077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859319+0000","last_change":"2024-09-17T05:32:42.849649+0000","last_active":"2024-09-17T05:32:43.859319+0000","last_peered":"2024-09-17T05:32:43.859319+0000","last_clean":"2024-09-17T05:32:43.859319+0000","last_became_active":"2024-09-17T05:32:42.849063+0000","last_became_peered":"2024-09-17T05:32:42.849063+0000","last_unstale":"2024-09-17T05:32:43.859319+0000","last_undegraded":"2024-09-17T05:32:43.859319+0000","last_fullsized":"2024-09-17T05:32:43.859319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:38.098767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.846809+0000","last_change":"2024-09-17T05:32:42.844865+0000","last_active":"2024-09-17T05:32:42.846809+0000","last_peered":"2024-09-17T05:32:42.846809+0000","last_clean":"2024-09-17T05:32:42.846809+0000","last_became_active":"2024-09-17T05:32:42.844601+0000","last_became_peered":"2024-09-17T05:32:42.844601+0000","last_unstale":"2024-09-17T05:32:42.846809+0000","last_undegraded":"2024-09-17T05:32:42.846809+0000","last_fullsized":"2024-09-17T05:32:42.846809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:11.080390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.856985+0000","last_change":"2024-09-17T05:32:42.855906+0000","last_active":"2024-09-17T05:32:42.856985+0000","last_peered":"2024-09-17T05:32:42.856985+0000","last_clean":"2024-09-17T05:32:42.856985+0000","last_became_active":"2024-09-17T05:32:42.853667+0000","last_became_peered":"2024-09-17T05:32:42.853667+0000","last_unstale":"2024-09-17T05:32:42.856985+0000","last_undegraded":"2024-09-17T05:32:42.856985+0000","last_fullsized":"2024-09-17T05:32:42.856985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:16.343319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859897+0000","last_change":"2024-09-17T05:32:42.857054+0000","last_active":"2024-09-17T05:32:43.859897+0000","last_peered":"2024-09-17T05:32:43.859897+0000","last_clean":"2024-09-17T05:32:43.859897+0000","last_became_active":"2024-09-17T05:32:42.855169+0000","last_became_peered":"2024-09-17T05:32:42.855169+0000","last_unstale":"2024-09-17T05:32:43.859897+0000","last_undegraded":"2024-09-17T05:32:43.859897+0000","last_fullsized":"2024-09-17T05:32:43.859897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:54.177514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.849946+0000","last_change":"2024-09-17T05:32:42.847954+0000","last_active":"2024-09-17T05:32:42.849946+0000","last_peered":"2024-09-17T05:32:42.849946+0000","last_clean":"2024-09-17T05:32:42.849946+0000","last_became_active":"2024-09-17T05:32:42.847119+0000","last_became_peered":"2024-09-17T05:32:42.847119+0000","last_unstale":"2024-09-17T05:32:42.849946+0000","last_undegraded":"2024-09-17T05:32:42.849946+0000","last_fullsized":"2024-09-17T05:32:42.849946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.129651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.846854+0000","last_change":"2024-09-17T05:32:42.838921+0000","last_active":"2024-09-17T05:32:42.846854+0000","last_peered":"2024-09-17T05:32:42.846854+0000","last_clean":"2024-09-17T05:32:42.846854+0000","last_became_active":"2024-09-17T05:32:42.835703+0000","last_became_peered":"2024-09-17T05:32:42.835703+0000","last_unstale":"2024-09-17T05:32:42.846854+0000","last_undegraded":"2024-09-17T05:32:42.846854+0000","last_fullsized":"2024-09-17T05:32:42.846854+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:27.393969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830106+0000","last_change":"2024-09-17T05:32:42.842689+0000","last_active":"2024-09-17T05:32:43.830106+0000","last_peered":"2024-09-17T05:32:43.830106+0000","last_clean":"2024-09-17T05:32:43.830106+0000","last_became_active":"2024-09-17T05:32:42.842150+0000","last_became_peered":"2024-09-17T05:32:42.842150+0000","last_unstale":"2024-09-17T05:32:43.830106+0000","last_undegraded":"2024-09-17T05:32:43.830106+0000","last_fullsized":"2024-09-17T05:32:43.830106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:21.467351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862019+0000","last_change":"2024-09-17T05:32:42.841369+0000","last_active":"2024-09-17T05:32:43.862019+0000","last_peered":"2024-09-17T05:32:43.862019+0000","last_clean":"2024-09-17T05:32:43.862019+0000","last_became_active":"2024-09-17T05:32:42.841089+0000","last_became_peered":"2024-09-17T05:32:42.841089+0000","last_unstale":"2024-09-17T05:32:43.862019+0000","last_undegraded":"2024-09-17T05:32:43.862019+0000","last_fullsized":"2024-09-17T05:32:43.862019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:47.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859601+0000","last_change":"2024-09-17T05:32:42.843051+0000","last_active":"2024-09-17T05:32:43.859601+0000","last_peered":"2024-09-17T05:32:43.859601+0000","last_clean":"2024-09-17T05:32:43.859601+0000","last_became_active":"2024-09-17T05:32:42.842487+0000","last_became_peered":"2024-09-17T05:32:42.842487+0000","last_unstale":"2024-09-17T05:32:43.859601+0000","last_undegraded":"2024-09-17T05:32:43.859601+0000","last_fullsized":"2024-09-17T05:32:43.859601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:02.160849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862536+0000","last_change":"2024-09-17T05:32:42.843814+0000","last_active":"2024-09-17T05:32:43.862536+0000","last_peered":"2024-09-17T05:32:43.862536+0000","last_clean":"2024-09-17T05:32:43.862536+0000","last_became_active":"2024-09-17T05:32:42.843546+0000","last_became_peered":"2024-09-17T05:32:42.843546+0000","last_unstale":"2024-09-17T05:32:43.862536+0000","last_undegraded":"2024-09-17T05:32:43.862536+0000","last_fullsized":"2024-09-17T05:32:43.862536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:29.948078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859593+0000","last_change":"2024-09-17T05:32:42.852704+0000","last_active":"2024-09-17T05:32:43.859593+0000","last_peered":"2024-09-17T05:32:43.859593+0000","last_clean":"2024-09-17T05:32:43.859593+0000","last_became_active":"2024-09-17T05:32:42.852417+0000","last_became_peered":"2024-09-17T05:32:42.852417+0000","last_unstale":"2024-09-17T05:32:43.859593+0000","last_undegraded":"2024-09-17T05:32:43.859593+0000","last_fullsized":"2024-09-17T05:32:43.859593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:11.869608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.857488+0000","last_change":"2024-09-17T05:32:42.856683+0000","last_active":"2024-09-17T05:32:42.857488+0000","last_peered":"2024-09-17T05:32:42.857488+0000","last_clean":"2024-09-17T05:32:42.857488+0000","last_became_active":"2024-09-17T05:32:42.853765+0000","last_became_peered":"2024-09-17T05:32:42.853765+0000","last_unstale":"2024-09-17T05:32:42.857488+0000","last_undegraded":"2024-09-17T05:32:42.857488+0000","last_fullsized":"2024-09-17T05:32:42.857488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:48.793159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.837808+0000","last_change":"2024-09-17T05:32:42.834701+0000","last_active":"2024-09-17T05:32:42.837808+0000","last_peered":"2024-09-17T05:32:42.837808+0000","last_clean":"2024-09-17T05:32:42.837808+0000","last_became_active":"2024-09-17T05:32:42.834136+0000","last_became_peered":"2024-09-17T05:32:42.834136+0000","last_unstale":"2024-09-17T05:32:42.837808+0000","last_undegraded":"2024-09-17T05:32:42.837808+0000","last_fullsized":"2024-09-17T05:32:42.837808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:42.066596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834470+0000","last_change":"2024-09-17T05:32:42.857138+0000","last_active":"2024-09-17T05:32:43.834470+0000","last_peered":"2024-09-17T05:32:43.834470+0000","last_clean":"2024-09-17T05:32:43.834470+0000","last_became_active":"2024-09-17T05:32:42.856330+0000","last_became_peered":"2024-09-17T05:32:42.856330+0000","last_unstale":"2024-09-17T05:32:43.834470+0000","last_undegraded":"2024-09-17T05:32:43.834470+0000","last_fullsized":"2024-09-17T05:32:43.834470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:55:00.511249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859396+0000","last_change":"2024-09-17T05:32:42.854582+0000","last_active":"2024-09-17T05:32:43.859396+0000","last_peered":"2024-09-17T05:32:43.859396+0000","last_clean":"2024-09-17T05:32:43.859396+0000","last_became_active":"2024-09-17T05:32:42.851102+0000","last_became_peered":"2024-09-17T05:32:42.851102+0000","last_unstale":"2024-09-17T05:32:43.859396+0000","last_undegraded":"2024-09-17T05:32:43.859396+0000","last_fullsized":"2024-09-17T05:32:43.859396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:29:36.194568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.845649+0000","last_change":"2024-09-17T05:32:42.838120+0000","last_active":"2024-09-17T05:32:42.845649+0000","last_peered":"2024-09-17T05:32:42.845649+0000","last_clean":"2024-09-17T05:32:42.845649+0000","last_became_active":"2024-09-17T05:32:42.837493+0000","last_became_peered":"2024-09-17T05:32:42.837493+0000","last_unstale":"2024-09-17T05:32:42.845649+0000","last_undegraded":"2024-09-17T05:32:42.845649+0000","last_fullsized":"2024-09-17T05:32:42.845649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:31.345540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.847492+0000","last_change":"2024-09-17T05:32:42.839041+0000","last_active":"2024-09-17T05:32:42.847492+0000","last_peered":"2024-09-17T05:32:42.847492+0000","last_clean":"2024-09-17T05:32:42.847492+0000","last_became_active":"2024-09-17T05:32:42.837764+0000","last_became_peered":"2024-09-17T05:32:42.837764+0000","last_unstale":"2024-09-17T05:32:42.847492+0000","last_undegraded":"2024-09-17T05:32:42.847492+0000","last_fullsized":"2024-09-17T05:32:42.847492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:50.652095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859860+0000","last_change":"2024-09-17T05:32:42.849801+0000","last_active":"2024-09-17T05:32:43.859860+0000","last_peered":"2024-09-17T05:32:43.859860+0000","last_clean":"2024-09-17T05:32:43.859860+0000","last_became_active":"2024-09-17T05:32:42.849466+0000","last_became_peered":"2024-09-17T05:32:42.849466+0000","last_unstale":"2024-09-17T05:32:43.859860+0000","last_undegraded":"2024-09-17T05:32:43.859860+0000","last_fullsized":"2024-09-17T05:32:43.859860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:22.064362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830499+0000","last_change":"2024-09-17T05:32:41.836537+0000","last_active":"2024-09-17T05:32:43.830499+0000","last_peered":"2024-09-17T05:32:43.830499+0000","last_clean":"2024-09-17T05:32:43.830499+0000","last_became_active":"2024-09-17T05:32:41.836235+0000","last_became_peered":"2024-09-17T05:32:41.836235+0000","last_unstale":"2024-09-17T05:32:43.830499+0000","last_undegraded":"2024-09-17T05:32:43.830499+0000","last_fullsized":"2024-09-17T05:32:43.830499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:02.523088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834766+0000","last_change":"2024-09-17T05:32:42.838027+0000","last_active":"2024-09-17T05:32:43.834766+0000","last_peered":"2024-09-17T05:32:43.834766+0000","last_clean":"2024-09-17T05:32:43.834766+0000","last_became_active":"2024-09-17T05:32:42.837670+0000","last_became_peered":"2024-09-17T05:32:42.837670+0000","last_unstale":"2024-09-17T05:32:43.834766+0000","last_undegraded":"2024-09-17T05:32:43.834766+0000","last_fullsized":"2024-09-17T05:32:43.834766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:23.463934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.865124+0000","last_change":"2024-09-17T05:32:41.829757+0000","last_active":"2024-09-17T05:32:42.865124+0000","last_peered":"2024-09-17T05:32:42.865124+0000","last_clean":"2024-09-17T05:32:42.865124+0000","last_became_active":"2024-09-17T05:32:41.829431+0000","last_became_peered":"2024-09-17T05:32:41.829431+0000","last_unstale":"2024-09-17T05:32:42.865124+0000","last_undegraded":"2024-09-17T05:32:42.865124+0000","last_fullsized":"2024-09-17T05:32:42.865124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:01.717928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830388+0000","last_change":"2024-09-17T05:32:42.851631+0000","last_active":"2024-09-17T05:32:43.830388+0000","last_peered":"2024-09-17T05:32:43.830388+0000","last_clean":"2024-09-17T05:32:43.830388+0000","last_became_active":"2024-09-17T05:32:42.851346+0000","last_became_peered":"2024-09-17T05:32:42.851346+0000","last_unstale":"2024-09-17T05:32:43.830388+0000","last_undegraded":"2024-09-17T05:32:43.830388+0000","last_fullsized":"2024-09-17T05:32:43.830388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:54.048872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859957+0000","last_change":"2024-09-17T05:32:41.828925+0000","last_active":"2024-09-17T05:32:43.859957+0000","last_peered":"2024-09-17T05:32:43.859957+0000","last_clean":"2024-09-17T05:32:43.859957+0000","last_became_active":"2024-09-17T05:32:41.828635+0000","last_became_peered":"2024-09-17T05:32:41.828635+0000","last_unstale":"2024-09-17T05:32:43.859957+0000","last_undegraded":"2024-09-17T05:32:43.859957+0000","last_fullsized":"2024-09-17T05:32:43.859957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:06.765661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862312+0000","last_change":"2024-09-17T05:32:42.849922+0000","last_active":"2024-09-17T05:32:43.862312+0000","last_peered":"2024-09-17T05:32:43.862312+0000","last_clean":"2024-09-17T05:32:43.862312+0000","last_became_active":"2024-09-17T05:32:42.849657+0000","last_became_peered":"2024-09-17T05:32:42.849657+0000","last_unstale":"2024-09-17T05:32:43.862312+0000","last_undegraded":"2024-09-17T05:32:43.862312+0000","last_fullsized":"2024-09-17T05:32:43.862312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:01.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830357+0000","last_change":"2024-09-17T05:32:41.831126+0000","last_active":"2024-09-17T05:32:43.830357+0000","last_peered":"2024-09-17T05:32:43.830357+0000","last_clean":"2024-09-17T05:32:43.830357+0000","last_became_active":"2024-09-17T05:32:41.830364+0000","last_became_peered":"2024-09-17T05:32:41.830364+0000","last_unstale":"2024-09-17T05:32:43.830357+0000","last_undegraded":"2024-09-17T05:32:43.830357+0000","last_fullsized":"2024-09-17T05:32:43.830357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:35.004736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859615+0000","last_change":"2024-09-17T05:32:42.854364+0000","last_active":"2024-09-17T05:32:43.859615+0000","last_peered":"2024-09-17T05:32:43.859615+0000","last_clean":"2024-09-17T05:32:43.859615+0000","last_became_active":"2024-09-17T05:32:42.851511+0000","last_became_peered":"2024-09-17T05:32:42.851511+0000","last_unstale":"2024-09-17T05:32:43.859615+0000","last_undegraded":"2024-09-17T05:32:43.859615+0000","last_fullsized":"2024-09-17T05:32:43.859615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:51.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834135+0000","last_change":"2024-09-17T05:32:41.836873+0000","last_active":"2024-09-17T05:32:43.834135+0000","last_peered":"2024-09-17T05:32:43.834135+0000","last_clean":"2024-09-17T05:32:43.834135+0000","last_became_active":"2024-09-17T05:32:41.836282+0000","last_became_peered":"2024-09-17T05:32:41.836282+0000","last_unstale":"2024-09-17T05:32:43.834135+0000","last_undegraded":"2024-09-17T05:32:43.834135+0000","last_fullsized":"2024-09-17T05:32:43.834135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:26.386511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830132+0000","last_change":"2024-09-17T05:32:42.857518+0000","last_active":"2024-09-17T05:32:43.830132+0000","last_peered":"2024-09-17T05:32:43.830132+0000","last_clean":"2024-09-17T05:32:43.830132+0000","last_became_active":"2024-09-17T05:32:42.857086+0000","last_became_peered":"2024-09-17T05:32:42.857086+0000","last_unstale":"2024-09-17T05:32:43.830132+0000","last_undegraded":"2024-09-17T05:32:43.830132+0000","last_fullsized":"2024-09-17T05:32:43.830132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:14.876812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861891+0000","last_change":"2024-09-17T05:32:41.837982+0000","last_active":"2024-09-17T05:32:43.861891+0000","last_peered":"2024-09-17T05:32:43.861891+0000","last_clean":"2024-09-17T05:32:43.861891+0000","last_became_active":"2024-09-17T05:32:41.837654+0000","last_became_peered":"2024-09-17T05:32:41.837654+0000","last_unstale":"2024-09-17T05:32:43.861891+0000","last_undegraded":"2024-09-17T05:32:43.861891+0000","last_fullsized":"2024-09-17T05:32:43.861891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:11.616776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.847792+0000","last_change":"2024-09-17T05:32:42.838731+0000","last_active":"2024-09-17T05:32:42.847792+0000","last_peered":"2024-09-17T05:32:42.847792+0000","last_clean":"2024-09-17T05:32:42.847792+0000","last_became_active":"2024-09-17T05:32:42.838470+0000","last_became_peered":"2024-09-17T05:32:42.838470+0000","last_unstale":"2024-09-17T05:32:42.847792+0000","last_undegraded":"2024-09-17T05:32:42.847792+0000","last_fullsized":"2024-09-17T05:32:42.847792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:10.082230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.834629+0000","last_change":"2024-09-17T05:32:41.829675+0000","last_active":"2024-09-17T05:32:42.834629+0000","last_peered":"2024-09-17T05:32:42.834629+0000","last_clean":"2024-09-17T05:32:42.834629+0000","last_became_active":"2024-09-17T05:32:41.829371+0000","last_became_peered":"2024-09-17T05:32:41.829371+0000","last_unstale":"2024-09-17T05:32:42.834629+0000","last_undegraded":"2024-09-17T05:32:42.834629+0000","last_fullsized":"2024-09-17T05:32:42.834629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:36.177885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834208+0000","last_change":"2024-09-17T05:32:42.852109+0000","last_active":"2024-09-17T05:32:43.834208+0000","last_peered":"2024-09-17T05:32:43.834208+0000","last_clean":"2024-09-17T05:32:43.834208+0000","last_became_active":"2024-09-17T05:32:42.851898+0000","last_became_peered":"2024-09-17T05:32:42.851898+0000","last_unstale":"2024-09-17T05:32:43.834208+0000","last_undegraded":"2024-09-17T05:32:43.834208+0000","last_fullsized":"2024-09-17T05:32:43.834208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:12.286653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833352+0000","last_change":"2024-09-17T05:32:41.829035+0000","last_active":"2024-09-17T05:32:43.833352+0000","last_peered":"2024-09-17T05:32:43.833352+0000","last_clean":"2024-09-17T05:32:43.833352+0000","last_became_active":"2024-09-17T05:32:41.828157+0000","last_became_peered":"2024-09-17T05:32:41.828157+0000","last_unstale":"2024-09-17T05:32:43.833352+0000","last_undegraded":"2024-09-17T05:32:43.833352+0000","last_fullsized":"2024-09-17T05:32:43.833352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:30.338082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862535+0000","last_change":"2024-09-17T05:32:42.846029+0000","last_active":"2024-09-17T05:32:43.862535+0000","last_peered":"2024-09-17T05:32:43.862535+0000","last_clean":"2024-09-17T05:32:43.862535+0000","last_became_active":"2024-09-17T05:32:42.845221+0000","last_became_peered":"2024-09-17T05:32:42.845221+0000","last_unstale":"2024-09-17T05:32:43.862535+0000","last_undegraded":"2024-09-17T05:32:43.862535+0000","last_fullsized":"2024-09-17T05:32:43.862535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:33.410911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859536+0000","last_change":"2024-09-17T05:32:41.827749+0000","last_active":"2024-09-17T05:32:43.859536+0000","last_peered":"2024-09-17T05:32:43.859536+0000","last_clean":"2024-09-17T05:32:43.859536+0000","last_became_active":"2024-09-17T05:32:41.826846+0000","last_became_peered":"2024-09-17T05:32:41.826846+0000","last_unstale":"2024-09-17T05:32:43.859536+0000","last_undegraded":"2024-09-17T05:32:43.859536+0000","last_fullsized":"2024-09-17T05:32:43.859536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:36.321174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.838095+0000","last_change":"2024-09-17T05:32:42.836467+0000","last_active":"2024-09-17T05:32:42.838095+0000","last_peered":"2024-09-17T05:32:42.838095+0000","last_clean":"2024-09-17T05:32:42.838095+0000","last_became_active":"2024-09-17T05:32:42.836200+0000","last_became_peered":"2024-09-17T05:32:42.836200+0000","last_unstale":"2024-09-17T05:32:42.838095+0000","last_undegraded":"2024-09-17T05:32:42.838095+0000","last_fullsized":"2024-09-17T05:32:42.838095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:35.386328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.829272+0000","last_change":"2024-09-17T05:32:41.833156+0000","last_active":"2024-09-17T05:32:42.829272+0000","last_peered":"2024-09-17T05:32:42.829272+0000","last_clean":"2024-09-17T05:32:42.829272+0000","last_became_active":"2024-09-17T05:32:41.832879+0000","last_became_peered":"2024-09-17T05:32:41.832879+0000","last_unstale":"2024-09-17T05:32:42.829272+0000","last_undegraded":"2024-09-17T05:32:42.829272+0000","last_fullsized":"2024-09-17T05:32:42.829272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:36.291632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859591+0000","last_change":"2024-09-17T05:32:42.852975+0000","last_active":"2024-09-17T05:32:43.859591+0000","last_peered":"2024-09-17T05:32:43.859591+0000","last_clean":"2024-09-17T05:32:43.859591+0000","last_became_active":"2024-09-17T05:32:42.852533+0000","last_became_peered":"2024-09-17T05:32:42.852533+0000","last_unstale":"2024-09-17T05:32:43.859591+0000","last_undegraded":"2024-09-17T05:32:43.859591+0000","last_fullsized":"2024-09-17T05:32:43.859591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:39.601349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833745+0000","last_change":"2024-09-17T05:32:41.827745+0000","last_active":"2024-09-17T05:32:43.833745+0000","last_peered":"2024-09-17T05:32:43.833745+0000","last_clean":"2024-09-17T05:32:43.833745+0000","last_became_active":"2024-09-17T05:32:41.827417+0000","last_became_peered":"2024-09-17T05:32:41.827417+0000","last_unstale":"2024-09-17T05:32:43.833745+0000","last_undegraded":"2024-09-17T05:32:43.833745+0000","last_fullsized":"2024-09-17T05:32:43.833745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:49.644637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.856897+0000","last_change":"2024-09-17T05:32:42.853323+0000","last_active":"2024-09-17T05:32:42.856897+0000","last_peered":"2024-09-17T05:32:42.856897+0000","last_clean":"2024-09-17T05:32:42.856897+0000","last_became_active":"2024-09-17T05:32:42.852662+0000","last_became_peered":"2024-09-17T05:32:42.852662+0000","last_unstale":"2024-09-17T05:32:42.856897+0000","last_undegraded":"2024-09-17T05:32:42.856897+0000","last_fullsized":"2024-09-17T05:32:42.856897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:13.592379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.869451+0000","last_change":"2024-09-17T05:32:41.828667+0000","last_active":"2024-09-17T05:32:42.869451+0000","last_peered":"2024-09-17T05:32:42.869451+0000","last_clean":"2024-09-17T05:32:42.869451+0000","last_became_active":"2024-09-17T05:32:41.828227+0000","last_became_peered":"2024-09-17T05:32:41.828227+0000","last_unstale":"2024-09-17T05:32:42.869451+0000","last_undegraded":"2024-09-17T05:32:42.869451+0000","last_fullsized":"2024-09-17T05:32:42.869451+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:50.015189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830430+0000","last_change":"2024-09-17T05:32:42.850662+0000","last_active":"2024-09-17T05:32:43.830430+0000","last_peered":"2024-09-17T05:32:43.830430+0000","last_clean":"2024-09-17T05:32:43.830430+0000","last_became_active":"2024-09-17T05:32:42.850318+0000","last_became_peered":"2024-09-17T05:32:42.850318+0000","last_unstale":"2024-09-17T05:32:43.830430+0000","last_undegraded":"2024-09-17T05:32:43.830430+0000","last_fullsized":"2024-09-17T05:32:43.830430+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:35.833067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.866716+0000","last_change":"2024-09-17T05:32:41.831821+0000","last_active":"2024-09-17T05:32:42.866716+0000","last_peered":"2024-09-17T05:32:42.866716+0000","last_clean":"2024-09-17T05:32:42.866716+0000","last_became_active":"2024-09-17T05:32:41.831522+0000","last_became_peered":"2024-09-17T05:32:41.831522+0000","last_unstale":"2024-09-17T05:32:42.866716+0000","last_undegraded":"2024-09-17T05:32:42.866716+0000","last_fullsized":"2024-09-17T05:32:42.866716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.456476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862110+0000","last_change":"2024-09-17T05:32:42.853771+0000","last_active":"2024-09-17T05:32:43.862110+0000","last_peered":"2024-09-17T05:32:43.862110+0000","last_clean":"2024-09-17T05:32:43.862110+0000","last_became_active":"2024-09-17T05:32:42.853463+0000","last_became_peered":"2024-09-17T05:32:42.853463+0000","last_unstale":"2024-09-17T05:32:43.862110+0000","last_undegraded":"2024-09-17T05:32:43.862110+0000","last_fullsized":"2024-09-17T05:32:43.862110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:52.537869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861940+0000","last_change":"2024-09-17T05:32:41.836050+0000","last_active":"2024-09-17T05:32:43.861940+0000","last_peered":"2024-09-17T05:32:43.861940+0000","last_clean":"2024-09-17T05:32:43.861940+0000","last_became_active":"2024-09-17T05:32:41.835786+0000","last_became_peered":"2024-09-17T05:32:41.835786+0000","last_unstale":"2024-09-17T05:32:43.861940+0000","last_undegraded":"2024-09-17T05:32:43.861940+0000","last_fullsized":"2024-09-17T05:32:43.861940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:53.644735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.848555+0000","last_change":"2024-09-17T05:32:42.846841+0000","last_active":"2024-09-17T05:32:42.848555+0000","last_peered":"2024-09-17T05:32:42.848555+0000","last_clean":"2024-09-17T05:32:42.848555+0000","last_became_active":"2024-09-17T05:32:42.846162+0000","last_became_peered":"2024-09-17T05:32:42.846162+0000","last_unstale":"2024-09-17T05:32:42.848555+0000","last_undegraded":"2024-09-17T05:32:42.848555+0000","last_fullsized":"2024-09-17T05:32:42.848555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:16:22.306659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862096+0000","last_change":"2024-09-17T05:32:41.839596+0000","last_active":"2024-09-17T05:32:43.862096+0000","last_peered":"2024-09-17T05:32:43.862096+0000","last_clean":"2024-09-17T05:32:43.862096+0000","last_became_active":"2024-09-17T05:32:41.839321+0000","last_became_peered":"2024-09-17T05:32:41.839321+0000","last_unstale":"2024-09-17T05:32:43.862096+0000","last_undegraded":"2024-09-17T05:32:43.862096+0000","last_fullsized":"2024-09-17T05:32:43.862096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:05.044603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833828+0000","last_change":"2024-09-17T05:32:42.838750+0000","last_active":"2024-09-17T05:32:43.833828+0000","last_peered":"2024-09-17T05:32:43.833828+0000","last_clean":"2024-09-17T05:32:43.833828+0000","last_became_active":"2024-09-17T05:32:42.838385+0000","last_became_peered":"2024-09-17T05:32:42.838385+0000","last_unstale":"2024-09-17T05:32:43.833828+0000","last_undegraded":"2024-09-17T05:32:43.833828+0000","last_fullsized":"2024-09-17T05:32:43.833828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:16.783092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861747+0000","last_change":"2024-09-17T05:32:41.830042+0000","last_active":"2024-09-17T05:32:43.861747+0000","last_peered":"2024-09-17T05:32:43.861747+0000","last_clean":"2024-09-17T05:32:43.861747+0000","last_became_active":"2024-09-17T05:32:41.829786+0000","last_became_peered":"2024-09-17T05:32:41.829786+0000","last_unstale":"2024-09-17T05:32:43.861747+0000","last_undegraded":"2024-09-17T05:32:43.861747+0000","last_fullsized":"2024-09-17T05:32:43.861747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:07.831467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.848331+0000","last_change":"2024-09-17T05:32:42.841555+0000","last_active":"2024-09-17T05:32:42.848331+0000","last_peered":"2024-09-17T05:32:42.848331+0000","last_clean":"2024-09-17T05:32:42.848331+0000","last_became_active":"2024-09-17T05:32:42.841285+0000","last_became_peered":"2024-09-17T05:32:42.841285+0000","last_unstale":"2024-09-17T05:32:42.848331+0000","last_undegraded":"2024-09-17T05:32:42.848331+0000","last_fullsized":"2024-09-17T05:32:42.848331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:50.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859522+0000","last_change":"2024-09-17T05:32:41.828304+0000","last_active":"2024-09-17T05:32:43.859522+0000","last_peered":"2024-09-17T05:32:43.859522+0000","last_clean":"2024-09-17T05:32:43.859522+0000","last_became_active":"2024-09-17T05:32:41.827502+0000","last_became_peered":"2024-09-17T05:32:41.827502+0000","last_unstale":"2024-09-17T05:32:43.859522+0000","last_undegraded":"2024-09-17T05:32:43.859522+0000","last_fullsized":"2024-09-17T05:32:43.859522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:35.429655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.854041+0000","last_change":"2024-09-17T05:32:42.850973+0000","last_active":"2024-09-17T05:32:42.854041+0000","last_peered":"2024-09-17T05:32:42.854041+0000","last_clean":"2024-09-17T05:32:42.854041+0000","last_became_active":"2024-09-17T05:32:42.849135+0000","last_became_peered":"2024-09-17T05:32:42.849135+0000","last_unstale":"2024-09-17T05:32:42.854041+0000","last_undegraded":"2024-09-17T05:32:42.854041+0000","last_fullsized":"2024-09-17T05:32:42.854041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:14.791743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.863110+0000","last_change":"2024-09-17T05:32:41.837157+0000","last_active":"2024-09-17T05:32:42.863110+0000","last_peered":"2024-09-17T05:32:42.863110+0000","last_clean":"2024-09-17T05:32:42.863110+0000","last_became_active":"2024-09-17T05:32:41.836908+0000","last_became_peered":"2024-09-17T05:32:41.836908+0000","last_unstale":"2024-09-17T05:32:42.863110+0000","last_undegraded":"2024-09-17T05:32:42.863110+0000","last_fullsized":"2024-09-17T05:32:42.863110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:50.015189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833812+0000","last_change":"2024-09-17T05:32:42.852176+0000","last_active":"2024-09-17T05:32:43.833812+0000","last_peered":"2024-09-17T05:32:43.833812+0000","last_clean":"2024-09-17T05:32:43.833812+0000","last_became_active":"2024-09-17T05:32:42.851841+0000","last_became_peered":"2024-09-17T05:32:42.851841+0000","last_unstale":"2024-09-17T05:32:43.833812+0000","last_undegraded":"2024-09-17T05:32:43.833812+0000","last_fullsized":"2024-09-17T05:32:43.833812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:17.936678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.866668+0000","last_change":"2024-09-17T05:32:41.833579+0000","last_active":"2024-09-17T05:32:42.866668+0000","last_peered":"2024-09-17T05:32:42.866668+0000","last_clean":"2024-09-17T05:32:42.866668+0000","last_became_active":"2024-09-17T05:32:41.833302+0000","last_became_peered":"2024-09-17T05:32:41.833302+0000","last_unstale":"2024-09-17T05:32:42.866668+0000","last_undegraded":"2024-09-17T05:32:42.866668+0000","last_fullsized":"2024-09-17T05:32:42.866668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.456476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833513+0000","last_change":"2024-09-17T05:32:42.856029+0000","last_active":"2024-09-17T05:32:43.833513+0000","last_peered":"2024-09-17T05:32:43.833513+0000","last_clean":"2024-09-17T05:32:43.833513+0000","last_became_active":"2024-09-17T05:32:42.855638+0000","last_became_peered":"2024-09-17T05:32:42.855638+0000","last_unstale":"2024-09-17T05:32:43.833513+0000","last_undegraded":"2024-09-17T05:32:43.833513+0000","last_fullsized":"2024-09-17T05:32:43.833513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:50.976912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830457+0000","last_change":"2024-09-17T05:32:41.831433+0000","last_active":"2024-09-17T05:32:43.830457+0000","last_peered":"2024-09-17T05:32:43.830457+0000","last_clean":"2024-09-17T05:32:43.830457+0000","last_became_active":"2024-09-17T05:32:41.830819+0000","last_became_peered":"2024-09-17T05:32:41.830819+0000","last_unstale":"2024-09-17T05:32:43.830457+0000","last_undegraded":"2024-09-17T05:32:43.830457+0000","last_fullsized":"2024-09-17T05:32:43.830457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:49.829434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.847842+0000","last_change":"2024-09-17T05:32:42.837749+0000","last_active":"2024-09-17T05:32:42.847842+0000","last_peered":"2024-09-17T05:32:42.847842+0000","last_clean":"2024-09-17T05:32:42.847842+0000","last_became_active":"2024-09-17T05:32:42.837454+0000","last_became_peered":"2024-09-17T05:32:42.837454+0000","last_unstale":"2024-09-17T05:32:42.847842+0000","last_undegraded":"2024-09-17T05:32:42.847842+0000","last_fullsized":"2024-09-17T05:32:42.847842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:42.066596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830223+0000","last_change":"2024-09-17T05:32:41.837349+0000","last_active":"2024-09-17T05:32:43.830223+0000","last_peered":"2024-09-17T05:32:43.830223+0000","last_clean":"2024-09-17T05:32:43.830223+0000","last_became_active":"2024-09-17T05:32:41.836973+0000","last_became_peered":"2024-09-17T05:32:41.836973+0000","last_unstale":"2024-09-17T05:32:43.830223+0000","last_undegraded":"2024-09-17T05:32:43.830223+0000","last_fullsized":"2024-09-17T05:32:43.830223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:31.224807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.855283+0000","last_change":"2024-09-17T05:32:42.851739+0000","last_active":"2024-09-17T05:32:42.855283+0000","last_peered":"2024-09-17T05:32:42.855283+0000","last_clean":"2024-09-17T05:32:42.855283+0000","last_became_active":"2024-09-17T05:32:42.851446+0000","last_became_peered":"2024-09-17T05:32:42.851446+0000","last_unstale":"2024-09-17T05:32:42.855283+0000","last_undegraded":"2024-09-17T05:32:42.855283+0000","last_fullsized":"2024-09-17T05:32:42.855283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:35.386328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.828731+0000","last_change":"2024-09-17T05:32:41.832200+0000","last_active":"2024-09-17T05:32:42.828731+0000","last_peered":"2024-09-17T05:32:42.828731+0000","last_clean":"2024-09-17T05:32:42.828731+0000","last_became_active":"2024-09-17T05:32:41.831449+0000","last_became_peered":"2024-09-17T05:32:41.831449+0000","last_unstale":"2024-09-17T05:32:42.828731+0000","last_undegraded":"2024-09-17T05:32:42.828731+0000","last_fullsized":"2024-09-17T05:32:42.828731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:54.661383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834323+0000","last_change":"2024-09-17T05:32:42.854816+0000","last_active":"2024-09-17T05:32:43.834323+0000","last_peered":"2024-09-17T05:32:43.834323+0000","last_clean":"2024-09-17T05:32:43.834323+0000","last_became_active":"2024-09-17T05:32:42.849886+0000","last_became_peered":"2024-09-17T05:32:42.849886+0000","last_unstale":"2024-09-17T05:32:43.834323+0000","last_undegraded":"2024-09-17T05:32:43.834323+0000","last_fullsized":"2024-09-17T05:32:43.834323+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:39.193716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.842110+0000","last_change":"2024-09-17T05:32:41.832336+0000","last_active":"2024-09-17T05:32:42.842110+0000","last_peered":"2024-09-17T05:32:42.842110+0000","last_clean":"2024-09-17T05:32:42.842110+0000","last_became_active":"2024-09-17T05:32:41.832053+0000","last_became_peered":"2024-09-17T05:32:41.832053+0000","last_unstale":"2024-09-17T05:32:42.842110+0000","last_undegraded":"2024-09-17T05:32:42.842110+0000","last_fullsized":"2024-09-17T05:32:42.842110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:22:45.631798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.856102+0000","last_change":"2024-09-17T05:32:42.853030+0000","last_active":"2024-09-17T05:32:42.856102+0000","last_peered":"2024-09-17T05:32:42.856102+0000","last_clean":"2024-09-17T05:32:42.856102+0000","last_became_active":"2024-09-17T05:32:42.852758+0000","last_became_peered":"2024-09-17T05:32:42.852758+0000","last_unstale":"2024-09-17T05:32:42.856102+0000","last_undegraded":"2024-09-17T05:32:42.856102+0000","last_fullsized":"2024-09-17T05:32:42.856102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:41.544536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859688+0000","last_change":"2024-09-17T05:32:41.828055+0000","last_active":"2024-09-17T05:32:43.859688+0000","last_peered":"2024-09-17T05:32:43.859688+0000","last_clean":"2024-09-17T05:32:43.859688+0000","last_became_active":"2024-09-17T05:32:41.827184+0000","last_became_peered":"2024-09-17T05:32:41.827184+0000","last_unstale":"2024-09-17T05:32:43.859688+0000","last_undegraded":"2024-09-17T05:32:43.859688+0000","last_fullsized":"2024-09-17T05:32:43.859688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:35.618186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862701+0000","last_change":"2024-09-17T05:32:42.838296+0000","last_active":"2024-09-17T05:32:43.862701+0000","last_peered":"2024-09-17T05:32:43.862701+0000","last_clean":"2024-09-17T05:32:43.862701+0000","last_became_active":"2024-09-17T05:32:42.837476+0000","last_became_peered":"2024-09-17T05:32:42.837476+0000","last_unstale":"2024-09-17T05:32:43.862701+0000","last_undegraded":"2024-09-17T05:32:43.862701+0000","last_fullsized":"2024-09-17T05:32:43.862701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:13.110950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861401+0000","last_change":"2024-09-17T05:32:41.829414+0000","last_active":"2024-09-17T05:32:43.861401+0000","last_peered":"2024-09-17T05:32:43.861401+0000","last_clean":"2024-09-17T05:32:43.861401+0000","last_became_active":"2024-09-17T05:32:41.829122+0000","last_became_peered":"2024-09-17T05:32:41.829122+0000","last_unstale":"2024-09-17T05:32:43.861401+0000","last_undegraded":"2024-09-17T05:32:43.861401+0000","last_fullsized":"2024-09-17T05:32:43.861401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:34.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.846258+0000","last_change":"2024-09-17T05:32:42.840541+0000","last_active":"2024-09-17T05:32:42.846258+0000","last_peered":"2024-09-17T05:32:42.846258+0000","last_clean":"2024-09-17T05:32:42.846258+0000","last_became_active":"2024-09-17T05:32:42.840284+0000","last_became_peered":"2024-09-17T05:32:42.840284+0000","last_unstale":"2024-09-17T05:32:42.846258+0000","last_undegraded":"2024-09-17T05:32:42.846258+0000","last_fullsized":"2024-09-17T05:32:42.846258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:29.189795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833860+0000","last_change":"2024-09-17T05:32:41.828449+0000","last_active":"2024-09-17T05:32:43.833860+0000","last_peered":"2024-09-17T05:32:43.833860+0000","last_clean":"2024-09-17T05:32:43.833860+0000","last_became_active":"2024-09-17T05:32:41.828161+0000","last_became_peered":"2024-09-17T05:32:41.828161+0000","last_unstale":"2024-09-17T05:32:43.833860+0000","last_undegraded":"2024-09-17T05:32:43.833860+0000","last_fullsized":"2024-09-17T05:32:43.833860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:54.450949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862387+0000","last_change":"2024-09-17T05:32:42.842387+0000","last_active":"2024-09-17T05:32:43.862387+0000","last_peered":"2024-09-17T05:32:43.862387+0000","last_clean":"2024-09-17T05:32:43.862387+0000","last_became_active":"2024-09-17T05:32:42.842113+0000","last_became_peered":"2024-09-17T05:32:42.842113+0000","last_unstale":"2024-09-17T05:32:43.862387+0000","last_undegraded":"2024-09-17T05:32:43.862387+0000","last_fullsized":"2024-09-17T05:32:43.862387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:03.521253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833850+0000","last_change":"2024-09-17T05:32:41.826546+0000","last_active":"2024-09-17T05:32:43.833850+0000","last_peered":"2024-09-17T05:32:43.833850+0000","last_clean":"2024-09-17T05:32:43.833850+0000","last_became_active":"2024-09-17T05:32:41.826239+0000","last_became_peered":"2024-09-17T05:32:41.826239+0000","last_unstale":"2024-09-17T05:32:43.833850+0000","last_undegraded":"2024-09-17T05:32:43.833850+0000","last_fullsized":"2024-09-17T05:32:43.833850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:49.504003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.848375+0000","last_change":"2024-09-17T05:32:42.846398+0000","last_active":"2024-09-17T05:32:42.848375+0000","last_peered":"2024-09-17T05:32:42.848375+0000","last_clean":"2024-09-17T05:32:42.848375+0000","last_became_active":"2024-09-17T05:32:42.846115+0000","last_became_peered":"2024-09-17T05:32:42.846115+0000","last_unstale":"2024-09-17T05:32:42.848375+0000","last_undegraded":"2024-09-17T05:32:42.848375+0000","last_fullsized":"2024-09-17T05:32:42.848375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:52.521645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834104+0000","last_change":"2024-09-17T05:32:41.836576+0000","last_active":"2024-09-17T05:32:43.834104+0000","last_peered":"2024-09-17T05:32:43.834104+0000","last_clean":"2024-09-17T05:32:43.834104+0000","last_became_active":"2024-09-17T05:32:41.835909+0000","last_became_peered":"2024-09-17T05:32:41.835909+0000","last_unstale":"2024-09-17T05:32:43.834104+0000","last_undegraded":"2024-09-17T05:32:43.834104+0000","last_fullsized":"2024-09-17T05:32:43.834104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:28.182337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859785+0000","last_change":"2024-09-17T05:32:42.849931+0000","last_active":"2024-09-17T05:32:43.859785+0000","last_peered":"2024-09-17T05:32:43.859785+0000","last_clean":"2024-09-17T05:32:43.859785+0000","last_became_active":"2024-09-17T05:32:42.849631+0000","last_became_peered":"2024-09-17T05:32:42.849631+0000","last_unstale":"2024-09-17T05:32:43.859785+0000","last_undegraded":"2024-09-17T05:32:43.859785+0000","last_fullsized":"2024-09-17T05:32:43.859785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:04.686919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.842994+0000","last_change":"2024-09-17T05:32:41.834411+0000","last_active":"2024-09-17T05:32:42.842994+0000","last_peered":"2024-09-17T05:32:42.842994+0000","last_clean":"2024-09-17T05:32:42.842994+0000","last_became_active":"2024-09-17T05:32:41.833919+0000","last_became_peered":"2024-09-17T05:32:41.833919+0000","last_unstale":"2024-09-17T05:32:42.842994+0000","last_undegraded":"2024-09-17T05:32:42.842994+0000","last_fullsized":"2024-09-17T05:32:42.842994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:06.236255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.835552+0000","last_change":"2024-09-17T05:32:42.834174+0000","last_active":"2024-09-17T05:32:42.835552+0000","last_peered":"2024-09-17T05:32:42.835552+0000","last_clean":"2024-09-17T05:32:42.835552+0000","last_became_active":"2024-09-17T05:32:42.833873+0000","last_became_peered":"2024-09-17T05:32:42.833873+0000","last_unstale":"2024-09-17T05:32:42.835552+0000","last_undegraded":"2024-09-17T05:32:42.835552+0000","last_fullsized":"2024-09-17T05:32:42.835552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:37.299090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.829424+0000","last_change":"2024-09-17T05:32:41.834497+0000","last_active":"2024-09-17T05:32:42.829424+0000","last_peered":"2024-09-17T05:32:42.829424+0000","last_clean":"2024-09-17T05:32:42.829424+0000","last_became_active":"2024-09-17T05:32:41.832177+0000","last_became_peered":"2024-09-17T05:32:41.832177+0000","last_unstale":"2024-09-17T05:32:42.829424+0000","last_undegraded":"2024-09-17T05:32:42.829424+0000","last_fullsized":"2024-09-17T05:32:42.829424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:36.177885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.837699+0000","last_change":"2024-09-17T05:32:42.835372+0000","last_active":"2024-09-17T05:32:42.837699+0000","last_peered":"2024-09-17T05:32:42.837699+0000","last_clean":"2024-09-17T05:32:42.837699+0000","last_became_active":"2024-09-17T05:32:42.835076+0000","last_became_peered":"2024-09-17T05:32:42.835076+0000","last_unstale":"2024-09-17T05:32:42.837699+0000","last_undegraded":"2024-09-17T05:32:42.837699+0000","last_fullsized":"2024-09-17T05:32:42.837699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:10.082230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.829895+0000","last_change":"2024-09-17T05:32:41.835605+0000","last_active":"2024-09-17T05:32:43.829895+0000","last_peered":"2024-09-17T05:32:43.829895+0000","last_clean":"2024-09-17T05:32:43.829895+0000","last_became_active":"2024-09-17T05:32:41.835081+0000","last_became_peered":"2024-09-17T05:32:41.835081+0000","last_unstale":"2024-09-17T05:32:43.829895+0000","last_undegraded":"2024-09-17T05:32:43.829895+0000","last_fullsized":"2024-09-17T05:32:43.829895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:27.646267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.855990+0000","last_change":"2024-09-17T05:32:42.852181+0000","last_active":"2024-09-17T05:32:42.855990+0000","last_peered":"2024-09-17T05:32:42.855990+0000","last_clean":"2024-09-17T05:32:42.855990+0000","last_became_active":"2024-09-17T05:32:42.851740+0000","last_became_peered":"2024-09-17T05:32:42.851740+0000","last_unstale":"2024-09-17T05:32:42.855990+0000","last_undegraded":"2024-09-17T05:32:42.855990+0000","last_fullsized":"2024-09-17T05:32:42.855990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:59.516118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862099+0000","last_change":"2024-09-17T05:32:41.839819+0000","last_active":"2024-09-17T05:32:43.862099+0000","last_peered":"2024-09-17T05:32:43.862099+0000","last_clean":"2024-09-17T05:32:43.862099+0000","last_became_active":"2024-09-17T05:32:41.839431+0000","last_became_peered":"2024-09-17T05:32:41.839431+0000","last_unstale":"2024-09-17T05:32:43.862099+0000","last_undegraded":"2024-09-17T05:32:43.862099+0000","last_fullsized":"2024-09-17T05:32:43.862099+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:56.006570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":213,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861719+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:32:43.861719+0000","last_peered":"2024-09-17T05:32:43.861719+0000","last_clean":"2024-09-17T05:32:43.861719+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:32:43.861719+0000","last_undegraded":"2024-09-17T05:32:43.861719+0000","last_fullsized":"2024-09-17T05:32:43.861719+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":905216,"data_stored":380982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":909312,"data_stored":381016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54284,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317556,"statfs":{"total":96636764160,"available":96581177344,"internally_reserved":0,"allocated":905216,"data_stored":380064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509536,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":905216,"data_stored":379496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509537,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1486848,"data_stored":969738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50116,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321724,"statfs":{"total":96636764160,"available":96585445376,"internally_reserved":0,"allocated":897024,"data_stored":379440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54856,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316984,"statfs":{"total":96636764160,"available":96580591616,"internally_reserved":0,"allocated":1490944,"data_stored":969828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:32:47.095 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:32:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- 172.21.15.122:0/3147396851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec06e810 msgr2=0x7fdfec0cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 --2- 172.21.15.122:0/3147396851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec06e810 0x7fdfec0cf130 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdfec06e730 tx=0x7fdfd801c5d0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- 172.21.15.122:0/3147396851 shutdown_connections 2024-09-17T05:32:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 --2- 172.21.15.122:0/3147396851 >> v2:172.21.15.173:3300/0 conn(0x7fdfec0b4bd0 0x7fdfec15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 --2- 172.21.15.122:0/3147396851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec06e810 0x7fdfec0cf130 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 --2- 172.21.15.122:0/3147396851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfec15ade0 0x7fdfec0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- 172.21.15.122:0/3147396851 >> 172.21.15.122:0/3147396851 conn(0x7fdfec0b18f0 msgr2=0x7fdfec0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:47.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- 172.21.15.122:0/3147396851 shutdown_connections 2024-09-17T05:32:47.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- 172.21.15.122:0/3147396851 wait complete. 2024-09-17T05:32:47.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 Processor -- start 2024-09-17T05:32:47.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- start start 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfec06e810 0x7fdfec09c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec0b4bd0 0x7fdfec152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec15ade0 0x7fdfec152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec06f150 con 0x7fdfec15ade0 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec06efd0 con 0x7fdfec0b4bd0 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff4400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdfec06f2d0 con 0x7fdfec06e810 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff2400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec15ade0 0x7fdfec152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff2400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec15ade0 0x7fdfec152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35448/0 (socket says 172.21.15.122:35448) 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff2400640 1 -- 172.21.15.122:0/2500266568 learned_addr learned my addr 172.21.15.122:0/2500266568 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff1a00640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfec06e810 0x7fdfec09c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.276+0000 7fdff1000640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec0b4bd0 0x7fdfec152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff2400640 1 -- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfec06e810 msgr2=0x7fdfec09c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff2400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfec06e810 0x7fdfec09c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff2400640 1 -- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec0b4bd0 msgr2=0x7fdfec152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff2400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec0b4bd0 0x7fdfec152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff2400640 1 -- 172.21.15.122:0/2500266568 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec09af10 con 0x7fdfec15ade0 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff2400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec15ade0 0x7fdfec152890 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e990 tx=0x7fdfe400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdfe9600640 1 -- 172.21.15.122:0/2500266568 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4004d30 con 0x7fdfec15ade0 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdfe9600640 1 -- 172.21.15.122:0/2500266568 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfe400cb00 con 0x7fdfec15ade0 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdfe9600640 1 -- 172.21.15.122:0/2500266568 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4005020 con 0x7fdfec15ade0 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec09b1f0 con 0x7fdfec15ade0 2024-09-17T05:32:47.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec09b7b0 con 0x7fdfec15ade0 2024-09-17T05:32:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfec069410 con 0x7fdfec15ade0 2024-09-17T05:32:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdfe9600640 1 -- 172.21.15.122:0/2500266568 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fdfe40051c0 con 0x7fdfec15ade0 2024-09-17T05:32:47.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdfe9600640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfcc073e60 0x7fdfcc076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.280+0000 7fdfe9600640 1 -- 172.21.15.122:0/2500266568 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fdfe40934d0 con 0x7fdfec15ade0 2024-09-17T05:32:47.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.284+0000 7fdff1a00640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfcc073e60 0x7fdfcc076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.284+0000 7fdff1a00640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfcc073e60 0x7fdfcc076320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0004630 tx=0x7fdfe000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:47.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.284+0000 7fdfe9600640 1 -- 172.21.15.122:0/2500266568 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdfe4016030 con 0x7fdfec15ade0 2024-09-17T05:32:47.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.452+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfec15e900 con 0x7fdfcc073e60 2024-09-17T05:32:47.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.456+0000 7fdfe9600640 1 -- 172.21.15.122:0/2500266568 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252058 (secure 0 0 0) 0x7fdfec15e900 con 0x7fdfcc073e60 2024-09-17T05:32:47.460 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:32:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfcc073e60 msgr2=0x7fdfcc076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfcc073e60 0x7fdfcc076320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0004630 tx=0x7fdfe000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec15ade0 msgr2=0x7fdfec152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec15ade0 0x7fdfec152890 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e990 tx=0x7fdfe400ee50 comp rx=0 tx=0).stop 2024-09-17T05:32:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 shutdown_connections 2024-09-17T05:32:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfcc073e60 0x7fdfcc076320 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfec15ade0 0x7fdfec152890 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfec0b4bd0 0x7fdfec152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 --2- 172.21.15.122:0/2500266568 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfec06e810 0x7fdfec09c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 >> 172.21.15.122:0/2500266568 conn(0x7fdfec0b18f0 msgr2=0x7fdfec0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:47.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 shutdown_connections 2024-09-17T05:32:47.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.460+0000 7fdff4400640 1 -- 172.21.15.122:0/2500266568 wait complete. 2024-09-17T05:32:47.481 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-17T05:32:47.004125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406268,"kb_used_data":8200,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754568452,"statfs":{"total":773094113280,"available":772678094848,"internally_reserved":0,"allocated":8396800,"data_stored":4220516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":407424566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.190566"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859812+0000","last_change":"2024-09-17T05:32:42.850446+0000","last_active":"2024-09-17T05:32:43.859812+0000","last_peered":"2024-09-17T05:32:43.859812+0000","last_clean":"2024-09-17T05:32:43.859812+0000","last_became_active":"2024-09-17T05:32:42.850158+0000","last_became_peered":"2024-09-17T05:32:42.850158+0000","last_unstale":"2024-09-17T05:32:43.859812+0000","last_undegraded":"2024-09-17T05:32:43.859812+0000","last_fullsized":"2024-09-17T05:32:43.859812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:41.844548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.837778+0000","last_change":"2024-09-17T05:32:42.836753+0000","last_active":"2024-09-17T05:32:42.837778+0000","last_peered":"2024-09-17T05:32:42.837778+0000","last_clean":"2024-09-17T05:32:42.837778+0000","last_became_active":"2024-09-17T05:32:42.836490+0000","last_became_peered":"2024-09-17T05:32:42.836490+0000","last_unstale":"2024-09-17T05:32:42.837778+0000","last_undegraded":"2024-09-17T05:32:42.837778+0000","last_fullsized":"2024-09-17T05:32:42.837778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:21.791821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833439+0000","last_change":"2024-09-17T05:32:42.841990+0000","last_active":"2024-09-17T05:32:43.833439+0000","last_peered":"2024-09-17T05:32:43.833439+0000","last_clean":"2024-09-17T05:32:43.833439+0000","last_became_active":"2024-09-17T05:32:42.841706+0000","last_became_peered":"2024-09-17T05:32:42.841706+0000","last_unstale":"2024-09-17T05:32:43.833439+0000","last_undegraded":"2024-09-17T05:32:43.833439+0000","last_fullsized":"2024-09-17T05:32:43.833439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:23.083468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834035+0000","last_change":"2024-09-17T05:32:42.842588+0000","last_active":"2024-09-17T05:32:43.834035+0000","last_peered":"2024-09-17T05:32:43.834035+0000","last_clean":"2024-09-17T05:32:43.834035+0000","last_became_active":"2024-09-17T05:32:42.842309+0000","last_became_peered":"2024-09-17T05:32:42.842309+0000","last_unstale":"2024-09-17T05:32:43.834035+0000","last_undegraded":"2024-09-17T05:32:43.834035+0000","last_fullsized":"2024-09-17T05:32:43.834035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:14.342414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.851405+0000","last_change":"2024-09-17T05:32:42.849299+0000","last_active":"2024-09-17T05:32:42.851405+0000","last_peered":"2024-09-17T05:32:42.851405+0000","last_clean":"2024-09-17T05:32:42.851405+0000","last_became_active":"2024-09-17T05:32:42.849065+0000","last_became_peered":"2024-09-17T05:32:42.849065+0000","last_unstale":"2024-09-17T05:32:42.851405+0000","last_undegraded":"2024-09-17T05:32:42.851405+0000","last_fullsized":"2024-09-17T05:32:42.851405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:01:23.217270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830086+0000","last_change":"2024-09-17T05:32:42.843322+0000","last_active":"2024-09-17T05:32:43.830086+0000","last_peered":"2024-09-17T05:32:43.830086+0000","last_clean":"2024-09-17T05:32:43.830086+0000","last_became_active":"2024-09-17T05:32:42.843074+0000","last_became_peered":"2024-09-17T05:32:42.843074+0000","last_unstale":"2024-09-17T05:32:43.830086+0000","last_undegraded":"2024-09-17T05:32:43.830086+0000","last_fullsized":"2024-09-17T05:32:43.830086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:24.780866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861994+0000","last_change":"2024-09-17T05:32:42.847172+0000","last_active":"2024-09-17T05:32:43.861994+0000","last_peered":"2024-09-17T05:32:43.861994+0000","last_clean":"2024-09-17T05:32:43.861994+0000","last_became_active":"2024-09-17T05:32:42.846910+0000","last_became_peered":"2024-09-17T05:32:42.846910+0000","last_unstale":"2024-09-17T05:32:43.861994+0000","last_undegraded":"2024-09-17T05:32:43.861994+0000","last_fullsized":"2024-09-17T05:32:43.861994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:23.944825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859324+0000","last_change":"2024-09-17T05:32:42.851761+0000","last_active":"2024-09-17T05:32:43.859324+0000","last_peered":"2024-09-17T05:32:43.859324+0000","last_clean":"2024-09-17T05:32:43.859324+0000","last_became_active":"2024-09-17T05:32:42.850931+0000","last_became_peered":"2024-09-17T05:32:42.850931+0000","last_unstale":"2024-09-17T05:32:43.859324+0000","last_undegraded":"2024-09-17T05:32:43.859324+0000","last_fullsized":"2024-09-17T05:32:43.859324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:07.602801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834573+0000","last_change":"2024-09-17T05:32:42.851748+0000","last_active":"2024-09-17T05:32:43.834573+0000","last_peered":"2024-09-17T05:32:43.834573+0000","last_clean":"2024-09-17T05:32:43.834573+0000","last_became_active":"2024-09-17T05:32:42.851447+0000","last_became_peered":"2024-09-17T05:32:42.851447+0000","last_unstale":"2024-09-17T05:32:43.834573+0000","last_undegraded":"2024-09-17T05:32:43.834573+0000","last_fullsized":"2024-09-17T05:32:43.834573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:55.668841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862570+0000","last_change":"2024-09-17T05:32:42.848133+0000","last_active":"2024-09-17T05:32:43.862570+0000","last_peered":"2024-09-17T05:32:43.862570+0000","last_clean":"2024-09-17T05:32:43.862570+0000","last_became_active":"2024-09-17T05:32:42.845525+0000","last_became_peered":"2024-09-17T05:32:42.845525+0000","last_unstale":"2024-09-17T05:32:43.862570+0000","last_undegraded":"2024-09-17T05:32:43.862570+0000","last_fullsized":"2024-09-17T05:32:43.862570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:26.270649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859933+0000","last_change":"2024-09-17T05:32:42.857839+0000","last_active":"2024-09-17T05:32:43.859933+0000","last_peered":"2024-09-17T05:32:43.859933+0000","last_clean":"2024-09-17T05:32:43.859933+0000","last_became_active":"2024-09-17T05:32:42.857439+0000","last_became_peered":"2024-09-17T05:32:42.857439+0000","last_unstale":"2024-09-17T05:32:43.859933+0000","last_undegraded":"2024-09-17T05:32:43.859933+0000","last_fullsized":"2024-09-17T05:32:43.859933+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:26.339928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833718+0000","last_change":"2024-09-17T05:32:42.851174+0000","last_active":"2024-09-17T05:32:43.833718+0000","last_peered":"2024-09-17T05:32:43.833718+0000","last_clean":"2024-09-17T05:32:43.833718+0000","last_became_active":"2024-09-17T05:32:42.850696+0000","last_became_peered":"2024-09-17T05:32:42.850696+0000","last_unstale":"2024-09-17T05:32:43.833718+0000","last_undegraded":"2024-09-17T05:32:43.833718+0000","last_fullsized":"2024-09-17T05:32:43.833718+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:59.785005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834233+0000","last_change":"2024-09-17T05:32:42.842338+0000","last_active":"2024-09-17T05:32:43.834233+0000","last_peered":"2024-09-17T05:32:43.834233+0000","last_clean":"2024-09-17T05:32:43.834233+0000","last_became_active":"2024-09-17T05:32:42.841976+0000","last_became_peered":"2024-09-17T05:32:42.841976+0000","last_unstale":"2024-09-17T05:32:43.834233+0000","last_undegraded":"2024-09-17T05:32:43.834233+0000","last_fullsized":"2024-09-17T05:32:43.834233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:18.728423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833859+0000","last_change":"2024-09-17T05:32:42.852925+0000","last_active":"2024-09-17T05:32:43.833859+0000","last_peered":"2024-09-17T05:32:43.833859+0000","last_clean":"2024-09-17T05:32:43.833859+0000","last_became_active":"2024-09-17T05:32:42.852588+0000","last_became_peered":"2024-09-17T05:32:42.852588+0000","last_unstale":"2024-09-17T05:32:43.833859+0000","last_undegraded":"2024-09-17T05:32:43.833859+0000","last_fullsized":"2024-09-17T05:32:43.833859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:10.758147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834093+0000","last_change":"2024-09-17T05:32:42.855395+0000","last_active":"2024-09-17T05:32:43.834093+0000","last_peered":"2024-09-17T05:32:43.834093+0000","last_clean":"2024-09-17T05:32:43.834093+0000","last_became_active":"2024-09-17T05:32:42.855175+0000","last_became_peered":"2024-09-17T05:32:42.855175+0000","last_unstale":"2024-09-17T05:32:43.834093+0000","last_undegraded":"2024-09-17T05:32:43.834093+0000","last_fullsized":"2024-09-17T05:32:43.834093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:43.482077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859319+0000","last_change":"2024-09-17T05:32:42.849649+0000","last_active":"2024-09-17T05:32:43.859319+0000","last_peered":"2024-09-17T05:32:43.859319+0000","last_clean":"2024-09-17T05:32:43.859319+0000","last_became_active":"2024-09-17T05:32:42.849063+0000","last_became_peered":"2024-09-17T05:32:42.849063+0000","last_unstale":"2024-09-17T05:32:43.859319+0000","last_undegraded":"2024-09-17T05:32:43.859319+0000","last_fullsized":"2024-09-17T05:32:43.859319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:38.098767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.846809+0000","last_change":"2024-09-17T05:32:42.844865+0000","last_active":"2024-09-17T05:32:42.846809+0000","last_peered":"2024-09-17T05:32:42.846809+0000","last_clean":"2024-09-17T05:32:42.846809+0000","last_became_active":"2024-09-17T05:32:42.844601+0000","last_became_peered":"2024-09-17T05:32:42.844601+0000","last_unstale":"2024-09-17T05:32:42.846809+0000","last_undegraded":"2024-09-17T05:32:42.846809+0000","last_fullsized":"2024-09-17T05:32:42.846809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:11.080390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.856985+0000","last_change":"2024-09-17T05:32:42.855906+0000","last_active":"2024-09-17T05:32:42.856985+0000","last_peered":"2024-09-17T05:32:42.856985+0000","last_clean":"2024-09-17T05:32:42.856985+0000","last_became_active":"2024-09-17T05:32:42.853667+0000","last_became_peered":"2024-09-17T05:32:42.853667+0000","last_unstale":"2024-09-17T05:32:42.856985+0000","last_undegraded":"2024-09-17T05:32:42.856985+0000","last_fullsized":"2024-09-17T05:32:42.856985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:16.343319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859897+0000","last_change":"2024-09-17T05:32:42.857054+0000","last_active":"2024-09-17T05:32:43.859897+0000","last_peered":"2024-09-17T05:32:43.859897+0000","last_clean":"2024-09-17T05:32:43.859897+0000","last_became_active":"2024-09-17T05:32:42.855169+0000","last_became_peered":"2024-09-17T05:32:42.855169+0000","last_unstale":"2024-09-17T05:32:43.859897+0000","last_undegraded":"2024-09-17T05:32:43.859897+0000","last_fullsized":"2024-09-17T05:32:43.859897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:54.177514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.849946+0000","last_change":"2024-09-17T05:32:42.847954+0000","last_active":"2024-09-17T05:32:42.849946+0000","last_peered":"2024-09-17T05:32:42.849946+0000","last_clean":"2024-09-17T05:32:42.849946+0000","last_became_active":"2024-09-17T05:32:42.847119+0000","last_became_peered":"2024-09-17T05:32:42.847119+0000","last_unstale":"2024-09-17T05:32:42.849946+0000","last_undegraded":"2024-09-17T05:32:42.849946+0000","last_fullsized":"2024-09-17T05:32:42.849946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:09.129651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.846854+0000","last_change":"2024-09-17T05:32:42.838921+0000","last_active":"2024-09-17T05:32:42.846854+0000","last_peered":"2024-09-17T05:32:42.846854+0000","last_clean":"2024-09-17T05:32:42.846854+0000","last_became_active":"2024-09-17T05:32:42.835703+0000","last_became_peered":"2024-09-17T05:32:42.835703+0000","last_unstale":"2024-09-17T05:32:42.846854+0000","last_undegraded":"2024-09-17T05:32:42.846854+0000","last_fullsized":"2024-09-17T05:32:42.846854+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:27.393969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830106+0000","last_change":"2024-09-17T05:32:42.842689+0000","last_active":"2024-09-17T05:32:43.830106+0000","last_peered":"2024-09-17T05:32:43.830106+0000","last_clean":"2024-09-17T05:32:43.830106+0000","last_became_active":"2024-09-17T05:32:42.842150+0000","last_became_peered":"2024-09-17T05:32:42.842150+0000","last_unstale":"2024-09-17T05:32:43.830106+0000","last_undegraded":"2024-09-17T05:32:43.830106+0000","last_fullsized":"2024-09-17T05:32:43.830106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:21.467351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862019+0000","last_change":"2024-09-17T05:32:42.841369+0000","last_active":"2024-09-17T05:32:43.862019+0000","last_peered":"2024-09-17T05:32:43.862019+0000","last_clean":"2024-09-17T05:32:43.862019+0000","last_became_active":"2024-09-17T05:32:42.841089+0000","last_became_peered":"2024-09-17T05:32:42.841089+0000","last_unstale":"2024-09-17T05:32:43.862019+0000","last_undegraded":"2024-09-17T05:32:43.862019+0000","last_fullsized":"2024-09-17T05:32:43.862019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:47.119283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859601+0000","last_change":"2024-09-17T05:32:42.843051+0000","last_active":"2024-09-17T05:32:43.859601+0000","last_peered":"2024-09-17T05:32:43.859601+0000","last_clean":"2024-09-17T05:32:43.859601+0000","last_became_active":"2024-09-17T05:32:42.842487+0000","last_became_peered":"2024-09-17T05:32:42.842487+0000","last_unstale":"2024-09-17T05:32:43.859601+0000","last_undegraded":"2024-09-17T05:32:43.859601+0000","last_fullsized":"2024-09-17T05:32:43.859601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:02.160849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862536+0000","last_change":"2024-09-17T05:32:42.843814+0000","last_active":"2024-09-17T05:32:43.862536+0000","last_peered":"2024-09-17T05:32:43.862536+0000","last_clean":"2024-09-17T05:32:43.862536+0000","last_became_active":"2024-09-17T05:32:42.843546+0000","last_became_peered":"2024-09-17T05:32:42.843546+0000","last_unstale":"2024-09-17T05:32:43.862536+0000","last_undegraded":"2024-09-17T05:32:43.862536+0000","last_fullsized":"2024-09-17T05:32:43.862536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:29.948078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859593+0000","last_change":"2024-09-17T05:32:42.852704+0000","last_active":"2024-09-17T05:32:43.859593+0000","last_peered":"2024-09-17T05:32:43.859593+0000","last_clean":"2024-09-17T05:32:43.859593+0000","last_became_active":"2024-09-17T05:32:42.852417+0000","last_became_peered":"2024-09-17T05:32:42.852417+0000","last_unstale":"2024-09-17T05:32:43.859593+0000","last_undegraded":"2024-09-17T05:32:43.859593+0000","last_fullsized":"2024-09-17T05:32:43.859593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:11.869608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.857488+0000","last_change":"2024-09-17T05:32:42.856683+0000","last_active":"2024-09-17T05:32:42.857488+0000","last_peered":"2024-09-17T05:32:42.857488+0000","last_clean":"2024-09-17T05:32:42.857488+0000","last_became_active":"2024-09-17T05:32:42.853765+0000","last_became_peered":"2024-09-17T05:32:42.853765+0000","last_unstale":"2024-09-17T05:32:42.857488+0000","last_undegraded":"2024-09-17T05:32:42.857488+0000","last_fullsized":"2024-09-17T05:32:42.857488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:48.793159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.837808+0000","last_change":"2024-09-17T05:32:42.834701+0000","last_active":"2024-09-17T05:32:42.837808+0000","last_peered":"2024-09-17T05:32:42.837808+0000","last_clean":"2024-09-17T05:32:42.837808+0000","last_became_active":"2024-09-17T05:32:42.834136+0000","last_became_peered":"2024-09-17T05:32:42.834136+0000","last_unstale":"2024-09-17T05:32:42.837808+0000","last_undegraded":"2024-09-17T05:32:42.837808+0000","last_fullsized":"2024-09-17T05:32:42.837808+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:42.066596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834470+0000","last_change":"2024-09-17T05:32:42.857138+0000","last_active":"2024-09-17T05:32:43.834470+0000","last_peered":"2024-09-17T05:32:43.834470+0000","last_clean":"2024-09-17T05:32:43.834470+0000","last_became_active":"2024-09-17T05:32:42.856330+0000","last_became_peered":"2024-09-17T05:32:42.856330+0000","last_unstale":"2024-09-17T05:32:43.834470+0000","last_undegraded":"2024-09-17T05:32:43.834470+0000","last_fullsized":"2024-09-17T05:32:43.834470+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:55:00.511249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859396+0000","last_change":"2024-09-17T05:32:42.854582+0000","last_active":"2024-09-17T05:32:43.859396+0000","last_peered":"2024-09-17T05:32:43.859396+0000","last_clean":"2024-09-17T05:32:43.859396+0000","last_became_active":"2024-09-17T05:32:42.851102+0000","last_became_peered":"2024-09-17T05:32:42.851102+0000","last_unstale":"2024-09-17T05:32:43.859396+0000","last_undegraded":"2024-09-17T05:32:43.859396+0000","last_fullsized":"2024-09-17T05:32:43.859396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:29:36.194568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.845649+0000","last_change":"2024-09-17T05:32:42.838120+0000","last_active":"2024-09-17T05:32:42.845649+0000","last_peered":"2024-09-17T05:32:42.845649+0000","last_clean":"2024-09-17T05:32:42.845649+0000","last_became_active":"2024-09-17T05:32:42.837493+0000","last_became_peered":"2024-09-17T05:32:42.837493+0000","last_unstale":"2024-09-17T05:32:42.845649+0000","last_undegraded":"2024-09-17T05:32:42.845649+0000","last_fullsized":"2024-09-17T05:32:42.845649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:31.345540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.847492+0000","last_change":"2024-09-17T05:32:42.839041+0000","last_active":"2024-09-17T05:32:42.847492+0000","last_peered":"2024-09-17T05:32:42.847492+0000","last_clean":"2024-09-17T05:32:42.847492+0000","last_became_active":"2024-09-17T05:32:42.837764+0000","last_became_peered":"2024-09-17T05:32:42.837764+0000","last_unstale":"2024-09-17T05:32:42.847492+0000","last_undegraded":"2024-09-17T05:32:42.847492+0000","last_fullsized":"2024-09-17T05:32:42.847492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:50.652095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859860+0000","last_change":"2024-09-17T05:32:42.849801+0000","last_active":"2024-09-17T05:32:43.859860+0000","last_peered":"2024-09-17T05:32:43.859860+0000","last_clean":"2024-09-17T05:32:43.859860+0000","last_became_active":"2024-09-17T05:32:42.849466+0000","last_became_peered":"2024-09-17T05:32:42.849466+0000","last_unstale":"2024-09-17T05:32:43.859860+0000","last_undegraded":"2024-09-17T05:32:43.859860+0000","last_fullsized":"2024-09-17T05:32:43.859860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:22.064362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830499+0000","last_change":"2024-09-17T05:32:41.836537+0000","last_active":"2024-09-17T05:32:43.830499+0000","last_peered":"2024-09-17T05:32:43.830499+0000","last_clean":"2024-09-17T05:32:43.830499+0000","last_became_active":"2024-09-17T05:32:41.836235+0000","last_became_peered":"2024-09-17T05:32:41.836235+0000","last_unstale":"2024-09-17T05:32:43.830499+0000","last_undegraded":"2024-09-17T05:32:43.830499+0000","last_fullsized":"2024-09-17T05:32:43.830499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:02.523088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834766+0000","last_change":"2024-09-17T05:32:42.838027+0000","last_active":"2024-09-17T05:32:43.834766+0000","last_peered":"2024-09-17T05:32:43.834766+0000","last_clean":"2024-09-17T05:32:43.834766+0000","last_became_active":"2024-09-17T05:32:42.837670+0000","last_became_peered":"2024-09-17T05:32:42.837670+0000","last_unstale":"2024-09-17T05:32:43.834766+0000","last_undegraded":"2024-09-17T05:32:43.834766+0000","last_fullsized":"2024-09-17T05:32:43.834766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:23.463934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.865124+0000","last_change":"2024-09-17T05:32:41.829757+0000","last_active":"2024-09-17T05:32:42.865124+0000","last_peered":"2024-09-17T05:32:42.865124+0000","last_clean":"2024-09-17T05:32:42.865124+0000","last_became_active":"2024-09-17T05:32:41.829431+0000","last_became_peered":"2024-09-17T05:32:41.829431+0000","last_unstale":"2024-09-17T05:32:42.865124+0000","last_undegraded":"2024-09-17T05:32:42.865124+0000","last_fullsized":"2024-09-17T05:32:42.865124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:01.717928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830388+0000","last_change":"2024-09-17T05:32:42.851631+0000","last_active":"2024-09-17T05:32:43.830388+0000","last_peered":"2024-09-17T05:32:43.830388+0000","last_clean":"2024-09-17T05:32:43.830388+0000","last_became_active":"2024-09-17T05:32:42.851346+0000","last_became_peered":"2024-09-17T05:32:42.851346+0000","last_unstale":"2024-09-17T05:32:43.830388+0000","last_undegraded":"2024-09-17T05:32:43.830388+0000","last_fullsized":"2024-09-17T05:32:43.830388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:54.048872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859957+0000","last_change":"2024-09-17T05:32:41.828925+0000","last_active":"2024-09-17T05:32:43.859957+0000","last_peered":"2024-09-17T05:32:43.859957+0000","last_clean":"2024-09-17T05:32:43.859957+0000","last_became_active":"2024-09-17T05:32:41.828635+0000","last_became_peered":"2024-09-17T05:32:41.828635+0000","last_unstale":"2024-09-17T05:32:43.859957+0000","last_undegraded":"2024-09-17T05:32:43.859957+0000","last_fullsized":"2024-09-17T05:32:43.859957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:06.765661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862312+0000","last_change":"2024-09-17T05:32:42.849922+0000","last_active":"2024-09-17T05:32:43.862312+0000","last_peered":"2024-09-17T05:32:43.862312+0000","last_clean":"2024-09-17T05:32:43.862312+0000","last_became_active":"2024-09-17T05:32:42.849657+0000","last_became_peered":"2024-09-17T05:32:42.849657+0000","last_unstale":"2024-09-17T05:32:43.862312+0000","last_undegraded":"2024-09-17T05:32:43.862312+0000","last_fullsized":"2024-09-17T05:32:43.862312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:01.497059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830357+0000","last_change":"2024-09-17T05:32:41.831126+0000","last_active":"2024-09-17T05:32:43.830357+0000","last_peered":"2024-09-17T05:32:43.830357+0000","last_clean":"2024-09-17T05:32:43.830357+0000","last_became_active":"2024-09-17T05:32:41.830364+0000","last_became_peered":"2024-09-17T05:32:41.830364+0000","last_unstale":"2024-09-17T05:32:43.830357+0000","last_undegraded":"2024-09-17T05:32:43.830357+0000","last_fullsized":"2024-09-17T05:32:43.830357+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:35.004736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859615+0000","last_change":"2024-09-17T05:32:42.854364+0000","last_active":"2024-09-17T05:32:43.859615+0000","last_peered":"2024-09-17T05:32:43.859615+0000","last_clean":"2024-09-17T05:32:43.859615+0000","last_became_active":"2024-09-17T05:32:42.851511+0000","last_became_peered":"2024-09-17T05:32:42.851511+0000","last_unstale":"2024-09-17T05:32:43.859615+0000","last_undegraded":"2024-09-17T05:32:43.859615+0000","last_fullsized":"2024-09-17T05:32:43.859615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:08:51.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834135+0000","last_change":"2024-09-17T05:32:41.836873+0000","last_active":"2024-09-17T05:32:43.834135+0000","last_peered":"2024-09-17T05:32:43.834135+0000","last_clean":"2024-09-17T05:32:43.834135+0000","last_became_active":"2024-09-17T05:32:41.836282+0000","last_became_peered":"2024-09-17T05:32:41.836282+0000","last_unstale":"2024-09-17T05:32:43.834135+0000","last_undegraded":"2024-09-17T05:32:43.834135+0000","last_fullsized":"2024-09-17T05:32:43.834135+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:26.386511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830132+0000","last_change":"2024-09-17T05:32:42.857518+0000","last_active":"2024-09-17T05:32:43.830132+0000","last_peered":"2024-09-17T05:32:43.830132+0000","last_clean":"2024-09-17T05:32:43.830132+0000","last_became_active":"2024-09-17T05:32:42.857086+0000","last_became_peered":"2024-09-17T05:32:42.857086+0000","last_unstale":"2024-09-17T05:32:43.830132+0000","last_undegraded":"2024-09-17T05:32:43.830132+0000","last_fullsized":"2024-09-17T05:32:43.830132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:14.876812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861891+0000","last_change":"2024-09-17T05:32:41.837982+0000","last_active":"2024-09-17T05:32:43.861891+0000","last_peered":"2024-09-17T05:32:43.861891+0000","last_clean":"2024-09-17T05:32:43.861891+0000","last_became_active":"2024-09-17T05:32:41.837654+0000","last_became_peered":"2024-09-17T05:32:41.837654+0000","last_unstale":"2024-09-17T05:32:43.861891+0000","last_undegraded":"2024-09-17T05:32:43.861891+0000","last_fullsized":"2024-09-17T05:32:43.861891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:11.616776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.847792+0000","last_change":"2024-09-17T05:32:42.838731+0000","last_active":"2024-09-17T05:32:42.847792+0000","last_peered":"2024-09-17T05:32:42.847792+0000","last_clean":"2024-09-17T05:32:42.847792+0000","last_became_active":"2024-09-17T05:32:42.838470+0000","last_became_peered":"2024-09-17T05:32:42.838470+0000","last_unstale":"2024-09-17T05:32:42.847792+0000","last_undegraded":"2024-09-17T05:32:42.847792+0000","last_fullsized":"2024-09-17T05:32:42.847792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:10.082230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.834629+0000","last_change":"2024-09-17T05:32:41.829675+0000","last_active":"2024-09-17T05:32:42.834629+0000","last_peered":"2024-09-17T05:32:42.834629+0000","last_clean":"2024-09-17T05:32:42.834629+0000","last_became_active":"2024-09-17T05:32:41.829371+0000","last_became_peered":"2024-09-17T05:32:41.829371+0000","last_unstale":"2024-09-17T05:32:42.834629+0000","last_undegraded":"2024-09-17T05:32:42.834629+0000","last_fullsized":"2024-09-17T05:32:42.834629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:36.177885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834208+0000","last_change":"2024-09-17T05:32:42.852109+0000","last_active":"2024-09-17T05:32:43.834208+0000","last_peered":"2024-09-17T05:32:43.834208+0000","last_clean":"2024-09-17T05:32:43.834208+0000","last_became_active":"2024-09-17T05:32:42.851898+0000","last_became_peered":"2024-09-17T05:32:42.851898+0000","last_unstale":"2024-09-17T05:32:43.834208+0000","last_undegraded":"2024-09-17T05:32:43.834208+0000","last_fullsized":"2024-09-17T05:32:43.834208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:12.286653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833352+0000","last_change":"2024-09-17T05:32:41.829035+0000","last_active":"2024-09-17T05:32:43.833352+0000","last_peered":"2024-09-17T05:32:43.833352+0000","last_clean":"2024-09-17T05:32:43.833352+0000","last_became_active":"2024-09-17T05:32:41.828157+0000","last_became_peered":"2024-09-17T05:32:41.828157+0000","last_unstale":"2024-09-17T05:32:43.833352+0000","last_undegraded":"2024-09-17T05:32:43.833352+0000","last_fullsized":"2024-09-17T05:32:43.833352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:30.338082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862535+0000","last_change":"2024-09-17T05:32:42.846029+0000","last_active":"2024-09-17T05:32:43.862535+0000","last_peered":"2024-09-17T05:32:43.862535+0000","last_clean":"2024-09-17T05:32:43.862535+0000","last_became_active":"2024-09-17T05:32:42.845221+0000","last_became_peered":"2024-09-17T05:32:42.845221+0000","last_unstale":"2024-09-17T05:32:43.862535+0000","last_undegraded":"2024-09-17T05:32:43.862535+0000","last_fullsized":"2024-09-17T05:32:43.862535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:33.410911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859536+0000","last_change":"2024-09-17T05:32:41.827749+0000","last_active":"2024-09-17T05:32:43.859536+0000","last_peered":"2024-09-17T05:32:43.859536+0000","last_clean":"2024-09-17T05:32:43.859536+0000","last_became_active":"2024-09-17T05:32:41.826846+0000","last_became_peered":"2024-09-17T05:32:41.826846+0000","last_unstale":"2024-09-17T05:32:43.859536+0000","last_undegraded":"2024-09-17T05:32:43.859536+0000","last_fullsized":"2024-09-17T05:32:43.859536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:36.321174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.838095+0000","last_change":"2024-09-17T05:32:42.836467+0000","last_active":"2024-09-17T05:32:42.838095+0000","last_peered":"2024-09-17T05:32:42.838095+0000","last_clean":"2024-09-17T05:32:42.838095+0000","last_became_active":"2024-09-17T05:32:42.836200+0000","last_became_peered":"2024-09-17T05:32:42.836200+0000","last_unstale":"2024-09-17T05:32:42.838095+0000","last_undegraded":"2024-09-17T05:32:42.838095+0000","last_fullsized":"2024-09-17T05:32:42.838095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:35.386328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.829272+0000","last_change":"2024-09-17T05:32:41.833156+0000","last_active":"2024-09-17T05:32:42.829272+0000","last_peered":"2024-09-17T05:32:42.829272+0000","last_clean":"2024-09-17T05:32:42.829272+0000","last_became_active":"2024-09-17T05:32:41.832879+0000","last_became_peered":"2024-09-17T05:32:41.832879+0000","last_unstale":"2024-09-17T05:32:42.829272+0000","last_undegraded":"2024-09-17T05:32:42.829272+0000","last_fullsized":"2024-09-17T05:32:42.829272+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:36.291632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859591+0000","last_change":"2024-09-17T05:32:42.852975+0000","last_active":"2024-09-17T05:32:43.859591+0000","last_peered":"2024-09-17T05:32:43.859591+0000","last_clean":"2024-09-17T05:32:43.859591+0000","last_became_active":"2024-09-17T05:32:42.852533+0000","last_became_peered":"2024-09-17T05:32:42.852533+0000","last_unstale":"2024-09-17T05:32:43.859591+0000","last_undegraded":"2024-09-17T05:32:43.859591+0000","last_fullsized":"2024-09-17T05:32:43.859591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:37:39.601349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833745+0000","last_change":"2024-09-17T05:32:41.827745+0000","last_active":"2024-09-17T05:32:43.833745+0000","last_peered":"2024-09-17T05:32:43.833745+0000","last_clean":"2024-09-17T05:32:43.833745+0000","last_became_active":"2024-09-17T05:32:41.827417+0000","last_became_peered":"2024-09-17T05:32:41.827417+0000","last_unstale":"2024-09-17T05:32:43.833745+0000","last_undegraded":"2024-09-17T05:32:43.833745+0000","last_fullsized":"2024-09-17T05:32:43.833745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:49.644637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.856897+0000","last_change":"2024-09-17T05:32:42.853323+0000","last_active":"2024-09-17T05:32:42.856897+0000","last_peered":"2024-09-17T05:32:42.856897+0000","last_clean":"2024-09-17T05:32:42.856897+0000","last_became_active":"2024-09-17T05:32:42.852662+0000","last_became_peered":"2024-09-17T05:32:42.852662+0000","last_unstale":"2024-09-17T05:32:42.856897+0000","last_undegraded":"2024-09-17T05:32:42.856897+0000","last_fullsized":"2024-09-17T05:32:42.856897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:13.592379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.869451+0000","last_change":"2024-09-17T05:32:41.828667+0000","last_active":"2024-09-17T05:32:42.869451+0000","last_peered":"2024-09-17T05:32:42.869451+0000","last_clean":"2024-09-17T05:32:42.869451+0000","last_became_active":"2024-09-17T05:32:41.828227+0000","last_became_peered":"2024-09-17T05:32:41.828227+0000","last_unstale":"2024-09-17T05:32:42.869451+0000","last_undegraded":"2024-09-17T05:32:42.869451+0000","last_fullsized":"2024-09-17T05:32:42.869451+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:50.015189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830430+0000","last_change":"2024-09-17T05:32:42.850662+0000","last_active":"2024-09-17T05:32:43.830430+0000","last_peered":"2024-09-17T05:32:43.830430+0000","last_clean":"2024-09-17T05:32:43.830430+0000","last_became_active":"2024-09-17T05:32:42.850318+0000","last_became_peered":"2024-09-17T05:32:42.850318+0000","last_unstale":"2024-09-17T05:32:43.830430+0000","last_undegraded":"2024-09-17T05:32:43.830430+0000","last_fullsized":"2024-09-17T05:32:43.830430+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:35.833067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.866716+0000","last_change":"2024-09-17T05:32:41.831821+0000","last_active":"2024-09-17T05:32:42.866716+0000","last_peered":"2024-09-17T05:32:42.866716+0000","last_clean":"2024-09-17T05:32:42.866716+0000","last_became_active":"2024-09-17T05:32:41.831522+0000","last_became_peered":"2024-09-17T05:32:41.831522+0000","last_unstale":"2024-09-17T05:32:42.866716+0000","last_undegraded":"2024-09-17T05:32:42.866716+0000","last_fullsized":"2024-09-17T05:32:42.866716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.456476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862110+0000","last_change":"2024-09-17T05:32:42.853771+0000","last_active":"2024-09-17T05:32:43.862110+0000","last_peered":"2024-09-17T05:32:43.862110+0000","last_clean":"2024-09-17T05:32:43.862110+0000","last_became_active":"2024-09-17T05:32:42.853463+0000","last_became_peered":"2024-09-17T05:32:42.853463+0000","last_unstale":"2024-09-17T05:32:43.862110+0000","last_undegraded":"2024-09-17T05:32:43.862110+0000","last_fullsized":"2024-09-17T05:32:43.862110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:52.537869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861940+0000","last_change":"2024-09-17T05:32:41.836050+0000","last_active":"2024-09-17T05:32:43.861940+0000","last_peered":"2024-09-17T05:32:43.861940+0000","last_clean":"2024-09-17T05:32:43.861940+0000","last_became_active":"2024-09-17T05:32:41.835786+0000","last_became_peered":"2024-09-17T05:32:41.835786+0000","last_unstale":"2024-09-17T05:32:43.861940+0000","last_undegraded":"2024-09-17T05:32:43.861940+0000","last_fullsized":"2024-09-17T05:32:43.861940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:53.644735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.848555+0000","last_change":"2024-09-17T05:32:42.846841+0000","last_active":"2024-09-17T05:32:42.848555+0000","last_peered":"2024-09-17T05:32:42.848555+0000","last_clean":"2024-09-17T05:32:42.848555+0000","last_became_active":"2024-09-17T05:32:42.846162+0000","last_became_peered":"2024-09-17T05:32:42.846162+0000","last_unstale":"2024-09-17T05:32:42.848555+0000","last_undegraded":"2024-09-17T05:32:42.848555+0000","last_fullsized":"2024-09-17T05:32:42.848555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:16:22.306659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862096+0000","last_change":"2024-09-17T05:32:41.839596+0000","last_active":"2024-09-17T05:32:43.862096+0000","last_peered":"2024-09-17T05:32:43.862096+0000","last_clean":"2024-09-17T05:32:43.862096+0000","last_became_active":"2024-09-17T05:32:41.839321+0000","last_became_peered":"2024-09-17T05:32:41.839321+0000","last_unstale":"2024-09-17T05:32:43.862096+0000","last_undegraded":"2024-09-17T05:32:43.862096+0000","last_fullsized":"2024-09-17T05:32:43.862096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:05.044603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833828+0000","last_change":"2024-09-17T05:32:42.838750+0000","last_active":"2024-09-17T05:32:43.833828+0000","last_peered":"2024-09-17T05:32:43.833828+0000","last_clean":"2024-09-17T05:32:43.833828+0000","last_became_active":"2024-09-17T05:32:42.838385+0000","last_became_peered":"2024-09-17T05:32:42.838385+0000","last_unstale":"2024-09-17T05:32:43.833828+0000","last_undegraded":"2024-09-17T05:32:43.833828+0000","last_fullsized":"2024-09-17T05:32:43.833828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:16.783092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861747+0000","last_change":"2024-09-17T05:32:41.830042+0000","last_active":"2024-09-17T05:32:43.861747+0000","last_peered":"2024-09-17T05:32:43.861747+0000","last_clean":"2024-09-17T05:32:43.861747+0000","last_became_active":"2024-09-17T05:32:41.829786+0000","last_became_peered":"2024-09-17T05:32:41.829786+0000","last_unstale":"2024-09-17T05:32:43.861747+0000","last_undegraded":"2024-09-17T05:32:43.861747+0000","last_fullsized":"2024-09-17T05:32:43.861747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:07.831467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.848331+0000","last_change":"2024-09-17T05:32:42.841555+0000","last_active":"2024-09-17T05:32:42.848331+0000","last_peered":"2024-09-17T05:32:42.848331+0000","last_clean":"2024-09-17T05:32:42.848331+0000","last_became_active":"2024-09-17T05:32:42.841285+0000","last_became_peered":"2024-09-17T05:32:42.841285+0000","last_unstale":"2024-09-17T05:32:42.848331+0000","last_undegraded":"2024-09-17T05:32:42.848331+0000","last_fullsized":"2024-09-17T05:32:42.848331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:50.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859522+0000","last_change":"2024-09-17T05:32:41.828304+0000","last_active":"2024-09-17T05:32:43.859522+0000","last_peered":"2024-09-17T05:32:43.859522+0000","last_clean":"2024-09-17T05:32:43.859522+0000","last_became_active":"2024-09-17T05:32:41.827502+0000","last_became_peered":"2024-09-17T05:32:41.827502+0000","last_unstale":"2024-09-17T05:32:43.859522+0000","last_undegraded":"2024-09-17T05:32:43.859522+0000","last_fullsized":"2024-09-17T05:32:43.859522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:35.429655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.854041+0000","last_change":"2024-09-17T05:32:42.850973+0000","last_active":"2024-09-17T05:32:42.854041+0000","last_peered":"2024-09-17T05:32:42.854041+0000","last_clean":"2024-09-17T05:32:42.854041+0000","last_became_active":"2024-09-17T05:32:42.849135+0000","last_became_peered":"2024-09-17T05:32:42.849135+0000","last_unstale":"2024-09-17T05:32:42.854041+0000","last_undegraded":"2024-09-17T05:32:42.854041+0000","last_fullsized":"2024-09-17T05:32:42.854041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:14.791743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.863110+0000","last_change":"2024-09-17T05:32:41.837157+0000","last_active":"2024-09-17T05:32:42.863110+0000","last_peered":"2024-09-17T05:32:42.863110+0000","last_clean":"2024-09-17T05:32:42.863110+0000","last_became_active":"2024-09-17T05:32:41.836908+0000","last_became_peered":"2024-09-17T05:32:41.836908+0000","last_unstale":"2024-09-17T05:32:42.863110+0000","last_undegraded":"2024-09-17T05:32:42.863110+0000","last_fullsized":"2024-09-17T05:32:42.863110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:50.015189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833812+0000","last_change":"2024-09-17T05:32:42.852176+0000","last_active":"2024-09-17T05:32:43.833812+0000","last_peered":"2024-09-17T05:32:43.833812+0000","last_clean":"2024-09-17T05:32:43.833812+0000","last_became_active":"2024-09-17T05:32:42.851841+0000","last_became_peered":"2024-09-17T05:32:42.851841+0000","last_unstale":"2024-09-17T05:32:43.833812+0000","last_undegraded":"2024-09-17T05:32:43.833812+0000","last_fullsized":"2024-09-17T05:32:43.833812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:17.936678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.866668+0000","last_change":"2024-09-17T05:32:41.833579+0000","last_active":"2024-09-17T05:32:42.866668+0000","last_peered":"2024-09-17T05:32:42.866668+0000","last_clean":"2024-09-17T05:32:42.866668+0000","last_became_active":"2024-09-17T05:32:41.833302+0000","last_became_peered":"2024-09-17T05:32:41.833302+0000","last_unstale":"2024-09-17T05:32:42.866668+0000","last_undegraded":"2024-09-17T05:32:42.866668+0000","last_fullsized":"2024-09-17T05:32:42.866668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:22.456476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833513+0000","last_change":"2024-09-17T05:32:42.856029+0000","last_active":"2024-09-17T05:32:43.833513+0000","last_peered":"2024-09-17T05:32:43.833513+0000","last_clean":"2024-09-17T05:32:43.833513+0000","last_became_active":"2024-09-17T05:32:42.855638+0000","last_became_peered":"2024-09-17T05:32:42.855638+0000","last_unstale":"2024-09-17T05:32:43.833513+0000","last_undegraded":"2024-09-17T05:32:43.833513+0000","last_fullsized":"2024-09-17T05:32:43.833513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:50.976912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830457+0000","last_change":"2024-09-17T05:32:41.831433+0000","last_active":"2024-09-17T05:32:43.830457+0000","last_peered":"2024-09-17T05:32:43.830457+0000","last_clean":"2024-09-17T05:32:43.830457+0000","last_became_active":"2024-09-17T05:32:41.830819+0000","last_became_peered":"2024-09-17T05:32:41.830819+0000","last_unstale":"2024-09-17T05:32:43.830457+0000","last_undegraded":"2024-09-17T05:32:43.830457+0000","last_fullsized":"2024-09-17T05:32:43.830457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:49.829434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.847842+0000","last_change":"2024-09-17T05:32:42.837749+0000","last_active":"2024-09-17T05:32:42.847842+0000","last_peered":"2024-09-17T05:32:42.847842+0000","last_clean":"2024-09-17T05:32:42.847842+0000","last_became_active":"2024-09-17T05:32:42.837454+0000","last_became_peered":"2024-09-17T05:32:42.837454+0000","last_unstale":"2024-09-17T05:32:42.847842+0000","last_undegraded":"2024-09-17T05:32:42.847842+0000","last_fullsized":"2024-09-17T05:32:42.847842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:42.066596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.830223+0000","last_change":"2024-09-17T05:32:41.837349+0000","last_active":"2024-09-17T05:32:43.830223+0000","last_peered":"2024-09-17T05:32:43.830223+0000","last_clean":"2024-09-17T05:32:43.830223+0000","last_became_active":"2024-09-17T05:32:41.836973+0000","last_became_peered":"2024-09-17T05:32:41.836973+0000","last_unstale":"2024-09-17T05:32:43.830223+0000","last_undegraded":"2024-09-17T05:32:43.830223+0000","last_fullsized":"2024-09-17T05:32:43.830223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:31.224807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.855283+0000","last_change":"2024-09-17T05:32:42.851739+0000","last_active":"2024-09-17T05:32:42.855283+0000","last_peered":"2024-09-17T05:32:42.855283+0000","last_clean":"2024-09-17T05:32:42.855283+0000","last_became_active":"2024-09-17T05:32:42.851446+0000","last_became_peered":"2024-09-17T05:32:42.851446+0000","last_unstale":"2024-09-17T05:32:42.855283+0000","last_undegraded":"2024-09-17T05:32:42.855283+0000","last_fullsized":"2024-09-17T05:32:42.855283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:35.386328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.828731+0000","last_change":"2024-09-17T05:32:41.832200+0000","last_active":"2024-09-17T05:32:42.828731+0000","last_peered":"2024-09-17T05:32:42.828731+0000","last_clean":"2024-09-17T05:32:42.828731+0000","last_became_active":"2024-09-17T05:32:41.831449+0000","last_became_peered":"2024-09-17T05:32:41.831449+0000","last_unstale":"2024-09-17T05:32:42.828731+0000","last_undegraded":"2024-09-17T05:32:42.828731+0000","last_fullsized":"2024-09-17T05:32:42.828731+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:54:54.661383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834323+0000","last_change":"2024-09-17T05:32:42.854816+0000","last_active":"2024-09-17T05:32:43.834323+0000","last_peered":"2024-09-17T05:32:43.834323+0000","last_clean":"2024-09-17T05:32:43.834323+0000","last_became_active":"2024-09-17T05:32:42.849886+0000","last_became_peered":"2024-09-17T05:32:42.849886+0000","last_unstale":"2024-09-17T05:32:43.834323+0000","last_undegraded":"2024-09-17T05:32:43.834323+0000","last_fullsized":"2024-09-17T05:32:43.834323+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:39.193716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.842110+0000","last_change":"2024-09-17T05:32:41.832336+0000","last_active":"2024-09-17T05:32:42.842110+0000","last_peered":"2024-09-17T05:32:42.842110+0000","last_clean":"2024-09-17T05:32:42.842110+0000","last_became_active":"2024-09-17T05:32:41.832053+0000","last_became_peered":"2024-09-17T05:32:41.832053+0000","last_unstale":"2024-09-17T05:32:42.842110+0000","last_undegraded":"2024-09-17T05:32:42.842110+0000","last_fullsized":"2024-09-17T05:32:42.842110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:22:45.631798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.856102+0000","last_change":"2024-09-17T05:32:42.853030+0000","last_active":"2024-09-17T05:32:42.856102+0000","last_peered":"2024-09-17T05:32:42.856102+0000","last_clean":"2024-09-17T05:32:42.856102+0000","last_became_active":"2024-09-17T05:32:42.852758+0000","last_became_peered":"2024-09-17T05:32:42.852758+0000","last_unstale":"2024-09-17T05:32:42.856102+0000","last_undegraded":"2024-09-17T05:32:42.856102+0000","last_fullsized":"2024-09-17T05:32:42.856102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:41.544536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859688+0000","last_change":"2024-09-17T05:32:41.828055+0000","last_active":"2024-09-17T05:32:43.859688+0000","last_peered":"2024-09-17T05:32:43.859688+0000","last_clean":"2024-09-17T05:32:43.859688+0000","last_became_active":"2024-09-17T05:32:41.827184+0000","last_became_peered":"2024-09-17T05:32:41.827184+0000","last_unstale":"2024-09-17T05:32:43.859688+0000","last_undegraded":"2024-09-17T05:32:43.859688+0000","last_fullsized":"2024-09-17T05:32:43.859688+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:50:35.618186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862701+0000","last_change":"2024-09-17T05:32:42.838296+0000","last_active":"2024-09-17T05:32:43.862701+0000","last_peered":"2024-09-17T05:32:43.862701+0000","last_clean":"2024-09-17T05:32:43.862701+0000","last_became_active":"2024-09-17T05:32:42.837476+0000","last_became_peered":"2024-09-17T05:32:42.837476+0000","last_unstale":"2024-09-17T05:32:43.862701+0000","last_undegraded":"2024-09-17T05:32:43.862701+0000","last_fullsized":"2024-09-17T05:32:43.862701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:13.110950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861401+0000","last_change":"2024-09-17T05:32:41.829414+0000","last_active":"2024-09-17T05:32:43.861401+0000","last_peered":"2024-09-17T05:32:43.861401+0000","last_clean":"2024-09-17T05:32:43.861401+0000","last_became_active":"2024-09-17T05:32:41.829122+0000","last_became_peered":"2024-09-17T05:32:41.829122+0000","last_unstale":"2024-09-17T05:32:43.861401+0000","last_undegraded":"2024-09-17T05:32:43.861401+0000","last_fullsized":"2024-09-17T05:32:43.861401+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:05:34.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.846258+0000","last_change":"2024-09-17T05:32:42.840541+0000","last_active":"2024-09-17T05:32:42.846258+0000","last_peered":"2024-09-17T05:32:42.846258+0000","last_clean":"2024-09-17T05:32:42.846258+0000","last_became_active":"2024-09-17T05:32:42.840284+0000","last_became_peered":"2024-09-17T05:32:42.840284+0000","last_unstale":"2024-09-17T05:32:42.846258+0000","last_undegraded":"2024-09-17T05:32:42.846258+0000","last_fullsized":"2024-09-17T05:32:42.846258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:29.189795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833860+0000","last_change":"2024-09-17T05:32:41.828449+0000","last_active":"2024-09-17T05:32:43.833860+0000","last_peered":"2024-09-17T05:32:43.833860+0000","last_clean":"2024-09-17T05:32:43.833860+0000","last_became_active":"2024-09-17T05:32:41.828161+0000","last_became_peered":"2024-09-17T05:32:41.828161+0000","last_unstale":"2024-09-17T05:32:43.833860+0000","last_undegraded":"2024-09-17T05:32:43.833860+0000","last_fullsized":"2024-09-17T05:32:43.833860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:54.450949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862387+0000","last_change":"2024-09-17T05:32:42.842387+0000","last_active":"2024-09-17T05:32:43.862387+0000","last_peered":"2024-09-17T05:32:43.862387+0000","last_clean":"2024-09-17T05:32:43.862387+0000","last_became_active":"2024-09-17T05:32:42.842113+0000","last_became_peered":"2024-09-17T05:32:42.842113+0000","last_unstale":"2024-09-17T05:32:43.862387+0000","last_undegraded":"2024-09-17T05:32:43.862387+0000","last_fullsized":"2024-09-17T05:32:43.862387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:03.521253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.833850+0000","last_change":"2024-09-17T05:32:41.826546+0000","last_active":"2024-09-17T05:32:43.833850+0000","last_peered":"2024-09-17T05:32:43.833850+0000","last_clean":"2024-09-17T05:32:43.833850+0000","last_became_active":"2024-09-17T05:32:41.826239+0000","last_became_peered":"2024-09-17T05:32:41.826239+0000","last_unstale":"2024-09-17T05:32:43.833850+0000","last_undegraded":"2024-09-17T05:32:43.833850+0000","last_fullsized":"2024-09-17T05:32:43.833850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:49.504003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.848375+0000","last_change":"2024-09-17T05:32:42.846398+0000","last_active":"2024-09-17T05:32:42.848375+0000","last_peered":"2024-09-17T05:32:42.848375+0000","last_clean":"2024-09-17T05:32:42.848375+0000","last_became_active":"2024-09-17T05:32:42.846115+0000","last_became_peered":"2024-09-17T05:32:42.846115+0000","last_unstale":"2024-09-17T05:32:42.848375+0000","last_undegraded":"2024-09-17T05:32:42.848375+0000","last_fullsized":"2024-09-17T05:32:42.848375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:52.521645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.834104+0000","last_change":"2024-09-17T05:32:41.836576+0000","last_active":"2024-09-17T05:32:43.834104+0000","last_peered":"2024-09-17T05:32:43.834104+0000","last_clean":"2024-09-17T05:32:43.834104+0000","last_became_active":"2024-09-17T05:32:41.835909+0000","last_became_peered":"2024-09-17T05:32:41.835909+0000","last_unstale":"2024-09-17T05:32:43.834104+0000","last_undegraded":"2024-09-17T05:32:43.834104+0000","last_fullsized":"2024-09-17T05:32:43.834104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:14:28.182337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.859785+0000","last_change":"2024-09-17T05:32:42.849931+0000","last_active":"2024-09-17T05:32:43.859785+0000","last_peered":"2024-09-17T05:32:43.859785+0000","last_clean":"2024-09-17T05:32:43.859785+0000","last_became_active":"2024-09-17T05:32:42.849631+0000","last_became_peered":"2024-09-17T05:32:42.849631+0000","last_unstale":"2024-09-17T05:32:43.859785+0000","last_undegraded":"2024-09-17T05:32:43.859785+0000","last_fullsized":"2024-09-17T05:32:43.859785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:04.686919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.842994+0000","last_change":"2024-09-17T05:32:41.834411+0000","last_active":"2024-09-17T05:32:42.842994+0000","last_peered":"2024-09-17T05:32:42.842994+0000","last_clean":"2024-09-17T05:32:42.842994+0000","last_became_active":"2024-09-17T05:32:41.833919+0000","last_became_peered":"2024-09-17T05:32:41.833919+0000","last_unstale":"2024-09-17T05:32:42.842994+0000","last_undegraded":"2024-09-17T05:32:42.842994+0000","last_fullsized":"2024-09-17T05:32:42.842994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:06.236255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.835552+0000","last_change":"2024-09-17T05:32:42.834174+0000","last_active":"2024-09-17T05:32:42.835552+0000","last_peered":"2024-09-17T05:32:42.835552+0000","last_clean":"2024-09-17T05:32:42.835552+0000","last_became_active":"2024-09-17T05:32:42.833873+0000","last_became_peered":"2024-09-17T05:32:42.833873+0000","last_unstale":"2024-09-17T05:32:42.835552+0000","last_undegraded":"2024-09-17T05:32:42.835552+0000","last_fullsized":"2024-09-17T05:32:42.835552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:37.299090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.829424+0000","last_change":"2024-09-17T05:32:41.834497+0000","last_active":"2024-09-17T05:32:42.829424+0000","last_peered":"2024-09-17T05:32:42.829424+0000","last_clean":"2024-09-17T05:32:42.829424+0000","last_became_active":"2024-09-17T05:32:41.832177+0000","last_became_peered":"2024-09-17T05:32:41.832177+0000","last_unstale":"2024-09-17T05:32:42.829424+0000","last_undegraded":"2024-09-17T05:32:42.829424+0000","last_fullsized":"2024-09-17T05:32:42.829424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:36.177885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.837699+0000","last_change":"2024-09-17T05:32:42.835372+0000","last_active":"2024-09-17T05:32:42.837699+0000","last_peered":"2024-09-17T05:32:42.837699+0000","last_clean":"2024-09-17T05:32:42.837699+0000","last_became_active":"2024-09-17T05:32:42.835076+0000","last_became_peered":"2024-09-17T05:32:42.835076+0000","last_unstale":"2024-09-17T05:32:42.837699+0000","last_undegraded":"2024-09-17T05:32:42.837699+0000","last_fullsized":"2024-09-17T05:32:42.837699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:10.082230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.829895+0000","last_change":"2024-09-17T05:32:41.835605+0000","last_active":"2024-09-17T05:32:43.829895+0000","last_peered":"2024-09-17T05:32:43.829895+0000","last_clean":"2024-09-17T05:32:43.829895+0000","last_became_active":"2024-09-17T05:32:41.835081+0000","last_became_peered":"2024-09-17T05:32:41.835081+0000","last_unstale":"2024-09-17T05:32:43.829895+0000","last_undegraded":"2024-09-17T05:32:43.829895+0000","last_fullsized":"2024-09-17T05:32:43.829895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:27.646267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:32:42.855990+0000","last_change":"2024-09-17T05:32:42.852181+0000","last_active":"2024-09-17T05:32:42.855990+0000","last_peered":"2024-09-17T05:32:42.855990+0000","last_clean":"2024-09-17T05:32:42.855990+0000","last_became_active":"2024-09-17T05:32:42.851740+0000","last_became_peered":"2024-09-17T05:32:42.851740+0000","last_unstale":"2024-09-17T05:32:42.855990+0000","last_undegraded":"2024-09-17T05:32:42.855990+0000","last_fullsized":"2024-09-17T05:32:42.855990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:41.806839+0000","last_clean_scrub_stamp":"2024-09-17T05:32:41.806839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:59.516118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.862099+0000","last_change":"2024-09-17T05:32:41.839819+0000","last_active":"2024-09-17T05:32:43.862099+0000","last_peered":"2024-09-17T05:32:43.862099+0000","last_clean":"2024-09-17T05:32:43.862099+0000","last_became_active":"2024-09-17T05:32:41.839431+0000","last_became_peered":"2024-09-17T05:32:41.839431+0000","last_unstale":"2024-09-17T05:32:43.862099+0000","last_undegraded":"2024-09-17T05:32:43.862099+0000","last_fullsized":"2024-09-17T05:32:43.862099+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:32:40.799381+0000","last_clean_scrub_stamp":"2024-09-17T05:32:40.799381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:56.006570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":213,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:32:43.861719+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:32:43.861719+0000","last_peered":"2024-09-17T05:32:43.861719+0000","last_clean":"2024-09-17T05:32:43.861719+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:32:43.861719+0000","last_undegraded":"2024-09-17T05:32:43.861719+0000","last_fullsized":"2024-09-17T05:32:43.861719+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":905216,"data_stored":380982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":909312,"data_stored":381016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54284,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317556,"statfs":{"total":96636764160,"available":96581177344,"internally_reserved":0,"allocated":905216,"data_stored":380064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509536,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50120,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321720,"statfs":{"total":96636764160,"available":96585441280,"internally_reserved":0,"allocated":905216,"data_stored":379496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509537,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":897024,"data_stored":379952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1486848,"data_stored":969738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50116,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321724,"statfs":{"total":96636764160,"available":96585445376,"internally_reserved":0,"allocated":897024,"data_stored":379440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54856,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316984,"statfs":{"total":96636764160,"available":96580591616,"internally_reserved":0,"allocated":1490944,"data_stored":969828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:32:47.484 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:32:47.484 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:32:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- 172.21.15.122:0/2266128721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74180cd100 msgr2=0x7f741806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 --2- 172.21.15.122:0/2266128721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74180cd100 0x7f741806f7b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7400009900 tx=0x7f740001c7c0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- 172.21.15.122:0/2266128721 shutdown_connections 2024-09-17T05:32:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 --2- 172.21.15.122:0/2266128721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74180b49e0 0x7f74180acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 --2- 172.21.15.122:0/2266128721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74180cd100 0x7f741806f7b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- 172.21.15.122:0/2266128721 >> 172.21.15.122:0/2266128721 conn(0x7f74180b18f0 msgr2=0x7f74180b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- 172.21.15.122:0/2266128721 shutdown_connections 2024-09-17T05:32:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- 172.21.15.122:0/2266128721 wait complete. 2024-09-17T05:32:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 Processor -- start 2024-09-17T05:32:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- start start 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74180b49e0 0x7f7418147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74180cd100 0x7f74181634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741815dae0 0x7f7418163a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74180cf0d0 con 0x7f741815dae0 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74180cef50 con 0x7f74180cd100 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741f000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f74180cf250 con 0x7f74180b49e0 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741815dae0 0x7f7418163a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f740f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74180cd100 0x7f74181634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f7417e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74180b49e0 0x7f7418147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741815dae0 0x7f7418163a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35460/0 (socket says 172.21.15.122:35460) 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741d000640 1 -- 172.21.15.122:0/1836985262 learned_addr learned my addr 172.21.15.122:0/1836985262 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741d000640 1 -- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74180b49e0 msgr2=0x7f7418147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741d000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74180b49e0 0x7f7418147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741d000640 1 -- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74180cd100 msgr2=0x7f74181634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741d000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74180cd100 0x7f74181634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.672+0000 7f741d000640 1 -- 172.21.15.122:0/1836985262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74181f7ac0 con 0x7f741815dae0 2024-09-17T05:32:47.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f741d000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741815dae0 0x7f7418163a00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f741000afd0 tx=0x7f741000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:47.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f7414c00640 1 -- 172.21.15.122:0/1836985262 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410003b80 con 0x7f741815dae0 2024-09-17T05:32:47.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f7414c00640 1 -- 172.21.15.122:0/1836985262 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7410003d20 con 0x7f741815dae0 2024-09-17T05:32:47.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74181fbbf0 con 0x7f741815dae0 2024-09-17T05:32:47.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f7414c00640 1 -- 172.21.15.122:0/1836985262 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410004280 con 0x7f741815dae0 2024-09-17T05:32:47.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181f8060 con 0x7f741815dae0 2024-09-17T05:32:47.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74181f78d0 con 0x7f741815dae0 2024-09-17T05:32:47.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f7414c00640 1 -- 172.21.15.122:0/1836985262 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f7410004420 con 0x7f741815dae0 2024-09-17T05:32:47.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f7414c00640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f73f4073e60 0x7f73f4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:47.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.676+0000 7f7414c00640 1 -- 172.21.15.122:0/1836985262 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f7410093750 con 0x7f741815dae0 2024-09-17T05:32:47.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.680+0000 7f7417e00640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f73f4073e60 0x7f73f4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:47.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.680+0000 7f7417e00640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f73f4073e60 0x7f73f4076320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7418070d60 tx=0x7f7408009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:47.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.680+0000 7f7414c00640 1 -- 172.21.15.122:0/1836985262 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f741005f980 con 0x7f741815dae0 2024-09-17T05:32:47.805 INFO:tasks.ceph.mds.c.smithi122.stderr:2024-09-17T05:32:47.804+0000 7f5161c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:32:47.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.908+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f74180ad8f0 con 0x7f741815dae0 2024-09-17T05:32:47.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f7414c00640 1 -- 172.21.15.122:0/1836985262 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f74100643b0 con 0x7f741815dae0 2024-09-17T05:32:47.912 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:32:47.912 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:32:47.912 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:32:47.912 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:32:47.913 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:32:47.913 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:32:47.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f73f4073e60 msgr2=0x7f73f4076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f73f4073e60 0x7f73f4076320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7418070d60 tx=0x7f7408009040 comp rx=0 tx=0).stop 2024-09-17T05:32:47.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741815dae0 msgr2=0x7f7418163a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741815dae0 0x7f7418163a00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f741000afd0 tx=0x7f741000c7a0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 shutdown_connections 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f73f4073e60 0x7f73f4076320 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741815dae0 0x7f7418163a00 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74180cd100 0x7f74181634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 --2- 172.21.15.122:0/1836985262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74180b49e0 0x7f7418147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 >> 172.21.15.122:0/1836985262 conn(0x7f74180b18f0 msgr2=0x7f74180ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 shutdown_connections 2024-09-17T05:32:47.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:47.912+0000 7f741f000640 1 -- 172.21.15.122:0/1836985262 wait complete. 2024-09-17T05:32:47.933 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:32:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.076+0000 7fb8fe400640 1 -- 172.21.15.122:0/611349907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 msgr2=0x7fb8f80b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.076+0000 7fb8fe400640 1 --2- 172.21.15.122:0/611349907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 0x7fb8f80b42a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80cc630 tx=0x7fb8e001daa0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.076+0000 7fb8fe400640 1 -- 172.21.15.122:0/611349907 shutdown_connections 2024-09-17T05:32:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.076+0000 7fb8fe400640 1 --2- 172.21.15.122:0/611349907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 0x7fb8f80b42a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.076+0000 7fb8fe400640 1 --2- 172.21.15.122:0/611349907 >> v2:172.21.15.173:3300/0 conn(0x7fb8f815a4d0 0x7fb8f815ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.076+0000 7fb8fe400640 1 -- 172.21.15.122:0/611349907 >> 172.21.15.122:0/611349907 conn(0x7fb8f80b18f0 msgr2=0x7fb8f80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.076+0000 7fb8fe400640 1 -- 172.21.15.122:0/611349907 shutdown_connections 2024-09-17T05:32:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.076+0000 7fb8fe400640 1 -- 172.21.15.122:0/611349907 wait complete. 2024-09-17T05:32:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 Processor -- start 2024-09-17T05:32:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 -- start start 2024-09-17T05:32:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 0x7fb8f8158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8f80b47e0 0x7fb8f809aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8f815a4d0 0x7fb8f809b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f80ccd00 con 0x7fb8f80703d0 2024-09-17T05:32:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f80ccb80 con 0x7fb8f815a4d0 2024-09-17T05:32:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f80cce80 con 0x7fb8f80b47e0 2024-09-17T05:32:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 0x7fb8f8158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 0x7fb8f8158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35474/0 (socket says 172.21.15.122:35474) 2024-09-17T05:32:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 -- 172.21.15.122:0/3405555276 learned_addr learned my addr 172.21.15.122:0/3405555276 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8eea00640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8f80b47e0 0x7fb8f809aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 -- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8f80b47e0 msgr2=0x7fb8f809aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7e00640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8f815a4d0 0x7fb8f809b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8f80b47e0 0x7fb8f809aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 -- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8f815a4d0 msgr2=0x7fb8f809b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8f815a4d0 0x7fb8f809b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 -- 172.21.15.122:0/3405555276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f815bb20 con 0x7fb8f80703d0 2024-09-17T05:32:48.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8eea00640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8f80b47e0 0x7fb8f809aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:48.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7e00640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8f815a4d0 0x7fb8f809b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:32:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8f7400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 0x7fb8f8158a90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800db00 tx=0x7fb8e800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8efe00640 1 -- 172.21.15.122:0/3405555276 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8017070 con 0x7fb8f80703d0 2024-09-17T05:32:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8efe00640 1 -- 172.21.15.122:0/3405555276 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8e8004b30 con 0x7fb8f80703d0 2024-09-17T05:32:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8efe00640 1 -- 172.21.15.122:0/3405555276 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8005020 con 0x7fb8f80703d0 2024-09-17T05:32:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f81f7420 con 0x7fb8f80703d0 2024-09-17T05:32:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f81f7cc0 con 0x7fb8f80703d0 2024-09-17T05:32:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.080+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f8069410 con 0x7fb8f80703d0 2024-09-17T05:32:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.084+0000 7fb8efe00640 1 -- 172.21.15.122:0/3405555276 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fb8e80051c0 con 0x7fb8f80703d0 2024-09-17T05:32:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.084+0000 7fb8efe00640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8d4073e60 0x7fb8d4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.084+0000 7fb8efe00640 1 -- 172.21.15.122:0/3405555276 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fb8e8093120 con 0x7fb8f80703d0 2024-09-17T05:32:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.084+0000 7fb8eea00640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8d4073e60 0x7fb8d4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.084+0000 7fb8eea00640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8d4073e60 0x7fb8d4076320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00b0000 tx=0x7fb8e001c2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.088+0000 7fb8efe00640 1 -- 172.21.15.122:0/3405555276 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb8e8020070 con 0x7fb8f80703d0 2024-09-17T05:32:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.292+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8f80b42a0 con 0x7fb8f80703d0 2024-09-17T05:32:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.292+0000 7fb8efe00640 1 -- 172.21.15.122:0/3405555276 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5234 (secure 0 0 0) 0x7fb8e805fc20 con 0x7fb8f80703d0 2024-09-17T05:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:32:48.293 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":46,"btime":"2024-09-17T05:32:45:992259+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3900952275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3900952275},{"type":"v1","addr":"172.21.15.173:6836","nonce":3900952275}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5711,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/645422945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":645422945},{"type":"v1","addr":"172.21.15.173:6837","nonce":645422945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6408,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1968037162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1968037162},{"type":"v1","addr":"172.21.15.122:6833","nonce":1968037162}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:32:42.215056+0000","modified":"2024-09-17T05:32:45.239781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6426},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6426":{"gid":6426,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2505208774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2505208774},{"type":"v1","addr":"172.21.15.122:6835","nonce":2505208774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6426,"qdb_cluster":[6426]},"id":5}]} 2024-09-17T05:32:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 46 2024-09-17T05:32:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8d4073e60 msgr2=0x7fb8d4076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8d4073e60 0x7fb8d4076320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00b0000 tx=0x7fb8e001c2e0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 msgr2=0x7fb8f8158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 0x7fb8f8158a90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800db00 tx=0x7fb8e800b260 comp rx=0 tx=0).stop 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 shutdown_connections 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8d4073e60 0x7fb8d4076320 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8f815a4d0 0x7fb8f809b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8f80b47e0 0x7fb8f809aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 --2- 172.21.15.122:0/3405555276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8f80703d0 0x7fb8f8158a90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 >> 172.21.15.122:0/3405555276 conn(0x7fb8f80b18f0 msgr2=0x7fb8f8163d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 shutdown_connections 2024-09-17T05:32:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.296+0000 7fb8fe400640 1 -- 172.21.15.122:0/3405555276 wait complete. 2024-09-17T05:32:48.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:32:42.215056+0000', 'modified': '2024-09-17T05:32:45.239781+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6426}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6426': {'gid': 6426, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6835/2505208774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 2505208774}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 2505208774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6426, 'qdb_cluster': [6426]} 2024-09-17T05:32:48.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:32:48.717 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:32:48.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.864+0000 7fa878a00640 1 -- 172.21.15.122:0/3982506878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 msgr2=0x7fa858004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.864+0000 7fa878a00640 1 --2- 172.21.15.122:0/3982506878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 0x7fa858004c90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009b30 tx=0x7fa85c01d230 comp rx=0 tx=0).stop 2024-09-17T05:32:48.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.864+0000 7fa872800640 1 -- 172.21.15.122:0/3982506878 reap_dead start 2024-09-17T05:32:48.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.864+0000 7fa878a00640 1 -- 172.21.15.122:0/3982506878 shutdown_connections 2024-09-17T05:32:48.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.864+0000 7fa878a00640 1 -- 172.21.15.122:0/3982506878 >> 172.21.15.122:0/3982506878 conn(0x7fa8740d1e40 msgr2=0x7fa8740d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.864+0000 7fa878a00640 1 -- 172.21.15.122:0/3982506878 shutdown_connections 2024-09-17T05:32:48.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.864+0000 7fa878a00640 1 -- 172.21.15.122:0/3982506878 wait complete. 2024-09-17T05:32:48.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 Processor -- start 2024-09-17T05:32:48.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 -- start start 2024-09-17T05:32:48.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 0x7fa8740b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8740cd900 0x7fa8740cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8740b69c0 0x7fa8740b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8740b06d0 con 0x7fa858004810 2024-09-17T05:32:48.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa8740b0550 con 0x7fa8740b69c0 2024-09-17T05:32:48.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa8740b0850 con 0x7fa8740cd900 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 0x7fa8740b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 0x7fa8740b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35496/0 (socket says 172.21.15.122:35496) 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 -- 172.21.15.122:0/3447657728 learned_addr learned my addr 172.21.15.122:0/3447657728 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa869400640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8740cd900 0x7fa8740cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa872800640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8740b69c0 0x7fa8740b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 -- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8740cd900 msgr2=0x7fa8740cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8740cd900 0x7fa8740cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 -- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8740b69c0 msgr2=0x7fa8740b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8740b69c0 0x7fa8740b6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 -- 172.21.15.122:0/3447657728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8741f8940 con 0x7fa858004810 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa871e00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 0x7fa8740b6230 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa864006680 tx=0x7fa8640039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa86aa00640 1 -- 172.21.15.122:0/3447657728 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864015070 con 0x7fa858004810 2024-09-17T05:32:48.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa86aa00640 1 -- 172.21.15.122:0/3447657728 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa864011900 con 0x7fa858004810 2024-09-17T05:32:48.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa86aa00640 1 -- 172.21.15.122:0/3447657728 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864011d50 con 0x7fa858004810 2024-09-17T05:32:48.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 -- 172.21.15.122:0/3447657728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8740b6770 con 0x7fa858004810 2024-09-17T05:32:48.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.868+0000 7fa878a00640 1 -- 172.21.15.122:0/3447657728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741f7e60 con 0x7fa858004810 2024-09-17T05:32:48.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa86aa00640 1 -- 172.21.15.122:0/3447657728 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fa864018070 con 0x7fa858004810 2024-09-17T05:32:48.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa86aa00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa850073e90 0x7fa850076350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa87bd43000 1 Processor -- start 2024-09-17T05:32:48.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa86aa00640 1 -- 172.21.15.122:0/3447657728 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fa864093580 con 0x7fa858004810 2024-09-17T05:32:48.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa87bd43000 1 -- start start 2024-09-17T05:32:48.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa850073e90 0x7fa850076350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314ce2f0 0x5628314818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa850073e90 0x7fa850076350 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0ae000 tx=0x7fa85c019fd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562831481e10 0x5628314821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa87bd43000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5628313a23f0 con 0x5628314ce2f0 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa87bd43000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5628312dfb20 con 0x562831481e10 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa87bd43000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5628312cef50 con 0x5628314cd9d0 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314ce2f0 0x5628314818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314ce2f0 0x5628314818d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35506/0 (socket says 172.21.15.122:35506) 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 -- 172.21.15.122:0/677633319 learned_addr learned my addr 172.21.15.122:0/677633319 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa871e00640 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa872800640 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562831481e10 0x5628314821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 -- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 msgr2=0x5628314cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 -- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562831481e10 msgr2=0x5628314821f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562831481e10 0x5628314821f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa869400640 1 -- 172.21.15.122:0/677633319 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562831296590 con 0x5628314ce2f0 2024-09-17T05:32:48.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa872800640 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562831481e10 0x5628314821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.872+0000 7fa871e00640 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa869400640 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314ce2f0 0x5628314818d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa85c01bab0 tx=0x7fa85c0b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b9426ee28ec38d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa862a00640 1 -- 172.21.15.122:0/677633319 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0b1040 con 0x5628314ce2f0 2024-09-17T05:32:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa862a00640 1 -- 172.21.15.122:0/677633319 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa85c0bc030 con 0x5628314ce2f0 2024-09-17T05:32:48.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa862a00640 1 -- 172.21.15.122:0/677633319 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c004b80 con 0x5628314ce2f0 2024-09-17T05:32:48.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- 172.21.15.122:0/677633319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314ce2f0 msgr2=0x5628314818d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314ce2f0 0x5628314818d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa85c01bab0 tx=0x7fa85c0b6000 comp rx=0 tx=0).stop 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- 172.21.15.122:0/677633319 shutdown_connections 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562831481e10 0x5628314821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314ce2f0 0x5628314818d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 --2- 172.21.15.122:0/677633319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- 172.21.15.122:0/677633319 >> 172.21.15.122:0/677633319 conn(0x56283136b130 msgr2=0x5628314b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- 172.21.15.122:0/677633319 shutdown_connections 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- 172.21.15.122:0/677633319 wait complete. 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- start start 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 Processor -- start 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x5628314ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 0x5628314af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5628312695d0 con 0x562831481e10 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5628312d2d60 con 0x5628314ce2f0 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x562831275500 con 0x5628314cd9d0 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa87bd43000 10 client.0 _ll_register_callbacks cb 0x562831279810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa871e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x5628314ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa871e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x5628314ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35516/0 (socket says 172.21.15.122:35516) 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa871e00640 1 -- 172.21.15.122:0/3291312539 learned_addr learned my addr 172.21.15.122:0/3291312539 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa872800640 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 0x5628314af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa869400640 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa871e00640 1 -- 172.21.15.122:0/3291312539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 msgr2=0x5628314aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa871e00640 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa871e00640 1 -- 172.21.15.122:0/3291312539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 msgr2=0x5628314af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa871e00640 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 0x5628314af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.876+0000 7fa871e00640 1 -- 172.21.15.122:0/3291312539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283126d170 con 0x562831481e10 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa871e00640 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x5628314ae6a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa86405f680 tx=0x7fa86405efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c24489a73c830b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 1 -- 172.21.15.122:0/3291312539 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864085070 con 0x562831481e10 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 1 -- 172.21.15.122:0/3291312539 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fa86404cb60 con 0x562831481e10 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 -- 172.21.15.122:0/3291312539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5628312670f0 con 0x562831481e10 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 1 -- 172.21.15.122:0/3291312539 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa86401f070 con 0x562831481e10 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 10 client.6537 fetch_fsmap learned FSMap version 46 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 -- 172.21.15.122:0/3291312539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x562831353ef0 con 0x562831481e10 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 1 -- 172.21.15.122:0/3291312539 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5628312670f0 con 0x562831481e10 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 1 -- 172.21.15.122:0/3291312539 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7fa86405e650 con 0x562831481e10 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 10 client.6537 fetch_fsmap finished waiting for FSMap version 46 2024-09-17T05:32:48.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 10 client.6537 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:32:48.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 20 client.6537 populate_metadata read hostname 'smithi122' 2024-09-17T05:32:48.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] conn(0x5628314b00f0 0x5628314dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 4 client.6537 mds_command: new command op to 6426 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:32:48.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 -- 172.21.15.122:0/3291312539 --> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628312e1a70 con 0x5628314b00f0 2024-09-17T05:32:48.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa869400640 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] conn(0x5628314b00f0 0x5628314dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 10 client.6537 ms_handle_connect on v2:172.21.15.122:6834/2505208774 2024-09-17T05:32:48.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa869400640 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] conn(0x5628314b00f0 0x5628314dfc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6c20cc620de43c server_cookie=c0954c071112e50e in_seq=0 out_seq=0 2024-09-17T05:32:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 1 -- 172.21.15.122:0/3291312539 <== mds.0 v2:172.21.15.122:6834/2505208774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628312e1a70 con 0x5628314b00f0 2024-09-17T05:32:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 10 client.6537 handle_command_reply: tid=0 2024-09-17T05:32:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 client.6537 shutdown 2024-09-17T05:32:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 -- 172.21.15.122:0/3291312539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 msgr2=0x5628314ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x5628314ae6a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa86405f680 tx=0x7fa86405efb0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 -- 172.21.15.122:0/3291312539 shutdown_connections 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] conn(0x5628314b00f0 0x5628314dfc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 0x5628314af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314cd9d0 0x5628314aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 --2- 172.21.15.122:0/3291312539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x5628314ae6a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa857400640 0 client.6537 ms_handle_reset on v2:172.21.15.122:6834/2505208774 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 -- 172.21.15.122:0/3291312539 >> 172.21.15.122:0/3291312539 conn(0x56283136b130 msgr2=0x562831480fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 -- 172.21.15.122:0/3291312539 shutdown_connections 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 1 -- 172.21.15.122:0/3291312539 wait complete. 2024-09-17T05:32:48.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.880+0000 7fa87bd43000 20 client.6537 trim_cache size 0 max 16384 2024-09-17T05:32:48.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 Processor -- start 2024-09-17T05:32:48.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- start start 2024-09-17T05:32:48.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x56283153fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314cd9d0 0x562831540310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314ce2f0 0x562831540850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa874204ce0 con 0x562831481e10 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa874203d00 con 0x5628314cd9d0 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa87406ea00 con 0x5628314ce2f0 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x56283153fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x56283153fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35526/0 (socket says 172.21.15.122:35526) 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 -- 172.21.15.122:0/400313739 learned_addr learned my addr 172.21.15.122:0/400313739 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa869400640 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314cd9d0 0x562831540310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa872800640 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314ce2f0 0x562831540850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 -- 172.21.15.122:0/400313739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314ce2f0 msgr2=0x562831540850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314ce2f0 0x562831540850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 -- 172.21.15.122:0/400313739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314cd9d0 msgr2=0x562831540310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314cd9d0 0x562831540310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 -- 172.21.15.122:0/400313739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283132ad90 con 0x562831481e10 2024-09-17T05:32:48.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa871e00640 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x56283153fdd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa864000c10 tx=0x7fa8640012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32d644665728965 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa862000640 1 -- 172.21.15.122:0/400313739 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864060080 con 0x562831481e10 2024-09-17T05:32:48.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa862000640 1 -- 172.21.15.122:0/400313739 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa86405f980 con 0x562831481e10 2024-09-17T05:32:48.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa862000640 1 -- 172.21.15.122:0/400313739 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864064460 con 0x562831481e10 2024-09-17T05:32:48.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- 172.21.15.122:0/400313739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 msgr2=0x56283153fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x56283153fdd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa864000c10 tx=0x7fa8640012d0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- 172.21.15.122:0/400313739 shutdown_connections 2024-09-17T05:32:48.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5628314ce2f0 0x562831540850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314cd9d0 0x562831540310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 --2- 172.21.15.122:0/400313739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562831481e10 0x56283153fdd0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- 172.21.15.122:0/400313739 >> 172.21.15.122:0/400313739 conn(0x56283136b130 msgr2=0x562831489a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- 172.21.15.122:0/400313739 shutdown_connections 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.900+0000 7fa87bd43000 1 -- 172.21.15.122:0/400313739 wait complete. 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 -- start start 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 Processor -- start 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562831481e10 0x562831543c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314cd9d0 0x562831544140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 0x562831544680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5628312695d0 con 0x5628314cd9d0 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5628314e4ce0 con 0x5628314ce2f0 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5628312d7390 con 0x562831481e10 2024-09-17T05:32:48.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 10 client.0 _ll_register_callbacks cb 0x7fa864011ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314cd9d0 0x562831544140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314cd9d0 0x562831544140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35534/0 (socket says 172.21.15.122:35534) 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 -- 172.21.15.122:0/2057647600 learned_addr learned my addr 172.21.15.122:0/2057647600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa871e00640 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562831481e10 0x562831543c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa872800640 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 0x562831544680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 -- 172.21.15.122:0/2057647600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562831481e10 msgr2=0x562831543c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562831481e10 0x562831543c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 -- 172.21.15.122:0/2057647600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 msgr2=0x562831544680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 0x562831544680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 -- 172.21.15.122:0/2057647600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562831295a90 con 0x5628314cd9d0 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa869400640 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314cd9d0 0x562831544140 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa85c01bab0 tx=0x7fa85c01a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e4baaeada80494 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa857400640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:32:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa857400640 1 -- 172.21.15.122:0/2057647600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0b1040 con 0x5628314cd9d0 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa857400640 1 -- 172.21.15.122:0/2057647600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fa85c0b2040 con 0x5628314cd9d0 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 -- 172.21.15.122:0/2057647600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86405f980 con 0x5628314cd9d0 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa857400640 1 -- 172.21.15.122:0/2057647600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa85c000bd0 con 0x5628314cd9d0 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 10 client.6543 fetch_fsmap learned FSMap version 46 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 -- 172.21.15.122:0/2057647600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x562831274050 con 0x5628314cd9d0 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa857400640 1 -- 172.21.15.122:0/2057647600 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa86405f980 con 0x5628314cd9d0 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa857400640 1 -- 172.21.15.122:0/2057647600 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7fa85c004a50 con 0x5628314cd9d0 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 10 client.6543 fetch_fsmap finished waiting for FSMap version 46 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 10 client.6543 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 20 client.6543 populate_metadata read hostname 'smithi122' 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] conn(0x5628314e23c0 0x5628314e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 4 client.6543 mds_command: new command op to 6426 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa87bd43000 1 -- 172.21.15.122:0/2057647600 --> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] -- command(tid 0: {"prefix": "status"}) -- 0x5628312dd520 con 0x5628314e23c0 2024-09-17T05:32:48.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.904+0000 7fa871e00640 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] conn(0x5628314e23c0 0x5628314e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa871e00640 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] conn(0x5628314e23c0 0x5628314e27a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3061be5880a90402 server_cookie=3bbdd8b66ac6c8d5 in_seq=0 out_seq=0 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa857400640 10 client.6543 ms_handle_connect on v2:172.21.15.122:6834/2505208774 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa857400640 1 -- 172.21.15.122:0/2057647600 <== mds.0 v2:172.21.15.122:6834/2505208774 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5628312dd520 con 0x5628314e23c0 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa857400640 10 client.6543 handle_command_reply: tid=0 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 client.6543 shutdown 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 -- 172.21.15.122:0/2057647600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314cd9d0 msgr2=0x562831544140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314cd9d0 0x562831544140 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa85c01bab0 tx=0x7fa85c01a350 comp rx=0 tx=0).stop 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 -- 172.21.15.122:0/2057647600 shutdown_connections 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.122:6834/2505208774,v1:172.21.15.122:6835/2505208774] conn(0x5628314e23c0 0x5628314e27a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5628314ce2f0 0x562831544680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5628314cd9d0 0x562831544140 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 --2- 172.21.15.122:0/2057647600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562831481e10 0x562831543c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 -- 172.21.15.122:0/2057647600 >> 172.21.15.122:0/2057647600 conn(0x56283136b130 msgr2=0x5628314cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa857400640 0 client.6543 ms_handle_reset on v2:172.21.15.122:6834/2505208774 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 -- 172.21.15.122:0/2057647600 shutdown_connections 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 1 -- 172.21.15.122:0/2057647600 wait complete. 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa87bd43000 20 client.6543 trim_cache size 0 max 16384 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 6426, 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:32:48.912 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:32:48.913 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 6.0702436430000004, 2024-09-17T05:32:48.913 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 46, 2024-09-17T05:32:48.913 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 77, 2024-09-17T05:32:48.913 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 75, 2024-09-17T05:32:48.913 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 11.653232961000001 2024-09-17T05:32:48.913 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 -- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa850073e90 msgr2=0x7fa850076350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa850073e90 0x7fa850076350 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0ae000 tx=0x7fa85c019fd0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 -- 172.21.15.122:0/3447657728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 msgr2=0x7fa8740b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 0x7fa8740b6230 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa864006680 tx=0x7fa8640039d0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 -- 172.21.15.122:0/3447657728 shutdown_connections 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa850073e90 0x7fa850076350 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8740b69c0 0x7fa8740b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8740cd900 0x7fa8740cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 --2- 172.21.15.122:0/3447657728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa858004810 0x7fa8740b6230 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 -- 172.21.15.122:0/3447657728 >> 172.21.15.122:0/3447657728 conn(0x7fa8740d1e40 msgr2=0x7fa8740b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 -- 172.21.15.122:0/3447657728 shutdown_connections 2024-09-17T05:32:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:48.908+0000 7fa878a00640 1 -- 172.21.15.122:0/3447657728 wait complete. 2024-09-17T05:32:48.928 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- 172.21.15.122:0/3027634261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 msgr2=0x7f000c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 --2- 172.21.15.122:0/3027634261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 0x7f000c0aeef0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0004009bc0 tx=0x7f000401cba0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- 172.21.15.122:0/3027634261 shutdown_connections 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 --2- 172.21.15.122:0/3027634261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 0x7f000c0aeef0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 --2- 172.21.15.122:0/3027634261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f000c0ced80 0x7f000c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- 172.21.15.122:0/3027634261 >> 172.21.15.122:0/3027634261 conn(0x7f000c0d1dc0 msgr2=0x7f000c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- 172.21.15.122:0/3027634261 shutdown_connections 2024-09-17T05:32:49.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- 172.21.15.122:0/3027634261 wait complete. 2024-09-17T05:32:49.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 Processor -- start 2024-09-17T05:32:49.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- start start 2024-09-17T05:32:49.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 0x7f000c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f000c0ced80 0x7f000c1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f000c15bea0 0x7f000c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f000c169340 con 0x7f000c0cdb30 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f000c1691c0 con 0x7f000c15bea0 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0012c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f000c1694c0 con 0x7f000c0ced80 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f000be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 0x7f000c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f000be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 0x7f000c0b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35554/0 (socket says 172.21.15.122:35554) 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f000be00640 1 -- 172.21.15.122:0/1488532195 learned_addr learned my addr 172.21.15.122:0/1488532195 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0003400640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f000c0ced80 0x7f000c1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f0010c00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f000c15bea0 0x7f000c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f000be00640 1 -- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f000c0ced80 msgr2=0x7f000c1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f000be00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f000c0ced80 0x7f000c1502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f000be00640 1 -- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f000c15bea0 msgr2=0x7f000c06f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f000be00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f000c15bea0 0x7f000c06f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.072+0000 7f000be00640 1 -- 172.21.15.122:0/1488532195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c06dd30 con 0x7f000c0cdb30 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.076+0000 7f000be00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 0x7f000c0b1570 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c430 tx=0x7efffc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.076+0000 7f0008c00640 1 -- 172.21.15.122:0/1488532195 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc011790 con 0x7f000c0cdb30 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.076+0000 7f0008c00640 1 -- 172.21.15.122:0/1488532195 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efffc017070 con 0x7f000c0cdb30 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.076+0000 7f0008c00640 1 -- 172.21.15.122:0/1488532195 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc011ce0 con 0x7f000c0cdb30 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.076+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c06e010 con 0x7f000c0cdb30 2024-09-17T05:32:49.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.076+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c06e5d0 con 0x7f000c0cdb30 2024-09-17T05:32:49.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.076+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c0ce670 con 0x7f000c0cdb30 2024-09-17T05:32:49.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.076+0000 7f0008c00640 1 -- 172.21.15.122:0/1488532195 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7efffc01c460 con 0x7f000c0cdb30 2024-09-17T05:32:49.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.080+0000 7f0008c00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effe8073e60 0x7effe8076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:49.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.080+0000 7f0008c00640 1 -- 172.21.15.122:0/1488532195 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7efffc092990 con 0x7f000c0cdb30 2024-09-17T05:32:49.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.080+0000 7f0003400640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effe8073e60 0x7effe8076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:49.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.080+0000 7f0003400640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effe8073e60 0x7effe8076320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efff40058d0 tx=0x7efff4005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:49.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.084+0000 7f0008c00640 1 -- 172.21.15.122:0/1488532195 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efffc05f550 con 0x7f000c0cdb30 2024-09-17T05:32:49.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.288+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f000c0cea50 con 0x7f000c0cdb30 2024-09-17T05:32:49.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.288+0000 7f0008c00640 1 -- 172.21.15.122:0/1488532195 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7efffc05ee50 con 0x7f000c0cdb30 2024-09-17T05:32:49.289 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:32:49.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effe8073e60 msgr2=0x7effe8076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effe8073e60 0x7effe8076320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efff40058d0 tx=0x7efff4005860 comp rx=0 tx=0).stop 2024-09-17T05:32:49.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 msgr2=0x7f000c0b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:49.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 0x7f000c0b1570 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c430 tx=0x7efffc00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 shutdown_connections 2024-09-17T05:32:49.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effe8073e60 0x7effe8076320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f000c15bea0 0x7f000c06f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f000c0ced80 0x7f000c1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 --2- 172.21.15.122:0/1488532195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f000c0cdb30 0x7f000c0b1570 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:49.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 >> 172.21.15.122:0/1488532195 conn(0x7f000c0d1dc0 msgr2=0x7f000c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:49.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 shutdown_connections 2024-09-17T05:32:49.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49.292+0000 7f0012c00640 1 -- 172.21.15.122:0/1488532195 wait complete. 2024-09-17T05:32:49.310 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:32:49.310 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:32:49.310 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:32:49.310 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:32:49.310 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:49.311 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:32:49.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:32:49.311 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:32:49.311 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:49.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:49.311 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:32:49.320 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:32:49.320 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:32:49.320 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:49.321 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 41998sec preferred_lft 41998sec 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:32:49.322 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:32:49.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:49.323 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:32:49.384 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:32:49.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:49.386 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:32:49.438 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:32:49.438 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:32:49.438 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:32:49.438 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:32:49.438 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:32:49.438 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:32:49.438 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:32:49.439 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:32:49.439 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:32:49.439 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:32:49.439 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:32:49.439 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:32:49.439 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:32:49.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:32:49.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:32:49.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:32:49.439 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:32:49.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:32:49.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:32:49.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:49.570 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:32:49.618 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:32:49.618 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:32:49.619 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:49.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:49.619 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:32:49.682 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:32:49.682 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:49.730 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:32:49.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:32:49.730 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:49.777 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:32:49.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:49.778 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:49.867 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:49.868 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:49.868 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:32:49.883 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:49.883 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:49.883 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:32:49.883 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:32:49.883 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:32:49.894 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:32:49.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:32:49.894 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:32:49.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:32:49.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:32:49.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:32:49.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:32:49.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:32:49.898 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:32:49.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:49.898 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:32:49.918 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:32:50.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:32:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:32:50.024 INFO:teuthology.orchestra.run.smithi122.stdout:client6283 2024-09-17T05:32:50.025 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:32:50.025 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:50.033 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:32:50.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.216+0000 7f6cf5200640 1 -- 172.21.15.122:0/1596466825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015be10 msgr2=0x7f6cf00ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.216+0000 7f6cf5200640 1 --2- 172.21.15.122:0/1596466825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015be10 0x7f6cf00ce700 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009c80 tx=0x7f6ce001cda0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 -- 172.21.15.122:0/1596466825 shutdown_connections 2024-09-17T05:32:50.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 --2- 172.21.15.122:0/1596466825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf00b0b60 0x7f6cf00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 --2- 172.21.15.122:0/1596466825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015be10 0x7f6cf00ce700 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 -- 172.21.15.122:0/1596466825 >> 172.21.15.122:0/1596466825 conn(0x7f6cf00d1da0 msgr2=0x7f6cf00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:50.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 -- 172.21.15.122:0/1596466825 shutdown_connections 2024-09-17T05:32:50.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 -- 172.21.15.122:0/1596466825 wait complete. 2024-09-17T05:32:50.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 Processor -- start 2024-09-17T05:32:50.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 -- start start 2024-09-17T05:32:50.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf00b0b60 0x7f6cf01504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf015be10 0x7f6cf006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015c1f0 0x7f6cf006e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0169040 con 0x7f6cf015c1f0 2024-09-17T05:32:50.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0168ec0 con 0x7f6cf015be10 2024-09-17T05:32:50.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cf5200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf01691c0 con 0x7f6cf00b0b60 2024-09-17T05:32:50.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015c1f0 0x7f6cf006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cede00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf00b0b60 0x7f6cf01504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015c1f0 0x7f6cf006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35578/0 (socket says 172.21.15.122:35578) 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 -- 172.21.15.122:0/4260432446 learned_addr learned my addr 172.21.15.122:0/4260432446 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6ce5400640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf015be10 0x7f6cf006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 -- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf00b0b60 msgr2=0x7f6cf01504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf00b0b60 0x7f6cf01504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 -- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf015be10 msgr2=0x7f6cf006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf015be10 0x7f6cf006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 -- 172.21.15.122:0/4260432446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf0159770 con 0x7f6cf015c1f0 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.220+0000 7f6cee800640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015c1f0 0x7f6cf006e700 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800d750 tx=0x7f6ce800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6ce6a00640 1 -- 172.21.15.122:0/4260432446 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8017070 con 0x7f6cf015c1f0 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6ce6a00640 1 -- 172.21.15.122:0/4260432446 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ce8009d20 con 0x7f6cf015c1f0 2024-09-17T05:32:50.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6ce6a00640 1 -- 172.21.15.122:0/4260432446 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800de60 con 0x7f6cf015c1f0 2024-09-17T05:32:50.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf0159a50 con 0x7f6cf015c1f0 2024-09-17T05:32:50.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf015a010 con 0x7f6cf015c1f0 2024-09-17T05:32:50.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf00ccbd0 con 0x7f6cf015c1f0 2024-09-17T05:32:50.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6ce6a00640 1 -- 172.21.15.122:0/4260432446 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f6ce8005280 con 0x7f6cf015c1f0 2024-09-17T05:32:50.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6ce6a00640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6ccc073e60 0x7f6ccc076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:50.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6ce6a00640 1 -- 172.21.15.122:0/4260432446 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f6ce8092ef0 con 0x7f6cf015c1f0 2024-09-17T05:32:50.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.224+0000 7f6cede00640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6ccc073e60 0x7f6ccc076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:50.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.228+0000 7f6cede00640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6ccc073e60 0x7f6ccc076320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00af000 tx=0x7f6ce001b5e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:50.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.228+0000 7f6ce6a00640 1 -- 172.21.15.122:0/4260432446 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6ce805f120 con 0x7f6cf015c1f0 2024-09-17T05:32:50.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:50.432+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f6cf00b0310 con 0x7f6cf015c1f0 2024-09-17T05:32:51.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.024+0000 7f6ce6a00640 1 -- 172.21.15.122:0/4260432446 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f6ce8063b50 con 0x7f6cf015c1f0 2024-09-17T05:32:51.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.024+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6ccc073e60 msgr2=0x7f6ccc076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6ccc073e60 0x7f6ccc076320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00af000 tx=0x7f6ce001b5e0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015c1f0 msgr2=0x7f6cf006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015c1f0 0x7f6cf006e700 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800d750 tx=0x7f6ce800dc10 comp rx=0 tx=0).stop 2024-09-17T05:32:51.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 shutdown_connections 2024-09-17T05:32:51.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6ccc073e60 0x7f6ccc076320 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6cf015c1f0 0x7f6cf006e700 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf015be10 0x7f6cf006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 --2- 172.21.15.122:0/4260432446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf00b0b60 0x7f6cf01504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 >> 172.21.15.122:0/4260432446 conn(0x7f6cf00d1da0 msgr2=0x7f6cf0151ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 shutdown_connections 2024-09-17T05:32:51.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.028+0000 7f6cf5200640 1 -- 172.21.15.122:0/4260432446 wait complete. 2024-09-17T05:32:51.049 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:32:51.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 -- 172.21.15.122:0/606435984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c15dae0 msgr2=0x7f845c0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 --2- 172.21.15.122:0/606435984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c15dae0 0x7f845c0acd40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f844c00a950 tx=0x7f844c01dae0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 -- 172.21.15.122:0/606435984 shutdown_connections 2024-09-17T05:32:51.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 --2- 172.21.15.122:0/606435984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f845c0cd100 0x7f845c0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 --2- 172.21.15.122:0/606435984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c15dae0 0x7f845c0acd40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 -- 172.21.15.122:0/606435984 >> 172.21.15.122:0/606435984 conn(0x7f845c0b18f0 msgr2=0x7f845c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 -- 172.21.15.122:0/606435984 shutdown_connections 2024-09-17T05:32:51.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 -- 172.21.15.122:0/606435984 wait complete. 2024-09-17T05:32:51.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 Processor -- start 2024-09-17T05:32:51.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.192+0000 7f8460e00640 1 -- start start 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f845c0b4810 0x7f845c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c0cd100 0x7f845c157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f845c15dae0 0x7f845c1ed900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f845c1674b0 con 0x7f845c0cd100 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f845c167330 con 0x7f845c15dae0 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f845c167630 con 0x7f845c0b4810 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c0cd100 0x7f845c157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c0cd100 0x7f845c157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35598/0 (socket says 172.21.15.122:35598) 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 -- 172.21.15.122:0/554293075 learned_addr learned my addr 172.21.15.122:0/554293075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f845a800640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f845c15dae0 0x7f845c1ed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8459e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f845c0b4810 0x7f845c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 -- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f845c0b4810 msgr2=0x7f845c09aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f845c0b4810 0x7f845c09aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 -- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f845c15dae0 msgr2=0x7f845c1ed900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f845c15dae0 0x7f845c1ed900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 -- 172.21.15.122:0/554293075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c157fc0 con 0x7f845c0cd100 2024-09-17T05:32:51.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8453e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c0cd100 0x7f845c157a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f844400d980 tx=0x7f844400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8452000640 1 -- 172.21.15.122:0/554293075 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444017070 con 0x7f845c0cd100 2024-09-17T05:32:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8452000640 1 -- 172.21.15.122:0/554293075 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8444009d20 con 0x7f845c0cd100 2024-09-17T05:32:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8452000640 1 -- 172.21.15.122:0/554293075 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444004280 con 0x7f845c0cd100 2024-09-17T05:32:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845c15b0f0 con 0x7f845c0cd100 2024-09-17T05:32:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845c15b6b0 con 0x7f845c0cd100 2024-09-17T05:32:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845c0b42d0 con 0x7f845c0cd100 2024-09-17T05:32:51.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.196+0000 7f8452000640 1 -- 172.21.15.122:0/554293075 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f8444003600 con 0x7f845c0cd100 2024-09-17T05:32:51.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.200+0000 7f8452000640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8438073e60 0x7f8438076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.200+0000 7f8452000640 1 -- 172.21.15.122:0/554293075 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f8444092fb0 con 0x7f845c0cd100 2024-09-17T05:32:51.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.200+0000 7f8459e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8438073e60 0x7f8438076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.200+0000 7f8459e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8438073e60 0x7f8438076320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f844c0b0000 tx=0x7f844c01c320 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.204+0000 7f8452000640 1 -- 172.21.15.122:0/554293075 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f844405f830 con 0x7f845c0cd100 2024-09-17T05:32:51.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.392+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f845c0af480 con 0x7f8438073e60 2024-09-17T05:32:51.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8452000640 1 -- 172.21.15.122:0/554293075 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f845c0af480 con 0x7f8438073e60 2024-09-17T05:32:51.397 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:32:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:32:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:32:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:32:51.397 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8438073e60 msgr2=0x7f8438076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8438073e60 0x7f8438076320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f844c0b0000 tx=0x7f844c01c320 comp rx=0 tx=0).stop 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c0cd100 msgr2=0x7f845c157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c0cd100 0x7f845c157a80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f844400d980 tx=0x7f844400de40 comp rx=0 tx=0).stop 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 shutdown_connections 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8438073e60 0x7f8438076320 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f845c15dae0 0x7f845c1ed900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f845c0cd100 0x7f845c157a80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 --2- 172.21.15.122:0/554293075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f845c0b4810 0x7f845c09aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 >> 172.21.15.122:0/554293075 conn(0x7f845c0b18f0 msgr2=0x7f845c0add40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 shutdown_connections 2024-09-17T05:32:51.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.396+0000 7f8460e00640 1 -- 172.21.15.122:0/554293075 wait complete. 2024-09-17T05:32:51.418 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:32:51.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 -- 172.21.15.122:0/1918637247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380d22c0 msgr2=0x7f88380cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 --2- 172.21.15.122:0/1918637247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380d22c0 0x7f88380cc620 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8830009bc0 tx=0x7f883001cba0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 -- 172.21.15.122:0/1918637247 shutdown_connections 2024-09-17T05:32:51.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 --2- 172.21.15.122:0/1918637247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380d22c0 0x7f88380cc620 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 --2- 172.21.15.122:0/1918637247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f883815bd40 0x7f88380d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 -- 172.21.15.122:0/1918637247 >> 172.21.15.122:0/1918637247 conn(0x7f883809aa10 msgr2=0x7f883809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 -- 172.21.15.122:0/1918637247 shutdown_connections 2024-09-17T05:32:51.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 -- 172.21.15.122:0/1918637247 wait complete. 2024-09-17T05:32:51.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 Processor -- start 2024-09-17T05:32:51.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.564+0000 7f8837e00640 1 -- start start 2024-09-17T05:32:51.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380b0d10 0x7f883815ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88380d22c0 0x7f883815de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f883815bd40 0x7f883815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8838168d00 con 0x7f88380b0d10 2024-09-17T05:32:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8838168b80 con 0x7f883815bd40 2024-09-17T05:32:51.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8838168e80 con 0x7f88380d22c0 2024-09-17T05:32:51.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380b0d10 0x7f883815ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380b0d10 0x7f883815ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46946/0 (socket says 172.21.15.122:46946) 2024-09-17T05:32:51.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 -- 172.21.15.122:0/1323605935 learned_addr learned my addr 172.21.15.122:0/1323605935 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:51.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 -- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88380d22c0 msgr2=0x7f883815de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88380d22c0 0x7f883815de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 -- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f883815bd40 msgr2=0x7f883815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f883815bd40 0x7f883815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 -- 172.21.15.122:0/1323605935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883815d140 con 0x7f88380b0d10 2024-09-17T05:32:51.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8836a00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380b0d10 0x7f883815ea30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f882800e7d0 tx=0x7f882800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f882fe00640 1 -- 172.21.15.122:0/1323605935 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882800ccc0 con 0x7f88380b0d10 2024-09-17T05:32:51.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f882fe00640 1 -- 172.21.15.122:0/1323605935 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8828010430 con 0x7f88380b0d10 2024-09-17T05:32:51.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883815d420 con 0x7f88380b0d10 2024-09-17T05:32:51.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f882fe00640 1 -- 172.21.15.122:0/1323605935 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828010860 con 0x7f88380b0d10 2024-09-17T05:32:51.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883815d9e0 con 0x7f88380b0d10 2024-09-17T05:32:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883815dc20 con 0x7f88380b0d10 2024-09-17T05:32:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.568+0000 7f882fe00640 1 -- 172.21.15.122:0/1323605935 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f8828010a00 con 0x7f88380b0d10 2024-09-17T05:32:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.572+0000 7f882fe00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8814073e60 0x7f8814076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.572+0000 7f882fe00640 1 -- 172.21.15.122:0/1323605935 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f8828093630 con 0x7f88380b0d10 2024-09-17T05:32:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.572+0000 7f882e000640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8814073e60 0x7f8814076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.572+0000 7f882e000640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8814073e60 0x7f8814076320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f88200046c0 tx=0x7f8820007910 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.576+0000 7f882fe00640 1 -- 172.21.15.122:0/1323605935 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f883815dc20 con 0x7f88380b0d10 2024-09-17T05:32:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.740+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f88380cc620 con 0x7f88380b0d10 2024-09-17T05:32:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.740+0000 7f882fe00640 1 -- 172.21.15.122:0/1323605935 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f8828060130 con 0x7f88380b0d10 2024-09-17T05:32:51.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8814073e60 msgr2=0x7f8814076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8814073e60 0x7f8814076320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f88200046c0 tx=0x7f8820007910 comp rx=0 tx=0).stop 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380b0d10 msgr2=0x7f883815ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380b0d10 0x7f883815ea30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f882800e7d0 tx=0x7f882800ec90 comp rx=0 tx=0).stop 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 shutdown_connections 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8814073e60 0x7f8814076320 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f883815bd40 0x7f883815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88380d22c0 0x7f883815de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 --2- 172.21.15.122:0/1323605935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88380b0d10 0x7f883815ea30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 >> 172.21.15.122:0/1323605935 conn(0x7f883809aa10 msgr2=0x7f8838071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 shutdown_connections 2024-09-17T05:32:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.744+0000 7f8837e00640 1 -- 172.21.15.122:0/1323605935 wait complete. 2024-09-17T05:32:51.765 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:32:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- 172.21.15.122:0/2145762156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 msgr2=0x7fa1280b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 --2- 172.21.15.122:0/2145762156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 0x7fa1280b2830 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa120009bc0 tx=0x7fa12001cb10 comp rx=0 tx=0).stop 2024-09-17T05:32:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- 172.21.15.122:0/2145762156 shutdown_connections 2024-09-17T05:32:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 --2- 172.21.15.122:0/2145762156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 0x7fa1280b2830 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 --2- 172.21.15.122:0/2145762156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12815ba60 0x7fa1281602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- 172.21.15.122:0/2145762156 >> 172.21.15.122:0/2145762156 conn(0x7fa12809aa10 msgr2=0x7fa12809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- 172.21.15.122:0/2145762156 shutdown_connections 2024-09-17T05:32:51.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- 172.21.15.122:0/2145762156 wait complete. 2024-09-17T05:32:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 Processor -- start 2024-09-17T05:32:51.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- start start 2024-09-17T05:32:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1280d0190 0x7fa12815a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 0x7fa1280c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12815ba60 0x7fa128165d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa12815c810 con 0x7fa1280d05f0 2024-09-17T05:32:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa12815c690 con 0x7fa1280d0190 2024-09-17T05:32:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12e600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa12815c990 con 0x7fa12815ba60 2024-09-17T05:32:51.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 0x7fa1280c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 0x7fa1280c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46966/0 (socket says 172.21.15.122:46966) 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa11fe00640 1 -- 172.21.15.122:0/3660723500 learned_addr learned my addr 172.21.15.122:0/3660723500 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12d200640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1280d0190 0x7fa12815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.908+0000 7fa12dc00640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12815ba60 0x7fa128165d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa11fe00640 1 -- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12815ba60 msgr2=0x7fa128165d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa11fe00640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12815ba60 0x7fa128165d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa11fe00640 1 -- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1280d0190 msgr2=0x7fa12815a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa11fe00640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1280d0190 0x7fa12815a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa11fe00640 1 -- 172.21.15.122:0/3660723500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1281fb380 con 0x7fa1280d05f0 2024-09-17T05:32:51.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa11fe00640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 0x7fa1280c68f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c460 tx=0x7fa11000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa125600640 1 -- 172.21.15.122:0/3660723500 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110016020 con 0x7fa1280d05f0 2024-09-17T05:32:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa125600640 1 -- 172.21.15.122:0/3660723500 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa110012920 con 0x7fa1280d05f0 2024-09-17T05:32:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa125600640 1 -- 172.21.15.122:0/3660723500 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110012d50 con 0x7fa1280d05f0 2024-09-17T05:32:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1281fb660 con 0x7fa1280d05f0 2024-09-17T05:32:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12815ab50 con 0x7fa1280d05f0 2024-09-17T05:32:51.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1280cbf20 con 0x7fa1280d05f0 2024-09-17T05:32:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa125600640 1 -- 172.21.15.122:0/3660723500 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fa110019020 con 0x7fa1280d05f0 2024-09-17T05:32:51.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.912+0000 7fa125600640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa104073e60 0x7fa104076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:51.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.916+0000 7fa125600640 1 -- 172.21.15.122:0/3660723500 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fa110092c50 con 0x7fa1280d05f0 2024-09-17T05:32:51.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.916+0000 7fa12d200640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa104073e60 0x7fa104076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:51.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.916+0000 7fa12d200640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa104073e60 0x7fa104076320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa118009b40 tx=0x7fa1180091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:51.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:51.916+0000 7fa125600640 1 -- 172.21.15.122:0/3660723500 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa110020020 con 0x7fa1280d05f0 2024-09-17T05:32:52.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.084+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fa1280d2a60 con 0x7fa1280d05f0 2024-09-17T05:32:52.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa125600640 1 -- 172.21.15.122:0/3660723500 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v37) ==== 116+0+2 (secure 0 0 0) 0x7fa11005eec0 con 0x7fa1280d05f0 2024-09-17T05:32:52.087 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:32:52.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa104073e60 msgr2=0x7fa104076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa104073e60 0x7fa104076320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa118009b40 tx=0x7fa1180091e0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 msgr2=0x7fa1280c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 0x7fa1280c68f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c460 tx=0x7fa11000c920 comp rx=0 tx=0).stop 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 shutdown_connections 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa104073e60 0x7fa104076320 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12815ba60 0x7fa128165d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1280d05f0 0x7fa1280c68f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 --2- 172.21.15.122:0/3660723500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1280d0190 0x7fa12815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 >> 172.21.15.122:0/3660723500 conn(0x7fa12809aa10 msgr2=0x7fa1280cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 shutdown_connections 2024-09-17T05:32:52.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.088+0000 7fa12e600640 1 -- 172.21.15.122:0/3660723500 wait complete. 2024-09-17T05:32:52.108 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:32:52.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 -- 172.21.15.122:0/1009109147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00703d0 msgr2=0x7fddc015a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 --2- 172.21.15.122:0/1009109147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00703d0 0x7fddc015a8b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fddc009e090 tx=0x7fddb801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 -- 172.21.15.122:0/1009109147 shutdown_connections 2024-09-17T05:32:52.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 --2- 172.21.15.122:0/1009109147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00703d0 0x7fddc015a8b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 --2- 172.21.15.122:0/1009109147 >> v2:172.21.15.173:3300/0 conn(0x7fddc00b49b0 0x7fddc015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 -- 172.21.15.122:0/1009109147 >> 172.21.15.122:0/1009109147 conn(0x7fddc00b18f0 msgr2=0x7fddc00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 -- 172.21.15.122:0/1009109147 shutdown_connections 2024-09-17T05:32:52.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 -- 172.21.15.122:0/1009109147 wait complete. 2024-09-17T05:32:52.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 Processor -- start 2024-09-17T05:32:52.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.252+0000 7fddc4a00640 1 -- start start 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc00703d0 0x7fddc0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00b49b0 0x7fddc01466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddc015a4d0 0x7fddc0158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fddc00ccf90 con 0x7fddc00b49b0 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fddc00cce10 con 0x7fddc00703d0 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fddc00cd110 con 0x7fddc015a4d0 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00b49b0 0x7fddc01466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00b49b0 0x7fddc01466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46994/0 (socket says 172.21.15.122:46994) 2024-09-17T05:32:52.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 -- 172.21.15.122:0/3372441910 learned_addr learned my addr 172.21.15.122:0/3372441910 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbde00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc00703d0 0x7fddc0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbe800640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddc015a4d0 0x7fddc0158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 -- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddc015a4d0 msgr2=0x7fddc0158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddc015a4d0 0x7fddc0158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 -- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc00703d0 msgr2=0x7fddc0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc00703d0 0x7fddc0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 -- 172.21.15.122:0/3372441910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc0072f70 con 0x7fddc00b49b0 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddbd400640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00b49b0 0x7fddc01466e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b550 tx=0x7fddac00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddb6000640 1 -- 172.21.15.122:0/3372441910 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac014070 con 0x7fddc00b49b0 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddb6000640 1 -- 172.21.15.122:0/3372441910 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddac010980 con 0x7fddc00b49b0 2024-09-17T05:32:52.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddb6000640 1 -- 172.21.15.122:0/3372441910 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac010dd0 con 0x7fddc00b49b0 2024-09-17T05:32:52.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc0073250 con 0x7fddc00b49b0 2024-09-17T05:32:52.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc009bde0 con 0x7fddc00b49b0 2024-09-17T05:32:52.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.256+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc0069410 con 0x7fddc00b49b0 2024-09-17T05:32:52.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.260+0000 7fddb6000640 1 -- 172.21.15.122:0/3372441910 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7fddac017070 con 0x7fddc00b49b0 2024-09-17T05:32:52.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.260+0000 7fddb6000640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdda0073e60 0x7fdda0076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.260+0000 7fddb6000640 1 -- 172.21.15.122:0/3372441910 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fddac092cc0 con 0x7fddc00b49b0 2024-09-17T05:32:52.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.260+0000 7fddbde00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdda0073e60 0x7fdda0076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.260+0000 7fddbde00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdda0073e60 0x7fdda0076320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fddb000b810 tx=0x7fddb0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.264+0000 7fddb6000640 1 -- 172.21.15.122:0/3372441910 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fddac05f540 con 0x7fddc00b49b0 2024-09-17T05:32:52.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.428+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fddc0167aa0 con 0x7fddc00b49b0 2024-09-17T05:32:52.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.432+0000 7fddb6000640 1 -- 172.21.15.122:0/3372441910 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v38)=0 v38) ==== 131+0+0 (secure 0 0 0) 0x7fddac05ee40 con 0x7fddc00b49b0 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdda0073e60 msgr2=0x7fdda0076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdda0073e60 0x7fdda0076320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fddb000b810 tx=0x7fddb0008040 comp rx=0 tx=0).stop 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00b49b0 msgr2=0x7fddc01466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00b49b0 0x7fddc01466e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b550 tx=0x7fddac00ba10 comp rx=0 tx=0).stop 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 shutdown_connections 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdda0073e60 0x7fdda0076320 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddc015a4d0 0x7fddc0158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fddc00b49b0 0x7fddc01466e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 --2- 172.21.15.122:0/3372441910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddc00703d0 0x7fddc0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 >> 172.21.15.122:0/3372441910 conn(0x7fddc00b18f0 msgr2=0x7fddc0074170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 shutdown_connections 2024-09-17T05:32:52.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.436+0000 7fddc4a00640 1 -- 172.21.15.122:0/3372441910 wait complete. 2024-09-17T05:32:52.454 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:52.456+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:52.454 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:52.456+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:52.454 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:52.456+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:52.454 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:52.456+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:52.454 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:52.456+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:52.456 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-17T05:32:52.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.600+0000 7f5bda400640 1 -- 172.21.15.122:0/2377910280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb8004d50 msgr2=0x7f5bb80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.600+0000 7f5bda400640 1 --2- 172.21.15.122:0/2377910280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bb8004d50 0x7f5bb80051d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009ee0 tx=0x7f5bcc01d630 comp rx=0 tx=0).stop 2024-09-17T05:32:52.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.600+0000 7f5bd3e00640 1 -- 172.21.15.122:0/2377910280 reap_dead start 2024-09-17T05:32:52.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.600+0000 7f5bda400640 1 -- 172.21.15.122:0/2377910280 shutdown_connections 2024-09-17T05:32:52.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.600+0000 7f5bda400640 1 -- 172.21.15.122:0/2377910280 >> 172.21.15.122:0/2377910280 conn(0x7f5bd40b18f0 msgr2=0x7f5bd40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.600+0000 7f5bda400640 1 -- 172.21.15.122:0/2377910280 shutdown_connections 2024-09-17T05:32:52.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.600+0000 7f5bda400640 1 -- 172.21.15.122:0/2377910280 wait complete. 2024-09-17T05:32:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 Processor -- start 2024-09-17T05:32:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 -- start start 2024-09-17T05:32:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8004d50 0x7f5bd414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd41578c0 0x7f5bd41f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd4159d00 0x7f5bd415a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd40b4df0 con 0x7f5bd4159d00 2024-09-17T05:32:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd40b4c70 con 0x7f5bb8004d50 2024-09-17T05:32:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd40cca10 con 0x7f5bd41578c0 2024-09-17T05:32:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd4159d00 0x7f5bd415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd4159d00 0x7f5bd415a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47010/0 (socket says 172.21.15.122:47010) 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 -- 172.21.15.122:0/484126787 learned_addr learned my addr 172.21.15.122:0/484126787 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8004d50 0x7f5bd414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd2a00640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd41578c0 0x7f5bd41f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 -- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd41578c0 msgr2=0x7f5bd41f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd41578c0 0x7f5bd41f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 -- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8004d50 msgr2=0x7f5bd414d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8004d50 0x7f5bd414d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 -- 172.21.15.122:0/484126787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd414dc90 con 0x7f5bd4159d00 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8004d50 0x7f5bd414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bd3e00640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd4159d00 0x7f5bd415a180 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009ee0 tx=0x7f5bcc004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bcb400640 1 -- 172.21.15.122:0/484126787 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0ad050 con 0x7f5bd4159d00 2024-09-17T05:32:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bcb400640 1 -- 172.21.15.122:0/484126787 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bcc0a7ad0 con 0x7f5bd4159d00 2024-09-17T05:32:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd41f84d0 con 0x7f5bd4159d00 2024-09-17T05:32:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bcb400640 1 -- 172.21.15.122:0/484126787 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0a7e80 con 0x7f5bd4159d00 2024-09-17T05:32:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.604+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd415c660 con 0x7f5bd4159d00 2024-09-17T05:32:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.608+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd4069410 con 0x7f5bd4159d00 2024-09-17T05:32:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.608+0000 7f5bcb400640 1 -- 172.21.15.122:0/484126787 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f5bcc0b0070 con 0x7f5bd4159d00 2024-09-17T05:32:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.608+0000 7f5bcb400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bb4073e90 0x7f5bb4076350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.608+0000 7f5bcb400640 1 -- 172.21.15.122:0/484126787 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f5bcc12d9e0 con 0x7f5bd4159d00 2024-09-17T05:32:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.608+0000 7f5bd3400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bb4073e90 0x7f5bb4076350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.608+0000 7f5bd3400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bb4073e90 0x7f5bb4076350 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4001020 tx=0x7f5bc4006730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.612+0000 7f5bcb400640 1 -- 172.21.15.122:0/484126787 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5bcc017340 con 0x7f5bd4159d00 2024-09-17T05:32:52.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.780+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f5bd40b3390 con 0x7f5bd4159d00 2024-09-17T05:32:52.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.784+0000 7f5bcb400640 1 -- 172.21.15.122:0/484126787 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v39)=0 v39) ==== 136+0+0 (secure 0 0 0) 0x7f5bcc0f9ed0 con 0x7f5bd4159d00 2024-09-17T05:32:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bb4073e90 msgr2=0x7f5bb4076350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bb4073e90 0x7f5bb4076350 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4001020 tx=0x7f5bc4006730 comp rx=0 tx=0).stop 2024-09-17T05:32:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd4159d00 msgr2=0x7f5bd415a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd4159d00 0x7f5bd415a180 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009ee0 tx=0x7f5bcc004f50 comp rx=0 tx=0).stop 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 shutdown_connections 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bb4073e90 0x7f5bb4076350 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd4159d00 0x7f5bd415a180 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd41578c0 0x7f5bd41f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 --2- 172.21.15.122:0/484126787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bb8004d50 0x7f5bd414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 >> 172.21.15.122:0/484126787 conn(0x7f5bd40b18f0 msgr2=0x7f5bd40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 shutdown_connections 2024-09-17T05:32:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.788+0000 7f5bda400640 1 -- 172.21.15.122:0/484126787 wait complete. 2024-09-17T05:32:52.807 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T05:32:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- 172.21.15.122:0/317925658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d40aef60 msgr2=0x7f86d40ae660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 --2- 172.21.15.122:0/317925658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d40aef60 0x7f86d40ae660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009900 tx=0x7f86c401c850 comp rx=0 tx=0).stop 2024-09-17T05:32:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- 172.21.15.122:0/317925658 shutdown_connections 2024-09-17T05:32:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 --2- 172.21.15.122:0/317925658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86d415e740 0x7f86d415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 --2- 172.21.15.122:0/317925658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d40aef60 0x7f86d40ae660 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- 172.21.15.122:0/317925658 >> 172.21.15.122:0/317925658 conn(0x7f86d40cfbc0 msgr2=0x7f86d40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- 172.21.15.122:0/317925658 shutdown_connections 2024-09-17T05:32:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- 172.21.15.122:0/317925658 wait complete. 2024-09-17T05:32:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 Processor -- start 2024-09-17T05:32:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- start start 2024-09-17T05:32:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86d40aef60 0x7f86d4149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d415e360 0x7f86d4156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d415e740 0x7f86d4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86d41680b0 con 0x7f86d415e360 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f86d4167f30 con 0x7f86d415e740 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86dc800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f86d4168230 con 0x7f86d40aef60 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d415e360 0x7f86d4156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d415e360 0x7f86d4156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47028/0 (socket says 172.21.15.122:47028) 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9400640 1 -- 172.21.15.122:0/3106786163 learned_addr learned my addr 172.21.15.122:0/3106786163 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9e00640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86d40aef60 0x7f86d4149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86da800640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d415e740 0x7f86d4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9400640 1 -- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86d40aef60 msgr2=0x7f86d4149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9400640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86d40aef60 0x7f86d4149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9400640 1 -- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d415e740 msgr2=0x7f86d4159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9400640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d415e740 0x7f86d4159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86d9400640 1 -- 172.21.15.122:0/3106786163 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d4158280 con 0x7f86d415e360 2024-09-17T05:32:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.952+0000 7f86da800640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d415e740 0x7f86d4159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:32:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86d9400640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d415e360 0x7f86d4156bd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f86c40b0040 tx=0x7f86c4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86ce000640 1 -- 172.21.15.122:0/3106786163 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c40b2040 con 0x7f86d415e360 2024-09-17T05:32:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86ce000640 1 -- 172.21.15.122:0/3106786163 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86c40bd070 con 0x7f86d415e360 2024-09-17T05:32:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d4158560 con 0x7f86d415e360 2024-09-17T05:32:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86ce000640 1 -- 172.21.15.122:0/3106786163 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4003600 con 0x7f86d415e360 2024-09-17T05:32:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d41566b0 con 0x7f86d415e360 2024-09-17T05:32:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d4069410 con 0x7f86d415e360 2024-09-17T05:32:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86ce000640 1 -- 172.21.15.122:0/3106786163 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f86c40037a0 con 0x7f86d415e360 2024-09-17T05:32:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86ce000640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86b8073e60 0x7f86b8076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86ce000640 1 -- 172.21.15.122:0/3106786163 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f86c412e0e0 con 0x7f86d415e360 2024-09-17T05:32:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.956+0000 7f86d9e00640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86b8073e60 0x7f86b8076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.960+0000 7f86d9e00640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86b8073e60 0x7f86b8076320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f86c8000f70 tx=0x7f86c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:52.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:52.960+0000 7f86ce000640 1 -- 172.21.15.122:0/3106786163 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f86c40c7020 con 0x7f86d415e360 2024-09-17T05:32:53.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.152+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f86d40d0850 con 0x7f86b8073e60 2024-09-17T05:32:53.165 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:53.168+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:53.166 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:53.168+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:53.166 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:53.168+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:53.166 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:53.168+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:53.166 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:32:53.168+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:32:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.200+0000 7f86ce000640 1 -- 172.21.15.122:0/3106786163 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86d40d0850 con 0x7f86b8073e60 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86b8073e60 msgr2=0x7f86b8076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86b8073e60 0x7f86b8076320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f86c8000f70 tx=0x7f86c8009040 comp rx=0 tx=0).stop 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d415e360 msgr2=0x7f86d4156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d415e360 0x7f86d4156bd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f86c40b0040 tx=0x7f86c4005e00 comp rx=0 tx=0).stop 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 shutdown_connections 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86b8073e60 0x7f86b8076320 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d415e740 0x7f86d4159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86d415e360 0x7f86d4156bd0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 --2- 172.21.15.122:0/3106786163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86d40aef60 0x7f86d4149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 >> 172.21.15.122:0/3106786163 conn(0x7f86d40cfbc0 msgr2=0x7f86d40cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:53.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 shutdown_connections 2024-09-17T05:32:53.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.204+0000 7f86dc800640 1 -- 172.21.15.122:0/3106786163 wait complete. 2024-09-17T05:32:53.223 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:32:53.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 -- 172.21.15.122:0/3259723531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e5015bbc0 msgr2=0x7f3e5006fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 --2- 172.21.15.122:0/3259723531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e5015bbc0 0x7f3e5006fbd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3e50074840 tx=0x7f3e4001f0b0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 -- 172.21.15.122:0/3259723531 shutdown_connections 2024-09-17T05:32:53.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 --2- 172.21.15.122:0/3259723531 >> v2:172.21.15.173:3300/0 conn(0x7f3e50161c10 0x7f3e501620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 --2- 172.21.15.122:0/3259723531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e500b4540 0x7f3e500b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 --2- 172.21.15.122:0/3259723531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e5015bbc0 0x7f3e5006fbd0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 -- 172.21.15.122:0/3259723531 >> 172.21.15.122:0/3259723531 conn(0x7f3e500b18f0 msgr2=0x7f3e500b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:53.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 -- 172.21.15.122:0/3259723531 shutdown_connections 2024-09-17T05:32:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 -- 172.21.15.122:0/3259723531 wait complete. 2024-09-17T05:32:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 Processor -- start 2024-09-17T05:32:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.368+0000 7f3e56c00640 1 -- start start 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e500b4540 0x7f3e501f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5015bbc0 0x7f3e501f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e50161c10 0x7f3e50160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50072c00 con 0x7f3e500b4540 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50072a80 con 0x7f3e5015bbc0 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3e50072d80 con 0x7f3e50161c10 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e500b4540 0x7f3e501f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e500b4540 0x7f3e501f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47054/0 (socket says 172.21.15.122:47054) 2024-09-17T05:32:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 -- 172.21.15.122:0/1589159036 learned_addr learned my addr 172.21.15.122:0/1589159036 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:32:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e54c00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e50161c10 0x7f3e50160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e47400640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5015bbc0 0x7f3e501f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 -- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e50161c10 msgr2=0x7f3e50160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e50161c10 0x7f3e50160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 -- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5015bbc0 msgr2=0x7f3e501f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5015bbc0 0x7f3e501f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 -- 172.21.15.122:0/1589159036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5009c0d0 con 0x7f3e500b4540 2024-09-17T05:32:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4fe00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e500b4540 0x7f3e501f5620 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3e4001d760 tx=0x7f3e40005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4cc00640 1 -- 172.21.15.122:0/1589159036 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e400be070 con 0x7f3e500b4540 2024-09-17T05:32:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4cc00640 1 -- 172.21.15.122:0/1589159036 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e400053e0 con 0x7f3e500b4540 2024-09-17T05:32:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5009c350 con 0x7f3e500b4540 2024-09-17T05:32:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4cc00640 1 -- 172.21.15.122:0/1589159036 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e400b5040 con 0x7f3e500b4540 2024-09-17T05:32:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5009c910 con 0x7f3e500b4540 2024-09-17T05:32:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e50069410 con 0x7f3e500b4540 2024-09-17T05:32:53.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.372+0000 7f3e4cc00640 1 -- 172.21.15.122:0/1589159036 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 98356+0+0 (secure 0 0 0) 0x7f3e400c1070 con 0x7f3e500b4540 2024-09-17T05:32:53.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.376+0000 7f3e4cc00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e2c073e60 0x7f3e2c076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:32:53.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.376+0000 7f3e4cc00640 1 -- 172.21.15.122:0/1589159036 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f3e4012f930 con 0x7f3e500b4540 2024-09-17T05:32:53.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.376+0000 7f3e47400640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e2c073e60 0x7f3e2c076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:32:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.376+0000 7f3e47400640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e2c073e60 0x7f3e2c076320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3e500aed30 tx=0x7f3e3800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:32:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.380+0000 7f3e4cc00640 1 -- 172.21.15.122:0/1589159036 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3e5009c0d0 con 0x7f3e500b4540 2024-09-17T05:32:53.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.444+0000 7f3e4cc00640 1 -- 172.21.15.122:0/1589159036 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 27) ==== 98407+0+0 (secure 0 0 0) 0x7f3e400f8050 con 0x7f3e500b4540 2024-09-17T05:32:53.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.576+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f3e5006add0 con 0x7f3e2c073e60 2024-09-17T05:32:53.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.580+0000 7f3e4cc00640 1 -- 172.21.15.122:0/1589159036 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f3e5006add0 con 0x7f3e2c073e60 2024-09-17T05:32:53.582 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1 2024-09-17T05:32:53.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e2c073e60 msgr2=0x7f3e2c076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e2c073e60 0x7f3e2c076320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3e500aed30 tx=0x7f3e3800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e500b4540 msgr2=0x7f3e501f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:32:53.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e500b4540 0x7f3e501f5620 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3e4001d760 tx=0x7f3e40005760 comp rx=0 tx=0).stop 2024-09-17T05:32:53.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 shutdown_connections 2024-09-17T05:32:53.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e2c073e60 0x7f3e2c076320 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e50161c10 0x7f3e50160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5015bbc0 0x7f3e501f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 --2- 172.21.15.122:0/1589159036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e500b4540 0x7f3e501f5620 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:32:53.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 >> 172.21.15.122:0/1589159036 conn(0x7f3e500b18f0 msgr2=0x7f3e500cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:32:53.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 shutdown_connections 2024-09-17T05:32:53.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:32:53.584+0000 7f3e56c00640 1 -- 172.21.15.122:0/1589159036 wait complete. 2024-09-17T05:32:53.603 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1 2024-09-17T05:32:53.603 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:32:53.603 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:32:53.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:32:53.612 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:33:02.456 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:33:02.457 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:33:02.457 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.78292 s, 122 MB/s 2024-09-17T05:33:02.458 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:33:02.458 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:02.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:02.467 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:33:02.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:33:02.815 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:33:11.572 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:33:11.572 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:33:11.572 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.05199 s, 119 MB/s 2024-09-17T05:33:11.573 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:33:11.573 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:11.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:11.582 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:33:20.492 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:33:20.492 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:33:20.492 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.85642 s, 121 MB/s 2024-09-17T05:33:20.493 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T05:33:20.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- 172.21.15.122:0/1869850515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 msgr2=0x7fb85c0d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 --2- 172.21.15.122:0/1869850515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 0x7fb85c0d0340 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb844009840 tx=0x7fb84401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- 172.21.15.122:0/1869850515 shutdown_connections 2024-09-17T05:33:20.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 --2- 172.21.15.122:0/1869850515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 0x7fb85c0d0340 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- 172.21.15.122:0/1869850515 >> 172.21.15.122:0/1869850515 conn(0x7fb85c0d1dc0 msgr2=0x7fb85c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:20.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- 172.21.15.122:0/1869850515 shutdown_connections 2024-09-17T05:33:20.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- 172.21.15.122:0/1869850515 wait complete. 2024-09-17T05:33:20.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 Processor -- start 2024-09-17T05:33:20.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- start start 2024-09-17T05:33:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb85c0af850 0x7fb85c0c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 0x7fb85c0b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb85c15f390 0x7fb85c149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c169eb0 con 0x7fb85c0d28e0 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c169d30 con 0x7fb85c15f390 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb861400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb85c16a030 con 0x7fb85c0af850 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb851e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 0x7fb85c0b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb851e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 0x7fb85c0b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36426/0 (socket says 172.21.15.122:36426) 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb851e00640 1 -- 172.21.15.122:0/977892651 learned_addr learned my addr 172.21.15.122:0/977892651 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb85a800640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb85c0af850 0x7fb85c0c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.640+0000 7fb85b200640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb85c15f390 0x7fb85c149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb851e00640 1 -- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb85c0af850 msgr2=0x7fb85c0c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb851e00640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb85c0af850 0x7fb85c0c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb851e00640 1 -- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb85c15f390 msgr2=0x7fb85c149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb851e00640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb85c15f390 0x7fb85c149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb851e00640 1 -- 172.21.15.122:0/977892651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c0c8250 con 0x7fb85c0d28e0 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb851e00640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 0x7fb85c0b5720 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb844009790 tx=0x7fb844004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb853400640 1 -- 172.21.15.122:0/977892651 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440b2040 con 0x7fb85c0d28e0 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb853400640 1 -- 172.21.15.122:0/977892651 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb844005310 con 0x7fb85c0d28e0 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85c148250 con 0x7fb85c0d28e0 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb853400640 1 -- 172.21.15.122:0/977892651 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440044d0 con 0x7fb85c0d28e0 2024-09-17T05:33:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85c148810 con 0x7fb85c0d28e0 2024-09-17T05:33:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85c0afc30 con 0x7fb85c0d28e0 2024-09-17T05:33:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb853400640 1 -- 172.21.15.122:0/977892651 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 28) ==== 98458+0+0 (secure 0 0 0) 0x7fb8440a5830 con 0x7fb85c0d28e0 2024-09-17T05:33:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.644+0000 7fb853400640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb838073f80 0x7fb838076440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.648+0000 7fb853400640 1 -- 172.21.15.122:0/977892651 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fb84412cb70 con 0x7fb85c0d28e0 2024-09-17T05:33:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.648+0000 7fb85a800640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb838073f80 0x7fb838076440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.648+0000 7fb85a800640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb838073f80 0x7fb838076440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0012f0 tx=0x7fb84c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:20.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.652+0000 7fb853400640 1 -- 172.21.15.122:0/977892651 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb8440bc060 con 0x7fb85c0d28e0 2024-09-17T05:33:20.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.848+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fb85c150500 con 0x7fb838073f80 2024-09-17T05:33:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.860+0000 7fb853400640 1 -- 172.21.15.122:0/977892651 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb85c150500 con 0x7fb838073f80 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb838073f80 msgr2=0x7fb838076440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb838073f80 0x7fb838076440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0012f0 tx=0x7fb84c009040 comp rx=0 tx=0).stop 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 msgr2=0x7fb85c0b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 0x7fb85c0b5720 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb844009790 tx=0x7fb844004220 comp rx=0 tx=0).stop 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 shutdown_connections 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb838073f80 0x7fb838076440 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb85c15f390 0x7fb85c149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c0d28e0 0x7fb85c0b5720 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 --2- 172.21.15.122:0/977892651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb85c0af850 0x7fb85c0c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 >> 172.21.15.122:0/977892651 conn(0x7fb85c0d1dc0 msgr2=0x7fb85c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 shutdown_connections 2024-09-17T05:33:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:20.864+0000 7fb861400640 1 -- 172.21.15.122:0/977892651 wait complete. 2024-09-17T05:33:20.882 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:33:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- 172.21.15.122:0/4094796601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f580b1650 msgr2=0x7f5f580b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 --2- 172.21.15.122:0/4094796601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f580b1650 0x7f5f580b1a30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009900 tx=0x7f5f4401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- 172.21.15.122:0/4094796601 shutdown_connections 2024-09-17T05:33:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 --2- 172.21.15.122:0/4094796601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f580b1650 0x7f5f580b1a30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 --2- 172.21.15.122:0/4094796601 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f581605d0 0x7f5f580d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- 172.21.15.122:0/4094796601 >> 172.21.15.122:0/4094796601 conn(0x7f5f580d1dd0 msgr2=0x7f5f580d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- 172.21.15.122:0/4094796601 shutdown_connections 2024-09-17T05:33:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- 172.21.15.122:0/4094796601 wait complete. 2024-09-17T05:33:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 Processor -- start 2024-09-17T05:33:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- start start 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f580aeb00 0x7f5f5814ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f580b1650 0x7f5f58165f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f581605d0 0x7f5f5814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f580b0810 con 0x7f5f580b1650 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5f580b0ee0 con 0x7f5f581605d0 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f5d600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5f580b1220 con 0x7f5f580aeb00 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f580b1650 0x7f5f58165f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f580b1650 0x7f5f58165f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36438/0 (socket says 172.21.15.122:36438) 2024-09-17T05:33:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56000640 1 -- 172.21.15.122:0/40954557 learned_addr learned my addr 172.21.15.122:0/40954557 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f57400640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f581605d0 0x7f5f5814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56a00640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f580aeb00 0x7f5f5814ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56000640 1 -- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f580aeb00 msgr2=0x7f5f5814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56000640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f580aeb00 0x7f5f5814ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56000640 1 -- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f581605d0 msgr2=0x7f5f5814f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56000640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f581605d0 0x7f5f5814f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.028+0000 7f5f56000640 1 -- 172.21.15.122:0/40954557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5814fd50 con 0x7f5f580b1650 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f56000640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f580b1650 0x7f5f58165f90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5f4401a510 tx=0x7f5f440a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f4ea00640 1 -- 172.21.15.122:0/40954557 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f440b2040 con 0x7f5f580b1650 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f4ea00640 1 -- 172.21.15.122:0/40954557 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f44004470 con 0x7f5f580b1650 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f4ea00640 1 -- 172.21.15.122:0/40954557 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44005290 con 0x7f5f580b1650 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f58154630 con 0x7f5f580b1650 2024-09-17T05:33:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f58155a90 con 0x7f5f580b1650 2024-09-17T05:33:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f58155c60 con 0x7f5f580b1650 2024-09-17T05:33:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f4ea00640 1 -- 172.21.15.122:0/40954557 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 28) ==== 98458+0+0 (secure 0 0 0) 0x7f5f44005430 con 0x7f5f580b1650 2024-09-17T05:33:21.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.032+0000 7f5f4ea00640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f38073fb0 0x7f5f38076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:21.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.036+0000 7f5f4ea00640 1 -- 172.21.15.122:0/40954557 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f5f4412c1d0 con 0x7f5f580b1650 2024-09-17T05:33:21.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.036+0000 7f5f56a00640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f38073fb0 0x7f5f38076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:21.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.036+0000 7f5f56a00640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f38073fb0 0x7f5f38076470 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5f580ce3e0 tx=0x7f5f480091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:21.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.036+0000 7f5f4ea00640 1 -- 172.21.15.122:0/40954557 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5f58154630 con 0x7f5f580b1650 2024-09-17T05:33:21.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.232+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f5f580b1a90 con 0x7f5f38073fb0 2024-09-17T05:33:21.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f4ea00640 1 -- 172.21.15.122:0/40954557 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f5f580b1a90 con 0x7f5f38073fb0 2024-09-17T05:33:21.239 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1 2024-09-17T05:33:21.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f38073fb0 msgr2=0x7f5f38076470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f38073fb0 0x7f5f38076470 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5f580ce3e0 tx=0x7f5f480091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f580b1650 msgr2=0x7f5f58165f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f580b1650 0x7f5f58165f90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5f4401a510 tx=0x7f5f440a55a0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 shutdown_connections 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f38073fb0 0x7f5f38076470 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f581605d0 0x7f5f5814f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f580b1650 0x7f5f58165f90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 --2- 172.21.15.122:0/40954557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f580aeb00 0x7f5f5814ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 >> 172.21.15.122:0/40954557 conn(0x7f5f580d1dd0 msgr2=0x7f5f58156970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 shutdown_connections 2024-09-17T05:33:21.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:21.240+0000 7f5f5d600640 1 -- 172.21.15.122:0/40954557 wait complete. 2024-09-17T05:33:21.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:21.261 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1 2024-09-17T05:33:21.261 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:33:21.270 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1 2024-09-17T05:33:21.270 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T05:33:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:21.271 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T05:33:23.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:23.272 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1 2024-09-17T05:33:23.272 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:33:23.283 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/545efc83-ff2b-4cc5-b0a7-6e578e855be1 2024-09-17T05:33:23.283 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T05:33:23.283 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:23.284 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T05:33:23.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone67490 2024-09-17T05:33:23.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.480+0000 7f1e7fe00640 1 -- 172.21.15.122:0/236171259 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e7815f340 msgr2=0x7f1e780b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.480+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/236171259 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e7815f340 0x7f1e780b34a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009840 tx=0x7f1e6c01c820 comp rx=0 tx=0).stop 2024-09-17T05:33:23.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 -- 172.21.15.122:0/236171259 shutdown_connections 2024-09-17T05:33:23.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/236171259 >> v2:172.21.15.173:3300/0 conn(0x7f1e780b39e0 0x7f1e780b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/236171259 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e7815f340 0x7f1e780b34a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 -- 172.21.15.122:0/236171259 >> 172.21.15.122:0/236171259 conn(0x7f1e780d1e00 msgr2=0x7f1e780d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:23.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 -- 172.21.15.122:0/236171259 shutdown_connections 2024-09-17T05:33:23.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 -- 172.21.15.122:0/236171259 wait complete. 2024-09-17T05:33:23.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 Processor -- start 2024-09-17T05:33:23.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 -- start start 2024-09-17T05:33:23.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e780b2a00 0x7f1e7814ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e780b39e0 0x7f1e78156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e7815f340 0x7f1e7814d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78168cd0 con 0x7f1e7815f340 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78168b50 con 0x7f1e780b2a00 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7fe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1e78168e50 con 0x7f1e780b39e0 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e7815f340 0x7f1e7814d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e7815f340 0x7f1e7814d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57414/0 (socket says 172.21.15.122:57414) 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 -- 172.21.15.122:0/3733810494 learned_addr learned my addr 172.21.15.122:0/3733810494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7d400640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e780b2a00 0x7f1e7814ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7ca00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e780b39e0 0x7f1e78156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 -- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e780b39e0 msgr2=0x7f1e78156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e780b39e0 0x7f1e78156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 -- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e780b2a00 msgr2=0x7f1e7814ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e780b2a00 0x7f1e7814ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 -- 172.21.15.122:0/3733810494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7814bfd0 con 0x7f1e7815f340 2024-09-17T05:33:23.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.484+0000 7f1e7de00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e7815f340 0x7f1e7814d950 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000eb20 tx=0x7f1e7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:23.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e75600640 1 -- 172.21.15.122:0/3733810494 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70004b30 con 0x7f1e7815f340 2024-09-17T05:33:23.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e75600640 1 -- 172.21.15.122:0/3733810494 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e70004cf0 con 0x7f1e7815f340 2024-09-17T05:33:23.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e75600640 1 -- 172.21.15.122:0/3733810494 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70005280 con 0x7f1e7815f340 2024-09-17T05:33:23.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e78156ab0 con 0x7f1e7815f340 2024-09-17T05:33:23.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e78157070 con 0x7f1e7815f340 2024-09-17T05:33:23.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e780b34a0 con 0x7f1e7815f340 2024-09-17T05:33:23.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e75600640 1 -- 172.21.15.122:0/3733810494 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 28) ==== 98458+0+0 (secure 0 0 0) 0x7f1e70005420 con 0x7f1e7815f340 2024-09-17T05:33:23.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e75600640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1e58073f80 0x7f1e58076440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e75600640 1 -- 172.21.15.122:0/3733810494 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f1e70093bc0 con 0x7f1e7815f340 2024-09-17T05:33:23.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.488+0000 7f1e7d400640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1e58073f80 0x7f1e58076440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.492+0000 7f1e7d400640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1e58073f80 0x7f1e58076440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0af000 tx=0x7f1e6c01b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:23.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.492+0000 7f1e75600640 1 -- 172.21.15.122:0/3733810494 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1e7005fd40 con 0x7f1e7815f340 2024-09-17T05:33:23.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.692+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone67490", "target": ["mon-mgr", ""]}) -- 0x7f1e780afe00 con 0x7f1e58073f80 2024-09-17T05:33:23.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e75600640 1 -- 172.21.15.122:0/3733810494 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1e780afe00 con 0x7f1e58073f80 2024-09-17T05:33:23.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1e58073f80 msgr2=0x7f1e58076440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1e58073f80 0x7f1e58076440 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0af000 tx=0x7f1e6c01b060 comp rx=0 tx=0).stop 2024-09-17T05:33:23.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e7815f340 msgr2=0x7f1e7814d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e7815f340 0x7f1e7814d950 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000eb20 tx=0x7f1e7000c2d0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 shutdown_connections 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1e58073f80 0x7f1e58076440 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e7815f340 0x7f1e7814d950 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e780b39e0 0x7f1e78156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 --2- 172.21.15.122:0/3733810494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e780b2a00 0x7f1e7814ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 >> 172.21.15.122:0/3733810494 conn(0x7f1e780d1e00 msgr2=0x7f1e780ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 shutdown_connections 2024-09-17T05:33:23.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.764+0000 7f1e7fe00640 1 -- 172.21.15.122:0/3733810494 wait complete. 2024-09-17T05:33:23.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.768+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.768+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.768+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.768+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.768+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.784 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.788+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.785 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.788+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.785 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.788+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.785 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.788+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.785 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:23.788+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:23.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone64781 2024-09-17T05:33:23.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- 172.21.15.122:0/2384404986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa406e7a0 msgr2=0x7f7aa40ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 --2- 172.21.15.122:0/2384404986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa406e7a0 0x7f7aa40ce2e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009c80 tx=0x7f7a9c01cd70 comp rx=0 tx=0).stop 2024-09-17T05:33:23.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- 172.21.15.122:0/2384404986 shutdown_connections 2024-09-17T05:33:23.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 --2- 172.21.15.122:0/2384404986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa406e7a0 0x7f7aa40ce2e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 --2- 172.21.15.122:0/2384404986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7aa40cf1d0 0x7f7aa4160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- 172.21.15.122:0/2384404986 >> 172.21.15.122:0/2384404986 conn(0x7f7aa40b18f0 msgr2=0x7f7aa40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:23.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- 172.21.15.122:0/2384404986 shutdown_connections 2024-09-17T05:33:23.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- 172.21.15.122:0/2384404986 wait complete. 2024-09-17T05:33:23.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 Processor -- start 2024-09-17T05:33:23.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- start start 2024-09-17T05:33:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7aa406e7a0 0x7f7aa4154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa40cf1d0 0x7f7aa4165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa415f9a0 0x7f7aa41656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4169200 con 0x7f7aa415f9a0 2024-09-17T05:33:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4169080 con 0x7f7aa40cf1d0 2024-09-17T05:33:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aaa400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa4169380 con 0x7f7aa406e7a0 2024-09-17T05:33:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aa3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa415f9a0 0x7f7aa41656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aa3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa415f9a0 0x7f7aa41656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57430/0 (socket says 172.21.15.122:57430) 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.940+0000 7f7aa3e00640 1 -- 172.21.15.122:0/4156131734 learned_addr learned my addr 172.21.15.122:0/4156131734 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7a9aa00640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa40cf1d0 0x7f7aa4165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aa3400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7aa406e7a0 0x7f7aa4154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aa3e00640 1 -- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7aa406e7a0 msgr2=0x7f7aa4154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aa3e00640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7aa406e7a0 0x7f7aa4154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aa3e00640 1 -- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa40cf1d0 msgr2=0x7f7aa4165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aa3e00640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa40cf1d0 0x7f7aa4165160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aa3e00640 1 -- 172.21.15.122:0/4156131734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa40b31f0 con 0x7f7aa415f9a0 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7a9aa00640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa40cf1d0 0x7f7aa4165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aa3e00640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa415f9a0 0x7f7aa41656a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009bd0 tx=0x7f7a9c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7a9be00640 1 -- 172.21.15.122:0/4156131734 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0bb070 con 0x7f7aa415f9a0 2024-09-17T05:33:23.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7a9be00640 1 -- 172.21.15.122:0/4156131734 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a9c004d40 con 0x7f7aa415f9a0 2024-09-17T05:33:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7a9be00640 1 -- 172.21.15.122:0/4156131734 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0b2040 con 0x7f7aa415f9a0 2024-09-17T05:33:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa40b34d0 con 0x7f7aa415f9a0 2024-09-17T05:33:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa40b3a90 con 0x7f7aa415f9a0 2024-09-17T05:33:23.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa40ce2e0 con 0x7f7aa415f9a0 2024-09-17T05:33:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.944+0000 7f7a9be00640 1 -- 172.21.15.122:0/4156131734 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 28) ==== 98458+0+0 (secure 0 0 0) 0x7f7a9c0ad850 con 0x7f7aa415f9a0 2024-09-17T05:33:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.948+0000 7f7a9be00640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7a80073f80 0x7f7a80076440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.948+0000 7f7a9be00640 1 -- 172.21.15.122:0/4156131734 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f7a9c12cb80 con 0x7f7aa415f9a0 2024-09-17T05:33:23.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.948+0000 7f7aa3400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7a80073f80 0x7f7a80076440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:23.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.948+0000 7f7aa3400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7a80073f80 0x7f7a80076440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4165d20 tx=0x7f7a94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:23.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:23.952+0000 7f7a9be00640 1 -- 172.21.15.122:0/4156131734 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7a9c016990 con 0x7f7aa415f9a0 2024-09-17T05:33:24.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.156+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone64781", "target": ["mon-mgr", ""]}) -- 0x7f7aa4165e30 con 0x7f7a80073f80 2024-09-17T05:33:24.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:24.768+0000 7fc08ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:24.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:24.768+0000 7fc08ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:24.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:24.768+0000 7fc08ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:24.768 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:24.768+0000 7fc08ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:24.769 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:24.768+0000 7fc08ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:24.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.848+0000 7f7a9be00640 1 -- 172.21.15.122:0/4156131734 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7aa4165e30 con 0x7f7a80073f80 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7a80073f80 msgr2=0x7f7a80076440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7a80073f80 0x7f7a80076440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4165d20 tx=0x7f7a94009040 comp rx=0 tx=0).stop 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa415f9a0 msgr2=0x7f7aa41656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa415f9a0 0x7f7aa41656a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009bd0 tx=0x7f7a9c0092c0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 shutdown_connections 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7a80073f80 0x7f7a80076440 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa415f9a0 0x7f7aa41656a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7aa40cf1d0 0x7f7aa4165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 --2- 172.21.15.122:0/4156131734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7aa406e7a0 0x7f7aa4154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 >> 172.21.15.122:0/4156131734 conn(0x7f7aa40b18f0 msgr2=0x7f7aa415a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 shutdown_connections 2024-09-17T05:33:24.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:24.852+0000 7f7aaa400640 1 -- 172.21.15.122:0/4156131734 wait complete. 2024-09-17T05:33:24.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone77222 2024-09-17T05:33:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 -- 172.21.15.122:0/3550190450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad7c0cee00 msgr2=0x7fad7c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 --2- 172.21.15.122:0/3550190450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad7c0cee00 0x7fad7c15e8a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0b3890 tx=0x7fad7401c830 comp rx=0 tx=0).stop 2024-09-17T05:33:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 -- 172.21.15.122:0/3550190450 shutdown_connections 2024-09-17T05:33:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 --2- 172.21.15.122:0/3550190450 >> v2:172.21.15.173:3300/0 conn(0x7fad7c073d00 0x7fad7c0ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 --2- 172.21.15.122:0/3550190450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad7c0cee00 0x7fad7c15e8a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 --2- 172.21.15.122:0/3550190450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad7c155450 0x7fad7c0737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 -- 172.21.15.122:0/3550190450 >> 172.21.15.122:0/3550190450 conn(0x7fad7c0d0cf0 msgr2=0x7fad7c0d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 -- 172.21.15.122:0/3550190450 shutdown_connections 2024-09-17T05:33:25.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.112+0000 7fad82800640 1 -- 172.21.15.122:0/3550190450 wait complete. 2024-09-17T05:33:25.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad82800640 1 Processor -- start 2024-09-17T05:33:25.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad82800640 1 -- start start 2024-09-17T05:33:25.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad82800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad7c073d00 0x7fad7c1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:25.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad82800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad7c0cee00 0x7fad7c15a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:25.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad82800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad7c155450 0x7fad7c1f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:25.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad82800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c06d320 con 0x7fad7c155450 2024-09-17T05:33:25.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad82800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c06d1a0 con 0x7fad7c073d00 2024-09-17T05:33:25.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad82800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fad7c06d4a0 con 0x7fad7c0cee00 2024-09-17T05:33:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad7b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad7c073d00 0x7fad7c1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad7b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad7c073d00 0x7fad7c1580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:45032/0 (socket says 172.21.15.122:45032) 2024-09-17T05:33:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad7b400640 1 -- 172.21.15.122:0/627812703 learned_addr learned my addr 172.21.15.122:0/627812703 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad7b400640 1 -- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad7c0cee00 msgr2=0x7fad7c15a020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:33:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad7b400640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad7c0cee00 0x7fad7c15a020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad7b400640 1 -- 172.21.15.122:0/627812703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad7c155450 msgr2=0x7fad7c1f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad7b400640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad7c155450 0x7fad7c1f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.116+0000 7fad7b400640 1 -- 172.21.15.122:0/627812703 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c0b2240 con 0x7fad7c073d00 2024-09-17T05:33:25.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.120+0000 7fad7b400640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad7c073d00 0x7fad7c1580e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0058d0 tx=0x7fad6c013420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:25.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.120+0000 7fad73400640 1 -- 172.21.15.122:0/627812703 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c022070 con 0x7fad7c073d00 2024-09-17T05:33:25.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.120+0000 7fad73400640 1 -- 172.21.15.122:0/627812703 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad6c01d400 con 0x7fad7c073d00 2024-09-17T05:33:25.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.120+0000 7fad73400640 1 -- 172.21.15.122:0/627812703 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c01d7f0 con 0x7fad7c073d00 2024-09-17T05:33:25.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.120+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c0b24e0 con 0x7fad7c073d00 2024-09-17T05:33:25.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.120+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c1f7f10 con 0x7fad7c073d00 2024-09-17T05:33:25.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.120+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7c06d950 con 0x7fad7c073d00 2024-09-17T05:33:25.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.124+0000 7fad73400640 1 -- 172.21.15.122:0/627812703 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 28) ==== 98458+0+0 (secure 0 0 0) 0x7fad6c025070 con 0x7fad7c073d00 2024-09-17T05:33:25.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.128+0000 7fad73400640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad5c073f80 0x7fad5c076440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:25.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.128+0000 7fad73400640 1 -- 172.21.15.122:0/627812703 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fad6c09b220 con 0x7fad7c073d00 2024-09-17T05:33:25.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.128+0000 7fad7aa00640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad5c073f80 0x7fad5c076440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:25.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.132+0000 7fad73400640 1 -- 172.21.15.122:0/627812703 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fad6c018070 con 0x7fad7c073d00 2024-09-17T05:33:25.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.132+0000 7fad7aa00640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad5c073f80 0x7fad5c076440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad7c158620 tx=0x7fad6800b2f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:25.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.148+0000 7fad73400640 1 -- 172.21.15.122:0/627812703 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mgrmap(e 29) ==== 98560+0+0 (secure 0 0 0) 0x7fad6c0635e0 con 0x7fad7c073d00 2024-09-17T05:33:25.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.456+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone77222", "target": ["mon-mgr", ""]}) -- 0x7fad7c0cb0b0 con 0x7fad5c073f80 2024-09-17T05:33:25.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:25.996+0000 7fad73400640 1 -- 172.21.15.122:0/627812703 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad7c0cb0b0 con 0x7fad5c073f80 2024-09-17T05:33:26.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.000+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad5c073f80 msgr2=0x7fad5c076440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:26.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.000+0000 7fad82800640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad5c073f80 0x7fad5c076440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad7c158620 tx=0x7fad6800b2f0 comp rx=0 tx=0).stop 2024-09-17T05:33:26.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.000+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad7c073d00 msgr2=0x7fad7c1580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:26.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.000+0000 7fad82800640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad7c073d00 0x7fad7c1580e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0058d0 tx=0x7fad6c013420 comp rx=0 tx=0).stop 2024-09-17T05:33:26.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.004+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 shutdown_connections 2024-09-17T05:33:26.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.004+0000 7fad82800640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fad5c073f80 0x7fad5c076440 secure :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fad7c158620 tx=0x7fad6800b2f0 comp rx=0 tx=0).stop 2024-09-17T05:33:26.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.004+0000 7fad82800640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad7c155450 0x7fad7c1f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:26.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.004+0000 7fad82800640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad7c0cee00 0x7fad7c15a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:26.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.004+0000 7fad82800640 1 --2- 172.21.15.122:0/627812703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad7c073d00 0x7fad7c1580e0 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0058d0 tx=0x7fad6c013420 comp rx=0 tx=0).stop 2024-09-17T05:33:26.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.004+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 >> 172.21.15.122:0/627812703 conn(0x7fad7c0d0cf0 msgr2=0x7fad7c070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:26.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.004+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 shutdown_connections 2024-09-17T05:33:26.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.004+0000 7fad82800640 1 -- 172.21.15.122:0/627812703 wait complete. 2024-09-17T05:33:26.014 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.016+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.015 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.016+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.015 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.016+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.015 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.016+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.015 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.016+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.035 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone36743 2024-09-17T05:33:26.064 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.064+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.064 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.064+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.064 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.064+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.064 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.064+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.064 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:26.064+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:26.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.276+0000 7fd32b400640 1 -- 172.21.15.122:0/1891426392 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324009be0 con 0x7fd314004e70 2024-09-17T05:33:26.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.276+0000 7fd333400640 1 -- 172.21.15.122:0/1891426392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd314004e70 msgr2=0x7fd31400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:26.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.276+0000 7fd333400640 1 --2- 172.21.15.122:0/1891426392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd314004e70 0x7fd31400b9d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd324009f90 tx=0x7fd32401c990 comp rx=0 tx=0).stop 2024-09-17T05:33:26.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.276+0000 7fd331400640 1 -- 172.21.15.122:0/1891426392 reap_dead start 2024-09-17T05:33:26.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.276+0000 7fd333400640 1 -- 172.21.15.122:0/1891426392 shutdown_connections 2024-09-17T05:33:26.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.276+0000 7fd333400640 1 -- 172.21.15.122:0/1891426392 >> 172.21.15.122:0/1891426392 conn(0x7fd32c0d0c70 msgr2=0x7fd32c0d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:26.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.276+0000 7fd333400640 1 -- 172.21.15.122:0/1891426392 shutdown_connections 2024-09-17T05:33:26.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.276+0000 7fd333400640 1 -- 172.21.15.122:0/1891426392 wait complete. 2024-09-17T05:33:26.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.280+0000 7fd333400640 1 Processor -- start 2024-09-17T05:33:26.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.280+0000 7fd333400640 1 -- start start 2024-09-17T05:33:26.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.280+0000 7fd333400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd32c155b60 0x7fd32c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:26.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.280+0000 7fd333400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd32c0b42c0 0x7fd32c0c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:26.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.280+0000 7fd333400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd32c0c63a0 0x7fd32c160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:26.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.280+0000 7fd333400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c164bd0 con 0x7fd32c155b60 2024-09-17T05:33:26.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.280+0000 7fd333400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c164a50 con 0x7fd32c0b42c0 2024-09-17T05:33:26.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.280+0000 7fd333400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c164d50 con 0x7fd32c0c63a0 2024-09-17T05:33:26.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd32c155b60 0x7fd32c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:26.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd32c155b60 0x7fd32c15a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57456/0 (socket says 172.21.15.122:57456) 2024-09-17T05:33:26.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 -- 172.21.15.122:0/2150209636 learned_addr learned my addr 172.21.15.122:0/2150209636 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:26.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd331400640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd32c0c63a0 0x7fd32c160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:26.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 -- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd32c0c63a0 msgr2=0x7fd32c160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:26.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd32c0c63a0 0x7fd32c160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:26.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 -- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd32c0b42c0 msgr2=0x7fd32c0c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:26.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd32c0b42c0 0x7fd32c0c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:26.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 -- 172.21.15.122:0/2150209636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c15b440 con 0x7fd32c155b60 2024-09-17T05:33:26.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd330a00640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd32c155b60 0x7fd32c15a090 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd32000b4b0 tx=0x7fd32000b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:26.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd328c00640 1 -- 172.21.15.122:0/2150209636 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320014070 con 0x7fd32c155b60 2024-09-17T05:33:26.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd328c00640 1 -- 172.21.15.122:0/2150209636 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd320010610 con 0x7fd32c155b60 2024-09-17T05:33:26.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd328c00640 1 -- 172.21.15.122:0/2150209636 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320010a60 con 0x7fd32c155b60 2024-09-17T05:33:26.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c1592d0 con 0x7fd32c155b60 2024-09-17T05:33:26.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.284+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c159890 con 0x7fd32c155b60 2024-09-17T05:33:26.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.288+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c070e30 con 0x7fd32c155b60 2024-09-17T05:33:26.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.288+0000 7fd328c00640 1 -- 172.21.15.122:0/2150209636 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 29) ==== 98560+0+0 (secure 0 0 0) 0x7fd320017070 con 0x7fd32c155b60 2024-09-17T05:33:26.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.292+0000 7fd328c00640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd30c074120 0x7fd30c0765e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:26.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.292+0000 7fd328c00640 1 -- 172.21.15.122:0/2150209636 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fd320092a00 con 0x7fd32c155b60 2024-09-17T05:33:26.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.292+0000 7fd32be00640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd30c074120 0x7fd30c0765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:26.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.296+0000 7fd328c00640 1 -- 172.21.15.122:0/2150209636 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd32005f220 con 0x7fd32c155b60 2024-09-17T05:33:26.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.308+0000 7fd32be00640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd30c074120 0x7fd30c0765e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0d1390 tx=0x7fd318000fa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:26.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:26.696+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone36743", "target": ["mon-mgr", ""]}) -- 0x7fd32c0b6440 con 0x7fd30c074120 2024-09-17T05:33:27.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.248+0000 7fd328c00640 1 -- 172.21.15.122:0/2150209636 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 30) ==== 98713+0+0 (secure 0 0 0) 0x7fd32005ad90 con 0x7fd32c155b60 2024-09-17T05:33:27.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.596+0000 7fd328c00640 1 -- 172.21.15.122:0/2150209636 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd32c0b6440 con 0x7fd30c074120 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd30c074120 msgr2=0x7fd30c0765e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd30c074120 0x7fd30c0765e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0d1390 tx=0x7fd318000fa0 comp rx=0 tx=0).stop 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd32c155b60 msgr2=0x7fd32c15a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd32c155b60 0x7fd32c15a090 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd32000b4b0 tx=0x7fd32000b970 comp rx=0 tx=0).stop 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 shutdown_connections 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd30c074120 0x7fd30c0765e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd32c0c63a0 0x7fd32c160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd32c0b42c0 0x7fd32c0c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:27.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 --2- 172.21.15.122:0/2150209636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd32c155b60 0x7fd32c15a090 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 >> 172.21.15.122:0/2150209636 conn(0x7fd32c0d0c70 msgr2=0x7fd32c15ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 shutdown_connections 2024-09-17T05:33:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.600+0000 7fd333400640 1 -- 172.21.15.122:0/2150209636 wait complete. 2024-09-17T05:33:27.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone66604 2024-09-17T05:33:27.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.908+0000 7f7fee400640 1 -- 172.21.15.122:0/1006105528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fcc004d50 msgr2=0x7f7fcc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:27.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.908+0000 7f7fee400640 1 --2- 172.21.15.122:0/1006105528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fcc004d50 0x7f7fcc0051d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c610 tx=0x7f7fe001d2e0 comp rx=0 tx=0).stop 2024-09-17T05:33:27.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.912+0000 7f7fe7e00640 1 -- 172.21.15.122:0/1006105528 reap_dead start 2024-09-17T05:33:27.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.912+0000 7f7fee400640 1 -- 172.21.15.122:0/1006105528 shutdown_connections 2024-09-17T05:33:27.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.912+0000 7f7fee400640 1 -- 172.21.15.122:0/1006105528 >> 172.21.15.122:0/1006105528 conn(0x7f7fe80cd740 msgr2=0x7f7fe80cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:27.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.912+0000 7f7fee400640 1 -- 172.21.15.122:0/1006105528 shutdown_connections 2024-09-17T05:33:27.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.912+0000 7f7fee400640 1 -- 172.21.15.122:0/1006105528 wait complete. 2024-09-17T05:33:27.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.916+0000 7f7fee400640 1 Processor -- start 2024-09-17T05:33:27.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.916+0000 7f7fee400640 1 -- start start 2024-09-17T05:33:27.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fee400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fe81eea50 0x7f7fe80b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:27.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fee400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe815da40 0x7f7fe80a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:27.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fee400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fe80a0f20 0x7f7fe809ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:27.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fee400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe80cf420 con 0x7f7fe81eea50 2024-09-17T05:33:27.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fee400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe80cf2a0 con 0x7f7fe815da40 2024-09-17T05:33:27.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fee400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe80cf5a0 con 0x7f7fe80a0f20 2024-09-17T05:33:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe815da40 0x7f7fe80a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fe81eea50 0x7f7fe80b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe815da40 0x7f7fe80a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:45072/0 (socket says 172.21.15.122:45072) 2024-09-17T05:33:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe6a00640 1 -- 172.21.15.122:0/3964676001 learned_addr learned my addr 172.21.15.122:0/3964676001 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:27.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe7e00640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fe80a0f20 0x7f7fe809ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe7400640 1 -- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fe80a0f20 msgr2=0x7f7fe809ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe7400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fe80a0f20 0x7f7fe809ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe7400640 1 -- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe815da40 msgr2=0x7f7fe80a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe7400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe815da40 0x7f7fe80a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:27.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe7400640 1 -- 172.21.15.122:0/3964676001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe81f5570 con 0x7f7fe81eea50 2024-09-17T05:33:27.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fe7400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fe81eea50 0x7f7fe80b4b40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800d6b0 tx=0x7f7fd800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:27.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fdf400640 1 -- 172.21.15.122:0/3964676001 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8017070 con 0x7f7fe81eea50 2024-09-17T05:33:27.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.920+0000 7f7fdf400640 1 -- 172.21.15.122:0/3964676001 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fd8009ab0 con 0x7f7fe81eea50 2024-09-17T05:33:27.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.924+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe81f5810 con 0x7f7fe81eea50 2024-09-17T05:33:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.924+0000 7f7fdf400640 1 -- 172.21.15.122:0/3964676001 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8010460 con 0x7f7fe81eea50 2024-09-17T05:33:27.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.924+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe80b5080 con 0x7f7fe81eea50 2024-09-17T05:33:27.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.928+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe80b2740 con 0x7f7fe81eea50 2024-09-17T05:33:27.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.928+0000 7f7fdf400640 1 -- 172.21.15.122:0/3964676001 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 30) ==== 98713+0+0 (secure 0 0 0) 0x7f7fd801a070 con 0x7f7fe81eea50 2024-09-17T05:33:27.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.928+0000 7f7fdf400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7fc80742d0 0x7f7fc8076790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:27.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.928+0000 7f7fdf400640 1 -- 172.21.15.122:0/3964676001 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f7fd8093070 con 0x7f7fe81eea50 2024-09-17T05:33:27.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.936+0000 7f7fe6a00640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7fc80742d0 0x7f7fc8076790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:27.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.936+0000 7f7fdf400640 1 -- 172.21.15.122:0/3964676001 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7fd805f1a0 con 0x7f7fe81eea50 2024-09-17T05:33:27.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:27.948+0000 7f7fe6a00640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7fc80742d0 0x7f7fc8076790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40058d0 tx=0x7f7fd400b3c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.248+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone66604", "target": ["mon-mgr", ""]}) -- 0x7f7fe81f59e0 con 0x7f7fc80742d0 2024-09-17T05:33:28.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fdf400640 1 -- 172.21.15.122:0/3964676001 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7fe81f59e0 con 0x7f7fc80742d0 2024-09-17T05:33:28.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7fc80742d0 msgr2=0x7f7fc8076790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:28.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7fc80742d0 0x7f7fc8076790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40058d0 tx=0x7f7fd400b3c0 comp rx=0 tx=0).stop 2024-09-17T05:33:28.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fe81eea50 msgr2=0x7f7fe80b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:28.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fe81eea50 0x7f7fe80b4b40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800d6b0 tx=0x7f7fd800db70 comp rx=0 tx=0).stop 2024-09-17T05:33:28.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 shutdown_connections 2024-09-17T05:33:28.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7fc80742d0 0x7f7fc8076790 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:28.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fe80a0f20 0x7f7fe809ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:28.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fe815da40 0x7f7fe80a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:28.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 --2- 172.21.15.122:0/3964676001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fe81eea50 0x7f7fe80b4b40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:28.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 >> 172.21.15.122:0/3964676001 conn(0x7f7fe80cd740 msgr2=0x7f7fe806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:28.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 shutdown_connections 2024-09-17T05:33:28.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:28.904+0000 7f7fee400640 1 -- 172.21.15.122:0/3964676001 wait complete. 2024-09-17T05:33:28.952 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone34415 2024-09-17T05:33:29.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.212+0000 7f0921000640 1 -- 172.21.15.122:0/2508970534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 msgr2=0x7f091c06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:29.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.212+0000 7f0921000640 1 --2- 172.21.15.122:0/2508970534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 0x7f091c06f8f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f091c0d06f0 tx=0x7f091401cd70 comp rx=0 tx=0).stop 2024-09-17T05:33:29.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.212+0000 7f0921000640 1 -- 172.21.15.122:0/2508970534 shutdown_connections 2024-09-17T05:33:29.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.212+0000 7f0921000640 1 --2- 172.21.15.122:0/2508970534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 0x7f091c06f8f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:29.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.212+0000 7f0921000640 1 --2- 172.21.15.122:0/2508970534 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f091c15ac70 0x7f091c0cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:29.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.212+0000 7f0921000640 1 -- 172.21.15.122:0/2508970534 >> 172.21.15.122:0/2508970534 conn(0x7f091c0cd740 msgr2=0x7f091c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:29.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.212+0000 7f0921000640 1 -- 172.21.15.122:0/2508970534 shutdown_connections 2024-09-17T05:33:29.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.212+0000 7f0921000640 1 -- 172.21.15.122:0/2508970534 wait complete. 2024-09-17T05:33:29.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0921000640 1 Processor -- start 2024-09-17T05:33:29.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0921000640 1 -- start start 2024-09-17T05:33:29.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0921000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 0x7f091c15a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:29.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0921000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f091c0cbdb0 0x7f091c155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:29.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0921000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f091c15ac70 0x7f091c1549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:29.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0921000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f091c167b50 con 0x7f091c0b2e50 2024-09-17T05:33:29.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0921000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1679d0 con 0x7f091c0cbdb0 2024-09-17T05:33:29.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0921000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f091c167cd0 con 0x7f091c15ac70 2024-09-17T05:33:29.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0919e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 0x7f091c15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:29.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0919e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 0x7f091c15a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57496/0 (socket says 172.21.15.122:57496) 2024-09-17T05:33:29.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.216+0000 7f0919e00640 1 -- 172.21.15.122:0/3074503624 learned_addr learned my addr 172.21.15.122:0/3074503624 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f091a800640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f091c15ac70 0x7f091c1549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0919e00640 1 -- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f091c15ac70 msgr2=0x7f091c1549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0919e00640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f091c15ac70 0x7f091c1549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0919e00640 1 -- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f091c0cbdb0 msgr2=0x7f091c155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0919e00640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f091c0cbdb0 0x7f091c155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0919e00640 1 -- 172.21.15.122:0/3074503624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c1fb8d0 con 0x7f091c0b2e50 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0919e00640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 0x7f091c15a000 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f090c00b410 tx=0x7f090c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0912000640 1 -- 172.21.15.122:0/3074503624 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c015020 con 0x7f091c0b2e50 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c162cb0 con 0x7f091c0b2e50 2024-09-17T05:33:29.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0912000640 1 -- 172.21.15.122:0/3074503624 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f090c011920 con 0x7f091c0b2e50 2024-09-17T05:33:29.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0912000640 1 -- 172.21.15.122:0/3074503624 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c011d70 con 0x7f091c0b2e50 2024-09-17T05:33:29.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c15b560 con 0x7f091c0b2e50 2024-09-17T05:33:29.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.220+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f091c06f8f0 con 0x7f091c0b2e50 2024-09-17T05:33:29.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.224+0000 7f0912000640 1 -- 172.21.15.122:0/3074503624 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 30) ==== 98713+0+0 (secure 0 0 0) 0x7f090c018020 con 0x7f091c0b2e50 2024-09-17T05:33:29.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.228+0000 7f0912000640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08fc074250 0x7f08fc076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:29.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.228+0000 7f0912000640 1 -- 172.21.15.122:0/3074503624 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f090c093070 con 0x7f091c0b2e50 2024-09-17T05:33:29.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.228+0000 7f0912000640 1 -- 172.21.15.122:0/3074503624 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f090c01f020 con 0x7f091c0b2e50 2024-09-17T05:33:29.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.232+0000 7f0919400640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08fc074250 0x7f08fc076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:29.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.232+0000 7f0919400640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08fc074250 0x7f08fc076710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0908000f70 tx=0x7f09080079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:29.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:29.628+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone34415", "target": ["mon-mgr", ""]}) -- 0x7f091c1fb680 con 0x7f08fc074250 2024-09-17T05:33:31.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.415+0000 7f0912000640 1 -- 172.21.15.122:0/3074503624 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f091c1fb680 con 0x7f08fc074250 2024-09-17T05:33:31.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08fc074250 msgr2=0x7f08fc076710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:31.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08fc074250 0x7f08fc076710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0908000f70 tx=0x7f09080079c0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 msgr2=0x7f091c15a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:31.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 0x7f091c15a000 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f090c00b410 tx=0x7f090c00b8d0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 shutdown_connections 2024-09-17T05:33:31.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08fc074250 0x7f08fc076710 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f091c15ac70 0x7f091c1549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f091c0cbdb0 0x7f091c155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 --2- 172.21.15.122:0/3074503624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f091c0b2e50 0x7f091c15a000 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 >> 172.21.15.122:0/3074503624 conn(0x7f091c0cd740 msgr2=0x7f091c06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:31.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 shutdown_connections 2024-09-17T05:33:31.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.419+0000 7f0921000640 1 -- 172.21.15.122:0/3074503624 wait complete. 2024-09-17T05:33:31.456 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone73276 2024-09-17T05:33:31.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 -- 172.21.15.122:0/3453376543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 msgr2=0x7f4d80003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:31.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 --2- 172.21.15.122:0/3453376543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 0x7f4d80003ea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009940 tx=0x7f4d8c01c720 comp rx=0 tx=0).stop 2024-09-17T05:33:31.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 -- 172.21.15.122:0/3453376543 shutdown_connections 2024-09-17T05:33:31.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 --2- 172.21.15.122:0/3453376543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 0x7f4d80003ea0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 -- 172.21.15.122:0/3453376543 >> 172.21.15.122:0/3453376543 conn(0x7f4d9c0d1c40 msgr2=0x7f4d9c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:31.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 -- 172.21.15.122:0/3453376543 shutdown_connections 2024-09-17T05:33:31.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 -- 172.21.15.122:0/3453376543 wait complete. 2024-09-17T05:33:31.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 Processor -- start 2024-09-17T05:33:31.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.711+0000 7f4d9be00640 1 -- start start 2024-09-17T05:33:31.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 0x7f4d9c09b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:31.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c157c50 0x7f4d9c09c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d9c158090 0x7f4d9c09cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c0cf400 con 0x7f4d80003a60 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c0cf280 con 0x7f4d9c158090 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9c0b56a0 con 0x7f4d9c157c50 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 0x7f4d9c09b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 0x7f4d9c09b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45918/0 (socket says 172.21.15.122:45918) 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 -- 172.21.15.122:0/359637062 learned_addr learned my addr 172.21.15.122:0/359637062 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 -- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c157c50 msgr2=0x7f4d9c09c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c157c50 0x7f4d9c09c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 -- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d9c158090 msgr2=0x7f4d9c09cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d9c158090 0x7f4d9c09cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:31.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 -- 172.21.15.122:0/359637062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9c158780 con 0x7f4d80003a60 2024-09-17T05:33:31.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9aa00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 0x7f4d9c09b990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c01c550 tx=0x7f4d8c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:31.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d93e00640 1 -- 172.21.15.122:0/359637062 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c005bf0 con 0x7f4d80003a60 2024-09-17T05:33:31.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d93e00640 1 -- 172.21.15.122:0/359637062 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d8c0a9410 con 0x7f4d80003a60 2024-09-17T05:33:31.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d93e00640 1 -- 172.21.15.122:0/359637062 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0a9a20 con 0x7f4d80003a60 2024-09-17T05:33:31.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d9c1fbf80 con 0x7f4d80003a60 2024-09-17T05:33:31.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.715+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d9c1fc540 con 0x7f4d80003a60 2024-09-17T05:33:31.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.719+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9c0b5c00 con 0x7f4d80003a60 2024-09-17T05:33:31.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.719+0000 7f4d93e00640 1 -- 172.21.15.122:0/359637062 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 30) ==== 98713+0+0 (secure 0 0 0) 0x7f4d8c0a9bc0 con 0x7f4d80003a60 2024-09-17T05:33:31.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.719+0000 7f4d93e00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d78074250 0x7f4d78076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:31.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.719+0000 7f4d93e00640 1 -- 172.21.15.122:0/359637062 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f4d8c12c780 con 0x7f4d80003a60 2024-09-17T05:33:31.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.723+0000 7f4d93e00640 1 -- 172.21.15.122:0/359637062 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4d8c0c1070 con 0x7f4d80003a60 2024-09-17T05:33:31.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.723+0000 7f4d92000640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d78074250 0x7f4d78076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:31.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:31.735+0000 7f4d92000640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d78074250 0x7f4d78076710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d84001400 tx=0x7f4d840012b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:32.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:32.083+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone73276", "target": ["mon-mgr", ""]}) -- 0x7f4d9c0cc070 con 0x7f4d78074250 2024-09-17T05:33:32.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:33:32.882 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:33:33.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.095+0000 7f4d93e00640 1 -- 172.21.15.122:0/359637062 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d9c0cc070 con 0x7f4d78074250 2024-09-17T05:33:33.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d78074250 msgr2=0x7f4d78076710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d78074250 0x7f4d78076710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d84001400 tx=0x7f4d840012b0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 msgr2=0x7f4d9c09b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 0x7f4d9c09b990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c01c550 tx=0x7f4d8c004dc0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 shutdown_connections 2024-09-17T05:33:33.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d78074250 0x7f4d78076710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d9c158090 0x7f4d9c09cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c157c50 0x7f4d9c09c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 --2- 172.21.15.122:0/359637062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d80003a60 0x7f4d9c09b990 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 >> 172.21.15.122:0/359637062 conn(0x7f4d9c0d1c40 msgr2=0x7f4d9c15b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:33.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 shutdown_connections 2024-09-17T05:33:33.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.099+0000 7f4d9be00640 1 -- 172.21.15.122:0/359637062 wait complete. 2024-09-17T05:33:33.128 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:33:33.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.359+0000 7f2e91a00640 1 -- 172.21.15.122:0/3588939728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 msgr2=0x7f2e8c0ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.359+0000 7f2e91a00640 1 --2- 172.21.15.122:0/3588939728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 0x7f2e8c0ccbc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009840 tx=0x7f2e7c01c820 comp rx=0 tx=0).stop 2024-09-17T05:33:33.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.359+0000 7f2e91a00640 1 -- 172.21.15.122:0/3588939728 shutdown_connections 2024-09-17T05:33:33.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.359+0000 7f2e91a00640 1 --2- 172.21.15.122:0/3588939728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e8c071b60 0x7f2e8c073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.359+0000 7f2e91a00640 1 --2- 172.21.15.122:0/3588939728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 0x7f2e8c0ccbc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.359+0000 7f2e91a00640 1 -- 172.21.15.122:0/3588939728 >> 172.21.15.122:0/3588939728 conn(0x7f2e8c0d0cd0 msgr2=0x7f2e8c0d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:33.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.359+0000 7f2e91a00640 1 -- 172.21.15.122:0/3588939728 shutdown_connections 2024-09-17T05:33:33.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.359+0000 7f2e91a00640 1 -- 172.21.15.122:0/3588939728 wait complete. 2024-09-17T05:33:33.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e91a00640 1 Processor -- start 2024-09-17T05:33:33.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e91a00640 1 -- start start 2024-09-17T05:33:33.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e91a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e8c071b60 0x7f2e8c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e91a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e8c073820 0x7f2e8c14ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e91a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 0x7f2e8c14b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e91a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c167fa0 con 0x7f2e8c15bd80 2024-09-17T05:33:33.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e91a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c167e20 con 0x7f2e8c073820 2024-09-17T05:33:33.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e91a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8c168120 con 0x7f2e8c071b60 2024-09-17T05:33:33.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 0x7f2e8c14b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 0x7f2e8c14b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45926/0 (socket says 172.21.15.122:45926) 2024-09-17T05:33:33.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8b400640 1 -- 172.21.15.122:0/1853716756 learned_addr learned my addr 172.21.15.122:0/1853716756 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:33.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8aa00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e8c071b60 0x7f2e8c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8b400640 1 -- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e8c071b60 msgr2=0x7f2e8c158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8b400640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e8c071b60 0x7f2e8c158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8b400640 1 -- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e8c073820 msgr2=0x7f2e8c14ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8b400640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e8c073820 0x7f2e8c14ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.363+0000 7f2e8b400640 1 -- 172.21.15.122:0/1853716756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c06ac90 con 0x7f2e8c15bd80 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.367+0000 7f2e8b400640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 0x7f2e8c14b3b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009f60 tx=0x7f2e84009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.367+0000 7f2e82a00640 1 -- 172.21.15.122:0/1853716756 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8401d070 con 0x7f2e8c15bd80 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.367+0000 7f2e82a00640 1 -- 172.21.15.122:0/1853716756 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e84018530 con 0x7f2e8c15bd80 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.367+0000 7f2e82a00640 1 -- 172.21.15.122:0/1853716756 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e840189c0 con 0x7f2e8c15bd80 2024-09-17T05:33:33.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.367+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c15ab20 con 0x7f2e8c15bd80 2024-09-17T05:33:33.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.367+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8c0b6780 con 0x7f2e8c15bd80 2024-09-17T05:33:33.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.367+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8c073c60 con 0x7f2e8c15bd80 2024-09-17T05:33:33.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.371+0000 7f2e82a00640 1 -- 172.21.15.122:0/1853716756 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 30) ==== 98713+0+0 (secure 0 0 0) 0x7f2e84020070 con 0x7f2e8c15bd80 2024-09-17T05:33:33.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.371+0000 7f2e82a00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e6c074250 0x7f2e6c076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:33.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.371+0000 7f2e82a00640 1 -- 172.21.15.122:0/1853716756 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f2e84097230 con 0x7f2e8c15bd80 2024-09-17T05:33:33.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.379+0000 7f2e82a00640 1 -- 172.21.15.122:0/1853716756 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2e8c15ab20 con 0x7f2e8c15bd80 2024-09-17T05:33:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.383+0000 7f2e8aa00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e6c074250 0x7f2e6c076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.403+0000 7f2e8aa00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e6c074250 0x7f2e6c076710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0af000 tx=0x7f2e7c01b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:33.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.731+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2e8c1659e0 con 0x7f2e8c15bd80 2024-09-17T05:33:33.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.731+0000 7f2e82a00640 1 -- 172.21.15.122:0/1853716756 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2636 (secure 0 0 0) 0x7f2e84063a10 con 0x7f2e8c15bd80 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:33:33.729 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 534, 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:33:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 77, 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:33:33.731 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 1440, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 5889996926, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 11784159232, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 761309954048, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 235132813, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 261026256, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 57, 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 74 2024-09-17T05:33:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 47, 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:32:51:021047+0000", 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 5, 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 5, 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "c", 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:33:33.733 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 6426 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:33:33.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 13, 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:33:32.583961+0000", 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "services": { 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "daemons": { 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "summary": "", 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "c": { 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "start_epoch": 0, 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "start_stamp": "0.000000", 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 0, 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T05:33:33.735 INFO:teuthology.orchestra.run.smithi122.stdout: "metadata": {}, 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: "task_status": {} 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "4 ongoing clones - average progress is 37.044% (6s)\n [==========..................] (remaining: 11s)", 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.37044000625610352, 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:33.736 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:33:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e6c074250 msgr2=0x7f2e6c076710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e6c074250 0x7f2e6c076710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0af000 tx=0x7f2e7c01b060 comp rx=0 tx=0).stop 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 msgr2=0x7f2e8c14b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 0x7f2e8c14b3b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009f60 tx=0x7f2e84009f90 comp rx=0 tx=0).stop 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 shutdown_connections 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e6c074250 0x7f2e6c076710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e8c15bd80 0x7f2e8c14b3b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e8c073820 0x7f2e8c14ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 --2- 172.21.15.122:0/1853716756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e8c071b60 0x7f2e8c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 >> 172.21.15.122:0/1853716756 conn(0x7f2e8c0d0cd0 msgr2=0x7f2e8c15e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 shutdown_connections 2024-09-17T05:33:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:33.735+0000 7f2e91a00640 1 -- 172.21.15.122:0/1853716756 wait complete. 2024-09-17T05:33:34.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:33:35.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.011+0000 7f7149a00640 1 -- 172.21.15.122:0/659787146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71440b2d00 msgr2=0x7f714415fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:35.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.011+0000 7f7149a00640 1 --2- 172.21.15.122:0/659787146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71440b2d00 0x7f714415fec0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7130009900 tx=0x7f713001c850 comp rx=0 tx=0).stop 2024-09-17T05:33:35.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.011+0000 7f7149a00640 1 -- 172.21.15.122:0/659787146 shutdown_connections 2024-09-17T05:33:35.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.011+0000 7f7149a00640 1 --2- 172.21.15.122:0/659787146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71440b2d00 0x7f714415fec0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.011+0000 7f7149a00640 1 --2- 172.21.15.122:0/659787146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71440cfe80 0x7f71440d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.011+0000 7f7149a00640 1 -- 172.21.15.122:0/659787146 >> 172.21.15.122:0/659787146 conn(0x7f71440d1e40 msgr2=0x7f71440d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:35.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.011+0000 7f7149a00640 1 -- 172.21.15.122:0/659787146 shutdown_connections 2024-09-17T05:33:35.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.011+0000 7f7149a00640 1 -- 172.21.15.122:0/659787146 wait complete. 2024-09-17T05:33:35.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.015+0000 7f7149a00640 1 Processor -- start 2024-09-17T05:33:35.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.015+0000 7f7149a00640 1 -- start start 2024-09-17T05:33:35.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.019+0000 7f7149a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71440b2d00 0x7f71440cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:35.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.019+0000 7f7149a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71440cfe80 0x7f71440b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:35.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.019+0000 7f7149a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7144160400 0x7f714415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:35.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.019+0000 7f7149a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7144169520 con 0x7f7144160400 2024-09-17T05:33:35.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.019+0000 7f7149a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71441693a0 con 0x7f71440b2d00 2024-09-17T05:33:35.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.019+0000 7f7149a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f71441696a0 con 0x7f71440cfe80 2024-09-17T05:33:35.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71440cfe80 0x7f71440b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:35.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71440b2d00 0x7f71440cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:35.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71440cfe80 0x7f71440b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:52182/0 (socket says 172.21.15.122:52182) 2024-09-17T05:33:35.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71440b2d00 0x7f71440cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:42958/0 (socket says 172.21.15.122:42958) 2024-09-17T05:33:35.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142000640 1 -- 172.21.15.122:0/3745714378 learned_addr learned my addr 172.21.15.122:0/3745714378 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:35.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7143400640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7144160400 0x7f714415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:35.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142a00640 1 -- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71440cfe80 msgr2=0x7f71440b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:35.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71440cfe80 0x7f71440b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142a00640 1 -- 172.21.15.122:0/3745714378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7144160400 msgr2=0x7f714415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:35.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7144160400 0x7f714415cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.023+0000 7f7142a00640 1 -- 172.21.15.122:0/3745714378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71440b7310 con 0x7f71440b2d00 2024-09-17T05:33:35.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.031+0000 7f7142a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71440b2d00 0x7f71440cbe90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f713400b3f0 tx=0x7f713400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.031+0000 7f713aa00640 1 -- 172.21.15.122:0/3745714378 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134015020 con 0x7f71440b2d00 2024-09-17T05:33:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.031+0000 7f713aa00640 1 -- 172.21.15.122:0/3745714378 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7134011920 con 0x7f71440b2d00 2024-09-17T05:33:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.031+0000 7f713aa00640 1 -- 172.21.15.122:0/3745714378 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134011d70 con 0x7f71440b2d00 2024-09-17T05:33:35.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.035+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71440c9b30 con 0x7f71440b2d00 2024-09-17T05:33:35.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.035+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71440c5140 con 0x7f71440b2d00 2024-09-17T05:33:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.035+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714415fec0 con 0x7f71440b2d00 2024-09-17T05:33:35.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.043+0000 7f713aa00640 1 -- 172.21.15.122:0/3745714378 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 30) ==== 98713+0+0 (secure 0 0 0) 0x7f7134018020 con 0x7f71440b2d00 2024-09-17T05:33:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.047+0000 7f713aa00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71240742a0 0x7f7124076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:35.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.047+0000 7f713aa00640 1 -- 172.21.15.122:0/3745714378 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f7134093490 con 0x7f71440b2d00 2024-09-17T05:33:35.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.055+0000 7f713aa00640 1 -- 172.21.15.122:0/3745714378 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f713401f020 con 0x7f71440b2d00 2024-09-17T05:33:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.067+0000 7f7142000640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71240742a0 0x7f7124076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:35.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.095+0000 7f7142000640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71240742a0 0x7f7124076760 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f713001af00 tx=0x7f713001b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.503+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f71440b4840 con 0x7f71440b2d00 2024-09-17T05:33:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.507+0000 7f713aa00640 1 -- 172.21.15.122:0/3745714378 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2638 (secure 0 0 0) 0x7f713405fcb0 con 0x7f71440b2d00 2024-09-17T05:33:35.504 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:35.504 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:33:35.504 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:35.505 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 536, 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 77, 2024-09-17T05:33:35.506 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:35.507 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 2323, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 9593780806, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 19095449600, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 753998663680, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 544687461, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 569552799, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 132, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 154 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:35.508 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 47, 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:32:51:021047+0000", 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 5, 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 5, 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "c", 2024-09-17T05:33:35.509 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 6426 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:33:35.510 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 13, 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:33:32.583961+0000", 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "services": { 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "daemons": { 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "summary": "", 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "c": { 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "start_epoch": 0, 2024-09-17T05:33:35.511 INFO:teuthology.orchestra.run.smithi122.stdout: "start_stamp": "0.000000", 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 0, 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: "metadata": {}, 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: "task_status": {} 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:33:35.512 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "4 ongoing clones - average progress is 37.044% (6s)\n [==========..................] (remaining: 11s)", 2024-09-17T05:33:35.513 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.37044000625610352, 2024-09-17T05:33:35.513 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:33:35.513 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:35.513 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:35.513 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:33:35.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71240742a0 msgr2=0x7f7124076760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:35.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71240742a0 0x7f7124076760 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f713001af00 tx=0x7f713001b090 comp rx=0 tx=0).stop 2024-09-17T05:33:35.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71440b2d00 msgr2=0x7f71440cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:35.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71440b2d00 0x7f71440cbe90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f713400b3f0 tx=0x7f713400b8b0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 shutdown_connections 2024-09-17T05:33:35.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71240742a0 0x7f7124076760 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7144160400 0x7f714415cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71440cfe80 0x7f71440b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 --2- 172.21.15.122:0/3745714378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71440b2d00 0x7f71440cbe90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:35.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 >> 172.21.15.122:0/3745714378 conn(0x7f71440d1e40 msgr2=0x7f71440cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:35.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 shutdown_connections 2024-09-17T05:33:35.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:35.511+0000 7f7149a00640 1 -- 172.21.15.122:0/3745714378 wait complete. 2024-09-17T05:33:36.565 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:33:36.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.843+0000 7f1fdb200640 1 -- 172.21.15.122:0/3477045813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc0cd190 msgr2=0x7f1fdc15e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:36.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.843+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3477045813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc0cd190 0x7f1fdc15e100 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400ad90 tx=0x7f1fd401db60 comp rx=0 tx=0).stop 2024-09-17T05:33:36.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.843+0000 7f1fdb200640 1 -- 172.21.15.122:0/3477045813 shutdown_connections 2024-09-17T05:33:36.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.843+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3477045813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc0cd190 0x7f1fdc15e100 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.843+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3477045813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fdc0ad070 0x7f1fdc0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.843+0000 7f1fdb200640 1 -- 172.21.15.122:0/3477045813 >> 172.21.15.122:0/3477045813 conn(0x7f1fdc0742d0 msgr2=0x7f1fdc0746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:36.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.843+0000 7f1fdb200640 1 -- 172.21.15.122:0/3477045813 shutdown_connections 2024-09-17T05:33:36.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.843+0000 7f1fdb200640 1 -- 172.21.15.122:0/3477045813 wait complete. 2024-09-17T05:33:36.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.847+0000 7f1fdb200640 1 Processor -- start 2024-09-17T05:33:36.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.847+0000 7f1fdb200640 1 -- start start 2024-09-17T05:33:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.847+0000 7f1fdb200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0ad070 0x7f1fdc070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.847+0000 7f1fdb200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc0cd190 0x7f1fdc097be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fdb200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fdc157f90 0x7f1fdc0c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fdb200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc15e9d0 con 0x7f1fdc0ad070 2024-09-17T05:33:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fdb200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc15e850 con 0x7f1fdc0cd190 2024-09-17T05:33:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fdb200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1fdc15eb50 con 0x7f1fdc157f90 2024-09-17T05:33:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fd9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0ad070 0x7f1fdc070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fd9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0ad070 0x7f1fdc070bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45956/0 (socket says 172.21.15.122:45956) 2024-09-17T05:33:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fd9e00640 1 -- 172.21.15.122:0/3284617040 learned_addr learned my addr 172.21.15.122:0/3284617040 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fda800640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fdc157f90 0x7f1fdc0c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fd9e00640 1 -- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fdc157f90 msgr2=0x7f1fdc0c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fd9e00640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fdc157f90 0x7f1fdc0c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fd9e00640 1 -- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc0cd190 msgr2=0x7f1fdc097be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fd9e00640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc0cd190 0x7f1fdc097be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.851+0000 7f1fd9e00640 1 -- 172.21.15.122:0/3284617040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc159f80 con 0x7f1fdc0ad070 2024-09-17T05:33:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.855+0000 7f1fd9e00640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0ad070 0x7f1fdc070bb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00c3c0 tx=0x7f1fcc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.855+0000 7f1fd2000640 1 -- 172.21.15.122:0/3284617040 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc017070 con 0x7f1fdc0ad070 2024-09-17T05:33:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.855+0000 7f1fd2000640 1 -- 172.21.15.122:0/3284617040 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fcc0053c0 con 0x7f1fdc0ad070 2024-09-17T05:33:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.855+0000 7f1fd2000640 1 -- 172.21.15.122:0/3284617040 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc003860 con 0x7f1fdc0ad070 2024-09-17T05:33:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.855+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc0c1ec0 con 0x7f1fdc0ad070 2024-09-17T05:33:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.855+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc0c2480 con 0x7f1fdc0ad070 2024-09-17T05:33:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.855+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fdc0ccc50 con 0x7f1fdc0ad070 2024-09-17T05:33:36.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.859+0000 7f1fd2000640 1 -- 172.21.15.122:0/3284617040 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 30) ==== 98713+0+0 (secure 0 0 0) 0x7f1fcc01a070 con 0x7f1fdc0ad070 2024-09-17T05:33:36.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.859+0000 7f1fd2000640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fbc074250 0x7f1fbc076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:36.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.859+0000 7f1fd2000640 1 -- 172.21.15.122:0/3284617040 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f1fcc092840 con 0x7f1fdc0ad070 2024-09-17T05:33:36.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.867+0000 7f1fd2000640 1 -- 172.21.15.122:0/3284617040 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1fcc021020 con 0x7f1fdc0ad070 2024-09-17T05:33:36.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.887+0000 7f1fd9400640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fbc074250 0x7f1fbc076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:36.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:36.935+0000 7f1fd9400640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fbc074250 0x7f1fbc076710 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc1f8a80 tx=0x7f1fc8006ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1fdc0731a0 con 0x7f1fdc0ad070 2024-09-17T05:33:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fd2000640 1 -- 172.21.15.122:0/3284617040 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2877 (secure 0 0 0) 0x7f1fdc0cd620 con 0x7f1fdc0ad070 2024-09-17T05:33:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:37.285 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:33:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:33:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:33:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:33:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:33:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 538, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 77, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 2323, 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 9593780806, 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 19095449600, 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 753998663680, 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 544651500, 2024-09-17T05:33:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 530728098, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 132, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 143 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 47, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:32:51:021047+0000", 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 5, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 5, 2024-09-17T05:33:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "c", 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 6426 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:33:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 13, 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:33:32.583961+0000", 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "services": { 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "daemons": { 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "summary": "", 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "c": { 2024-09-17T05:33:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "start_epoch": 0, 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "start_stamp": "0.000000", 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 0, 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "metadata": {}, 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "task_status": {} 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:33:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "4 ongoing clones - average progress is 20.898% (9s)\n [=====.......................] (remaining: 37s)", 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.20897999405860901, 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-total-clones": { 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "Total 7 clones - average progress is 31.213% (0s)\n [========....................] ", 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.31213000416755676, 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:37.293 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:33:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fbc074250 msgr2=0x7f1fbc076710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fbc074250 0x7f1fbc076710 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc1f8a80 tx=0x7f1fc8006ca0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0ad070 msgr2=0x7f1fdc070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0ad070 0x7f1fdc070bb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00c3c0 tx=0x7f1fcc00c880 comp rx=0 tx=0).stop 2024-09-17T05:33:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 shutdown_connections 2024-09-17T05:33:37.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fbc074250 0x7f1fbc076710 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fdc157f90 0x7f1fdc0c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fdc0cd190 0x7f1fdc097be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 --2- 172.21.15.122:0/3284617040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0ad070 0x7f1fdc070bb0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.287+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 >> 172.21.15.122:0/3284617040 conn(0x7f1fdc0742d0 msgr2=0x7f1fdc154690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:37.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.291+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 shutdown_connections 2024-09-17T05:33:37.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.291+0000 7f1fdb200640 1 -- 172.21.15.122:0/3284617040 wait complete. 2024-09-17T05:33:37.310 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone67490 2024-09-17T05:33:37.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.575+0000 7f80aea00640 1 -- 172.21.15.122:0/3814935571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a80b2cf0 msgr2=0x7f80a80d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:37.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.575+0000 7f80aea00640 1 --2- 172.21.15.122:0/3814935571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a80b2cf0 0x7f80a80d0000 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f80a809fb60 tx=0x7f809401c760 comp rx=0 tx=0).stop 2024-09-17T05:33:37.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.575+0000 7f80aea00640 1 -- 172.21.15.122:0/3814935571 shutdown_connections 2024-09-17T05:33:37.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.575+0000 7f80aea00640 1 --2- 172.21.15.122:0/3814935571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a80b2cf0 0x7f80a80d0000 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.575+0000 7f80aea00640 1 --2- 172.21.15.122:0/3814935571 >> v2:172.21.15.173:3300/0 conn(0x7f80a815f290 0x7f80a80cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.575+0000 7f80aea00640 1 -- 172.21.15.122:0/3814935571 >> 172.21.15.122:0/3814935571 conn(0x7f80a80d1bc0 msgr2=0x7f80a80d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:37.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.575+0000 7f80aea00640 1 -- 172.21.15.122:0/3814935571 shutdown_connections 2024-09-17T05:33:37.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.575+0000 7f80aea00640 1 -- 172.21.15.122:0/3814935571 wait complete. 2024-09-17T05:33:37.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 Processor -- start 2024-09-17T05:33:37.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 -- start start 2024-09-17T05:33:37.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a80b3160 0x7f80a8148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:37.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a815f290 0x7f80a81516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:37.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a81495f0 0x7f80a80c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:37.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80a8166e00 con 0x7f80a815f290 2024-09-17T05:33:37.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80a8166c80 con 0x7f80a81495f0 2024-09-17T05:33:37.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f80a8166f80 con 0x7f80a80b3160 2024-09-17T05:33:37.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a815f290 0x7f80a81516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:37.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a815f290 0x7f80a81516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45964/0 (socket says 172.21.15.122:45964) 2024-09-17T05:33:37.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 -- 172.21.15.122:0/4272441922 learned_addr learned my addr 172.21.15.122:0/4272441922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:37.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80ad600640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a80b3160 0x7f80a8148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:37.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 -- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a80b3160 msgr2=0x7f80a8148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:37.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80ae000640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a81495f0 0x7f80a80c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:37.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a80b3160 0x7f80a8148d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 -- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a81495f0 msgr2=0x7f80a80c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:37.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a81495f0 0x7f80a80c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:37.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 -- 172.21.15.122:0/4272441922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a8152030 con 0x7f80a815f290 2024-09-17T05:33:37.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80ae000640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a81495f0 0x7f80a80c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:37.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80acc00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a815f290 0x7f80a81516c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f809401cc40 tx=0x7f8094005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:37.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80a5600640 1 -- 172.21.15.122:0/4272441922 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940be070 con 0x7f80a815f290 2024-09-17T05:33:37.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80a5600640 1 -- 172.21.15.122:0/4272441922 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80940a8550 con 0x7f80a815f290 2024-09-17T05:33:37.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80a5600640 1 -- 172.21.15.122:0/4272441922 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940a8bc0 con 0x7f80a815f290 2024-09-17T05:33:37.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a8152310 con 0x7f80a815f290 2024-09-17T05:33:37.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.579+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a814e180 con 0x7f80a815f290 2024-09-17T05:33:37.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.583+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a8167480 con 0x7f80a815f290 2024-09-17T05:33:37.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.583+0000 7f80a5600640 1 -- 172.21.15.122:0/4272441922 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 30) ==== 98713+0+0 (secure 0 0 0) 0x7f80940a6040 con 0x7f80a815f290 2024-09-17T05:33:37.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.587+0000 7f80a5600640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8088074250 0x7f8088076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:37.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.587+0000 7f80a5600640 1 -- 172.21.15.122:0/4272441922 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f809412d280 con 0x7f80a815f290 2024-09-17T05:33:37.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.591+0000 7f80a5600640 1 -- 172.21.15.122:0/4272441922 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f80940c7020 con 0x7f80a815f290 2024-09-17T05:33:37.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.603+0000 7f80ad600640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8088074250 0x7f8088076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:37.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.639+0000 7f80ad600640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8088074250 0x7f8088076710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f809c0058d0 tx=0x7f809c005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:37.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:37.971+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone67490", "target": ["mon-mgr", ""]}) -- 0x7f80a81676f0 con 0x7f8088074250 2024-09-17T05:33:38.326 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:38.328+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:38.326 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:38.328+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:38.326 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:38.328+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:38.326 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:38.328+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:38.326 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:38.328+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:38.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:38.831+0000 7f80a5600640 1 -- 172.21.15.122:0/4272441922 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f80940f55d0 con 0x7f80a815f290 2024-09-17T05:33:40.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.267+0000 7f80a5600640 1 -- 172.21.15.122:0/4272441922 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80a81676f0 con 0x7f8088074250 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8088074250 msgr2=0x7f8088076710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8088074250 0x7f8088076710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f809c0058d0 tx=0x7f809c005860 comp rx=0 tx=0).stop 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a815f290 msgr2=0x7f80a81516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a815f290 0x7f80a81516c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f809401cc40 tx=0x7f8094005df0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 shutdown_connections 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8088074250 0x7f8088076710 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a81495f0 0x7f80a80c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80a815f290 0x7f80a81516c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 --2- 172.21.15.122:0/4272441922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a80b3160 0x7f80a8148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 >> 172.21.15.122:0/4272441922 conn(0x7f80a80d1bc0 msgr2=0x7f80a80d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:40.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 shutdown_connections 2024-09-17T05:33:40.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.271+0000 7f80aea00640 1 -- 172.21.15.122:0/4272441922 wait complete. 2024-09-17T05:33:40.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone64781 2024-09-17T05:33:40.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- 172.21.15.122:0/2357490229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d340d0060 msgr2=0x7f5d340d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:40.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 --2- 172.21.15.122:0/2357490229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d340d0060 0x7f5d340d0510 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009bc0 tx=0x7f5d2c01cb10 comp rx=0 tx=0).stop 2024-09-17T05:33:40.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- 172.21.15.122:0/2357490229 shutdown_connections 2024-09-17T05:33:40.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 --2- 172.21.15.122:0/2357490229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d340d0060 0x7f5d340d0510 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 --2- 172.21.15.122:0/2357490229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d3406d330 0x7f5d34161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- 172.21.15.122:0/2357490229 >> 172.21.15.122:0/2357490229 conn(0x7f5d340cd740 msgr2=0x7f5d340cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:40.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- 172.21.15.122:0/2357490229 shutdown_connections 2024-09-17T05:33:40.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- 172.21.15.122:0/2357490229 wait complete. 2024-09-17T05:33:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 Processor -- start 2024-09-17T05:33:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- start start 2024-09-17T05:33:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3406d330 0x7f5d341f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d340cd060 0x7f5d3415a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d340d0060 0x7f5d340ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3416b140 con 0x7f5d340d0060 2024-09-17T05:33:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3416afc0 con 0x7f5d3406d330 2024-09-17T05:33:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.595+0000 7f5d39e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3416b2c0 con 0x7f5d340cd060 2024-09-17T05:33:40.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d340d0060 0x7f5d340ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d340d0060 0x7f5d340ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45988/0 (socket says 172.21.15.122:45988) 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 -- 172.21.15.122:0/3744200279 learned_addr learned my addr 172.21.15.122:0/3744200279 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d32000640 1 --2- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d340cd060 0x7f5d3415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 -- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d340cd060 msgr2=0x7f5d3415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 --2- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d340cd060 0x7f5d3415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 -- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3406d330 msgr2=0x7f5d341f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 --2- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d3406d330 0x7f5d341f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 -- 172.21.15.122:0/3744200279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3415afa0 con 0x7f5d340d0060 2024-09-17T05:33:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d32000640 1 --2- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d340cd060 0x7f5d3415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:40.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d33400640 1 --2- 172.21.15.122:0/3744200279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d340d0060 0x7f5d340ca0a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c01a740 tx=0x7f5d2c0a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:40.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d2b400640 1 -- 172.21.15.122:0/3744200279 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0bb070 con 0x7f5d340d0060 2024-09-17T05:33:40.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d39e00640 1 -- 172.21.15.122:0/3744200279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3415b280 con 0x7f5d340d0060 2024-09-17T05:33:40.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d2b400640 1 -- 172.21.15.122:0/3744200279 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d2c004d40 con 0x7f5d340d0060 2024-09-17T05:33:40.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d2b400640 1 -- 172.21.15.122:0/3744200279 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0b2040 con 0x7f5d340d0060 2024-09-17T05:33:40.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d39e00640 1 -- 172.21.15.122:0/3744200279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3415b840 con 0x7f5d340d0060 2024-09-17T05:33:40.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.599+0000 7f5d39e00640 1 -- 172.21.15.122:0/3744200279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d34161a10 con 0x7f5d340d0060 2024-09-17T05:33:40.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.603+0000 7f5d2b400640 1 -- 172.21.15.122:0/3744200279 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f5d2c004320 con 0x7f5d340d0060 2024-09-17T05:33:40.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.603+0000 7f5d2b400640 1 --2- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d140742e0 0x7f5d140767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:40.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.603+0000 7f5d2b400640 1 -- 172.21.15.122:0/3744200279 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f5d2c12cd10 con 0x7f5d340d0060 2024-09-17T05:33:40.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.615+0000 7f5d2b400640 1 -- 172.21.15.122:0/3744200279 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5d2c016790 con 0x7f5d340d0060 2024-09-17T05:33:40.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.623+0000 7f5d32a00640 1 --2- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d140742e0 0x7f5d140767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.639+0000 7f5d32a00640 1 --2- 172.21.15.122:0/3744200279 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d140742e0 0x7f5d140767a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5d240015d0 tx=0x7f5d24007a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:40.983+0000 7f5d39e00640 1 -- 172.21.15.122:0/3744200279 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone64781", "target": ["mon-mgr", ""]}) -- 0x7f5d340b6c40 con 0x7f5d140742e0 2024-09-17T05:33:41.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.295+0000 7f5d2b400640 1 -- 172.21.15.122:0/3744200279 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f5d340b6c40 con 0x7f5d140742e0 2024-09-17T05:33:41.292 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:33:41.296 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:41.292+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:33:41.307 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:33:41.308 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone77222 2024-09-17T05:33:41.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.635+0000 7fcb6aa00640 1 -- 172.21.15.122:0/2681307188 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0a6d90 con 0x7fcb6c1605d0 2024-09-17T05:33:41.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- 172.21.15.122:0/2681307188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb6c1605d0 msgr2=0x7fcb6c0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:41.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 --2- 172.21.15.122:0/2681307188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb6c1605d0 0x7fcb6c0d0690 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009840 tx=0x7fcb5c01c820 comp rx=0 tx=0).stop 2024-09-17T05:33:41.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- 172.21.15.122:0/2681307188 shutdown_connections 2024-09-17T05:33:41.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 --2- 172.21.15.122:0/2681307188 >> v2:172.21.15.122:3300/0 conn(0x7fcb6c0afb10 0x7fcb6c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 --2- 172.21.15.122:0/2681307188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb6c1605d0 0x7fcb6c0d0690 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- 172.21.15.122:0/2681307188 >> 172.21.15.122:0/2681307188 conn(0x7fcb6c0d1dd0 msgr2=0x7fcb6c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:41.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- 172.21.15.122:0/2681307188 shutdown_connections 2024-09-17T05:33:41.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- 172.21.15.122:0/2681307188 wait complete. 2024-09-17T05:33:41.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 Processor -- start 2024-09-17T05:33:41.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- start start 2024-09-17T05:33:41.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb6c0afb10 0x7fcb6c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:41.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb6c0b1650 0x7fcb6c0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:41.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb6c1605d0 0x7fcb6c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:41.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c0b0e70 con 0x7fcb6c1605d0 2024-09-17T05:33:41.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6c0b1430 con 0x7fcb6c0b1650 2024-09-17T05:33:41.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.639+0000 7fcb72a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6c169540 con 0x7fcb6c0afb10 2024-09-17T05:33:41.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb70a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb6c1605d0 0x7fcb6c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:41.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb6be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb6c0afb10 0x7fcb6c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:41.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb70a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb6c1605d0 0x7fcb6c0cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51688/0 (socket says 172.21.15.122:51688) 2024-09-17T05:33:41.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb70a00640 1 -- 172.21.15.122:0/769913052 learned_addr learned my addr 172.21.15.122:0/769913052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:41.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb6b400640 1 --2- 172.21.15.122:0/769913052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb6c0b1650 0x7fcb6c0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:41.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb6be00640 1 -- 172.21.15.122:0/769913052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb6c0b1650 msgr2=0x7fcb6c0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:41.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb6be00640 1 --2- 172.21.15.122:0/769913052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb6c0b1650 0x7fcb6c0c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb6be00640 1 -- 172.21.15.122:0/769913052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb6c1605d0 msgr2=0x7fcb6c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:41.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb6be00640 1 --2- 172.21.15.122:0/769913052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb6c1605d0 0x7fcb6c0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:41.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb6be00640 1 -- 172.21.15.122:0/769913052 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6c159810 con 0x7fcb6c0afb10 2024-09-17T05:33:41.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb6be00640 1 --2- 172.21.15.122:0/769913052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb6c0afb10 0x7fcb6c1592d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0ac000 tx=0x7fcb5c005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:41.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb63e00640 1 -- 172.21.15.122:0/769913052 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0bd020 con 0x7fcb6c0afb10 2024-09-17T05:33:41.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb63e00640 1 -- 172.21.15.122:0/769913052 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb5c004330 con 0x7fcb6c0afb10 2024-09-17T05:33:41.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb63e00640 1 -- 172.21.15.122:0/769913052 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c003820 con 0x7fcb6c0afb10 2024-09-17T05:33:41.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb72a00640 1 -- 172.21.15.122:0/769913052 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6c0c62b0 con 0x7fcb6c0afb10 2024-09-17T05:33:41.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb72a00640 1 -- 172.21.15.122:0/769913052 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb6c0c8fd0 con 0x7fcb6c0afb10 2024-09-17T05:33:41.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.643+0000 7fcb72a00640 1 -- 172.21.15.122:0/769913052 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6c0affc0 con 0x7fcb6c0afb10 2024-09-17T05:33:41.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.647+0000 7fcb63e00640 1 -- 172.21.15.122:0/769913052 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7fcb5c0c0020 con 0x7fcb6c0afb10 2024-09-17T05:33:41.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.647+0000 7fcb63e00640 1 --2- 172.21.15.122:0/769913052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb4c0742e0 0x7fcb4c0767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:41.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.647+0000 7fcb63e00640 1 -- 172.21.15.122:0/769913052 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fcb5c12dcc0 con 0x7fcb6c0afb10 2024-09-17T05:33:41.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.647+0000 7fcb6b400640 1 --2- 172.21.15.122:0/769913052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb4c0742e0 0x7fcb4c0767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.651+0000 7fcb6b400640 1 --2- 172.21.15.122:0/769913052 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb4c0742e0 0x7fcb4c0767a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c166cf0 tx=0x7fcb58001420 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:41.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:41.659+0000 7fcb63e00640 1 -- 172.21.15.122:0/769913052 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcb5c0c7030 con 0x7fcb6c0afb10 2024-09-17T05:33:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.023+0000 7fcb72a00640 1 -- 172.21.15.122:0/769913052 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone77222", "target": ["mon-mgr", ""]}) -- 0x7fcb6c0b1ef0 con 0x7fcb4c0742e0 2024-09-17T05:33:42.448 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:42.448+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:33:42.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.459+0000 7fcb63e00640 1 -- 172.21.15.122:0/769913052 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fcb6c0b1ef0 con 0x7fcb4c0742e0 2024-09-17T05:33:42.455 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:33:42.473 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:33:42.474 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone36743 2024-09-17T05:33:42.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.751+0000 7fa5cb000640 1 -- 172.21.15.122:0/2027609779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5c4155080 msgr2=0x7fa5c40cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.751+0000 7fa5cb000640 1 --2- 172.21.15.122:0/2027609779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5c4155080 0x7fa5c40cd510 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40d0980 tx=0x7fa5b001c800 comp rx=0 tx=0).stop 2024-09-17T05:33:42.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.751+0000 7fa5cb000640 1 -- 172.21.15.122:0/2027609779 shutdown_connections 2024-09-17T05:33:42.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.751+0000 7fa5cb000640 1 --2- 172.21.15.122:0/2027609779 >> v2:172.21.15.173:3300/0 conn(0x7fa5c40cba70 0x7fa5c40cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.751+0000 7fa5cb000640 1 --2- 172.21.15.122:0/2027609779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5c4155080 0x7fa5c40cd510 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.751+0000 7fa5cb000640 1 -- 172.21.15.122:0/2027609779 >> 172.21.15.122:0/2027609779 conn(0x7fa5c406dd30 msgr2=0x7fa5c406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:42.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.751+0000 7fa5cb000640 1 -- 172.21.15.122:0/2027609779 shutdown_connections 2024-09-17T05:33:42.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.751+0000 7fa5cb000640 1 -- 172.21.15.122:0/2027609779 wait complete. 2024-09-17T05:33:42.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5cb000640 1 Processor -- start 2024-09-17T05:33:42.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5cb000640 1 -- start start 2024-09-17T05:33:42.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5cb000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c40cba70 0x7fa5c415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5cb000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5c40cda50 0x7fa5c4159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5cb000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5c4155080 0x7fa5c40c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5cb000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4167cf0 con 0x7fa5c4155080 2024-09-17T05:33:42.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5cb000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4167b70 con 0x7fa5c40cba70 2024-09-17T05:33:42.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5cb000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c4167e70 con 0x7fa5c40cda50 2024-09-17T05:33:42.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5c4155080 0x7fa5c40c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5c4155080 0x7fa5c40c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51694/0 (socket says 172.21.15.122:51694) 2024-09-17T05:33:42.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 -- 172.21.15.122:0/2815620673 learned_addr learned my addr 172.21.15.122:0/2815620673 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:42.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 -- 172.21.15.122:0/2815620673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5c40cda50 msgr2=0x7fa5c4159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 --2- 172.21.15.122:0/2815620673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5c40cda50 0x7fa5c4159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 -- 172.21.15.122:0/2815620673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c40cba70 msgr2=0x7fa5c415c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:42.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 --2- 172.21.15.122:0/2815620673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5c40cba70 0x7fa5c415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:42.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 -- 172.21.15.122:0/2815620673 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c415c6b0 con 0x7fa5c4155080 2024-09-17T05:33:42.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.755+0000 7fa5ca600640 1 --2- 172.21.15.122:0/2815620673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5c4155080 0x7fa5c40c6e20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00b970 tx=0x7fa5bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.759+0000 7fa5c1600640 1 -- 172.21.15.122:0/2815620673 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc019070 con 0x7fa5c4155080 2024-09-17T05:33:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.759+0000 7fa5c1600640 1 -- 172.21.15.122:0/2815620673 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5bc00cc30 con 0x7fa5c4155080 2024-09-17T05:33:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.759+0000 7fa5c1600640 1 -- 172.21.15.122:0/2815620673 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0146a0 con 0x7fa5c4155080 2024-09-17T05:33:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.759+0000 7fa5cb000640 1 -- 172.21.15.122:0/2815620673 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c41fb490 con 0x7fa5c4155080 2024-09-17T05:33:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.759+0000 7fa5cb000640 1 -- 172.21.15.122:0/2815620673 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41fba50 con 0x7fa5c4155080 2024-09-17T05:33:42.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.767+0000 7fa5cb000640 1 -- 172.21.15.122:0/2815620673 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c40cea20 con 0x7fa5c4155080 2024-09-17T05:33:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.771+0000 7fa5c1600640 1 -- 172.21.15.122:0/2815620673 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7fa5bc01c070 con 0x7fa5c4155080 2024-09-17T05:33:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.771+0000 7fa5c1600640 1 --2- 172.21.15.122:0/2815620673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa5a4074330 0x7fa5a40767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:42.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.775+0000 7fa5c1600640 1 -- 172.21.15.122:0/2815620673 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fa5bc05b5a0 con 0x7fa5c4155080 2024-09-17T05:33:42.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.787+0000 7fa5c9c00640 1 --2- 172.21.15.122:0/2815620673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa5a4074330 0x7fa5a40767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:42.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.787+0000 7fa5c1600640 1 -- 172.21.15.122:0/2815620673 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa5bc023020 con 0x7fa5c4155080 2024-09-17T05:33:42.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:42.799+0000 7fa5c9c00640 1 --2- 172.21.15.122:0/2815620673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa5a4074330 0x7fa5a40767f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00af000 tx=0x7fa5b00195a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:43.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.227+0000 7fa5cb000640 1 -- 172.21.15.122:0/2815620673 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone36743", "target": ["mon-mgr", ""]}) -- 0x7fa5c40b0340 con 0x7fa5a4074330 2024-09-17T05:33:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.323+0000 7fa5c1600640 1 -- 172.21.15.122:0/2815620673 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fa5c40b0340 con 0x7fa5a4074330 2024-09-17T05:33:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:33:43.323 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:43.324+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:33:43.338 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:33:43.339 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone66604 2024-09-17T05:33:43.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.587+0000 7f6536800640 1 -- 172.21.15.122:0/3763277130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300cff20 msgr2=0x7f65300d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:43.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.587+0000 7f6536800640 1 --2- 172.21.15.122:0/3763277130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300cff20 0x7f65300d0300 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6520009900 tx=0x7f652001f0d0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.587+0000 7f6536800640 1 -- 172.21.15.122:0/3763277130 shutdown_connections 2024-09-17T05:33:43.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.587+0000 7f6536800640 1 --2- 172.21.15.122:0/3763277130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65300b2d80 0x7f65300d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.587+0000 7f6536800640 1 --2- 172.21.15.122:0/3763277130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300cff20 0x7f65300d0300 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.587+0000 7f6536800640 1 -- 172.21.15.122:0/3763277130 >> 172.21.15.122:0/3763277130 conn(0x7f65300d1dc0 msgr2=0x7f65300d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:43.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.587+0000 7f6536800640 1 -- 172.21.15.122:0/3763277130 shutdown_connections 2024-09-17T05:33:43.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.587+0000 7f6536800640 1 -- 172.21.15.122:0/3763277130 wait complete. 2024-09-17T05:33:43.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.591+0000 7f6536800640 1 Processor -- start 2024-09-17T05:33:43.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.591+0000 7f6536800640 1 -- start start 2024-09-17T05:33:43.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6536800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65300b14c0 0x7f653014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6536800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300b2d80 0x7f653014d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6536800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65300cff20 0x7f653015f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6536800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65300af880 con 0x7f65300b2d80 2024-09-17T05:33:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6536800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f65300af700 con 0x7f65300cff20 2024-09-17T05:33:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6536800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f65300afa00 con 0x7f65300b14c0 2024-09-17T05:33:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300b2d80 0x7f653014d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300b2d80 0x7f653014d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51710/0 (socket says 172.21.15.122:51710) 2024-09-17T05:33:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 -- 172.21.15.122:0/597016277 learned_addr learned my addr 172.21.15.122:0/597016277 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 -- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65300b14c0 msgr2=0x7f653014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65300b14c0 0x7f653014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 -- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65300cff20 msgr2=0x7f653015f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65300cff20 0x7f653015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 -- 172.21.15.122:0/597016277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65301f60c0 con 0x7f65300b2d80 2024-09-17T05:33:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f652ea00640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300b2d80 0x7f653014d780 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f651c00d770 tx=0x7f651c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6527400640 1 -- 172.21.15.122:0/597016277 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c017020 con 0x7f65300b2d80 2024-09-17T05:33:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6527400640 1 -- 172.21.15.122:0/597016277 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f651c009820 con 0x7f65300b2d80 2024-09-17T05:33:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6527400640 1 -- 172.21.15.122:0/597016277 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c009c50 con 0x7f65300b2d80 2024-09-17T05:33:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65301f63a0 con 0x7f65300b2d80 2024-09-17T05:33:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.595+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301f6960 con 0x7f65300b2d80 2024-09-17T05:33:43.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.599+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65301f6d40 con 0x7f65300b2d80 2024-09-17T05:33:43.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.599+0000 7f6527400640 1 -- 172.21.15.122:0/597016277 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f651c01a020 con 0x7f65300b2d80 2024-09-17T05:33:43.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.603+0000 7f6527400640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65100743b0 0x7f6510076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:43.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.603+0000 7f6527400640 1 -- 172.21.15.122:0/597016277 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f651c093830 con 0x7f65300b2d80 2024-09-17T05:33:43.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.607+0000 7f6527400640 1 -- 172.21.15.122:0/597016277 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f65301f6d40 con 0x7f65300b2d80 2024-09-17T05:33:43.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.635+0000 7f652f400640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65100743b0 0x7f6510076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:43.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:43.679+0000 7f652f400640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65100743b0 0x7f6510076870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65200b2000 tx=0x7f652001be70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:44.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:44.019+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone66604", "target": ["mon-mgr", ""]}) -- 0x7f65301f5e70 con 0x7f65100743b0 2024-09-17T05:33:45.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.615+0000 7f6527400640 1 -- 172.21.15.122:0/597016277 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f65301f5e70 con 0x7f65100743b0 2024-09-17T05:33:45.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.619+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65100743b0 msgr2=0x7f6510076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.619+0000 7f6536800640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65100743b0 0x7f6510076870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65200b2000 tx=0x7f652001be70 comp rx=0 tx=0).stop 2024-09-17T05:33:45.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.619+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300b2d80 msgr2=0x7f653014d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.619+0000 7f6536800640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300b2d80 0x7f653014d780 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f651c00d770 tx=0x7f651c00dc30 comp rx=0 tx=0).stop 2024-09-17T05:33:45.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.623+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 shutdown_connections 2024-09-17T05:33:45.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.623+0000 7f6536800640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65100743b0 0x7f6510076870 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.623+0000 7f6536800640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65300cff20 0x7f653015f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.623+0000 7f6536800640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65300b2d80 0x7f653014d780 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.623+0000 7f6536800640 1 --2- 172.21.15.122:0/597016277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65300b14c0 0x7f653014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.623+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 >> 172.21.15.122:0/597016277 conn(0x7f65300d1dc0 msgr2=0x7f65300d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:45.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.623+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 shutdown_connections 2024-09-17T05:33:45.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.623+0000 7f6536800640 1 -- 172.21.15.122:0/597016277 wait complete. 2024-09-17T05:33:45.670 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone34415 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 -- 172.21.15.122:0/4089982277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e0155080 msgr2=0x7f66e00cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 --2- 172.21.15.122:0/4089982277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e0155080 0x7f66e00cd510 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009900 tx=0x7f66d001c880 comp rx=0 tx=0).stop 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 -- 172.21.15.122:0/4089982277 shutdown_connections 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 --2- 172.21.15.122:0/4089982277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00cba70 0x7f66e00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 --2- 172.21.15.122:0/4089982277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e0155080 0x7f66e00cd510 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 -- 172.21.15.122:0/4089982277 >> 172.21.15.122:0/4089982277 conn(0x7f66e006dd30 msgr2=0x7f66e006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 -- 172.21.15.122:0/4089982277 shutdown_connections 2024-09-17T05:33:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 -- 172.21.15.122:0/4089982277 wait complete. 2024-09-17T05:33:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 Processor -- start 2024-09-17T05:33:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.939+0000 7f66e6000640 1 -- start start 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e00cba70 0x7f66e0157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e6000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e00cda50 0x7f66e00b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e6000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e0155080 0x7f66e00b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e6000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0167ae0 con 0x7f66e00cba70 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e6000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0167960 con 0x7f66e00cda50 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e6000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f66e0167c60 con 0x7f66e0155080 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e00cba70 0x7f66e0157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e00cba70 0x7f66e0157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51728/0 (socket says 172.21.15.122:51728) 2024-09-17T05:33:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 -- 172.21.15.122:0/1446008111 learned_addr learned my addr 172.21.15.122:0/1446008111 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 -- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e0155080 msgr2=0x7f66e00b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e0155080 0x7f66e00b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 -- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e00cda50 msgr2=0x7f66e00b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e00cda50 0x7f66e00b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 -- 172.21.15.122:0/1446008111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e00b5ba0 con 0x7f66e00cba70 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e4c00640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e00cba70 0x7f66e0157e90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009850 tx=0x7f66d0004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66dd600640 1 -- 172.21.15.122:0/1446008111 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00b2040 con 0x7f66e00cba70 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66dd600640 1 -- 172.21.15.122:0/1446008111 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66d0005110 con 0x7f66e00cba70 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66dd600640 1 -- 172.21.15.122:0/1446008111 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0005560 con 0x7f66e00cba70 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e01583d0 con 0x7f66e00cba70 2024-09-17T05:33:45.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.943+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e006d790 con 0x7f66e00cba70 2024-09-17T05:33:45.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.947+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e00cea20 con 0x7f66e00cba70 2024-09-17T05:33:45.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.947+0000 7f66dd600640 1 -- 172.21.15.122:0/1446008111 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f66d00046d0 con 0x7f66e00cba70 2024-09-17T05:33:45.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.947+0000 7f66dd600640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66c00742e0 0x7f66c00767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:45.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.947+0000 7f66dd600640 1 -- 172.21.15.122:0/1446008111 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f66d012ccc0 con 0x7f66e00cba70 2024-09-17T05:33:45.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.951+0000 7f66d7e00640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66c00742e0 0x7f66c00767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:45.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.955+0000 7f66dd600640 1 -- 172.21.15.122:0/1446008111 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f66d00bd070 con 0x7f66e00cba70 2024-09-17T05:33:45.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:45.963+0000 7f66d7e00640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66c00742e0 0x7f66c00767a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f66e00b1bd0 tx=0x7f66cc0079f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:46.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.327+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone34415", "target": ["mon-mgr", ""]}) -- 0x7f66e00ce2f0 con 0x7f66c00742e0 2024-09-17T05:33:46.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.827+0000 7f66dd600640 1 -- 172.21.15.122:0/1446008111 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f66e00ce2f0 con 0x7f66c00742e0 2024-09-17T05:33:46.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66c00742e0 msgr2=0x7f66c00767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66c00742e0 0x7f66c00767a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f66e00b1bd0 tx=0x7f66cc0079f0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e00cba70 msgr2=0x7f66e0157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:46.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e00cba70 0x7f66e0157e90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009850 tx=0x7f66d0004220 comp rx=0 tx=0).stop 2024-09-17T05:33:46.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 shutdown_connections 2024-09-17T05:33:46.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66c00742e0 0x7f66c00767a0 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f66e00b1bd0 tx=0x7f66cc0079f0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e0155080 0x7f66e00b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e00cda50 0x7f66e00b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 --2- 172.21.15.122:0/1446008111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66e00cba70 0x7f66e0157e90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:46.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 >> 172.21.15.122:0/1446008111 conn(0x7f66e006dd30 msgr2=0x7f66e006cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:46.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.831+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 shutdown_connections 2024-09-17T05:33:46.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:46.839+0000 7f66e6000640 1 -- 172.21.15.122:0/1446008111 wait complete. 2024-09-17T05:33:46.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone73276 2024-09-17T05:33:47.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.099+0000 7fd130e00640 1 -- 172.21.15.122:0/771749864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd12c070700 msgr2=0x7fd12c15e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.099+0000 7fd130e00640 1 --2- 172.21.15.122:0/771749864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd12c070700 0x7fd12c15e6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009900 tx=0x7fd11c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.099+0000 7fd130e00640 1 -- 172.21.15.122:0/771749864 shutdown_connections 2024-09-17T05:33:47.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.099+0000 7fd130e00640 1 --2- 172.21.15.122:0/771749864 >> v2:172.21.15.173:3300/0 conn(0x7fd12c0d0520 0x7fd12c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.099+0000 7fd130e00640 1 --2- 172.21.15.122:0/771749864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd12c070700 0x7fd12c15e6b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.099+0000 7fd130e00640 1 -- 172.21.15.122:0/771749864 >> 172.21.15.122:0/771749864 conn(0x7fd12c0cfa00 msgr2=0x7fd12c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:47.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.099+0000 7fd130e00640 1 -- 172.21.15.122:0/771749864 shutdown_connections 2024-09-17T05:33:47.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.099+0000 7fd130e00640 1 -- 172.21.15.122:0/771749864 wait complete. 2024-09-17T05:33:47.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd130e00640 1 Processor -- start 2024-09-17T05:33:47.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd130e00640 1 -- start start 2024-09-17T05:33:47.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd130e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd12c070700 0x7fd12c201560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd130e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd12c0cda20 0x7fd12c0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd130e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd12c0d0520 0x7fd12c1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd130e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c0ce910 con 0x7fd12c070700 2024-09-17T05:33:47.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd130e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c0ce790 con 0x7fd12c0cda20 2024-09-17T05:33:47.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd130e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c06f8d0 con 0x7fd12c0d0520 2024-09-17T05:33:47.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd12b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd12c070700 0x7fd12c201560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd12b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd12c070700 0x7fd12c201560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51742/0 (socket says 172.21.15.122:51742) 2024-09-17T05:33:47.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd12b400640 1 -- 172.21.15.122:0/2379543191 learned_addr learned my addr 172.21.15.122:0/2379543191 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:47.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd123e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd12c0cda20 0x7fd12c0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.103+0000 7fd12be00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd12c0d0520 0x7fd12c1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd12b400640 1 -- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd12c0d0520 msgr2=0x7fd12c1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd12b400640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd12c0d0520 0x7fd12c1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd12b400640 1 -- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd12c0cda20 msgr2=0x7fd12c0b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd12b400640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd12c0cda20 0x7fd12c0b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd12b400640 1 -- 172.21.15.122:0/2379543191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c207a90 con 0x7fd12c070700 2024-09-17T05:33:47.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd12b400640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd12c070700 0x7fd12c201560 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0b0040 tx=0x7fd11c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:47.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd123400640 1 -- 172.21.15.122:0/2379543191 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0b2040 con 0x7fd12c070700 2024-09-17T05:33:47.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd123400640 1 -- 172.21.15.122:0/2379543191 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd11c0053e0 con 0x7fd12c070700 2024-09-17T05:33:47.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12c207d70 con 0x7fd12c070700 2024-09-17T05:33:47.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd123400640 1 -- 172.21.15.122:0/2379543191 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0046d0 con 0x7fd12c070700 2024-09-17T05:33:47.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12c208330 con 0x7fd12c070700 2024-09-17T05:33:47.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12c15e6b0 con 0x7fd12c070700 2024-09-17T05:33:47.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.107+0000 7fd123400640 1 -- 172.21.15.122:0/2379543191 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7fd11c0be060 con 0x7fd12c070700 2024-09-17T05:33:47.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.111+0000 7fd123400640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd1080742e0 0x7fd1080767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.111+0000 7fd123400640 1 -- 172.21.15.122:0/2379543191 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fd11c12caa0 con 0x7fd12c070700 2024-09-17T05:33:47.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.111+0000 7fd123e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd1080742e0 0x7fd1080767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.111+0000 7fd123e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd1080742e0 0x7fd1080767a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd12c164df0 tx=0x7fd114010040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:47.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.115+0000 7fd123400640 1 -- 172.21.15.122:0/2379543191 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd11c0164d0 con 0x7fd12c070700 2024-09-17T05:33:47.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.411+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone73276", "target": ["mon-mgr", ""]}) -- 0x7fd12c069890 con 0x7fd1080742e0 2024-09-17T05:33:47.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd123400640 1 -- 172.21.15.122:0/2379543191 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd12c069890 con 0x7fd1080742e0 2024-09-17T05:33:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd1080742e0 msgr2=0x7fd1080767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd1080742e0 0x7fd1080767a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd12c164df0 tx=0x7fd114010040 comp rx=0 tx=0).stop 2024-09-17T05:33:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd12c070700 msgr2=0x7fd12c201560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd12c070700 0x7fd12c201560 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0b0040 tx=0x7fd11c005e70 comp rx=0 tx=0).stop 2024-09-17T05:33:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 shutdown_connections 2024-09-17T05:33:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd1080742e0 0x7fd1080767a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd12c0d0520 0x7fd12c1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd12c0cda20 0x7fd12c0b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 --2- 172.21.15.122:0/2379543191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd12c070700 0x7fd12c201560 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 >> 172.21.15.122:0/2379543191 conn(0x7fd12c0cfa00 msgr2=0x7fd12c156900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 shutdown_connections 2024-09-17T05:33:47.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.607+0000 7fd130e00640 1 -- 172.21.15.122:0/2379543191 wait complete. 2024-09-17T05:33:47.625 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:33:47.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- 172.21.15.122:0/1025990551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b815bea0 msgr2=0x7ff9b80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 --2- 172.21.15.122:0/1025990551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b815bea0 0x7ff9b80ce670 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009900 tx=0x7ff9a801c850 comp rx=0 tx=0).stop 2024-09-17T05:33:47.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- 172.21.15.122:0/1025990551 shutdown_connections 2024-09-17T05:33:47.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 --2- 172.21.15.122:0/1025990551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b815bea0 0x7ff9b80ce670 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- 172.21.15.122:0/1025990551 >> 172.21.15.122:0/1025990551 conn(0x7ff9b80d1dc0 msgr2=0x7ff9b80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:47.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- 172.21.15.122:0/1025990551 shutdown_connections 2024-09-17T05:33:47.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- 172.21.15.122:0/1025990551 wait complete. 2024-09-17T05:33:47.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 Processor -- start 2024-09-17T05:33:47.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- start start 2024-09-17T05:33:47.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b80cdb30 0x7ff9b80c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b80d28e0 0x7ff9b80cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b815bea0 0x7ff9b814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b816a930 con 0x7ff9b80d28e0 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b816a7b0 con 0x7ff9b815bea0 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9be800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b816aab0 con 0x7ff9b80cdb30 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b80d28e0 0x7ff9b80cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b80d28e0 0x7ff9b80cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51772/0 (socket says 172.21.15.122:51772) 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 -- 172.21.15.122:0/2269818866 learned_addr learned my addr 172.21.15.122:0/2269818866 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9b7e00640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b815bea0 0x7ff9b814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9b7400640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b80cdb30 0x7ff9b80c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 -- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b80cdb30 msgr2=0x7ff9b80c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b80cdb30 0x7ff9b80c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 -- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b815bea0 msgr2=0x7ff9b814edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b815bea0 0x7ff9b814edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 -- 172.21.15.122:0/2269818866 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b80cbdc0 con 0x7ff9b80d28e0 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.771+0000 7ff9afe00640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b80d28e0 0x7ff9b80cb880 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c530 tx=0x7ff9a000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9af400640 1 -- 172.21.15.122:0/2269818866 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0017070 con 0x7ff9b80d28e0 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9af400640 1 -- 172.21.15.122:0/2269818866 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9a00053c0 con 0x7ff9b80d28e0 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9af400640 1 -- 172.21.15.122:0/2269818866 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0012890 con 0x7ff9b80d28e0 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b80ca1b0 con 0x7ff9b80d28e0 2024-09-17T05:33:47.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b8158ab0 con 0x7ff9b80d28e0 2024-09-17T05:33:47.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b80ce670 con 0x7ff9b80d28e0 2024-09-17T05:33:47.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9af400640 1 -- 172.21.15.122:0/2269818866 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7ff9a001a070 con 0x7ff9b80d28e0 2024-09-17T05:33:47.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9af400640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff994074310 0x7ff9940767d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:47.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9af400640 1 -- 172.21.15.122:0/2269818866 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7ff9a0092410 con 0x7ff9b80d28e0 2024-09-17T05:33:47.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.775+0000 7ff9b7400640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff994074310 0x7ff9940767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:47.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.779+0000 7ff9b7400640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff994074310 0x7ff9940767d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80af000 tx=0x7ff9a801b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:47.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.779+0000 7ff9af400640 1 -- 172.21.15.122:0/2269818866 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff9b80cbdc0 con 0x7ff9b80d28e0 2024-09-17T05:33:47.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.975+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff9b80d2480 con 0x7ff994074310 2024-09-17T05:33:47.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.975+0000 7ff9af400640 1 -- 172.21.15.122:0/2269818866 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff9b80d2480 con 0x7ff994074310 2024-09-17T05:33:47.973 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:33:47.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:47.973 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:33:47.973 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:47.973 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff994074310 msgr2=0x7ff9940767d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff994074310 0x7ff9940767d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80af000 tx=0x7ff9a801b090 comp rx=0 tx=0).stop 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b80d28e0 msgr2=0x7ff9b80cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b80d28e0 0x7ff9b80cb880 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c530 tx=0x7ff9a000c9f0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 shutdown_connections 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff994074310 0x7ff9940767d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b815bea0 0x7ff9b814edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9b80d28e0 0x7ff9b80cb880 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 --2- 172.21.15.122:0/2269818866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b80cdb30 0x7ff9b80c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 >> 172.21.15.122:0/2269818866 conn(0x7ff9b80d1dc0 msgr2=0x7ff9b80b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:47.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 shutdown_connections 2024-09-17T05:33:47.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:47.979+0000 7ff9be800640 1 -- 172.21.15.122:0/2269818866 wait complete. 2024-09-17T05:33:47.995 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:33:48.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- 172.21.15.122:0/23078850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08600af220 msgr2=0x7f086015ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 --2- 172.21.15.122:0/23078850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08600af220 0x7f086015ef00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0848009840 tx=0x7f084801c790 comp rx=0 tx=0).stop 2024-09-17T05:33:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- 172.21.15.122:0/23078850 shutdown_connections 2024-09-17T05:33:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 --2- 172.21.15.122:0/23078850 >> v2:172.21.15.173:3300/0 conn(0x7f08600cddd0 0x7f08600ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 --2- 172.21.15.122:0/23078850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08600af220 0x7f086015ef00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- 172.21.15.122:0/23078850 >> 172.21.15.122:0/23078850 conn(0x7f08600cfc20 msgr2=0x7f08600d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- 172.21.15.122:0/23078850 shutdown_connections 2024-09-17T05:33:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- 172.21.15.122:0/23078850 wait complete. 2024-09-17T05:33:48.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 Processor -- start 2024-09-17T05:33:48.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- start start 2024-09-17T05:33:48.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f086006fea0 0x7f08601588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08600af220 0x7f08601fa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08600cddd0 0x7f08601faec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0860168360 con 0x7f08600cddd0 2024-09-17T05:33:48.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08601681e0 con 0x7f08600af220 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0866a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f08601684e0 con 0x7f086006fea0 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.143+0000 7f0864a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08600cddd0 0x7f08601faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0864a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08600cddd0 0x7f08601faec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51778/0 (socket says 172.21.15.122:51778) 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0864a00640 1 -- 172.21.15.122:0/2436396483 learned_addr learned my addr 172.21.15.122:0/2436396483 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f085fe00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f086006fea0 0x7f08601588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0857400640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08600af220 0x7f08601fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0864a00640 1 -- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f086006fea0 msgr2=0x7f08601588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0864a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f086006fea0 0x7f08601588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0864a00640 1 -- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08600af220 msgr2=0x7f08601fa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0864a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08600af220 0x7f08601fa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0864a00640 1 -- 172.21.15.122:0/2436396483 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08601fb5c0 con 0x7f08600cddd0 2024-09-17T05:33:48.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0864a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08600cddd0 0x7f08601faec0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f085800efd0 tx=0x7f085800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:48.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f085cc00640 1 -- 172.21.15.122:0/2436396483 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858003b80 con 0x7f08600cddd0 2024-09-17T05:33:48.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f085cc00640 1 -- 172.21.15.122:0/2436396483 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0858003d20 con 0x7f08600cddd0 2024-09-17T05:33:48.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08601fb8a0 con 0x7f08600cddd0 2024-09-17T05:33:48.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f085cc00640 1 -- 172.21.15.122:0/2436396483 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858004280 con 0x7f08600cddd0 2024-09-17T05:33:48.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08601fbe60 con 0x7f08600cddd0 2024-09-17T05:33:48.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08600ca270 con 0x7f08600cddd0 2024-09-17T05:33:48.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.147+0000 7f085cc00640 1 -- 172.21.15.122:0/2436396483 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f0858004420 con 0x7f08600cddd0 2024-09-17T05:33:48.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.151+0000 7f085cc00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f083c0742e0 0x7f083c0767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.151+0000 7f085cc00640 1 -- 172.21.15.122:0/2436396483 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f0858093db0 con 0x7f08600cddd0 2024-09-17T05:33:48.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.151+0000 7f085fe00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f083c0742e0 0x7f083c0767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.151+0000 7f085fe00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f083c0742e0 0x7f083c0767a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08600b5c60 tx=0x7f08500013a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.155+0000 7f085cc00640 1 -- 172.21.15.122:0/2436396483 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f085805fea0 con 0x7f08600cddd0 2024-09-17T05:33:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.343+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08600d0810 con 0x7f083c0742e0 2024-09-17T05:33:48.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.351+0000 7f085cc00640 1 -- 172.21.15.122:0/2436396483 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f08600d0810 con 0x7f083c0742e0 2024-09-17T05:33:48.349 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:48.349 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:33:48.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.349 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sv1" 2024-09-17T05:33:48.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone64781" 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone67490" 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone73276" 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone36743" 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:48.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone66604" 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone77222" 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone34415" 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:48.351 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:33:48.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f083c0742e0 msgr2=0x7f083c0767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f083c0742e0 0x7f083c0767a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08600b5c60 tx=0x7f08500013a0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08600cddd0 msgr2=0x7f08601faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08600cddd0 0x7f08601faec0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f085800efd0 tx=0x7f085800c6a0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 shutdown_connections 2024-09-17T05:33:48.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f083c0742e0 0x7f083c0767a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08600cddd0 0x7f08601faec0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08600af220 0x7f08601fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 --2- 172.21.15.122:0/2436396483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f086006fea0 0x7f08601588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 >> 172.21.15.122:0/2436396483 conn(0x7f08600cfc20 msgr2=0x7f08600b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:48.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 shutdown_connections 2024-09-17T05:33:48.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.355+0000 7f0866a00640 1 -- 172.21.15.122:0/2436396483 wait complete. 2024-09-17T05:33:48.372 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T05:33:48.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.519+0000 7f2cc6c00640 1 -- 172.21.15.122:0/525259666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca4004d50 msgr2=0x7f2ca40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.519+0000 7f2cc6c00640 1 --2- 172.21.15.122:0/525259666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca4004d50 0x7f2ca40051d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009f10 tx=0x7f2cb801d5d0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.519+0000 7f2cc4c00640 1 -- 172.21.15.122:0/525259666 reap_dead start 2024-09-17T05:33:48.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.519+0000 7f2cc6c00640 1 -- 172.21.15.122:0/525259666 shutdown_connections 2024-09-17T05:33:48.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.519+0000 7f2cc6c00640 1 -- 172.21.15.122:0/525259666 >> 172.21.15.122:0/525259666 conn(0x7f2cc00d1da0 msgr2=0x7f2cc00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:48.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.519+0000 7f2cc6c00640 1 -- 172.21.15.122:0/525259666 shutdown_connections 2024-09-17T05:33:48.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.519+0000 7f2cc6c00640 1 -- 172.21.15.122:0/525259666 wait complete. 2024-09-17T05:33:48.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 Processor -- start 2024-09-17T05:33:48.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 -- start start 2024-09-17T05:33:48.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca4004d50 0x7f2cc00b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cc00b79a0 0x7f2cc00ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00caa00 0x7f2cc01f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc00ccd50 con 0x7f2cc00caa00 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc00ccbd0 con 0x7f2ca4004d50 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc00cced0 con 0x7f2cc00b79a0 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00caa00 0x7f2cc01f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cb7400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cc00b79a0 0x7f2cc00ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00caa00 0x7f2cc01f7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51804/0 (socket says 172.21.15.122:51804) 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 -- 172.21.15.122:0/1257375371 learned_addr learned my addr 172.21.15.122:0/1257375371 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cbfe00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca4004d50 0x7f2cc00b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 -- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cc00b79a0 msgr2=0x7f2cc00ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cc00b79a0 0x7f2cc00ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 -- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca4004d50 msgr2=0x7f2cc00b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca4004d50 0x7f2cc00b7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 -- 172.21.15.122:0/1257375371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc00cb050 con 0x7f2cc00caa00 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cb7400640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cc00b79a0 0x7f2cc00ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:48.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc4c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00caa00 0x7f2cc01f7ce0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009f10 tx=0x7f2cb8009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:48.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cbcc00640 1 -- 172.21.15.122:0/1257375371 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8009200 con 0x7f2cc00caa00 2024-09-17T05:33:48.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cbcc00640 1 -- 172.21.15.122:0/1257375371 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb80a9510 con 0x7f2cc00caa00 2024-09-17T05:33:48.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cbcc00640 1 -- 172.21.15.122:0/1257375371 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80a9cf0 con 0x7f2cc00caa00 2024-09-17T05:33:48.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc00cb330 con 0x7f2cc00caa00 2024-09-17T05:33:48.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.523+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc00c5e00 con 0x7f2cc00caa00 2024-09-17T05:33:48.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.527+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc00cb850 con 0x7f2cc00caa00 2024-09-17T05:33:48.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.527+0000 7f2cbcc00640 1 -- 172.21.15.122:0/1257375371 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f2cb80b9030 con 0x7f2cc00caa00 2024-09-17T05:33:48.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.527+0000 7f2cbcc00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c0742e0 0x7f2c9c0767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.527+0000 7f2cbcc00640 1 -- 172.21.15.122:0/1257375371 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f2cb812dc00 con 0x7f2cc00caa00 2024-09-17T05:33:48.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.527+0000 7f2cbfe00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c0742e0 0x7f2c9c0767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.527+0000 7f2cbfe00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c0742e0 0x7f2c9c0767a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009e60 tx=0x7f2cb00091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:48.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.531+0000 7f2cbcc00640 1 -- 172.21.15.122:0/1257375371 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2cc00cb850 con 0x7f2cc00caa00 2024-09-17T05:33:48.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.727+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cc00d2a70 con 0x7f2c9c0742e0 2024-09-17T05:33:48.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.743+0000 7f2cbcc00640 1 -- 172.21.15.122:0/1257375371 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2cc00d2a70 con 0x7f2c9c0742e0 2024-09-17T05:33:48.740 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:48.741 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:33:48.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:33:48.741 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1" 2024-09-17T05:33:48.741 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:33:48.741 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c0742e0 msgr2=0x7f2c9c0767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c0742e0 0x7f2c9c0767a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009e60 tx=0x7f2cb00091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00caa00 msgr2=0x7f2cc01f7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00caa00 0x7f2cc01f7ce0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009f10 tx=0x7f2cb8009480 comp rx=0 tx=0).stop 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 shutdown_connections 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c9c0742e0 0x7f2c9c0767a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cc00caa00 0x7f2cc01f7ce0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cc00b79a0 0x7f2cc00ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 --2- 172.21.15.122:0/1257375371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca4004d50 0x7f2cc00b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 >> 172.21.15.122:0/1257375371 conn(0x7f2cc00d1da0 msgr2=0x7f2cc00715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:48.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 shutdown_connections 2024-09-17T05:33:48.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.747+0000 7f2cc6c00640 1 -- 172.21.15.122:0/1257375371 wait complete. 2024-09-17T05:33:48.761 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T05:33:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.911+0000 7f122d400640 1 -- 172.21.15.122:0/890423220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280b2e50 msgr2=0x7f122806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.911+0000 7f122d400640 1 --2- 172.21.15.122:0/890423220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280b2e50 0x7f122806f8f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f12280d06f0 tx=0x7f122001cb70 comp rx=0 tx=0).stop 2024-09-17T05:33:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.911+0000 7f122d400640 1 -- 172.21.15.122:0/890423220 shutdown_connections 2024-09-17T05:33:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.911+0000 7f122d400640 1 --2- 172.21.15.122:0/890423220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280b2e50 0x7f122806f8f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.911+0000 7f122d400640 1 --2- 172.21.15.122:0/890423220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122815ac70 0x7f12280cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.911+0000 7f122d400640 1 -- 172.21.15.122:0/890423220 >> 172.21.15.122:0/890423220 conn(0x7f12280cd740 msgr2=0x7f12280cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.911+0000 7f122d400640 1 -- 172.21.15.122:0/890423220 shutdown_connections 2024-09-17T05:33:48.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.911+0000 7f122d400640 1 -- 172.21.15.122:0/890423220 wait complete. 2024-09-17T05:33:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 Processor -- start 2024-09-17T05:33:48.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 -- start start 2024-09-17T05:33:48.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12280b2e50 0x7f1228154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280cbdb0 0x7f12280c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122815ac70 0x7f12280c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1228167b50 con 0x7f12280cbdb0 2024-09-17T05:33:48.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f12281679d0 con 0x7f12280b2e50 2024-09-17T05:33:48.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1228167cd0 con 0x7f122815ac70 2024-09-17T05:33:48.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280cbdb0 0x7f12280c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280cbdb0 0x7f12280c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51816/0 (socket says 172.21.15.122:51816) 2024-09-17T05:33:48.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 -- 172.21.15.122:0/3182540141 learned_addr learned my addr 172.21.15.122:0/3182540141 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:48.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f1226800640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12280b2e50 0x7f1228154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f1227200640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122815ac70 0x7f12280c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 -- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122815ac70 msgr2=0x7f12280c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122815ac70 0x7f12280c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 -- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12280b2e50 msgr2=0x7f1228154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12280b2e50 0x7f1228154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:48.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 -- 172.21.15.122:0/3182540141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12280c8110 con 0x7f12280cbdb0 2024-09-17T05:33:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121de00640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280cbdb0 0x7f12280c6bb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f121000c430 tx=0x7f121000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121f400640 1 -- 172.21.15.122:0/3182540141 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210017070 con 0x7f12280cbdb0 2024-09-17T05:33:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121f400640 1 -- 172.21.15.122:0/3182540141 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1210003600 con 0x7f12280cbdb0 2024-09-17T05:33:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12280c83f0 con 0x7f12280cbdb0 2024-09-17T05:33:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f121f400640 1 -- 172.21.15.122:0/3182540141 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210003a50 con 0x7f12280cbdb0 2024-09-17T05:33:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.915+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12280cacf0 con 0x7f12280cbdb0 2024-09-17T05:33:48.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.919+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122806f8f0 con 0x7f12280cbdb0 2024-09-17T05:33:48.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.919+0000 7f121f400640 1 -- 172.21.15.122:0/3182540141 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f121001a070 con 0x7f12280cbdb0 2024-09-17T05:33:48.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.919+0000 7f121f400640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1204074310 0x7f12040767d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:48.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.919+0000 7f121f400640 1 -- 172.21.15.122:0/3182540141 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f12100922e0 con 0x7f12280cbdb0 2024-09-17T05:33:48.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.919+0000 7f1226800640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1204074310 0x7f12040767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:48.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.923+0000 7f1226800640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1204074310 0x7f12040767d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f121800bb50 tx=0x7f121800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:48.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:48.923+0000 7f121f400640 1 -- 172.21.15.122:0/3182540141 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f121009c080 con 0x7f12280cbdb0 2024-09-17T05:33:49.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.127+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12280c79c0 con 0x7f1204074310 2024-09-17T05:33:49.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.151+0000 7f121f400640 1 -- 172.21.15.122:0/3182540141 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12280c79c0 con 0x7f1204074310 2024-09-17T05:33:49.149 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1204074310 msgr2=0x7f12040767d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1204074310 0x7f12040767d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f121800bb50 tx=0x7f121800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280cbdb0 msgr2=0x7f12280c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280cbdb0 0x7f12280c6bb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f121000c430 tx=0x7f121000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 shutdown_connections 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1204074310 0x7f12040767d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f122815ac70 0x7f12280c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12280cbdb0 0x7f12280c6bb0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 --2- 172.21.15.122:0/3182540141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12280b2e50 0x7f1228154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 >> 172.21.15.122:0/3182540141 conn(0x7f12280cd740 msgr2=0x7f122815b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 shutdown_connections 2024-09-17T05:33:49.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.155+0000 7f122d400640 1 -- 172.21.15.122:0/3182540141 wait complete. 2024-09-17T05:33:49.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T05:33:49.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.319+0000 7f63cd400640 1 -- 172.21.15.122:0/1293436865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c815db50 msgr2=0x7f63c80acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.319+0000 7f63cd400640 1 --2- 172.21.15.122:0/1293436865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c815db50 0x7f63c80acd40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f63b800a950 tx=0x7f63b801dae0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.319+0000 7f63cd400640 1 -- 172.21.15.122:0/1293436865 shutdown_connections 2024-09-17T05:33:49.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.319+0000 7f63cd400640 1 --2- 172.21.15.122:0/1293436865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63c80b4a50 0x7f63c80ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.319+0000 7f63cd400640 1 --2- 172.21.15.122:0/1293436865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c815db50 0x7f63c80acd40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.319+0000 7f63cd400640 1 -- 172.21.15.122:0/1293436865 >> 172.21.15.122:0/1293436865 conn(0x7f63c80b18f0 msgr2=0x7f63c80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:49.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.319+0000 7f63cd400640 1 -- 172.21.15.122:0/1293436865 shutdown_connections 2024-09-17T05:33:49.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.319+0000 7f63cd400640 1 -- 172.21.15.122:0/1293436865 wait complete. 2024-09-17T05:33:49.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 Processor -- start 2024-09-17T05:33:49.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 -- start start 2024-09-17T05:33:49.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c80b4a50 0x7f63c81523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:49.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63c80cd100 0x7f63c815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:49.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63c815db50 0x7f63c815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:49.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8167120 con 0x7f63c80b4a50 2024-09-17T05:33:49.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f63c80cf550 con 0x7f63c815db50 2024-09-17T05:33:49.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f63c81672a0 con 0x7f63c80cd100 2024-09-17T05:33:49.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c80b4a50 0x7f63c81523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c80b4a50 0x7f63c81523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51840/0 (socket says 172.21.15.122:51840) 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 -- 172.21.15.122:0/3584731501 learned_addr learned my addr 172.21.15.122:0/3584731501 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c7200640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63c815db50 0x7f63c815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c5e00640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63c80cd100 0x7f63c815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 -- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63c80cd100 msgr2=0x7f63c815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63c80cd100 0x7f63c815c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 -- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63c815db50 msgr2=0x7f63c815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63c815db50 0x7f63c815c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 -- 172.21.15.122:0/3584731501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c81631f0 con 0x7f63c80b4a50 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c7200640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63c815db50 0x7f63c815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63c6800640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c80b4a50 0x7f63c81523e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f63b800a8a0 tx=0x7f63b8004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63bea00640 1 -- 172.21.15.122:0/3584731501 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80bb020 con 0x7f63c80b4a50 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63bea00640 1 -- 172.21.15.122:0/3584731501 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63b8005320 con 0x7f63c80b4a50 2024-09-17T05:33:49.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63bea00640 1 -- 172.21.15.122:0/3584731501 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80b3040 con 0x7f63c80b4a50 2024-09-17T05:33:49.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c81fb4b0 con 0x7f63c80b4a50 2024-09-17T05:33:49.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81fba70 con 0x7f63c80b4a50 2024-09-17T05:33:49.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.323+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c80ce810 con 0x7f63c80b4a50 2024-09-17T05:33:49.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.327+0000 7f63bea00640 1 -- 172.21.15.122:0/3584731501 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f63b80046d0 con 0x7f63c80b4a50 2024-09-17T05:33:49.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.327+0000 7f63bea00640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63a80742e0 0x7f63a80767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:49.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.327+0000 7f63bea00640 1 -- 172.21.15.122:0/3584731501 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f63b812d310 con 0x7f63c80b4a50 2024-09-17T05:33:49.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.327+0000 7f63c5e00640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63a80742e0 0x7f63a80767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:49.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.327+0000 7f63c5e00640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63a80742e0 0x7f63a80767a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f63c809d9a0 tx=0x7f63b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:49.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.331+0000 7f63bea00640 1 -- 172.21.15.122:0/3584731501 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f63b80c5060 con 0x7f63c80b4a50 2024-09-17T05:33:49.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.523+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f63c81598b0 con 0x7f63a80742e0 2024-09-17T05:33:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.543+0000 7f63bea00640 1 -- 172.21.15.122:0/3584731501 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63c81598b0 con 0x7f63a80742e0 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63a80742e0 msgr2=0x7f63a80767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63a80742e0 0x7f63a80767a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f63c809d9a0 tx=0x7f63b4009040 comp rx=0 tx=0).stop 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c80b4a50 msgr2=0x7f63c81523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c80b4a50 0x7f63c81523e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f63b800a8a0 tx=0x7f63b8004620 comp rx=0 tx=0).stop 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 shutdown_connections 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63a80742e0 0x7f63a80767a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63c815db50 0x7f63c815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63c80cd100 0x7f63c815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 --2- 172.21.15.122:0/3584731501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63c80b4a50 0x7f63c81523e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 >> 172.21.15.122:0/3584731501 conn(0x7f63c80b18f0 msgr2=0x7f63c80b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:49.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 shutdown_connections 2024-09-17T05:33:49.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.547+0000 7f63cd400640 1 -- 172.21.15.122:0/3584731501 wait complete. 2024-09-17T05:33:49.553 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:49.555+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:49.553 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:49.555+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:49.553 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:49.555+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:49.553 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:49.555+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:49.553 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:33:49.555+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:33:49.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone64781 2024-09-17T05:33:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- 172.21.15.122:0/4098780175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b80d20a0 msgr2=0x7f26b80d2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 --2- 172.21.15.122:0/4098780175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b80d20a0 0x7f26b80d2540 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f26b8162ca0 tx=0x7f26b001cd50 comp rx=0 tx=0).stop 2024-09-17T05:33:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- 172.21.15.122:0/4098780175 shutdown_connections 2024-09-17T05:33:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 --2- 172.21.15.122:0/4098780175 >> v2:172.21.15.173:3300/0 conn(0x7f26b815e220 0x7f26b80d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 --2- 172.21.15.122:0/4098780175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b80d20a0 0x7f26b80d2540 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 --2- 172.21.15.122:0/4098780175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b815de40 0x7f26b80b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- 172.21.15.122:0/4098780175 >> 172.21.15.122:0/4098780175 conn(0x7f26b80d3f90 msgr2=0x7f26b80d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- 172.21.15.122:0/4098780175 shutdown_connections 2024-09-17T05:33:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- 172.21.15.122:0/4098780175 wait complete. 2024-09-17T05:33:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 Processor -- start 2024-09-17T05:33:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- start start 2024-09-17T05:33:49.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26b80d20a0 0x7f26b80caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:49.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b815de40 0x7f26b80cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:49.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b815e220 0x7f26b80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:49.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8161f70 con 0x7f26b815e220 2024-09-17T05:33:49.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f26b8161df0 con 0x7f26b80d20a0 2024-09-17T05:33:49.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26be000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f26b81620f0 con 0x7f26b815de40 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.719+0000 7f26b7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b815e220 0x7f26b80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b815e220 0x7f26b80cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51858/0 (socket says 172.21.15.122:51858) 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7e00640 1 -- 172.21.15.122:0/3634931970 learned_addr learned my addr 172.21.15.122:0/3634931970 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7400640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26b80d20a0 0x7f26b80caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26afe00640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b815de40 0x7f26b80cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7e00640 1 -- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b815de40 msgr2=0x7f26b80cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7e00640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b815de40 0x7f26b80cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7e00640 1 -- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26b80d20a0 msgr2=0x7f26b80caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7e00640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26b80d20a0 0x7f26b80caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7e00640 1 -- 172.21.15.122:0/3634931970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b81f8f00 con 0x7f26b815e220 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26b7e00640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b815e220 0x7f26b80cb5a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f26b001b400 tx=0x7f26b0009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26af400640 1 -- 172.21.15.122:0/3634931970 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b00bb070 con 0x7f26b815e220 2024-09-17T05:33:49.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26af400640 1 -- 172.21.15.122:0/3634931970 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26b0004d40 con 0x7f26b815e220 2024-09-17T05:33:49.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26af400640 1 -- 172.21.15.122:0/3634931970 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b00b2040 con 0x7f26b815e220 2024-09-17T05:33:49.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b81fb900 con 0x7f26b815e220 2024-09-17T05:33:49.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b81fbec0 con 0x7f26b815e220 2024-09-17T05:33:49.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b81fb710 con 0x7f26b815e220 2024-09-17T05:33:49.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.723+0000 7f26af400640 1 -- 172.21.15.122:0/3634931970 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f26b0004200 con 0x7f26b815e220 2024-09-17T05:33:49.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.727+0000 7f26af400640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26940742e0 0x7f26940767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:49.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.727+0000 7f26af400640 1 -- 172.21.15.122:0/3634931970 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f26b012ca80 con 0x7f26b815e220 2024-09-17T05:33:49.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.727+0000 7f26b7400640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26940742e0 0x7f26940767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:49.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.727+0000 7f26b7400640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26940742e0 0x7f26940767a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f26a8004630 tx=0x7f26a800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:49.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.731+0000 7f26af400640 1 -- 172.21.15.122:0/3634931970 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f26b0016970 con 0x7f26b815e220 2024-09-17T05:33:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.931+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone64781", "target": ["mon-mgr", ""]}) -- 0x7f26b80b35a0 con 0x7f26940742e0 2024-09-17T05:33:49.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26af400640 1 -- 172.21.15.122:0/3634931970 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26b80b35a0 con 0x7f26940742e0 2024-09-17T05:33:49.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26940742e0 msgr2=0x7f26940767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26940742e0 0x7f26940767a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f26a8004630 tx=0x7f26a800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b815e220 msgr2=0x7f26b80cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:49.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b815e220 0x7f26b80cb5a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f26b001b400 tx=0x7f26b0009370 comp rx=0 tx=0).stop 2024-09-17T05:33:49.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 shutdown_connections 2024-09-17T05:33:49.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26940742e0 0x7f26940767a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26b815e220 0x7f26b80cb5a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b815de40 0x7f26b80cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 --2- 172.21.15.122:0/3634931970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f26b80d20a0 0x7f26b80caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:49.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 >> 172.21.15.122:0/3634931970 conn(0x7f26b80d3f90 msgr2=0x7f26b80b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:49.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 shutdown_connections 2024-09-17T05:33:49.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:49.955+0000 7f26be000640 1 -- 172.21.15.122:0/3634931970 wait complete. 2024-09-17T05:33:49.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone67490 2024-09-17T05:33:50.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- 172.21.15.122:0/2214766203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c0ce890 msgr2=0x7f428c06eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 --2- 172.21.15.122:0/2214766203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c0ce890 0x7f428c06eb40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f428400ad90 tx=0x7f428401dcb0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- 172.21.15.122:0/2214766203 shutdown_connections 2024-09-17T05:33:50.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 --2- 172.21.15.122:0/2214766203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c0ce890 0x7f428c06eb40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 --2- 172.21.15.122:0/2214766203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f428c15a9d0 0x7f428c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- 172.21.15.122:0/2214766203 >> 172.21.15.122:0/2214766203 conn(0x7f428c0cfbc0 msgr2=0x7f428c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:50.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- 172.21.15.122:0/2214766203 shutdown_connections 2024-09-17T05:33:50.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- 172.21.15.122:0/2214766203 wait complete. 2024-09-17T05:33:50.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 Processor -- start 2024-09-17T05:33:50.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- start start 2024-09-17T05:33:50.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f428c0734f0 0x7f428c1648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f428c0ce890 0x7f428c09d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c15a9d0 0x7f428c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1674e0 con 0x7f428c15a9d0 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f428c167360 con 0x7f428c0734f0 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4291400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f428c167660 con 0x7f428c0ce890 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c15a9d0 0x7f428c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c15a9d0 0x7f428c159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51872/0 (socket says 172.21.15.122:51872) 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 -- 172.21.15.122:0/3338270880 learned_addr learned my addr 172.21.15.122:0/3338270880 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f4281e00640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f428c0ce890 0x7f428c09d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428a800640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f428c0734f0 0x7f428c1648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 -- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f428c0ce890 msgr2=0x7f428c09d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f428c0ce890 0x7f428c09d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 -- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f428c0734f0 msgr2=0x7f428c1648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f428c0734f0 0x7f428c1648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 -- 172.21.15.122:0/3338270880 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c164e30 con 0x7f428c15a9d0 2024-09-17T05:33:50.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.127+0000 7f428b200640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c15a9d0 0x7f428c159480 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f428400ace0 tx=0x7f4284004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:50.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4283400640 1 -- 172.21.15.122:0/3338270880 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840b3040 con 0x7f428c15a9d0 2024-09-17T05:33:50.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4283400640 1 -- 172.21.15.122:0/3338270880 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4284004b40 con 0x7f428c15a9d0 2024-09-17T05:33:50.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4283400640 1 -- 172.21.15.122:0/3338270880 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284004460 con 0x7f428c15a9d0 2024-09-17T05:33:50.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c0ad9a0 con 0x7f428c15a9d0 2024-09-17T05:33:50.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c0adf60 con 0x7f428c15a9d0 2024-09-17T05:33:50.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c069410 con 0x7f428c15a9d0 2024-09-17T05:33:50.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4283400640 1 -- 172.21.15.122:0/3338270880 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f42840bf060 con 0x7f428c15a9d0 2024-09-17T05:33:50.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4283400640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42680742e0 0x7f42680767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:50.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f4283400640 1 -- 172.21.15.122:0/3338270880 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f428412d130 con 0x7f428c15a9d0 2024-09-17T05:33:50.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.131+0000 7f428a800640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42680742e0 0x7f42680767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:50.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.135+0000 7f428a800640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42680742e0 0x7f42680767a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f427c007fd0 tx=0x7f427c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:50.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.135+0000 7f4283400640 1 -- 172.21.15.122:0/3338270880 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f428c164e30 con 0x7f428c15a9d0 2024-09-17T05:33:50.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.331+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone67490", "target": ["mon-mgr", ""]}) -- 0x7f428c0ad750 con 0x7f42680742e0 2024-09-17T05:33:50.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.347+0000 7f4283400640 1 -- 172.21.15.122:0/3338270880 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f428c0ad750 con 0x7f42680742e0 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42680742e0 msgr2=0x7f42680767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42680742e0 0x7f42680767a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f427c007fd0 tx=0x7f427c009040 comp rx=0 tx=0).stop 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c15a9d0 msgr2=0x7f428c159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c15a9d0 0x7f428c159480 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f428400ace0 tx=0x7f4284004270 comp rx=0 tx=0).stop 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 shutdown_connections 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42680742e0 0x7f42680767a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f428c15a9d0 0x7f428c159480 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f428c0ce890 0x7f428c09d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 --2- 172.21.15.122:0/3338270880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f428c0734f0 0x7f428c1648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 >> 172.21.15.122:0/3338270880 conn(0x7f428c0cfbc0 msgr2=0x7f428c15bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 shutdown_connections 2024-09-17T05:33:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.351+0000 7f4291400640 1 -- 172.21.15.122:0/3338270880 wait complete. 2024-09-17T05:33:50.373 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone73276 2024-09-17T05:33:50.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- 172.21.15.122:0/930148835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 msgr2=0x7fca840ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 --2- 172.21.15.122:0/930148835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 0x7fca840ce700 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fca74009840 tx=0x7fca7401c820 comp rx=0 tx=0).stop 2024-09-17T05:33:50.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- 172.21.15.122:0/930148835 shutdown_connections 2024-09-17T05:33:50.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 --2- 172.21.15.122:0/930148835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca840b0b60 0x7fca840ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 --2- 172.21.15.122:0/930148835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 0x7fca840ce700 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- 172.21.15.122:0/930148835 >> 172.21.15.122:0/930148835 conn(0x7fca840d1da0 msgr2=0x7fca840d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:50.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- 172.21.15.122:0/930148835 shutdown_connections 2024-09-17T05:33:50.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- 172.21.15.122:0/930148835 wait complete. 2024-09-17T05:33:50.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 Processor -- start 2024-09-17T05:33:50.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- start start 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca840b0b60 0x7fca84152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 0x7fca8414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca8415c1f0 0x7fca841fa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca84169040 con 0x7fca8415be10 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fca84168ec0 con 0x7fca8415c1f0 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca8b000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fca841691c0 con 0x7fca840b0b60 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca7b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 0x7fca8414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca7b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 0x7fca8414ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51884/0 (socket says 172.21.15.122:51884) 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca7b400640 1 -- 172.21.15.122:0/2966712561 learned_addr learned my addr 172.21.15.122:0/2966712561 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca83e00640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca840b0b60 0x7fca84152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.523+0000 7fca89000640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca8415c1f0 0x7fca841fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca7b400640 1 -- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca840b0b60 msgr2=0x7fca84152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca7b400640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca840b0b60 0x7fca84152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca7b400640 1 -- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca8415c1f0 msgr2=0x7fca841fa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca7b400640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca8415c1f0 0x7fca841fa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca7b400640 1 -- 172.21.15.122:0/2966712561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8414f520 con 0x7fca8415be10 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca7b400640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 0x7fca8414ce90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fca6c001ec0 tx=0x7fca6c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca80c00640 1 -- 172.21.15.122:0/2966712561 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c017070 con 0x7fca8415be10 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca80c00640 1 -- 172.21.15.122:0/2966712561 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca6c003600 con 0x7fca8415be10 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca84153180 con 0x7fca8415be10 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca80c00640 1 -- 172.21.15.122:0/2966712561 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c003a50 con 0x7fca8415be10 2024-09-17T05:33:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca84153740 con 0x7fca8415be10 2024-09-17T05:33:50.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca840ccbd0 con 0x7fca8415be10 2024-09-17T05:33:50.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.527+0000 7fca80c00640 1 -- 172.21.15.122:0/2966712561 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7fca6c01a070 con 0x7fca8415be10 2024-09-17T05:33:50.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.531+0000 7fca80c00640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca600742e0 0x7fca600767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:50.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.531+0000 7fca80c00640 1 -- 172.21.15.122:0/2966712561 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fca6c092310 con 0x7fca8415be10 2024-09-17T05:33:50.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.531+0000 7fca83e00640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca600742e0 0x7fca600767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:50.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.531+0000 7fca83e00640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca600742e0 0x7fca600767a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fca740af000 tx=0x7fca7401b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:50.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.535+0000 7fca80c00640 1 -- 172.21.15.122:0/2966712561 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fca6c05e480 con 0x7fca8415be10 2024-09-17T05:33:50.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.727+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone73276", "target": ["mon-mgr", ""]}) -- 0x7fca84070350 con 0x7fca600742e0 2024-09-17T05:33:50.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca80c00640 1 -- 172.21.15.122:0/2966712561 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca84070350 con 0x7fca600742e0 2024-09-17T05:33:50.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca600742e0 msgr2=0x7fca600767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca600742e0 0x7fca600767a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fca740af000 tx=0x7fca7401b060 comp rx=0 tx=0).stop 2024-09-17T05:33:50.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 msgr2=0x7fca8414ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:50.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 0x7fca8414ce90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fca6c001ec0 tx=0x7fca6c00c890 comp rx=0 tx=0).stop 2024-09-17T05:33:50.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 shutdown_connections 2024-09-17T05:33:50.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca600742e0 0x7fca600767a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca8415c1f0 0x7fca841fa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca8415be10 0x7fca8414ce90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 --2- 172.21.15.122:0/2966712561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca840b0b60 0x7fca84152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:50.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 >> 172.21.15.122:0/2966712561 conn(0x7fca840d1da0 msgr2=0x7fca840d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:50.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.839+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 shutdown_connections 2024-09-17T05:33:50.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:50.843+0000 7fca8b000640 1 -- 172.21.15.122:0/2966712561 wait complete. 2024-09-17T05:33:50.859 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone36743 2024-09-17T05:33:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 -- 172.21.15.122:0/3903250579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c0b0b60 msgr2=0x7f800c0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 --2- 172.21.15.122:0/3903250579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c0b0b60 0x7f800c0ccbd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8008009bc0 tx=0x7f800801f3b0 comp rx=0 tx=0).stop 2024-09-17T05:33:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 -- 172.21.15.122:0/3903250579 shutdown_connections 2024-09-17T05:33:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 --2- 172.21.15.122:0/3903250579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c0b0b60 0x7f800c0ccbd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 --2- 172.21.15.122:0/3903250579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f800c15c1f0 0x7f800c0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 -- 172.21.15.122:0/3903250579 >> 172.21.15.122:0/3903250579 conn(0x7f800c0d1da0 msgr2=0x7f800c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 -- 172.21.15.122:0/3903250579 shutdown_connections 2024-09-17T05:33:51.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 -- 172.21.15.122:0/3903250579 wait complete. 2024-09-17T05:33:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 Processor -- start 2024-09-17T05:33:51.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.011+0000 7f8014800640 1 -- start start 2024-09-17T05:33:51.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c0b0b60 0x7f800c1f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:51.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c15be10 0x7f800c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f800c15c1f0 0x7f800c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f800c0cd320 con 0x7f800c15be10 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f800c0cf590 con 0x7f800c0b0b60 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f800c0cd4a0 con 0x7f800c15c1f0 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c15be10 0x7f800c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c15be10 0x7f800c06ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51914/0 (socket says 172.21.15.122:51914) 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 -- 172.21.15.122:0/1808255000 learned_addr learned my addr 172.21.15.122:0/1808255000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8012800640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f800c15c1f0 0x7f800c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011e00640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c0b0b60 0x7f800c1f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 -- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f800c15c1f0 msgr2=0x7f800c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f800c15c1f0 0x7f800c06f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 -- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c0b0b60 msgr2=0x7f800c1f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c0b0b60 0x7f800c1f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 -- 172.21.15.122:0/1808255000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c15f030 con 0x7f800c15be10 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8012800640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f800c15c1f0 0x7f800c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011e00640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c0b0b60 0x7f800c1f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8011400640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c15be10 0x7f800c06ed80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00d950 tx=0x7f7ffc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8006000640 1 -- 172.21.15.122:0/1808255000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc017070 con 0x7f800c15be10 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8006000640 1 -- 172.21.15.122:0/1808255000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ffc009d20 con 0x7f800c15be10 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8006000640 1 -- 172.21.15.122:0/1808255000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0106c0 con 0x7f800c15be10 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c15f310 con 0x7f800c15be10 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c0c93a0 con 0x7f800c15be10 2024-09-17T05:33:51.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c0ccbd0 con 0x7f800c15be10 2024-09-17T05:33:51.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.015+0000 7f8006000640 1 -- 172.21.15.122:0/1808255000 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 31) ==== 98764+0+0 (secure 0 0 0) 0x7f7ffc01a070 con 0x7f800c15be10 2024-09-17T05:33:51.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.019+0000 7f8006000640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ff00742e0 0x7f7ff00767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:51.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.019+0000 7f8006000640 1 -- 172.21.15.122:0/1808255000 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7f7ffc093250 con 0x7f800c15be10 2024-09-17T05:33:51.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.019+0000 7f8011e00640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ff00742e0 0x7f7ff00767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:51.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.019+0000 7f8011e00640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ff00742e0 0x7f7ff00767a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8000000f70 tx=0x7f8000009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:51.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.023+0000 7f8006000640 1 -- 172.21.15.122:0/1808255000 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7ffc05fa10 con 0x7f800c15be10 2024-09-17T05:33:51.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.067+0000 7f8006000640 1 -- 172.21.15.122:0/1808255000 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f7ffc05b580 con 0x7f800c15be10 2024-09-17T05:33:51.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:51.219+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone36743", "target": ["mon-mgr", ""]}) -- 0x7f800c0cb890 con 0x7f7ff00742e0 2024-09-17T05:33:52.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.295+0000 7f8006000640 1 -- 172.21.15.122:0/1808255000 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8000001d10 con 0x7f7ff00742e0 2024-09-17T05:33:52.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ff00742e0 msgr2=0x7f7ff00767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ff00742e0 0x7f7ff00767a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8000000f70 tx=0x7f8000009040 comp rx=0 tx=0).stop 2024-09-17T05:33:52.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c15be10 msgr2=0x7f800c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c15be10 0x7f800c06ed80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00d950 tx=0x7f7ffc00de10 comp rx=0 tx=0).stop 2024-09-17T05:33:52.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 shutdown_connections 2024-09-17T05:33:52.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ff00742e0 0x7f7ff00767a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f800c15c1f0 0x7f800c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f800c15be10 0x7f800c06ed80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 --2- 172.21.15.122:0/1808255000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c0b0b60 0x7f800c1f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.299+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 >> 172.21.15.122:0/1808255000 conn(0x7f800c0d1da0 msgr2=0x7f800c070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:52.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.303+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 shutdown_connections 2024-09-17T05:33:52.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.303+0000 7f8014800640 1 -- 172.21.15.122:0/1808255000 wait complete. 2024-09-17T05:33:52.339 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone66604 2024-09-17T05:33:52.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.603+0000 7fe1a2600640 1 -- 172.21.15.122:0/528610259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe180004d50 msgr2=0x7fe1800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.603+0000 7fe19b400640 1 -- 172.21.15.122:0/528610259 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c006c80 con 0x7fe180004d50 2024-09-17T05:33:52.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.603+0000 7fe1a2600640 1 --2- 172.21.15.122:0/528610259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe180004d50 0x7fe1800051d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c2e0 tx=0x7fe18c01f720 comp rx=0 tx=0).stop 2024-09-17T05:33:52.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.603+0000 7fe1a1c00640 1 -- 172.21.15.122:0/528610259 reap_dead start 2024-09-17T05:33:52.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.603+0000 7fe1a2600640 1 -- 172.21.15.122:0/528610259 shutdown_connections 2024-09-17T05:33:52.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.603+0000 7fe1a2600640 1 -- 172.21.15.122:0/528610259 >> 172.21.15.122:0/528610259 conn(0x7fe19c0d3f50 msgr2=0x7fe19c0d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:52.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.603+0000 7fe1a2600640 1 -- 172.21.15.122:0/528610259 shutdown_connections 2024-09-17T05:33:52.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.603+0000 7fe1a2600640 1 -- 172.21.15.122:0/528610259 wait complete. 2024-09-17T05:33:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a2600640 1 Processor -- start 2024-09-17T05:33:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a2600640 1 -- start start 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a2600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe19c1f3840 0x7fe19c0ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c153d80 0x7fe19c1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a2600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe19c0cced0 0x7fe19c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a2600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c0d1000 con 0x7fe19c153d80 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a2600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c0d0e80 con 0x7fe19c1f3840 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a2600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c1630f0 con 0x7fe19c0cced0 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe19be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c153d80 0x7fe19c1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe19be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c153d80 0x7fe19c1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46942/0 (socket says 172.21.15.122:46942) 2024-09-17T05:33:52.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe19be00640 1 -- 172.21.15.122:0/4072530811 learned_addr learned my addr 172.21.15.122:0/4072530811 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.607+0000 7fe1a1c00640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe19c0cced0 0x7fe19c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a1c00640 1 -- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe19c1f3840 msgr2=0x7fe19c0ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a1c00640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe19c1f3840 0x7fe19c0ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a1c00640 1 -- 172.21.15.122:0/4072530811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c153d80 msgr2=0x7fe19c1512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a1c00640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c153d80 0x7fe19c1512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a1c00640 1 -- 172.21.15.122:0/4072530811 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c0cadb0 con 0x7fe19c0cced0 2024-09-17T05:33:52.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a1c00640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe19c0cced0 0x7fe19c0cd350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0ad000 tx=0x7fe18c005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe198c00640 1 -- 172.21.15.122:0/4072530811 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0b3040 con 0x7fe19c0cced0 2024-09-17T05:33:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe198c00640 1 -- 172.21.15.122:0/4072530811 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe18c006c60 con 0x7fe19c0cced0 2024-09-17T05:33:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe198c00640 1 -- 172.21.15.122:0/4072530811 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c005cb0 con 0x7fe19c0cced0 2024-09-17T05:33:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c0cc110 con 0x7fe19c0cced0 2024-09-17T05:33:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c0cc6d0 con 0x7fe19c0cced0 2024-09-17T05:33:52.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.611+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c0d1ec0 con 0x7fe19c0cced0 2024-09-17T05:33:52.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.615+0000 7fe198c00640 1 -- 172.21.15.122:0/4072530811 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fe18c005e50 con 0x7fe19c0cced0 2024-09-17T05:33:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.615+0000 7fe198c00640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe17c0743f0 0x7fe17c0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.615+0000 7fe198c00640 1 -- 172.21.15.122:0/4072530811 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(77..77 src has 1..77) ==== 5823+0+0 (secure 0 0 0) 0x7fe18c1307a0 con 0x7fe19c0cced0 2024-09-17T05:33:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.619+0000 7fe1a1200640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe17c0743f0 0x7fe17c0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.619+0000 7fe1a1200640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe17c0743f0 0x7fe17c0768b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe188009ae0 tx=0x7fe188009580 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.623+0000 7fe198c00640 1 -- 172.21.15.122:0/4072530811 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe18c0194d0 con 0x7fe19c0cced0 2024-09-17T05:33:52.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:52.995+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone66604", "target": ["mon-mgr", ""]}) -- 0x7fe19c076160 con 0x7fe17c0743f0 2024-09-17T05:33:53.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe198c00640 1 -- 172.21.15.122:0/4072530811 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe19c076160 con 0x7fe17c0743f0 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe17c0743f0 msgr2=0x7fe17c0768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe17c0743f0 0x7fe17c0768b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe188009ae0 tx=0x7fe188009580 comp rx=0 tx=0).stop 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe19c0cced0 msgr2=0x7fe19c0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe19c0cced0 0x7fe19c0cd350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0ad000 tx=0x7fe18c005c40 comp rx=0 tx=0).stop 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 shutdown_connections 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe17c0743f0 0x7fe17c0768b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe19c0cced0 0x7fe19c0cd350 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c153d80 0x7fe19c1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 --2- 172.21.15.122:0/4072530811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe19c1f3840 0x7fe19c0ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.031+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 >> 172.21.15.122:0/4072530811 conn(0x7fe19c0d3f50 msgr2=0x7fe19c0d2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.035+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 shutdown_connections 2024-09-17T05:33:53.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.035+0000 7fe1a2600640 1 -- 172.21.15.122:0/4072530811 wait complete. 2024-09-17T05:33:53.066 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone77222 2024-09-17T05:33:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.315+0000 7f432ea00640 1 -- 172.21.15.122:0/2985492657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 msgr2=0x7f4310004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.315+0000 7f432ea00640 1 --2- 172.21.15.122:0/2985492657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 0x7f4310004090 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c3c0 tx=0x7f431c01ce10 comp rx=0 tx=0).stop 2024-09-17T05:33:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 -- 172.21.15.122:0/2985492657 shutdown_connections 2024-09-17T05:33:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 --2- 172.21.15.122:0/2985492657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 0x7f4310004090 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 -- 172.21.15.122:0/2985492657 >> 172.21.15.122:0/2985492657 conn(0x7f43280d1da0 msgr2=0x7f43280d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 -- 172.21.15.122:0/2985492657 shutdown_connections 2024-09-17T05:33:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 -- 172.21.15.122:0/2985492657 wait complete. 2024-09-17T05:33:53.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 Processor -- start 2024-09-17T05:33:53.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 -- start start 2024-09-17T05:33:53.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 0x7f43280c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f432815e580 0x7f43281fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43281fb510 0x7f43280ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43280b1980 con 0x7f4310003c90 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f43280b1800 con 0x7f43281fb510 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432ea00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f43280b1b00 con 0x7f432815e580 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 0x7f43280c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 0x7f43280c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46960/0 (socket says 172.21.15.122:46960) 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 -- 172.21.15.122:0/3681116767 learned_addr learned my addr 172.21.15.122:0/3681116767 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432cc00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f432815e580 0x7f43281fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432e000640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43281fb510 0x7f43280ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 -- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f432815e580 msgr2=0x7f43281fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f432815e580 0x7f43281fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 -- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43281fb510 msgr2=0x7f43280ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43281fb510 0x7f43280ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 -- 172.21.15.122:0/3681116767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43280c8f10 con 0x7f4310003c90 2024-09-17T05:33:53.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.319+0000 7f432d600640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 0x7f43280c9fb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f431c098a70 tx=0x7f431c0aea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:53.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f4325600640 1 -- 172.21.15.122:0/3681116767 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c006d10 con 0x7f4310003c90 2024-09-17T05:33:53.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f4325600640 1 -- 172.21.15.122:0/3681116767 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f431c0016d0 con 0x7f4310003c90 2024-09-17T05:33:53.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f4325600640 1 -- 172.21.15.122:0/3681116767 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c001880 con 0x7f4310003c90 2024-09-17T05:33:53.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43280c91f0 con 0x7f4310003c90 2024-09-17T05:33:53.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43280c97b0 con 0x7f4310003c90 2024-09-17T05:33:53.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43280d0730 con 0x7f4310003c90 2024-09-17T05:33:53.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f4325600640 1 -- 172.21.15.122:0/3681116767 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f431c003600 con 0x7f4310003c90 2024-09-17T05:33:53.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f4325600640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4308074370 0x7f4308076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f4325600640 1 -- 172.21.15.122:0/3681116767 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7f431c12d7c0 con 0x7f4310003c90 2024-09-17T05:33:53.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.323+0000 7f432cc00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4308074370 0x7f4308076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.327+0000 7f432cc00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4308074370 0x7f4308076830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f43280b6d30 tx=0x7f43140091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:53.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.327+0000 7f4325600640 1 -- 172.21.15.122:0/3681116767 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f43280b08f0 con 0x7f4310003c90 2024-09-17T05:33:53.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.519+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone77222", "target": ["mon-mgr", ""]}) -- 0x7f4328069e60 con 0x7f4308074370 2024-09-17T05:33:53.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.539+0000 7f4325600640 1 -- 172.21.15.122:0/3681116767 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4314009e10 con 0x7f4308074370 2024-09-17T05:33:53.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4308074370 msgr2=0x7f4308076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4308074370 0x7f4308076830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f43280b6d30 tx=0x7f43140091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 msgr2=0x7f43280c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 0x7f43280c9fb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f431c098a70 tx=0x7f431c0aea80 comp rx=0 tx=0).stop 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 shutdown_connections 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4308074370 0x7f4308076830 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43281fb510 0x7f43280ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f432815e580 0x7f43281fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 --2- 172.21.15.122:0/3681116767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4310003c90 0x7f43280c9fb0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 >> 172.21.15.122:0/3681116767 conn(0x7f43280d1da0 msgr2=0x7f43280b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 shutdown_connections 2024-09-17T05:33:53.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.543+0000 7f432ea00640 1 -- 172.21.15.122:0/3681116767 wait complete. 2024-09-17T05:33:53.565 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone34415 2024-09-17T05:33:53.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.715+0000 7fe35cc00640 1 -- 172.21.15.122:0/568921808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe35815f600 msgr2=0x7fe3580b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.715+0000 7fe35cc00640 1 --2- 172.21.15.122:0/568921808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe35815f600 0x7fe3580b2ed0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe348009cb0 tx=0x7fe34801cd70 comp rx=0 tx=0).stop 2024-09-17T05:33:53.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.715+0000 7fe35cc00640 1 -- 172.21.15.122:0/568921808 shutdown_connections 2024-09-17T05:33:53.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.715+0000 7fe35cc00640 1 --2- 172.21.15.122:0/568921808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3580d0500 0x7fe3580af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.715+0000 7fe35cc00640 1 --2- 172.21.15.122:0/568921808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe35815f600 0x7fe3580b2ed0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.715+0000 7fe35cc00640 1 -- 172.21.15.122:0/568921808 >> 172.21.15.122:0/568921808 conn(0x7fe3580d1e80 msgr2=0x7fe3580d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.715+0000 7fe35cc00640 1 -- 172.21.15.122:0/568921808 shutdown_connections 2024-09-17T05:33:53.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.715+0000 7fe35cc00640 1 -- 172.21.15.122:0/568921808 wait complete. 2024-09-17T05:33:53.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 Processor -- start 2024-09-17T05:33:53.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 -- start start 2024-09-17T05:33:53.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3580b3410 0x7fe35815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3580d0500 0x7fe35814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe35815f600 0x7fe3580b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe358168c40 con 0x7fe3580d0500 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe3580b0580 con 0x7fe3580b3410 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe358168dc0 con 0x7fe35815f600 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3580d0500 0x7fe35814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3580b3410 0x7fe35815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3580d0500 0x7fe35814fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46986/0 (socket says 172.21.15.122:46986) 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3580b3410 0x7fe35815d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:55516/0 (socket says 172.21.15.122:55516) 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe356800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe35815f600 0x7fe3580b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 -- 172.21.15.122:0/3181642222 learned_addr learned my addr 172.21.15.122:0/3181642222 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 -- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe35815f600 msgr2=0x7fe3580b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe35815f600 0x7fe3580b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 -- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3580b3410 msgr2=0x7fe35815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3580b3410 0x7fe35815d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 -- 172.21.15.122:0/3181642222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35815c0a0 con 0x7fe3580d0500 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe355400640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3580d0500 0x7fe35814fa10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe34400c4b0 tx=0x7fe34400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe34e000640 1 -- 172.21.15.122:0/3181642222 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344017070 con 0x7fe3580d0500 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe34e000640 1 -- 172.21.15.122:0/3181642222 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3440053c0 con 0x7fe3580d0500 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe34e000640 1 -- 172.21.15.122:0/3181642222 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344003860 con 0x7fe3580d0500 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35815c380 con 0x7fe3580d0500 2024-09-17T05:33:53.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.719+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35815c940 con 0x7fe3580d0500 2024-09-17T05:33:53.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.723+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35815ce30 con 0x7fe3580d0500 2024-09-17T05:33:53.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.723+0000 7fe34e000640 1 -- 172.21.15.122:0/3181642222 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fe34401a070 con 0x7fe3580d0500 2024-09-17T05:33:53.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.723+0000 7fe34e000640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe3380743a0 0x7fe338076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:53.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.723+0000 7fe34e000640 1 -- 172.21.15.122:0/3181642222 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7fe344092380 con 0x7fe3580d0500 2024-09-17T05:33:53.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.723+0000 7fe355e00640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe3380743a0 0x7fe338076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:53.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.723+0000 7fe355e00640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe3380743a0 0x7fe338076860 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe3480af000 tx=0x7fe34801b5b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:53.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.727+0000 7fe34e000640 1 -- 172.21.15.122:0/3181642222 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe34405e4e0 con 0x7fe3580d0500 2024-09-17T05:33:53.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.927+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone34415", "target": ["mon-mgr", ""]}) -- 0x7fe3580af2a0 con 0x7fe3380743a0 2024-09-17T05:33:53.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.943+0000 7fe34e000640 1 -- 172.21.15.122:0/3181642222 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe3580af2a0 con 0x7fe3380743a0 2024-09-17T05:33:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe3380743a0 msgr2=0x7fe338076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe3380743a0 0x7fe338076860 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe3480af000 tx=0x7fe34801b5b0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3580d0500 msgr2=0x7fe35814fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3580d0500 0x7fe35814fa10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe34400c4b0 tx=0x7fe34400c8b0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 shutdown_connections 2024-09-17T05:33:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe3380743a0 0x7fe338076860 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe35815f600 0x7fe3580b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3580d0500 0x7fe35814fa10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 --2- 172.21.15.122:0/3181642222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3580b3410 0x7fe35815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 >> 172.21.15.122:0/3181642222 conn(0x7fe3580d1e80 msgr2=0x7fe35815de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 shutdown_connections 2024-09-17T05:33:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:53.947+0000 7fe35cc00640 1 -- 172.21.15.122:0/3181642222 wait complete. 2024-09-17T05:33:53.966 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:33:54.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:33:54.020 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:33:54.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:33:54.027 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:33:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- 172.21.15.122:0/907395151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd80b27a0 msgr2=0x7f2cd80ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 --2- 172.21.15.122:0/907395151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd80b27a0 0x7f2cd80ce0e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009900 tx=0x7f2cc001c7c0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- 172.21.15.122:0/907395151 shutdown_connections 2024-09-17T05:33:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 --2- 172.21.15.122:0/907395151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cd80ce620 0x7f2cd80cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 --2- 172.21.15.122:0/907395151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd80b27a0 0x7f2cd80ce0e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- 172.21.15.122:0/907395151 >> 172.21.15.122:0/907395151 conn(0x7f2cd80d1e30 msgr2=0x7f2cd80d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- 172.21.15.122:0/907395151 shutdown_connections 2024-09-17T05:33:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- 172.21.15.122:0/907395151 wait complete. 2024-09-17T05:33:54.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 Processor -- start 2024-09-17T05:33:54.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- start start 2024-09-17T05:33:54.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cd80b27a0 0x7f2cd81f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cd80ce620 0x7f2cd81f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd815f640 0x7f2cd814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd80b1010 con 0x7f2cd815f640 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd80b0e90 con 0x7f2cd80b27a0 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cde600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd8169390 con 0x7f2cd80ce620 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd815f640 0x7f2cd814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd815f640 0x7f2cd814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47018/0 (socket says 172.21.15.122:47018) 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7e00640 1 -- 172.21.15.122:0/2352538267 learned_addr learned my addr 172.21.15.122:0/2352538267 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2ccea00640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cd80ce620 0x7f2cd81f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7400640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cd80b27a0 0x7f2cd81f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7e00640 1 -- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cd80ce620 msgr2=0x7f2cd81f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7e00640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cd80ce620 0x7f2cd81f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7e00640 1 -- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cd80b27a0 msgr2=0x7f2cd81f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7e00640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cd80b27a0 0x7f2cd81f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.223+0000 7f2cd7e00640 1 -- 172.21.15.122:0/2352538267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd81f6090 con 0x7f2cd815f640 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2cd7e00640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd815f640 0x7f2cd814ff50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000d9b0 tx=0x7f2cd000de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2ccfe00640 1 -- 172.21.15.122:0/2352538267 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0017030 con 0x7f2cd815f640 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2ccfe00640 1 -- 172.21.15.122:0/2352538267 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cd0004020 con 0x7f2cd815f640 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2ccfe00640 1 -- 172.21.15.122:0/2352538267 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0004470 con 0x7f2cd815f640 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd81f6370 con 0x7f2cd815f640 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd81f8e10 con 0x7f2cd815f640 2024-09-17T05:33:54.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd80ce0e0 con 0x7f2cd815f640 2024-09-17T05:33:54.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2ccfe00640 1 -- 172.21.15.122:0/2352538267 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f2cd001a070 con 0x7f2cd815f640 2024-09-17T05:33:54.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2ccfe00640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cb40743a0 0x7f2cb4076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2ccfe00640 1 -- 172.21.15.122:0/2352538267 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7f2cd00931a0 con 0x7f2cd815f640 2024-09-17T05:33:54.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.227+0000 7f2cd7400640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cb40743a0 0x7f2cb4076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.231+0000 7f2cd7400640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cb40743a0 0x7f2cb4076860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8001e80 tx=0x7f2cc8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:54.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.231+0000 7f2ccfe00640 1 -- 172.21.15.122:0/2352538267 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2cd005f360 con 0x7f2cd815f640 2024-09-17T05:33:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.399+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f2cd81f5d90 con 0x7f2cd815f640 2024-09-17T05:33:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.399+0000 7f2ccfe00640 1 -- 172.21.15.122:0/2352538267 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7f2cd0063d90 con 0x7f2cd815f640 2024-09-17T05:33:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cb40743a0 msgr2=0x7f2cb4076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cb40743a0 0x7f2cb4076860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8001e80 tx=0x7f2cc8009040 comp rx=0 tx=0).stop 2024-09-17T05:33:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd815f640 msgr2=0x7f2cd814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd815f640 0x7f2cd814ff50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000d9b0 tx=0x7f2cd000de70 comp rx=0 tx=0).stop 2024-09-17T05:33:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 shutdown_connections 2024-09-17T05:33:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cb40743a0 0x7f2cb4076860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cd815f640 0x7f2cd814ff50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cd80ce620 0x7f2cd81f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 --2- 172.21.15.122:0/2352538267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cd80b27a0 0x7f2cd81f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 >> 172.21.15.122:0/2352538267 conn(0x7f2cd80d1e30 msgr2=0x7f2cd815c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 shutdown_connections 2024-09-17T05:33:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.403+0000 7f2cde600640 1 -- 172.21.15.122:0/2352538267 wait complete. 2024-09-17T05:33:54.419 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:33:54.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:33:54.512 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:33:54.603 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:33:54.603 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:33:54.603 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:33:54.603 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:54.612 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:33:54.612 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:54.666 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:33:54.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.859+0000 7f7fabe00640 1 -- 172.21.15.122:0/1561483346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac073820 msgr2=0x7f7fac073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.859+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1561483346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac073820 0x7f7fac073c70 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0748c0 tx=0x7f7fa401cc30 comp rx=0 tx=0).stop 2024-09-17T05:33:54.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.859+0000 7f7fabe00640 1 -- 172.21.15.122:0/1561483346 shutdown_connections 2024-09-17T05:33:54.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.859+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1561483346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac073820 0x7f7fac073c70 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.859+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1561483346 >> v2:172.21.15.173:3300/0 conn(0x7f7fac1518b0 0x7f7fac158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.859+0000 7f7fabe00640 1 -- 172.21.15.122:0/1561483346 >> 172.21.15.122:0/1561483346 conn(0x7f7fac06dd30 msgr2=0x7f7fac06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:54.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.859+0000 7f7fabe00640 1 -- 172.21.15.122:0/1561483346 shutdown_connections 2024-09-17T05:33:54.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.859+0000 7f7fabe00640 1 -- 172.21.15.122:0/1561483346 wait complete. 2024-09-17T05:33:54.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 Processor -- start 2024-09-17T05:33:54.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 -- start start 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fac073820 0x7f7fac0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fac1518b0 0x7f7fac0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac1590a0 0x7f7fac1504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac1676a0 con 0x7f7fac1590a0 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac0b5390 con 0x7f7fac1518b0 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac167820 con 0x7f7fac073820 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac1590a0 0x7f7fac1504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7faaa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fac073820 0x7f7fac0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac1590a0 0x7f7fac1504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47026/0 (socket says 172.21.15.122:47026) 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 -- 172.21.15.122:0/1683970169 learned_addr learned my addr 172.21.15.122:0/1683970169 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7faa000640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fac1518b0 0x7f7fac0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 -- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fac073820 msgr2=0x7f7fac0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fac073820 0x7f7fac0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 -- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fac1518b0 msgr2=0x7f7fac0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fac1518b0 0x7f7fac0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 -- 172.21.15.122:0/1683970169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac0b2390 con 0x7f7fac1590a0 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7faaa00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fac073820 0x7f7fac0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fab400640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac1590a0 0x7f7fac1504a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4019840 tx=0x7f7fa4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fa2a00640 1 -- 172.21.15.122:0/1683970169 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40bb070 con 0x7f7fac1590a0 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fa2a00640 1 -- 172.21.15.122:0/1683970169 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fa4004d40 con 0x7f7fac1590a0 2024-09-17T05:33:54.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac0b2600 con 0x7f7fac1590a0 2024-09-17T05:33:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fa2a00640 1 -- 172.21.15.122:0/1683970169 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40b2040 con 0x7f7fac1590a0 2024-09-17T05:33:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.863+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac0b2bc0 con 0x7f7fac1590a0 2024-09-17T05:33:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.867+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fac06ade0 con 0x7f7fac1590a0 2024-09-17T05:33:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.867+0000 7f7fa2a00640 1 -- 172.21.15.122:0/1683970169 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f7fa40be070 con 0x7f7fac1590a0 2024-09-17T05:33:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.867+0000 7f7fa2a00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f8c074370 0x7f7f8c076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.867+0000 7f7fa2a00640 1 -- 172.21.15.122:0/1683970169 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7f7fa412cd50 con 0x7f7fac1590a0 2024-09-17T05:33:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.867+0000 7f7faaa00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f8c074370 0x7f7f8c076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.867+0000 7f7faaa00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f8c074370 0x7f7f8c076830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fac159050 tx=0x7f7f9c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:54.871+0000 7f7fa2a00640 1 -- 172.21.15.122:0/1683970169 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7fa4016970 con 0x7f7fac1590a0 2024-09-17T05:33:55.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.183+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7fac0b2d90 con 0x7f7fac1590a0 2024-09-17T05:33:55.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.183+0000 7f7fa2a00640 1 -- 172.21.15.122:0/1683970169 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7f7fa40f8eb0 con 0x7f7fac1590a0 2024-09-17T05:33:55.181 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:33:55.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f8c074370 msgr2=0x7f7f8c076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f8c074370 0x7f7f8c076830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fac159050 tx=0x7f7f9c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac1590a0 msgr2=0x7f7fac1504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac1590a0 0x7f7fac1504a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4019840 tx=0x7f7fa4003f60 comp rx=0 tx=0).stop 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 shutdown_connections 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f8c074370 0x7f7f8c076830 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac1590a0 0x7f7fac1504a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fac1518b0 0x7f7fac0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 --2- 172.21.15.122:0/1683970169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fac073820 0x7f7fac0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 >> 172.21.15.122:0/1683970169 conn(0x7f7fac06dd30 msgr2=0x7f7fac14c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 shutdown_connections 2024-09-17T05:33:55.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.187+0000 7f7fabe00640 1 -- 172.21.15.122:0/1683970169 wait complete. 2024-09-17T05:33:55.219 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:33:55.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:55.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:33:55.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:33:55.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:33:55.284 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:33:55.315 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:33:55.315 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:33:55.315 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:55.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:33:55.326 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:33:55.326 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:33:55.326 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:55.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:33:55.373 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:33:55.373 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:33:55.373 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:33:55.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:33:55.386 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:55.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- 172.21.15.122:0/3222382238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb0068960 msgr2=0x7f3eb00ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 --2- 172.21.15.122:0/3222382238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb0068960 0x7f3eb00ccca0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009bc0 tx=0x7f3ea801cba0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- 172.21.15.122:0/3222382238 shutdown_connections 2024-09-17T05:33:55.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 --2- 172.21.15.122:0/3222382238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb0068960 0x7f3eb00ccca0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 --2- 172.21.15.122:0/3222382238 >> v2:172.21.15.173:3300/0 conn(0x7f3eb015a420 0x7f3eb00af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- 172.21.15.122:0/3222382238 >> 172.21.15.122:0/3222382238 conn(0x7f3eb0073490 msgr2=0x7f3eb00d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:55.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- 172.21.15.122:0/3222382238 shutdown_connections 2024-09-17T05:33:55.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- 172.21.15.122:0/3222382238 wait complete. 2024-09-17T05:33:55.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 Processor -- start 2024-09-17T05:33:55.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- start start 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb0068960 0x7f3eb00c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00cde90 0x7f3eb00c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb015a420 0x7f3eb0155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb015edf0 con 0x7f3eb015a420 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb00ada20 con 0x7f3eb0068960 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eafe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb015ef70 con 0x7f3eb00cde90 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb015a420 0x7f3eb0155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb015a420 0x7f3eb0155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47036/0 (socket says 172.21.15.122:47036) 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 -- 172.21.15.122:0/192966633 learned_addr learned my addr 172.21.15.122:0/192966633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:55.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaea00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb0068960 0x7f3eb00c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eae000640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00cde90 0x7f3eb00c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 -- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00cde90 msgr2=0x7f3eb00c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00cde90 0x7f3eb00c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 -- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb0068960 msgr2=0x7f3eb00c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb0068960 0x7f3eb00c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 -- 172.21.15.122:0/192966633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb0165c50 con 0x7f3eb015a420 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eae000640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00cde90 0x7f3eb00c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaea00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb0068960 0x7f3eb00c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:55.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.583+0000 7f3eaf400640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb015a420 0x7f3eb0155800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3ea801a7d0 tx=0x7f3ea8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3ea6a00640 1 -- 172.21.15.122:0/192966633 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80b2040 con 0x7f3eb015a420 2024-09-17T05:33:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3ea6a00640 1 -- 172.21.15.122:0/192966633 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ea8004d40 con 0x7f3eb015a420 2024-09-17T05:33:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb0098c90 con 0x7f3eb015a420 2024-09-17T05:33:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3ea6a00640 1 -- 172.21.15.122:0/192966633 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8004210 con 0x7f3eb015a420 2024-09-17T05:33:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb0099250 con 0x7f3eb015a420 2024-09-17T05:33:55.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb00c35e0 con 0x7f3eb015a420 2024-09-17T05:33:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3ea6a00640 1 -- 172.21.15.122:0/192966633 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f3ea80be060 con 0x7f3eb015a420 2024-09-17T05:33:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3ea6a00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e90074370 0x7f3e90076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3ea6a00640 1 -- 172.21.15.122:0/192966633 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7f3ea812ccf0 con 0x7f3eb015a420 2024-09-17T05:33:55.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.587+0000 7f3eaea00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e90074370 0x7f3e90076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.591+0000 7f3eaea00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e90074370 0x7f3e90076830 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3eb009a050 tx=0x7f3ea0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:55.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.591+0000 7f3ea6a00640 1 -- 172.21.15.122:0/192966633 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3ea80a5e20 con 0x7f3eb015a420 2024-09-17T05:33:55.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3eb00ccca0 con 0x7f3eb015a420 2024-09-17T05:33:55.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3ea6a00640 1 -- 172.21.15.122:0/192966633 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5234 (secure 0 0 0) 0x7f3ea8016790 con 0x7f3eb015a420 2024-09-17T05:33:55.796 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:55.796 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":47,"btime":"2024-09-17T05:32:51:021047+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3900952275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3900952275},{"type":"v1","addr":"172.21.15.173:6836","nonce":3900952275}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5711,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/645422945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":645422945},{"type":"v1","addr":"172.21.15.173:6837","nonce":645422945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6408,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1968037162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1968037162},{"type":"v1","addr":"172.21.15.122:6833","nonce":1968037162}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:32:42.215056+0000","modified":"2024-09-17T05:32:45.239781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6426},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6426":{"gid":6426,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2505208774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2505208774},{"type":"v1","addr":"172.21.15.122:6835","nonce":2505208774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6426,"qdb_cluster":[6426]},"id":5}]} 2024-09-17T05:33:55.796 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 47 2024-09-17T05:33:55.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e90074370 msgr2=0x7f3e90076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e90074370 0x7f3e90076830 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3eb009a050 tx=0x7f3ea0009040 comp rx=0 tx=0).stop 2024-09-17T05:33:55.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb015a420 msgr2=0x7f3eb0155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb015a420 0x7f3eb0155800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3ea801a7d0 tx=0x7f3ea8003f60 comp rx=0 tx=0).stop 2024-09-17T05:33:55.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 shutdown_connections 2024-09-17T05:33:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e90074370 0x7f3e90076830 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb015a420 0x7f3eb0155800 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00cde90 0x7f3eb00c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 --2- 172.21.15.122:0/192966633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb0068960 0x7f3eb00c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 >> 172.21.15.122:0/192966633 conn(0x7f3eb0073490 msgr2=0x7f3eb0158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.799+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 shutdown_connections 2024-09-17T05:33:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.803+0000 7f3eafe00640 1 -- 172.21.15.122:0/192966633 wait complete. 2024-09-17T05:33:55.817 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:55.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.991+0000 7faba6800640 1 -- 172.21.15.122:0/523334764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faba00737c0 msgr2=0x7faba0073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.991+0000 7faba6800640 1 --2- 172.21.15.122:0/523334764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faba00737c0 0x7faba0073c20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faba0074c60 tx=0x7fab8c01c820 comp rx=0 tx=0).stop 2024-09-17T05:33:55.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.991+0000 7faba6800640 1 -- 172.21.15.122:0/523334764 shutdown_connections 2024-09-17T05:33:55.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.991+0000 7faba6800640 1 --2- 172.21.15.122:0/523334764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faba00737c0 0x7faba0073c20 secure :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faba0074c60 tx=0x7fab8c01c820 comp rx=0 tx=0).stop 2024-09-17T05:33:55.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.991+0000 7faba6800640 1 --2- 172.21.15.122:0/523334764 >> v2:172.21.15.173:3300/0 conn(0x7faba015af60 0x7faba00cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.991+0000 7faba6800640 1 -- 172.21.15.122:0/523334764 >> 172.21.15.122:0/523334764 conn(0x7faba00d0c90 msgr2=0x7faba00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:55.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.991+0000 7faba6800640 1 -- 172.21.15.122:0/523334764 shutdown_connections 2024-09-17T05:33:55.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.991+0000 7faba6800640 1 -- 172.21.15.122:0/523334764 wait complete. 2024-09-17T05:33:55.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.995+0000 7faba6800640 1 Processor -- start 2024-09-17T05:33:55.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.995+0000 7faba6800640 1 -- start start 2024-09-17T05:33:55.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.995+0000 7faba6800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faba0074160 0x7faba00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.995+0000 7faba6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faba015af60 0x7faba0156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.995+0000 7faba6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba0156b90 0x7faba00b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:55.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.995+0000 7faba6800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faba0167cc0 con 0x7faba015af60 2024-09-17T05:33:55.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.995+0000 7faba6800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faba0167b40 con 0x7faba0156b90 2024-09-17T05:33:55.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.995+0000 7faba6800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faba0167e40 con 0x7faba0074160 2024-09-17T05:33:55.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faba0074160 0x7faba00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faba0074160 0x7faba00b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:46964/0 (socket says 172.21.15.122:46964) 2024-09-17T05:33:55.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 -- 172.21.15.122:0/3080457376 learned_addr learned my addr 172.21.15.122:0/3080457376 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:55.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9ea00640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faba015af60 0x7faba0156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9fe00640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba0156b90 0x7faba00b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:55.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 -- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba0156b90 msgr2=0x7faba00b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba0156b90 0x7faba00b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 -- 172.21.15.122:0/3080457376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faba015af60 msgr2=0x7faba0156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:55.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faba015af60 0x7faba0156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:55.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 -- 172.21.15.122:0/3080457376 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba00b2b20 con 0x7faba0074160 2024-09-17T05:33:55.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab9f400640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faba0074160 0x7faba00b12e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b440 tx=0x7fab9000b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:55.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab97400640 1 -- 172.21.15.122:0/3080457376 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90015020 con 0x7faba0074160 2024-09-17T05:33:56.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab97400640 1 -- 172.21.15.122:0/3080457376 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab900119a0 con 0x7faba0074160 2024-09-17T05:33:56.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7fab97400640 1 -- 172.21.15.122:0/3080457376 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90011df0 con 0x7faba0074160 2024-09-17T05:33:56.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba00b2840 con 0x7faba0074160 2024-09-17T05:33:56.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:55.999+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faba00b0ae0 con 0x7faba0074160 2024-09-17T05:33:56.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.003+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba0073c20 con 0x7faba0074160 2024-09-17T05:33:56.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.003+0000 7fab97400640 1 -- 172.21.15.122:0/3080457376 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fab90018020 con 0x7faba0074160 2024-09-17T05:33:56.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.003+0000 7fab97400640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab800743c0 0x7fab80076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.007+0000 7fab97400640 1 -- 172.21.15.122:0/3080457376 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7fab90093060 con 0x7faba0074160 2024-09-17T05:33:56.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.007+0000 7fab9ea00640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab800743c0 0x7fab80076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.007+0000 7fab9ea00640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab800743c0 0x7fab80076880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0af000 tx=0x7fab8c01b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.011+0000 7fab97400640 1 -- 172.21.15.122:0/3080457376 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fab9001f020 con 0x7faba0074160 2024-09-17T05:33:56.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.447+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faba006da40 con 0x7faba0074160 2024-09-17T05:33:56.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.451+0000 7fab97400640 1 -- 172.21.15.122:0/3080457376 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5234 (secure 0 0 0) 0x7fab9000d040 con 0x7faba0074160 2024-09-17T05:33:56.448 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:56.448 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":47,"btime":"2024-09-17T05:32:51:021047+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3900952275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3900952275},{"type":"v1","addr":"172.21.15.173:6836","nonce":3900952275}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5711,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/645422945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":645422945},{"type":"v1","addr":"172.21.15.173:6837","nonce":645422945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6408,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1968037162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1968037162},{"type":"v1","addr":"172.21.15.122:6833","nonce":1968037162}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:32:42.215056+0000","modified":"2024-09-17T05:32:45.239781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6426},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6426":{"gid":6426,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2505208774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2505208774},{"type":"v1","addr":"172.21.15.122:6835","nonce":2505208774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6426,"qdb_cluster":[6426]},"id":5}]} 2024-09-17T05:33:56.449 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 47 2024-09-17T05:33:56.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab800743c0 msgr2=0x7fab80076880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab800743c0 0x7fab80076880 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0af000 tx=0x7fab8c01b060 comp rx=0 tx=0).stop 2024-09-17T05:33:56.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faba0074160 msgr2=0x7faba00b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faba0074160 0x7faba00b12e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b440 tx=0x7fab9000b900 comp rx=0 tx=0).stop 2024-09-17T05:33:56.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 shutdown_connections 2024-09-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab800743c0 0x7fab80076880 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faba0156b90 0x7faba00b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faba015af60 0x7faba0156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 --2- 172.21.15.122:0/3080457376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faba0074160 0x7faba00b12e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 >> 172.21.15.122:0/3080457376 conn(0x7faba00d0c90 msgr2=0x7faba0070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 shutdown_connections 2024-09-17T05:33:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.455+0000 7faba6800640 1 -- 172.21.15.122:0/3080457376 wait complete. 2024-09-17T05:33:56.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:33:56.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.623+0000 7fab28c00640 1 -- 172.21.15.122:0/1461284827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab24158240 msgr2=0x7fab2415ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.623+0000 7fab28c00640 1 --2- 172.21.15.122:0/1461284827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab24158240 0x7fab2415ede0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fab240b3820 tx=0x7fab1401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.623+0000 7fab28c00640 1 -- 172.21.15.122:0/1461284827 shutdown_connections 2024-09-17T05:33:56.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.623+0000 7fab28c00640 1 --2- 172.21.15.122:0/1461284827 >> v2:172.21.15.173:3300/0 conn(0x7fab240cc8c0 0x7fab240cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.623+0000 7fab28c00640 1 --2- 172.21.15.122:0/1461284827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab24158240 0x7fab2415ede0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.623+0000 7fab28c00640 1 -- 172.21.15.122:0/1461284827 >> 172.21.15.122:0/1461284827 conn(0x7fab240d0c90 msgr2=0x7fab240d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.623+0000 7fab28c00640 1 -- 172.21.15.122:0/1461284827 shutdown_connections 2024-09-17T05:33:56.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.623+0000 7fab28c00640 1 -- 172.21.15.122:0/1461284827 wait complete. 2024-09-17T05:33:56.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 Processor -- start 2024-09-17T05:33:56.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 -- start start 2024-09-17T05:33:56.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab240cc8c0 0x7fab240c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24158240 0x7fab240b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab24158620 0x7fab240b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab240ce860 con 0x7fab240cc8c0 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab24072190 con 0x7fab24158240 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fab240ce9e0 con 0x7fab24158620 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab240cc8c0 0x7fab240c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab240cc8c0 0x7fab240c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47080/0 (socket says 172.21.15.122:47080) 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 -- 172.21.15.122:0/2078753799 learned_addr learned my addr 172.21.15.122:0/2078753799 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab22800640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab24158620 0x7fab240b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 -- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab24158620 msgr2=0x7fab240b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab24158620 0x7fab240b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 -- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24158240 msgr2=0x7fab240b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21400640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24158240 0x7fab240b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24158240 0x7fab240b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 -- 172.21.15.122:0/2078753799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2415f930 con 0x7fab240cc8c0 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab22800640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab24158620 0x7fab240b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21400640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24158240 0x7fab240b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab21e00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab240cc8c0 0x7fab240c3c00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fab1401ccb0 tx=0x7fab140a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab1a000640 1 -- 172.21.15.122:0/2078753799 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140bb070 con 0x7fab240cc8c0 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab1a000640 1 -- 172.21.15.122:0/2078753799 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab14004430 con 0x7fab240cc8c0 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab1a000640 1 -- 172.21.15.122:0/2078753799 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140b2040 con 0x7fab240cc8c0 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab24153d20 con 0x7fab240cc8c0 2024-09-17T05:33:56.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab240b65a0 con 0x7fab240cc8c0 2024-09-17T05:33:56.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.627+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab240b67a0 con 0x7fab240cc8c0 2024-09-17T05:33:56.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.631+0000 7fab1a000640 1 -- 172.21.15.122:0/2078753799 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fab140be070 con 0x7fab240cc8c0 2024-09-17T05:33:56.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.631+0000 7fab1a000640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab04074370 0x7fab04076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.631+0000 7fab1a000640 1 -- 172.21.15.122:0/2078753799 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7fab1412c9a0 con 0x7fab240cc8c0 2024-09-17T05:33:56.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.631+0000 7fab21400640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab04074370 0x7fab04076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.631+0000 7fab21400640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab04074370 0x7fab04076830 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fab24070530 tx=0x7fab10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.635+0000 7fab1a000640 1 -- 172.21.15.122:0/2078753799 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fab240b67a0 con 0x7fab240cc8c0 2024-09-17T05:33:56.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.803+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab240cccd0 con 0x7fab240cc8c0 2024-09-17T05:33:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.803+0000 7fab1a000640 1 -- 172.21.15.122:0/2078753799 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+17023 (secure 0 0 0) 0x7fab140163f0 con 0x7fab240cc8c0 2024-09-17T05:33:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:56.801 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":78,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:33:53.071323+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:32:39.907333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:32:41.177668+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":78,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"11.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab04074370 msgr2=0x7fab04076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab04074370 0x7fab04076830 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fab24070530 tx=0x7fab10009040 comp rx=0 tx=0).stop 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab240cc8c0 msgr2=0x7fab240c3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab240cc8c0 0x7fab240c3c00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fab1401ccb0 tx=0x7fab140a57a0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 shutdown_connections 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fab04074370 0x7fab04076830 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab24158620 0x7fab240b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab24158240 0x7fab240b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 --2- 172.21.15.122:0/2078753799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab240cc8c0 0x7fab240c3c00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 >> 172.21.15.122:0/2078753799 conn(0x7fab240d0c90 msgr2=0x7fab2406f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 shutdown_connections 2024-09-17T05:33:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.807+0000 7fab28c00640 1 -- 172.21.15.122:0/2078753799 wait complete. 2024-09-17T05:33:56.823 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:33:56.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:56.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.967+0000 7f0d69000640 1 -- 172.21.15.122:0/3844761197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 msgr2=0x7f0d640b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.967+0000 7f0d69000640 1 --2- 172.21.15.122:0/3844761197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 0x7f0d640b2bf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009900 tx=0x7f0d4c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.967+0000 7f0d69000640 1 -- 172.21.15.122:0/3844761197 shutdown_connections 2024-09-17T05:33:56.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.967+0000 7f0d69000640 1 --2- 172.21.15.122:0/3844761197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 0x7f0d640b2bf0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.967+0000 7f0d69000640 1 -- 172.21.15.122:0/3844761197 >> 172.21.15.122:0/3844761197 conn(0x7f0d640b18f0 msgr2=0x7f0d640b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:56.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.967+0000 7f0d69000640 1 -- 172.21.15.122:0/3844761197 shutdown_connections 2024-09-17T05:33:56.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.967+0000 7f0d69000640 1 -- 172.21.15.122:0/3844761197 wait complete. 2024-09-17T05:33:56.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 Processor -- start 2024-09-17T05:33:56.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 -- start start 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d640b3130 0x7f0d640c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 0x7f0d6409b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6415b580 0x7f0d6409b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d64168fa0 con 0x7f0d640b4bd0 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d64168e20 con 0x7f0d640b3130 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0d64169120 con 0x7f0d6415b580 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 0x7f0d6409b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 0x7f0d6409b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47104/0 (socket says 172.21.15.122:47104) 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 -- 172.21.15.122:0/1477612178 learned_addr learned my addr 172.21.15.122:0/1477612178 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d61e00640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d640b3130 0x7f0d640c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d62800640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6415b580 0x7f0d6409b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 -- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6415b580 msgr2=0x7f0d6409b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6415b580 0x7f0d6409b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 -- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d640b3130 msgr2=0x7f0d640c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d640b3130 0x7f0d640c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 -- 172.21.15.122:0/1477612178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d641666a0 con 0x7f0d640b4bd0 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d59400640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 0x7f0d6409b420 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009850 tx=0x7f0d4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d5aa00640 1 -- 172.21.15.122:0/1477612178 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0b2040 con 0x7f0d640b4bd0 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d5aa00640 1 -- 172.21.15.122:0/1477612178 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d4c005020 con 0x7f0d640b4bd0 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d5aa00640 1 -- 172.21.15.122:0/1477612178 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c005430 con 0x7f0d640b4bd0 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d641663c0 con 0x7f0d640b4bd0 2024-09-17T05:33:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.971+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6409d6c0 con 0x7f0d640b4bd0 2024-09-17T05:33:56.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.975+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d640b2bf0 con 0x7f0d640b4bd0 2024-09-17T05:33:56.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.975+0000 7f0d5aa00640 1 -- 172.21.15.122:0/1477612178 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f0d4c0046d0 con 0x7f0d640b4bd0 2024-09-17T05:33:56.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.975+0000 7f0d5aa00640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d40074370 0x7f0d40076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:56.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.975+0000 7f0d5aa00640 1 -- 172.21.15.122:0/1477612178 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7f0d4c12cd50 con 0x7f0d640b4bd0 2024-09-17T05:33:56.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.975+0000 7f0d61e00640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d40074370 0x7f0d40076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:56.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.975+0000 7f0d61e00640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d40074370 0x7f0d40076830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0d54001e40 tx=0x7f0d54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:56.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:56.979+0000 7f0d5aa00640 1 -- 172.21.15.122:0/1477612178 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0d4c0164d0 con 0x7f0d640b4bd0 2024-09-17T05:33:57.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.183+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d6406f740 con 0x7f0d640b4bd0 2024-09-17T05:33:57.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d5aa00640 1 -- 172.21.15.122:0/1477612178 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5234 (secure 0 0 0) 0x7f0d4c0f9200 con 0x7f0d640b4bd0 2024-09-17T05:33:57.182 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:57.183 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":47,"btime":"2024-09-17T05:32:51:021047+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3900952275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3900952275},{"type":"v1","addr":"172.21.15.173:6836","nonce":3900952275}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5711,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/645422945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":645422945},{"type":"v1","addr":"172.21.15.173:6837","nonce":645422945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6408,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1968037162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1968037162},{"type":"v1","addr":"172.21.15.122:6833","nonce":1968037162}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:32:42.215056+0000","modified":"2024-09-17T05:32:45.239781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6426},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6426":{"gid":6426,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2505208774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2505208774},{"type":"v1","addr":"172.21.15.122:6835","nonce":2505208774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6426,"qdb_cluster":[6426]},"id":5}]} 2024-09-17T05:33:57.183 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 47 2024-09-17T05:33:57.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d40074370 msgr2=0x7f0d40076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d40074370 0x7f0d40076830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0d54001e40 tx=0x7f0d54009040 comp rx=0 tx=0).stop 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 msgr2=0x7f0d6409b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 0x7f0d6409b420 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009850 tx=0x7f0d4c004040 comp rx=0 tx=0).stop 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 shutdown_connections 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d40074370 0x7f0d40076830 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6415b580 0x7f0d6409b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d640b4bd0 0x7f0d6409b420 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 --2- 172.21.15.122:0/1477612178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d640b3130 0x7f0d640c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 >> 172.21.15.122:0/1477612178 conn(0x7f0d640b18f0 msgr2=0x7f0d6406fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 shutdown_connections 2024-09-17T05:33:57.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.187+0000 7f0d69000640 1 -- 172.21.15.122:0/1477612178 wait complete. 2024-09-17T05:33:57.203 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:33:57.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- 172.21.15.122:0/2541451299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfac0b2e80 msgr2=0x7fdfac0b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2541451299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfac0b2e80 0x7fdfac0b1bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009900 tx=0x7fdf9801c790 comp rx=0 tx=0).stop 2024-09-17T05:33:57.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- 172.21.15.122:0/2541451299 shutdown_connections 2024-09-17T05:33:57.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2541451299 >> v2:172.21.15.173:3300/0 conn(0x7fdfac0afca0 0x7fdfac0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2541451299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfac0b2e80 0x7fdfac0b1bd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- 172.21.15.122:0/2541451299 >> 172.21.15.122:0/2541451299 conn(0x7fdfac0d1e30 msgr2=0x7fdfac0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:57.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- 172.21.15.122:0/2541451299 shutdown_connections 2024-09-17T05:33:57.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- 172.21.15.122:0/2541451299 wait complete. 2024-09-17T05:33:57.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 Processor -- start 2024-09-17T05:33:57.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- start start 2024-09-17T05:33:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac0afca0 0x7fdfac06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac0b2e80 0x7fdfac06f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfac15f490 0x7fdfac06e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac168c80 con 0x7fdfac0afca0 2024-09-17T05:33:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac0b2320 con 0x7fdfac0b2e80 2024-09-17T05:33:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb4200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdfac168e00 con 0x7fdfac15f490 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb1800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac0afca0 0x7fdfac06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb1800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac0afca0 0x7fdfac06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47116/0 (socket says 172.21.15.122:47116) 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb1800640 1 -- 172.21.15.122:0/2120633987 learned_addr learned my addr 172.21.15.122:0/2120633987 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb0e00640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac0b2e80 0x7fdfac06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.351+0000 7fdfb2200640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfac15f490 0x7fdfac06e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb1800640 1 -- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfac15f490 msgr2=0x7fdfac06e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb1800640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfac15f490 0x7fdfac06e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb1800640 1 -- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac0b2e80 msgr2=0x7fdfac06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb1800640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac0b2e80 0x7fdfac06f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb1800640 1 -- 172.21.15.122:0/2120633987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac06e270 con 0x7fdfac0afca0 2024-09-17T05:33:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb1800640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac0afca0 0x7fdfac06dd30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c760 tx=0x7fdfa000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfa9600640 1 -- 172.21.15.122:0/2120633987 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0003b80 con 0x7fdfac0afca0 2024-09-17T05:33:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfa9600640 1 -- 172.21.15.122:0/2120633987 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa0003d40 con 0x7fdfac0afca0 2024-09-17T05:33:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac0cf180 con 0x7fdfac0afca0 2024-09-17T05:33:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfa9600640 1 -- 172.21.15.122:0/2120633987 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000aab0 con 0x7fdfac0afca0 2024-09-17T05:33:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac0ceea0 con 0x7fdfac0afca0 2024-09-17T05:33:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac0b0130 con 0x7fdfac0afca0 2024-09-17T05:33:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfa9600640 1 -- 172.21.15.122:0/2120633987 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fdfa000ac50 con 0x7fdfac0afca0 2024-09-17T05:33:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfa9600640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf8c074370 0x7fdf8c076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.355+0000 7fdfa9600640 1 -- 172.21.15.122:0/2120633987 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7fdfa0092e40 con 0x7fdfac0afca0 2024-09-17T05:33:57.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.359+0000 7fdfb0e00640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf8c074370 0x7fdf8c076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.359+0000 7fdfb0e00640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf8c074370 0x7fdf8c076830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdf980af000 tx=0x7fdf9801afd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:57.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.359+0000 7fdfa9600640 1 -- 172.21.15.122:0/2120633987 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdfa005efa0 con 0x7fdfac0afca0 2024-09-17T05:33:57.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.559+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfac15bf80 con 0x7fdfac0afca0 2024-09-17T05:33:57.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.559+0000 7fdfa9600640 1 -- 172.21.15.122:0/2120633987 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5234 (secure 0 0 0) 0x7fdfa00639d0 con 0x7fdfac0afca0 2024-09-17T05:33:57.557 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:57.557 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":47,"btime":"2024-09-17T05:32:51:021047+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3900952275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3900952275},{"type":"v1","addr":"172.21.15.173:6836","nonce":3900952275}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5711,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/645422945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":645422945},{"type":"v1","addr":"172.21.15.173:6837","nonce":645422945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6408,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1968037162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1968037162},{"type":"v1","addr":"172.21.15.122:6833","nonce":1968037162}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:32:42.215056+0000","modified":"2024-09-17T05:32:45.239781+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6426},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6426":{"gid":6426,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2505208774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2505208774},{"type":"v1","addr":"172.21.15.122:6835","nonce":2505208774}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6426,"qdb_cluster":[6426]},"id":5}]} 2024-09-17T05:33:57.557 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 47 2024-09-17T05:33:57.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf8c074370 msgr2=0x7fdf8c076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf8c074370 0x7fdf8c076830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdf980af000 tx=0x7fdf9801afd0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac0afca0 msgr2=0x7fdfac06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac0afca0 0x7fdfac06dd30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c760 tx=0x7fdfa000cc20 comp rx=0 tx=0).stop 2024-09-17T05:33:57.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 shutdown_connections 2024-09-17T05:33:57.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf8c074370 0x7fdf8c076830 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfac15f490 0x7fdfac06e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfac0b2e80 0x7fdfac06f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 --2- 172.21.15.122:0/2120633987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfac0afca0 0x7fdfac06dd30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 >> 172.21.15.122:0/2120633987 conn(0x7fdfac0d1e30 msgr2=0x7fdfac15c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:57.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 shutdown_connections 2024-09-17T05:33:57.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.563+0000 7fdfb4200640 1 -- 172.21.15.122:0/2120633987 wait complete. 2024-09-17T05:33:57.579 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:33:57.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.723+0000 7faa2ec00640 1 -- 172.21.15.122:0/4207350725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 msgr2=0x7faa10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.723+0000 7faa2ec00640 1 --2- 172.21.15.122:0/4207350725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 0x7faa10004810 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c4a0 tx=0x7faa1401cf00 comp rx=0 tx=0).stop 2024-09-17T05:33:57.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.723+0000 7faa2e200640 1 -- 172.21.15.122:0/4207350725 reap_dead start 2024-09-17T05:33:57.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.723+0000 7faa2ec00640 1 -- 172.21.15.122:0/4207350725 shutdown_connections 2024-09-17T05:33:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.723+0000 7faa2ec00640 1 -- 172.21.15.122:0/4207350725 >> 172.21.15.122:0/4207350725 conn(0x7faa2809aa10 msgr2=0x7faa2809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.723+0000 7faa2ec00640 1 -- 172.21.15.122:0/4207350725 shutdown_connections 2024-09-17T05:33:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.723+0000 7faa2ec00640 1 -- 172.21.15.122:0/4207350725 wait complete. 2024-09-17T05:33:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 Processor -- start 2024-09-17T05:33:57.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 -- start start 2024-09-17T05:33:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 0x7faa28071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa28156d40 0x7faa28154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa281545d0 0x7faa28144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa28166c70 con 0x7faa10004030 2024-09-17T05:33:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa28166af0 con 0x7faa281545d0 2024-09-17T05:33:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faa28166df0 con 0x7faa28156d40 2024-09-17T05:33:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 0x7faa28071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa28156d40 0x7faa28154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2e200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa281545d0 0x7faa28144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa28156d40 0x7faa28154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:47032/0 (socket says 172.21.15.122:47032) 2024-09-17T05:33:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 0x7faa28071b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47134/0 (socket says 172.21.15.122:47134) 2024-09-17T05:33:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ce00640 1 -- 172.21.15.122:0/2375737685 learned_addr learned my addr 172.21.15.122:0/2375737685 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2e200640 1 -- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa28156d40 msgr2=0x7faa28154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2e200640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa28156d40 0x7faa28154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2e200640 1 -- 172.21.15.122:0/2375737685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 msgr2=0x7faa28071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2e200640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 0x7faa28071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2e200640 1 -- 172.21.15.122:0/2375737685 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa281454e0 con 0x7faa281545d0 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2d800640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 0x7faa28071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2e200640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa281545d0 0x7faa28144790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faa280732a0 tx=0x7faa20011ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa25600640 1 -- 172.21.15.122:0/2375737685 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20004b30 con 0x7faa281545d0 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa25600640 1 -- 172.21.15.122:0/2375737685 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa20004cd0 con 0x7faa281545d0 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa28156280 con 0x7faa281545d0 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa25600640 1 -- 172.21.15.122:0/2375737685 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20005280 con 0x7faa281545d0 2024-09-17T05:33:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa28154a50 con 0x7faa281545d0 2024-09-17T05:33:57.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.727+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa28075e50 con 0x7faa281545d0 2024-09-17T05:33:57.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.731+0000 7faa25600640 1 -- 172.21.15.122:0/2375737685 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7faa20005420 con 0x7faa281545d0 2024-09-17T05:33:57.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.731+0000 7faa25600640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa080743a0 0x7faa08076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:57.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.731+0000 7faa25600640 1 -- 172.21.15.122:0/2375737685 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7faa20093a20 con 0x7faa281545d0 2024-09-17T05:33:57.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.731+0000 7faa2d800640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa080743a0 0x7faa08076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:57.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.731+0000 7faa2d800640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa080743a0 0x7faa08076860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faa1c012560 tx=0x7faa1c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:57.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.735+0000 7faa25600640 1 -- 172.21.15.122:0/2375737685 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7faa2001f060 con 0x7faa281545d0 2024-09-17T05:33:57.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.903+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa28156450 con 0x7faa281545d0 2024-09-17T05:33:57.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.903+0000 7faa25600640 1 -- 172.21.15.122:0/2375737685 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+17023 (secure 0 0 0) 0x7faa200601d0 con 0x7faa281545d0 2024-09-17T05:33:57.902 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:33:57.902 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":78,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:33:53.071323+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:32:39.907333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-17T05:32:41.177668+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":78,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"11.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:33:57.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa080743a0 msgr2=0x7faa08076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa080743a0 0x7faa08076860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faa1c012560 tx=0x7faa1c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa281545d0 msgr2=0x7faa28144790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa281545d0 0x7faa28144790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faa280732a0 tx=0x7faa20011ef0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 shutdown_connections 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa080743a0 0x7faa08076860 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa281545d0 0x7faa28144790 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa28156d40 0x7faa28154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 --2- 172.21.15.122:0/2375737685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa10004030 0x7faa28071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 >> 172.21.15.122:0/2375737685 conn(0x7faa2809aa10 msgr2=0x7faa2815e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 shutdown_connections 2024-09-17T05:33:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:57.907+0000 7faa2ec00640 1 -- 172.21.15.122:0/2375737685 wait complete. 2024-09-17T05:33:57.925 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:33:58.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 -- 172.21.15.122:0/2068011452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400cd100 msgr2=0x7f04400acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:58.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 --2- 172.21.15.122:0/2068011452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400cd100 0x7f04400acd40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f043800acd0 tx=0x7f043801de60 comp rx=0 tx=0).stop 2024-09-17T05:33:58.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 -- 172.21.15.122:0/2068011452 shutdown_connections 2024-09-17T05:33:58.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 --2- 172.21.15.122:0/2068011452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400cd100 0x7f04400acd40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 -- 172.21.15.122:0/2068011452 >> 172.21.15.122:0/2068011452 conn(0x7f04400b18f0 msgr2=0x7f04400b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:58.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 -- 172.21.15.122:0/2068011452 shutdown_connections 2024-09-17T05:33:58.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 -- 172.21.15.122:0/2068011452 wait complete. 2024-09-17T05:33:58.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 Processor -- start 2024-09-17T05:33:58.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.071+0000 7f0446a00640 1 -- start start 2024-09-17T05:33:58.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400b4a50 0x7f04400b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04400cd100 0x7f044009bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f044015db50 0x7f044015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0440168130 con 0x7f04400b4a50 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04400ccc80 con 0x7f04400cd100 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f04401682b0 con 0x7f044015db50 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400b4a50 0x7f04400b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400b4a50 0x7f04400b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47146/0 (socket says 172.21.15.122:47146) 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 -- 172.21.15.122:0/386160399 learned_addr learned my addr 172.21.15.122:0/386160399 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0444a00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f044015db50 0x7f044015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0437400640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04400cd100 0x7f044009bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 -- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f044015db50 msgr2=0x7f044015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:58.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f044015db50 0x7f044015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 -- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04400cd100 msgr2=0x7f044009bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04400cd100 0x7f044009bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 -- 172.21.15.122:0/386160399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04400b13d0 con 0x7f04400b4a50 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0437400640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04400cd100 0x7f044009bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043fe00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400b4a50 0x7f04400b0e90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f043000c430 tx=0x7f043000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043cc00640 1 -- 172.21.15.122:0/386160399 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430011790 con 0x7f04400b4a50 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043cc00640 1 -- 172.21.15.122:0/386160399 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0430017070 con 0x7f04400b4a50 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044009ac20 con 0x7f04400b4a50 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043cc00640 1 -- 172.21.15.122:0/386160399 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0430011ce0 con 0x7f04400b4a50 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044009b1e0 con 0x7f04400b4a50 2024-09-17T05:33:58.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0440069410 con 0x7f04400b4a50 2024-09-17T05:33:58.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.075+0000 7f043cc00640 1 -- 172.21.15.122:0/386160399 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f043001c460 con 0x7f04400b4a50 2024-09-17T05:33:58.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.079+0000 7f043cc00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f041c074370 0x7f041c076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:58.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.079+0000 7f043cc00640 1 -- 172.21.15.122:0/386160399 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5879+0+0 (secure 0 0 0) 0x7f0430092de0 con 0x7f04400b4a50 2024-09-17T05:33:58.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.079+0000 7f0437400640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f041c074370 0x7f041c076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:58.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.079+0000 7f0437400640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f041c074370 0x7f041c076830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f04400cbe70 tx=0x7f0428009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:58.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.083+0000 7f043cc00640 1 -- 172.21.15.122:0/386160399 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f043005f690 con 0x7f04400b4a50 2024-09-17T05:33:58.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.287+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0440159430 con 0x7f04400b4a50 2024-09-17T05:33:58.283 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:33:58.287+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:33:58.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.287+0000 7f043cc00640 1 -- 172.21.15.122:0/386160399 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f043005ef90 con 0x7f04400b4a50 2024-09-17T05:33:58.286 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f041c074370 msgr2=0x7f041c076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f041c074370 0x7f041c076830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f04400cbe70 tx=0x7f0428009040 comp rx=0 tx=0).stop 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400b4a50 msgr2=0x7f04400b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400b4a50 0x7f04400b0e90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f043000c430 tx=0x7f043000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 shutdown_connections 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f041c074370 0x7f041c076830 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f044015db50 0x7f044015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04400cd100 0x7f044009bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 --2- 172.21.15.122:0/386160399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04400b4a50 0x7f04400b0e90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 >> 172.21.15.122:0/386160399 conn(0x7f04400b18f0 msgr2=0x7f044006fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:58.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 shutdown_connections 2024-09-17T05:33:58.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.291+0000 7f0446a00640 1 -- 172.21.15.122:0/386160399 wait complete. 2024-09-17T05:33:58.311 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:33:58.355 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:33:58.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- 172.21.15.122:0/737734092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 msgr2=0x7f0f740cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:58.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 --2- 172.21.15.122:0/737734092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 0x7f0f740cf1a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009900 tx=0x7f0f6c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- 172.21.15.122:0/737734092 shutdown_connections 2024-09-17T05:33:58.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 --2- 172.21.15.122:0/737734092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 0x7f0f740cf1a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 --2- 172.21.15.122:0/737734092 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f740b2d60 0x7f0f7415f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- 172.21.15.122:0/737734092 >> 172.21.15.122:0/737734092 conn(0x7f0f74071420 msgr2=0x7f0f7406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:58.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- 172.21.15.122:0/737734092 shutdown_connections 2024-09-17T05:33:58.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- 172.21.15.122:0/737734092 wait complete. 2024-09-17T05:33:58.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 Processor -- start 2024-09-17T05:33:58.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- start start 2024-09-17T05:33:58.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f740b2d60 0x7f0f741fac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:58.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 0x7f0f740cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:58.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f741584d0 0x7f0f741f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:58.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7416ad20 con 0x7f0f740ced30 2024-09-17T05:33:58.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7416aba0 con 0x7f0f741584d0 2024-09-17T05:33:58.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f79600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7416aea0 con 0x7f0f740b2d60 2024-09-17T05:33:58.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f6a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 0x7f0f740cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f6a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 0x7f0f740cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47194/0 (socket says 172.21.15.122:47194) 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f6a000640 1 -- 172.21.15.122:0/3653826983 learned_addr learned my addr 172.21.15.122:0/3653826983 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f72a00640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f740b2d60 0x7f0f741fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.463+0000 7f0f73400640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f741584d0 0x7f0f741f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6a000640 1 -- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f740b2d60 msgr2=0x7f0f741fac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6a000640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f740b2d60 0x7f0f741fac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6a000640 1 -- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f741584d0 msgr2=0x7f0f741f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6a000640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f741584d0 0x7f0f741f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6a000640 1 -- 172.21.15.122:0/3653826983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f741fb3b0 con 0x7f0f740ced30 2024-09-17T05:33:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6a000640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 0x7f0f740cd250 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00d9b0 tx=0x7f0f5c00de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6b400640 1 -- 172.21.15.122:0/3653826983 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c017070 con 0x7f0f740ced30 2024-09-17T05:33:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6b400640 1 -- 172.21.15.122:0/3653826983 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f5c004310 con 0x7f0f740ced30 2024-09-17T05:33:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6b400640 1 -- 172.21.15.122:0/3653826983 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c009c30 con 0x7f0f740ced30 2024-09-17T05:33:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f741fb690 con 0x7f0f740ced30 2024-09-17T05:33:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f741f7880 con 0x7f0f740ced30 2024-09-17T05:33:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f741fbbb0 con 0x7f0f740ced30 2024-09-17T05:33:58.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6b400640 1 -- 172.21.15.122:0/3653826983 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f0f5c003600 con 0x7f0f740ced30 2024-09-17T05:33:58.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6b400640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f50074370 0x7f0f50076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:58.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.467+0000 7f0f6b400640 1 -- 172.21.15.122:0/3653826983 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5937+0+0 (secure 0 0 0) 0x7f0f5c093190 con 0x7f0f740ced30 2024-09-17T05:33:58.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.471+0000 7f0f72a00640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f50074370 0x7f0f50076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:58.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.471+0000 7f0f72a00640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f50074370 0x7f0f50076830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0f64001e40 tx=0x7f0f64009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:58.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.471+0000 7f0f6b400640 1 -- 172.21.15.122:0/3653826983 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0f5c05f2f0 con 0x7f0f740ced30 2024-09-17T05:33:58.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:58.675+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0f7415f600 con 0x7f0f740ced30 2024-09-17T05:33:59.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.295+0000 7f0f6b400640 1 -- 172.21.15.122:0/3653826983 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f0f5c063d20 con 0x7f0f740ced30 2024-09-17T05:33:59.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.295+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f50074370 msgr2=0x7f0f50076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.295+0000 7f0f79600640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f50074370 0x7f0f50076830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0f64001e40 tx=0x7f0f64009040 comp rx=0 tx=0).stop 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.295+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 msgr2=0x7f0f740cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.295+0000 7f0f79600640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 0x7f0f740cd250 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00d9b0 tx=0x7f0f5c00de70 comp rx=0 tx=0).stop 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.299+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 shutdown_connections 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.299+0000 7f0f79600640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f50074370 0x7f0f50076830 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.299+0000 7f0f79600640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f741584d0 0x7f0f741f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.299+0000 7f0f79600640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f740ced30 0x7f0f740cd250 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.299+0000 7f0f79600640 1 --2- 172.21.15.122:0/3653826983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f740b2d60 0x7f0f741fac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.299+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 >> 172.21.15.122:0/3653826983 conn(0x7f0f74071420 msgr2=0x7f0f74154130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.299+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 shutdown_connections 2024-09-17T05:33:59.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.299+0000 7f0f79600640 1 -- 172.21.15.122:0/3653826983 wait complete. 2024-09-17T05:33:59.315 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:33:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 -- 172.21.15.122:0/3798980729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415c1f0 msgr2=0x7f6e240cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 --2- 172.21.15.122:0/3798980729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415c1f0 0x7f6e240cec40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6e24073e00 tx=0x7f6e0c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 -- 172.21.15.122:0/3798980729 shutdown_connections 2024-09-17T05:33:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 --2- 172.21.15.122:0/3798980729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e240b0b60 0x7f6e240cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 --2- 172.21.15.122:0/3798980729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415c1f0 0x7f6e240cec40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 --2- 172.21.15.122:0/3798980729 >> v2:172.21.15.173:3300/0 conn(0x7f6e2415be10 0x7f6e240ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 -- 172.21.15.122:0/3798980729 >> 172.21.15.122:0/3798980729 conn(0x7f6e240d1da0 msgr2=0x7f6e240d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:33:59.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 -- 172.21.15.122:0/3798980729 shutdown_connections 2024-09-17T05:33:59.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.507+0000 7f6e2b000640 1 -- 172.21.15.122:0/3798980729 wait complete. 2024-09-17T05:33:59.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 Processor -- start 2024-09-17T05:33:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 -- start start 2024-09-17T05:33:59.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e240b0b60 0x7f6e24166780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:59.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415be10 0x7f6e2414b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:59.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2415c1f0 0x7f6e240af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:59.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e240ccf60 con 0x7f6e2415be10 2024-09-17T05:33:59.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e240ccde0 con 0x7f6e240b0b60 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6e240cd0e0 con 0x7f6e2415c1f0 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415be10 0x7f6e2414b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415be10 0x7f6e2414b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47218/0 (socket says 172.21.15.122:47218) 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 -- 172.21.15.122:0/986833076 learned_addr learned my addr 172.21.15.122:0/986833076 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e23e00640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e240b0b60 0x7f6e24166780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e29000640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2415c1f0 0x7f6e240af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 -- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2415c1f0 msgr2=0x7f6e240af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2415c1f0 0x7f6e240af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 -- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e240b0b60 msgr2=0x7f6e24166780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e240b0b60 0x7f6e24166780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 -- 172.21.15.122:0/986833076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2406f310 con 0x7f6e2415be10 2024-09-17T05:33:59.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e1b400640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415be10 0x7f6e2414b6a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c01af70 tx=0x7f6e0c0a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:59.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e20c00640 1 -- 172.21.15.122:0/986833076 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c0ba020 con 0x7f6e2415be10 2024-09-17T05:33:59.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e20c00640 1 -- 172.21.15.122:0/986833076 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e0c0043b0 con 0x7f6e2415be10 2024-09-17T05:33:59.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e20c00640 1 -- 172.21.15.122:0/986833076 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c0b2040 con 0x7f6e2415be10 2024-09-17T05:33:59.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2406f5b0 con 0x7f6e2415be10 2024-09-17T05:33:59.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2406fb70 con 0x7f6e2415be10 2024-09-17T05:33:59.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.511+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e240cd590 con 0x7f6e2415be10 2024-09-17T05:33:59.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.515+0000 7f6e20c00640 1 -- 172.21.15.122:0/986833076 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f6e0c005290 con 0x7f6e2415be10 2024-09-17T05:33:59.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.515+0000 7f6e20c00640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e00074370 0x7f6e00076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:33:59.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.515+0000 7f6e20c00640 1 -- 172.21.15.122:0/986833076 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5937+0+0 (secure 0 0 0) 0x7f6e0c12ca40 con 0x7f6e2415be10 2024-09-17T05:33:59.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.515+0000 7f6e23e00640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e00074370 0x7f6e00076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:33:59.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.515+0000 7f6e23e00640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e00074370 0x7f6e00076830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6e14005d90 tx=0x7f6e14005c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:33:59.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.519+0000 7f6e20c00640 1 -- 172.21.15.122:0/986833076 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6e0c0164b0 con 0x7f6e2415be10 2024-09-17T05:33:59.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:33:59.691+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6e2414bbe0 con 0x7f6e2415be10 2024-09-17T05:34:00.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.295+0000 7f6e20c00640 1 -- 172.21.15.122:0/986833076 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) ==== 181+0+0 (secure 0 0 0) 0x7f6e0c0f9540 con 0x7f6e2415be10 2024-09-17T05:34:00.294 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:34:00.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.299+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e00074370 msgr2=0x7f6e00076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:00.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.299+0000 7f6e2b000640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e00074370 0x7f6e00076830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6e14005d90 tx=0x7f6e14005c60 comp rx=0 tx=0).stop 2024-09-17T05:34:00.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.299+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415be10 msgr2=0x7f6e2414b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:00.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.299+0000 7f6e2b000640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415be10 0x7f6e2414b6a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c01af70 tx=0x7f6e0c0a5630 comp rx=0 tx=0).stop 2024-09-17T05:34:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.303+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 shutdown_connections 2024-09-17T05:34:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.303+0000 7f6e2b000640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e00074370 0x7f6e00076830 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.303+0000 7f6e2b000640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e2415c1f0 0x7f6e240af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.303+0000 7f6e2b000640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e2415be10 0x7f6e2414b6a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.303+0000 7f6e2b000640 1 --2- 172.21.15.122:0/986833076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e240b0b60 0x7f6e24166780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.303+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 >> 172.21.15.122:0/986833076 conn(0x7f6e240d1da0 msgr2=0x7f6e240afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:00.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.303+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 shutdown_connections 2024-09-17T05:34:00.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.303+0000 7f6e2b000640 1 -- 172.21.15.122:0/986833076 wait complete. 2024-09-17T05:34:00.322 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:34:00.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.471+0000 7f7ee8000640 1 -- 172.21.15.122:0/1081367488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec8004d50 msgr2=0x7f7ec80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:00.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.471+0000 7f7ee8000640 1 --2- 172.21.15.122:0/1081367488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec8004d50 0x7f7ec80051d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009f10 tx=0x7f7ed801d5d0 comp rx=0 tx=0).stop 2024-09-17T05:34:00.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.471+0000 7f7ee6000640 1 -- 172.21.15.122:0/1081367488 reap_dead start 2024-09-17T05:34:00.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.471+0000 7f7ee8000640 1 -- 172.21.15.122:0/1081367488 shutdown_connections 2024-09-17T05:34:00.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.471+0000 7f7ee8000640 1 -- 172.21.15.122:0/1081367488 >> 172.21.15.122:0/1081367488 conn(0x7f7ee00cfbc0 msgr2=0x7f7ee00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:00.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.471+0000 7f7ee8000640 1 -- 172.21.15.122:0/1081367488 shutdown_connections 2024-09-17T05:34:00.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.471+0000 7f7ee8000640 1 -- 172.21.15.122:0/1081367488 wait complete. 2024-09-17T05:34:00.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.471+0000 7f7ee8000640 1 Processor -- start 2024-09-17T05:34:00.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 -- start start 2024-09-17T05:34:00.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ec8004d50 0x7f7ee014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:00.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ee0159480 0x7f7ee0159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:00.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee00b47f0 0x7f7ee00b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee00ce610 con 0x7f7ee00b47f0 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee00ce490 con 0x7f7ec8004d50 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee00ce790 con 0x7f7ee0159480 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee00b47f0 0x7f7ee00b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee00b47f0 0x7f7ee00b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47242/0 (socket says 172.21.15.122:47242) 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 -- 172.21.15.122:0/889470363 learned_addr learned my addr 172.21.15.122:0/889470363 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee5600640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ec8004d50 0x7f7ee014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee4c00640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ee0159480 0x7f7ee0159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 -- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ee0159480 msgr2=0x7f7ee0159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ee0159480 0x7f7ee0159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 -- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ec8004d50 msgr2=0x7f7ee014f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:00.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ec8004d50 0x7f7ee014f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:00.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 -- 172.21.15.122:0/889470363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee01f49a0 con 0x7f7ee00b47f0 2024-09-17T05:34:00.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee6000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee00b47f0 0x7f7ee00b4c70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009f10 tx=0x7f7ed8009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:00.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7edd600640 1 -- 172.21.15.122:0/889470363 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8009200 con 0x7f7ee00b47f0 2024-09-17T05:34:00.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7edd600640 1 -- 172.21.15.122:0/889470363 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ed80a9510 con 0x7f7ee00b47f0 2024-09-17T05:34:00.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7edd600640 1 -- 172.21.15.122:0/889470363 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80a9d10 con 0x7f7ee00b47f0 2024-09-17T05:34:00.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee01f4c40 con 0x7f7ee00b47f0 2024-09-17T05:34:00.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee01f5200 con 0x7f7ee00b47f0 2024-09-17T05:34:00.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.475+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee0069410 con 0x7f7ee00b47f0 2024-09-17T05:34:00.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.479+0000 7f7edd600640 1 -- 172.21.15.122:0/889470363 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f7ed80a7040 con 0x7f7ee00b47f0 2024-09-17T05:34:00.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.479+0000 7f7edd600640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec0074370 0x7f7ec0076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:00.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.479+0000 7f7edd600640 1 -- 172.21.15.122:0/889470363 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5442+0+0 (secure 0 0 0) 0x7f7ed812d9e0 con 0x7f7ee00b47f0 2024-09-17T05:34:00.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.479+0000 7f7ee5600640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec0074370 0x7f7ec0076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:00.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.479+0000 7f7ee5600640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec0074370 0x7f7ec0076830 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4001870 tx=0x7f7ed4006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:00.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.483+0000 7f7edd600640 1 -- 172.21.15.122:0/889470363 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7ed80ad030 con 0x7f7ee00b47f0 2024-09-17T05:34:00.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:00.651+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7ee00cdac0 con 0x7f7ee00b47f0 2024-09-17T05:34:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.303+0000 7f7edd600640 1 -- 172.21.15.122:0/889470363 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) ==== 169+0+0 (secure 0 0 0) 0x7f7ed8017320 con 0x7f7ee00b47f0 2024-09-17T05:34:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec0074370 msgr2=0x7f7ec0076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec0074370 0x7f7ec0076830 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4001870 tx=0x7f7ed4006040 comp rx=0 tx=0).stop 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee00b47f0 msgr2=0x7f7ee00b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee00b47f0 0x7f7ee00b4c70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009f10 tx=0x7f7ed8009480 comp rx=0 tx=0).stop 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 shutdown_connections 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec0074370 0x7f7ec0076830 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ee00b47f0 0x7f7ee00b4c70 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ee0159480 0x7f7ee0159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 --2- 172.21.15.122:0/889470363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ec8004d50 0x7f7ee014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 >> 172.21.15.122:0/889470363 conn(0x7f7ee00cfbc0 msgr2=0x7f7ee015d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 shutdown_connections 2024-09-17T05:34:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.307+0000 7f7ee8000640 1 -- 172.21.15.122:0/889470363 wait complete. 2024-09-17T05:34:01.338 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:34:01.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.487+0000 7f3c59e00640 1 -- 172.21.15.122:0/1287291908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c540b2e80 msgr2=0x7f3c540b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.487+0000 7f3c59e00640 1 --2- 172.21.15.122:0/1287291908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c540b2e80 0x7f3c540b1a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009840 tx=0x7f3c3c01c5f0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.487+0000 7f3c59e00640 1 -- 172.21.15.122:0/1287291908 shutdown_connections 2024-09-17T05:34:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.487+0000 7f3c59e00640 1 --2- 172.21.15.122:0/1287291908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c540b2e80 0x7f3c540b1a00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.487+0000 7f3c59e00640 1 --2- 172.21.15.122:0/1287291908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c5415f490 0x7f3c540b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.487+0000 7f3c59e00640 1 -- 172.21.15.122:0/1287291908 >> 172.21.15.122:0/1287291908 conn(0x7f3c540d1e30 msgr2=0x7f3c540d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.487+0000 7f3c59e00640 1 -- 172.21.15.122:0/1287291908 shutdown_connections 2024-09-17T05:34:01.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.487+0000 7f3c59e00640 1 -- 172.21.15.122:0/1287291908 wait complete. 2024-09-17T05:34:01.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 Processor -- start 2024-09-17T05:34:01.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 -- start start 2024-09-17T05:34:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c540b1f40 0x7f3c540c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c540b2e80 0x7f3c540b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5415f490 0x7f3c540b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54168e00 con 0x7f3c540b1f40 2024-09-17T05:34:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54168c80 con 0x7f3c5415f490 2024-09-17T05:34:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3c54168f80 con 0x7f3c540b2e80 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c540b1f40 0x7f3c540c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c540b1f40 0x7f3c540c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37258/0 (socket says 172.21.15.122:37258) 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 -- 172.21.15.122:0/3370087417 learned_addr learned my addr 172.21.15.122:0/3370087417 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c4be00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c540b2e80 0x7f3c540b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c53400640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5415f490 0x7f3c540b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 -- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c540b2e80 msgr2=0x7f3c540b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c540b2e80 0x7f3c540b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 -- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5415f490 msgr2=0x7f3c540b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5415f490 0x7f3c540b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 -- 172.21.15.122:0/3370087417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c540b4c60 con 0x7f3c540b1f40 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c53400640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5415f490 0x7f3c540b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c52a00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c540b1f40 0x7f3c540c6370 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400e7d0 tx=0x7f3c4400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c4b400640 1 -- 172.21.15.122:0/3370087417 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4400ccc0 con 0x7f3c540b1f40 2024-09-17T05:34:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c4b400640 1 -- 172.21.15.122:0/3370087417 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c44010430 con 0x7f3c540b1f40 2024-09-17T05:34:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c4b400640 1 -- 172.21.15.122:0/3370087417 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44010860 con 0x7f3c540b1f40 2024-09-17T05:34:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c540b3ec0 con 0x7f3c540b1f40 2024-09-17T05:34:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.491+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c540c6c70 con 0x7f3c540b1f40 2024-09-17T05:34:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.495+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c540cfda0 con 0x7f3c540b1f40 2024-09-17T05:34:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.495+0000 7f3c4b400640 1 -- 172.21.15.122:0/3370087417 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f3c44010a00 con 0x7f3c540b1f40 2024-09-17T05:34:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.495+0000 7f3c4b400640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c30074370 0x7f3c30076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.495+0000 7f3c4b400640 1 -- 172.21.15.122:0/3370087417 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7f3c44093a90 con 0x7f3c540b1f40 2024-09-17T05:34:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.495+0000 7f3c4be00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c30074370 0x7f3c30076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.495+0000 7f3c4be00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c30074370 0x7f3c30076830 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0af000 tx=0x7f3c3c01ae30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.499+0000 7f3c4b400640 1 -- 172.21.15.122:0/3370087417 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3c4405fbf0 con 0x7f3c540b1f40 2024-09-17T05:34:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.675+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f3c540af1e0 con 0x7f3c540b1f40 2024-09-17T05:34:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.679+0000 7f3c4b400640 1 -- 172.21.15.122:0/3370087417 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v40) ==== 71+0+0 (secure 0 0 0) 0x7f3c44064620 con 0x7f3c540b1f40 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c30074370 msgr2=0x7f3c30076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c30074370 0x7f3c30076830 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0af000 tx=0x7f3c3c01ae30 comp rx=0 tx=0).stop 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c540b1f40 msgr2=0x7f3c540c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c540b1f40 0x7f3c540c6370 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400e7d0 tx=0x7f3c4400ec90 comp rx=0 tx=0).stop 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 shutdown_connections 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3c30074370 0x7f3c30076830 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c5415f490 0x7f3c540b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c540b2e80 0x7f3c540b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 --2- 172.21.15.122:0/3370087417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c540b1f40 0x7f3c540c6370 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 >> 172.21.15.122:0/3370087417 conn(0x7f3c540d1e30 msgr2=0x7f3c54157460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 shutdown_connections 2024-09-17T05:34:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.683+0000 7f3c59e00640 1 -- 172.21.15.122:0/3370087417 wait complete. 2024-09-17T05:34:01.698 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-17T05:34:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.847+0000 7f420aa00640 1 -- 172.21.15.122:0/1028552626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 msgr2=0x7f42040cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.847+0000 7f420aa00640 1 --2- 172.21.15.122:0/1028552626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 0x7f42040cce50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009840 tx=0x7f41f001c820 comp rx=0 tx=0).stop 2024-09-17T05:34:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.847+0000 7f420aa00640 1 -- 172.21.15.122:0/1028552626 shutdown_connections 2024-09-17T05:34:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.847+0000 7f420aa00640 1 --2- 172.21.15.122:0/1028552626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 0x7f42040cce50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.847+0000 7f420aa00640 1 --2- 172.21.15.122:0/1028552626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f420415a240 0x7f420415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.847+0000 7f420aa00640 1 -- 172.21.15.122:0/1028552626 >> 172.21.15.122:0/1028552626 conn(0x7f4204073490 msgr2=0x7f42040d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.847+0000 7f420aa00640 1 -- 172.21.15.122:0/1028552626 shutdown_connections 2024-09-17T05:34:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.847+0000 7f420aa00640 1 -- 172.21.15.122:0/1028552626 wait complete. 2024-09-17T05:34:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 Processor -- start 2024-09-17T05:34:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 -- start start 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4204074340 0x7f42041f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 0x7f42041541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f420415a240 0x7f4204154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4204165550 con 0x7f42040cca70 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f42041653d0 con 0x7f4204074340 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f42041656d0 con 0x7f420415a240 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 0x7f42041541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 0x7f42041541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37274/0 (socket says 172.21.15.122:37274) 2024-09-17T05:34:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 -- 172.21.15.122:0/2258334539 learned_addr learned my addr 172.21.15.122:0/2258334539 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4209600640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4204074340 0x7f42041f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420a000640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f420415a240 0x7f4204154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 -- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f420415a240 msgr2=0x7f4204154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f420415a240 0x7f4204154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 -- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4204074340 msgr2=0x7f42041f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4204074340 0x7f42041f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 -- 172.21.15.122:0/2258334539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42041f7c60 con 0x7f42040cca70 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4209600640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4204074340 0x7f42041f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4208c00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 0x7f42041541c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009790 tx=0x7f41f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4201600640 1 -- 172.21.15.122:0/2258334539 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00042f0 con 0x7f42040cca70 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4201600640 1 -- 172.21.15.122:0/2258334539 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41f0005030 con 0x7f42040cca70 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f4201600640 1 -- 172.21.15.122:0/2258334539 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0005460 con 0x7f42040cca70 2024-09-17T05:34:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42041f7f40 con 0x7f42040cca70 2024-09-17T05:34:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.851+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42041f8500 con 0x7f42040cca70 2024-09-17T05:34:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.855+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4204166380 con 0x7f42040cca70 2024-09-17T05:34:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.855+0000 7f4201600640 1 -- 172.21.15.122:0/2258334539 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f41f00c0020 con 0x7f42040cca70 2024-09-17T05:34:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.855+0000 7f4201600640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f41e4074370 0x7f41e4076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.855+0000 7f4201600640 1 -- 172.21.15.122:0/2258334539 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7f41f012d5b0 con 0x7f42040cca70 2024-09-17T05:34:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.855+0000 7f4209600640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f41e4074370 0x7f41e4076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.855+0000 7f4209600640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f41e4074370 0x7f41e4076830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f42040ae580 tx=0x7f41f8005eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:01.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:01.859+0000 7f4201600640 1 -- 172.21.15.122:0/2258334539 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f41f00c7020 con 0x7f42040cca70 2024-09-17T05:34:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.083+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f42040d09d0 con 0x7f42040cca70 2024-09-17T05:34:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f4201600640 1 -- 172.21.15.122:0/2258334539 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v398) ==== 167+0+0 (secure 0 0 0) 0x7f41f0016410 con 0x7f42040cca70 2024-09-17T05:34:02.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f41e4074370 msgr2=0x7f41e4076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f41e4074370 0x7f41e4076830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f42040ae580 tx=0x7f41f8005eb0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 msgr2=0x7f42041541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 0x7f42041541c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009790 tx=0x7f41f0004040 comp rx=0 tx=0).stop 2024-09-17T05:34:02.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 shutdown_connections 2024-09-17T05:34:02.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f41e4074370 0x7f41e4076830 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f420415a240 0x7f4204154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f42040cca70 0x7f42041541c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 --2- 172.21.15.122:0/2258334539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4204074340 0x7f42041f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 >> 172.21.15.122:0/2258334539 conn(0x7f4204073490 msgr2=0x7f42040b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:02.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 shutdown_connections 2024-09-17T05:34:02.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.307+0000 7f420aa00640 1 -- 172.21.15.122:0/2258334539 wait complete. 2024-09-17T05:34:02.325 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed in output of "ceph status" ... ok 2024-09-17T05:34:02.325 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-09-17T05:34:02.325 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:34:02.326 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:02.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47c800640 1 -- 172.21.15.122:0/4245981846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc45c004da0 msgr2=0x7fc45c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47c800640 1 --2- 172.21.15.122:0/4245981846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc45c004da0 0x7fc45c005220 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc470009e00 tx=0x7fc47001d130 comp rx=0 tx=0).stop 2024-09-17T05:34:02.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47a800640 1 -- 172.21.15.122:0/4245981846 reap_dead start 2024-09-17T05:34:02.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47c800640 1 -- 172.21.15.122:0/4245981846 shutdown_connections 2024-09-17T05:34:02.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47c800640 1 -- 172.21.15.122:0/4245981846 >> 172.21.15.122:0/4245981846 conn(0x7fc47406dd30 msgr2=0x7fc47406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:02.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47c800640 1 -- 172.21.15.122:0/4245981846 shutdown_connections 2024-09-17T05:34:02.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47c800640 1 -- 172.21.15.122:0/4245981846 wait complete. 2024-09-17T05:34:02.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47c800640 1 Processor -- start 2024-09-17T05:34:02.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.471+0000 7fc47c800640 1 -- start start 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc45c004da0 0x7fc47415d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4740cd2c0 0x7fc4740cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4740cd740 0x7fc4740d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc47406f830 con 0x7fc4740cd740 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc47406f6b0 con 0x7fc45c004da0 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc47406f9b0 con 0x7fc4740cd2c0 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4740cd740 0x7fc4740d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4740cd740 0x7fc4740d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37304/0 (socket says 172.21.15.122:37304) 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 -- 172.21.15.122:0/1581083506 learned_addr learned my addr 172.21.15.122:0/1581083506 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc479e00640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc45c004da0 0x7fc47415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:02.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc479400640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4740cd2c0 0x7fc4740cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:02.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 -- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4740cd2c0 msgr2=0x7fc4740cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4740cd2c0 0x7fc4740cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 -- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc45c004da0 msgr2=0x7fc47415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc45c004da0 0x7fc47415d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 -- 172.21.15.122:0/1581083506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47415dbc0 con 0x7fc4740cd740 2024-09-17T05:34:02.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47a800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4740cd740 0x7fc4740d02c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc4700af040 tx=0x7fc4700addc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc46e000640 1 -- 172.21.15.122:0/1581083506 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700b1040 con 0x7fc4740cd740 2024-09-17T05:34:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc46e000640 1 -- 172.21.15.122:0/1581083506 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4700bc070 con 0x7fc4740cd740 2024-09-17T05:34:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc46e000640 1 -- 172.21.15.122:0/1581083506 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700ade30 con 0x7fc4740cd740 2024-09-17T05:34:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741f5390 con 0x7fc4740cd740 2024-09-17T05:34:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4740cf2a0 con 0x7fc4740cd740 2024-09-17T05:34:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474074010 con 0x7fc4740cd740 2024-09-17T05:34:02.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.475+0000 7fc46e000640 1 -- 172.21.15.122:0/1581083506 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fc470004a50 con 0x7fc4740cd740 2024-09-17T05:34:02.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.479+0000 7fc46e000640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4580743a0 0x7fc458076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:02.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.479+0000 7fc46e000640 1 -- 172.21.15.122:0/1581083506 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7fc47012de10 con 0x7fc4740cd740 2024-09-17T05:34:02.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.479+0000 7fc479e00640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4580743a0 0x7fc458076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:02.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.479+0000 7fc479e00640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4580743a0 0x7fc458076860 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c10 tx=0x7fc4680011a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:02.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.483+0000 7fc46e000640 1 -- 172.21.15.122:0/1581083506 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc4700a5e20 con 0x7fc4740cd740 2024-09-17T05:34:02.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.651+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc474161350 con 0x7fc4740cd740 2024-09-17T05:34:02.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.651+0000 7fc46e000640 1 -- 172.21.15.122:0/1581083506 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+13182 (secure 0 0 0) 0x7fc470016e40 con 0x7fc4740cd740 2024-09-17T05:34:02.648 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:02.649 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":82,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:34:01.299928+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6835/2505208774":"2024-09-18T05:33:58.289177+0000","172.21.15.122:6834/2505208774":"2024-09-18T05:33:58.289177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:02.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4580743a0 msgr2=0x7fc458076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4580743a0 0x7fc458076860 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc468000c10 tx=0x7fc4680011a0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4740cd740 msgr2=0x7fc4740d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4740cd740 0x7fc4740d02c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc4700af040 tx=0x7fc4700addc0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 shutdown_connections 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4580743a0 0x7fc458076860 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4740cd740 0x7fc4740d02c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4740cd2c0 0x7fc4740cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 --2- 172.21.15.122:0/1581083506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc45c004da0 0x7fc47415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 >> 172.21.15.122:0/1581083506 conn(0x7fc47406dd30 msgr2=0x7fc4740b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 shutdown_connections 2024-09-17T05:34:02.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.655+0000 7fc47c800640 1 -- 172.21.15.122:0/1581083506 wait complete. 2024-09-17T05:34:02.670 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:34:02.670 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:34:02.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- 172.21.15.122:0/66121652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a780b1650 msgr2=0x7f3a780b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 --2- 172.21.15.122:0/66121652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a780b1650 0x7f3a780b1a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009900 tx=0x7f3a6001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- 172.21.15.122:0/66121652 shutdown_connections 2024-09-17T05:34:02.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 --2- 172.21.15.122:0/66121652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a780b1650 0x7f3a780b1a30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 --2- 172.21.15.122:0/66121652 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a781605d0 0x7f3a780d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- 172.21.15.122:0/66121652 >> 172.21.15.122:0/66121652 conn(0x7f3a780d1dd0 msgr2=0x7f3a780d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:02.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- 172.21.15.122:0/66121652 shutdown_connections 2024-09-17T05:34:02.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- 172.21.15.122:0/66121652 wait complete. 2024-09-17T05:34:02.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 Processor -- start 2024-09-17T05:34:02.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- start start 2024-09-17T05:34:02.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a780aeb00 0x7f3a7815f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:02.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a780b1650 0x7f3a7815dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:02.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a781605d0 0x7f3a7806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:02.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a780b0810 con 0x7f3a780b1650 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a780b0ee0 con 0x7f3a780aeb00 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a7e400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3a780b1220 con 0x7f3a781605d0 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a6ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a780b1650 0x7f3a7815dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a6ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a780b1650 0x7f3a7815dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37312/0 (socket says 172.21.15.122:37312) 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.819+0000 7f3a6ea00640 1 -- 172.21.15.122:0/731558947 learned_addr learned my addr 172.21.15.122:0/731558947 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a77400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a780aeb00 0x7f3a7815f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a77e00640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a781605d0 0x7f3a7806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6ea00640 1 -- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a781605d0 msgr2=0x7f3a7806f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6ea00640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a781605d0 0x7f3a7806f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6ea00640 1 -- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a780aeb00 msgr2=0x7f3a7815f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6ea00640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a780aeb00 0x7f3a7815f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6ea00640 1 -- 172.21.15.122:0/731558947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a781ff480 con 0x7f3a780b1650 2024-09-17T05:34:02.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a77e00640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a781605d0 0x7f3a7806f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:02.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6ea00640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a780b1650 0x7f3a7815dfe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3a6001a510 tx=0x7f3a600a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:02.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6fe00640 1 -- 172.21.15.122:0/731558947 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600b2040 con 0x7f3a780b1650 2024-09-17T05:34:02.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6fe00640 1 -- 172.21.15.122:0/731558947 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a60004470 con 0x7f3a780b1650 2024-09-17T05:34:02.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a781ff760 con 0x7f3a780b1650 2024-09-17T05:34:02.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6fe00640 1 -- 172.21.15.122:0/731558947 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60005290 con 0x7f3a780b1650 2024-09-17T05:34:02.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a781ffd20 con 0x7f3a780b1650 2024-09-17T05:34:02.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a780b1a30 con 0x7f3a780b1650 2024-09-17T05:34:02.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.823+0000 7f3a6fe00640 1 -- 172.21.15.122:0/731558947 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f3a60005430 con 0x7f3a780b1650 2024-09-17T05:34:02.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.827+0000 7f3a6fe00640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a54074370 0x7f3a54076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:02.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.827+0000 7f3a6fe00640 1 -- 172.21.15.122:0/731558947 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7f3a6012c480 con 0x7f3a780b1650 2024-09-17T05:34:02.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.827+0000 7f3a77400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a54074370 0x7f3a54076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:02.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.827+0000 7f3a77400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a54074370 0x7f3a54076830 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009b40 tx=0x7f3a680091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:02.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.831+0000 7f3a6fe00640 1 -- 172.21.15.122:0/731558947 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3a600164d0 con 0x7f3a780b1650 2024-09-17T05:34:02.984 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:34:02.988 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:34:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a780d1080 con 0x7f3a780b1650 2024-09-17T05:34:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a6fe00640 1 -- 172.21.15.122:0/731558947 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f3a600f88b0 con 0x7f3a780b1650 2024-09-17T05:34:02.995 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:34:02.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a54074370 msgr2=0x7f3a54076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a54074370 0x7f3a54076830 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009b40 tx=0x7f3a680091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a780b1650 msgr2=0x7f3a7815dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a780b1650 0x7f3a7815dfe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3a6001a510 tx=0x7f3a600a55a0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 shutdown_connections 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3a54074370 0x7f3a54076830 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a781605d0 0x7f3a7806f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a780b1650 0x7f3a7815dfe0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 --2- 172.21.15.122:0/731558947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a780aeb00 0x7f3a7815f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 >> 172.21.15.122:0/731558947 conn(0x7f3a780d1dd0 msgr2=0x7f3a78156970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 shutdown_connections 2024-09-17T05:34:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:02.999+0000 7f3a7e400640 1 -- 172.21.15.122:0/731558947 wait complete. 2024-09-17T05:34:03.018 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-17T05:34:03.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.215+0000 7fe243600640 1 -- 172.21.15.122:0/3102362058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c0cf400 msgr2=0x7fe23c0cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.215+0000 7fe243600640 1 --2- 172.21.15.122:0/3102362058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c0cf400 0x7fe23c0cc870 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe224009900 tx=0x7fe22401f0d0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 -- 172.21.15.122:0/3102362058 shutdown_connections 2024-09-17T05:34:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 --2- 172.21.15.122:0/3102362058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c0cf400 0x7fe23c0cc870 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 --2- 172.21.15.122:0/3102362058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe23c15bee0 0x7fe23c0cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 -- 172.21.15.122:0/3102362058 >> 172.21.15.122:0/3102362058 conn(0x7fe23c0b18f0 msgr2=0x7fe23c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 -- 172.21.15.122:0/3102362058 shutdown_connections 2024-09-17T05:34:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 -- 172.21.15.122:0/3102362058 wait complete. 2024-09-17T05:34:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 Processor -- start 2024-09-17T05:34:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 -- start start 2024-09-17T05:34:03.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe23c0ccdb0 0x7fe23c153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe23c0cf400 0x7fe23c1f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c15bee0 0x7fe23c140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c1603b0 con 0x7fe23c15bee0 2024-09-17T05:34:03.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c160230 con 0x7fe23c0cf400 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe243600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c160530 con 0x7fe23c0ccdb0 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c15bee0 0x7fe23c140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c15bee0 0x7fe23c140d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37334/0 (socket says 172.21.15.122:37334) 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 -- 172.21.15.122:0/2338776814 learned_addr learned my addr 172.21.15.122:0/2338776814 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe240c00640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe23c0ccdb0 0x7fe23c153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe233e00640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe23c0cf400 0x7fe23c1f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 -- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe23c0ccdb0 msgr2=0x7fe23c153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe23c0ccdb0 0x7fe23c153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 -- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe23c0cf400 msgr2=0x7fe23c1f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe23c0cf400 0x7fe23c1f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:03.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 -- 172.21.15.122:0/2338776814 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c1f6d10 con 0x7fe23c15bee0 2024-09-17T05:34:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe241600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c15bee0 0x7fe23c140d20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c7c0 tx=0x7fe23400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.219+0000 7fe239600640 1 -- 172.21.15.122:0/2338776814 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234003b80 con 0x7fe23c15bee0 2024-09-17T05:34:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe239600640 1 -- 172.21.15.122:0/2338776814 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe234003d20 con 0x7fe23c15bee0 2024-09-17T05:34:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe239600640 1 -- 172.21.15.122:0/2338776814 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23400aa50 con 0x7fe23c15bee0 2024-09-17T05:34:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c154430 con 0x7fe23c15bee0 2024-09-17T05:34:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c1549f0 con 0x7fe23c15bee0 2024-09-17T05:34:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23c069410 con 0x7fe23c15bee0 2024-09-17T05:34:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe239600640 1 -- 172.21.15.122:0/2338776814 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fe23400abf0 con 0x7fe23c15bee0 2024-09-17T05:34:03.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe239600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe218074370 0x7fe218076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:03.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe239600640 1 -- 172.21.15.122:0/2338776814 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7fe234092e00 con 0x7fe23c15bee0 2024-09-17T05:34:03.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.223+0000 7fe240c00640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe218074370 0x7fe218076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.227+0000 7fe240c00640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe218074370 0x7fe218076830 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00bb50 tx=0x7fe22c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:03.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.227+0000 7fe239600640 1 -- 172.21.15.122:0/2338776814 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe23c1f6d10 con 0x7fe23c15bee0 2024-09-17T05:34:03.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:03.459+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7fe23c15fea0 con 0x7fe23c15bee0 2024-09-17T05:34:04.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe239600640 1 -- 172.21.15.122:0/2338776814 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v400) ==== 177+0+0 (secure 0 0 0) 0x7fe23405ef40 con 0x7fe23c15bee0 2024-09-17T05:34:04.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe218074370 msgr2=0x7fe218076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe218074370 0x7fe218076830 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00bb50 tx=0x7fe22c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c15bee0 msgr2=0x7fe23c140d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c15bee0 0x7fe23c140d20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe23400c7c0 tx=0x7fe23400cc80 comp rx=0 tx=0).stop 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 shutdown_connections 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe218074370 0x7fe218076830 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe23c15bee0 0x7fe23c140d20 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe23c0cf400 0x7fe23c1f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 --2- 172.21.15.122:0/2338776814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe23c0ccdb0 0x7fe23c153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 >> 172.21.15.122:0/2338776814 conn(0x7fe23c0b18f0 msgr2=0x7fe23c0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 shutdown_connections 2024-09-17T05:34:04.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.315+0000 7fe243600640 1 -- 172.21.15.122:0/2338776814 wait complete. 2024-09-17T05:34:04.335 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:34:04.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.479+0000 7f8c7e800640 1 -- 172.21.15.122:0/837418845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c5c004da0 msgr2=0x7f8c5c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.479+0000 7f8c7e800640 1 --2- 172.21.15.122:0/837418845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c5c004da0 0x7f8c5c005220 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8c70006fd0 tx=0x7f8c7001ca70 comp rx=0 tx=0).stop 2024-09-17T05:34:04.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.479+0000 7f8c77e00640 1 -- 172.21.15.122:0/837418845 reap_dead start 2024-09-17T05:34:04.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.479+0000 7f8c7e800640 1 -- 172.21.15.122:0/837418845 shutdown_connections 2024-09-17T05:34:04.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.479+0000 7f8c7e800640 1 -- 172.21.15.122:0/837418845 >> 172.21.15.122:0/837418845 conn(0x7f8c780d1ea0 msgr2=0x7f8c780d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.479+0000 7f8c7e800640 1 -- 172.21.15.122:0/837418845 shutdown_connections 2024-09-17T05:34:04.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.479+0000 7f8c7e800640 1 -- 172.21.15.122:0/837418845 wait complete. 2024-09-17T05:34:04.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.479+0000 7f8c7e800640 1 Processor -- start 2024-09-17T05:34:04.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 -- start start 2024-09-17T05:34:04.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c781f7a30 0x7f8c7809bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c781576f0 0x7f8c7809f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c780cb140 0x7f8c780cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c780cc940 con 0x7f8c780cb140 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8c780cf7e0 con 0x7f8c781576f0 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8c780ccac0 con 0x7f8c781f7a30 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c780cb140 0x7f8c780cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c780cb140 0x7f8c780cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37354/0 (socket says 172.21.15.122:37354) 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 -- 172.21.15.122:0/1357992141 learned_addr learned my addr 172.21.15.122:0/1357992141 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c76a00640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c781576f0 0x7f8c7809f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77400640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c781f7a30 0x7f8c7809bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 -- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c781f7a30 msgr2=0x7f8c7809bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c781f7a30 0x7f8c7809bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 -- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c781576f0 msgr2=0x7f8c7809f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c781576f0 0x7f8c7809f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 -- 172.21.15.122:0/1357992141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c780ce070 con 0x7f8c780cb140 2024-09-17T05:34:04.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c77e00640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c780cb140 0x7f8c780cb5c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8c7001b170 tx=0x7f8c70003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:04.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c6f400640 1 -- 172.21.15.122:0/1357992141 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70006cb0 con 0x7f8c780cb140 2024-09-17T05:34:04.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c6f400640 1 -- 172.21.15.122:0/1357992141 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c700a5cf0 con 0x7f8c780cb140 2024-09-17T05:34:04.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7806df80 con 0x7f8c780cb140 2024-09-17T05:34:04.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c6f400640 1 -- 172.21.15.122:0/1357992141 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70006330 con 0x7f8c780cb140 2024-09-17T05:34:04.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7806e540 con 0x7f8c780cb140 2024-09-17T05:34:04.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.483+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c780ce5e0 con 0x7f8c780cb140 2024-09-17T05:34:04.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.487+0000 7f8c6f400640 1 -- 172.21.15.122:0/1357992141 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f8c700064d0 con 0x7f8c780cb140 2024-09-17T05:34:04.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.487+0000 7f8c6f400640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8c58074370 0x7f8c58076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.487+0000 7f8c6f400640 1 -- 172.21.15.122:0/1357992141 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7f8c7012c740 con 0x7f8c780cb140 2024-09-17T05:34:04.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.487+0000 7f8c77400640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8c58074370 0x7f8c58076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.487+0000 7f8c77400640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8c58074370 0x7f8c58076830 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c68003680 tx=0x7f8c68000f60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:04.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.491+0000 7f8c6f400640 1 -- 172.21.15.122:0/1357992141 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8c700167d0 con 0x7f8c780cb140 2024-09-17T05:34:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.659+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f8c780cfaf0 con 0x7f8c780cb140 2024-09-17T05:34:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.659+0000 7f8c6f400640 1 -- 172.21.15.122:0/1357992141 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) ==== 59+0+1998 (secure 0 0 0) 0x7f8c700f9300 con 0x7f8c780cb140 2024-09-17T05:34:04.656 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:34:04.656 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:34:04.656 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:34:04.656 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:34:04.657 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:34:04.658 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:34:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8c58074370 msgr2=0x7f8c58076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8c58074370 0x7f8c58076830 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c68003680 tx=0x7f8c68000f60 comp rx=0 tx=0).stop 2024-09-17T05:34:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c780cb140 msgr2=0x7f8c780cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c780cb140 0x7f8c780cb5c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8c7001b170 tx=0x7f8c70003ef0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 shutdown_connections 2024-09-17T05:34:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8c58074370 0x7f8c58076830 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c780cb140 0x7f8c780cb5c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c781576f0 0x7f8c7809f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 --2- 172.21.15.122:0/1357992141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c781f7a30 0x7f8c7809bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 >> 172.21.15.122:0/1357992141 conn(0x7f8c780d1ea0 msgr2=0x7f8c780d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 shutdown_connections 2024-09-17T05:34:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.663+0000 7f8c7e800640 1 -- 172.21.15.122:0/1357992141 wait complete. 2024-09-17T05:34:04.677 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:34:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.823+0000 7fa081600640 1 -- 172.21.15.122:0/117542431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa07c0cf1d0 msgr2=0x7fa07c160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.823+0000 7fa081600640 1 --2- 172.21.15.122:0/117542431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa07c0cf1d0 0x7fa07c160490 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa068009840 tx=0x7fa06801c7f0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.823+0000 7fa081600640 1 -- 172.21.15.122:0/117542431 shutdown_connections 2024-09-17T05:34:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.823+0000 7fa081600640 1 --2- 172.21.15.122:0/117542431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa07c0cf1d0 0x7fa07c160490 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.823+0000 7fa081600640 1 -- 172.21.15.122:0/117542431 >> 172.21.15.122:0/117542431 conn(0x7fa07c0b18f0 msgr2=0x7fa07c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.823+0000 7fa081600640 1 -- 172.21.15.122:0/117542431 shutdown_connections 2024-09-17T05:34:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.823+0000 7fa081600640 1 -- 172.21.15.122:0/117542431 wait complete. 2024-09-17T05:34:04.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 Processor -- start 2024-09-17T05:34:04.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 -- start start 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa07c06e7a0 0x7fa07c166d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa07c0cf1d0 0x7fa07c0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa07c15f9a0 0x7fa07c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c16a7c0 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa07c16a940 con 0x7fa07c06e7a0 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa07c16aac0 con 0x7fa07c15f9a0 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa07c0cf1d0 0x7fa07c0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa07c0cf1d0 0x7fa07c0b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37370/0 (socket says 172.21.15.122:37370) 2024-09-17T05:34:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 -- 172.21.15.122:0/3584828498 learned_addr learned my addr 172.21.15.122:0/3584828498 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07aa00640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa07c06e7a0 0x7fa07c166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07b400640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa07c15f9a0 0x7fa07c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 -- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa07c15f9a0 msgr2=0x7fa07c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa07c15f9a0 0x7fa07c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 -- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa07c06e7a0 msgr2=0x7fa07c166d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa07c06e7a0 0x7fa07c166d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 -- 172.21.15.122:0/3584828498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07c1fb990 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa07a000640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa07c0cf1d0 0x7fa07c0b44f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa068009790 tx=0x7fa068005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa072a00640 1 -- 172.21.15.122:0/3584828498 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0680b2040 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa072a00640 1 -- 172.21.15.122:0/3584828498 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa068004300 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.827+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07c1fbc70 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.831+0000 7fa072a00640 1 -- 172.21.15.122:0/3584828498 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068005220 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.831+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07c0b4a30 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.831+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07c069410 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.831+0000 7fa072a00640 1 -- 172.21.15.122:0/3584828498 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fa0680a5830 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.831+0000 7fa072a00640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c0743a0 0x7fa05c076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:04.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.831+0000 7fa072a00640 1 -- 172.21.15.122:0/3584828498 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7fa06812cb00 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.831+0000 7fa07aa00640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c0743a0 0x7fa05c076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:04.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.835+0000 7fa07aa00640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c0743a0 0x7fa05c076860 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa06c007fd0 tx=0x7fa06c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:04.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.835+0000 7fa072a00640 1 -- 172.21.15.122:0/3584828498 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa068016410 con 0x7fa07c0cf1d0 2024-09-17T05:34:04.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:04.999+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa07c0ce2e0 con 0x7fa07c0cf1d0 2024-09-17T05:34:05.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.003+0000 7fa072a00640 1 -- 172.21.15.122:0/3584828498 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7fa0680044b0 con 0x7fa07c0cf1d0 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c0743a0 msgr2=0x7fa05c076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c0743a0 0x7fa05c076860 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa06c007fd0 tx=0x7fa06c009040 comp rx=0 tx=0).stop 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa07c0cf1d0 msgr2=0x7fa07c0b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa07c0cf1d0 0x7fa07c0b44f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa068009790 tx=0x7fa068005db0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 shutdown_connections 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c0743a0 0x7fa05c076860 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa07c15f9a0 0x7fa07c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa07c0cf1d0 0x7fa07c0b44f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 --2- 172.21.15.122:0/3584828498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa07c06e7a0 0x7fa07c166d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 >> 172.21.15.122:0/3584828498 conn(0x7fa07c0b18f0 msgr2=0x7fa07c0740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:05.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 shutdown_connections 2024-09-17T05:34:05.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.007+0000 7fa081600640 1 -- 172.21.15.122:0/3584828498 wait complete. 2024-09-17T05:34:05.023 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:34:05.023 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:34:05.100 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:34:05.179 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:34:05.179 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:34:05.179 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:05.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:34:05.187 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:34:05.188 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:05.197 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:34:05.197 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:34:05.197 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:05.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:34:05.238 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:34:05.238 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:05.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:34:05.248 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:34:05.248 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:05.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ee00640 1 -- 172.21.15.122:0/205975953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 msgr2=0x7f936c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ee00640 1 --2- 172.21.15.122:0/205975953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 0x7f936c004810 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9370006f50 tx=0x7f937001cea0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ce00640 1 -- 172.21.15.122:0/205975953 reap_dead start 2024-09-17T05:34:05.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ee00640 1 -- 172.21.15.122:0/205975953 shutdown_connections 2024-09-17T05:34:05.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ee00640 1 -- 172.21.15.122:0/205975953 >> 172.21.15.122:0/205975953 conn(0x7f93880b18f0 msgr2=0x7f93880b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:05.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ee00640 1 -- 172.21.15.122:0/205975953 shutdown_connections 2024-09-17T05:34:05.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ee00640 1 -- 172.21.15.122:0/205975953 wait complete. 2024-09-17T05:34:05.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ee00640 1 Processor -- start 2024-09-17T05:34:05.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.427+0000 7f938ee00640 1 -- start start 2024-09-17T05:34:05.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 0x7f93881f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93880cb600 0x7f93880b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93880b0930 0x7f93881f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93880cdc60 con 0x7f936c003f80 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93880cdae0 con 0x7f93880cb600 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f93880cdde0 con 0x7f93880b0930 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 0x7f93881f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 0x7f93881f0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37388/0 (socket says 172.21.15.122:37388) 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 -- 172.21.15.122:0/3286822768 learned_addr learned my addr 172.21.15.122:0/3286822768 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f937f400640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93880cb600 0x7f93880b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ce00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93880b0930 0x7f93881f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 -- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93880b0930 msgr2=0x7f93881f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93880b0930 0x7f93881f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 -- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93880cb600 msgr2=0x7f93880b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93880cb600 0x7f93880b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 -- 172.21.15.122:0/3286822768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93881f0f90 con 0x7f936c003f80 2024-09-17T05:34:05.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9387e00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 0x7f93881f0a50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f93780115d0 tx=0x7f9378011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:05.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9384c00640 1 -- 172.21.15.122:0/3286822768 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378013720 con 0x7f936c003f80 2024-09-17T05:34:05.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9384c00640 1 -- 172.21.15.122:0/3286822768 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93780138c0 con 0x7f936c003f80 2024-09-17T05:34:05.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f9384c00640 1 -- 172.21.15.122:0/3286822768 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378013cf0 con 0x7f936c003f80 2024-09-17T05:34:05.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93880c50c0 con 0x7f936c003f80 2024-09-17T05:34:05.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93880c5680 con 0x7f936c003f80 2024-09-17T05:34:05.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.431+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93880b0d40 con 0x7f936c003f80 2024-09-17T05:34:05.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.435+0000 7f9384c00640 1 -- 172.21.15.122:0/3286822768 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f9378024440 con 0x7f936c003f80 2024-09-17T05:34:05.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.435+0000 7f9384c00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9364074370 0x7f9364076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.435+0000 7f9384c00640 1 -- 172.21.15.122:0/3286822768 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7f93780966b0 con 0x7f936c003f80 2024-09-17T05:34:05.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.435+0000 7f937f400640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9364074370 0x7f9364076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.435+0000 7f937f400640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9364074370 0x7f9364076830 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f93700ac000 tx=0x7f9370019ce0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:05.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.439+0000 7f9384c00640 1 -- 172.21.15.122:0/3286822768 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f93880b0d40 con 0x7f936c003f80 2024-09-17T05:34:05.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.639+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f938815c460 con 0x7f936c003f80 2024-09-17T05:34:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.639+0000 7f9384c00640 1 -- 172.21.15.122:0/3286822768 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3920 (secure 0 0 0) 0x7f93881f0f90 con 0x7f936c003f80 2024-09-17T05:34:05.637 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:05.637 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":49,"btime":"2024-09-17T05:33:59:291790+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5708,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3900952275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3900952275},{"type":"v1","addr":"172.21.15.173:6836","nonce":3900952275}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5711,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/645422945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":645422945},{"type":"v1","addr":"172.21.15.173:6837","nonce":645422945}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6408,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/1968037162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1968037162},{"type":"v1","addr":"172.21.15.122:6833","nonce":1968037162}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6939,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2529487370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2529487370},{"type":"v1","addr":"172.21.15.122:6835","nonce":2529487370}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-09-17T05:34:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 49 2024-09-17T05:34:05.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9364074370 msgr2=0x7f9364076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9364074370 0x7f9364076830 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f93700ac000 tx=0x7f9370019ce0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 msgr2=0x7f93881f0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 0x7f93881f0a50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f93780115d0 tx=0x7f9378011a90 comp rx=0 tx=0).stop 2024-09-17T05:34:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 shutdown_connections 2024-09-17T05:34:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9364074370 0x7f9364076830 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93880b0930 0x7f93881f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93880cb600 0x7f93880b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 --2- 172.21.15.122:0/3286822768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f936c003f80 0x7f93881f0a50 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 >> 172.21.15.122:0/3286822768 conn(0x7f93880b18f0 msgr2=0x7f938815dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 shutdown_connections 2024-09-17T05:34:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.643+0000 7f938ee00640 1 -- 172.21.15.122:0/3286822768 wait complete. 2024-09-17T05:34:05.658 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:34:05.658 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:34:05.659 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:34:05.659 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:34:05.659 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:34:05.659 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:34:05.659 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:34:05.659 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:34:05.660 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:34:05.660 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:34:05.660 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:34:05.660 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:34:05.660 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:34:05.660 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:34:05.661 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:34:05.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:34:05.671 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:34:05.671 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:34:05.672 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:34:05.672 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:34:05.675 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:34:05.675 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:34:05.676 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:34:05.676 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:34:05.719 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:34:05.719 DEBUG:teuthology.parallel:result is None 2024-09-17T05:34:05.720 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:34:05.720 DEBUG:teuthology.parallel:result is None 2024-09-17T05:34:05.721 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:34:05.721 DEBUG:teuthology.parallel:result is None 2024-09-17T05:34:05.722 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:34:05.722 DEBUG:teuthology.parallel:result is None 2024-09-17T05:34:05.722 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:34:05.807 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:34:05.807 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:34:05.809 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:34:05.810 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:34:05.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.875+0000 7f3e9c800640 1 -- 172.21.15.122:0/2329575371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 msgr2=0x7f3e940ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.875+0000 7f3e9c800640 1 --2- 172.21.15.122:0/2329575371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 0x7f3e940ce670 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009900 tx=0x7f3e8801c850 comp rx=0 tx=0).stop 2024-09-17T05:34:05.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.875+0000 7f3e9c800640 1 -- 172.21.15.122:0/2329575371 shutdown_connections 2024-09-17T05:34:05.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.875+0000 7f3e9c800640 1 --2- 172.21.15.122:0/2329575371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 0x7f3e940ce670 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.875+0000 7f3e9c800640 1 -- 172.21.15.122:0/2329575371 >> 172.21.15.122:0/2329575371 conn(0x7f3e940d1dc0 msgr2=0x7f3e940d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:05.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.875+0000 7f3e9c800640 1 -- 172.21.15.122:0/2329575371 shutdown_connections 2024-09-17T05:34:05.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.875+0000 7f3e9c800640 1 -- 172.21.15.122:0/2329575371 wait complete. 2024-09-17T05:34:05.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 Processor -- start 2024-09-17T05:34:05.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 -- start start 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e940cdb30 0x7f3e9415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e940d28e0 0x7f3e940c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 0x7f3e940cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9416a930 con 0x7f3e9415bea0 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9416a7b0 con 0x7f3e940d28e0 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9416aab0 con 0x7f3e940cdb30 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e99400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e940d28e0 0x7f3e940c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 0x7f3e940cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e99e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e940cdb30 0x7f3e9415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 0x7f3e940cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37444/0 (socket says 172.21.15.122:37444) 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 -- 172.21.15.122:0/2632848983 learned_addr learned my addr 172.21.15.122:0/2632848983 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e99400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e940d28e0 0x7f3e940c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:35478/0 (socket says 172.21.15.122:35478) 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 -- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e940cdb30 msgr2=0x7f3e9415a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e940cdb30 0x7f3e9415a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 -- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e940d28e0 msgr2=0x7f3e940c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e940d28e0 0x7f3e940c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 -- 172.21.15.122:0/2632848983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e940c6620 con 0x7f3e9415bea0 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e99e00640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e940cdb30 0x7f3e9415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9a800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 0x7f3e940cb6f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c7a0 tx=0x7f3e9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e8e000640 1 -- 172.21.15.122:0/2632848983 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e90003b80 con 0x7f3e9415bea0 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e8e000640 1 -- 172.21.15.122:0/2632848983 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e90003d40 con 0x7f3e9415bea0 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e8e000640 1 -- 172.21.15.122:0/2632848983 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000aab0 con 0x7f3e9415bea0 2024-09-17T05:34:05.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e940c6900 con 0x7f3e9415bea0 2024-09-17T05:34:05.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e940cb1d0 con 0x7f3e9415bea0 2024-09-17T05:34:05.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.879+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e940ce670 con 0x7f3e9415bea0 2024-09-17T05:34:05.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.883+0000 7f3e8e000640 1 -- 172.21.15.122:0/2632848983 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f3e9000ac50 con 0x7f3e9415bea0 2024-09-17T05:34:05.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.883+0000 7f3e8e000640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e780743a0 0x7f3e78076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:05.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.883+0000 7f3e8e000640 1 -- 172.21.15.122:0/2632848983 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4999+0+0 (secure 0 0 0) 0x7f3e90092dd0 con 0x7f3e9415bea0 2024-09-17T05:34:05.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.883+0000 7f3e99e00640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e780743a0 0x7f3e78076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:05.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.883+0000 7f3e99e00640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e780743a0 0x7f3e78076860 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3e880af000 tx=0x7f3e8801b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:05.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:05.887+0000 7f3e8e000640 1 -- 172.21.15.122:0/2632848983 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3e9005ef30 con 0x7f3e9415bea0 2024-09-17T05:34:06.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.067+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3e940cb3a0 con 0x7f3e9415bea0 2024-09-17T05:34:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.331+0000 7f3e8e000640 1 -- 172.21.15.122:0/2632848983 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) ==== 97+0+0 (secure 0 0 0) 0x7f3e90063960 con 0x7f3e9415bea0 2024-09-17T05:34:06.328 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:34:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e780743a0 msgr2=0x7f3e78076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e780743a0 0x7f3e78076860 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3e880af000 tx=0x7f3e8801b090 comp rx=0 tx=0).stop 2024-09-17T05:34:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 msgr2=0x7f3e940cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 0x7f3e940cb6f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c7a0 tx=0x7f3e9000cc60 comp rx=0 tx=0).stop 2024-09-17T05:34:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 shutdown_connections 2024-09-17T05:34:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e780743a0 0x7f3e78076860 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e9415bea0 0x7f3e940cb6f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e940d28e0 0x7f3e940c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 --2- 172.21.15.122:0/2632848983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e940cdb30 0x7f3e9415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 >> 172.21.15.122:0/2632848983 conn(0x7f3e940d1dc0 msgr2=0x7f3e940d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 shutdown_connections 2024-09-17T05:34:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.335+0000 7f3e9c800640 1 -- 172.21.15.122:0/2632848983 wait complete. 2024-09-17T05:34:06.352 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:34:06.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 -- 172.21.15.122:0/3394098985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c0744c0 msgr2=0x7efd3c074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 --2- 172.21.15.122:0/3394098985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c0744c0 0x7efd3c074930 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0ae010 tx=0x7efd3401cd80 comp rx=0 tx=0).stop 2024-09-17T05:34:06.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 -- 172.21.15.122:0/3394098985 shutdown_connections 2024-09-17T05:34:06.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 --2- 172.21.15.122:0/3394098985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c0744c0 0x7efd3c074930 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 --2- 172.21.15.122:0/3394098985 >> v2:172.21.15.173:3300/0 conn(0x7efd3c072c90 0x7efd3c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 -- 172.21.15.122:0/3394098985 >> 172.21.15.122:0/3394098985 conn(0x7efd3c0b1c20 msgr2=0x7efd3c0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:06.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 -- 172.21.15.122:0/3394098985 shutdown_connections 2024-09-17T05:34:06.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 -- 172.21.15.122:0/3394098985 wait complete. 2024-09-17T05:34:06.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 Processor -- start 2024-09-17T05:34:06.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.499+0000 7efd43400640 1 -- start start 2024-09-17T05:34:06.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd3c072c90 0x7efd3c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd3c0744c0 0x7efd3c13ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c15c5e0 0x7efd3c1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c1523e0 con 0x7efd3c15c5e0 2024-09-17T05:34:06.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c152260 con 0x7efd3c072c90 2024-09-17T05:34:06.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c152560 con 0x7efd3c0744c0 2024-09-17T05:34:06.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c15c5e0 0x7efd3c1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c15c5e0 0x7efd3c1504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37464/0 (socket says 172.21.15.122:37464) 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 -- 172.21.15.122:0/215476285 learned_addr learned my addr 172.21.15.122:0/215476285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd40a00640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd3c072c90 0x7efd3c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd33e00640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd3c0744c0 0x7efd3c13ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 -- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd3c0744c0 msgr2=0x7efd3c13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd3c0744c0 0x7efd3c13ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 -- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd3c072c90 msgr2=0x7efd3c15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd3c072c90 0x7efd3c15ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 -- 172.21.15.122:0/215476285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c13c430 con 0x7efd3c15c5e0 2024-09-17T05:34:06.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd41400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c15c5e0 0x7efd3c1504d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7efd34019990 tx=0x7efd340092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:06.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd39600640 1 -- 172.21.15.122:0/215476285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd340bb070 con 0x7efd3c15c5e0 2024-09-17T05:34:06.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd39600640 1 -- 172.21.15.122:0/215476285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd34004d40 con 0x7efd3c15c5e0 2024-09-17T05:34:06.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c0cf200 con 0x7efd3c15c5e0 2024-09-17T05:34:06.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd39600640 1 -- 172.21.15.122:0/215476285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd340b2040 con 0x7efd3c15c5e0 2024-09-17T05:34:06.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c13c9b0 con 0x7efd3c15c5e0 2024-09-17T05:34:06.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.503+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c13ccc0 con 0x7efd3c15c5e0 2024-09-17T05:34:06.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.507+0000 7efd39600640 1 -- 172.21.15.122:0/215476285 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7efd340be070 con 0x7efd3c15c5e0 2024-09-17T05:34:06.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.507+0000 7efd39600640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efd180743a0 0x7efd18076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.507+0000 7efd39600640 1 -- 172.21.15.122:0/215476285 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4913+0+0 (secure 0 0 0) 0x7efd3412cba0 con 0x7efd3c15c5e0 2024-09-17T05:34:06.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.507+0000 7efd40a00640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efd180743a0 0x7efd18076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.507+0000 7efd40a00640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efd180743a0 0x7efd18076860 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efd2c007fd0 tx=0x7efd2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:06.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.511+0000 7efd39600640 1 -- 172.21.15.122:0/215476285 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efd34016970 con 0x7efd3c15c5e0 2024-09-17T05:34:06.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efd3c06ec50 con 0x7efd3c15c5e0 2024-09-17T05:34:06.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd39600640 1 -- 172.21.15.122:0/215476285 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7efd340f8c80 con 0x7efd3c15c5e0 2024-09-17T05:34:06.735 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:06.735 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:34:06.735 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:34:06.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:34:06.737 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:34:06.738 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:34:06.739 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.740 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:06.740 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:34:06.740 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.740 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.740 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:34:06.740 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:34:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.742 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:34:06.743 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.745 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow r", 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:34:06.747 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.748 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:34:06.749 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:34:06.749 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:34:06.749 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:34:06.749 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.749 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:34:06.749 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:34:06.750 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.751 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:34:06.752 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:34:06.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efd180743a0 msgr2=0x7efd18076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efd180743a0 0x7efd18076860 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efd2c007fd0 tx=0x7efd2c009040 comp rx=0 tx=0).stop 2024-09-17T05:34:06.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c15c5e0 msgr2=0x7efd3c1504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c15c5e0 0x7efd3c1504d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7efd34019990 tx=0x7efd340092e0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 shutdown_connections 2024-09-17T05:34:06.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efd180743a0 0x7efd18076860 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd3c15c5e0 0x7efd3c1504d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd3c0744c0 0x7efd3c13ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 --2- 172.21.15.122:0/215476285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd3c072c90 0x7efd3c15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 >> 172.21.15.122:0/215476285 conn(0x7efd3c0b1c20 msgr2=0x7efd3c09c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:06.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 shutdown_connections 2024-09-17T05:34:06.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.739+0000 7efd43400640 1 -- 172.21.15.122:0/215476285 wait complete. 2024-09-17T05:34:06.757 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:34:06.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- 172.21.15.122:0/108765647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c0b2e80 msgr2=0x7ff55c0b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 --2- 172.21.15.122:0/108765647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c0b2e80 0x7ff55c0b1a00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff548009900 tx=0x7ff54801c790 comp rx=0 tx=0).stop 2024-09-17T05:34:06.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- 172.21.15.122:0/108765647 shutdown_connections 2024-09-17T05:34:06.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 --2- 172.21.15.122:0/108765647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c0b2e80 0x7ff55c0b1a00 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 --2- 172.21.15.122:0/108765647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff55c15f490 0x7ff55c0b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- 172.21.15.122:0/108765647 >> 172.21.15.122:0/108765647 conn(0x7ff55c0d1e30 msgr2=0x7ff55c0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:06.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- 172.21.15.122:0/108765647 shutdown_connections 2024-09-17T05:34:06.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- 172.21.15.122:0/108765647 wait complete. 2024-09-17T05:34:06.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 Processor -- start 2024-09-17T05:34:06.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- start start 2024-09-17T05:34:06.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff55c0b1f40 0x7ff55c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff55c0b2e80 0x7ff55c152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c15f490 0x7ff55c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c168e90 con 0x7ff55c15f490 2024-09-17T05:34:06.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c15f9e0 con 0x7ff55c0b1f40 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff561a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff55c169010 con 0x7ff55c0b2e80 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff55b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c15f490 0x7ff55c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff55b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c15f490 0x7ff55c0c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37470/0 (socket says 172.21.15.122:37470) 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff55b400640 1 -- 172.21.15.122:0/2076959513 learned_addr learned my addr 172.21.15.122:0/2076959513 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff55aa00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff55c0b1f40 0x7ff55c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.903+0000 7ff55a000640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff55c0b2e80 0x7ff55c152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff55b400640 1 -- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff55c0b2e80 msgr2=0x7ff55c152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff55b400640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff55c0b2e80 0x7ff55c152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff55b400640 1 -- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff55c0b1f40 msgr2=0x7ff55c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff55b400640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff55c0b1f40 0x7ff55c157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff55b400640 1 -- 172.21.15.122:0/2076959513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c0cd830 con 0x7ff55c15f490 2024-09-17T05:34:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff55b400640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c15f490 0x7ff55c0c8230 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c780 tx=0x7ff55400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:06.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff552a00640 1 -- 172.21.15.122:0/2076959513 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554004b70 con 0x7ff55c15f490 2024-09-17T05:34:06.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff552a00640 1 -- 172.21.15.122:0/2076959513 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff554004e00 con 0x7ff55c15f490 2024-09-17T05:34:06.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff552a00640 1 -- 172.21.15.122:0/2076959513 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55400aa90 con 0x7ff55c15f490 2024-09-17T05:34:06.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c0c8770 con 0x7ff55c15f490 2024-09-17T05:34:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c152840 con 0x7ff55c15f490 2024-09-17T05:34:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c0cfda0 con 0x7ff55c15f490 2024-09-17T05:34:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.907+0000 7ff552a00640 1 -- 172.21.15.122:0/2076959513 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7ff55400ac30 con 0x7ff55c15f490 2024-09-17T05:34:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.911+0000 7ff552a00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff53c0743a0 0x7ff53c076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.911+0000 7ff552a00640 1 -- 172.21.15.122:0/2076959513 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4913+0+0 (secure 0 0 0) 0x7ff5540932b0 con 0x7ff55c15f490 2024-09-17T05:34:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.911+0000 7ff55aa00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff53c0743a0 0x7ff53c076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:06.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.911+0000 7ff55aa00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff53c0743a0 0x7ff53c076860 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00bb50 tx=0x7ff54c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:06.911+0000 7ff552a00640 1 -- 172.21.15.122:0/2076959513 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff554020070 con 0x7ff55c15f490 2024-09-17T05:34:07.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff55c158fc0 con 0x7ff55c15f490 2024-09-17T05:34:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff552a00640 1 -- 172.21.15.122:0/2076959513 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) ==== 84+0+65 (secure 0 0 0) 0x7ff55c0c8770 con 0x7ff55c15f490 2024-09-17T05:34:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:07.079 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:34:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:34:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:34:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:07.079 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:34:07.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff53c0743a0 msgr2=0x7ff53c076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff53c0743a0 0x7ff53c076860 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00bb50 tx=0x7ff54c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c15f490 msgr2=0x7ff55c0c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c15f490 0x7ff55c0c8230 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c780 tx=0x7ff55400cc40 comp rx=0 tx=0).stop 2024-09-17T05:34:07.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 shutdown_connections 2024-09-17T05:34:07.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff53c0743a0 0x7ff53c076860 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff55c15f490 0x7ff55c0c8230 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff55c0b2e80 0x7ff55c152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 --2- 172.21.15.122:0/2076959513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff55c0b1f40 0x7ff55c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 >> 172.21.15.122:0/2076959513 conn(0x7ff55c0d1e30 msgr2=0x7ff55c15c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:07.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 shutdown_connections 2024-09-17T05:34:07.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.083+0000 7ff561a00640 1 -- 172.21.15.122:0/2076959513 wait complete. 2024-09-17T05:34:07.100 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:34:07.100 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:34:07.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.247+0000 7f4470e00640 1 -- 172.21.15.122:0/4263591515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 msgr2=0x7f446c0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.247+0000 7f4470e00640 1 --2- 172.21.15.122:0/4263591515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 0x7f446c0ccbd0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4464009bc0 tx=0x7f446401f3b0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.247+0000 7f4470e00640 1 -- 172.21.15.122:0/4263591515 shutdown_connections 2024-09-17T05:34:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.247+0000 7f4470e00640 1 --2- 172.21.15.122:0/4263591515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 0x7f446c0ccbd0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.247+0000 7f4470e00640 1 --2- 172.21.15.122:0/4263591515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f446c15be10 0x7f446c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.247+0000 7f4470e00640 1 -- 172.21.15.122:0/4263591515 >> 172.21.15.122:0/4263591515 conn(0x7f446c0d1da0 msgr2=0x7f446c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.247+0000 7f4470e00640 1 -- 172.21.15.122:0/4263591515 shutdown_connections 2024-09-17T05:34:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.247+0000 7f4470e00640 1 -- 172.21.15.122:0/4263591515 wait complete. 2024-09-17T05:34:07.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 Processor -- start 2024-09-17T05:34:07.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 -- start start 2024-09-17T05:34:07.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 0x7f446c0c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f446c15be10 0x7f446c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c15c1f0 0x7f446c1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0cd320 con 0x7f446c0b0b60 2024-09-17T05:34:07.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0cf590 con 0x7f446c15c1f0 2024-09-17T05:34:07.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f446c0cd4a0 con 0x7f446c15be10 2024-09-17T05:34:07.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 0x7f446c0c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 0x7f446c0c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37492/0 (socket says 172.21.15.122:37492) 2024-09-17T05:34:07.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 -- 172.21.15.122:0/86061784 learned_addr learned my addr 172.21.15.122:0/86061784 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:07.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f446a800640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c15c1f0 0x7f446c1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4461400640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f446c15be10 0x7f446c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 -- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f446c15be10 msgr2=0x7f446c156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f446c15be10 0x7f446c156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 -- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c15c1f0 msgr2=0x7f446c1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:07.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c15c1f0 0x7f446c1f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:07.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 -- 172.21.15.122:0/86061784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c156840 con 0x7f446c0b0b60 2024-09-17T05:34:07.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4469e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 0x7f446c0c9b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b580 tx=0x7f445c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4462a00640 1 -- 172.21.15.122:0/86061784 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c014070 con 0x7f446c0b0b60 2024-09-17T05:34:07.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4462a00640 1 -- 172.21.15.122:0/86061784 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f445c010980 con 0x7f446c0b0b60 2024-09-17T05:34:07.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c1f5090 con 0x7f446c0b0b60 2024-09-17T05:34:07.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4462a00640 1 -- 172.21.15.122:0/86061784 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c010dd0 con 0x7f446c0b0b60 2024-09-17T05:34:07.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.251+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c1f5330 con 0x7f446c0b0b60 2024-09-17T05:34:07.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.255+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f446c0ccbd0 con 0x7f446c0b0b60 2024-09-17T05:34:07.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.255+0000 7f4462a00640 1 -- 172.21.15.122:0/86061784 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f445c017070 con 0x7f446c0b0b60 2024-09-17T05:34:07.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.255+0000 7f4462a00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4448074370 0x7f4448076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:07.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.255+0000 7f4462a00640 1 -- 172.21.15.122:0/86061784 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4913+0+0 (secure 0 0 0) 0x7f445c092f50 con 0x7f446c0b0b60 2024-09-17T05:34:07.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.255+0000 7f4461400640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4448074370 0x7f4448076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:07.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.255+0000 7f4461400640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4448074370 0x7f4448076830 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f446c0c9390 tx=0x7f4454007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:07.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.259+0000 7f4462a00640 1 -- 172.21.15.122:0/86061784 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f445c05f030 con 0x7f446c0b0b60 2024-09-17T05:34:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:07.431+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f446c070350 con 0x7f446c0b0b60 2024-09-17T05:34:08.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4462a00640 1 -- 172.21.15.122:0/86061784 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) ==== 138+0+0 (secure 0 0 0) 0x7f445c063a60 con 0x7f446c0b0b60 2024-09-17T05:34:08.347 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4448074370 msgr2=0x7f4448076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4448074370 0x7f4448076830 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f446c0c9390 tx=0x7f4454007ea0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 msgr2=0x7f446c0c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 0x7f446c0c9b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b580 tx=0x7f445c00ba40 comp rx=0 tx=0).stop 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 shutdown_connections 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4448074370 0x7f4448076830 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f446c0c9390 tx=0x7f4454007ea0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f446c15c1f0 0x7f446c1f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f446c15be10 0x7f446c156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 --2- 172.21.15.122:0/86061784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446c0b0b60 0x7f446c0c9b60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 >> 172.21.15.122:0/86061784 conn(0x7f446c0d1da0 msgr2=0x7f446c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:08.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 shutdown_connections 2024-09-17T05:34:08.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.351+0000 7f4470e00640 1 -- 172.21.15.122:0/86061784 wait complete. 2024-09-17T05:34:08.373 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:34:08.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 -- 172.21.15.122:0/2359470898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15df50 msgr2=0x7f29dc0d4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 --2- 172.21.15.122:0/2359470898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15df50 0x7f29dc0d4990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009900 tx=0x7f29cc01c8b0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 -- 172.21.15.122:0/2359470898 shutdown_connections 2024-09-17T05:34:08.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 --2- 172.21.15.122:0/2359470898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc15ef20 0x7f29dc0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 --2- 172.21.15.122:0/2359470898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15df50 0x7f29dc0d4990 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 -- 172.21.15.122:0/2359470898 >> 172.21.15.122:0/2359470898 conn(0x7f29dc0d3e70 msgr2=0x7f29dc0d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:08.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 -- 172.21.15.122:0/2359470898 shutdown_connections 2024-09-17T05:34:08.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 -- 172.21.15.122:0/2359470898 wait complete. 2024-09-17T05:34:08.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 Processor -- start 2024-09-17T05:34:08.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.527+0000 7f29e1600640 1 -- start start 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0d23c0 0x7f29dc0ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc15df50 0x7f29dc159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15ef20 0x7f29dc15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc16b2f0 con 0x7f29dc15ef20 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc16b170 con 0x7f29dc0d23c0 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f29dc16b470 con 0x7f29dc15df50 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15ef20 0x7f29dc15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15ef20 0x7f29dc15a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37522/0 (socket says 172.21.15.122:37522) 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 -- 172.21.15.122:0/848712981 learned_addr learned my addr 172.21.15.122:0/848712981 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29dbe00640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0d23c0 0x7f29dc0ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:08.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29d3400640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc15df50 0x7f29dc159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 -- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc15df50 msgr2=0x7f29dc159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc15df50 0x7f29dc159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 -- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0d23c0 msgr2=0x7f29dc0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0d23c0 0x7f29dc0ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 -- 172.21.15.122:0/848712981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc0b8b20 con 0x7f29dc15ef20 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e0c00640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15ef20 0x7f29dc15a010 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c7c0 tx=0x7f29d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29d8c00640 1 -- 172.21.15.122:0/848712981 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4003b80 con 0x7f29dc15ef20 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29d8c00640 1 -- 172.21.15.122:0/848712981 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29d4003d20 con 0x7f29dc15ef20 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc0b8e00 con 0x7f29dc15ef20 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29d8c00640 1 -- 172.21.15.122:0/848712981 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d400aab0 con 0x7f29dc15ef20 2024-09-17T05:34:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc0ca620 con 0x7f29dc15ef20 2024-09-17T05:34:08.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc0d4990 con 0x7f29dc15ef20 2024-09-17T05:34:08.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.531+0000 7f29d8c00640 1 -- 172.21.15.122:0/848712981 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f29d400ac50 con 0x7f29dc15ef20 2024-09-17T05:34:08.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.535+0000 7f29d8c00640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f29b8074370 0x7f29b8076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:08.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.535+0000 7f29d8c00640 1 -- 172.21.15.122:0/848712981 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5283+0+0 (secure 0 0 0) 0x7f29d4093920 con 0x7f29dc15ef20 2024-09-17T05:34:08.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.535+0000 7f29dbe00640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f29b8074370 0x7f29b8076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:08.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.535+0000 7f29dbe00640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f29b8074370 0x7f29b8076830 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0af000 tx=0x7f29cc01b0f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:08.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.535+0000 7f29d8c00640 1 -- 172.21.15.122:0/848712981 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f29d405fa80 con 0x7f29dc15ef20 2024-09-17T05:34:08.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:08.707+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f29dc0d0660 con 0x7f29dc15ef20 2024-09-17T05:34:09.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.359+0000 7f29d8c00640 1 -- 172.21.15.122:0/848712981 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) ==== 170+0+0 (secure 0 0 0) 0x7f29d40644b0 con 0x7f29dc15ef20 2024-09-17T05:34:09.356 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:34:09.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f29b8074370 msgr2=0x7f29b8076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:09.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f29b8074370 0x7f29b8076830 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0af000 tx=0x7f29cc01b0f0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15ef20 msgr2=0x7f29dc15a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:09.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15ef20 0x7f29dc15a010 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c7c0 tx=0x7f29d400cc80 comp rx=0 tx=0).stop 2024-09-17T05:34:09.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 shutdown_connections 2024-09-17T05:34:09.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f29b8074370 0x7f29b8076830 secure :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0af000 tx=0x7f29cc01b0f0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29dc15ef20 0x7f29dc15a010 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc15df50 0x7f29dc159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 --2- 172.21.15.122:0/848712981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0d23c0 0x7f29dc0ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 >> 172.21.15.122:0/848712981 conn(0x7f29dc0d3e70 msgr2=0x7f29dc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:09.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 shutdown_connections 2024-09-17T05:34:09.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.363+0000 7f29e1600640 1 -- 172.21.15.122:0/848712981 wait complete. 2024-09-17T05:34:09.395 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:34:09.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 -- 172.21.15.122:0/250721990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e806fe00 msgr2=0x7f90e80cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:09.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 --2- 172.21.15.122:0/250721990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e806fe00 0x7f90e80cc740 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f90e806e730 tx=0x7f90d001c570 comp rx=0 tx=0).stop 2024-09-17T05:34:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 -- 172.21.15.122:0/250721990 shutdown_connections 2024-09-17T05:34:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 --2- 172.21.15.122:0/250721990 >> v2:172.21.15.173:3300/0 conn(0x7f90e80ccc80 0x7f90e806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 --2- 172.21.15.122:0/250721990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e806fe00 0x7f90e80cc740 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 --2- 172.21.15.122:0/250721990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90e8070960 0x7f90e806f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 -- 172.21.15.122:0/250721990 >> 172.21.15.122:0/250721990 conn(0x7f90e80b18f0 msgr2=0x7f90e80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 -- 172.21.15.122:0/250721990 shutdown_connections 2024-09-17T05:34:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.543+0000 7f90ef000640 1 -- 172.21.15.122:0/250721990 wait complete. 2024-09-17T05:34:09.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 Processor -- start 2024-09-17T05:34:09.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 -- start start 2024-09-17T05:34:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90e806fe00 0x7f90e8157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90e8070960 0x7f90e81f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e80ccc80 0x7f90e81fb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f90e80b3eb0 con 0x7f90e80ccc80 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90e80b3d30 con 0x7f90e806fe00 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f90e80b4030 con 0x7f90e8070960 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e80ccc80 0x7f90e81fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e80ccc80 0x7f90e81fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37538/0 (socket says 172.21.15.122:37538) 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 -- 172.21.15.122:0/2700038265 learned_addr learned my addr 172.21.15.122:0/2700038265 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90e7e00640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90e806fe00 0x7f90e8157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90df400640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90e8070960 0x7f90e81f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 -- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90e8070960 msgr2=0x7f90e81f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90e8070960 0x7f90e81f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 -- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90e806fe00 msgr2=0x7f90e8157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90e806fe00 0x7f90e8157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 -- 172.21.15.122:0/2700038265 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e8157b50 con 0x7f90e80ccc80 2024-09-17T05:34:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ed000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e80ccc80 0x7f90e81fb660 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f90e000e970 tx=0x7f90e000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:09.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90e4c00640 1 -- 172.21.15.122:0/2700038265 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0004d30 con 0x7f90e80ccc80 2024-09-17T05:34:09.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90e4c00640 1 -- 172.21.15.122:0/2700038265 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90e000cb20 con 0x7f90e80ccc80 2024-09-17T05:34:09.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90e4c00640 1 -- 172.21.15.122:0/2700038265 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0005020 con 0x7f90e80ccc80 2024-09-17T05:34:09.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e8074110 con 0x7f90e80ccc80 2024-09-17T05:34:09.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.547+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e80746d0 con 0x7f90e80ccc80 2024-09-17T05:34:09.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.551+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e8069410 con 0x7f90e80ccc80 2024-09-17T05:34:09.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.551+0000 7f90e4c00640 1 -- 172.21.15.122:0/2700038265 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f90e00051c0 con 0x7f90e80ccc80 2024-09-17T05:34:09.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.551+0000 7f90e4c00640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90c4074370 0x7f90c4076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:09.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.551+0000 7f90e4c00640 1 -- 172.21.15.122:0/2700038265 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5665+0+0 (secure 0 0 0) 0x7f90e0093980 con 0x7f90e80ccc80 2024-09-17T05:34:09.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.551+0000 7f90e7e00640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90c4074370 0x7f90c4076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:09.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.551+0000 7f90e7e00640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90c4074370 0x7f90c4076830 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f90e815ec80 tx=0x7f90d800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:09.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.555+0000 7f90e4c00640 1 -- 172.21.15.122:0/2700038265 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f90e8157b50 con 0x7f90e80ccc80 2024-09-17T05:34:09.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:09.759+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f90e80d09f0 con 0x7f90e80ccc80 2024-09-17T05:34:10.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.359+0000 7f90e4c00640 1 -- 172.21.15.122:0/2700038265 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) ==== 174+0+0 (secure 0 0 0) 0x7f90e0016030 con 0x7f90e80ccc80 2024-09-17T05:34:10.358 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 12 and data pool 13 2024-09-17T05:34:10.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90c4074370 msgr2=0x7f90c4076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90c4074370 0x7f90c4076830 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f90e815ec80 tx=0x7f90d800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e80ccc80 msgr2=0x7f90e81fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e80ccc80 0x7f90e81fb660 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f90e000e970 tx=0x7f90e000ee30 comp rx=0 tx=0).stop 2024-09-17T05:34:10.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 shutdown_connections 2024-09-17T05:34:10.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f90c4074370 0x7f90c4076830 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90e80ccc80 0x7f90e81fb660 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90e8070960 0x7f90e81f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 --2- 172.21.15.122:0/2700038265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90e806fe00 0x7f90e8157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 >> 172.21.15.122:0/2700038265 conn(0x7f90e80b18f0 msgr2=0x7f90e80b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 shutdown_connections 2024-09-17T05:34:10.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.363+0000 7f90ef000640 1 -- 172.21.15.122:0/2700038265 wait complete. 2024-09-17T05:34:10.407 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.551+0000 7f8fcbc00640 1 -- 172.21.15.122:0/1850493165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 msgr2=0x7f8fa8004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.551+0000 7f8fcbc00640 1 --2- 172.21.15.122:0/1850493165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 0x7f8fa8004020 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009b90 tx=0x7f8fb401d270 comp rx=0 tx=0).stop 2024-09-17T05:34:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.551+0000 7f8fc9c00640 1 -- 172.21.15.122:0/1850493165 reap_dead start 2024-09-17T05:34:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.551+0000 7f8fcbc00640 1 -- 172.21.15.122:0/1850493165 shutdown_connections 2024-09-17T05:34:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.551+0000 7f8fcbc00640 1 -- 172.21.15.122:0/1850493165 >> 172.21.15.122:0/1850493165 conn(0x7f8fc40d1dc0 msgr2=0x7f8fc40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 -- 172.21.15.122:0/1850493165 shutdown_connections 2024-09-17T05:34:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 -- 172.21.15.122:0/1850493165 wait complete. 2024-09-17T05:34:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 Processor -- start 2024-09-17T05:34:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 -- start start 2024-09-17T05:34:10.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 0x7f8fc40b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:10.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fc4158ce0 0x7f8fc40cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:10.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fc40cb510 0x7f8fc4163870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:10.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4167d00 con 0x7f8fa8003c20 2024-09-17T05:34:10.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4167b80 con 0x7f8fc4158ce0 2024-09-17T05:34:10.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fcbc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc4167e80 con 0x7f8fc40cb510 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 0x7f8fc40b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 0x7f8fc40b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37548/0 (socket says 172.21.15.122:37548) 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 -- 172.21.15.122:0/4016426918 learned_addr learned my addr 172.21.15.122:0/4016426918 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fbbe00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fc4158ce0 0x7f8fc40cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9c00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fc40cb510 0x7f8fc4163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 -- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fc40cb510 msgr2=0x7f8fc4163870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fc40cb510 0x7f8fc4163870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 -- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fc4158ce0 msgr2=0x7f8fc40cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fc4158ce0 0x7f8fc40cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 -- 172.21.15.122:0/4016426918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc40b3f30 con 0x7f8fa8003c20 2024-09-17T05:34:10.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.555+0000 7f8fc9200640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 0x7f8fc40b3950 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009b90 tx=0x7f8fb4005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:10.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fc1600640 1 -- 172.21.15.122:0/4016426918 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40b1040 con 0x7f8fa8003c20 2024-09-17T05:34:10.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fc1600640 1 -- 172.21.15.122:0/4016426918 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fb40ac400 con 0x7f8fa8003c20 2024-09-17T05:34:10.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc406e870 con 0x7f8fa8003c20 2024-09-17T05:34:10.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fc1600640 1 -- 172.21.15.122:0/4016426918 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4004290 con 0x7f8fa8003c20 2024-09-17T05:34:10.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc406ee30 con 0x7f8fa8003c20 2024-09-17T05:34:10.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc41590c0 con 0x7f8fa8003c20 2024-09-17T05:34:10.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fc1600640 1 -- 172.21.15.122:0/4016426918 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f8fb40bd020 con 0x7f8fa8003c20 2024-09-17T05:34:10.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fc1600640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fa0074370 0x7f8fa0076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:10.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fc1600640 1 -- 172.21.15.122:0/4016426918 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7f8fb412e1a0 con 0x7f8fa8003c20 2024-09-17T05:34:10.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.559+0000 7f8fbbe00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fa0074370 0x7f8fa0076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:10.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.563+0000 7f8fbbe00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fa0074370 0x7f8fa0076830 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8fac001ce0 tx=0x7f8fac001b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:10.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.563+0000 7f8fc1600640 1 -- 172.21.15.122:0/4016426918 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8fb40c4020 con 0x7f8fa8003c20 2024-09-17T05:34:10.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.731+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fc40cb8f0 con 0x7f8fa8003c20 2024-09-17T05:34:10.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fc1600640 1 -- 172.21.15.122:0/4016426918 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16892 (secure 0 0 0) 0x7f8fb4016fc0 con 0x7f8fa8003c20 2024-09-17T05:34:10.730 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:10.731 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":89,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:34:09.761874+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:07.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:08.718769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:10.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fa0074370 msgr2=0x7f8fa0076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fa0074370 0x7f8fa0076830 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8fac001ce0 tx=0x7f8fac001b90 comp rx=0 tx=0).stop 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 msgr2=0x7f8fc40b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 0x7f8fc40b3950 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009b90 tx=0x7f8fb4005590 comp rx=0 tx=0).stop 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 shutdown_connections 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fa0074370 0x7f8fa0076830 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fc40cb510 0x7f8fc4163870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8fc4158ce0 0x7f8fc40cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 --2- 172.21.15.122:0/4016426918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003c20 0x7f8fc40b3950 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 >> 172.21.15.122:0/4016426918 conn(0x7f8fc40d1dc0 msgr2=0x7f8fc415ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 shutdown_connections 2024-09-17T05:34:10.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.735+0000 7f8fcbc00640 1 -- 172.21.15.122:0/4016426918 wait complete. 2024-09-17T05:34:10.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- 172.21.15.122:0/3577127954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16015c2d0 msgr2=0x7fd160074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 --2- 172.21.15.122:0/3577127954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16015c2d0 0x7fd160074060 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd1600ce850 tx=0x7fd14801c740 comp rx=0 tx=0).stop 2024-09-17T05:34:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- 172.21.15.122:0/3577127954 shutdown_connections 2024-09-17T05:34:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 --2- 172.21.15.122:0/3577127954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16015c2d0 0x7fd160074060 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 --2- 172.21.15.122:0/3577127954 >> v2:172.21.15.173:3300/0 conn(0x7fd1600b6350 0x7fd16006fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- 172.21.15.122:0/3577127954 >> 172.21.15.122:0/3577127954 conn(0x7fd1600d0cd0 msgr2=0x7fd1600d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- 172.21.15.122:0/3577127954 shutdown_connections 2024-09-17T05:34:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- 172.21.15.122:0/3577127954 wait complete. 2024-09-17T05:34:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 Processor -- start 2024-09-17T05:34:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- start start 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1600b6350 0x7fd16014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16015c2d0 0x7fd16015e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd160154c70 0x7fd16015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd160168da0 con 0x7fd1600b6350 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd160168c20 con 0x7fd16015c2d0 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd166c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd160168f20 con 0x7fd160154c70 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1600b6350 0x7fd16014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1600b6350 0x7fd16014f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37574/0 (socket says 172.21.15.122:37574) 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.899+0000 7fd15fe00640 1 -- 172.21.15.122:0/2843533416 learned_addr learned my addr 172.21.15.122:0/2843533416 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd157400640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16015c2d0 0x7fd16015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd164c00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd160154c70 0x7fd16015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15fe00640 1 -- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd160154c70 msgr2=0x7fd16015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15fe00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd160154c70 0x7fd16015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15fe00640 1 -- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16015c2d0 msgr2=0x7fd16015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15fe00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16015c2d0 0x7fd16015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15fe00640 1 -- 172.21.15.122:0/2843533416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1600afd30 con 0x7fd1600b6350 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd164c00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd160154c70 0x7fd16015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15fe00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1600b6350 0x7fd16014f360 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd15000ea20 tx=0x7fd15000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15cc00640 1 -- 172.21.15.122:0/2843533416 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15000cd80 con 0x7fd1600b6350 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15cc00640 1 -- 172.21.15.122:0/2843533416 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd150010430 con 0x7fd1600b6350 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15cc00640 1 -- 172.21.15.122:0/2843533416 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150010860 con 0x7fd1600b6350 2024-09-17T05:34:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1600c4480 con 0x7fd1600b6350 2024-09-17T05:34:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1600affd0 con 0x7fd1600b6350 2024-09-17T05:34:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16006fff0 con 0x7fd1600b6350 2024-09-17T05:34:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.903+0000 7fd15cc00640 1 -- 172.21.15.122:0/2843533416 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fd150010a00 con 0x7fd1600b6350 2024-09-17T05:34:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.907+0000 7fd15cc00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd13c074370 0x7fd13c076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:10.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.907+0000 7fd15cc00640 1 -- 172.21.15.122:0/2843533416 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7fd150093bf0 con 0x7fd1600b6350 2024-09-17T05:34:10.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.907+0000 7fd157400640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd13c074370 0x7fd13c076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:10.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.907+0000 7fd157400640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd13c074370 0x7fd13c076830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd1480af000 tx=0x7fd14801af80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:10.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:10.911+0000 7fd15cc00640 1 -- 172.21.15.122:0/2843533416 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd15005fdb0 con 0x7fd1600b6350 2024-09-17T05:34:11.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.079+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1600d1b00 con 0x7fd1600b6350 2024-09-17T05:34:11.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.079+0000 7fd15cc00640 1 -- 172.21.15.122:0/2843533416 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16892 (secure 0 0 0) 0x7fd1500647e0 con 0x7fd1600b6350 2024-09-17T05:34:11.076 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:11.077 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":89,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:34:09.761874+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:07.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:08.718769+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:11.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd13c074370 msgr2=0x7fd13c076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:11.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd13c074370 0x7fd13c076830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd1480af000 tx=0x7fd14801af80 comp rx=0 tx=0).stop 2024-09-17T05:34:11.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1600b6350 msgr2=0x7fd16014f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:11.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1600b6350 0x7fd16014f360 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd15000ea20 tx=0x7fd15000eee0 comp rx=0 tx=0).stop 2024-09-17T05:34:11.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 shutdown_connections 2024-09-17T05:34:11.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd13c074370 0x7fd13c076830 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:11.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd160154c70 0x7fd16015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:11.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16015c2d0 0x7fd16015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:11.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 --2- 172.21.15.122:0/2843533416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1600b6350 0x7fd16014f360 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:11.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 >> 172.21.15.122:0/2843533416 conn(0x7fd1600d0cd0 msgr2=0x7fd1600b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:11.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 shutdown_connections 2024-09-17T05:34:11.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.083+0000 7fd166c00640 1 -- 172.21.15.122:0/2843533416 wait complete. 2024-09-17T05:34:11.099 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:34:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.247+0000 7fbf1ce00640 1 -- 172.21.15.122:0/2356408702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbefc004d50 msgr2=0x7fbefc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.247+0000 7fbf1ce00640 1 --2- 172.21.15.122:0/2356408702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbefc004d50 0x7fbefc0051d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009ee0 tx=0x7fbf1001d630 comp rx=0 tx=0).stop 2024-09-17T05:34:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.247+0000 7fbf16800640 1 -- 172.21.15.122:0/2356408702 reap_dead start 2024-09-17T05:34:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.247+0000 7fbf1ce00640 1 -- 172.21.15.122:0/2356408702 shutdown_connections 2024-09-17T05:34:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.247+0000 7fbf1ce00640 1 -- 172.21.15.122:0/2356408702 >> 172.21.15.122:0/2356408702 conn(0x7fbf180b18f0 msgr2=0x7fbf180b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.247+0000 7fbf1ce00640 1 -- 172.21.15.122:0/2356408702 shutdown_connections 2024-09-17T05:34:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.247+0000 7fbf1ce00640 1 -- 172.21.15.122:0/2356408702 wait complete. 2024-09-17T05:34:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 Processor -- start 2024-09-17T05:34:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 -- start start 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbefc004d50 0x7fbf181f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1815ea60 0x7fbf1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf180ae0f0 0x7fbf180ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf18160610 con 0x7fbf180ae0f0 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf18160490 con 0x7fbf1815ea60 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbf18160790 con 0x7fbefc004d50 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf180ae0f0 0x7fbf180ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf180ae0f0 0x7fbf180ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37592/0 (socket says 172.21.15.122:37592) 2024-09-17T05:34:11.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 -- 172.21.15.122:0/696635469 learned_addr learned my addr 172.21.15.122:0/696635469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:11.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf15400640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1815ea60 0x7fbf1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:11.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf15e00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbefc004d50 0x7fbf181f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:11.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 -- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbefc004d50 msgr2=0x7fbf181f0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:11.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbefc004d50 0x7fbf181f0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 -- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1815ea60 msgr2=0x7fbf1806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1815ea60 0x7fbf1806ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 -- 172.21.15.122:0/696635469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf181f3ca0 con 0x7fbf180ae0f0 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf16800640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf180ae0f0 0x7fbf180ae890 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009ee0 tx=0x7fbf10004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf0e000640 1 -- 172.21.15.122:0/696635469 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100ad050 con 0x7fbf180ae0f0 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf0e000640 1 -- 172.21.15.122:0/696635469 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf100a7ad0 con 0x7fbf180ae0f0 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf0e000640 1 -- 172.21.15.122:0/696635469 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100a7ea0 con 0x7fbf180ae0f0 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf181f3f80 con 0x7fbf180ae0f0 2024-09-17T05:34:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.251+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf181f4540 con 0x7fbf180ae0f0 2024-09-17T05:34:11.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.255+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf18069410 con 0x7fbf180ae0f0 2024-09-17T05:34:11.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.255+0000 7fbf0e000640 1 -- 172.21.15.122:0/696635469 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fbf100b0070 con 0x7fbf180ae0f0 2024-09-17T05:34:11.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.255+0000 7fbf0e000640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbef8074370 0x7fbef8076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:11.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.255+0000 7fbf0e000640 1 -- 172.21.15.122:0/696635469 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7fbf1012dc50 con 0x7fbf180ae0f0 2024-09-17T05:34:11.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.255+0000 7fbf15e00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbef8074370 0x7fbef8076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:11.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.255+0000 7fbf15e00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbef8074370 0x7fbef8076830 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbf08001020 tx=0x7fbf08006730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:11.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.259+0000 7fbf0e000640 1 -- 172.21.15.122:0/696635469 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbf10017340 con 0x7fbf180ae0f0 2024-09-17T05:34:11.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:11.467+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbf180ce2e0 con 0x7fbf180ae0f0 2024-09-17T05:34:12.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.391+0000 7fbf0e000640 1 -- 172.21.15.122:0/696635469 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7fbf100fa640 con 0x7fbf180ae0f0 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbef8074370 msgr2=0x7fbef8076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbef8074370 0x7fbef8076830 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbf08001020 tx=0x7fbf08006730 comp rx=0 tx=0).stop 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf180ae0f0 msgr2=0x7fbf180ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf180ae0f0 0x7fbf180ae890 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009ee0 tx=0x7fbf10004f30 comp rx=0 tx=0).stop 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 shutdown_connections 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbef8074370 0x7fbef8076830 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf180ae0f0 0x7fbf180ae890 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1815ea60 0x7fbf1806ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 --2- 172.21.15.122:0/696635469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbefc004d50 0x7fbf181f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:12.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 >> 172.21.15.122:0/696635469 conn(0x7fbf180b18f0 msgr2=0x7fbf1815fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:12.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 shutdown_connections 2024-09-17T05:34:12.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.395+0000 7fbf1ce00640 1 -- 172.21.15.122:0/696635469 wait complete. 2024-09-17T05:34:12.413 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:34:12.413 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:34:12.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.563+0000 7fbba7a00640 1 -- 172.21.15.122:0/2119589715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 msgr2=0x7fbb88004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:12.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.563+0000 7fbba7a00640 1 --2- 172.21.15.122:0/2119589715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 0x7fbb88004810 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c001660 tx=0x7fbb8c01c780 comp rx=0 tx=0).stop 2024-09-17T05:34:12.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.563+0000 7fbba5a00640 1 -- 172.21.15.122:0/2119589715 reap_dead start 2024-09-17T05:34:12.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.563+0000 7fbba7a00640 1 -- 172.21.15.122:0/2119589715 shutdown_connections 2024-09-17T05:34:12.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.563+0000 7fbba7a00640 1 -- 172.21.15.122:0/2119589715 >> 172.21.15.122:0/2119589715 conn(0x7fbba00b18f0 msgr2=0x7fbba00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:12.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.563+0000 7fbba7a00640 1 -- 172.21.15.122:0/2119589715 shutdown_connections 2024-09-17T05:34:12.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.563+0000 7fbba7a00640 1 -- 172.21.15.122:0/2119589715 wait complete. 2024-09-17T05:34:12.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 Processor -- start 2024-09-17T05:34:12.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 -- start start 2024-09-17T05:34:12.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 0x7fbba014c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbba01f5880 0x7fbba01f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba01f53c0 0x7fbba0157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbba00ced70 con 0x7fbb88004060 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbba00cebf0 con 0x7fbba01f53c0 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbba00ceef0 con 0x7fbba01f5880 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 0x7fbba014c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 0x7fbba014c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53210/0 (socket says 172.21.15.122:53210) 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 -- 172.21.15.122:0/2555019832 learned_addr learned my addr 172.21.15.122:0/2555019832 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbb9fe00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbba01f5880 0x7fbba01f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5a00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba01f53c0 0x7fbba0157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 -- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbba01f5880 msgr2=0x7fbba01f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbba01f5880 0x7fbba01f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 -- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba01f53c0 msgr2=0x7fbba0157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba01f53c0 0x7fbba0157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 -- 172.21.15.122:0/2555019832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba00acde0 con 0x7fbb88004060 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5a00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba01f53c0 0x7fbba0157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba5000640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 0x7fbba014c960 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbb94011ae0 tx=0x7fbb94009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbb9cc00640 1 -- 172.21.15.122:0/2555019832 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9401e070 con 0x7fbb88004060 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbb9cc00640 1 -- 172.21.15.122:0/2555019832 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb9400fe40 con 0x7fbb88004060 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbb9cc00640 1 -- 172.21.15.122:0/2555019832 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb940042b0 con 0x7fbb88004060 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba00ad0c0 con 0x7fbb88004060 2024-09-17T05:34:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba00ad680 con 0x7fbb88004060 2024-09-17T05:34:12.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.567+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba006f1a0 con 0x7fbb88004060 2024-09-17T05:34:12.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.571+0000 7fbb9cc00640 1 -- 172.21.15.122:0/2555019832 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fbb94021070 con 0x7fbb88004060 2024-09-17T05:34:12.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.571+0000 7fbb9cc00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbb80074370 0x7fbb80076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:12.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.571+0000 7fbb9cc00640 1 -- 172.21.15.122:0/2555019832 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fbb940944a0 con 0x7fbb88004060 2024-09-17T05:34:12.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.571+0000 7fbb9fe00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbb80074370 0x7fbb80076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:12.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.571+0000 7fbb9fe00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbb80074370 0x7fbb80076830 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbba00cbcf0 tx=0x7fbb8c019520 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:12.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.575+0000 7fbb9cc00640 1 -- 172.21.15.122:0/2555019832 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbb940606c0 con 0x7fbb88004060 2024-09-17T05:34:12.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:12.775+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbba0069b00 con 0x7fbb88004060 2024-09-17T05:34:13.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.403+0000 7fbb9cc00640 1 -- 172.21.15.122:0/2555019832 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7fbb940650f0 con 0x7fbb88004060 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbb80074370 msgr2=0x7fbb80076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbb80074370 0x7fbb80076830 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbba00cbcf0 tx=0x7fbb8c019520 comp rx=0 tx=0).stop 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 msgr2=0x7fbba014c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 0x7fbba014c960 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbb94011ae0 tx=0x7fbb94009ea0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 shutdown_connections 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbb80074370 0x7fbb80076830 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba01f53c0 0x7fbba0157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbba01f5880 0x7fbba01f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 --2- 172.21.15.122:0/2555019832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb88004060 0x7fbba014c960 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 >> 172.21.15.122:0/2555019832 conn(0x7fbba00b18f0 msgr2=0x7fbba00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:13.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 shutdown_connections 2024-09-17T05:34:13.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.407+0000 7fbba7a00640 1 -- 172.21.15.122:0/2555019832 wait complete. 2024-09-17T05:34:13.423 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:34:13.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:13.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 -- 172.21.15.122:0/2957727899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe40cdb30 msgr2=0x7fabe40af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 --2- 172.21.15.122:0/2957727899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe40cdb30 0x7fabe40af0c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009bc0 tx=0x7fabdc01cba0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 -- 172.21.15.122:0/2957727899 shutdown_connections 2024-09-17T05:34:13.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 --2- 172.21.15.122:0/2957727899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe40cdb30 0x7fabe40af0c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 -- 172.21.15.122:0/2957727899 >> 172.21.15.122:0/2957727899 conn(0x7fabe40d1dc0 msgr2=0x7fabe40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:13.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 -- 172.21.15.122:0/2957727899 shutdown_connections 2024-09-17T05:34:13.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 -- 172.21.15.122:0/2957727899 wait complete. 2024-09-17T05:34:13.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 Processor -- start 2024-09-17T05:34:13.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.567+0000 7fabeb400640 1 -- start start 2024-09-17T05:34:13.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabe40cdb30 0x7fabe406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabe40ced80 0x7fabe406ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe415bea0 0x7fabe406f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabe416a840 con 0x7fabe415bea0 2024-09-17T05:34:13.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fabe416a6c0 con 0x7fabe40cdb30 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fabe416a9c0 con 0x7fabe40ced80 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe415bea0 0x7fabe406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabe40cdb30 0x7fabe406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe415bea0 0x7fabe406f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53232/0 (socket says 172.21.15.122:53232) 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 -- 172.21.15.122:0/1717764538 learned_addr learned my addr 172.21.15.122:0/1717764538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe3e00640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabe40ced80 0x7fabe406ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 -- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabe40ced80 msgr2=0x7fabe406ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabe40ced80 0x7fabe406ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 -- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabe40cdb30 msgr2=0x7fabe406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabe40cdb30 0x7fabe406ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 -- 172.21.15.122:0/1717764538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe415e060 con 0x7fabe415bea0 2024-09-17T05:34:13.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe9400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe415bea0 0x7fabe406f480 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b0040 tx=0x7fabdc009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:13.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe0c00640 1 -- 172.21.15.122:0/1717764538 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0bb070 con 0x7fabe415bea0 2024-09-17T05:34:13.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe0c00640 1 -- 172.21.15.122:0/1717764538 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabdc004d40 con 0x7fabe415bea0 2024-09-17T05:34:13.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabe0c00640 1 -- 172.21.15.122:0/1717764538 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0b2040 con 0x7fabe415bea0 2024-09-17T05:34:13.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe415e2e0 con 0x7fabe415bea0 2024-09-17T05:34:13.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe415e8a0 con 0x7fabe415bea0 2024-09-17T05:34:13.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.571+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe40b0eb0 con 0x7fabe415bea0 2024-09-17T05:34:13.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.575+0000 7fabe0c00640 1 -- 172.21.15.122:0/1717764538 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fabdc0a5ac0 con 0x7fabe415bea0 2024-09-17T05:34:13.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.575+0000 7fabe0c00640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fabc4074370 0x7fabc4076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.575+0000 7fabe0c00640 1 -- 172.21.15.122:0/1717764538 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fabdc12cf40 con 0x7fabe415bea0 2024-09-17T05:34:13.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.575+0000 7fabe8a00640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fabc4074370 0x7fabc4076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.575+0000 7fabe8a00640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fabc4074370 0x7fabc4076830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fabd8001870 tx=0x7fabd8001380 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.579+0000 7fabe0c00640 1 -- 172.21.15.122:0/1717764538 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fabe415e060 con 0x7fabe415bea0 2024-09-17T05:34:13.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabe40c8510 con 0x7fabe415bea0 2024-09-17T05:34:13.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabe0c00640 1 -- 172.21.15.122:0/1717764538 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5237 (secure 0 0 0) 0x7fabdc0a5e20 con 0x7fabe415bea0 2024-09-17T05:34:13.775 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:13.775 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":56,"btime":"2024-09-17T05:34:13:395200+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1117224890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1117224890},{"type":"v1","addr":"172.21.15.122:6835","nonce":1117224890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2271477440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2271477440},{"type":"v1","addr":"172.21.15.173:6837","nonce":2271477440}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3081382903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3081382903},{"type":"v1","addr":"172.21.15.173:6836","nonce":3081382903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:09.761813+0000","modified":"2024-09-17T05:34:12.779372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6834/3516694104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3516694104},{"type":"v1","addr":"172.21.15.122:6834","nonce":3516694104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-17T05:34:13.775 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 56 2024-09-17T05:34:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fabc4074370 msgr2=0x7fabc4076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fabc4074370 0x7fabc4076830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fabd8001870 tx=0x7fabd8001380 comp rx=0 tx=0).stop 2024-09-17T05:34:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe415bea0 msgr2=0x7fabe406f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe415bea0 0x7fabe406f480 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b0040 tx=0x7fabdc009200 comp rx=0 tx=0).stop 2024-09-17T05:34:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 shutdown_connections 2024-09-17T05:34:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fabc4074370 0x7fabc4076830 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabe415bea0 0x7fabe406f480 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabe40ced80 0x7fabe406ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 --2- 172.21.15.122:0/1717764538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabe40cdb30 0x7fabe406ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 >> 172.21.15.122:0/1717764538 conn(0x7fabe40d1dc0 msgr2=0x7fabe40d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 shutdown_connections 2024-09-17T05:34:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.779+0000 7fabeb400640 1 -- 172.21.15.122:0/1717764538 wait complete. 2024-09-17T05:34:13.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:34:13.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 -- 172.21.15.122:0/3316084001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015c130 msgr2=0x7faa000cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 --2- 172.21.15.122:0/3316084001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015c130 0x7faa000cce80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009c80 tx=0x7fa9f801cc80 comp rx=0 tx=0).stop 2024-09-17T05:34:13.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 -- 172.21.15.122:0/3316084001 shutdown_connections 2024-09-17T05:34:13.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 --2- 172.21.15.122:0/3316084001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015c130 0x7faa000cce80 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 --2- 172.21.15.122:0/3316084001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa000cecf0 0x7faa000cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 -- 172.21.15.122:0/3316084001 >> 172.21.15.122:0/3316084001 conn(0x7faa000d1ea0 msgr2=0x7faa000d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:13.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 -- 172.21.15.122:0/3316084001 shutdown_connections 2024-09-17T05:34:13.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 -- 172.21.15.122:0/3316084001 wait complete. 2024-09-17T05:34:13.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 Processor -- start 2024-09-17T05:34:13.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.943+0000 7faa06a00640 1 -- start start 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa000cecf0 0x7faa00157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015bd50 0x7faa0006f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa0015c130 0x7faa0006f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa001695e0 con 0x7faa0015bd50 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa00169460 con 0x7faa0015c130 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faa00169760 con 0x7faa000cecf0 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015bd50 0x7faa0006f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015bd50 0x7faa0006f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53260/0 (socket says 172.21.15.122:53260) 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 -- 172.21.15.122:0/3676841266 learned_addr learned my addr 172.21.15.122:0/3676841266 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa04a00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa0015c130 0x7faa0006f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9ffe00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa000cecf0 0x7faa00157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 -- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa000cecf0 msgr2=0x7faa00157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa000cecf0 0x7faa00157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 -- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa0015c130 msgr2=0x7faa0006f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa0015c130 0x7faa0006f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 -- 172.21.15.122:0/3676841266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa001584f0 con 0x7faa0015bd50 2024-09-17T05:34:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa04a00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa0015c130 0x7faa0006f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9f7400640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015bd50 0x7faa0006f240 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b550 tx=0x7fa9e800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9fcc00640 1 -- 172.21.15.122:0/3676841266 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8014070 con 0x7faa0015bd50 2024-09-17T05:34:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9fcc00640 1 -- 172.21.15.122:0/3676841266 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9e8010720 con 0x7faa0015bd50 2024-09-17T05:34:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9fcc00640 1 -- 172.21.15.122:0/3676841266 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8010b70 con 0x7faa0015bd50 2024-09-17T05:34:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001f8de0 con 0x7faa0015bd50 2024-09-17T05:34:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001fbb20 con 0x7faa0015bd50 2024-09-17T05:34:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa000ce5e0 con 0x7faa0015bd50 2024-09-17T05:34:13.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.947+0000 7fa9fcc00640 1 -- 172.21.15.122:0/3676841266 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fa9e8017070 con 0x7faa0015bd50 2024-09-17T05:34:13.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.951+0000 7fa9fcc00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa9dc074370 0x7fa9dc076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:13.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.951+0000 7fa9fcc00640 1 -- 172.21.15.122:0/3676841266 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fa9e8092ad0 con 0x7faa0015bd50 2024-09-17T05:34:13.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.951+0000 7fa9ffe00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa9dc074370 0x7fa9dc076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.951+0000 7fa9ffe00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa9dc074370 0x7fa9dc076830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00016c0 tx=0x7fa9f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:13.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:13.955+0000 7fa9fcc00640 1 -- 172.21.15.122:0/3676841266 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa9e805f2e0 con 0x7faa0015bd50 2024-09-17T05:34:14.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.123+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa000c8130 con 0x7faa0015bd50 2024-09-17T05:34:14.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7fa9fcc00640 1 -- 172.21.15.122:0/3676841266 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16879 (secure 0 0 0) 0x7fa9e805ebe0 con 0x7faa0015bd50 2024-09-17T05:34:14.122 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:14.122 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:34:11.379297+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:07.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:08.718769+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:34:14.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa9dc074370 msgr2=0x7fa9dc076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa9dc074370 0x7fa9dc076830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00016c0 tx=0x7fa9f0009040 comp rx=0 tx=0).stop 2024-09-17T05:34:14.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015bd50 msgr2=0x7faa0006f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015bd50 0x7faa0006f240 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b550 tx=0x7fa9e800ba10 comp rx=0 tx=0).stop 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 shutdown_connections 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa9dc074370 0x7fa9dc076830 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa0015c130 0x7faa0006f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa0015bd50 0x7faa0006f240 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 --2- 172.21.15.122:0/3676841266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa000cecf0 0x7faa00157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 >> 172.21.15.122:0/3676841266 conn(0x7faa000d1ea0 msgr2=0x7faa000c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 shutdown_connections 2024-09-17T05:34:14.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.127+0000 7faa06a00640 1 -- 172.21.15.122:0/3676841266 wait complete. 2024-09-17T05:34:14.143 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:34:14.144 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:34:14.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.287+0000 7fdc23e00640 1 -- 172.21.15.122:0/3970515152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc1c0cd100 msgr2=0x7fdc1c0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.287+0000 7fdc23e00640 1 --2- 172.21.15.122:0/3970515152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc1c0cd100 0x7fdc1c0acd40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009870 tx=0x7fdc0801c7d0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.287+0000 7fdc23e00640 1 -- 172.21.15.122:0/3970515152 shutdown_connections 2024-09-17T05:34:14.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.287+0000 7fdc23e00640 1 --2- 172.21.15.122:0/3970515152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc1c0cd100 0x7fdc1c0acd40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.287+0000 7fdc23e00640 1 -- 172.21.15.122:0/3970515152 >> 172.21.15.122:0/3970515152 conn(0x7fdc1c0b18f0 msgr2=0x7fdc1c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.287+0000 7fdc23e00640 1 -- 172.21.15.122:0/3970515152 shutdown_connections 2024-09-17T05:34:14.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.287+0000 7fdc23e00640 1 -- 172.21.15.122:0/3970515152 wait complete. 2024-09-17T05:34:14.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 Processor -- start 2024-09-17T05:34:14.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 -- start start 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc1c06f8b0 0x7fdc1c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc1c0cd100 0x7fdc1c09d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc1c15db50 0x7fdc1c1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1694d0 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c169350 con 0x7fdc1c0cd100 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c169650 con 0x7fdc1c15db50 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc1c06f8b0 0x7fdc1c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc1c06f8b0 0x7fdc1c15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53272/0 (socket says 172.21.15.122:53272) 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 -- 172.21.15.122:0/3093246974 learned_addr learned my addr 172.21.15.122:0/3093246974 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc20a00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc1c0cd100 0x7fdc1c09d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21e00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc1c15db50 0x7fdc1c1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 -- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc1c15db50 msgr2=0x7fdc1c1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc1c15db50 0x7fdc1c1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 -- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc1c0cd100 msgr2=0x7fdc1c09d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc1c0cd100 0x7fdc1c09d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 -- 172.21.15.122:0/3093246974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c1f7ce0 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc21400640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc1c06f8b0 0x7fdc1c15cb20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c430 tx=0x7fdc1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc19600640 1 -- 172.21.15.122:0/3093246974 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10017070 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc19600640 1 -- 172.21.15.122:0/3093246974 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc100053c0 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc19600640 1 -- 172.21.15.122:0/3093246974 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10003860 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1f7f80 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.291+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c1f8540 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.295+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c069410 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.295+0000 7fdc19600640 1 -- 172.21.15.122:0/3093246974 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fdc1001a070 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.295+0000 7fdc19600640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbfc074370 0x7fdbfc076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.295+0000 7fdc19600640 1 -- 172.21.15.122:0/3093246974 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fdc10092500 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.295+0000 7fdc20a00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbfc074370 0x7fdbfc076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.299+0000 7fdc20a00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbfc074370 0x7fdbfc076830 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdc080af000 tx=0x7fdc0801b010 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:14.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.299+0000 7fdc19600640 1 -- 172.21.15.122:0/3093246974 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdc1005e660 con 0x7fdc1c06f8b0 2024-09-17T05:34:14.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.467+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc1c15d060 con 0x7fdbfc074370 2024-09-17T05:34:14.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.471+0000 7fdc19600640 1 -- 172.21.15.122:0/3093246974 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239856 (secure 0 0 0) 0x7fdc080040c0 con 0x7fdbfc074370 2024-09-17T05:34:14.471 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:14.472 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbfc074370 msgr2=0x7fdbfc076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbfc074370 0x7fdbfc076830 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdc080af000 tx=0x7fdc0801b010 comp rx=0 tx=0).stop 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc1c06f8b0 msgr2=0x7fdc1c15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc1c06f8b0 0x7fdc1c15cb20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c430 tx=0x7fdc1000c8f0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 shutdown_connections 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbfc074370 0x7fdbfc076830 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc1c15db50 0x7fdc1c1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc1c0cd100 0x7fdc1c09d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 --2- 172.21.15.122:0/3093246974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc1c06f8b0 0x7fdc1c15cb20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 >> 172.21.15.122:0/3093246974 conn(0x7fdc1c0b18f0 msgr2=0x7fdc1c0addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 shutdown_connections 2024-09-17T05:34:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.475+0000 7fdc23e00640 1 -- 172.21.15.122:0/3093246974 wait complete. 2024-09-17T05:34:14.491 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-17T05:34:12.599675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":501736,"kb_used_data":9388,"kb_used_omap":12,"kb_used_meta":492147,"kb_avail":754472984,"statfs":{"total":773094113280,"available":772580335616,"internally_reserved":0,"allocated":9613312,"data_stored":4791019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":503959094},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.244467"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.356100+0000","last_change":"2024-09-17T05:34:08.356100+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.356100+0000","last_undegraded":"2024-09-17T05:34:08.356100+0000","last_fullsized":"2024-09-17T05:34:08.356100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.360598+0000","last_change":"2024-09-17T05:34:08.360598+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.360598+0000","last_undegraded":"2024-09-17T05:34:08.360598+0000","last_fullsized":"2024-09-17T05:34:08.360598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.360624+0000","last_change":"2024-09-17T05:34:08.360624+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.360624+0000","last_undegraded":"2024-09-17T05:34:08.360624+0000","last_fullsized":"2024-09-17T05:34:08.360624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:08.349148+0000","last_change":"2024-09-17T05:34:08.349148+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.349148+0000","last_undegraded":"2024-09-17T05:34:08.349148+0000","last_fullsized":"2024-09-17T05:34:08.349148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.358369+0000","last_change":"2024-09-17T05:34:08.358369+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.358369+0000","last_undegraded":"2024-09-17T05:34:08.358369+0000","last_fullsized":"2024-09-17T05:34:08.358369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.360490+0000","last_change":"2024-09-17T05:34:08.360490+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.360490+0000","last_undegraded":"2024-09-17T05:34:08.360490+0000","last_fullsized":"2024-09-17T05:34:08.360490+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.357630+0000","last_change":"2024-09-17T05:34:08.357630+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.357630+0000","last_undegraded":"2024-09-17T05:34:08.357630+0000","last_fullsized":"2024-09-17T05:34:08.357630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.358194+0000","last_change":"2024-09-17T05:34:08.358194+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.358194+0000","last_undegraded":"2024-09-17T05:34:08.358194+0000","last_fullsized":"2024-09-17T05:34:08.358194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.361983+0000","last_change":"2024-09-17T05:34:08.361983+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.361983+0000","last_undegraded":"2024-09-17T05:34:08.361983+0000","last_fullsized":"2024-09-17T05:34:08.361983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.360122+0000","last_change":"2024-09-17T05:34:08.360122+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.360122+0000","last_undegraded":"2024-09-17T05:34:08.360122+0000","last_fullsized":"2024-09-17T05:34:08.360122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.361956+0000","last_change":"2024-09-17T05:34:08.361956+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.361956+0000","last_undegraded":"2024-09-17T05:34:08.361956+0000","last_fullsized":"2024-09-17T05:34:08.361956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.358980+0000","last_change":"2024-09-17T05:34:08.358980+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.358980+0000","last_undegraded":"2024-09-17T05:34:08.358980+0000","last_fullsized":"2024-09-17T05:34:08.358980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:08.349148+0000","last_change":"2024-09-17T05:34:08.349148+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.349148+0000","last_undegraded":"2024-09-17T05:34:08.349148+0000","last_fullsized":"2024-09-17T05:34:08.349148+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.360859+0000","last_change":"2024-09-17T05:34:08.360859+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.360859+0000","last_undegraded":"2024-09-17T05:34:08.360859+0000","last_fullsized":"2024-09-17T05:34:08.360859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.360034+0000","last_change":"2024-09-17T05:34:08.360034+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.360034+0000","last_undegraded":"2024-09-17T05:34:08.360034+0000","last_fullsized":"2024-09-17T05:34:08.360034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.361257+0000","last_change":"2024-09-17T05:34:08.361257+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.361257+0000","last_undegraded":"2024-09-17T05:34:08.361257+0000","last_fullsized":"2024-09-17T05:34:08.361257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.355599+0000","last_change":"2024-09-17T05:34:08.355599+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.355599+0000","last_undegraded":"2024-09-17T05:34:08.355599+0000","last_fullsized":"2024-09-17T05:34:08.355599+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.357540+0000","last_change":"2024-09-17T05:34:08.357540+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.357540+0000","last_undegraded":"2024-09-17T05:34:08.357540+0000","last_fullsized":"2024-09-17T05:34:08.357540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.357550+0000","last_change":"2024-09-17T05:34:08.357550+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.357550+0000","last_undegraded":"2024-09-17T05:34:08.357550+0000","last_fullsized":"2024-09-17T05:34:08.357550+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.357926+0000","last_change":"2024-09-17T05:34:08.357926+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.357926+0000","last_undegraded":"2024-09-17T05:34:08.357926+0000","last_fullsized":"2024-09-17T05:34:08.357926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.360179+0000","last_change":"2024-09-17T05:34:08.360179+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.360179+0000","last_undegraded":"2024-09-17T05:34:08.360179+0000","last_fullsized":"2024-09-17T05:34:08.360179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.359842+0000","last_change":"2024-09-17T05:34:08.359842+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.359842+0000","last_undegraded":"2024-09-17T05:34:08.359842+0000","last_fullsized":"2024-09-17T05:34:08.359842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.357440+0000","last_change":"2024-09-17T05:34:08.357440+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.357440+0000","last_undegraded":"2024-09-17T05:34:08.357440+0000","last_fullsized":"2024-09-17T05:34:08.357440+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.357574+0000","last_change":"2024-09-17T05:34:08.357574+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.357574+0000","last_undegraded":"2024-09-17T05:34:08.357574+0000","last_fullsized":"2024-09-17T05:34:08.357574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.359835+0000","last_change":"2024-09-17T05:34:08.359835+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.359835+0000","last_undegraded":"2024-09-17T05:34:08.359835+0000","last_fullsized":"2024-09-17T05:34:08.359835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.358328+0000","last_change":"2024-09-17T05:34:08.358328+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.358328+0000","last_undegraded":"2024-09-17T05:34:08.358328+0000","last_fullsized":"2024-09-17T05:34:08.358328+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.358410+0000","last_change":"2024-09-17T05:34:08.358410+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.358410+0000","last_undegraded":"2024-09-17T05:34:08.358410+0000","last_fullsized":"2024-09-17T05:34:08.358410+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.360190+0000","last_change":"2024-09-17T05:34:08.360190+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.360190+0000","last_undegraded":"2024-09-17T05:34:08.360190+0000","last_fullsized":"2024-09-17T05:34:08.360190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.358468+0000","last_change":"2024-09-17T05:34:08.358468+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.358468+0000","last_undegraded":"2024-09-17T05:34:08.358468+0000","last_fullsized":"2024-09-17T05:34:08.358468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.362672+0000","last_change":"2024-09-17T05:34:08.362672+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.362672+0000","last_undegraded":"2024-09-17T05:34:08.362672+0000","last_fullsized":"2024-09-17T05:34:08.362672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.358854+0000","last_change":"2024-09-17T05:34:08.358854+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.358854+0000","last_undegraded":"2024-09-17T05:34:08.358854+0000","last_fullsized":"2024-09-17T05:34:08.358854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:34:09.352772+0000","last_change":"2024-09-17T05:34:09.352772+0000","last_active":"2024-09-17T05:34:09.352772+0000","last_peered":"2024-09-17T05:34:09.352772+0000","last_clean":"2024-09-17T05:34:09.352772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:09.352772+0000","last_undegraded":"2024-09-17T05:34:09.352772+0000","last_fullsized":"2024-09-17T05:34:09.352772+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-09-17T05:34:08.355507+0000","last_change":"2024-09-17T05:34:08.355507+0000","last_active":"2024-09-17T05:34:08.349148+0000","last_peered":"2024-09-17T05:34:08.349148+0000","last_clean":"2024-09-17T05:34:08.349148+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:34:08.355507+0000","last_undegraded":"2024-09-17T05:34:08.355507+0000","last_fullsized":"2024-09-17T05:34:08.355507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":235,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:34:08.363648+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:34:08.363648+0000","last_peered":"2024-09-17T05:34:08.363648+0000","last_clean":"2024-09-17T05:34:08.363648+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:34:08.363648+0000","last_undegraded":"2024-09-17T05:34:08.363648+0000","last_fullsized":"2024-09-17T05:34:08.363648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58592,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313248,"statfs":{"total":96636764160,"available":96576765952,"internally_reserved":0,"allocated":1060864,"data_stored":454956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509554,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64100,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307740,"statfs":{"total":96636764160,"available":96571125760,"internally_reserved":0,"allocated":1060864,"data_stored":454956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84948,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":83902,"kb_avail":94286892,"statfs":{"total":96636764160,"available":96549777408,"internally_reserved":0,"allocated":1048576,"data_stored":449083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":85916101},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509553,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58588,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313252,"statfs":{"total":96636764160,"available":96576770048,"internally_reserved":0,"allocated":1048576,"data_stored":449083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58592,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313248,"statfs":{"total":96636764160,"available":96576765952,"internally_reserved":0,"allocated":1060864,"data_stored":454956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509555,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55000,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316840,"statfs":{"total":96636764160,"available":96580444160,"internally_reserved":0,"allocated":1642496,"data_stored":1039451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58592,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313248,"statfs":{"total":96636764160,"available":96576765952,"internally_reserved":0,"allocated":1048576,"data_stored":449083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542259,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63324,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308516,"statfs":{"total":96636764160,"available":96571920384,"internally_reserved":0,"allocated":1642496,"data_stored":1039451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:34:14.495 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:34:14.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 -- 172.21.15.122:0/1513365138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b4d80 msgr2=0x7f60f80b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 --2- 172.21.15.122:0/1513365138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b4d80 0x7f60f80b11a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009900 tx=0x7f60e801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 -- 172.21.15.122:0/1513365138 shutdown_connections 2024-09-17T05:34:14.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 --2- 172.21.15.122:0/1513365138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f80b5870 0x7f60f8165670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 --2- 172.21.15.122:0/1513365138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b4d80 0x7f60f80b11a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 -- 172.21.15.122:0/1513365138 >> 172.21.15.122:0/1513365138 conn(0x7f60f809aa10 msgr2=0x7f60f809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 -- 172.21.15.122:0/1513365138 shutdown_connections 2024-09-17T05:34:14.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 -- 172.21.15.122:0/1513365138 wait complete. 2024-09-17T05:34:14.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 Processor -- start 2024-09-17T05:34:14.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.683+0000 7f60f7200640 1 -- start start 2024-09-17T05:34:14.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b0ce0 0x7f60f81f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f80b4d80 0x7f60f815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f80b5870 0x7f60f81fb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8168a70 con 0x7f60f80b0ce0 2024-09-17T05:34:14.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81688f0 con 0x7f60f80b4d80 2024-09-17T05:34:14.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f60f8168bf0 con 0x7f60f80b5870 2024-09-17T05:34:14.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b0ce0 0x7f60f81f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b0ce0 0x7f60f81f7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53290/0 (socket says 172.21.15.122:53290) 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 -- 172.21.15.122:0/3071771268 learned_addr learned my addr 172.21.15.122:0/3071771268 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5400640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f80b4d80 0x7f60f815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f6800640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f80b5870 0x7f60f81fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 -- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f80b5870 msgr2=0x7f60f81fb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f80b5870 0x7f60f81fb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 -- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f80b4d80 msgr2=0x7f60f815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f80b4d80 0x7f60f815a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 -- 172.21.15.122:0/3071771268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f806eeb0 con 0x7f60f80b0ce0 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f6800640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f80b5870 0x7f60f81fb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f5e00640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b0ce0 0x7f60f81f7910 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009850 tx=0x7f60e80a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60ee000640 1 -- 172.21.15.122:0/3071771268 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e80b2040 con 0x7f60f80b0ce0 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60ee000640 1 -- 172.21.15.122:0/3071771268 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60e8004400 con 0x7f60f80b0ce0 2024-09-17T05:34:14.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60ee000640 1 -- 172.21.15.122:0/3071771268 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e80052d0 con 0x7f60f80b0ce0 2024-09-17T05:34:14.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f81f7fd0 con 0x7f60f80b0ce0 2024-09-17T05:34:14.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81f8590 con 0x7f60f80b0ce0 2024-09-17T05:34:14.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.687+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f80b11a0 con 0x7f60f80b0ce0 2024-09-17T05:34:14.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.691+0000 7f60ee000640 1 -- 172.21.15.122:0/3071771268 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f60e8005470 con 0x7f60f80b0ce0 2024-09-17T05:34:14.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.691+0000 7f60ee000640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60d8074370 0x7f60d8076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:14.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.691+0000 7f60ee000640 1 -- 172.21.15.122:0/3071771268 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f60e812c430 con 0x7f60f80b0ce0 2024-09-17T05:34:14.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.691+0000 7f60f5400640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60d8074370 0x7f60d8076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:14.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.691+0000 7f60f5400640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60d8074370 0x7f60d8076830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f60f8153b20 tx=0x7f60e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:14.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.695+0000 7f60ee000640 1 -- 172.21.15.122:0/3071771268 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f60e80164d0 con 0x7f60f80b0ce0 2024-09-17T05:34:14.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.859+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60f80ce0c0 con 0x7f60d8074370 2024-09-17T05:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.863+0000 7f60ee000640 1 -- 172.21.15.122:0/3071771268 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252074 (secure 0 0 0) 0x7f60f80ce0c0 con 0x7f60d8074370 2024-09-17T05:34:14.864 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:34:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60d8074370 msgr2=0x7f60d8076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60d8074370 0x7f60d8076830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f60f8153b20 tx=0x7f60e4009040 comp rx=0 tx=0).stop 2024-09-17T05:34:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b0ce0 msgr2=0x7f60f81f7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b0ce0 0x7f60f81f7910 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009850 tx=0x7f60e80a56b0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 shutdown_connections 2024-09-17T05:34:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f60d8074370 0x7f60d8076830 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f80b5870 0x7f60f81fb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f80b4d80 0x7f60f815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 --2- 172.21.15.122:0/3071771268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60f80b0ce0 0x7f60f81f7910 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 >> 172.21.15.122:0/3071771268 conn(0x7f60f809aa10 msgr2=0x7f60f80b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 shutdown_connections 2024-09-17T05:34:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:14.867+0000 7f60f7200640 1 -- 172.21.15.122:0/3071771268 wait complete. 2024-09-17T05:34:14.884 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-17T05:34:14.600116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":485460,"kb_used_data":9800,"kb_used_omap":12,"kb_used_meta":475443,"kb_avail":754489260,"statfs":{"total":773094113280,"available":772597002240,"internally_reserved":0,"allocated":10035200,"data_stored":4981348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":486854198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001423"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697757+0000","last_change":"2024-09-17T05:34:10.394337+0000","last_active":"2024-09-17T05:34:11.697757+0000","last_peered":"2024-09-17T05:34:11.697757+0000","last_clean":"2024-09-17T05:34:11.697757+0000","last_became_active":"2024-09-17T05:34:10.391501+0000","last_became_peered":"2024-09-17T05:34:10.391501+0000","last_unstale":"2024-09-17T05:34:11.697757+0000","last_undegraded":"2024-09-17T05:34:11.697757+0000","last_fullsized":"2024-09-17T05:34:11.697757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:34.240317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701589+0000","last_change":"2024-09-17T05:34:10.381773+0000","last_active":"2024-09-17T05:34:11.701589+0000","last_peered":"2024-09-17T05:34:11.701589+0000","last_clean":"2024-09-17T05:34:11.701589+0000","last_became_active":"2024-09-17T05:34:10.381418+0000","last_became_peered":"2024-09-17T05:34:10.381418+0000","last_unstale":"2024-09-17T05:34:11.701589+0000","last_undegraded":"2024-09-17T05:34:11.701589+0000","last_fullsized":"2024-09-17T05:34:11.701589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:03.740502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.389825+0000","last_change":"2024-09-17T05:34:10.390226+0000","last_active":"2024-09-17T05:34:11.389825+0000","last_peered":"2024-09-17T05:34:11.389825+0000","last_clean":"2024-09-17T05:34:11.389825+0000","last_became_active":"2024-09-17T05:34:10.389954+0000","last_became_peered":"2024-09-17T05:34:10.389954+0000","last_unstale":"2024-09-17T05:34:11.389825+0000","last_undegraded":"2024-09-17T05:34:11.389825+0000","last_fullsized":"2024-09-17T05:34:11.389825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:26.960767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700789+0000","last_change":"2024-09-17T05:34:10.394500+0000","last_active":"2024-09-17T05:34:11.700789+0000","last_peered":"2024-09-17T05:34:11.700789+0000","last_clean":"2024-09-17T05:34:11.700789+0000","last_became_active":"2024-09-17T05:34:10.393599+0000","last_became_peered":"2024-09-17T05:34:10.393599+0000","last_unstale":"2024-09-17T05:34:11.700789+0000","last_undegraded":"2024-09-17T05:34:11.700789+0000","last_fullsized":"2024-09-17T05:34:11.700789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:59.232739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390164+0000","last_change":"2024-09-17T05:34:10.396171+0000","last_active":"2024-09-17T05:34:11.390164+0000","last_peered":"2024-09-17T05:34:11.390164+0000","last_clean":"2024-09-17T05:34:11.390164+0000","last_became_active":"2024-09-17T05:34:10.395529+0000","last_became_peered":"2024-09-17T05:34:10.395529+0000","last_unstale":"2024-09-17T05:34:11.390164+0000","last_undegraded":"2024-09-17T05:34:11.390164+0000","last_fullsized":"2024-09-17T05:34:11.390164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:21.299083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.389286+0000","last_change":"2024-09-17T05:34:10.396131+0000","last_active":"2024-09-17T05:34:11.389286+0000","last_peered":"2024-09-17T05:34:11.389286+0000","last_clean":"2024-09-17T05:34:11.389286+0000","last_became_active":"2024-09-17T05:34:10.395705+0000","last_became_peered":"2024-09-17T05:34:10.395705+0000","last_unstale":"2024-09-17T05:34:11.389286+0000","last_undegraded":"2024-09-17T05:34:11.389286+0000","last_fullsized":"2024-09-17T05:34:11.389286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:23.597850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700160+0000","last_change":"2024-09-17T05:34:10.393868+0000","last_active":"2024-09-17T05:34:11.700160+0000","last_peered":"2024-09-17T05:34:11.700160+0000","last_clean":"2024-09-17T05:34:11.700160+0000","last_became_active":"2024-09-17T05:34:10.393515+0000","last_became_peered":"2024-09-17T05:34:10.393515+0000","last_unstale":"2024-09-17T05:34:11.700160+0000","last_undegraded":"2024-09-17T05:34:11.700160+0000","last_fullsized":"2024-09-17T05:34:11.700160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:39.626942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.458894+0000","last_change":"2024-09-17T05:34:10.385220+0000","last_active":"2024-09-17T05:34:11.458894+0000","last_peered":"2024-09-17T05:34:11.458894+0000","last_clean":"2024-09-17T05:34:11.458894+0000","last_became_active":"2024-09-17T05:34:10.384934+0000","last_became_peered":"2024-09-17T05:34:10.384934+0000","last_unstale":"2024-09-17T05:34:11.458894+0000","last_undegraded":"2024-09-17T05:34:11.458894+0000","last_fullsized":"2024-09-17T05:34:11.458894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:52.346623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697980+0000","last_change":"2024-09-17T05:34:10.393786+0000","last_active":"2024-09-17T05:34:11.697980+0000","last_peered":"2024-09-17T05:34:11.697980+0000","last_clean":"2024-09-17T05:34:11.697980+0000","last_became_active":"2024-09-17T05:34:10.381198+0000","last_became_peered":"2024-09-17T05:34:10.381198+0000","last_unstale":"2024-09-17T05:34:11.697980+0000","last_undegraded":"2024-09-17T05:34:11.697980+0000","last_fullsized":"2024-09-17T05:34:11.697980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:48.249989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390151+0000","last_change":"2024-09-17T05:34:10.377624+0000","last_active":"2024-09-17T05:34:11.390151+0000","last_peered":"2024-09-17T05:34:11.390151+0000","last_clean":"2024-09-17T05:34:11.390151+0000","last_became_active":"2024-09-17T05:34:10.377312+0000","last_became_peered":"2024-09-17T05:34:10.377312+0000","last_unstale":"2024-09-17T05:34:11.390151+0000","last_undegraded":"2024-09-17T05:34:11.390151+0000","last_fullsized":"2024-09-17T05:34:11.390151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:41.359555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.702115+0000","last_change":"2024-09-17T05:34:10.385677+0000","last_active":"2024-09-17T05:34:11.702115+0000","last_peered":"2024-09-17T05:34:11.702115+0000","last_clean":"2024-09-17T05:34:11.702115+0000","last_became_active":"2024-09-17T05:34:10.383908+0000","last_became_peered":"2024-09-17T05:34:10.383908+0000","last_unstale":"2024-09-17T05:34:11.702115+0000","last_undegraded":"2024-09-17T05:34:11.702115+0000","last_fullsized":"2024-09-17T05:34:11.702115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:29.792186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.459014+0000","last_change":"2024-09-17T05:34:10.390885+0000","last_active":"2024-09-17T05:34:11.459014+0000","last_peered":"2024-09-17T05:34:11.459014+0000","last_clean":"2024-09-17T05:34:11.459014+0000","last_became_active":"2024-09-17T05:34:10.390422+0000","last_became_peered":"2024-09-17T05:34:10.390422+0000","last_unstale":"2024-09-17T05:34:11.459014+0000","last_undegraded":"2024-09-17T05:34:11.459014+0000","last_fullsized":"2024-09-17T05:34:11.459014+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:43.060207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390379+0000","last_change":"2024-09-17T05:34:10.404908+0000","last_active":"2024-09-17T05:34:11.390379+0000","last_peered":"2024-09-17T05:34:11.390379+0000","last_clean":"2024-09-17T05:34:11.390379+0000","last_became_active":"2024-09-17T05:34:10.403964+0000","last_became_peered":"2024-09-17T05:34:10.403964+0000","last_unstale":"2024-09-17T05:34:11.390379+0000","last_undegraded":"2024-09-17T05:34:11.390379+0000","last_fullsized":"2024-09-17T05:34:11.390379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:39.668474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.391707+0000","last_change":"2024-09-17T05:34:10.395423+0000","last_active":"2024-09-17T05:34:11.391707+0000","last_peered":"2024-09-17T05:34:11.391707+0000","last_clean":"2024-09-17T05:34:11.391707+0000","last_became_active":"2024-09-17T05:34:10.395125+0000","last_became_peered":"2024-09-17T05:34:10.395125+0000","last_unstale":"2024-09-17T05:34:11.391707+0000","last_undegraded":"2024-09-17T05:34:11.391707+0000","last_fullsized":"2024-09-17T05:34:11.391707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:22.914089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700385+0000","last_change":"2024-09-17T05:34:10.384933+0000","last_active":"2024-09-17T05:34:11.700385+0000","last_peered":"2024-09-17T05:34:11.700385+0000","last_clean":"2024-09-17T05:34:11.700385+0000","last_became_active":"2024-09-17T05:34:10.384662+0000","last_became_peered":"2024-09-17T05:34:10.384662+0000","last_unstale":"2024-09-17T05:34:11.700385+0000","last_undegraded":"2024-09-17T05:34:11.700385+0000","last_fullsized":"2024-09-17T05:34:11.700385+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:05.644701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.388831+0000","last_change":"2024-09-17T05:34:10.390786+0000","last_active":"2024-09-17T05:34:11.388831+0000","last_peered":"2024-09-17T05:34:11.388831+0000","last_clean":"2024-09-17T05:34:11.388831+0000","last_became_active":"2024-09-17T05:34:10.390525+0000","last_became_peered":"2024-09-17T05:34:10.390525+0000","last_unstale":"2024-09-17T05:34:11.388831+0000","last_undegraded":"2024-09-17T05:34:11.388831+0000","last_fullsized":"2024-09-17T05:34:11.388831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:08.856150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.459446+0000","last_change":"2024-09-17T05:34:10.395831+0000","last_active":"2024-09-17T05:34:11.459446+0000","last_peered":"2024-09-17T05:34:11.459446+0000","last_clean":"2024-09-17T05:34:11.459446+0000","last_became_active":"2024-09-17T05:34:10.395555+0000","last_became_peered":"2024-09-17T05:34:10.395555+0000","last_unstale":"2024-09-17T05:34:11.459446+0000","last_undegraded":"2024-09-17T05:34:11.459446+0000","last_fullsized":"2024-09-17T05:34:11.459446+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:23.597850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.702033+0000","last_change":"2024-09-17T05:34:10.391057+0000","last_active":"2024-09-17T05:34:11.702033+0000","last_peered":"2024-09-17T05:34:11.702033+0000","last_clean":"2024-09-17T05:34:11.702033+0000","last_became_active":"2024-09-17T05:34:10.390794+0000","last_became_peered":"2024-09-17T05:34:10.390794+0000","last_unstale":"2024-09-17T05:34:11.702033+0000","last_undegraded":"2024-09-17T05:34:11.702033+0000","last_fullsized":"2024-09-17T05:34:11.702033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:04.555601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.392168+0000","last_change":"2024-09-17T05:34:10.385092+0000","last_active":"2024-09-17T05:34:11.392168+0000","last_peered":"2024-09-17T05:34:11.392168+0000","last_clean":"2024-09-17T05:34:11.392168+0000","last_became_active":"2024-09-17T05:34:10.384773+0000","last_became_peered":"2024-09-17T05:34:10.384773+0000","last_unstale":"2024-09-17T05:34:11.392168+0000","last_undegraded":"2024-09-17T05:34:11.392168+0000","last_fullsized":"2024-09-17T05:34:11.392168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:05.644701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.391953+0000","last_change":"2024-09-17T05:34:10.386358+0000","last_active":"2024-09-17T05:34:11.391953+0000","last_peered":"2024-09-17T05:34:11.391953+0000","last_clean":"2024-09-17T05:34:11.391953+0000","last_became_active":"2024-09-17T05:34:10.386088+0000","last_became_peered":"2024-09-17T05:34:10.386088+0000","last_unstale":"2024-09-17T05:34:11.391953+0000","last_undegraded":"2024-09-17T05:34:11.391953+0000","last_fullsized":"2024-09-17T05:34:11.391953+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:04.555601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700499+0000","last_change":"2024-09-17T05:34:10.398153+0000","last_active":"2024-09-17T05:34:11.700499+0000","last_peered":"2024-09-17T05:34:11.700499+0000","last_clean":"2024-09-17T05:34:11.700499+0000","last_became_active":"2024-09-17T05:34:10.397887+0000","last_became_peered":"2024-09-17T05:34:10.397887+0000","last_unstale":"2024-09-17T05:34:11.700499+0000","last_undegraded":"2024-09-17T05:34:11.700499+0000","last_fullsized":"2024-09-17T05:34:11.700499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:10.979557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.392214+0000","last_change":"2024-09-17T05:34:10.383738+0000","last_active":"2024-09-17T05:34:11.392214+0000","last_peered":"2024-09-17T05:34:11.392214+0000","last_clean":"2024-09-17T05:34:11.392214+0000","last_became_active":"2024-09-17T05:34:10.383445+0000","last_became_peered":"2024-09-17T05:34:10.383445+0000","last_unstale":"2024-09-17T05:34:11.392214+0000","last_undegraded":"2024-09-17T05:34:11.392214+0000","last_fullsized":"2024-09-17T05:34:11.392214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:55.751703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.458937+0000","last_change":"2024-09-17T05:34:10.389992+0000","last_active":"2024-09-17T05:34:11.458937+0000","last_peered":"2024-09-17T05:34:11.458937+0000","last_clean":"2024-09-17T05:34:11.458937+0000","last_became_active":"2024-09-17T05:34:10.389708+0000","last_became_peered":"2024-09-17T05:34:10.389708+0000","last_unstale":"2024-09-17T05:34:11.458937+0000","last_undegraded":"2024-09-17T05:34:11.458937+0000","last_fullsized":"2024-09-17T05:34:11.458937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:50.966091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700962+0000","last_change":"2024-09-17T05:34:10.390352+0000","last_active":"2024-09-17T05:34:11.700962+0000","last_peered":"2024-09-17T05:34:11.700962+0000","last_clean":"2024-09-17T05:34:11.700962+0000","last_became_active":"2024-09-17T05:34:10.390103+0000","last_became_peered":"2024-09-17T05:34:10.390103+0000","last_unstale":"2024-09-17T05:34:11.700962+0000","last_undegraded":"2024-09-17T05:34:11.700962+0000","last_fullsized":"2024-09-17T05:34:11.700962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:20.639523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.702138+0000","last_change":"2024-09-17T05:34:10.391862+0000","last_active":"2024-09-17T05:34:11.702138+0000","last_peered":"2024-09-17T05:34:11.702138+0000","last_clean":"2024-09-17T05:34:11.702138+0000","last_became_active":"2024-09-17T05:34:10.391552+0000","last_became_peered":"2024-09-17T05:34:10.391552+0000","last_unstale":"2024-09-17T05:34:11.702138+0000","last_undegraded":"2024-09-17T05:34:11.702138+0000","last_fullsized":"2024-09-17T05:34:11.702138+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:14.551837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390198+0000","last_change":"2024-09-17T05:34:10.398532+0000","last_active":"2024-09-17T05:34:11.390198+0000","last_peered":"2024-09-17T05:34:11.390198+0000","last_clean":"2024-09-17T05:34:11.390198+0000","last_became_active":"2024-09-17T05:34:10.398243+0000","last_became_peered":"2024-09-17T05:34:10.398243+0000","last_unstale":"2024-09-17T05:34:11.390198+0000","last_undegraded":"2024-09-17T05:34:11.390198+0000","last_fullsized":"2024-09-17T05:34:11.390198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:16.556151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390416+0000","last_change":"2024-09-17T05:34:10.399435+0000","last_active":"2024-09-17T05:34:11.390416+0000","last_peered":"2024-09-17T05:34:11.390416+0000","last_clean":"2024-09-17T05:34:11.390416+0000","last_became_active":"2024-09-17T05:34:10.397108+0000","last_became_peered":"2024-09-17T05:34:10.397108+0000","last_unstale":"2024-09-17T05:34:11.390416+0000","last_undegraded":"2024-09-17T05:34:11.390416+0000","last_fullsized":"2024-09-17T05:34:11.390416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:11.958180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697699+0000","last_change":"2024-09-17T05:34:10.393821+0000","last_active":"2024-09-17T05:34:11.697699+0000","last_peered":"2024-09-17T05:34:11.697699+0000","last_clean":"2024-09-17T05:34:11.697699+0000","last_became_active":"2024-09-17T05:34:10.393534+0000","last_became_peered":"2024-09-17T05:34:10.393534+0000","last_unstale":"2024-09-17T05:34:11.697699+0000","last_undegraded":"2024-09-17T05:34:11.697699+0000","last_fullsized":"2024-09-17T05:34:11.697699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:58.197805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701560+0000","last_change":"2024-09-17T05:34:10.397857+0000","last_active":"2024-09-17T05:34:11.701560+0000","last_peered":"2024-09-17T05:34:11.701560+0000","last_clean":"2024-09-17T05:34:11.701560+0000","last_became_active":"2024-09-17T05:34:10.397508+0000","last_became_peered":"2024-09-17T05:34:10.397508+0000","last_unstale":"2024-09-17T05:34:11.701560+0000","last_undegraded":"2024-09-17T05:34:11.701560+0000","last_fullsized":"2024-09-17T05:34:11.701560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:58.724793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.392083+0000","last_change":"2024-09-17T05:34:10.388022+0000","last_active":"2024-09-17T05:34:11.392083+0000","last_peered":"2024-09-17T05:34:11.392083+0000","last_clean":"2024-09-17T05:34:11.392083+0000","last_became_active":"2024-09-17T05:34:10.387738+0000","last_became_peered":"2024-09-17T05:34:10.387738+0000","last_unstale":"2024-09-17T05:34:11.392083+0000","last_undegraded":"2024-09-17T05:34:11.392083+0000","last_fullsized":"2024-09-17T05:34:11.392083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:27.857293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.388986+0000","last_change":"2024-09-17T05:34:10.386218+0000","last_active":"2024-09-17T05:34:11.388986+0000","last_peered":"2024-09-17T05:34:11.388986+0000","last_clean":"2024-09-17T05:34:11.388986+0000","last_became_active":"2024-09-17T05:34:10.385949+0000","last_became_peered":"2024-09-17T05:34:10.385949+0000","last_unstale":"2024-09-17T05:34:11.388986+0000","last_undegraded":"2024-09-17T05:34:11.388986+0000","last_fullsized":"2024-09-17T05:34:11.388986+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:57.117119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.388911+0000","last_change":"2024-09-17T05:34:10.382749+0000","last_active":"2024-09-17T05:34:11.388911+0000","last_peered":"2024-09-17T05:34:11.388911+0000","last_clean":"2024-09-17T05:34:11.388911+0000","last_became_active":"2024-09-17T05:34:10.382070+0000","last_became_peered":"2024-09-17T05:34:10.382070+0000","last_unstale":"2024-09-17T05:34:11.388911+0000","last_undegraded":"2024-09-17T05:34:11.388911+0000","last_fullsized":"2024-09-17T05:34:11.388911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:43.379308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.459399+0000","last_change":"2024-09-17T05:34:10.390802+0000","last_active":"2024-09-17T05:34:11.459399+0000","last_peered":"2024-09-17T05:34:11.459399+0000","last_clean":"2024-09-17T05:34:11.459399+0000","last_became_active":"2024-09-17T05:34:10.390556+0000","last_became_peered":"2024-09-17T05:34:10.390556+0000","last_unstale":"2024-09-17T05:34:11.459399+0000","last_undegraded":"2024-09-17T05:34:11.459399+0000","last_fullsized":"2024-09-17T05:34:11.459399+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:03.243859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390386+0000","last_change":"2024-09-17T05:34:09.377671+0000","last_active":"2024-09-17T05:34:11.390386+0000","last_peered":"2024-09-17T05:34:11.390386+0000","last_clean":"2024-09-17T05:34:11.390386+0000","last_became_active":"2024-09-17T05:34:09.377379+0000","last_became_peered":"2024-09-17T05:34:09.377379+0000","last_unstale":"2024-09-17T05:34:11.390386+0000","last_undegraded":"2024-09-17T05:34:11.390386+0000","last_fullsized":"2024-09-17T05:34:11.390386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:08.168602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.698043+0000","last_change":"2024-09-17T05:34:10.394067+0000","last_active":"2024-09-17T05:34:11.698043+0000","last_peered":"2024-09-17T05:34:11.698043+0000","last_clean":"2024-09-17T05:34:11.698043+0000","last_became_active":"2024-09-17T05:34:10.391149+0000","last_became_peered":"2024-09-17T05:34:10.391149+0000","last_unstale":"2024-09-17T05:34:11.698043+0000","last_undegraded":"2024-09-17T05:34:11.698043+0000","last_fullsized":"2024-09-17T05:34:11.698043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:20.639523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390164+0000","last_change":"2024-09-17T05:34:09.383370+0000","last_active":"2024-09-17T05:34:11.390164+0000","last_peered":"2024-09-17T05:34:11.390164+0000","last_clean":"2024-09-17T05:34:11.390164+0000","last_became_active":"2024-09-17T05:34:09.383110+0000","last_became_peered":"2024-09-17T05:34:09.383110+0000","last_unstale":"2024-09-17T05:34:11.390164+0000","last_undegraded":"2024-09-17T05:34:11.390164+0000","last_fullsized":"2024-09-17T05:34:11.390164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:55:00.812264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.459027+0000","last_change":"2024-09-17T05:34:10.388702+0000","last_active":"2024-09-17T05:34:11.459027+0000","last_peered":"2024-09-17T05:34:11.459027+0000","last_clean":"2024-09-17T05:34:11.459027+0000","last_became_active":"2024-09-17T05:34:10.388280+0000","last_became_peered":"2024-09-17T05:34:10.388280+0000","last_unstale":"2024-09-17T05:34:11.459027+0000","last_undegraded":"2024-09-17T05:34:11.459027+0000","last_fullsized":"2024-09-17T05:34:11.459027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:52.457105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701956+0000","last_change":"2024-09-17T05:34:09.388946+0000","last_active":"2024-09-17T05:34:11.701956+0000","last_peered":"2024-09-17T05:34:11.701956+0000","last_clean":"2024-09-17T05:34:11.701956+0000","last_became_active":"2024-09-17T05:34:09.388626+0000","last_became_peered":"2024-09-17T05:34:09.388626+0000","last_unstale":"2024-09-17T05:34:11.701956+0000","last_undegraded":"2024-09-17T05:34:11.701956+0000","last_fullsized":"2024-09-17T05:34:11.701956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:27.685593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.698004+0000","last_change":"2024-09-17T05:34:10.389723+0000","last_active":"2024-09-17T05:34:11.698004+0000","last_peered":"2024-09-17T05:34:11.698004+0000","last_clean":"2024-09-17T05:34:11.698004+0000","last_became_active":"2024-09-17T05:34:10.381507+0000","last_became_peered":"2024-09-17T05:34:10.381507+0000","last_unstale":"2024-09-17T05:34:11.698004+0000","last_undegraded":"2024-09-17T05:34:11.698004+0000","last_fullsized":"2024-09-17T05:34:11.698004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:42.947855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700838+0000","last_change":"2024-09-17T05:34:09.384263+0000","last_active":"2024-09-17T05:34:11.700838+0000","last_peered":"2024-09-17T05:34:11.700838+0000","last_clean":"2024-09-17T05:34:11.700838+0000","last_became_active":"2024-09-17T05:34:09.383155+0000","last_became_peered":"2024-09-17T05:34:09.383155+0000","last_unstale":"2024-09-17T05:34:11.700838+0000","last_undegraded":"2024-09-17T05:34:11.700838+0000","last_fullsized":"2024-09-17T05:34:11.700838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:44.916859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700597+0000","last_change":"2024-09-17T05:34:10.395235+0000","last_active":"2024-09-17T05:34:11.700597+0000","last_peered":"2024-09-17T05:34:11.700597+0000","last_clean":"2024-09-17T05:34:11.700597+0000","last_became_active":"2024-09-17T05:34:10.394942+0000","last_became_peered":"2024-09-17T05:34:10.394942+0000","last_unstale":"2024-09-17T05:34:11.700597+0000","last_undegraded":"2024-09-17T05:34:11.700597+0000","last_fullsized":"2024-09-17T05:34:11.700597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:30.357161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.458803+0000","last_change":"2024-09-17T05:34:09.376314+0000","last_active":"2024-09-17T05:34:11.458803+0000","last_peered":"2024-09-17T05:34:11.458803+0000","last_clean":"2024-09-17T05:34:11.458803+0000","last_became_active":"2024-09-17T05:34:09.375950+0000","last_became_peered":"2024-09-17T05:34:09.375950+0000","last_unstale":"2024-09-17T05:34:11.458803+0000","last_undegraded":"2024-09-17T05:34:11.458803+0000","last_fullsized":"2024-09-17T05:34:11.458803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:26.853670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.389629+0000","last_change":"2024-09-17T05:34:10.379937+0000","last_active":"2024-09-17T05:34:11.389629+0000","last_peered":"2024-09-17T05:34:11.389629+0000","last_clean":"2024-09-17T05:34:11.389629+0000","last_became_active":"2024-09-17T05:34:10.378769+0000","last_became_peered":"2024-09-17T05:34:10.378769+0000","last_unstale":"2024-09-17T05:34:11.389629+0000","last_undegraded":"2024-09-17T05:34:11.389629+0000","last_fullsized":"2024-09-17T05:34:11.389629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:25.491684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.389236+0000","last_change":"2024-09-17T05:34:09.383212+0000","last_active":"2024-09-17T05:34:11.389236+0000","last_peered":"2024-09-17T05:34:11.389236+0000","last_clean":"2024-09-17T05:34:11.389236+0000","last_became_active":"2024-09-17T05:34:09.382531+0000","last_became_peered":"2024-09-17T05:34:09.382531+0000","last_unstale":"2024-09-17T05:34:11.389236+0000","last_undegraded":"2024-09-17T05:34:11.389236+0000","last_fullsized":"2024-09-17T05:34:11.389236+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:54.748079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697629+0000","last_change":"2024-09-17T05:34:10.395869+0000","last_active":"2024-09-17T05:34:11.697629+0000","last_peered":"2024-09-17T05:34:11.697629+0000","last_clean":"2024-09-17T05:34:11.697629+0000","last_became_active":"2024-09-17T05:34:10.383611+0000","last_became_peered":"2024-09-17T05:34:10.383611+0000","last_unstale":"2024-09-17T05:34:11.697629+0000","last_undegraded":"2024-09-17T05:34:11.697629+0000","last_fullsized":"2024-09-17T05:34:11.697629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:43.130253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.391555+0000","last_change":"2024-09-17T05:34:09.389213+0000","last_active":"2024-09-17T05:34:11.391555+0000","last_peered":"2024-09-17T05:34:11.391555+0000","last_clean":"2024-09-17T05:34:11.391555+0000","last_became_active":"2024-09-17T05:34:09.388935+0000","last_became_peered":"2024-09-17T05:34:09.388935+0000","last_unstale":"2024-09-17T05:34:11.391555+0000","last_undegraded":"2024-09-17T05:34:11.391555+0000","last_fullsized":"2024-09-17T05:34:11.391555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:19.081107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.392136+0000","last_change":"2024-09-17T05:34:10.390216+0000","last_active":"2024-09-17T05:34:11.392136+0000","last_peered":"2024-09-17T05:34:11.392136+0000","last_clean":"2024-09-17T05:34:11.392136+0000","last_became_active":"2024-09-17T05:34:10.389924+0000","last_became_peered":"2024-09-17T05:34:10.389924+0000","last_unstale":"2024-09-17T05:34:11.392136+0000","last_undegraded":"2024-09-17T05:34:11.392136+0000","last_fullsized":"2024-09-17T05:34:11.392136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:39.626942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.391385+0000","last_change":"2024-09-17T05:34:09.379390+0000","last_active":"2024-09-17T05:34:11.391385+0000","last_peered":"2024-09-17T05:34:11.391385+0000","last_clean":"2024-09-17T05:34:11.391385+0000","last_became_active":"2024-09-17T05:34:09.378782+0000","last_became_peered":"2024-09-17T05:34:09.378782+0000","last_unstale":"2024-09-17T05:34:11.391385+0000","last_undegraded":"2024-09-17T05:34:11.391385+0000","last_fullsized":"2024-09-17T05:34:11.391385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:16.468005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.392391+0000","last_change":"2024-09-17T05:34:10.384933+0000","last_active":"2024-09-17T05:34:11.392391+0000","last_peered":"2024-09-17T05:34:11.392391+0000","last_clean":"2024-09-17T05:34:11.392391+0000","last_became_active":"2024-09-17T05:34:10.384660+0000","last_became_peered":"2024-09-17T05:34:10.384660+0000","last_unstale":"2024-09-17T05:34:11.392391+0000","last_undegraded":"2024-09-17T05:34:11.392391+0000","last_fullsized":"2024-09-17T05:34:11.392391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:03.740502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.698101+0000","last_change":"2024-09-17T05:34:09.381802+0000","last_active":"2024-09-17T05:34:11.698101+0000","last_peered":"2024-09-17T05:34:11.698101+0000","last_clean":"2024-09-17T05:34:11.698101+0000","last_became_active":"2024-09-17T05:34:09.381548+0000","last_became_peered":"2024-09-17T05:34:09.381548+0000","last_unstale":"2024-09-17T05:34:11.698101+0000","last_undegraded":"2024-09-17T05:34:11.698101+0000","last_fullsized":"2024-09-17T05:34:11.698101+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:48.000215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.698089+0000","last_change":"2024-09-17T05:34:10.391822+0000","last_active":"2024-09-17T05:34:11.698089+0000","last_peered":"2024-09-17T05:34:11.698089+0000","last_clean":"2024-09-17T05:34:11.698089+0000","last_became_active":"2024-09-17T05:34:10.382385+0000","last_became_peered":"2024-09-17T05:34:10.382385+0000","last_unstale":"2024-09-17T05:34:11.698089+0000","last_undegraded":"2024-09-17T05:34:11.698089+0000","last_fullsized":"2024-09-17T05:34:11.698089+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:55.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390203+0000","last_change":"2024-09-17T05:34:09.383720+0000","last_active":"2024-09-17T05:34:11.390203+0000","last_peered":"2024-09-17T05:34:11.390203+0000","last_clean":"2024-09-17T05:34:11.390203+0000","last_became_active":"2024-09-17T05:34:09.379668+0000","last_became_peered":"2024-09-17T05:34:09.379668+0000","last_unstale":"2024-09-17T05:34:11.390203+0000","last_undegraded":"2024-09-17T05:34:11.390203+0000","last_fullsized":"2024-09-17T05:34:11.390203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:17.554558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700907+0000","last_change":"2024-09-17T05:34:10.394332+0000","last_active":"2024-09-17T05:34:11.700907+0000","last_peered":"2024-09-17T05:34:11.700907+0000","last_clean":"2024-09-17T05:34:11.700907+0000","last_became_active":"2024-09-17T05:34:10.392091+0000","last_became_peered":"2024-09-17T05:34:10.392091+0000","last_unstale":"2024-09-17T05:34:11.700907+0000","last_undegraded":"2024-09-17T05:34:11.700907+0000","last_fullsized":"2024-09-17T05:34:11.700907+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:42.947855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697907+0000","last_change":"2024-09-17T05:34:09.380965+0000","last_active":"2024-09-17T05:34:11.697907+0000","last_peered":"2024-09-17T05:34:11.697907+0000","last_clean":"2024-09-17T05:34:11.697907+0000","last_became_active":"2024-09-17T05:34:09.380729+0000","last_became_peered":"2024-09-17T05:34:09.380729+0000","last_unstale":"2024-09-17T05:34:11.697907+0000","last_undegraded":"2024-09-17T05:34:11.697907+0000","last_fullsized":"2024-09-17T05:34:11.697907+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:05.723662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390294+0000","last_change":"2024-09-17T05:34:10.401623+0000","last_active":"2024-09-17T05:34:11.390294+0000","last_peered":"2024-09-17T05:34:11.390294+0000","last_clean":"2024-09-17T05:34:11.390294+0000","last_became_active":"2024-09-17T05:34:10.401191+0000","last_became_peered":"2024-09-17T05:34:10.401191+0000","last_unstale":"2024-09-17T05:34:11.390294+0000","last_undegraded":"2024-09-17T05:34:11.390294+0000","last_fullsized":"2024-09-17T05:34:11.390294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:56.507289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.459069+0000","last_change":"2024-09-17T05:34:09.384075+0000","last_active":"2024-09-17T05:34:11.459069+0000","last_peered":"2024-09-17T05:34:11.459069+0000","last_clean":"2024-09-17T05:34:11.459069+0000","last_became_active":"2024-09-17T05:34:09.383634+0000","last_became_peered":"2024-09-17T05:34:09.383634+0000","last_unstale":"2024-09-17T05:34:11.459069+0000","last_undegraded":"2024-09-17T05:34:11.459069+0000","last_fullsized":"2024-09-17T05:34:11.459069+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:38.623318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700494+0000","last_change":"2024-09-17T05:34:10.399121+0000","last_active":"2024-09-17T05:34:11.700494+0000","last_peered":"2024-09-17T05:34:11.700494+0000","last_clean":"2024-09-17T05:34:11.700494+0000","last_became_active":"2024-09-17T05:34:10.398119+0000","last_became_peered":"2024-09-17T05:34:10.398119+0000","last_unstale":"2024-09-17T05:34:11.700494+0000","last_undegraded":"2024-09-17T05:34:11.700494+0000","last_fullsized":"2024-09-17T05:34:11.700494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:32.451935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701408+0000","last_change":"2024-09-17T05:34:09.386810+0000","last_active":"2024-09-17T05:34:11.701408+0000","last_peered":"2024-09-17T05:34:11.701408+0000","last_clean":"2024-09-17T05:34:11.701408+0000","last_became_active":"2024-09-17T05:34:09.386566+0000","last_became_peered":"2024-09-17T05:34:09.386566+0000","last_unstale":"2024-09-17T05:34:11.701408+0000","last_undegraded":"2024-09-17T05:34:11.701408+0000","last_fullsized":"2024-09-17T05:34:11.701408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:40.760730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697735+0000","last_change":"2024-09-17T05:34:10.397874+0000","last_active":"2024-09-17T05:34:11.697735+0000","last_peered":"2024-09-17T05:34:11.697735+0000","last_clean":"2024-09-17T05:34:11.697735+0000","last_became_active":"2024-09-17T05:34:10.397578+0000","last_became_peered":"2024-09-17T05:34:10.397578+0000","last_unstale":"2024-09-17T05:34:11.697735+0000","last_undegraded":"2024-09-17T05:34:11.697735+0000","last_fullsized":"2024-09-17T05:34:11.697735+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:41.461898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701071+0000","last_change":"2024-09-17T05:34:09.380596+0000","last_active":"2024-09-17T05:34:11.701071+0000","last_peered":"2024-09-17T05:34:11.701071+0000","last_clean":"2024-09-17T05:34:11.701071+0000","last_became_active":"2024-09-17T05:34:09.380331+0000","last_became_peered":"2024-09-17T05:34:09.380331+0000","last_unstale":"2024-09-17T05:34:11.701071+0000","last_undegraded":"2024-09-17T05:34:11.701071+0000","last_fullsized":"2024-09-17T05:34:11.701071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:05.806220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.391950+0000","last_change":"2024-09-17T05:34:10.386496+0000","last_active":"2024-09-17T05:34:11.391950+0000","last_peered":"2024-09-17T05:34:11.391950+0000","last_clean":"2024-09-17T05:34:11.391950+0000","last_became_active":"2024-09-17T05:34:10.386225+0000","last_became_peered":"2024-09-17T05:34:10.386225+0000","last_unstale":"2024-09-17T05:34:11.391950+0000","last_undegraded":"2024-09-17T05:34:11.391950+0000","last_fullsized":"2024-09-17T05:34:11.391950+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:39.975947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.389083+0000","last_change":"2024-09-17T05:34:09.382610+0000","last_active":"2024-09-17T05:34:11.389083+0000","last_peered":"2024-09-17T05:34:11.389083+0000","last_clean":"2024-09-17T05:34:11.389083+0000","last_became_active":"2024-09-17T05:34:09.382306+0000","last_became_peered":"2024-09-17T05:34:09.382306+0000","last_unstale":"2024-09-17T05:34:11.389083+0000","last_undegraded":"2024-09-17T05:34:11.389083+0000","last_fullsized":"2024-09-17T05:34:11.389083+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:02.736878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.699958+0000","last_change":"2024-09-17T05:34:10.383616+0000","last_active":"2024-09-17T05:34:11.699958+0000","last_peered":"2024-09-17T05:34:11.699958+0000","last_clean":"2024-09-17T05:34:11.699958+0000","last_became_active":"2024-09-17T05:34:10.382928+0000","last_became_peered":"2024-09-17T05:34:10.382928+0000","last_unstale":"2024-09-17T05:34:11.699958+0000","last_undegraded":"2024-09-17T05:34:11.699958+0000","last_fullsized":"2024-09-17T05:34:11.699958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:34.240317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701038+0000","last_change":"2024-09-17T05:34:09.384817+0000","last_active":"2024-09-17T05:34:11.701038+0000","last_peered":"2024-09-17T05:34:11.701038+0000","last_clean":"2024-09-17T05:34:11.701038+0000","last_became_active":"2024-09-17T05:34:09.384440+0000","last_became_peered":"2024-09-17T05:34:09.384440+0000","last_unstale":"2024-09-17T05:34:11.701038+0000","last_undegraded":"2024-09-17T05:34:11.701038+0000","last_fullsized":"2024-09-17T05:34:11.701038+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:33:23.453870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.459452+0000","last_change":"2024-09-17T05:34:10.382118+0000","last_active":"2024-09-17T05:34:11.459452+0000","last_peered":"2024-09-17T05:34:11.459452+0000","last_clean":"2024-09-17T05:34:11.459452+0000","last_became_active":"2024-09-17T05:34:10.381890+0000","last_became_peered":"2024-09-17T05:34:10.381890+0000","last_unstale":"2024-09-17T05:34:11.459452+0000","last_undegraded":"2024-09-17T05:34:11.459452+0000","last_fullsized":"2024-09-17T05:34:11.459452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:41.173819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390538+0000","last_change":"2024-09-17T05:34:09.369093+0000","last_active":"2024-09-17T05:34:11.390538+0000","last_peered":"2024-09-17T05:34:11.390538+0000","last_clean":"2024-09-17T05:34:11.390538+0000","last_became_active":"2024-09-17T05:34:09.368099+0000","last_became_peered":"2024-09-17T05:34:09.368099+0000","last_unstale":"2024-09-17T05:34:11.390538+0000","last_undegraded":"2024-09-17T05:34:11.390538+0000","last_fullsized":"2024-09-17T05:34:11.390538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:53.813185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390991+0000","last_change":"2024-09-17T05:34:10.391861+0000","last_active":"2024-09-17T05:34:11.390991+0000","last_peered":"2024-09-17T05:34:11.390991+0000","last_clean":"2024-09-17T05:34:11.390991+0000","last_became_active":"2024-09-17T05:34:10.389792+0000","last_became_peered":"2024-09-17T05:34:10.389792+0000","last_unstale":"2024-09-17T05:34:11.390991+0000","last_undegraded":"2024-09-17T05:34:11.390991+0000","last_fullsized":"2024-09-17T05:34:11.390991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:50.924772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.392287+0000","last_change":"2024-09-17T05:34:09.387261+0000","last_active":"2024-09-17T05:34:11.392287+0000","last_peered":"2024-09-17T05:34:11.392287+0000","last_clean":"2024-09-17T05:34:11.392287+0000","last_became_active":"2024-09-17T05:34:09.386962+0000","last_became_peered":"2024-09-17T05:34:09.386962+0000","last_unstale":"2024-09-17T05:34:11.392287+0000","last_undegraded":"2024-09-17T05:34:11.392287+0000","last_fullsized":"2024-09-17T05:34:11.392287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:03.870942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.459143+0000","last_change":"2024-09-17T05:34:10.385788+0000","last_active":"2024-09-17T05:34:11.459143+0000","last_peered":"2024-09-17T05:34:11.459143+0000","last_clean":"2024-09-17T05:34:11.459143+0000","last_became_active":"2024-09-17T05:34:10.384533+0000","last_became_peered":"2024-09-17T05:34:10.384533+0000","last_unstale":"2024-09-17T05:34:11.459143+0000","last_undegraded":"2024-09-17T05:34:11.459143+0000","last_fullsized":"2024-09-17T05:34:11.459143+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:15.899383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.391842+0000","last_change":"2024-09-17T05:34:09.381259+0000","last_active":"2024-09-17T05:34:11.391842+0000","last_peered":"2024-09-17T05:34:11.391842+0000","last_clean":"2024-09-17T05:34:11.391842+0000","last_became_active":"2024-09-17T05:34:09.380936+0000","last_became_peered":"2024-09-17T05:34:09.380936+0000","last_unstale":"2024-09-17T05:34:11.391842+0000","last_undegraded":"2024-09-17T05:34:11.391842+0000","last_fullsized":"2024-09-17T05:34:11.391842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:58.165387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700922+0000","last_change":"2024-09-17T05:34:10.391854+0000","last_active":"2024-09-17T05:34:11.700922+0000","last_peered":"2024-09-17T05:34:11.700922+0000","last_clean":"2024-09-17T05:34:11.700922+0000","last_became_active":"2024-09-17T05:34:10.388336+0000","last_became_peered":"2024-09-17T05:34:10.388336+0000","last_unstale":"2024-09-17T05:34:11.700922+0000","last_undegraded":"2024-09-17T05:34:11.700922+0000","last_fullsized":"2024-09-17T05:34:11.700922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:55.751703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.391752+0000","last_change":"2024-09-17T05:34:09.382527+0000","last_active":"2024-09-17T05:34:11.391752+0000","last_peered":"2024-09-17T05:34:11.391752+0000","last_clean":"2024-09-17T05:34:11.391752+0000","last_became_active":"2024-09-17T05:34:09.382087+0000","last_became_peered":"2024-09-17T05:34:09.382087+0000","last_unstale":"2024-09-17T05:34:11.391752+0000","last_undegraded":"2024-09-17T05:34:11.391752+0000","last_fullsized":"2024-09-17T05:34:11.391752+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:48.315402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390532+0000","last_change":"2024-09-17T05:34:10.398824+0000","last_active":"2024-09-17T05:34:11.390532+0000","last_peered":"2024-09-17T05:34:11.390532+0000","last_clean":"2024-09-17T05:34:11.390532+0000","last_became_active":"2024-09-17T05:34:10.398544+0000","last_became_peered":"2024-09-17T05:34:10.398544+0000","last_unstale":"2024-09-17T05:34:11.390532+0000","last_undegraded":"2024-09-17T05:34:11.390532+0000","last_fullsized":"2024-09-17T05:34:11.390532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:56:50.966320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701195+0000","last_change":"2024-09-17T05:34:09.384816+0000","last_active":"2024-09-17T05:34:11.701195+0000","last_peered":"2024-09-17T05:34:11.701195+0000","last_clean":"2024-09-17T05:34:11.701195+0000","last_became_active":"2024-09-17T05:34:09.383804+0000","last_became_peered":"2024-09-17T05:34:09.383804+0000","last_unstale":"2024-09-17T05:34:11.701195+0000","last_undegraded":"2024-09-17T05:34:11.701195+0000","last_fullsized":"2024-09-17T05:34:11.701195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:06.496258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700067+0000","last_change":"2024-09-17T05:34:10.393222+0000","last_active":"2024-09-17T05:34:11.700067+0000","last_peered":"2024-09-17T05:34:11.700067+0000","last_clean":"2024-09-17T05:34:11.700067+0000","last_became_active":"2024-09-17T05:34:10.392938+0000","last_became_peered":"2024-09-17T05:34:10.392938+0000","last_unstale":"2024-09-17T05:34:11.700067+0000","last_undegraded":"2024-09-17T05:34:11.700067+0000","last_fullsized":"2024-09-17T05:34:11.700067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:24.492246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700023+0000","last_change":"2024-09-17T05:34:09.387043+0000","last_active":"2024-09-17T05:34:11.700023+0000","last_peered":"2024-09-17T05:34:11.700023+0000","last_clean":"2024-09-17T05:34:11.700023+0000","last_became_active":"2024-09-17T05:34:09.386780+0000","last_became_peered":"2024-09-17T05:34:09.386780+0000","last_unstale":"2024-09-17T05:34:11.700023+0000","last_undegraded":"2024-09-17T05:34:11.700023+0000","last_fullsized":"2024-09-17T05:34:11.700023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:27.599041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.458934+0000","last_change":"2024-09-17T05:34:10.385239+0000","last_active":"2024-09-17T05:34:11.458934+0000","last_peered":"2024-09-17T05:34:11.458934+0000","last_clean":"2024-09-17T05:34:11.458934+0000","last_became_active":"2024-09-17T05:34:10.384972+0000","last_became_peered":"2024-09-17T05:34:10.384972+0000","last_unstale":"2024-09-17T05:34:11.458934+0000","last_undegraded":"2024-09-17T05:34:11.458934+0000","last_fullsized":"2024-09-17T05:34:11.458934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:12.786037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.391533+0000","last_change":"2024-09-17T05:34:09.382763+0000","last_active":"2024-09-17T05:34:11.391533+0000","last_peered":"2024-09-17T05:34:11.391533+0000","last_clean":"2024-09-17T05:34:11.391533+0000","last_became_active":"2024-09-17T05:34:09.382306+0000","last_became_peered":"2024-09-17T05:34:09.382306+0000","last_unstale":"2024-09-17T05:34:11.391533+0000","last_undegraded":"2024-09-17T05:34:11.391533+0000","last_fullsized":"2024-09-17T05:34:11.391533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:02.466117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390233+0000","last_change":"2024-09-17T05:34:10.401163+0000","last_active":"2024-09-17T05:34:11.390233+0000","last_peered":"2024-09-17T05:34:11.390233+0000","last_clean":"2024-09-17T05:34:11.390233+0000","last_became_active":"2024-09-17T05:34:10.400799+0000","last_became_peered":"2024-09-17T05:34:10.400799+0000","last_unstale":"2024-09-17T05:34:11.390233+0000","last_undegraded":"2024-09-17T05:34:11.390233+0000","last_fullsized":"2024-09-17T05:34:11.390233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:20.891455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.392309+0000","last_change":"2024-09-17T05:34:09.385838+0000","last_active":"2024-09-17T05:34:11.392309+0000","last_peered":"2024-09-17T05:34:11.392309+0000","last_clean":"2024-09-17T05:34:11.392309+0000","last_became_active":"2024-09-17T05:34:09.384771+0000","last_became_peered":"2024-09-17T05:34:09.384771+0000","last_unstale":"2024-09-17T05:34:11.392309+0000","last_undegraded":"2024-09-17T05:34:11.392309+0000","last_fullsized":"2024-09-17T05:34:11.392309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:27.685593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390462+0000","last_change":"2024-09-17T05:34:10.399970+0000","last_active":"2024-09-17T05:34:11.390462+0000","last_peered":"2024-09-17T05:34:11.390462+0000","last_clean":"2024-09-17T05:34:11.390462+0000","last_became_active":"2024-09-17T05:34:10.398756+0000","last_became_peered":"2024-09-17T05:34:10.398756+0000","last_unstale":"2024-09-17T05:34:11.390462+0000","last_undegraded":"2024-09-17T05:34:11.390462+0000","last_fullsized":"2024-09-17T05:34:11.390462+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:51.251013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701956+0000","last_change":"2024-09-17T05:34:09.378876+0000","last_active":"2024-09-17T05:34:11.701956+0000","last_peered":"2024-09-17T05:34:11.701956+0000","last_clean":"2024-09-17T05:34:11.701956+0000","last_became_active":"2024-09-17T05:34:09.378541+0000","last_became_peered":"2024-09-17T05:34:09.378541+0000","last_unstale":"2024-09-17T05:34:11.701956+0000","last_undegraded":"2024-09-17T05:34:11.701956+0000","last_fullsized":"2024-09-17T05:34:11.701956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:48.315402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390888+0000","last_change":"2024-09-17T05:34:10.397875+0000","last_active":"2024-09-17T05:34:11.390888+0000","last_peered":"2024-09-17T05:34:11.390888+0000","last_clean":"2024-09-17T05:34:11.390888+0000","last_became_active":"2024-09-17T05:34:10.397632+0000","last_became_peered":"2024-09-17T05:34:10.397632+0000","last_unstale":"2024-09-17T05:34:11.390888+0000","last_undegraded":"2024-09-17T05:34:11.390888+0000","last_fullsized":"2024-09-17T05:34:11.390888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:37.094423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.459206+0000","last_change":"2024-09-17T05:34:09.377539+0000","last_active":"2024-09-17T05:34:11.459206+0000","last_peered":"2024-09-17T05:34:11.459206+0000","last_clean":"2024-09-17T05:34:11.459206+0000","last_became_active":"2024-09-17T05:34:09.377259+0000","last_became_peered":"2024-09-17T05:34:09.377259+0000","last_unstale":"2024-09-17T05:34:11.459206+0000","last_undegraded":"2024-09-17T05:34:11.459206+0000","last_fullsized":"2024-09-17T05:34:11.459206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:04.641077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.389888+0000","last_change":"2024-09-17T05:34:10.379564+0000","last_active":"2024-09-17T05:34:11.389888+0000","last_peered":"2024-09-17T05:34:11.389888+0000","last_clean":"2024-09-17T05:34:11.389888+0000","last_became_active":"2024-09-17T05:34:10.378747+0000","last_became_peered":"2024-09-17T05:34:10.378747+0000","last_unstale":"2024-09-17T05:34:11.389888+0000","last_undegraded":"2024-09-17T05:34:11.389888+0000","last_fullsized":"2024-09-17T05:34:11.389888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:37.201458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.389703+0000","last_change":"2024-09-17T05:34:09.388210+0000","last_active":"2024-09-17T05:34:11.389703+0000","last_peered":"2024-09-17T05:34:11.389703+0000","last_clean":"2024-09-17T05:34:11.389703+0000","last_became_active":"2024-09-17T05:34:09.387933+0000","last_became_peered":"2024-09-17T05:34:09.387933+0000","last_unstale":"2024-09-17T05:34:11.389703+0000","last_undegraded":"2024-09-17T05:34:11.389703+0000","last_fullsized":"2024-09-17T05:34:11.389703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:38.411918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697676+0000","last_change":"2024-09-17T05:34:10.393769+0000","last_active":"2024-09-17T05:34:11.697676+0000","last_peered":"2024-09-17T05:34:11.697676+0000","last_clean":"2024-09-17T05:34:11.697676+0000","last_became_active":"2024-09-17T05:34:10.382829+0000","last_became_peered":"2024-09-17T05:34:10.382829+0000","last_unstale":"2024-09-17T05:34:11.697676+0000","last_undegraded":"2024-09-17T05:34:11.697676+0000","last_fullsized":"2024-09-17T05:34:11.697676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:16.225080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701602+0000","last_change":"2024-09-17T05:34:09.382636+0000","last_active":"2024-09-17T05:34:11.701602+0000","last_peered":"2024-09-17T05:34:11.701602+0000","last_clean":"2024-09-17T05:34:11.701602+0000","last_became_active":"2024-09-17T05:34:09.381981+0000","last_became_peered":"2024-09-17T05:34:09.381981+0000","last_unstale":"2024-09-17T05:34:11.701602+0000","last_undegraded":"2024-09-17T05:34:11.701602+0000","last_fullsized":"2024-09-17T05:34:11.701602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:46.553012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.700612+0000","last_change":"2024-09-17T05:34:10.386606+0000","last_active":"2024-09-17T05:34:11.700612+0000","last_peered":"2024-09-17T05:34:11.700612+0000","last_clean":"2024-09-17T05:34:11.700612+0000","last_became_active":"2024-09-17T05:34:10.386318+0000","last_became_peered":"2024-09-17T05:34:10.386318+0000","last_unstale":"2024-09-17T05:34:11.700612+0000","last_undegraded":"2024-09-17T05:34:11.700612+0000","last_fullsized":"2024-09-17T05:34:11.700612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:48.249989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.389340+0000","last_change":"2024-09-17T05:34:09.383496+0000","last_active":"2024-09-17T05:34:11.389340+0000","last_peered":"2024-09-17T05:34:11.389340+0000","last_clean":"2024-09-17T05:34:11.389340+0000","last_became_active":"2024-09-17T05:34:09.382923+0000","last_became_peered":"2024-09-17T05:34:09.382923+0000","last_unstale":"2024-09-17T05:34:11.389340+0000","last_undegraded":"2024-09-17T05:34:11.389340+0000","last_fullsized":"2024-09-17T05:34:11.389340+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:06.143659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697650+0000","last_change":"2024-09-17T05:34:10.378393+0000","last_active":"2024-09-17T05:34:11.697650+0000","last_peered":"2024-09-17T05:34:11.697650+0000","last_clean":"2024-09-17T05:34:11.697650+0000","last_became_active":"2024-09-17T05:34:10.378140+0000","last_became_peered":"2024-09-17T05:34:10.378140+0000","last_unstale":"2024-09-17T05:34:11.697650+0000","last_undegraded":"2024-09-17T05:34:11.697650+0000","last_fullsized":"2024-09-17T05:34:11.697650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:37.944539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697489+0000","last_change":"2024-09-17T05:34:09.381047+0000","last_active":"2024-09-17T05:34:11.697489+0000","last_peered":"2024-09-17T05:34:11.697489+0000","last_clean":"2024-09-17T05:34:11.697489+0000","last_became_active":"2024-09-17T05:34:09.380804+0000","last_became_peered":"2024-09-17T05:34:09.380804+0000","last_unstale":"2024-09-17T05:34:11.697489+0000","last_undegraded":"2024-09-17T05:34:11.697489+0000","last_fullsized":"2024-09-17T05:34:11.697489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:29.390472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697902+0000","last_change":"2024-09-17T05:34:10.390640+0000","last_active":"2024-09-17T05:34:11.697902+0000","last_peered":"2024-09-17T05:34:11.697902+0000","last_clean":"2024-09-17T05:34:11.697902+0000","last_became_active":"2024-09-17T05:34:10.380901+0000","last_became_peered":"2024-09-17T05:34:10.380901+0000","last_unstale":"2024-09-17T05:34:11.697902+0000","last_undegraded":"2024-09-17T05:34:11.697902+0000","last_fullsized":"2024-09-17T05:34:11.697902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:32.451935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.458786+0000","last_change":"2024-09-17T05:34:09.369140+0000","last_active":"2024-09-17T05:34:11.458786+0000","last_peered":"2024-09-17T05:34:11.458786+0000","last_clean":"2024-09-17T05:34:11.458786+0000","last_became_active":"2024-09-17T05:34:09.368855+0000","last_became_peered":"2024-09-17T05:34:09.368855+0000","last_unstale":"2024-09-17T05:34:11.458786+0000","last_undegraded":"2024-09-17T05:34:11.458786+0000","last_fullsized":"2024-09-17T05:34:11.458786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:54.748079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.701204+0000","last_change":"2024-09-17T05:34:10.392370+0000","last_active":"2024-09-17T05:34:11.701204+0000","last_peered":"2024-09-17T05:34:11.701204+0000","last_clean":"2024-09-17T05:34:11.701204+0000","last_became_active":"2024-09-17T05:34:10.391731+0000","last_became_peered":"2024-09-17T05:34:10.391731+0000","last_unstale":"2024-09-17T05:34:11.701204+0000","last_undegraded":"2024-09-17T05:34:11.701204+0000","last_fullsized":"2024-09-17T05:34:11.701204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:09.352772+0000","last_clean_scrub_stamp":"2024-09-17T05:34:09.352772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:55.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.390452+0000","last_change":"2024-09-17T05:34:09.378336+0000","last_active":"2024-09-17T05:34:11.390452+0000","last_peered":"2024-09-17T05:34:11.390452+0000","last_clean":"2024-09-17T05:34:11.390452+0000","last_became_active":"2024-09-17T05:34:09.378005+0000","last_became_peered":"2024-09-17T05:34:09.378005+0000","last_unstale":"2024-09-17T05:34:11.390452+0000","last_undegraded":"2024-09-17T05:34:11.390452+0000","last_fullsized":"2024-09-17T05:34:11.390452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:34:08.349148+0000","last_clean_scrub_stamp":"2024-09-17T05:34:08.349148+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:58.580389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":241,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T05:34:11.697748+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:34:11.697748+0000","last_peered":"2024-09-17T05:34:11.697748+0000","last_clean":"2024-09-17T05:34:11.697748+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:34:11.697748+0000","last_undegraded":"2024-09-17T05:34:11.697748+0000","last_fullsized":"2024-09-17T05:34:11.697748+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509555,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58636,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313204,"statfs":{"total":96636764160,"available":96576720896,"internally_reserved":0,"allocated":1101824,"data_stored":474544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64148,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307692,"statfs":{"total":96636764160,"available":96571076608,"internally_reserved":0,"allocated":1105920,"data_stored":475146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64144,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307696,"statfs":{"total":96636764160,"available":96571080704,"internally_reserved":0,"allocated":1101824,"data_stored":474508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58644,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313196,"statfs":{"total":96636764160,"available":96576712704,"internally_reserved":0,"allocated":1110016,"data_stored":475148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509555,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58632,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313208,"statfs":{"total":96636764160,"available":96576724992,"internally_reserved":0,"allocated":1097728,"data_stored":474474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509556,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59212,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312628,"statfs":{"total":96636764160,"available":96576131072,"internally_reserved":0,"allocated":1695744,"data_stored":1064932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509554,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58652,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313188,"statfs":{"total":96636764160,"available":96576704512,"internally_reserved":0,"allocated":1114112,"data_stored":476108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63392,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308448,"statfs":{"total":96636764160,"available":96571850752,"internally_reserved":0,"allocated":1708032,"data_stored":1066488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:34:14.888 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:34:14.888 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:34:15.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.075+0000 7f9c17200640 1 -- 172.21.15.122:0/39473903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 msgr2=0x7f9c100acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.075+0000 7f9c17200640 1 --2- 172.21.15.122:0/39473903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 0x7f9c100acd40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009c80 tx=0x7f9c0001cd10 comp rx=0 tx=0).stop 2024-09-17T05:34:15.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.075+0000 7f9c17200640 1 -- 172.21.15.122:0/39473903 shutdown_connections 2024-09-17T05:34:15.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.075+0000 7f9c17200640 1 --2- 172.21.15.122:0/39473903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c100cd100 0x7f9c100b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.075+0000 7f9c17200640 1 --2- 172.21.15.122:0/39473903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 0x7f9c100acd40 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.075+0000 7f9c17200640 1 -- 172.21.15.122:0/39473903 >> 172.21.15.122:0/39473903 conn(0x7f9c100b18f0 msgr2=0x7f9c100b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:15.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.075+0000 7f9c17200640 1 -- 172.21.15.122:0/39473903 shutdown_connections 2024-09-17T05:34:15.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.075+0000 7f9c17200640 1 -- 172.21.15.122:0/39473903 wait complete. 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 Processor -- start 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 -- start start 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c100b4810 0x7f9c10073970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c100cd100 0x7f9c1009ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 0x7f9c1014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101674b0 con 0x7f9c1015dae0 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10167330 con 0x7f9c100cd100 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9c10167630 con 0x7f9c100b4810 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 0x7f9c1014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 0x7f9c1014b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53316/0 (socket says 172.21.15.122:53316) 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 -- 172.21.15.122:0/2993941375 learned_addr learned my addr 172.21.15.122:0/2993941375 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:15.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c0f400640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c100cd100 0x7f9c1009ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c0fe00640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c100b4810 0x7f9c10073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 -- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c100b4810 msgr2=0x7f9c10073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c100b4810 0x7f9c10073970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 -- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c100cd100 msgr2=0x7f9c1009ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c100cd100 0x7f9c1009ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 -- 172.21.15.122:0/2993941375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1009bcc0 con 0x7f9c1015dae0 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c15200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 0x7f9c1014b7a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c4d0 tx=0x7f9c0800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c07e00640 1 -- 172.21.15.122:0/2993941375 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080052d0 con 0x7f9c1015dae0 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c07e00640 1 -- 172.21.15.122:0/2993941375 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c08005470 con 0x7f9c1015dae0 2024-09-17T05:34:15.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c07e00640 1 -- 172.21.15.122:0/2993941375 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08012870 con 0x7f9c1015dae0 2024-09-17T05:34:15.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1009bfa0 con 0x7f9c1015dae0 2024-09-17T05:34:15.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1014bed0 con 0x7f9c1015dae0 2024-09-17T05:34:15.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.079+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c10069410 con 0x7f9c1015dae0 2024-09-17T05:34:15.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.083+0000 7f9c07e00640 1 -- 172.21.15.122:0/2993941375 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f9c08003600 con 0x7f9c1015dae0 2024-09-17T05:34:15.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.083+0000 7f9c07e00640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9bf0074370 0x7f9bf0076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.083+0000 7f9c07e00640 1 -- 172.21.15.122:0/2993941375 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f9c08092ab0 con 0x7f9c1015dae0 2024-09-17T05:34:15.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.083+0000 7f9c0fe00640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9bf0074370 0x7f9bf0076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.083+0000 7f9c0fe00640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9bf0074370 0x7f9bf0076830 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9c000af000 tx=0x7f9c0001b550 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:15.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.087+0000 7f9c07e00640 1 -- 172.21.15.122:0/2993941375 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9c0805ec10 con 0x7f9c1015dae0 2024-09-17T05:34:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.315+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9c100ae000 con 0x7f9c1015dae0 2024-09-17T05:34:15.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c07e00640 1 -- 172.21.15.122:0/2993941375 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f9c08063640 con 0x7f9c1015dae0 2024-09-17T05:34:15.315 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:34:15.315 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:34:15.315 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:34:15.315 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:34:15.315 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:34:15.316 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9bf0074370 msgr2=0x7f9bf0076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9bf0074370 0x7f9bf0076830 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9c000af000 tx=0x7f9c0001b550 comp rx=0 tx=0).stop 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 msgr2=0x7f9c1014b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 0x7f9c1014b7a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c4d0 tx=0x7f9c0800c990 comp rx=0 tx=0).stop 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 shutdown_connections 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9bf0074370 0x7f9bf0076830 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c1015dae0 0x7f9c1014b7a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c100cd100 0x7f9c1009ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 --2- 172.21.15.122:0/2993941375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c100b4810 0x7f9c10073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 >> 172.21.15.122:0/2993941375 conn(0x7f9c100b18f0 msgr2=0x7f9c100cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 shutdown_connections 2024-09-17T05:34:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.319+0000 7f9c17200640 1 -- 172.21.15.122:0/2993941375 wait complete. 2024-09-17T05:34:15.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:34:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.483+0000 7f2c70000640 1 -- 172.21.15.122:0/4102189813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680ced80 msgr2=0x7f2c680ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.483+0000 7f2c70000640 1 --2- 172.21.15.122:0/4102189813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680ced80 0x7f2c680ae9b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2c680cf340 tx=0x7f2c5401c800 comp rx=0 tx=0).stop 2024-09-17T05:34:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.483+0000 7f2c70000640 1 -- 172.21.15.122:0/4102189813 shutdown_connections 2024-09-17T05:34:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.483+0000 7f2c70000640 1 --2- 172.21.15.122:0/4102189813 >> v2:172.21.15.173:3300/0 conn(0x7f2c680cdb30 0x7f2c680aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.483+0000 7f2c70000640 1 --2- 172.21.15.122:0/4102189813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680ced80 0x7f2c680ae9b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.483+0000 7f2c70000640 1 -- 172.21.15.122:0/4102189813 >> 172.21.15.122:0/4102189813 conn(0x7f2c680d1dc0 msgr2=0x7f2c680d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.483+0000 7f2c70000640 1 -- 172.21.15.122:0/4102189813 shutdown_connections 2024-09-17T05:34:15.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.483+0000 7f2c70000640 1 -- 172.21.15.122:0/4102189813 wait complete. 2024-09-17T05:34:15.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 Processor -- start 2024-09-17T05:34:15.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 -- start start 2024-09-17T05:34:15.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680cdb30 0x7f2c681f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c680ced80 0x7f2c6814e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6815bea0 0x7f2c6814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c681692f0 con 0x7f2c680cdb30 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c68169170 con 0x7f2c6815bea0 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2c68169470 con 0x7f2c680ced80 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680cdb30 0x7f2c681f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680cdb30 0x7f2c681f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53328/0 (socket says 172.21.15.122:53328) 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 -- 172.21.15.122:0/3915756412 learned_addr learned my addr 172.21.15.122:0/3915756412 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6cc00640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c680ced80 0x7f2c6814e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6e000640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6815bea0 0x7f2c6814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 -- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c680ced80 msgr2=0x7f2c6814e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c680ced80 0x7f2c6814e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 -- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6815bea0 msgr2=0x7f2c6814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6815bea0 0x7f2c6814fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 -- 172.21.15.122:0/3915756412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c680cb130 con 0x7f2c680cdb30 2024-09-17T05:34:15.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c6d600640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680cdb30 0x7f2c681f98f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c400 tx=0x7f2c5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c65600640 1 -- 172.21.15.122:0/3915756412 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c011790 con 0x7f2c680cdb30 2024-09-17T05:34:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c65600640 1 -- 172.21.15.122:0/3915756412 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c5c017070 con 0x7f2c680cdb30 2024-09-17T05:34:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6814f250 con 0x7f2c680cdb30 2024-09-17T05:34:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c65600640 1 -- 172.21.15.122:0/3915756412 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c011cc0 con 0x7f2c680cdb30 2024-09-17T05:34:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.487+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c681508d0 con 0x7f2c680cdb30 2024-09-17T05:34:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.491+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c680ce670 con 0x7f2c680cdb30 2024-09-17T05:34:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.491+0000 7f2c65600640 1 -- 172.21.15.122:0/3915756412 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f2c5c01c460 con 0x7f2c680cdb30 2024-09-17T05:34:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.491+0000 7f2c65600640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c48074370 0x7f2c48076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.491+0000 7f2c65600640 1 -- 172.21.15.122:0/3915756412 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f2c5c092d30 con 0x7f2c680cdb30 2024-09-17T05:34:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.491+0000 7f2c6cc00640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c48074370 0x7f2c48076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.491+0000 7f2c6cc00640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c48074370 0x7f2c48076830 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2c540af000 tx=0x7f2c5401b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:15.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.495+0000 7f2c65600640 1 -- 172.21.15.122:0/3915756412 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2c5c05ee90 con 0x7f2c680cdb30 2024-09-17T05:34:15.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c680b0820 con 0x7f2c680cdb30 2024-09-17T05:34:15.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c65600640 1 -- 172.21.15.122:0/3915756412 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5237 (secure 0 0 0) 0x7f2c5c0638c0 con 0x7f2c680cdb30 2024-09-17T05:34:15.696 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:15.696 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":56,"btime":"2024-09-17T05:34:13:395200+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1117224890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1117224890},{"type":"v1","addr":"172.21.15.122:6835","nonce":1117224890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2271477440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2271477440},{"type":"v1","addr":"172.21.15.173:6837","nonce":2271477440}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3081382903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3081382903},{"type":"v1","addr":"172.21.15.173:6836","nonce":3081382903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:09.761813+0000","modified":"2024-09-17T05:34:12.779372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6834/3516694104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3516694104},{"type":"v1","addr":"172.21.15.122:6834","nonce":3516694104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-17T05:34:15.696 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 56 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c48074370 msgr2=0x7f2c48076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c48074370 0x7f2c48076830 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2c540af000 tx=0x7f2c5401b040 comp rx=0 tx=0).stop 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680cdb30 msgr2=0x7f2c681f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680cdb30 0x7f2c681f98f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c400 tx=0x7f2c5c00c8c0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 shutdown_connections 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c48074370 0x7f2c48076830 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6815bea0 0x7f2c6814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c680ced80 0x7f2c6814e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 --2- 172.21.15.122:0/3915756412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c680cdb30 0x7f2c681f98f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 >> 172.21.15.122:0/3915756412 conn(0x7f2c680d1dc0 msgr2=0x7f2c6815ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 shutdown_connections 2024-09-17T05:34:15.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:15.699+0000 7f2c70000640 1 -- 172.21.15.122:0/3915756412 wait complete. 2024-09-17T05:34:16.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:34:09.761813+0000', 'modified': '2024-09-17T05:34:12.779372+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7029}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7029': {'gid': 7029, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6834/3516694104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6832', 'nonce': 3516694104}, {'type': 'v1', 'addr': '172.21.15.122:6834', 'nonce': 3516694104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7029, 'qdb_cluster': [7029]} 2024-09-17T05:34:16.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:34:16.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:34:16.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- 172.21.15.122:0/2473326040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb406d240 msgr2=0x7fbcb4071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 --2- 172.21.15.122:0/2473326040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb406d240 0x7fbcb4071040 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009900 tx=0x7fbca401c880 comp rx=0 tx=0).stop 2024-09-17T05:34:16.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- 172.21.15.122:0/2473326040 shutdown_connections 2024-09-17T05:34:16.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 --2- 172.21.15.122:0/2473326040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbcb40b5b60 0x7fbcb4146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 --2- 172.21.15.122:0/2473326040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb406d240 0x7fbcb4071040 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- 172.21.15.122:0/2473326040 >> 172.21.15.122:0/2473326040 conn(0x7fbcb40b2890 msgr2=0x7fbcb40b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- 172.21.15.122:0/2473326040 shutdown_connections 2024-09-17T05:34:16.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- 172.21.15.122:0/2473326040 wait complete. 2024-09-17T05:34:16.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 Processor -- start 2024-09-17T05:34:16.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- start start 2024-09-17T05:34:16.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbcb406d240 0x7fbcb415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbcb4073f70 0x7fbcb40ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb40b5b60 0x7fbcb41435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb40d11b0 con 0x7fbcb40b5b60 2024-09-17T05:34:16.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb40d1030 con 0x7fbcb4073f70 2024-09-17T05:34:16.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcbac00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb40d1330 con 0x7fbcb406d240 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb40b5b60 0x7fbcb41435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb3e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbcb406d240 0x7fbcb415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb40b5b60 0x7fbcb41435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53360/0 (socket says 172.21.15.122:53360) 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb8c00640 1 -- 172.21.15.122:0/1747909327 learned_addr learned my addr 172.21.15.122:0/1747909327 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbcb406d240 0x7fbcb415b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:41870/0 (socket says 172.21.15.122:41870) 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcab400640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbcb4073f70 0x7fbcb40ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb3e00640 1 -- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbcb4073f70 msgr2=0x7fbcb40ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbcb4073f70 0x7fbcb40ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb3e00640 1 -- 172.21.15.122:0/1747909327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb40b5b60 msgr2=0x7fbcb41435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb40b5b60 0x7fbcb41435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb3e00640 1 -- 172.21.15.122:0/1747909327 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb413d800 con 0x7fbcb406d240 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.267+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb40b5b60 0x7fbcb41435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:34:16.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbcb406d240 0x7fbcb415b580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbca40ac000 tx=0x7fbca4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcb0c00640 1 -- 172.21.15.122:0/1747909327 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca40b2040 con 0x7fbcb406d240 2024-09-17T05:34:16.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcb0c00640 1 -- 172.21.15.122:0/1747909327 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbca4004450 con 0x7fbcb406d240 2024-09-17T05:34:16.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcb0c00640 1 -- 172.21.15.122:0/1747909327 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4005290 con 0x7fbcb406d240 2024-09-17T05:34:16.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcbac00640 1 -- 172.21.15.122:0/1747909327 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb413dae0 con 0x7fbcb406d240 2024-09-17T05:34:16.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcbac00640 1 -- 172.21.15.122:0/1747909327 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb413e230 con 0x7fbcb406d240 2024-09-17T05:34:16.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcb0c00640 1 -- 172.21.15.122:0/1747909327 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fbca4005430 con 0x7fbcb406d240 2024-09-17T05:34:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcbdfd6000 1 Processor -- start 2024-09-17T05:34:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcb0c00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc90074370 0x7fbc90076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcbdfd6000 1 -- start start 2024-09-17T05:34:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.271+0000 7fbcb0c00640 1 -- 172.21.15.122:0/1747909327 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fbca412c6c0 con 0x7fbcb406d240 2024-09-17T05:34:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc90074370 0x7fbc90076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac5adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac0e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afac0f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562afab2f3f0 con 0x562afac5a9d0 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562afaa6cb20 con 0x562afac5b2f0 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x562afaa5bf50 con 0x562afac0ee10 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac5adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc90074370 0x7fbc90076830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbcac011670 tx=0x7fbcac0095e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac5adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53370/0 (socket says 172.21.15.122:53370) 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 -- 172.21.15.122:0/1648434966 learned_addr learned my addr 172.21.15.122:0/1648434966 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcab400640 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac0e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afac0f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 -- 172.21.15.122:0/1648434966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 msgr2=0x562afac0f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afac0f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 -- 172.21.15.122:0/1648434966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 msgr2=0x562afac0e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac0e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 -- 172.21.15.122:0/1648434966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afaa23590 con 0x562afac5a9d0 2024-09-17T05:34:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac5adb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009900 tx=0x7fbca40f84c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67e0cd9210579b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbca8c00640 1 -- 172.21.15.122:0/1648434966 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca40f8b70 con 0x562afac5a9d0 2024-09-17T05:34:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbca8c00640 1 -- 172.21.15.122:0/1648434966 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbca40fd5a0 con 0x562afac5a9d0 2024-09-17T05:34:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbca8c00640 1 -- 172.21.15.122:0/1648434966 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4102450 con 0x562afac5a9d0 2024-09-17T05:34:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/1648434966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 msgr2=0x562afac5adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac5adb0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009900 tx=0x7fbca40f84c0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/1648434966 shutdown_connections 2024-09-17T05:34:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afac0f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac0e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/1648434966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac5adb0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/1648434966 >> 172.21.15.122:0/1648434966 conn(0x562afaaf8130 msgr2=0x562afac3de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/1648434966 shutdown_connections 2024-09-17T05:34:16.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/1648434966 wait complete. 2024-09-17T05:34:16.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 -- start start 2024-09-17T05:34:16.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.275+0000 7fbcbdfd6000 1 Processor -- start 2024-09-17T05:34:16.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afac3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac3be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac3c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562afa9f65d0 con 0x562afac5a9d0 2024-09-17T05:34:16.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562afaa5fd60 con 0x562afac5b2f0 2024-09-17T05:34:16.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x562afaa02500 con 0x562afac0ee10 2024-09-17T05:34:16.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 10 client.0 _ll_register_callbacks cb 0x562afaa06810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac3be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac3be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53382/0 (socket says 172.21.15.122:53382) 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 -- 172.21.15.122:0/575329794 learned_addr learned my addr 172.21.15.122:0/575329794 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afac3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac3c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 -- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 msgr2=0x562afac3b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afac3b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 -- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 msgr2=0x562afac3c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac3c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 -- 172.21.15.122:0/575329794 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afa9fa170 con 0x562afac5a9d0 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac3c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcab400640 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac3be70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c012570 tx=0x7fbc9c012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7adce47c8cc787c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 1 -- 172.21.15.122:0/575329794 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c019070 con 0x562afac5a9d0 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 1 -- 172.21.15.122:0/575329794 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fbc9c003b60 con 0x562afac5a9d0 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/575329794 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562afa9f40f0 con 0x562afac5a9d0 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 1 -- 172.21.15.122:0/575329794 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c024050 con 0x562afac5a9d0 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 10 client.7161 fetch_fsmap learned FSMap version 56 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/575329794 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x562afaae0ef0 con 0x562afac5a9d0 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 1 -- 172.21.15.122:0/575329794 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562afa9f40f0 con 0x562afac5a9d0 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 1 -- 172.21.15.122:0/575329794 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7fbc9c027020 con 0x562afac5a9d0 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 10 client.7161 fetch_fsmap finished waiting for FSMap version 56 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 10 client.7161 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 20 client.7161 populate_metadata read hostname 'smithi122' 2024-09-17T05:34:16.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] conn(0x562afac3d0f0 0x562afac6cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 4 client.7161 mds_command: new command op to 7029 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/575329794 --> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562afaa6ea70 con 0x562afac3d0f0 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] conn(0x562afac3d0f0 0x562afac6cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] conn(0x562afac3d0f0 0x562afac6cc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c710bbed9b0437d5 server_cookie=ed34b29e88a682c4 in_seq=0 out_seq=0 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 10 client.7161 ms_handle_connect on v2:172.21.15.122:6832/3516694104 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 1 -- 172.21.15.122:0/575329794 <== mds.0 v2:172.21.15.122:6832/3516694104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562afaa6ea70 con 0x562afac3d0f0 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 10 client.7161 handle_command_reply: tid=0 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 client.7161 shutdown 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/575329794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 msgr2=0x562afac3be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac3be70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c012570 tx=0x7fbc9c012a30 comp rx=0 tx=0).stop 2024-09-17T05:34:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/575329794 shutdown_connections 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] conn(0x562afac3d0f0 0x562afac6cc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5b2f0 0x562afac3c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5a9d0 0x562afac3be70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbca1600640 0 client.7161 ms_handle_reset on v2:172.21.15.122:6832/3516694104 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/575329794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afac3b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.279+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/575329794 >> 172.21.15.122:0/575329794 conn(0x562afaaf8130 msgr2=0x562afac0dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.283+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/575329794 shutdown_connections 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.283+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/575329794 wait complete. 2024-09-17T05:34:16.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.283+0000 7fbcbdfd6000 20 client.7161 trim_cache size 0 max 16384 2024-09-17T05:34:16.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.299+0000 7fbcbdfd6000 1 Processor -- start 2024-09-17T05:34:16.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.299+0000 7fbcbdfd6000 1 -- start start 2024-09-17T05:34:16.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.299+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afacccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.299+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 0x562afaccd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.299+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5b2f0 0x562afaccd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.299+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4204190 con 0x562afac5b2f0 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.299+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb42031b0 con 0x562afac5a9d0 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.299+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb41fb8d0 con 0x562afac0ee10 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5b2f0 0x562afaccd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5b2f0 0x562afaccd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53390/0 (socket says 172.21.15.122:53390) 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 -- 172.21.15.122:0/3405362504 learned_addr learned my addr 172.21.15.122:0/3405362504 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afacccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcab400640 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 0x562afaccd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 -- 172.21.15.122:0/3405362504 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 msgr2=0x562afacccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afacccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 -- 172.21.15.122:0/3405362504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 msgr2=0x562afaccd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:16.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 0x562afaccd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 -- 172.21.15.122:0/3405362504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afaab7d90 con 0x562afac5b2f0 2024-09-17T05:34:16.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5b2f0 0x562afaccd850 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00fce0 tx=0x7fbcac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55882d5b68c1b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbca3e00640 1 -- 172.21.15.122:0/3405362504 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac02a430 con 0x562afac5b2f0 2024-09-17T05:34:16.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbca3e00640 1 -- 172.21.15.122:0/3405362504 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcac02a6c0 con 0x562afac5b2f0 2024-09-17T05:34:16.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbca3e00640 1 -- 172.21.15.122:0/3405362504 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac02aad0 con 0x562afac5b2f0 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/3405362504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5b2f0 msgr2=0x562afaccd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5b2f0 0x562afaccd850 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00fce0 tx=0x7fbcac014000 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/3405362504 shutdown_connections 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac5b2f0 0x562afaccd850 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 0x562afaccd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/3405362504 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac0ee10 0x562afacccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/3405362504 >> 172.21.15.122:0/3405362504 conn(0x562afaaf8130 msgr2=0x562afac67fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/3405362504 shutdown_connections 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/3405362504 wait complete. 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 -- start start 2024-09-17T05:34:16.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.303+0000 7fbcbdfd6000 1 Processor -- start 2024-09-17T05:34:16.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac0ee10 0x562afacd0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 0x562afacd1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac5b2f0 0x562afacd1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562afaae0a60 con 0x562afac0ee10 2024-09-17T05:34:16.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562afab1b730 con 0x562afac5a9d0 2024-09-17T05:34:16.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x562afab2f3f0 con 0x562afac5b2f0 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 10 client.0 _ll_register_callbacks cb 0x7fbc9c004bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac0ee10 0x562afacd0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac0ee10 0x562afacd0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53400/0 (socket says 172.21.15.122:53400) 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 -- 172.21.15.122:0/677304938 learned_addr learned my addr 172.21.15.122:0/677304938 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcab400640 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 0x562afacd1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb8c00640 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac5b2f0 0x562afacd1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 -- 172.21.15.122:0/677304938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac5b2f0 msgr2=0x562afacd1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac5b2f0 0x562afacd1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 -- 172.21.15.122:0/677304938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 msgr2=0x562afacd1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 0x562afacd1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 -- 172.21.15.122:0/677304938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afaa22a90 con 0x562afac0ee10 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcb3e00640 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac0ee10 0x562afacd0c00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbca401cdf0 tx=0x7fbca4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea62e3de5b68ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 1 -- 172.21.15.122:0/677304938 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca40fd5a0 con 0x562afac0ee10 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 1 -- 172.21.15.122:0/677304938 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fbca412c940 con 0x562afac0ee10 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/677304938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcac02a6c0 con 0x562afac0ee10 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 1 -- 172.21.15.122:0/677304938 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fbca40f4de0 con 0x562afac0ee10 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 10 client.7167 fetch_fsmap learned FSMap version 56 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/677304938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x562afaa01050 con 0x562afac0ee10 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 1 -- 172.21.15.122:0/677304938 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcac02a6c0 con 0x562afac0ee10 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 1 -- 172.21.15.122:0/677304938 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7fbca40fd1e0 con 0x562afac0ee10 2024-09-17T05:34:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 10 client.7167 fetch_fsmap finished waiting for FSMap version 56 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 10 client.7167 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 20 client.7167 populate_metadata read hostname 'smithi122' 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] conn(0x562afac6ebe0 0x562afac6efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 4 client.7167 mds_command: new command op to 7029 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/677304938 --> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] -- command(tid 0: {"prefix": "status"}) -- 0x562afaa6a520 con 0x562afac6ebe0 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcab400640 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] conn(0x562afac6ebe0 0x562afac6efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 10 client.7167 ms_handle_connect on v2:172.21.15.122:6832/3516694104 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcab400640 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] conn(0x562afac6ebe0 0x562afac6efc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bdb534f09084288 server_cookie=203a146d893e9482 in_seq=0 out_seq=0 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 1 -- 172.21.15.122:0/677304938 <== mds.0 v2:172.21.15.122:6832/3516694104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562afaa6a520 con 0x562afac6ebe0 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 10 client.7167 handle_command_reply: tid=0 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 client.7167 shutdown 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/677304938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac0ee10 msgr2=0x562afacd0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac0ee10 0x562afacd0c00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbca401cdf0 tx=0x7fbca4005720 comp rx=0 tx=0).stop 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/677304938 shutdown_connections 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.122:6832/3516694104,v1:172.21.15.122:6834/3516694104] conn(0x562afac6ebe0 0x562afac6efc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x562afac5b2f0 0x562afacd1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562afac5a9d0 0x562afacd1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 --2- 172.21.15.122:0/677304938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562afac0ee10 0x562afacd0c00 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbca1600640 0 client.7167 ms_handle_reset on v2:172.21.15.122:6832/3516694104 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/677304938 >> 172.21.15.122:0/677304938 conn(0x562afaaf8130 msgr2=0x562afac16a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/677304938 shutdown_connections 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 1 -- 172.21.15.122:0/677304938 wait complete. 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbdfd6000 20 client.7167 trim_cache size 0 max 16384 2024-09-17T05:34:16.310 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:34:16.311 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:34:16.311 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:34:16.311 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 7029, 2024-09-17T05:34:16.311 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:34:16.311 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:34:16.311 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:34:16.311 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 5.9462531289999996, 2024-09-17T05:34:16.311 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 56, 2024-09-17T05:34:16.312 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 90, 2024-09-17T05:34:16.312 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-09-17T05:34:16.312 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 10.494842735000001 2024-09-17T05:34:16.312 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 -- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc90074370 msgr2=0x7fbc90076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc90074370 0x7fbc90076830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbcac011670 tx=0x7fbcac0095e0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 -- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbcb406d240 msgr2=0x7fbcb415b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbcb406d240 0x7fbcb415b580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbca40ac000 tx=0x7fbca4004040 comp rx=0 tx=0).stop 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 -- 172.21.15.122:0/1747909327 shutdown_connections 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc90074370 0x7fbc90076830 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbcb40b5b60 0x7fbcb41435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbcb4073f70 0x7fbcb40ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 --2- 172.21.15.122:0/1747909327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbcb406d240 0x7fbcb415b580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.307+0000 7fbcbac00640 1 -- 172.21.15.122:0/1747909327 >> 172.21.15.122:0/1747909327 conn(0x7fbcb40b2890 msgr2=0x7fbcb406e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.311+0000 7fbcbac00640 1 -- 172.21.15.122:0/1747909327 shutdown_connections 2024-09-17T05:34:16.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.311+0000 7fbcbac00640 1 -- 172.21.15.122:0/1747909327 wait complete. 2024-09-17T05:34:16.324 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:34:16.328 INFO:tasks.ceph.mds.c.smithi122.stderr:2024-09-17T05:34:16.331+0000 7f1874200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:34:16.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.471+0000 7f9aa7000640 1 -- 172.21.15.122:0/3595603301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a84003a60 msgr2=0x7f9a84003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.471+0000 7f9aa7000640 1 --2- 172.21.15.122:0/3595603301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a84003a60 0x7f9a84003e80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009b30 tx=0x7f9a9001d210 comp rx=0 tx=0).stop 2024-09-17T05:34:16.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.471+0000 7f9aa7000640 1 -- 172.21.15.122:0/3595603301 shutdown_connections 2024-09-17T05:34:16.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.471+0000 7f9aa7000640 1 --2- 172.21.15.122:0/3595603301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a84003a60 0x7f9a84003e80 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.471+0000 7f9aa7000640 1 -- 172.21.15.122:0/3595603301 >> 172.21.15.122:0/3595603301 conn(0x7f9aa00d1dc0 msgr2=0x7f9aa00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.471+0000 7f9aa7000640 1 -- 172.21.15.122:0/3595603301 shutdown_connections 2024-09-17T05:34:16.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.471+0000 7f9aa7000640 1 -- 172.21.15.122:0/3595603301 wait complete. 2024-09-17T05:34:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 Processor -- start 2024-09-17T05:34:16.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 -- start start 2024-09-17T05:34:16.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a84003a60 0x7f9aa00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9aa01565e0 0x7f9aa0156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aa01f8f60 0x7f9aa01f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa00cd080 con 0x7f9aa01f8f60 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa00ccf00 con 0x7f9a84003a60 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa00cd200 con 0x7f9aa01565e0 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aa01f8f60 0x7f9aa01f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aa01f8f60 0x7f9aa01f93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53418/0 (socket says 172.21.15.122:53418) 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 -- 172.21.15.122:0/866617479 learned_addr learned my addr 172.21.15.122:0/866617479 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9a97400640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9aa01565e0 0x7f9aa0156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9a9fe00640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a84003a60 0x7f9aa00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 -- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9aa01565e0 msgr2=0x7f9aa0156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9aa01565e0 0x7f9aa0156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 -- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a84003a60 msgr2=0x7f9aa00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a84003a60 0x7f9aa00cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 -- 172.21.15.122:0/866617479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa0157060 con 0x7f9aa01f8f60 2024-09-17T05:34:16.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa5000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aa01f8f60 0x7f9aa01f93e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0073f90 tx=0x7f9a980019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9a9cc00640 1 -- 172.21.15.122:0/866617479 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a980107d0 con 0x7f9aa01f8f60 2024-09-17T05:34:16.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9a9cc00640 1 -- 172.21.15.122:0/866617479 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a98016070 con 0x7f9aa01f8f60 2024-09-17T05:34:16.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa0157340 con 0x7f9aa01f8f60 2024-09-17T05:34:16.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9a9cc00640 1 -- 172.21.15.122:0/866617479 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98010ce0 con 0x7f9aa01f8f60 2024-09-17T05:34:16.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.475+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa0157900 con 0x7f9aa01f8f60 2024-09-17T05:34:16.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.479+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa00cd760 con 0x7f9aa01f8f60 2024-09-17T05:34:16.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.479+0000 7f9a9cc00640 1 -- 172.21.15.122:0/866617479 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f9a9801b460 con 0x7f9aa01f8f60 2024-09-17T05:34:16.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.479+0000 7f9a9cc00640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a7c074370 0x7f9a7c076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:16.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.479+0000 7f9a9cc00640 1 -- 172.21.15.122:0/866617479 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f9a98093940 con 0x7f9aa01f8f60 2024-09-17T05:34:16.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.479+0000 7f9a9fe00640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a7c074370 0x7f9a7c076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.479+0000 7f9a9fe00640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a7c074370 0x7f9a7c076830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9a900ae000 tx=0x7f9a90019fb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:16.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.483+0000 7f9a9cc00640 1 -- 172.21.15.122:0/866617479 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9a980600f0 con 0x7f9aa01f8f60 2024-09-17T05:34:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.687+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9aa006d080 con 0x7f9aa01f8f60 2024-09-17T05:34:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9a9cc00640 1 -- 172.21.15.122:0/866617479 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7f9a9805f9f0 con 0x7f9aa01f8f60 2024-09-17T05:34:16.686 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a7c074370 msgr2=0x7f9a7c076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a7c074370 0x7f9a7c076830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9a900ae000 tx=0x7f9a90019fb0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aa01f8f60 msgr2=0x7f9aa01f93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aa01f8f60 0x7f9aa01f93e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0073f90 tx=0x7f9a980019c0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 shutdown_connections 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a7c074370 0x7f9a7c076830 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9aa01f8f60 0x7f9aa01f93e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9aa01565e0 0x7f9aa0156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 --2- 172.21.15.122:0/866617479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a84003a60 0x7f9aa00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 >> 172.21.15.122:0/866617479 conn(0x7f9aa00d1dc0 msgr2=0x7f9aa00af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 shutdown_connections 2024-09-17T05:34:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16.691+0000 7f9aa7000640 1 -- 172.21.15.122:0/866617479 wait complete. 2024-09-17T05:34:16.708 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:34:16.708 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:34:16.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:34:16.708 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:34:16.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:16.708 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:34:16.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:34:16.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:34:16.709 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:16.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:16.709 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:34:16.719 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 41910sec preferred_lft 41910sec 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:34:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:16.722 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:34:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:34:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:34:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:34:16.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:16.722 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:34:16.778 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:34:16.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:16.781 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:34:16.832 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:34:16.833 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:34:16.833 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:34:16.833 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:34:16.833 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:34:16.833 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:34:16.833 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:34:16.833 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:34:16.833 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:34:16.834 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:34:16.834 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:34:16.834 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:34:16.834 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:34:16.834 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:34:16.834 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:34:16.835 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:34:16.835 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:34:16.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:34:16.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:16.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:16.966 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:34:17.017 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:34:17.017 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:34:17.018 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:17.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:17.019 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:34:17.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:34:17.081 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:17.133 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:34:17.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:34:17.134 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:17.185 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:34:17.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:17.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:17.273 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:17.273 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:17.273 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:34:17.288 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:17.289 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:17.289 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:34:17.289 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:34:17.289 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:34:17.297 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:34:17.297 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:34:17.298 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:34:17.298 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:34:17.298 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:34:17.298 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:34:17.298 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:34:17.298 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:34:17.301 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:34:17.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:17.302 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:34:17.321 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:34:17.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:34:17.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:34:17.429 INFO:teuthology.orchestra.run.smithi122.stdout:client6233 2024-09-17T05:34:17.429 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:34:17.429 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:17.438 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:34:17.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.627+0000 7f2e70200640 1 -- 172.21.15.122:0/2036548003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 msgr2=0x7f2e680ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.627+0000 7f2e70200640 1 --2- 172.21.15.122:0/2036548003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 0x7f2e680ce350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2e68051480 tx=0x7f2e5401dbc0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 -- 172.21.15.122:0/2036548003 shutdown_connections 2024-09-17T05:34:17.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 --2- 172.21.15.122:0/2036548003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 0x7f2e680ce350 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 --2- 172.21.15.122:0/2036548003 >> v2:172.21.15.173:3300/0 conn(0x7f2e680734f0 0x7f2e6815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 -- 172.21.15.122:0/2036548003 >> 172.21.15.122:0/2036548003 conn(0x7f2e680cfbc0 msgr2=0x7f2e680cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:17.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 -- 172.21.15.122:0/2036548003 shutdown_connections 2024-09-17T05:34:17.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 -- 172.21.15.122:0/2036548003 wait complete. 2024-09-17T05:34:17.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 Processor -- start 2024-09-17T05:34:17.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 -- start start 2024-09-17T05:34:17.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e680734f0 0x7f2e68143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e680ce890 0x7f2e6809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 0x7f2e681fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6815ed80 con 0x7f2e680cf440 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6815ec00 con 0x7f2e680ce890 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e70200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2e68167360 con 0x7f2e680734f0 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 0x7f2e681fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 0x7f2e681fb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53428/0 (socket says 172.21.15.122:53428) 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 -- 172.21.15.122:0/990539402 learned_addr learned my addr 172.21.15.122:0/990539402 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:17.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6ce00640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e680ce890 0x7f2e6809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6d800640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e680734f0 0x7f2e68143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 -- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e680734f0 msgr2=0x7f2e68143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e680734f0 0x7f2e68143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 -- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e680ce890 msgr2=0x7f2e6809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e680ce890 0x7f2e6809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 -- 172.21.15.122:0/990539402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e680b4060 con 0x7f2e680cf440 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6d800640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e680734f0 0x7f2e68143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6ce00640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e680ce890 0x7f2e6809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:17.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.631+0000 7f2e6e200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 0x7f2e681fb9f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c3a0 tx=0x7f2e6000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e65600640 1 -- 172.21.15.122:0/990539402 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60017070 con 0x7f2e680cf440 2024-09-17T05:34:17.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e65600640 1 -- 172.21.15.122:0/990539402 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e60003600 con 0x7f2e680cf440 2024-09-17T05:34:17.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e65600640 1 -- 172.21.15.122:0/990539402 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60003a50 con 0x7f2e680cf440 2024-09-17T05:34:17.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e680b4340 con 0x7f2e680cf440 2024-09-17T05:34:17.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e681fb4d0 con 0x7f2e680cf440 2024-09-17T05:34:17.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e68069410 con 0x7f2e680cf440 2024-09-17T05:34:17.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e65600640 1 -- 172.21.15.122:0/990539402 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f2e6001a070 con 0x7f2e680cf440 2024-09-17T05:34:17.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e65600640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e48074370 0x7f2e48076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:17.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e65600640 1 -- 172.21.15.122:0/990539402 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f2e600923c0 con 0x7f2e680cf440 2024-09-17T05:34:17.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.635+0000 7f2e6d800640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e48074370 0x7f2e48076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:17.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.639+0000 7f2e6d800640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e48074370 0x7f2e48076830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009f10 tx=0x7f2e5c0098f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:17.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.639+0000 7f2e65600640 1 -- 172.21.15.122:0/990539402 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2e6005eb70 con 0x7f2e680cf440 2024-09-17T05:34:17.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:17.843+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f2e6815a9d0 con 0x7f2e680cf440 2024-09-17T05:34:18.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.423+0000 7f2e65600640 1 -- 172.21.15.122:0/990539402 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) ==== 136+0+0 (secure 0 0 0) 0x7f2e6005e470 con 0x7f2e680cf440 2024-09-17T05:34:18.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e48074370 msgr2=0x7f2e48076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e48074370 0x7f2e48076830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009f10 tx=0x7f2e5c0098f0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 msgr2=0x7f2e681fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 0x7f2e681fb9f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000c3a0 tx=0x7f2e6000c860 comp rx=0 tx=0).stop 2024-09-17T05:34:18.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 shutdown_connections 2024-09-17T05:34:18.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2e48074370 0x7f2e48076830 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e680cf440 0x7f2e681fb9f0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e680ce890 0x7f2e6809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 --2- 172.21.15.122:0/990539402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e680734f0 0x7f2e68143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 >> 172.21.15.122:0/990539402 conn(0x7f2e680cfbc0 msgr2=0x7f2e681523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:18.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 shutdown_connections 2024-09-17T05:34:18.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.427+0000 7f2e70200640 1 -- 172.21.15.122:0/990539402 wait complete. 2024-09-17T05:34:18.444 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:34:18.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 -- 172.21.15.122:0/1950029403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b006e810 msgr2=0x7f25b006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 --2- 172.21.15.122:0/1950029403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b006e810 0x7f25b006f1f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009c80 tx=0x7f25a801cd70 comp rx=0 tx=0).stop 2024-09-17T05:34:18.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 -- 172.21.15.122:0/1950029403 shutdown_connections 2024-09-17T05:34:18.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 --2- 172.21.15.122:0/1950029403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b006e810 0x7f25b006f1f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 --2- 172.21.15.122:0/1950029403 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b00cf300 0x7f25b006ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 -- 172.21.15.122:0/1950029403 >> 172.21.15.122:0/1950029403 conn(0x7f25b00b18f0 msgr2=0x7f25b00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:18.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 -- 172.21.15.122:0/1950029403 shutdown_connections 2024-09-17T05:34:18.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 -- 172.21.15.122:0/1950029403 wait complete. 2024-09-17T05:34:18.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 Processor -- start 2024-09-17T05:34:18.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.591+0000 7f25b7e00640 1 -- start start 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b006e810 0x7f25b01f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b00cf300 0x7f25b009bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b015ade0 0x7f25b00cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0168840 con 0x7f25b015ade0 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f25b01686c0 con 0x7f25b006e810 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f25b01689c0 con 0x7f25b00cf300 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b015ade0 0x7f25b00cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b015ade0 0x7f25b00cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53452/0 (socket says 172.21.15.122:53452) 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 -- 172.21.15.122:0/116750188 learned_addr learned my addr 172.21.15.122:0/116750188 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:18.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5400640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b006e810 0x7f25b01f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b4a00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b00cf300 0x7f25b009bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 -- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b00cf300 msgr2=0x7f25b009bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b00cf300 0x7f25b009bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 -- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b006e810 msgr2=0x7f25b01f9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b006e810 0x7f25b01f9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 -- 172.21.15.122:0/116750188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00ade70 con 0x7f25b015ade0 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b4a00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b00cf300 0x7f25b009bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b5e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b015ade0 0x7f25b00cdae0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f25a801cba0 tx=0x7f25a80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25ad600640 1 -- 172.21.15.122:0/116750188 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a80bb070 con 0x7f25b015ade0 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25ad600640 1 -- 172.21.15.122:0/116750188 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a8004d60 con 0x7f25b015ade0 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b013e390 con 0x7f25b015ade0 2024-09-17T05:34:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25ad600640 1 -- 172.21.15.122:0/116750188 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a80b2040 con 0x7f25b015ade0 2024-09-17T05:34:18.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b013e950 con 0x7f25b015ade0 2024-09-17T05:34:18.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.595+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b013eb50 con 0x7f25b015ade0 2024-09-17T05:34:18.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.599+0000 7f25ad600640 1 -- 172.21.15.122:0/116750188 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f25a80be070 con 0x7f25b015ade0 2024-09-17T05:34:18.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.599+0000 7f25ad600640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2590074370 0x7f2590076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.599+0000 7f25ad600640 1 -- 172.21.15.122:0/116750188 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f25a812d650 con 0x7f25b015ade0 2024-09-17T05:34:18.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.599+0000 7f25b5400640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2590074370 0x7f2590076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.599+0000 7f25b5400640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2590074370 0x7f2590076830 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f25a400b810 tx=0x7f25a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:18.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.603+0000 7f25ad600640 1 -- 172.21.15.122:0/116750188 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f25b013eb50 con 0x7f25b015ade0 2024-09-17T05:34:18.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.791+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f25b00cf720 con 0x7f2590074370 2024-09-17T05:34:18.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25ad600640 1 -- 172.21.15.122:0/116750188 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f25b00cf720 con 0x7f2590074370 2024-09-17T05:34:18.790 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:34:18.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:34:18.791 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:18.791 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:34:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2590074370 msgr2=0x7f2590076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2590074370 0x7f2590076830 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f25a400b810 tx=0x7f25a4008040 comp rx=0 tx=0).stop 2024-09-17T05:34:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b015ade0 msgr2=0x7f25b00cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b015ade0 0x7f25b00cdae0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f25a801cba0 tx=0x7f25a80093e0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 shutdown_connections 2024-09-17T05:34:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2590074370 0x7f2590076830 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b015ade0 0x7f25b00cdae0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b00cf300 0x7f25b009bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 --2- 172.21.15.122:0/116750188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b006e810 0x7f25b01f9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 >> 172.21.15.122:0/116750188 conn(0x7f25b00b18f0 msgr2=0x7f25b00b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 shutdown_connections 2024-09-17T05:34:18.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.795+0000 7f25b7e00640 1 -- 172.21.15.122:0/116750188 wait complete. 2024-09-17T05:34:18.812 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:34:18.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 -- 172.21.15.122:0/1131700963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e40b4bd0 msgr2=0x7ff4e40b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/1131700963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e40b4bd0 0x7ff4e40b2bf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4069820 tx=0x7ff4cc01c800 comp rx=0 tx=0).stop 2024-09-17T05:34:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 -- 172.21.15.122:0/1131700963 shutdown_connections 2024-09-17T05:34:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/1131700963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e40b4bd0 0x7ff4e40b2bf0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/1131700963 >> v2:172.21.15.173:3300/0 conn(0x7ff4e415b580 0x7ff4e415b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 -- 172.21.15.122:0/1131700963 >> 172.21.15.122:0/1131700963 conn(0x7ff4e40b18f0 msgr2=0x7ff4e40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 -- 172.21.15.122:0/1131700963 shutdown_connections 2024-09-17T05:34:18.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 -- 172.21.15.122:0/1131700963 wait complete. 2024-09-17T05:34:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 Processor -- start 2024-09-17T05:34:18.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.959+0000 7ff4e9c00640 1 -- start start 2024-09-17T05:34:18.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e40b4bd0 0x7ff4e41fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e415b580 0x7ff4e40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e415bea0 0x7ff4e406f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4167f70 con 0x7ff4e415bea0 2024-09-17T05:34:18.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4167df0 con 0x7ff4e415b580 2024-09-17T05:34:18.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e41680f0 con 0x7ff4e40b4bd0 2024-09-17T05:34:18.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e415bea0 0x7ff4e406f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e40b4bd0 0x7ff4e41fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e40b4bd0 0x7ff4e41fbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:41942/0 (socket says 172.21.15.122:41942) 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e415bea0 0x7ff4e406f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53470/0 (socket says 172.21.15.122:53470) 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e2a00640 1 -- 172.21.15.122:0/2716209765 learned_addr learned my addr 172.21.15.122:0/2716209765 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4da000640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e415b580 0x7ff4e40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e3400640 1 -- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e40b4bd0 msgr2=0x7ff4e41fbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e3400640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e40b4bd0 0x7ff4e41fbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e3400640 1 -- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e415b580 msgr2=0x7ff4e40b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e3400640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e415b580 0x7ff4e40b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e3400640 1 -- 172.21.15.122:0/2716209765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40af440 con 0x7ff4e415bea0 2024-09-17T05:34:18.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e3400640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e415bea0 0x7ff4e406f200 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c390 tx=0x7ff4dc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:18.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4db400640 1 -- 172.21.15.122:0/2716209765 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc017070 con 0x7ff4e415bea0 2024-09-17T05:34:18.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4db400640 1 -- 172.21.15.122:0/2716209765 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4dc003600 con 0x7ff4e415bea0 2024-09-17T05:34:18.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e40af740 con 0x7ff4e415bea0 2024-09-17T05:34:18.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4db400640 1 -- 172.21.15.122:0/2716209765 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc003a50 con 0x7ff4e415bea0 2024-09-17T05:34:18.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e40b1580 con 0x7ff4e415bea0 2024-09-17T05:34:18.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e4069410 con 0x7ff4e415bea0 2024-09-17T05:34:18.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.963+0000 7ff4db400640 1 -- 172.21.15.122:0/2716209765 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7ff4dc01a070 con 0x7ff4e415bea0 2024-09-17T05:34:18.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.967+0000 7ff4db400640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4c0074370 0x7ff4c0076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:18.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.967+0000 7ff4db400640 1 -- 172.21.15.122:0/2716209765 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff4dc092210 con 0x7ff4e415bea0 2024-09-17T05:34:18.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.967+0000 7ff4e2a00640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4c0074370 0x7ff4c0076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:18.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.967+0000 7ff4e2a00640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4c0074370 0x7ff4c0076830 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4011550 tx=0x7ff4d4009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:18.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:18.971+0000 7ff4db400640 1 -- 172.21.15.122:0/2716209765 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff4e40af740 con 0x7ff4e415bea0 2024-09-17T05:34:19.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff4e40cbc40 con 0x7ff4e415bea0 2024-09-17T05:34:19.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4db400640 1 -- 172.21.15.122:0/2716209765 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7ff4dc05e370 con 0x7ff4e415bea0 2024-09-17T05:34:19.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4c0074370 msgr2=0x7ff4c0076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4c0074370 0x7ff4c0076830 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4011550 tx=0x7ff4d4009500 comp rx=0 tx=0).stop 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e415bea0 msgr2=0x7ff4e406f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e415bea0 0x7ff4e406f200 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c390 tx=0x7ff4dc00c850 comp rx=0 tx=0).stop 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 shutdown_connections 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4c0074370 0x7ff4c0076830 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e415bea0 0x7ff4e406f200 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e415b580 0x7ff4e40b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 --2- 172.21.15.122:0/2716209765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e40b4bd0 0x7ff4e41fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 >> 172.21.15.122:0/2716209765 conn(0x7ff4e40b18f0 msgr2=0x7ff4e4164d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 shutdown_connections 2024-09-17T05:34:19.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.139+0000 7ff4e9c00640 1 -- 172.21.15.122:0/2716209765 wait complete. 2024-09-17T05:34:19.158 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:34:19.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- 172.21.15.122:0/3749993226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00b0210 msgr2=0x7f2de01670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 --2- 172.21.15.122:0/3749993226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00b0210 0x7f2de01670a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009900 tx=0x7f2dd801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- 172.21.15.122:0/3749993226 shutdown_connections 2024-09-17T05:34:19.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 --2- 172.21.15.122:0/3749993226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00b0210 0x7f2de01670a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- 172.21.15.122:0/3749993226 >> 172.21.15.122:0/3749993226 conn(0x7f2de00d1e90 msgr2=0x7f2de00d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- 172.21.15.122:0/3749993226 shutdown_connections 2024-09-17T05:34:19.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- 172.21.15.122:0/3749993226 wait complete. 2024-09-17T05:34:19.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 Processor -- start 2024-09-17T05:34:19.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- start start 2024-09-17T05:34:19.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00b0210 0x7f2de00c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00d00f0 0x7f2de00ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00d29b0 0x7f2de00cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2de016a510 con 0x7f2de00d00f0 2024-09-17T05:34:19.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2de016a390 con 0x7f2de00d29b0 2024-09-17T05:34:19.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2de5c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2de016a690 con 0x7f2de00b0210 2024-09-17T05:34:19.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2dd6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00d00f0 0x7f2de00ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2dd6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00d00f0 0x7f2de00ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53490/0 (socket says 172.21.15.122:53490) 2024-09-17T05:34:19.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2dd6000640 1 -- 172.21.15.122:0/1009164276 learned_addr learned my addr 172.21.15.122:0/1009164276 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:19.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2ddf400640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00d29b0 0x7f2de00cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2ddea00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00b0210 0x7f2de00c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2dd6000640 1 -- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00b0210 msgr2=0x7f2de00c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2dd6000640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00b0210 0x7f2de00c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2dd6000640 1 -- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00d29b0 msgr2=0x7f2de00cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2dd6000640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00d29b0 0x7f2de00cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.307+0000 7f2dd6000640 1 -- 172.21.15.122:0/1009164276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de00b6930 con 0x7f2de00d00f0 2024-09-17T05:34:19.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2dd6000640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00d00f0 0x7f2de00ce870 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c3c0 tx=0x7f2dc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:19.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2dd7400640 1 -- 172.21.15.122:0/1009164276 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8017070 con 0x7f2de00d00f0 2024-09-17T05:34:19.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2dd7400640 1 -- 172.21.15.122:0/1009164276 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dc8003600 con 0x7f2de00d00f0 2024-09-17T05:34:19.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de00b6c10 con 0x7f2de00d00f0 2024-09-17T05:34:19.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2dd7400640 1 -- 172.21.15.122:0/1009164276 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8003a50 con 0x7f2de00d00f0 2024-09-17T05:34:19.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de00c5a60 con 0x7f2de00d00f0 2024-09-17T05:34:19.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de00b66c0 con 0x7f2de00d00f0 2024-09-17T05:34:19.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2dd7400640 1 -- 172.21.15.122:0/1009164276 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f2dc801a070 con 0x7f2de00d00f0 2024-09-17T05:34:19.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2dd7400640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2dbc074370 0x7f2dbc076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.311+0000 7f2dd7400640 1 -- 172.21.15.122:0/1009164276 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f2dc8092260 con 0x7f2de00d00f0 2024-09-17T05:34:19.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.315+0000 7f2ddea00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2dbc074370 0x7f2dbc076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.315+0000 7f2ddea00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2dbc074370 0x7f2dbc076830 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00012f0 tx=0x7f2dd0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:19.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.315+0000 7f2dd7400640 1 -- 172.21.15.122:0/1009164276 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2dc805e3c0 con 0x7f2de00d00f0 2024-09-17T05:34:19.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f2de00c5c30 con 0x7f2de00d00f0 2024-09-17T05:34:19.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2dd7400640 1 -- 172.21.15.122:0/1009164276 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v41) ==== 116+0+2 (secure 0 0 0) 0x7f2dc8062df0 con 0x7f2de00d00f0 2024-09-17T05:34:19.483 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:34:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2dbc074370 msgr2=0x7f2dbc076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2dbc074370 0x7f2dbc076830 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2dd00012f0 tx=0x7f2dd0009040 comp rx=0 tx=0).stop 2024-09-17T05:34:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00d00f0 msgr2=0x7f2de00ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00d00f0 0x7f2de00ce870 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c3c0 tx=0x7f2dc800c880 comp rx=0 tx=0).stop 2024-09-17T05:34:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 shutdown_connections 2024-09-17T05:34:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2dbc074370 0x7f2dbc076830 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00d29b0 0x7f2de00cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2de00d00f0 0x7f2de00ce870 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 --2- 172.21.15.122:0/1009164276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00b0210 0x7f2de00c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 >> 172.21.15.122:0/1009164276 conn(0x7f2de00d1e90 msgr2=0x7f2de0071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 shutdown_connections 2024-09-17T05:34:19.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.487+0000 7f2de5c00640 1 -- 172.21.15.122:0/1009164276 wait complete. 2024-09-17T05:34:19.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:34:19.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- 172.21.15.122:0/2021056959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37d00afb10 msgr2=0x7f37d00affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 --2- 172.21.15.122:0/2021056959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37d00afb10 0x7f37d00affc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009c80 tx=0x7f37c801cda0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- 172.21.15.122:0/2021056959 shutdown_connections 2024-09-17T05:34:19.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 --2- 172.21.15.122:0/2021056959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37d00afb10 0x7f37d00affc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 --2- 172.21.15.122:0/2021056959 >> v2:172.21.15.173:3300/0 conn(0x7f37d01605d0 0x7f37d00d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- 172.21.15.122:0/2021056959 >> 172.21.15.122:0/2021056959 conn(0x7f37d00d1dd0 msgr2=0x7f37d00d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- 172.21.15.122:0/2021056959 shutdown_connections 2024-09-17T05:34:19.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- 172.21.15.122:0/2021056959 wait complete. 2024-09-17T05:34:19.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 Processor -- start 2024-09-17T05:34:19.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- start start 2024-09-17T05:34:19.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37d00afb10 0x7f37d015f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37d00b1650 0x7f37d015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37d01605d0 0x7f37d015cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0169540 con 0x7f37d01605d0 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37d00b1430 con 0x7f37d00afb10 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d8400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f37d01696c0 con 0x7f37d00b1650 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37d01605d0 0x7f37d015cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37d01605d0 0x7f37d015cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53494/0 (socket says 172.21.15.122:53494) 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 -- 172.21.15.122:0/2363881376 learned_addr learned my addr 172.21.15.122:0/2363881376 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d5a00640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37d00afb10 0x7f37d015f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d5000640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37d00b1650 0x7f37d015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 -- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37d00b1650 msgr2=0x7f37d015caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37d00b1650 0x7f37d015caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 -- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37d00afb10 msgr2=0x7f37d015f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37d00afb10 0x7f37d015f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 -- 172.21.15.122:0/2363881376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d015b730 con 0x7f37d01605d0 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.655+0000 7f37d6400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37d01605d0 0x7f37d015cfe0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f37c801b450 tx=0x7f37c80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:19.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37cd600640 1 -- 172.21.15.122:0/2363881376 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c80bb070 con 0x7f37d01605d0 2024-09-17T05:34:19.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37cd600640 1 -- 172.21.15.122:0/2363881376 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37c8004c50 con 0x7f37d01605d0 2024-09-17T05:34:19.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d015ba10 con 0x7f37d01605d0 2024-09-17T05:34:19.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37cd600640 1 -- 172.21.15.122:0/2363881376 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c80b2040 con 0x7f37d01605d0 2024-09-17T05:34:19.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d0154390 con 0x7f37d01605d0 2024-09-17T05:34:19.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d00affc0 con 0x7f37d01605d0 2024-09-17T05:34:19.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37cd600640 1 -- 172.21.15.122:0/2363881376 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f37c80be070 con 0x7f37d01605d0 2024-09-17T05:34:19.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37cd600640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f37b0074370 0x7f37b0076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:19.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.659+0000 7f37cd600640 1 -- 172.21.15.122:0/2363881376 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f37c812cb90 con 0x7f37d01605d0 2024-09-17T05:34:19.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.663+0000 7f37d5a00640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f37b0074370 0x7f37b0076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:19.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.663+0000 7f37d5a00640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f37b0074370 0x7f37b0076830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009b40 tx=0x7f37c40091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:19.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.663+0000 7f37cd600640 1 -- 172.21.15.122:0/2363881376 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f37c8016990 con 0x7f37d01605d0 2024-09-17T05:34:19.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.831+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f37d0154560 con 0x7f37d01605d0 2024-09-17T05:34:19.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.835+0000 7f37cd600640 1 -- 172.21.15.122:0/2363881376 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v42)=0 v42) ==== 131+0+0 (secure 0 0 0) 0x7f37c8118e20 con 0x7f37d01605d0 2024-09-17T05:34:19.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f37b0074370 msgr2=0x7f37b0076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f37b0074370 0x7f37b0076830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009b40 tx=0x7f37c40091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37d01605d0 msgr2=0x7f37d015cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:19.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37d01605d0 0x7f37d015cfe0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f37c801b450 tx=0x7f37c80092e0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 shutdown_connections 2024-09-17T05:34:19.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f37b0074370 0x7f37b0076830 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37d01605d0 0x7f37d015cfe0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37d00b1650 0x7f37d015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 --2- 172.21.15.122:0/2363881376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37d00afb10 0x7f37d015f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:19.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 >> 172.21.15.122:0/2363881376 conn(0x7f37d00d1dd0 msgr2=0x7f37d00cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:19.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 shutdown_connections 2024-09-17T05:34:19.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:19.839+0000 7f37d8400640 1 -- 172.21.15.122:0/2363881376 wait complete. 2024-09-17T05:34:19.853 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:19.855+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:19.853 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:19.855+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:19.853 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:19.855+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:19.853 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:19.855+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:19.853 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:19.855+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:19.856 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-17T05:34:20.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.003+0000 7f03b9200640 1 -- 172.21.15.122:0/496662686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415c130 msgr2=0x7f03b40cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.003+0000 7f03b9200640 1 --2- 172.21.15.122:0/496662686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415c130 0x7f03b40cce80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009900 tx=0x7f03ac01c7c0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.003+0000 7f03b9200640 1 -- 172.21.15.122:0/496662686 shutdown_connections 2024-09-17T05:34:20.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.003+0000 7f03b9200640 1 --2- 172.21.15.122:0/496662686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415c130 0x7f03b40cce80 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.003+0000 7f03b9200640 1 --2- 172.21.15.122:0/496662686 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03b40cecf0 0x7f03b40cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.003+0000 7f03b9200640 1 -- 172.21.15.122:0/496662686 >> 172.21.15.122:0/496662686 conn(0x7f03b40d1ea0 msgr2=0x7f03b40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:20.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.003+0000 7f03b9200640 1 -- 172.21.15.122:0/496662686 shutdown_connections 2024-09-17T05:34:20.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.003+0000 7f03b9200640 1 -- 172.21.15.122:0/496662686 wait complete. 2024-09-17T05:34:20.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 Processor -- start 2024-09-17T05:34:20.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 -- start start 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03b40cecf0 0x7f03b414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415bd50 0x7f03b41fb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03b415c130 0x7f03b41587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41695e0 con 0x7f03b415bd50 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4169460 con 0x7f03b415c130 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f03b4169760 con 0x7f03b40cecf0 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415bd50 0x7f03b41fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415bd50 0x7f03b41fb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53520/0 (socket says 172.21.15.122:53520) 2024-09-17T05:34:20.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 -- 172.21.15.122:0/1166514813 learned_addr learned my addr 172.21.15.122:0/1166514813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b2800640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03b415c130 0x7f03b41587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b1e00640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03b40cecf0 0x7f03b414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 -- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03b40cecf0 msgr2=0x7f03b414df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03b40cecf0 0x7f03b414df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 -- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03b415c130 msgr2=0x7f03b41587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03b415c130 0x7f03b41587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 -- 172.21.15.122:0/1166514813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b414e450 con 0x7f03b415bd50 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03a9400640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415bd50 0x7f03b41fb620 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e9f0 tx=0x7f039c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03aaa00640 1 -- 172.21.15.122:0/1166514813 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c00cd80 con 0x7f03b415bd50 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03aaa00640 1 -- 172.21.15.122:0/1166514813 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f039c010430 con 0x7f03b415bd50 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b41fbb60 con 0x7f03b415bd50 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03aaa00640 1 -- 172.21.15.122:0/1166514813 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c010860 con 0x7f03b415bd50 2024-09-17T05:34:20.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b41fc3c0 con 0x7f03b415bd50 2024-09-17T05:34:20.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.007+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b40ce5e0 con 0x7f03b415bd50 2024-09-17T05:34:20.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.011+0000 7f03aaa00640 1 -- 172.21.15.122:0/1166514813 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7f039c010a00 con 0x7f03b415bd50 2024-09-17T05:34:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.011+0000 7f03aaa00640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0390074370 0x7f0390076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.011+0000 7f03aaa00640 1 -- 172.21.15.122:0/1166514813 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f039c093c00 con 0x7f03b415bd50 2024-09-17T05:34:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.011+0000 7f03b1e00640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0390074370 0x7f0390076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.011+0000 7f03b1e00640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0390074370 0x7f0390076830 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f03a4001ee0 tx=0x7f03a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:20.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.015+0000 7f03aaa00640 1 -- 172.21.15.122:0/1166514813 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f039c05fe60 con 0x7f03b415bd50 2024-09-17T05:34:20.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.183+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f03b4158d20 con 0x7f03b415bd50 2024-09-17T05:34:20.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.187+0000 7f03aaa00640 1 -- 172.21.15.122:0/1166514813 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v43)=0 v43) ==== 136+0+0 (secure 0 0 0) 0x7f039c064890 con 0x7f03b415bd50 2024-09-17T05:34:20.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0390074370 msgr2=0x7f0390076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0390074370 0x7f0390076830 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f03a4001ee0 tx=0x7f03a4009040 comp rx=0 tx=0).stop 2024-09-17T05:34:20.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415bd50 msgr2=0x7f03b41fb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415bd50 0x7f03b41fb620 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e9f0 tx=0x7f039c00eeb0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 shutdown_connections 2024-09-17T05:34:20.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0390074370 0x7f0390076830 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03b415c130 0x7f03b41587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b415bd50 0x7f03b41fb620 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 --2- 172.21.15.122:0/1166514813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03b40cecf0 0x7f03b414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 >> 172.21.15.122:0/1166514813 conn(0x7f03b40d1ea0 msgr2=0x7f03b40d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 shutdown_connections 2024-09-17T05:34:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.191+0000 7f03b9200640 1 -- 172.21.15.122:0/1166514813 wait complete. 2024-09-17T05:34:20.207 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T05:34:20.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.355+0000 7fd843e00640 1 -- 172.21.15.122:0/828641637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8440d09f0 msgr2=0x7fd8440d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.355+0000 7fd843e00640 1 --2- 172.21.15.122:0/828641637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8440d09f0 0x7fd8440d28c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd830009840 tx=0x7fd83001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.355+0000 7fd843e00640 1 -- 172.21.15.122:0/828641637 shutdown_connections 2024-09-17T05:34:20.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.355+0000 7fd843e00640 1 --2- 172.21.15.122:0/828641637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8440d09f0 0x7fd8440d28c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.355+0000 7fd843e00640 1 -- 172.21.15.122:0/828641637 >> 172.21.15.122:0/828641637 conn(0x7fd8440d1da0 msgr2=0x7fd8440d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.355+0000 7fd843e00640 1 -- 172.21.15.122:0/828641637 shutdown_connections 2024-09-17T05:34:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.355+0000 7fd843e00640 1 -- 172.21.15.122:0/828641637 wait complete. 2024-09-17T05:34:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.355+0000 7fd843e00640 1 Processor -- start 2024-09-17T05:34:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 -- start start 2024-09-17T05:34:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8440cff00 0x7fd8440b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd84415eff0 0x7fd844153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd844151220 0x7fd8440cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd844169ed0 con 0x7fd84415eff0 2024-09-17T05:34:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd844169d50 con 0x7fd8440cff00 2024-09-17T05:34:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd84416a050 con 0x7fd844151220 2024-09-17T05:34:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd84415eff0 0x7fd844153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd84415eff0 0x7fd844153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53536/0 (socket says 172.21.15.122:53536) 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 -- 172.21.15.122:0/1188742903 learned_addr learned my addr 172.21.15.122:0/1188742903 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843400640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd844151220 0x7fd8440cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842a00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8440cff00 0x7fd8440b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 -- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd844151220 msgr2=0x7fd8440cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd844151220 0x7fd8440cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 -- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8440cff00 msgr2=0x7fd8440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8440cff00 0x7fd8440b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 -- 172.21.15.122:0/1188742903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd844153a60 con 0x7fd84415eff0 2024-09-17T05:34:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842a00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8440cff00 0x7fd8440b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:34:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd842000640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd84415eff0 0x7fd844153520 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd830009790 tx=0x7fd830004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd83aa00640 1 -- 172.21.15.122:0/1188742903 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300b2040 con 0x7fd84415eff0 2024-09-17T05:34:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd83aa00640 1 -- 172.21.15.122:0/1188742903 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd830003600 con 0x7fd84415eff0 2024-09-17T05:34:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84415a680 con 0x7fd84415eff0 2024-09-17T05:34:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd83aa00640 1 -- 172.21.15.122:0/1188742903 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830003a30 con 0x7fd84415eff0 2024-09-17T05:34:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84415ac40 con 0x7fd84415eff0 2024-09-17T05:34:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.359+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8440d28c0 con 0x7fd84415eff0 2024-09-17T05:34:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.363+0000 7fd83aa00640 1 -- 172.21.15.122:0/1188742903 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7fd8300bd060 con 0x7fd84415eff0 2024-09-17T05:34:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.363+0000 7fd83aa00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd824074370 0x7fd824076830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.363+0000 7fd83aa00640 1 -- 172.21.15.122:0/1188742903 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fd83012ca70 con 0x7fd84415eff0 2024-09-17T05:34:20.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.363+0000 7fd842a00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd824074370 0x7fd824076830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.363+0000 7fd842a00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd824074370 0x7fd824076830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd834007fd0 tx=0x7fd834009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:20.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.367+0000 7fd83aa00640 1 -- 172.21.15.122:0/1188742903 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd830016410 con 0x7fd84415eff0 2024-09-17T05:34:20.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.559+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fd8441669c0 con 0x7fd824074370 2024-09-17T05:34:20.565 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:20.567+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:20.566 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:20.567+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:20.566 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:20.567+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:20.566 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:20.567+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:20.566 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:20.567+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:20.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.603+0000 7fd83aa00640 1 -- 172.21.15.122:0/1188742903 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8441669c0 con 0x7fd824074370 2024-09-17T05:34:20.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd824074370 msgr2=0x7fd824076830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd824074370 0x7fd824076830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd834007fd0 tx=0x7fd834009040 comp rx=0 tx=0).stop 2024-09-17T05:34:20.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd84415eff0 msgr2=0x7fd844153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd84415eff0 0x7fd844153520 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd830009790 tx=0x7fd830004040 comp rx=0 tx=0).stop 2024-09-17T05:34:20.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 shutdown_connections 2024-09-17T05:34:20.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd824074370 0x7fd824076830 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd844151220 0x7fd8440cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd84415eff0 0x7fd844153520 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 --2- 172.21.15.122:0/1188742903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8440cff00 0x7fd8440b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 >> 172.21.15.122:0/1188742903 conn(0x7fd8440d1da0 msgr2=0x7fd844154070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:20.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 shutdown_connections 2024-09-17T05:34:20.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.607+0000 7fd843e00640 1 -- 172.21.15.122:0/1188742903 wait complete. 2024-09-17T05:34:20.623 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:34:20.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.771+0000 7ff591800640 1 -- 172.21.15.122:0/1031735747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5700050e0 msgr2=0x7ff570005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.771+0000 7ff591800640 1 --2- 172.21.15.122:0/1031735747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5700050e0 0x7ff570005920 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff578009b30 tx=0x7ff57801d190 comp rx=0 tx=0).stop 2024-09-17T05:34:20.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.771+0000 7ff58b400640 1 -- 172.21.15.122:0/1031735747 reap_dead start 2024-09-17T05:34:20.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.771+0000 7ff591800640 1 -- 172.21.15.122:0/1031735747 shutdown_connections 2024-09-17T05:34:20.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.771+0000 7ff591800640 1 -- 172.21.15.122:0/1031735747 >> 172.21.15.122:0/1031735747 conn(0x7ff58c0b18f0 msgr2=0x7ff58c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:20.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.771+0000 7ff591800640 1 -- 172.21.15.122:0/1031735747 shutdown_connections 2024-09-17T05:34:20.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.771+0000 7ff591800640 1 -- 172.21.15.122:0/1031735747 wait complete. 2024-09-17T05:34:20.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 Processor -- start 2024-09-17T05:34:20.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 -- start start 2024-09-17T05:34:20.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5700050e0 0x7ff58c14f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c09ac20 0x7ff58c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff58c09b080 0x7ff58c0ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c0cc660 con 0x7ff58c09ac20 2024-09-17T05:34:20.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c06eaf0 con 0x7ff5700050e0 2024-09-17T05:34:20.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c0cc7e0 con 0x7ff58c09b080 2024-09-17T05:34:20.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c09ac20 0x7ff58c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c09ac20 0x7ff58c0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53566/0 (socket says 172.21.15.122:53566) 2024-09-17T05:34:20.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 -- 172.21.15.122:0/4195310702 learned_addr learned my addr 172.21.15.122:0/4195310702 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58aa00640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5700050e0 0x7ff58c14f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58b400640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff58c09b080 0x7ff58c0ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 -- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff58c09b080 msgr2=0x7ff58c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff58c09b080 0x7ff58c0ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 -- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5700050e0 msgr2=0x7ff58c14f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5700050e0 0x7ff58c14f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 -- 172.21.15.122:0/4195310702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c09d530 con 0x7ff58c09ac20 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff58a000640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c09ac20 0x7ff58c0ce050 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff5780af040 tx=0x7ff5780acb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff582a00640 1 -- 172.21.15.122:0/4195310702 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780bd020 con 0x7ff58c09ac20 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff582a00640 1 -- 172.21.15.122:0/4195310702 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5780acc70 con 0x7ff58c09ac20 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c09d7d0 con 0x7ff58c09ac20 2024-09-17T05:34:20.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff582a00640 1 -- 172.21.15.122:0/4195310702 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780b1040 con 0x7ff58c09ac20 2024-09-17T05:34:20.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.775+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c09dd90 con 0x7ff58c09ac20 2024-09-17T05:34:20.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.779+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c069410 con 0x7ff58c09ac20 2024-09-17T05:34:20.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.779+0000 7ff582a00640 1 -- 172.21.15.122:0/4195310702 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 32) ==== 98815+0+0 (secure 0 0 0) 0x7ff5780c0070 con 0x7ff58c09ac20 2024-09-17T05:34:20.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.779+0000 7ff582a00640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff56c0743a0 0x7ff56c076860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:20.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.779+0000 7ff582a00640 1 -- 172.21.15.122:0/4195310702 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff57812da20 con 0x7ff58c09ac20 2024-09-17T05:34:20.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.779+0000 7ff58aa00640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff56c0743a0 0x7ff56c076860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:20.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.779+0000 7ff58aa00640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff56c0743a0 0x7ff56c076860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff57c001410 tx=0x7ff57c006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:20.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.783+0000 7ff582a00640 1 -- 172.21.15.122:0/4195310702 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff578016ea0 con 0x7ff58c09ac20 2024-09-17T05:34:20.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.975+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff58c0cccd0 con 0x7ff56c0743a0 2024-09-17T05:34:20.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff582a00640 1 -- 172.21.15.122:0/4195310702 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff58c0cccd0 con 0x7ff56c0743a0 2024-09-17T05:34:20.979 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb 2024-09-17T05:34:20.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff56c0743a0 msgr2=0x7ff56c076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff56c0743a0 0x7ff56c076860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff57c001410 tx=0x7ff57c006040 comp rx=0 tx=0).stop 2024-09-17T05:34:20.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c09ac20 msgr2=0x7ff58c0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c09ac20 0x7ff58c0ce050 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff5780af040 tx=0x7ff5780acb00 comp rx=0 tx=0).stop 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 shutdown_connections 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff56c0743a0 0x7ff56c076860 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff58c09b080 0x7ff58c0ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff58c09ac20 0x7ff58c0ce050 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 --2- 172.21.15.122:0/4195310702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5700050e0 0x7ff58c14f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 >> 172.21.15.122:0/4195310702 conn(0x7ff58c0b18f0 msgr2=0x7ff58c158190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 shutdown_connections 2024-09-17T05:34:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:20.983+0000 7ff591800640 1 -- 172.21.15.122:0/4195310702 wait complete. 2024-09-17T05:34:21.001 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb 2024-09-17T05:34:21.001 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:34:21.002 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:21.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:21.011 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:34:29.956 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:34:29.957 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:34:29.957 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.88426 s, 121 MB/s 2024-09-17T05:34:29.958 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:34:29.958 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:29.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:29.967 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:34:33.035 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:34:33.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:34:39.109 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:34:39.110 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:34:39.110 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.09136 s, 118 MB/s 2024-09-17T05:34:39.111 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:34:39.111 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:34:39.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:39.121 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:34:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:34:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:34:48.296 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.12494 s, 118 MB/s 2024-09-17T05:34:48.297 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T05:34:48.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 -- 172.21.15.122:0/2327048204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280734f0 msgr2=0x7f5c2815a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 --2- 172.21.15.122:0/2327048204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280734f0 0x7f5c2815a9d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800aa10 tx=0x7f5c1801dc30 comp rx=0 tx=0).stop 2024-09-17T05:34:48.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 -- 172.21.15.122:0/2327048204 shutdown_connections 2024-09-17T05:34:48.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 --2- 172.21.15.122:0/2327048204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c280cf440 0x7f5c280ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 --2- 172.21.15.122:0/2327048204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280734f0 0x7f5c2815a9d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 -- 172.21.15.122:0/2327048204 >> 172.21.15.122:0/2327048204 conn(0x7f5c280cfbc0 msgr2=0x7f5c280cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:48.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 -- 172.21.15.122:0/2327048204 shutdown_connections 2024-09-17T05:34:48.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 -- 172.21.15.122:0/2327048204 wait complete. 2024-09-17T05:34:48.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 Processor -- start 2024-09-17T05:34:48.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.447+0000 7f5c2e600640 1 -- start start 2024-09-17T05:34:48.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c280734f0 0x7f5c281f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280ce890 0x7f5c281fb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c280cf440 0x7f5c281f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28167360 con 0x7f5c280ce890 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2815edb0 con 0x7f5c280734f0 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5c281674e0 con 0x7f5c280cf440 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280ce890 0x7f5c281fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280ce890 0x7f5c281fb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47504/0 (socket says 172.21.15.122:47504) 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 -- 172.21.15.122:0/1840625965 learned_addr learned my addr 172.21.15.122:0/1840625965 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c27400640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c280734f0 0x7f5c281f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c27e00640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c280cf440 0x7f5c281f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 -- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c280cf440 msgr2=0x7f5c281f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c280cf440 0x7f5c281f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 -- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c280734f0 msgr2=0x7f5c281f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c280734f0 0x7f5c281f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 -- 172.21.15.122:0/1840625965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c281f8be0 con 0x7f5c280ce890 2024-09-17T05:34:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c26a00640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280ce890 0x7f5c281fb030 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400e7a0 tx=0x7f5c1400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c1f400640 1 -- 172.21.15.122:0/1840625965 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400ccc0 con 0x7f5c280ce890 2024-09-17T05:34:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c1f400640 1 -- 172.21.15.122:0/1840625965 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c14010430 con 0x7f5c280ce890 2024-09-17T05:34:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c1f400640 1 -- 172.21.15.122:0/1840625965 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14010860 con 0x7f5c280ce890 2024-09-17T05:34:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c281f8ec0 con 0x7f5c280ce890 2024-09-17T05:34:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c281fb570 con 0x7f5c280ce890 2024-09-17T05:34:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c28069410 con 0x7f5c280ce890 2024-09-17T05:34:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.451+0000 7f5c1f400640 1 -- 172.21.15.122:0/1840625965 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 33) ==== 98917+0+0 (secure 0 0 0) 0x7f5c14010a00 con 0x7f5c280ce890 2024-09-17T05:34:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.455+0000 7f5c1f400640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c08074490 0x7f5c08076950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.455+0000 7f5c1f400640 1 -- 172.21.15.122:0/1840625965 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f5c14093710 con 0x7f5c280ce890 2024-09-17T05:34:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.455+0000 7f5c27400640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c08074490 0x7f5c08076950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.455+0000 7f5c27400640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c08074490 0x7f5c08076950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5c180b0000 tx=0x7f5c1801c470 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.459+0000 7f5c1f400640 1 -- 172.21.15.122:0/1840625965 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5c1405fc00 con 0x7f5c280ce890 2024-09-17T05:34:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.655+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f5c280d0850 con 0x7f5c08074490 2024-09-17T05:34:48.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.667+0000 7f5c1f400640 1 -- 172.21.15.122:0/1840625965 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5c280d0850 con 0x7f5c08074490 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c08074490 msgr2=0x7f5c08076950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c08074490 0x7f5c08076950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5c180b0000 tx=0x7f5c1801c470 comp rx=0 tx=0).stop 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280ce890 msgr2=0x7f5c281fb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280ce890 0x7f5c281fb030 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400e7a0 tx=0x7f5c1400ec60 comp rx=0 tx=0).stop 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 shutdown_connections 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c08074490 0x7f5c08076950 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c280cf440 0x7f5c281f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c280ce890 0x7f5c281fb030 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 --2- 172.21.15.122:0/1840625965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c280734f0 0x7f5c281f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 >> 172.21.15.122:0/1840625965 conn(0x7f5c280cfbc0 msgr2=0x7f5c280ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 shutdown_connections 2024-09-17T05:34:48.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.671+0000 7f5c2e600640 1 -- 172.21.15.122:0/1840625965 wait complete. 2024-09-17T05:34:48.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:34:48.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.835+0000 7f8977200640 1 -- 172.21.15.122:0/2361296497 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f897815ba60 msgr2=0x7f89781602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.835+0000 7f8977200640 1 --2- 172.21.15.122:0/2361296497 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f897815ba60 0x7f89781602b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8968009870 tx=0x7f896801c7d0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.835+0000 7f8977200640 1 -- 172.21.15.122:0/2361296497 shutdown_connections 2024-09-17T05:34:48.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.835+0000 7f8977200640 1 --2- 172.21.15.122:0/2361296497 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f897815ba60 0x7f89781602b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.835+0000 7f8977200640 1 -- 172.21.15.122:0/2361296497 >> 172.21.15.122:0/2361296497 conn(0x7f897809aa10 msgr2=0x7f897809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:48.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.835+0000 7f8977200640 1 -- 172.21.15.122:0/2361296497 shutdown_connections 2024-09-17T05:34:48.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.835+0000 7f8977200640 1 -- 172.21.15.122:0/2361296497 wait complete. 2024-09-17T05:34:48.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 Processor -- start 2024-09-17T05:34:48.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 -- start start 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89780d0190 0x7f897814ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89780d05f0 0x7f8978154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f897815ba60 0x7f8978154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f897816a770 con 0x7f89780d0190 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f897816a5f0 con 0x7f897815ba60 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f897816a8f0 con 0x7f89780d05f0 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89780d0190 0x7f897814ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89780d0190 0x7f897814ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47508/0 (socket says 172.21.15.122:47508) 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 -- 172.21.15.122:0/2462889605 learned_addr learned my addr 172.21.15.122:0/2462889605 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:48.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8976800640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f897815ba60 0x7f8978154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f896fe00640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89780d05f0 0x7f8978154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 -- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89780d05f0 msgr2=0x7f8978154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89780d05f0 0x7f8978154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 -- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f897815ba60 msgr2=0x7f8978154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:48.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f897815ba60 0x7f8978154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:48.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 -- 172.21.15.122:0/2462889605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89780afdd0 con 0x7f89780d0190 2024-09-17T05:34:48.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8975e00640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89780d0190 0x7f897814ebc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f896801ccb0 tx=0x7f8968004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:48.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f896e000640 1 -- 172.21.15.122:0/2462889605 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89680be070 con 0x7f89780d0190 2024-09-17T05:34:48.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f896e000640 1 -- 172.21.15.122:0/2462889605 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8968005040 con 0x7f89780d0190 2024-09-17T05:34:48.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f896e000640 1 -- 172.21.15.122:0/2462889605 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968005450 con 0x7f89780d0190 2024-09-17T05:34:48.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89780ca850 con 0x7f89780d0190 2024-09-17T05:34:48.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89780cae10 con 0x7f89780d0190 2024-09-17T05:34:48.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.839+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89781602b0 con 0x7f89780d0190 2024-09-17T05:34:48.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.843+0000 7f896e000640 1 -- 172.21.15.122:0/2462889605 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 33) ==== 98917+0+0 (secure 0 0 0) 0x7f89680a66e0 con 0x7f89780d0190 2024-09-17T05:34:48.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.843+0000 7f896e000640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8954074490 0x7f8954076950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:48.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.843+0000 7f896e000640 1 -- 172.21.15.122:0/2462889605 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f896812cf50 con 0x7f89780d0190 2024-09-17T05:34:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.843+0000 7f896fe00640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8954074490 0x7f8954076950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:48.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.843+0000 7f896fe00640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8954074490 0x7f8954076950 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f89781f94a0 tx=0x7f8960009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:48.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:48.847+0000 7f896e000640 1 -- 172.21.15.122:0/2462889605 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f89680ac030 con 0x7f89780d0190 2024-09-17T05:34:49.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.039+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f89780d2a60 con 0x7f8954074490 2024-09-17T05:34:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f896e000640 1 -- 172.21.15.122:0/2462889605 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f89780d2a60 con 0x7f8954074490 2024-09-17T05:34:49.039 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb 2024-09-17T05:34:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8954074490 msgr2=0x7f8954076950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8954074490 0x7f8954076950 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f89781f94a0 tx=0x7f8960009040 comp rx=0 tx=0).stop 2024-09-17T05:34:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89780d0190 msgr2=0x7f897814ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89780d0190 0x7f897814ebc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f896801ccb0 tx=0x7f8968004610 comp rx=0 tx=0).stop 2024-09-17T05:34:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 shutdown_connections 2024-09-17T05:34:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8954074490 0x7f8954076950 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f897815ba60 0x7f8978154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89780d05f0 0x7f8978154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 --2- 172.21.15.122:0/2462889605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89780d0190 0x7f897814ebc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 >> 172.21.15.122:0/2462889605 conn(0x7f897809aa10 msgr2=0x7f8978071320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 shutdown_connections 2024-09-17T05:34:49.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:49.043+0000 7f8977200640 1 -- 172.21.15.122:0/2462889605 wait complete. 2024-09-17T05:34:49.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:49.060 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb 2024-09-17T05:34:49.060 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:34:49.069 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb 2024-09-17T05:34:49.069 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="2972172288" 2024-09-17T05:34:49.069 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:49.070 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-17T05:34:51.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:34:51.071 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb 2024-09-17T05:34:51.071 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:34:51.081 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/8c2bd233-e6c4-45a1-a8a9-206b4ef3ebfb 2024-09-17T05:34:51.081 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T05:34:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:51.082 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T05:34:51.082 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone84910 2024-09-17T05:34:51.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- 172.21.15.122:0/1064937058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680d0840 msgr2=0x7ff5680b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:51.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 --2- 172.21.15.122:0/1064937058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680d0840 0x7ff5680b1bd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff56806d870 tx=0x7ff56001f580 comp rx=0 tx=0).stop 2024-09-17T05:34:51.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- 172.21.15.122:0/1064937058 shutdown_connections 2024-09-17T05:34:51.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 --2- 172.21.15.122:0/1064937058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680d0840 0x7ff5680b1bd0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 --2- 172.21.15.122:0/1064937058 >> v2:172.21.15.173:3300/0 conn(0x7ff5680cff20 0x7ff5680d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- 172.21.15.122:0/1064937058 >> 172.21.15.122:0/1064937058 conn(0x7ff5680d1dc0 msgr2=0x7ff5680d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:51.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- 172.21.15.122:0/1064937058 shutdown_connections 2024-09-17T05:34:51.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- 172.21.15.122:0/1064937058 wait complete. 2024-09-17T05:34:51.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 Processor -- start 2024-09-17T05:34:51.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- start start 2024-09-17T05:34:51.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5680b2d80 0x7ff5681f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680cff20 0x7ff56815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5680d0840 0x7ff56815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5680af800 con 0x7ff5680cff20 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff5680af680 con 0x7ff5680b2d80 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.283+0000 7ff56de00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff5680af980 con 0x7ff5680d0840 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680cff20 0x7ff56815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680cff20 0x7ff56815c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51910/0 (socket says 172.21.15.122:51910) 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 -- 172.21.15.122:0/2157472890 learned_addr learned my addr 172.21.15.122:0/2157472890 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff567400640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5680d0840 0x7ff56815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff566a00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5680b2d80 0x7ff5681f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 -- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5680d0840 msgr2=0x7ff56815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:51.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5680d0840 0x7ff56815cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 -- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5680b2d80 msgr2=0x7ff5681f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:51.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5680b2d80 0x7ff5681f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 -- 172.21.15.122:0/2157472890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5681f7000 con 0x7ff5680cff20 2024-09-17T05:34:51.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55e000640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680cff20 0x7ff56815c810 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c480 tx=0x7ff55000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:51.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55fe00640 1 -- 172.21.15.122:0/2157472890 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550012830 con 0x7ff5680cff20 2024-09-17T05:34:51.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55fe00640 1 -- 172.21.15.122:0/2157472890 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff550017070 con 0x7ff5680cff20 2024-09-17T05:34:51.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5680cd400 con 0x7ff5680cff20 2024-09-17T05:34:51.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55fe00640 1 -- 172.21.15.122:0/2157472890 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550012d80 con 0x7ff5680cff20 2024-09-17T05:34:51.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5681f6d20 con 0x7ff5680cff20 2024-09-17T05:34:51.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5680cd5d0 con 0x7ff5680cff20 2024-09-17T05:34:51.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.287+0000 7ff55fe00640 1 -- 172.21.15.122:0/2157472890 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 33) ==== 98917+0+0 (secure 0 0 0) 0x7ff55001c460 con 0x7ff5680cff20 2024-09-17T05:34:51.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.291+0000 7ff55fe00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff544074490 0x7ff544076950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:51.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.291+0000 7ff55fe00640 1 -- 172.21.15.122:0/2157472890 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff550092e90 con 0x7ff5680cff20 2024-09-17T05:34:51.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.291+0000 7ff566a00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff544074490 0x7ff544076950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:51.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.291+0000 7ff566a00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff544074490 0x7ff544076950 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff558004630 tx=0x7ff55800b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:51.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.295+0000 7ff55fe00640 1 -- 172.21.15.122:0/2157472890 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff55005f020 con 0x7ff5680cff20 2024-09-17T05:34:51.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.491+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone84910", "target": ["mon-mgr", ""]}) -- 0x7ff568070ce0 con 0x7ff544074490 2024-09-17T05:34:51.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.563+0000 7ff55fe00640 1 -- 172.21.15.122:0/2157472890 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff568070ce0 con 0x7ff544074490 2024-09-17T05:34:51.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.563+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff544074490 msgr2=0x7ff544076950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.563+0000 7ff56de00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff544074490 0x7ff544076950 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff558004630 tx=0x7ff55800b390 comp rx=0 tx=0).stop 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.563+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680cff20 msgr2=0x7ff56815c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.563+0000 7ff56de00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680cff20 0x7ff56815c810 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c480 tx=0x7ff55000c940 comp rx=0 tx=0).stop 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.567+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 shutdown_connections 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.567+0000 7ff56de00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff544074490 0x7ff544076950 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.567+0000 7ff56de00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5680d0840 0x7ff56815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.567+0000 7ff56de00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5680cff20 0x7ff56815c810 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.567+0000 7ff56de00640 1 --2- 172.21.15.122:0/2157472890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5680b2d80 0x7ff5681f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.567+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 >> 172.21.15.122:0/2157472890 conn(0x7ff5680d1dc0 msgr2=0x7ff568156600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.567+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 shutdown_connections 2024-09-17T05:34:51.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.567+0000 7ff56de00640 1 -- 172.21.15.122:0/2157472890 wait complete. 2024-09-17T05:34:51.564 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.571+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.564 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.571+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.564 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.571+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.565 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.571+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.565 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.571+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.581 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.587+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.581 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.587+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.581 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.587+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.581 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.587+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.581 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:51.587+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:51.583 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92681 2024-09-17T05:34:51.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- 172.21.15.122:0/3889019171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b0415bab0 msgr2=0x7f8b0415cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:51.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3889019171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b0415bab0 0x7f8b0415cbc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009840 tx=0x7f8af401c820 comp rx=0 tx=0).stop 2024-09-17T05:34:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- 172.21.15.122:0/3889019171 shutdown_connections 2024-09-17T05:34:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3889019171 >> v2:172.21.15.173:3300/0 conn(0x7f8b040689d0 0x7f8b040b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3889019171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b0415bab0 0x7f8b0415cbc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- 172.21.15.122:0/3889019171 >> 172.21.15.122:0/3889019171 conn(0x7f8b040b57c0 msgr2=0x7f8b040b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- 172.21.15.122:0/3889019171 shutdown_connections 2024-09-17T05:34:51.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- 172.21.15.122:0/3889019171 wait complete. 2024-09-17T05:34:51.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 Processor -- start 2024-09-17T05:34:51.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- start start 2024-09-17T05:34:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b040689d0 0x7f8b041662b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b040ae670 0x7f8b040d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b0415bab0 0x7f8b04158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b04168f20 con 0x7f8b040ae670 2024-09-17T05:34:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8b04168da0 con 0x7f8b0415bab0 2024-09-17T05:34:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b08c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8b041690a0 con 0x7f8b040689d0 2024-09-17T05:34:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b02a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b040ae670 0x7f8b040d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:51.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b02a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b040ae670 0x7f8b040d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51912/0 (socket says 172.21.15.122:51912) 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b02a00640 1 -- 172.21.15.122:0/3562357879 learned_addr learned my addr 172.21.15.122:0/3562357879 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b03400640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b040689d0 0x7f8b041662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.739+0000 7f8b03e00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b0415bab0 0x7f8b04158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b02a00640 1 -- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b040689d0 msgr2=0x7f8b041662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b02a00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b040689d0 0x7f8b041662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b02a00640 1 -- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b0415bab0 msgr2=0x7f8b04158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b02a00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b0415bab0 0x7f8b04158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b02a00640 1 -- 172.21.15.122:0/3562357879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b040b18c0 con 0x7f8b040ae670 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b02a00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b040ae670 0x7f8b040d0bb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8af000e9c0 tx=0x7f8af000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8afb400640 1 -- 172.21.15.122:0/3562357879 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af000cd80 con 0x7f8b040ae670 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8afb400640 1 -- 172.21.15.122:0/3562357879 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8af0004020 con 0x7f8b040ae670 2024-09-17T05:34:51.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8afb400640 1 -- 172.21.15.122:0/3562357879 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0004450 con 0x7f8b040ae670 2024-09-17T05:34:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0415a930 con 0x7f8b040ae670 2024-09-17T05:34:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b04149b90 con 0x7f8b040ae670 2024-09-17T05:34:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b040b1fa0 con 0x7f8b040ae670 2024-09-17T05:34:51.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.743+0000 7f8afb400640 1 -- 172.21.15.122:0/3562357879 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 33) ==== 98917+0+0 (secure 0 0 0) 0x7f8af0003600 con 0x7f8b040ae670 2024-09-17T05:34:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.747+0000 7f8afb400640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ae4074490 0x7f8ae4076950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:51.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.747+0000 7f8afb400640 1 -- 172.21.15.122:0/3562357879 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f8af0093760 con 0x7f8b040ae670 2024-09-17T05:34:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.747+0000 7f8b03400640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ae4074490 0x7f8ae4076950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:51.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.747+0000 7f8b03400640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ae4074490 0x7f8ae4076950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8af40af000 tx=0x7f8af401b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:51.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.751+0000 7f8afb400640 1 -- 172.21.15.122:0/3562357879 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8af005fc10 con 0x7f8b040ae670 2024-09-17T05:34:51.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:51.947+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92681", "target": ["mon-mgr", ""]}) -- 0x7f8b0414c960 con 0x7f8ae4074490 2024-09-17T05:34:52.565 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:52.571+0000 7fc05a400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:52.565 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:52.571+0000 7fc05a400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:52.565 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:52.571+0000 7fc05a400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:52.565 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:52.571+0000 7fc05a400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:52.565 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:52.571+0000 7fc05a400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:52.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.675+0000 7f8afb400640 1 -- 172.21.15.122:0/3562357879 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8b0414c960 con 0x7f8ae4074490 2024-09-17T05:34:52.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ae4074490 msgr2=0x7f8ae4076950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ae4074490 0x7f8ae4076950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8af40af000 tx=0x7f8af401b060 comp rx=0 tx=0).stop 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b040ae670 msgr2=0x7f8b040d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b040ae670 0x7f8b040d0bb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8af000e9c0 tx=0x7f8af000ee80 comp rx=0 tx=0).stop 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 shutdown_connections 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ae4074490 0x7f8ae4076950 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8af40af000 tx=0x7f8af401b060 comp rx=0 tx=0).stop 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b0415bab0 0x7f8b04158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b040ae670 0x7f8b040d0bb0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 --2- 172.21.15.122:0/3562357879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b040689d0 0x7f8b041662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:52.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 >> 172.21.15.122:0/3562357879 conn(0x7f8b040b57c0 msgr2=0x7f8b040af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:52.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 shutdown_connections 2024-09-17T05:34:52.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.679+0000 7f8b08c00640 1 -- 172.21.15.122:0/3562357879 wait complete. 2024-09-17T05:34:52.714 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone67922 2024-09-17T05:34:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.967+0000 7fca86000640 1 -- 172.21.15.122:0/2194761633 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800ad440 con 0x7fca880cd5d0 2024-09-17T05:34:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.967+0000 7fca8d000640 1 -- 172.21.15.122:0/2194761633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca880cd5d0 msgr2=0x7fca880cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.967+0000 7fca8d000640 1 --2- 172.21.15.122:0/2194761633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca880cd5d0 0x7fca880cecd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fca88076400 tx=0x7fca8001cb80 comp rx=0 tx=0).stop 2024-09-17T05:34:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.971+0000 7fca8d000640 1 -- 172.21.15.122:0/2194761633 shutdown_connections 2024-09-17T05:34:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.971+0000 7fca8d000640 1 --2- 172.21.15.122:0/2194761633 >> v2:172.21.15.122:3300/0 conn(0x7fca880cd9d0 0x7fca880cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.971+0000 7fca8d000640 1 --2- 172.21.15.122:0/2194761633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca880cd5d0 0x7fca880cecd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.971+0000 7fca8d000640 1 -- 172.21.15.122:0/2194761633 >> 172.21.15.122:0/2194761633 conn(0x7fca880cc4a0 msgr2=0x7fca880cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.971+0000 7fca8d000640 1 -- 172.21.15.122:0/2194761633 shutdown_connections 2024-09-17T05:34:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.971+0000 7fca8d000640 1 -- 172.21.15.122:0/2194761633 wait complete. 2024-09-17T05:34:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 Processor -- start 2024-09-17T05:34:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 -- start start 2024-09-17T05:34:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca880cd5d0 0x7fca881524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca880cd9d0 0x7fca880b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca880ce4a0 0x7fca880b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca8816acc0 con 0x7fca880cd5d0 2024-09-17T05:34:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fca8816ab40 con 0x7fca880cd9d0 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fca8816ae40 con 0x7fca880ce4a0 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca880cd5d0 0x7fca881524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca880cd5d0 0x7fca881524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51930/0 (socket says 172.21.15.122:51930) 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 -- 172.21.15.122:0/369430785 learned_addr learned my addr 172.21.15.122:0/369430785 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 -- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca880ce4a0 msgr2=0x7fca880b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca880ce4a0 0x7fca880b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 -- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca880cd9d0 msgr2=0x7fca880b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca880cd9d0 0x7fca880b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 -- 172.21.15.122:0/369430785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8815c5b0 con 0x7fca880cd5d0 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca87400640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca880cd5d0 0x7fca881524f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fca7400c430 tx=0x7fca7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca7f400640 1 -- 172.21.15.122:0/369430785 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74011790 con 0x7fca880cd5d0 2024-09-17T05:34:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca7f400640 1 -- 172.21.15.122:0/369430785 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca74017070 con 0x7fca880cd5d0 2024-09-17T05:34:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8815c890 con 0x7fca880cd5d0 2024-09-17T05:34:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca7f400640 1 -- 172.21.15.122:0/369430785 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74011ce0 con 0x7fca880cd5d0 2024-09-17T05:34:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.975+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca88152a30 con 0x7fca880cd5d0 2024-09-17T05:34:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.979+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca88074380 con 0x7fca880cd5d0 2024-09-17T05:34:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.979+0000 7fca7f400640 1 -- 172.21.15.122:0/369430785 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 34) ==== 99070+0+0 (secure 0 0 0) 0x7fca7401c460 con 0x7fca880cd5d0 2024-09-17T05:34:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.979+0000 7fca7f400640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca68074640 0x7fca68076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.979+0000 7fca7f400640 1 -- 172.21.15.122:0/369430785 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fca740930a0 con 0x7fca880cd5d0 2024-09-17T05:34:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.987+0000 7fca86a00640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca68074640 0x7fca68076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.987+0000 7fca7f400640 1 -- 172.21.15.122:0/369430785 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fca7405f7d0 con 0x7fca880cd5d0 2024-09-17T05:34:52.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:52.987+0000 7fca86a00640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca68074640 0x7fca68076b00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca800af000 tx=0x7fca80000f50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:53.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.339+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone67922", "target": ["mon-mgr", ""]}) -- 0x7fca88162890 con 0x7fca68074640 2024-09-17T05:34:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca7f400640 1 -- 172.21.15.122:0/369430785 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca80004a50 con 0x7fca68074640 2024-09-17T05:34:53.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca68074640 msgr2=0x7fca68076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:53.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca68074640 0x7fca68076b00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca800af000 tx=0x7fca80000f50 comp rx=0 tx=0).stop 2024-09-17T05:34:53.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca880cd5d0 msgr2=0x7fca881524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:53.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca880cd5d0 0x7fca881524f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fca7400c430 tx=0x7fca7400c8f0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 shutdown_connections 2024-09-17T05:34:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca68074640 0x7fca68076b00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca880ce4a0 0x7fca880b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca880cd9d0 0x7fca880b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 --2- 172.21.15.122:0/369430785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca880cd5d0 0x7fca881524f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 >> 172.21.15.122:0/369430785 conn(0x7fca880cc4a0 msgr2=0x7fca880cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 shutdown_connections 2024-09-17T05:34:53.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:53.683+0000 7fca8d000640 1 -- 172.21.15.122:0/369430785 wait complete. 2024-09-17T05:34:53.690 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.695+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.690 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.695+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.690 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.695+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.690 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.695+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.690 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.695+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.721 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone84053 2024-09-17T05:34:53.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.731+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.731+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.731+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.731+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:34:53.731+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 -- 172.21.15.122:0/4168726547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe65815bea0 msgr2=0x7fe6580b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 --2- 172.21.15.122:0/4168726547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe65815bea0 0x7fe6580b3130 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe658069820 tx=0x7fe65001cd80 comp rx=0 tx=0).stop 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 -- 172.21.15.122:0/4168726547 shutdown_connections 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 --2- 172.21.15.122:0/4168726547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe65815bea0 0x7fe6580b3130 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 --2- 172.21.15.122:0/4168726547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6580b4bd0 0x7fe6580b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 --2- 172.21.15.122:0/4168726547 >> v2:172.21.15.173:3300/0 conn(0x7fe65815b580 0x7fe65815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 -- 172.21.15.122:0/4168726547 >> 172.21.15.122:0/4168726547 conn(0x7fe6580b18f0 msgr2=0x7fe6580b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 -- 172.21.15.122:0/4168726547 shutdown_connections 2024-09-17T05:34:53.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 -- 172.21.15.122:0/4168726547 wait complete. 2024-09-17T05:34:54.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 Processor -- start 2024-09-17T05:34:54.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.003+0000 7fe65ca00640 1 -- start start 2024-09-17T05:34:54.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6580b4bd0 0x7fe658164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65815b580 0x7fe6581f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65815bea0 0x7fe6581f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6580cc8e0 con 0x7fe6580b4bd0 2024-09-17T05:34:54.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe6580cc760 con 0x7fe65815b580 2024-09-17T05:34:54.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe6580cca60 con 0x7fe65815bea0 2024-09-17T05:34:54.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6580b4bd0 0x7fe658164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6580b4bd0 0x7fe658164bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51934/0 (socket says 172.21.15.122:51934) 2024-09-17T05:34:54.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 -- 172.21.15.122:0/762020605 learned_addr learned my addr 172.21.15.122:0/762020605 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:54.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe656800640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65815bea0 0x7fe6581f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655400640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65815b580 0x7fe6581f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 -- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65815bea0 msgr2=0x7fe6581f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65815bea0 0x7fe6581f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 -- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65815b580 msgr2=0x7fe6581f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65815b580 0x7fe6581f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 -- 172.21.15.122:0/762020605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe658150540 con 0x7fe6580b4bd0 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe655e00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6580b4bd0 0x7fe658164bf0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe64800e9f0 tx=0x7fe64800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe64e000640 1 -- 172.21.15.122:0/762020605 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64800cd80 con 0x7fe6580b4bd0 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe64e000640 1 -- 172.21.15.122:0/762020605 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe648010430 con 0x7fe6580b4bd0 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe64e000640 1 -- 172.21.15.122:0/762020605 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648010860 con 0x7fe6580b4bd0 2024-09-17T05:34:54.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe658150820 con 0x7fe6580b4bd0 2024-09-17T05:34:54.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe658150de0 con 0x7fe6580b4bd0 2024-09-17T05:34:54.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.007+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe658069410 con 0x7fe6580b4bd0 2024-09-17T05:34:54.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.011+0000 7fe64e000640 1 -- 172.21.15.122:0/762020605 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 34) ==== 99070+0+0 (secure 0 0 0) 0x7fe648010a00 con 0x7fe6580b4bd0 2024-09-17T05:34:54.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.011+0000 7fe64e000640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe638074640 0x7fe638076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:54.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.011+0000 7fe64e000640 1 -- 172.21.15.122:0/762020605 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fe648093cc0 con 0x7fe6580b4bd0 2024-09-17T05:34:54.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.011+0000 7fe655400640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe638074640 0x7fe638076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:54.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.019+0000 7fe655400640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe638074640 0x7fe638076b00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe6581f8a60 tx=0x7fe6440091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:54.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.019+0000 7fe64e000640 1 -- 172.21.15.122:0/762020605 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe64805fe60 con 0x7fe6580b4bd0 2024-09-17T05:34:54.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.447+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone84053", "target": ["mon-mgr", ""]}) -- 0x7fe65815b960 con 0x7fe638074640 2024-09-17T05:34:54.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:54.887+0000 7fe64e000640 1 -- 172.21.15.122:0/762020605 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fe648010cf0 con 0x7fe6580b4bd0 2024-09-17T05:34:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.231+0000 7fe64e000640 1 -- 172.21.15.122:0/762020605 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe65815b960 con 0x7fe638074640 2024-09-17T05:34:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe638074640 msgr2=0x7fe638076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe638074640 0x7fe638076b00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe6581f8a60 tx=0x7fe6440091e0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6580b4bd0 msgr2=0x7fe658164bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6580b4bd0 0x7fe658164bf0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe64800e9f0 tx=0x7fe64800eeb0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 shutdown_connections 2024-09-17T05:34:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe638074640 0x7fe638076b00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65815bea0 0x7fe6581f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65815b580 0x7fe6581f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 --2- 172.21.15.122:0/762020605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6580b4bd0 0x7fe658164bf0 secure :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe64800e9f0 tx=0x7fe64800eeb0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 >> 172.21.15.122:0/762020605 conn(0x7fe6580b18f0 msgr2=0x7fe65815d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 shutdown_connections 2024-09-17T05:34:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.235+0000 7fe65ca00640 1 -- 172.21.15.122:0/762020605 wait complete. 2024-09-17T05:34:55.268 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone9004 2024-09-17T05:34:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.555+0000 7f7cd9800640 1 -- 172.21.15.122:0/1865005498 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cd406f940 msgr2=0x7f7cd406fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.555+0000 7f7cd9800640 1 --2- 172.21.15.122:0/1865005498 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cd406f940 0x7f7cd406fdd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009bc0 tx=0x7f7ccc01f3b0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.555+0000 7f7cd9800640 1 -- 172.21.15.122:0/1865005498 shutdown_connections 2024-09-17T05:34:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.555+0000 7f7cd9800640 1 --2- 172.21.15.122:0/1865005498 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cd406f940 0x7f7cd406fdd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.555+0000 7f7cd9800640 1 --2- 172.21.15.122:0/1865005498 >> v2:172.21.15.173:3300/0 conn(0x7f7cd415af10 0x7f7cd4073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.555+0000 7f7cd9800640 1 -- 172.21.15.122:0/1865005498 >> 172.21.15.122:0/1865005498 conn(0x7f7cd40d0b10 msgr2=0x7f7cd40d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.555+0000 7f7cd9800640 1 -- 172.21.15.122:0/1865005498 shutdown_connections 2024-09-17T05:34:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.555+0000 7f7cd9800640 1 -- 172.21.15.122:0/1865005498 wait complete. 2024-09-17T05:34:55.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd9800640 1 Processor -- start 2024-09-17T05:34:55.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd9800640 1 -- start start 2024-09-17T05:34:55.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd9800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cd406f940 0x7f7cd40ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd40cfe40 0x7f7cd41f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd9800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cd41f0b80 0x7f7cd40c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd9800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd40706a0 con 0x7f7cd40cfe40 2024-09-17T05:34:55.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd9800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4070520 con 0x7f7cd406f940 2024-09-17T05:34:55.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd9800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd4070820 con 0x7f7cd41f0b80 2024-09-17T05:34:55.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd40cfe40 0x7f7cd41f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd40cfe40 0x7f7cd41f0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51942/0 (socket says 172.21.15.122:51942) 2024-09-17T05:34:55.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd3400640 1 -- 172.21.15.122:0/3555685694 learned_addr learned my addr 172.21.15.122:0/3555685694 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd3400640 1 -- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cd41f0b80 msgr2=0x7f7cd40c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd3400640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cd41f0b80 0x7f7cd40c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd3400640 1 -- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cd406f940 msgr2=0x7f7cd40ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd3400640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cd406f940 0x7f7cd40ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.559+0000 7f7cd3400640 1 -- 172.21.15.122:0/3555685694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd41f1010 con 0x7f7cd40cfe40 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.563+0000 7f7cd3400640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd40cfe40 0x7f7cd41f0640 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000c4b0 tx=0x7f7cc000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.563+0000 7f7ccbe00640 1 -- 172.21.15.122:0/3555685694 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc00108a0 con 0x7f7cd40cfe40 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.563+0000 7f7ccbe00640 1 -- 172.21.15.122:0/3555685694 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cc0010b30 con 0x7f7cd40cfe40 2024-09-17T05:34:55.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.563+0000 7f7ccbe00640 1 -- 172.21.15.122:0/3555685694 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0011470 con 0x7f7cd40cfe40 2024-09-17T05:34:55.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.563+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd41f1500 con 0x7f7cd40cfe40 2024-09-17T05:34:55.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.563+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd40c6c00 con 0x7f7cd40cfe40 2024-09-17T05:34:55.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.563+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd406fdd0 con 0x7f7cd40cfe40 2024-09-17T05:34:55.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.567+0000 7f7ccbe00640 1 -- 172.21.15.122:0/3555685694 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f7cc0011610 con 0x7f7cd40cfe40 2024-09-17T05:34:55.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.567+0000 7f7ccbe00640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cb4074760 0x7f7cb4076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:55.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.567+0000 7f7ccbe00640 1 -- 172.21.15.122:0/3555685694 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f7cc0095450 con 0x7f7cd40cfe40 2024-09-17T05:34:55.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.571+0000 7f7ccbe00640 1 -- 172.21.15.122:0/3555685694 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7cc0029080 con 0x7f7cd40cfe40 2024-09-17T05:34:55.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.591+0000 7f7cd3e00640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cb4074760 0x7f7cb4076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:55.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.595+0000 7f7cd3e00640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cb4074760 0x7f7cb4076c20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40058d0 tx=0x7f7cc4005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:55.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:55.927+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone9004", "target": ["mon-mgr", ""]}) -- 0x7f7cd40b2310 con 0x7f7cb4074760 2024-09-17T05:34:56.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7ccbe00640 1 -- 172.21.15.122:0/3555685694 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7cd40b2310 con 0x7f7cb4074760 2024-09-17T05:34:56.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cb4074760 msgr2=0x7f7cb4076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cb4074760 0x7f7cb4076c20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40058d0 tx=0x7f7cc4005860 comp rx=0 tx=0).stop 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd40cfe40 msgr2=0x7f7cd41f0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd40cfe40 0x7f7cd41f0640 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000c4b0 tx=0x7f7cc000c970 comp rx=0 tx=0).stop 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 shutdown_connections 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7cb4074760 0x7f7cb4076c20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cd41f0b80 0x7f7cd40c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cd40cfe40 0x7f7cd41f0640 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 --2- 172.21.15.122:0/3555685694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cd406f940 0x7f7cd40ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 >> 172.21.15.122:0/3555685694 conn(0x7f7cd40d0b10 msgr2=0x7f7cd40ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 shutdown_connections 2024-09-17T05:34:56.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.715+0000 7f7cd9800640 1 -- 172.21.15.122:0/3555685694 wait complete. 2024-09-17T05:34:56.744 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:34:56.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.995+0000 7f5324a00640 1 -- 172.21.15.122:0/1581884569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53200b2db0 msgr2=0x7f5304005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:56.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.995+0000 7f5324a00640 1 --2- 172.21.15.122:0/1581884569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53200b2db0 0x7f5304005280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5318001dc0 tx=0x7f5318020ca0 comp rx=0 tx=0).stop 2024-09-17T05:34:56.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.999+0000 7f531e800640 1 -- 172.21.15.122:0/1581884569 reap_dead start 2024-09-17T05:34:56.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.999+0000 7f5324a00640 1 -- 172.21.15.122:0/1581884569 shutdown_connections 2024-09-17T05:34:56.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:56.999+0000 7f5324a00640 1 -- 172.21.15.122:0/1581884569 >> 172.21.15.122:0/1581884569 conn(0x7f532006cdf0 msgr2=0x7f532006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:56.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.003+0000 7f5324a00640 1 -- 172.21.15.122:0/1581884569 shutdown_connections 2024-09-17T05:34:57.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.003+0000 7f5324a00640 1 -- 172.21.15.122:0/1581884569 wait complete. 2024-09-17T05:34:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.011+0000 7f5324a00640 1 Processor -- start 2024-09-17T05:34:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.011+0000 7f5324a00640 1 -- start start 2024-09-17T05:34:57.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5324a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53200b2db0 0x7f53200aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:57.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5324a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53200ae930 0x7f5320201190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5324a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5320155910 0x7f5320155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5324a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f532015c0d0 con 0x7f53200b2db0 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5324a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f532015bf50 con 0x7f5320155910 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5324a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5320073ac0 con 0x7f53200ae930 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53200b2db0 0x7f53200aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53200b2db0 0x7f53200aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51956/0 (socket says 172.21.15.122:51956) 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 -- 172.21.15.122:0/4130439187 learned_addr learned my addr 172.21.15.122:0/4130439187 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531d400640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53200ae930 0x7f5320201190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 -- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53200ae930 msgr2=0x7f5320201190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53200ae930 0x7f5320201190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 -- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5320155910 msgr2=0x7f5320155da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:34:57.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5320155910 0x7f5320155da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 -- 172.21.15.122:0/4130439187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53202016d0 con 0x7f53200b2db0 2024-09-17T05:34:57.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f531de00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53200b2db0 0x7f53200aee60 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f530c00f500 tx=0x7f530c00f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:57.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5316000640 1 -- 172.21.15.122:0/4130439187 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c01a070 con 0x7f53200b2db0 2024-09-17T05:34:57.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5316000640 1 -- 172.21.15.122:0/4130439187 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f530c008570 con 0x7f53200b2db0 2024-09-17T05:34:57.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.027+0000 7f5316000640 1 -- 172.21.15.122:0/4130439187 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c006980 con 0x7f53200b2db0 2024-09-17T05:34:57.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.031+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532014ff70 con 0x7f53200b2db0 2024-09-17T05:34:57.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.031+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53200b14b0 con 0x7f53200b2db0 2024-09-17T05:34:57.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.035+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5320074050 con 0x7f53200b2db0 2024-09-17T05:34:57.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.039+0000 7f5316000640 1 -- 172.21.15.122:0/4130439187 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f530c01d070 con 0x7f53200b2db0 2024-09-17T05:34:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.039+0000 7f5316000640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f53000747b0 0x7f5300076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:57.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.043+0000 7f5316000640 1 -- 172.21.15.122:0/4130439187 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f530c05d9d0 con 0x7f53200b2db0 2024-09-17T05:34:57.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.051+0000 7f5316000640 1 -- 172.21.15.122:0/4130439187 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f530c061760 con 0x7f53200b2db0 2024-09-17T05:34:57.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.059+0000 7f531d400640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f53000747b0 0x7f5300076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:57.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.099+0000 7f531d400640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f53000747b0 0x7f5300076c70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f531000d4e0 tx=0x7f531000f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:57.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.479+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f532015cb90 con 0x7f53200b2db0 2024-09-17T05:34:57.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5316000640 1 -- 172.21.15.122:0/4130439187 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2180 (secure 0 0 0) 0x7f530c066190 con 0x7f53200b2db0 2024-09-17T05:34:57.478 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:57.478 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:34:57.479 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:34:57.480 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:34:57.480 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:57.480 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:34:57.480 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:34:57.480 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:34:57.480 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:34:57.480 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 618, 2024-09-17T05:34:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:34:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:34:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:34:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:34:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:34:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 90, 2024-09-17T05:34:57.482 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:34:57.482 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:34:57.482 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:34:57.482 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:34:57.482 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:34:57.482 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:34:57.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:57.483 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:34:57.483 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:34:57.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:57.483 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:34:57.483 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:34:57.483 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:57.483 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 983, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 3973017333, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 7696560128, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 765397553152, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 68822285, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 151989203, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 17, 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 42 2024-09-17T05:34:57.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 57, 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:34:18:421377+0000", 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 6, 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 6, 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "c", 2024-09-17T05:34:57.485 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 7029 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:34:57.486 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 14, 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:33:48.594120+0000", 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "3 ongoing clones - average progress is 7.552% (1s)\n [==..........................] (remaining: 15s)", 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.075520001351833344, 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:34:57.487 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:57.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:57.488 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:34:57.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f53000747b0 msgr2=0x7f5300076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:57.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f53000747b0 0x7f5300076c70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f531000d4e0 tx=0x7f531000f040 comp rx=0 tx=0).stop 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53200b2db0 msgr2=0x7f53200aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53200b2db0 0x7f53200aee60 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f530c00f500 tx=0x7f530c00f9c0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 shutdown_connections 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f53000747b0 0x7f5300076c70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5320155910 0x7f5320155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53200ae930 0x7f5320201190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 --2- 172.21.15.122:0/4130439187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53200b2db0 0x7f53200aee60 secure :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f530c00f500 tx=0x7f530c00f9c0 comp rx=0 tx=0).stop 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.483+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 >> 172.21.15.122:0/4130439187 conn(0x7f532006cdf0 msgr2=0x7f532006f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.487+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 shutdown_connections 2024-09-17T05:34:57.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:57.487+0000 7f5324a00640 1 -- 172.21.15.122:0/4130439187 wait complete. 2024-09-17T05:34:58.519 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:34:58.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.775+0000 7f668de00640 1 -- 172.21.15.122:0/2646250616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f668815a3e0 msgr2=0x7f668806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.775+0000 7f668de00640 1 --2- 172.21.15.122:0/2646250616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f668815a3e0 0x7f668806f090 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f66880cad70 tx=0x7f667801db50 comp rx=0 tx=0).stop 2024-09-17T05:34:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 -- 172.21.15.122:0/2646250616 shutdown_connections 2024-09-17T05:34:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 --2- 172.21.15.122:0/2646250616 >> v2:172.21.15.173:3300/0 conn(0x7f66880ce200 0x7f66880734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 --2- 172.21.15.122:0/2646250616 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66880cf160 0x7f6688072f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 --2- 172.21.15.122:0/2646250616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f668815a3e0 0x7f668806f090 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 -- 172.21.15.122:0/2646250616 >> 172.21.15.122:0/2646250616 conn(0x7f66880b2060 msgr2=0x7f66880b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 -- 172.21.15.122:0/2646250616 shutdown_connections 2024-09-17T05:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 -- 172.21.15.122:0/2646250616 wait complete. 2024-09-17T05:34:58.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 Processor -- start 2024-09-17T05:34:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.779+0000 7f668de00640 1 -- start start 2024-09-17T05:34:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f668de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66880ce200 0x7f66880c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f668de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66880cf160 0x7f66880ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f668de00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f668815a3e0 0x7f66881572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f668de00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66880cc6e0 con 0x7f66880cf160 2024-09-17T05:34:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f668de00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66880cc560 con 0x7f66880ce200 2024-09-17T05:34:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f668de00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f66880cc860 con 0x7f668815a3e0 2024-09-17T05:34:58.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f6686a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66880ce200 0x7f66880c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f6686a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66880ce200 0x7f66880c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:47458/0 (socket says 172.21.15.122:47458) 2024-09-17T05:34:58.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.783+0000 7f6686a00640 1 -- 172.21.15.122:0/3866060064 learned_addr learned my addr 172.21.15.122:0/3866060064 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:34:58.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.787+0000 7f6687400640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f668815a3e0 0x7f66881572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.787+0000 7f6686000640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66880cf160 0x7f66880ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.787+0000 7f6686a00640 1 -- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f668815a3e0 msgr2=0x7f66881572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.787+0000 7f6686a00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f668815a3e0 0x7f66881572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.787+0000 7f6686a00640 1 -- 172.21.15.122:0/3866060064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66880cf160 msgr2=0x7f66880ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.787+0000 7f6686a00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66880cf160 0x7f66880ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.787+0000 7f6686a00640 1 -- 172.21.15.122:0/3866060064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66881f69c0 con 0x7f66880ce200 2024-09-17T05:34:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.787+0000 7f6686000640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66880cf160 0x7f66880ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:34:58.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.791+0000 7f6686a00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66880ce200 0x7f66880c6780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f66780b1040 tx=0x7f66780a69f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:58.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.799+0000 7f667f400640 1 -- 172.21.15.122:0/3866060064 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66780b3070 con 0x7f66880ce200 2024-09-17T05:34:58.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.799+0000 7f667f400640 1 -- 172.21.15.122:0/3866060064 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6678003b80 con 0x7f66880ce200 2024-09-17T05:34:58.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.799+0000 7f667f400640 1 -- 172.21.15.122:0/3866060064 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66780ae440 con 0x7f66880ce200 2024-09-17T05:34:58.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.799+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66880c6cc0 con 0x7f66880ce200 2024-09-17T05:34:58.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.799+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66881f6120 con 0x7f66880ce200 2024-09-17T05:34:58.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.799+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668806f090 con 0x7f66880ce200 2024-09-17T05:34:58.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.807+0000 7f667f400640 1 -- 172.21.15.122:0/3866060064 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f66780bd060 con 0x7f66880ce200 2024-09-17T05:34:58.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.807+0000 7f667f400640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66680747b0 0x7f6668076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:34:58.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.807+0000 7f667f400640 1 -- 172.21.15.122:0/3866060064 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f667812d400 con 0x7f66880ce200 2024-09-17T05:34:58.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.819+0000 7f667f400640 1 -- 172.21.15.122:0/3866060064 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6678017800 con 0x7f66880ce200 2024-09-17T05:34:58.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.835+0000 7f6686000640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66680747b0 0x7f6668076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:34:58.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:58.919+0000 7f6686000640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66680747b0 0x7f6668076c70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6688159e00 tx=0x7f6674008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:34:59.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.295+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f66880afdf0 con 0x7f66880ce200 2024-09-17T05:34:59.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.311+0000 7f667f400640 1 -- 172.21.15.122:0/3866060064 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2181 (secure 0 0 0) 0x7f66880afdf0 con 0x7f66880ce200 2024-09-17T05:34:59.307 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:34:59.307 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:34:59.308 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 620, 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:34:59.309 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 90, 2024-09-17T05:34:59.310 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:34:59.310 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:34:59.310 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:34:59.310 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:34:59.310 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:34:59.310 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:34:59.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 1038, 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 4207898169, 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 8548290560, 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 764545822720, 2024-09-17T05:34:59.311 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 94669106, 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 171543838, 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 23, 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 46 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 57, 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:34:18:421377+0000", 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 6, 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:34:59.312 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:34:59.313 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:34:59.313 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:34:59.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:34:59.313 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 6, 2024-09-17T05:34:59.313 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:34:59.313 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "c", 2024-09-17T05:34:59.313 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 7029 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:34:59.314 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:34:59.315 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:34:59.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:59.315 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:34:59.315 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 14, 2024-09-17T05:34:59.315 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:33:48.594120+0000", 2024-09-17T05:34:59.315 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:34:59.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:34:59.316 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:34:59.316 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:34:59.316 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "4 ongoing clones - average progress is 20.573% (4s)\n [=====.......................] (remaining: 16s)", 2024-09-17T05:34:59.316 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.20573000609874725, 2024-09-17T05:34:59.316 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:34:59.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:59.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:34:59.316 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:34:59.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66680747b0 msgr2=0x7f6668076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66680747b0 0x7f6668076c70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6688159e00 tx=0x7f6674008040 comp rx=0 tx=0).stop 2024-09-17T05:34:59.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66880ce200 msgr2=0x7f66880c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:34:59.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66880ce200 0x7f66880c6780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f66780b1040 tx=0x7f66780a69f0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 shutdown_connections 2024-09-17T05:34:59.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66680747b0 0x7f6668076c70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f668815a3e0 0x7f66881572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66880cf160 0x7f66880ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 --2- 172.21.15.122:0/3866060064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66880ce200 0x7f66880c6780 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:34:59.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 >> 172.21.15.122:0/3866060064 conn(0x7f66880b2060 msgr2=0x7f66880b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:34:59.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 shutdown_connections 2024-09-17T05:34:59.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:34:59.323+0000 7f668de00640 1 -- 172.21.15.122:0/3866060064 wait complete. 2024-09-17T05:35:00.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:35:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.679+0000 7f0df9a00640 1 -- 172.21.15.122:0/2858840342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df4070b60 msgr2=0x7f0df40d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.679+0000 7f0df9a00640 1 --2- 172.21.15.122:0/2858840342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df4070b60 0x7f0df40d00e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009bc0 tx=0x7f0dec01cb60 comp rx=0 tx=0).stop 2024-09-17T05:35:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 -- 172.21.15.122:0/2858840342 shutdown_connections 2024-09-17T05:35:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 --2- 172.21.15.122:0/2858840342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df4070b60 0x7f0df40d00e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 --2- 172.21.15.122:0/2858840342 >> v2:172.21.15.173:3300/0 conn(0x7f0df40b4af0 0x7f0df406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 -- 172.21.15.122:0/2858840342 >> 172.21.15.122:0/2858840342 conn(0x7f0df406cdf0 msgr2=0x7f0df406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 -- 172.21.15.122:0/2858840342 shutdown_connections 2024-09-17T05:35:00.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 -- 172.21.15.122:0/2858840342 wait complete. 2024-09-17T05:35:00.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 Processor -- start 2024-09-17T05:35:00.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 -- start start 2024-09-17T05:35:00.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df4070b60 0x7f0df40aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:00.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df40b4af0 0x7f0df413a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:00.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0df40b4ed0 0x7f0df40ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:00.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0df41681a0 con 0x7f0df40b4af0 2024-09-17T05:35:00.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4168020 con 0x7f0df4070b60 2024-09-17T05:35:00.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df9a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0df4168320 con 0x7f0df40b4ed0 2024-09-17T05:35:00.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df2000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df40b4af0 0x7f0df413a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:00.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df2000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df40b4af0 0x7f0df413a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51988/0 (socket says 172.21.15.122:51988) 2024-09-17T05:35:00.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.683+0000 7f0df2000640 1 -- 172.21.15.122:0/831052871 learned_addr learned my addr 172.21.15.122:0/831052871 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:00.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0df2000640 1 -- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0df40b4ed0 msgr2=0x7f0df40ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0df2000640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0df40b4ed0 0x7f0df40ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0df2000640 1 -- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df4070b60 msgr2=0x7f0df40aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0df2000640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df4070b60 0x7f0df40aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0df2000640 1 -- 172.21.15.122:0/831052871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df40addc0 con 0x7f0df40b4af0 2024-09-17T05:35:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0df2000640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df40b4af0 0x7f0df413a880 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0de000c3c0 tx=0x7f0de000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0deaa00640 1 -- 172.21.15.122:0/831052871 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0011790 con 0x7f0df40b4af0 2024-09-17T05:35:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0deaa00640 1 -- 172.21.15.122:0/831052871 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0de0017070 con 0x7f0df40b4af0 2024-09-17T05:35:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0deaa00640 1 -- 172.21.15.122:0/831052871 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0011ce0 con 0x7f0df40b4af0 2024-09-17T05:35:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df414a150 con 0x7f0df40b4af0 2024-09-17T05:35:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.687+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df40ae320 con 0x7f0df40b4af0 2024-09-17T05:35:00.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.691+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df40d00e0 con 0x7f0df40b4af0 2024-09-17T05:35:00.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.691+0000 7f0deaa00640 1 -- 172.21.15.122:0/831052871 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f0de001c460 con 0x7f0df40b4af0 2024-09-17T05:35:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.691+0000 7f0deaa00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0dd40747b0 0x7f0dd4076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.695+0000 7f0deaa00640 1 -- 172.21.15.122:0/831052871 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f0de0092b10 con 0x7f0df40b4af0 2024-09-17T05:35:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.695+0000 7f0df2a00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0dd40747b0 0x7f0dd4076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:00.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.703+0000 7f0df2a00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0dd40747b0 0x7f0dd4076c70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009730 tx=0x7f0de4006ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:00.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:00.703+0000 7f0deaa00640 1 -- 172.21.15.122:0/831052871 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0df40addc0 con 0x7f0df40b4af0 2024-09-17T05:35:01.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.179+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0df406d420 con 0x7f0df40b4af0 2024-09-17T05:35:01.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.179+0000 7f0deaa00640 1 -- 172.21.15.122:0/831052871 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2423 (secure 0 0 0) 0x7f0de005ef80 con 0x7f0df40b4af0 2024-09-17T05:35:01.178 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:35:01.178 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:35:01.178 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:35:01.178 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:35:01.179 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:35:01.179 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:35:01.179 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:35:01.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:01.179 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:35:01.179 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:35:01.179 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:35:01.179 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:35:01.180 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:35:01.180 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:01.180 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:35:01.180 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:35:01.180 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:35:01.180 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:35:01.181 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:01.181 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 622, 2024-09-17T05:35:01.181 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:35:01.181 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:35:01.181 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:35:01.181 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:35:01.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:01.181 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 90, 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:35:01.182 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 1816, 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 7475492098, 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 14747025408, 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 758347087872, 2024-09-17T05:35:01.183 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 366330909, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 435519623, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 89, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 117 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 57, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:34:18:421377+0000", 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 6, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:35:01.184 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 6, 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "c", 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 7029 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:35:01.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:35:01.186 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 14, 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:33:48.594120+0000", 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "4 ongoing clones - average progress is 27.474% (7s)\n [=======.....................] (remaining: 19s)", 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.27474001049995422, 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-total-clones": { 2024-09-17T05:35:01.187 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "Total 5 clones - average progress is 30.729% (0s)\n [========....................] ", 2024-09-17T05:35:01.188 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.30728998780250549, 2024-09-17T05:35:01.188 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:35:01.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:35:01.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:35:01.188 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0dd40747b0 msgr2=0x7f0dd4076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0dd40747b0 0x7f0dd4076c70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009730 tx=0x7f0de4006ca0 comp rx=0 tx=0).stop 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df40b4af0 msgr2=0x7f0df413a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df40b4af0 0x7f0df413a880 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0de000c3c0 tx=0x7f0de000c880 comp rx=0 tx=0).stop 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 shutdown_connections 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0dd40747b0 0x7f0dd4076c70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0df40b4ed0 0x7f0df40ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0df40b4af0 0x7f0df413a880 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:01.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 --2- 172.21.15.122:0/831052871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0df4070b60 0x7f0df40aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:01.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 >> 172.21.15.122:0/831052871 conn(0x7f0df406cdf0 msgr2=0x7f0df40b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:01.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.187+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 shutdown_connections 2024-09-17T05:35:01.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.191+0000 7f0df9a00640 1 -- 172.21.15.122:0/831052871 wait complete. 2024-09-17T05:35:01.227 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 27.474% (7s)\n [=======.....................] (remaining: 19s)', 'progress': 0.2747400104999542, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 5 clones - average progress is 30.729% (0s)\n [========....................] ', 'progress': 0.3072899878025055, 'add_to_ceph_s': True}} 2024-09-17T05:35:01.227 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone25105 2024-09-17T05:35:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.519+0000 7f9605200640 1 -- 172.21.15.122:0/2381994578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95e4004810 msgr2=0x7f95e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.519+0000 7f9605200640 1 --2- 172.21.15.122:0/2381994578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95e4004810 0x7f95e4004c90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009b40 tx=0x7f95ec01d290 comp rx=0 tx=0).stop 2024-09-17T05:35:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.519+0000 7f9605200640 1 -- 172.21.15.122:0/2381994578 shutdown_connections 2024-09-17T05:35:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.519+0000 7f9605200640 1 --2- 172.21.15.122:0/2381994578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95e4004810 0x7f95e4004c90 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.519+0000 7f9605200640 1 -- 172.21.15.122:0/2381994578 >> 172.21.15.122:0/2381994578 conn(0x7f96000d0b80 msgr2=0x7f96000d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.519+0000 7f9605200640 1 -- 172.21.15.122:0/2381994578 shutdown_connections 2024-09-17T05:35:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.519+0000 7f9605200640 1 -- 172.21.15.122:0/2381994578 wait complete. 2024-09-17T05:35:01.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.523+0000 7f9605200640 1 Processor -- start 2024-09-17T05:35:01.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.523+0000 7f9605200640 1 -- start start 2024-09-17T05:35:01.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f9605200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e4004810 0x7f960015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:01.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f9605200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96001f7b60 0x7f96001f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:01.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f9605200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96000c8f00 0x7f96000c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:01.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f9605200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96000cc980 con 0x7f96000c8f00 2024-09-17T05:35:01.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f9605200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f96000cc800 con 0x7f96001f7b60 2024-09-17T05:35:01.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f9605200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9600073720 con 0x7f95e4004810 2024-09-17T05:35:01.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95ffe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96000c8f00 0x7f96000c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:01.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95ffe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96000c8f00 0x7f96000c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34274/0 (socket says 172.21.15.122:34274) 2024-09-17T05:35:01.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95ffe00640 1 -- 172.21.15.122:0/92084694 learned_addr learned my addr 172.21.15.122:0/92084694 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:01.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95fea00640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96001f7b60 0x7f96001f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:01.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95ffe00640 1 -- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e4004810 msgr2=0x7f960015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95ffe00640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e4004810 0x7f960015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95ffe00640 1 -- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96001f7b60 msgr2=0x7f96001f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95ffe00640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96001f7b60 0x7f96001f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.527+0000 7f95ffe00640 1 -- 172.21.15.122:0/92084694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96000c9d40 con 0x7f96000c8f00 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.531+0000 7f95ffe00640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96000c8f00 0x7f96000c93b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f95e4004790 tx=0x7f95f800e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.531+0000 7f95f7400640 1 -- 172.21.15.122:0/92084694 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f80107d0 con 0x7f96000c8f00 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.531+0000 7f95f7400640 1 -- 172.21.15.122:0/92084694 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95f8016070 con 0x7f96000c8f00 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.531+0000 7f95f7400640 1 -- 172.21.15.122:0/92084694 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8010d20 con 0x7f96000c8f00 2024-09-17T05:35:01.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.531+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96000ca020 con 0x7f96000c8f00 2024-09-17T05:35:01.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.531+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96000ca5e0 con 0x7f96000c8f00 2024-09-17T05:35:01.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.531+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9600073bf0 con 0x7f96000c8f00 2024-09-17T05:35:01.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.535+0000 7f95f7400640 1 -- 172.21.15.122:0/92084694 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f95f8011460 con 0x7f96000c8f00 2024-09-17T05:35:01.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.535+0000 7f95f7400640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95e0074760 0x7f95e0076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:01.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.535+0000 7f95f7400640 1 -- 172.21.15.122:0/92084694 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f95f8094290 con 0x7f96000c8f00 2024-09-17T05:35:01.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.543+0000 7f95ff400640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95e0074760 0x7f95e0076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:01.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.543+0000 7f95f7400640 1 -- 172.21.15.122:0/92084694 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f95f8060990 con 0x7f96000c8f00 2024-09-17T05:35:01.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.587+0000 7f95ff400640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95e0074760 0x7f95e0076c20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f95f0001000 tx=0x7f95f0001bf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:01.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:01.891+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone25105", "target": ["mon-mgr", ""]}) -- 0x7f96000b6b00 con 0x7f95e0074760 2024-09-17T05:35:03.123 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:03.134 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:03.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.135+0000 7f95f7400640 1 -- 172.21.15.122:0/92084694 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f96000b6b00 con 0x7f95e0074760 2024-09-17T05:35:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.143+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95e0074760 msgr2=0x7f95e0076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.143+0000 7f9605200640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95e0074760 0x7f95e0076c20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f95f0001000 tx=0x7f95f0001bf0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.143+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96000c8f00 msgr2=0x7f96000c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:03.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.143+0000 7f9605200640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96000c8f00 0x7f96000c93b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f95e4004790 tx=0x7f95f800e6b0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.147+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 shutdown_connections 2024-09-17T05:35:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.147+0000 7f9605200640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f95e0074760 0x7f95e0076c20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.147+0000 7f9605200640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96000c8f00 0x7f96000c93b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.147+0000 7f9605200640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96001f7b60 0x7f96001f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.147+0000 7f9605200640 1 --2- 172.21.15.122:0/92084694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e4004810 0x7f960015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.147+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 >> 172.21.15.122:0/92084694 conn(0x7f96000d0b80 msgr2=0x7f96000b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.147+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 shutdown_connections 2024-09-17T05:35:03.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.147+0000 7f9605200640 1 -- 172.21.15.122:0/92084694 wait complete. 2024-09-17T05:35:03.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone67336 2024-09-17T05:35:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.575+0000 7fa500a00640 1 -- 172.21.15.122:0/1926510614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 msgr2=0x7fa4e00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.575+0000 7fa500a00640 1 --2- 172.21.15.122:0/1926510614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 0x7fa4e00051d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009e30 tx=0x7fa4f401ce60 comp rx=0 tx=0).stop 2024-09-17T05:35:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.575+0000 7fa4fa800640 1 -- 172.21.15.122:0/1926510614 reap_dead start 2024-09-17T05:35:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.575+0000 7fa500a00640 1 -- 172.21.15.122:0/1926510614 shutdown_connections 2024-09-17T05:35:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.575+0000 7fa500a00640 1 -- 172.21.15.122:0/1926510614 >> 172.21.15.122:0/1926510614 conn(0x7fa4fc0d3f90 msgr2=0x7fa4fc0d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.575+0000 7fa500a00640 1 -- 172.21.15.122:0/1926510614 shutdown_connections 2024-09-17T05:35:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.575+0000 7fa500a00640 1 -- 172.21.15.122:0/1926510614 wait complete. 2024-09-17T05:35:03.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.579+0000 7fa500a00640 1 Processor -- start 2024-09-17T05:35:03.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.579+0000 7fa500a00640 1 -- start start 2024-09-17T05:35:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.579+0000 7fa500a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 0x7fa4fc14dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.579+0000 7fa500a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4fc14e830 0x7fa4fc0cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.579+0000 7fa500a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4fc0cc7e0 0x7fa4fc14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.579+0000 7fa500a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc0cfe50 con 0x7fa4e0004d50 2024-09-17T05:35:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.579+0000 7fa500a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc0cfcd0 con 0x7fa4fc0cc7e0 2024-09-17T05:35:03.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.579+0000 7fa500a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4fc167eb0 con 0x7fa4fc14e830 2024-09-17T05:35:03.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 0x7fa4fc14dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:03.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 0x7fa4fc14dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34284/0 (socket says 172.21.15.122:34284) 2024-09-17T05:35:03.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 -- 172.21.15.122:0/4111108425 learned_addr learned my addr 172.21.15.122:0/4111108425 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9400640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4fc14e830 0x7fa4fc0cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 -- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4fc14e830 msgr2=0x7fa4fc0cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4fc14e830 0x7fa4fc0cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 -- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4fc0cc7e0 msgr2=0x7fa4fc14bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4fc0cc7e0 0x7fa4fc14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 -- 172.21.15.122:0/4111108425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc0cb200 con 0x7fa4e0004d50 2024-09-17T05:35:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f9e00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 0x7fa4fc14dff0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00c530 tx=0x7fa4ec00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f2000640 1 -- 172.21.15.122:0/4111108425 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0052d0 con 0x7fa4e0004d50 2024-09-17T05:35:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f2000640 1 -- 172.21.15.122:0/4111108425 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4ec005470 con 0x7fa4e0004d50 2024-09-17T05:35:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa4f2000640 1 -- 172.21.15.122:0/4111108425 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0038b0 con 0x7fa4e0004d50 2024-09-17T05:35:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc0cb4e0 con 0x7fa4e0004d50 2024-09-17T05:35:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.583+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc0cbaa0 con 0x7fa4e0004d50 2024-09-17T05:35:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.587+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4fc0b0ae0 con 0x7fa4e0004d50 2024-09-17T05:35:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.587+0000 7fa4f2000640 1 -- 172.21.15.122:0/4111108425 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fa4ec003a50 con 0x7fa4e0004d50 2024-09-17T05:35:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.591+0000 7fa4f2000640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4dc074760 0x7fa4dc076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:03.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.591+0000 7fa4f2000640 1 -- 172.21.15.122:0/4111108425 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fa4ec092dd0 con 0x7fa4e0004d50 2024-09-17T05:35:03.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.595+0000 7fa4f2000640 1 -- 172.21.15.122:0/4111108425 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa4ec05f530 con 0x7fa4e0004d50 2024-09-17T05:35:03.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.647+0000 7fa4f9400640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4dc074760 0x7fa4dc076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:03.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.679+0000 7fa4f9400640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4dc074760 0x7fa4dc076c20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8001340 tx=0x7fa4e800c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:03.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:03.931+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone67336", "target": ["mon-mgr", ""]}) -- 0x7fa4fc0b9f10 con 0x7fa4dc074760 2024-09-17T05:35:05.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.035+0000 7fa4f2000640 1 -- 172.21.15.122:0/4111108425 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa4fc0b9f10 con 0x7fa4dc074760 2024-09-17T05:35:05.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4dc074760 msgr2=0x7fa4dc076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:05.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4dc074760 0x7fa4dc076c20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8001340 tx=0x7fa4e800c040 comp rx=0 tx=0).stop 2024-09-17T05:35:05.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 msgr2=0x7fa4fc14dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:05.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 0x7fa4fc14dff0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00c530 tx=0x7fa4ec00c9f0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 shutdown_connections 2024-09-17T05:35:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4dc074760 0x7fa4dc076c20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4fc0cc7e0 0x7fa4fc14bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4fc14e830 0x7fa4fc0cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 --2- 172.21.15.122:0/4111108425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4e0004d50 0x7fa4fc14dff0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 >> 172.21.15.122:0/4111108425 conn(0x7fa4fc0d3f90 msgr2=0x7fa4fc160370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 shutdown_connections 2024-09-17T05:35:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.043+0000 7fa500a00640 1 -- 172.21.15.122:0/4111108425 wait complete. 2024-09-17T05:35:05.090 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone21727 2024-09-17T05:35:05.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.387+0000 7fe009400640 1 -- 172.21.15.122:0/4007056229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe004071210 msgr2=0x7fe00415c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:05.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.387+0000 7fe009400640 1 --2- 172.21.15.122:0/4007056229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe004071210 0x7fe00415c0b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009840 tx=0x7fdfec01c820 comp rx=0 tx=0).stop 2024-09-17T05:35:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.391+0000 7fe009400640 1 -- 172.21.15.122:0/4007056229 shutdown_connections 2024-09-17T05:35:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.391+0000 7fe009400640 1 --2- 172.21.15.122:0/4007056229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe004071210 0x7fe00415c0b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.391+0000 7fe009400640 1 -- 172.21.15.122:0/4007056229 >> 172.21.15.122:0/4007056229 conn(0x7fe0040b1c70 msgr2=0x7fe0040b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.391+0000 7fe009400640 1 -- 172.21.15.122:0/4007056229 shutdown_connections 2024-09-17T05:35:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.391+0000 7fe009400640 1 -- 172.21.15.122:0/4007056229 wait complete. 2024-09-17T05:35:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe009400640 1 Processor -- start 2024-09-17T05:35:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe009400640 1 -- start start 2024-09-17T05:35:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe009400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe004071210 0x7fe0042000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe009400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0040cc8f0 0x7fe004145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe009400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe00415bcd0 0x7fe004164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe009400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe00416a130 con 0x7fe00415bcd0 2024-09-17T05:35:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe009400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe004169fb0 con 0x7fe004071210 2024-09-17T05:35:05.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe009400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe00416a2b0 con 0x7fe0040cc8f0 2024-09-17T05:35:05.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe00415bcd0 0x7fe004164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:05.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe00415bcd0 0x7fe004164f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34298/0 (socket says 172.21.15.122:34298) 2024-09-17T05:35:05.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 -- 172.21.15.122:0/4079995554 learned_addr learned my addr 172.21.15.122:0/4079995554 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 -- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0040cc8f0 msgr2=0x7fe004145c90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:35:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0040cc8f0 0x7fe004145c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 -- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe004071210 msgr2=0x7fe0042000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe004071210 0x7fe0042000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 -- 172.21.15.122:0/4079995554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0041461d0 con 0x7fe00415bcd0 2024-09-17T05:35:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.395+0000 7fe008a00640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe00415bcd0 0x7fe004164f40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c780 tx=0x7fdff400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:05.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.399+0000 7fdffbe00640 1 -- 172.21.15.122:0/4079995554 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4017070 con 0x7fe00415bcd0 2024-09-17T05:35:05.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.399+0000 7fdffbe00640 1 -- 172.21.15.122:0/4079995554 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdff4005070 con 0x7fe00415bcd0 2024-09-17T05:35:05.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.399+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00414e680 con 0x7fe00415bcd0 2024-09-17T05:35:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.399+0000 7fdffbe00640 1 -- 172.21.15.122:0/4079995554 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4005500 con 0x7fe00415bcd0 2024-09-17T05:35:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.399+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00414b980 con 0x7fe00415bcd0 2024-09-17T05:35:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.399+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00415c0b0 con 0x7fe00415bcd0 2024-09-17T05:35:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.403+0000 7fdffbe00640 1 -- 172.21.15.122:0/4079995554 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fdff401a070 con 0x7fe00415bcd0 2024-09-17T05:35:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.407+0000 7fdffbe00640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfe0074760 0x7fdfe0076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.407+0000 7fdffbe00640 1 -- 172.21.15.122:0/4079995554 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fdff40929e0 con 0x7fe00415bcd0 2024-09-17T05:35:05.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.411+0000 7fdffbe00640 1 -- 172.21.15.122:0/4079995554 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdff405f0e0 con 0x7fe00415bcd0 2024-09-17T05:35:05.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.419+0000 7fe003e00640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfe0074760 0x7fdfe0076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:05.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.423+0000 7fe003e00640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfe0074760 0x7fdfe0076c20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdffc000f70 tx=0x7fdffc0079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:05.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:05.767+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone21727", "target": ["mon-mgr", ""]}) -- 0x7fe004156a30 con 0x7fdfe0074760 2024-09-17T05:35:06.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.822+0000 7fdffbe00640 1 -- 172.21.15.122:0/4079995554 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe004156a30 con 0x7fdfe0074760 2024-09-17T05:35:06.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfe0074760 msgr2=0x7fdfe0076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfe0074760 0x7fdfe0076c20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdffc000f70 tx=0x7fdffc0079c0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe00415bcd0 msgr2=0x7fe004164f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:06.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe00415bcd0 0x7fe004164f40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c780 tx=0x7fdff400cc40 comp rx=0 tx=0).stop 2024-09-17T05:35:06.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 shutdown_connections 2024-09-17T05:35:06.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfe0074760 0x7fdfe0076c20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe00415bcd0 0x7fe004164f40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0040cc8f0 0x7fe004145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 --2- 172.21.15.122:0/4079995554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe004071210 0x7fe0042000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:06.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 >> 172.21.15.122:0/4079995554 conn(0x7fe0040b1c70 msgr2=0x7fe004156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:06.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 shutdown_connections 2024-09-17T05:35:06.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:06.826+0000 7fe009400640 1 -- 172.21.15.122:0/4079995554 wait complete. 2024-09-17T05:35:06.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone59558 2024-09-17T05:35:07.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.122+0000 7fa470e00640 1 -- 172.21.15.122:0/1685749517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 msgr2=0x7fa46c15aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.122+0000 7fa470e00640 1 --2- 172.21.15.122:0/1685749517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 0x7fa46c15aac0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa464009bc0 tx=0x7fa46401cba0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.126+0000 7fa470e00640 1 -- 172.21.15.122:0/1685749517 shutdown_connections 2024-09-17T05:35:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.126+0000 7fa470e00640 1 --2- 172.21.15.122:0/1685749517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 0x7fa46c15aac0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.126+0000 7fa470e00640 1 -- 172.21.15.122:0/1685749517 >> 172.21.15.122:0/1685749517 conn(0x7fa46c0b1c70 msgr2=0x7fa46c0b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.130+0000 7fa470e00640 1 -- 172.21.15.122:0/1685749517 shutdown_connections 2024-09-17T05:35:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.130+0000 7fa470e00640 1 -- 172.21.15.122:0/1685749517 wait complete. 2024-09-17T05:35:07.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.134+0000 7fa470e00640 1 Processor -- start 2024-09-17T05:35:07.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.134+0000 7fa470e00640 1 -- start start 2024-09-17T05:35:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.138+0000 7fa470e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa46c0b2f70 0x7fa46c1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.138+0000 7fa470e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 0x7fa46c1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.138+0000 7fa470e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa46c15b830 0x7fa46c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.138+0000 7fa470e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c15fb30 con 0x7fa46c0cc1f0 2024-09-17T05:35:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.138+0000 7fa470e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c15f9b0 con 0x7fa46c15b830 2024-09-17T05:35:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.138+0000 7fa470e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa46c15fcb0 con 0x7fa46c0b2f70 2024-09-17T05:35:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa46c15b830 0x7fa46c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 0x7fa46c1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa46c15b830 0x7fa46c151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:55248/0 (socket says 172.21.15.122:55248) 2024-09-17T05:35:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 0x7fa46c1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34314/0 (socket says 172.21.15.122:34314) 2024-09-17T05:35:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46be00640 1 -- 172.21.15.122:0/2956770220 learned_addr learned my addr 172.21.15.122:0/2956770220 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46b400640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa46c0b2f70 0x7fa46c1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46be00640 1 -- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa46c0b2f70 msgr2=0x7fa46c1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46be00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa46c0b2f70 0x7fa46c1603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.142+0000 7fa46be00640 1 -- 172.21.15.122:0/2956770220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 msgr2=0x7fa46c1512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:07.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.146+0000 7fa46be00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 0x7fa46c1512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:07.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.146+0000 7fa46be00640 1 -- 172.21.15.122:0/2956770220 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c150240 con 0x7fa46c15b830 2024-09-17T05:35:07.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.146+0000 7fa46aa00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 0x7fa46c1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:07.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.154+0000 7fa46be00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa46c15b830 0x7fa46c151820 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa45800b3e0 tx=0x7fa45800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.162+0000 7fa463400640 1 -- 172.21.15.122:0/2956770220 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458015020 con 0x7fa46c15b830 2024-09-17T05:35:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.162+0000 7fa463400640 1 -- 172.21.15.122:0/2956770220 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa458011940 con 0x7fa46c15b830 2024-09-17T05:35:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.162+0000 7fa463400640 1 -- 172.21.15.122:0/2956770220 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458011d90 con 0x7fa46c15b830 2024-09-17T05:35:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.162+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46c150520 con 0x7fa46c15b830 2024-09-17T05:35:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.166+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa46c150ae0 con 0x7fa46c15b830 2024-09-17T05:35:07.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.170+0000 7fa463400640 1 -- 172.21.15.122:0/2956770220 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fa458018020 con 0x7fa46c15b830 2024-09-17T05:35:07.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.170+0000 7fa463400640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa44c074810 0x7fa44c076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:07.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.170+0000 7fa463400640 1 -- 172.21.15.122:0/2956770220 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fa45805b170 con 0x7fa46c15b830 2024-09-17T05:35:07.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.174+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa46c15aac0 con 0x7fa46c15b830 2024-09-17T05:35:07.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.186+0000 7fa46b400640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa44c074810 0x7fa44c076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:07.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.206+0000 7fa463400640 1 -- 172.21.15.122:0/2956770220 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa45801f030 con 0x7fa46c15b830 2024-09-17T05:35:07.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.238+0000 7fa46b400640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa44c074810 0x7fa44c076cd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa4640af000 tx=0x7fa46401b3e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:07.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:07.538+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone59558", "target": ["mon-mgr", ""]}) -- 0x7fa46c15f290 con 0x7fa44c074810 2024-09-17T05:35:08.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa463400640 1 -- 172.21.15.122:0/2956770220 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa46c15f290 con 0x7fa44c074810 2024-09-17T05:35:08.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa44c074810 msgr2=0x7fa44c076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa44c074810 0x7fa44c076cd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa4640af000 tx=0x7fa46401b3e0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa46c15b830 msgr2=0x7fa46c151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:08.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa46c15b830 0x7fa46c151820 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa45800b3e0 tx=0x7fa45800b8a0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 shutdown_connections 2024-09-17T05:35:08.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa44c074810 0x7fa44c076cd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa46c15b830 0x7fa46c151820 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa46c0cc1f0 0x7fa46c1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 --2- 172.21.15.122:0/2956770220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa46c0b2f70 0x7fa46c1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:08.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 >> 172.21.15.122:0/2956770220 conn(0x7fa46c0b1c70 msgr2=0x7fa46c15c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:08.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 shutdown_connections 2024-09-17T05:35:08.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:08.762+0000 7fa470e00640 1 -- 172.21.15.122:0/2956770220 wait complete. 2024-09-17T05:35:08.796 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone44559 2024-09-17T05:35:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.042+0000 7f3ebfa00640 1 -- 172.21.15.122:0/2075314456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ea0004810 msgr2=0x7f3ea0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.042+0000 7f3ebfa00640 1 --2- 172.21.15.122:0/2075314456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ea0004810 0x7f3ea0004c90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4005fd0 tx=0x7f3ea401cd60 comp rx=0 tx=0).stop 2024-09-17T05:35:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.042+0000 7f3ebda00640 1 -- 172.21.15.122:0/2075314456 reap_dead start 2024-09-17T05:35:09.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.046+0000 7f3ebfa00640 1 -- 172.21.15.122:0/2075314456 shutdown_connections 2024-09-17T05:35:09.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.046+0000 7f3ebfa00640 1 -- 172.21.15.122:0/2075314456 >> 172.21.15.122:0/2075314456 conn(0x7f3eb80b2860 msgr2=0x7f3eb80b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:09.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.046+0000 7f3ebfa00640 1 -- 172.21.15.122:0/2075314456 shutdown_connections 2024-09-17T05:35:09.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.046+0000 7f3ebfa00640 1 -- 172.21.15.122:0/2075314456 wait complete. 2024-09-17T05:35:09.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.046+0000 7f3ebfa00640 1 Processor -- start 2024-09-17T05:35:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.046+0000 7f3ebfa00640 1 -- start start 2024-09-17T05:35:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebfa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea0004810 0x7f3eb81683e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebfa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb8145ca0 0x7f3eb81479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebfa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb8147f30 0x7f3eb815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebfa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb80ced30 con 0x7f3eb8145ca0 2024-09-17T05:35:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebfa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb815f6c0 con 0x7f3ea0004810 2024-09-17T05:35:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebfa00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb80743a0 con 0x7f3eb8147f30 2024-09-17T05:35:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebda00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb8147f30 0x7f3eb815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebda00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb8147f30 0x7f3eb815cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:33770/0 (socket says 172.21.15.122:33770) 2024-09-17T05:35:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebda00640 1 -- 172.21.15.122:0/3968142359 learned_addr learned my addr 172.21.15.122:0/3968142359 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3ebd000640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea0004810 0x7f3eb81683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.050+0000 7f3eb7e00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb8145ca0 0x7f3eb81479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb7e00640 1 -- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb8147f30 msgr2=0x7f3eb815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb7e00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb8147f30 0x7f3eb815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb7e00640 1 -- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea0004810 msgr2=0x7f3eb81683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb7e00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea0004810 0x7f3eb81683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb7e00640 1 -- 172.21.15.122:0/3968142359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb8150bd0 con 0x7f3eb8145ca0 2024-09-17T05:35:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb7e00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb8145ca0 0x7f3eb81479f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0004760 tx=0x7f3ea4005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb4c00640 1 -- 172.21.15.122:0/3968142359 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40ab040 con 0x7f3eb8145ca0 2024-09-17T05:35:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb4c00640 1 -- 172.21.15.122:0/3968142359 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ea40be070 con 0x7f3eb8145ca0 2024-09-17T05:35:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3eb4c00640 1 -- 172.21.15.122:0/3968142359 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4004930 con 0x7f3eb8145ca0 2024-09-17T05:35:09.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb8150eb0 con 0x7f3eb8145ca0 2024-09-17T05:35:09.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb81463c0 con 0x7f3eb8145ca0 2024-09-17T05:35:09.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.054+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb8070c50 con 0x7f3eb8145ca0 2024-09-17T05:35:09.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.058+0000 7f3eb4c00640 1 -- 172.21.15.122:0/3968142359 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f3ea40b6070 con 0x7f3eb8145ca0 2024-09-17T05:35:09.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.062+0000 7f3eb4c00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e98074790 0x7f3e98076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:09.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.062+0000 7f3eb4c00640 1 -- 172.21.15.122:0/3968142359 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f3ea412e720 con 0x7f3eb8145ca0 2024-09-17T05:35:09.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.062+0000 7f3ebd000640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e98074790 0x7f3e98076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:09.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.062+0000 7f3eb4c00640 1 -- 172.21.15.122:0/3968142359 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3ea4016a70 con 0x7f3eb8145ca0 2024-09-17T05:35:09.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.062+0000 7f3ebd000640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e98074790 0x7f3e98076c50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3eac005bd0 tx=0x7f3eac005aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:09.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:09.422+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone44559", "target": ["mon-mgr", ""]}) -- 0x7f3eb80cd200 con 0x7f3e98074790 2024-09-17T05:35:10.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.010+0000 7f3eb4c00640 1 -- 172.21.15.122:0/3968142359 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3eb80cd200 con 0x7f3e98074790 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e98074790 msgr2=0x7f3e98076c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e98074790 0x7f3e98076c50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3eac005bd0 tx=0x7f3eac005aa0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb8145ca0 msgr2=0x7f3eb81479f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb8145ca0 0x7f3eb81479f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0004760 tx=0x7f3ea4005700 comp rx=0 tx=0).stop 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 shutdown_connections 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3e98074790 0x7f3e98076c50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb8147f30 0x7f3eb815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb8145ca0 0x7f3eb81479f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 --2- 172.21.15.122:0/3968142359 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea0004810 0x7f3eb81683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 >> 172.21.15.122:0/3968142359 conn(0x7f3eb80b2860 msgr2=0x7f3eb8152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:10.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 shutdown_connections 2024-09-17T05:35:10.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.018+0000 7f3ebfa00640 1 -- 172.21.15.122:0/3968142359 wait complete. 2024-09-17T05:35:10.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone8110 2024-09-17T05:35:10.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 -- 172.21.15.122:0/2217693999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec15ef80 msgr2=0x7f71ec06cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 --2- 172.21.15.122:0/2217693999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec15ef80 0x7f71ec06cc20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009930 tx=0x7f71e001c8d0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 -- 172.21.15.122:0/2217693999 shutdown_connections 2024-09-17T05:35:10.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 --2- 172.21.15.122:0/2217693999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71ec0ccd80 0x7f71ec073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 --2- 172.21.15.122:0/2217693999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec15ef80 0x7f71ec06cc20 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 -- 172.21.15.122:0/2217693999 >> 172.21.15.122:0/2217693999 conn(0x7f71ec0b26f0 msgr2=0x7f71ec0b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:10.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 -- 172.21.15.122:0/2217693999 shutdown_connections 2024-09-17T05:35:10.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 -- 172.21.15.122:0/2217693999 wait complete. 2024-09-17T05:35:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 Processor -- start 2024-09-17T05:35:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.310+0000 7f71f2600640 1 -- start start 2024-09-17T05:35:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec06d160 0x7f71ec1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec0ccd80 0x7f71ec1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71ec15ef80 0x7f71ec1597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec168930 con 0x7f71ec06d160 2024-09-17T05:35:10.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec168ab0 con 0x7f71ec0ccd80 2024-09-17T05:35:10.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec168c30 con 0x7f71ec15ef80 2024-09-17T05:35:10.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec06d160 0x7f71ec1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec06d160 0x7f71ec1577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34356/0 (socket says 172.21.15.122:34356) 2024-09-17T05:35:10.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 -- 172.21.15.122:0/2385327694 learned_addr learned my addr 172.21.15.122:0/2385327694 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 -- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71ec15ef80 msgr2=0x7f71ec1597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71ec15ef80 0x7f71ec1597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 -- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec0ccd80 msgr2=0x7f71ec1560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec0ccd80 0x7f71ec1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 -- 172.21.15.122:0/2385327694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec0c4810 con 0x7f71ec06d160 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f1200640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec06d160 0x7f71ec1577a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f71e00b0040 tx=0x7f71e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71e8c00640 1 -- 172.21.15.122:0/2385327694 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e00b2040 con 0x7f71ec06d160 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71e8c00640 1 -- 172.21.15.122:0/2385327694 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e0003600 con 0x7f71ec06d160 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71e8c00640 1 -- 172.21.15.122:0/2385327694 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0003a50 con 0x7f71ec06d160 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec0c4af0 con 0x7f71ec06d160 2024-09-17T05:35:10.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec0c50b0 con 0x7f71ec06d160 2024-09-17T05:35:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.314+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec073af0 con 0x7f71ec06d160 2024-09-17T05:35:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.318+0000 7f71e8c00640 1 -- 172.21.15.122:0/2385327694 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f71e0003bf0 con 0x7f71ec06d160 2024-09-17T05:35:10.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.318+0000 7f71e8c00640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71cc074760 0x7f71cc076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:10.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.318+0000 7f71e8c00640 1 -- 172.21.15.122:0/2385327694 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f71e012cca0 con 0x7f71ec06d160 2024-09-17T05:35:10.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.322+0000 7f71e8c00640 1 -- 172.21.15.122:0/2385327694 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f71e00164f0 con 0x7f71ec06d160 2024-09-17T05:35:10.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.330+0000 7f71ebe00640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71cc074760 0x7f71cc076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:10.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.334+0000 7f71ebe00640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71cc074760 0x7f71cc076c20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0b00f0 tx=0x7f71d80079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:10.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:10.674+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone8110", "target": ["mon-mgr", ""]}) -- 0x7f71ec159260 con 0x7f71cc074760 2024-09-17T05:35:11.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.182+0000 7f71e8c00640 1 -- 172.21.15.122:0/2385327694 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71ec159260 con 0x7f71cc074760 2024-09-17T05:35:11.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71cc074760 msgr2=0x7f71cc076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71cc074760 0x7f71cc076c20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0b00f0 tx=0x7f71d80079c0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec06d160 msgr2=0x7f71ec1577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec06d160 0x7f71ec1577a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f71e00b0040 tx=0x7f71e0004040 comp rx=0 tx=0).stop 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 shutdown_connections 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71cc074760 0x7f71cc076c20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71ec15ef80 0x7f71ec1597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71ec0ccd80 0x7f71ec1560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 --2- 172.21.15.122:0/2385327694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71ec06d160 0x7f71ec1577a0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 >> 172.21.15.122:0/2385327694 conn(0x7f71ec0b26f0 msgr2=0x7f71ec0b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 shutdown_connections 2024-09-17T05:35:11.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.186+0000 7f71f2600640 1 -- 172.21.15.122:0/2385327694 wait complete. 2024-09-17T05:35:11.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone610611 2024-09-17T05:35:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.458+0000 7ff108e00640 1 -- 172.21.15.122:0/61616946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1041456b0 msgr2=0x7ff10415ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.458+0000 7ff108e00640 1 --2- 172.21.15.122:0/61616946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1041456b0 0x7ff10415ccf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009840 tx=0x7ff0f001c760 comp rx=0 tx=0).stop 2024-09-17T05:35:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.458+0000 7ff108e00640 1 -- 172.21.15.122:0/61616946 shutdown_connections 2024-09-17T05:35:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.458+0000 7ff108e00640 1 --2- 172.21.15.122:0/61616946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1041456b0 0x7ff10415ccf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.458+0000 7ff108e00640 1 --2- 172.21.15.122:0/61616946 >> v2:172.21.15.173:3300/0 conn(0x7ff10406cc20 0x7ff1040d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.458+0000 7ff108e00640 1 -- 172.21.15.122:0/61616946 >> 172.21.15.122:0/61616946 conn(0x7ff1040d0b30 msgr2=0x7ff1040d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:11.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.458+0000 7ff108e00640 1 -- 172.21.15.122:0/61616946 shutdown_connections 2024-09-17T05:35:11.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.462+0000 7ff108e00640 1 -- 172.21.15.122:0/61616946 wait complete. 2024-09-17T05:35:11.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.462+0000 7ff108e00640 1 Processor -- start 2024-09-17T05:35:11.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.462+0000 7ff108e00640 1 -- start start 2024-09-17T05:35:11.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff108e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10406cc20 0x7ff1040c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff108e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10406d000 0x7ff1040c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff108e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1041456b0 0x7ff1042032c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff108e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff104168e00 con 0x7ff10406d000 2024-09-17T05:35:11.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff108e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff104168c80 con 0x7ff10406cc20 2024-09-17T05:35:11.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff108e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff104168f80 con 0x7ff1041456b0 2024-09-17T05:35:11.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10406cc20 0x7ff1040c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10406cc20 0x7ff1040c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:37946/0 (socket says 172.21.15.122:37946) 2024-09-17T05:35:11.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 -- 172.21.15.122:0/4220446846 learned_addr learned my addr 172.21.15.122:0/4220446846 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:11.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103e00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1041456b0 0x7ff1042032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 -- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1041456b0 msgr2=0x7ff1042032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1041456b0 0x7ff1042032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 -- 172.21.15.122:0/4220446846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10406d000 msgr2=0x7ff1040c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:11.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10406d000 0x7ff1040c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:11.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 -- 172.21.15.122:0/4220446846 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1040ca4d0 con 0x7ff10406cc20 2024-09-17T05:35:11.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.466+0000 7ff103400640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10406cc20 0x7ff1040c9b80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400e9f0 tx=0x7ff0f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:11.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.470+0000 7ff0fb400640 1 -- 172.21.15.122:0/4220446846 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400cd20 con 0x7ff10406cc20 2024-09-17T05:35:11.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.470+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1040ca7b0 con 0x7ff10406cc20 2024-09-17T05:35:11.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.470+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff104159ef0 con 0x7ff10406cc20 2024-09-17T05:35:11.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.470+0000 7ff0fb400640 1 -- 172.21.15.122:0/4220446846 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0f4010430 con 0x7ff10406cc20 2024-09-17T05:35:11.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.470+0000 7ff0fb400640 1 -- 172.21.15.122:0/4220446846 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4010800 con 0x7ff10406cc20 2024-09-17T05:35:11.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.470+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc0052a0 con 0x7ff10406cc20 2024-09-17T05:35:11.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.478+0000 7ff0fb400640 1 -- 172.21.15.122:0/4220446846 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7ff0f40109a0 con 0x7ff10406cc20 2024-09-17T05:35:11.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.478+0000 7ff0fb400640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0e4074790 0x7ff0e4076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:11.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.478+0000 7ff0fb400640 1 -- 172.21.15.122:0/4220446846 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff0f4094060 con 0x7ff10406cc20 2024-09-17T05:35:11.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.478+0000 7ff102a00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0e4074790 0x7ff0e4076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.482+0000 7ff102a00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0e4074790 0x7ff0e4076c50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00af000 tx=0x7ff0f0019500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:11.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.486+0000 7ff0fb400640 1 -- 172.21.15.122:0/4220446846 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff0f4060510 con 0x7ff10406cc20 2024-09-17T05:35:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:11.854+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone610611", "target": ["mon-mgr", ""]}) -- 0x7ff0dc000c90 con 0x7ff0e4074790 2024-09-17T05:35:12.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.402+0000 7ff0fb400640 1 -- 172.21.15.122:0/4220446846 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff0dc000c90 con 0x7ff0e4074790 2024-09-17T05:35:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0e4074790 msgr2=0x7ff0e4076c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0e4074790 0x7ff0e4076c50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00af000 tx=0x7ff0f0019500 comp rx=0 tx=0).stop 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10406cc20 msgr2=0x7ff1040c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10406cc20 0x7ff1040c9b80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400e9f0 tx=0x7ff0f400eeb0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 shutdown_connections 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0e4074790 0x7ff0e4076c50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1041456b0 0x7ff1042032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10406d000 0x7ff1040c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 --2- 172.21.15.122:0/4220446846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff10406cc20 0x7ff1040c9b80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 >> 172.21.15.122:0/4220446846 conn(0x7ff1040d0b30 msgr2=0x7ff1040737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:12.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 shutdown_connections 2024-09-17T05:35:12.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.406+0000 7ff108e00640 1 -- 172.21.15.122:0/4220446846 wait complete. 2024-09-17T05:35:12.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone82612 2024-09-17T05:35:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.758+0000 7fd3d9800640 1 -- 172.21.15.122:0/3439834627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 msgr2=0x7fd3b8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.758+0000 7fd3d9800640 1 --2- 172.21.15.122:0/3439834627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 0x7fd3b8003e60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00c7a0 tx=0x7fd3cc01d290 comp rx=0 tx=0).stop 2024-09-17T05:35:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.758+0000 7fd3d3400640 1 -- 172.21.15.122:0/3439834627 reap_dead start 2024-09-17T05:35:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.758+0000 7fd3d9800640 1 -- 172.21.15.122:0/3439834627 shutdown_connections 2024-09-17T05:35:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.758+0000 7fd3d9800640 1 -- 172.21.15.122:0/3439834627 >> 172.21.15.122:0/3439834627 conn(0x7fd3d40cd740 msgr2=0x7fd3d40cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:12.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.758+0000 7fd3d9800640 1 -- 172.21.15.122:0/3439834627 shutdown_connections 2024-09-17T05:35:12.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.758+0000 7fd3d9800640 1 -- 172.21.15.122:0/3439834627 wait complete. 2024-09-17T05:35:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.762+0000 7fd3d9800640 1 Processor -- start 2024-09-17T05:35:12.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.762+0000 7fd3d9800640 1 -- start start 2024-09-17T05:35:12.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 0x7fd3d415dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d9800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d415bc80 0x7fd3d40afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d9800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40b4d60 0x7fd3d40b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d9800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d4166600 con 0x7fd3b8003a40 2024-09-17T05:35:12.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d9800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d4166480 con 0x7fd3d40b4d60 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 0x7fd3d415dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 0x7fd3d415dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53912/0 (socket says 172.21.15.122:53912) 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 -- 172.21.15.122:0/3360174270 learned_addr learned my addr 172.21.15.122:0/3360174270 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d9800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d4166780 con 0x7fd3d415bc80 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 -- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d415bc80 msgr2=0x7fd3d40afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d415bc80 0x7fd3d40afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 -- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40b4d60 msgr2=0x7fd3d40b51e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40b4d60 0x7fd3d40b51e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 -- 172.21.15.122:0/3360174270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d415c3f0 con 0x7fd3b8003a40 2024-09-17T05:35:12.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d2a00640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 0x7fd3d415dc40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc01b990 tx=0x7fd3cc00c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:12.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3caa00640 1 -- 172.21.15.122:0/3360174270 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc003980 con 0x7fd3b8003a40 2024-09-17T05:35:12.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3caa00640 1 -- 172.21.15.122:0/3360174270 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3cc003b20 con 0x7fd3b8003a40 2024-09-17T05:35:12.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3caa00640 1 -- 172.21.15.122:0/3360174270 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc0060d0 con 0x7fd3b8003a40 2024-09-17T05:35:12.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d415c6d0 con 0x7fd3b8003a40 2024-09-17T05:35:12.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.766+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d415cc90 con 0x7fd3b8003a40 2024-09-17T05:35:12.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.770+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d40b2740 con 0x7fd3b8003a40 2024-09-17T05:35:12.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.770+0000 7fd3caa00640 1 -- 172.21.15.122:0/3360174270 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fd3cc0b8070 con 0x7fd3b8003a40 2024-09-17T05:35:12.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.774+0000 7fd3caa00640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3b4074790 0x7fd3b4076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:12.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.774+0000 7fd3caa00640 1 -- 172.21.15.122:0/3360174270 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fd3cc12d710 con 0x7fd3b8003a40 2024-09-17T05:35:12.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.778+0000 7fd3caa00640 1 -- 172.21.15.122:0/3360174270 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd3d40cf6e0 con 0x7fd3b8003a40 2024-09-17T05:35:12.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.782+0000 7fd3d2000640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3b4074790 0x7fd3b4076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:12.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:12.786+0000 7fd3d2000640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3b4074790 0x7fd3b4076c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0007fd0 tx=0x7fd3c00079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:13.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.094+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone82612", "target": ["mon-mgr", ""]}) -- 0x7fd3d40c4590 con 0x7fd3b4074790 2024-09-17T05:35:13.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.986+0000 7fd3caa00640 1 -- 172.21.15.122:0/3360174270 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd3d40c4590 con 0x7fd3b4074790 2024-09-17T05:35:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3b4074790 msgr2=0x7fd3b4076c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3b4074790 0x7fd3b4076c50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0007fd0 tx=0x7fd3c00079c0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 msgr2=0x7fd3d415dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:13.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 0x7fd3d415dc40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc01b990 tx=0x7fd3cc00c3c0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 shutdown_connections 2024-09-17T05:35:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3b4074790 0x7fd3b4076c50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40b4d60 0x7fd3d40b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d415bc80 0x7fd3d40afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 --2- 172.21.15.122:0/3360174270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8003a40 0x7fd3d415dc40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 >> 172.21.15.122:0/3360174270 conn(0x7fd3d40cd740 msgr2=0x7fd3d4159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:13.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 shutdown_connections 2024-09-17T05:35:13.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:13.994+0000 7fd3d9800640 1 -- 172.21.15.122:0/3360174270 wait complete. 2024-09-17T05:35:14.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone281413 2024-09-17T05:35:14.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.334+0000 7f782c800640 1 -- 172.21.15.122:0/513319391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f782415ac70 msgr2=0x7f78240cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:14.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.334+0000 7f782c800640 1 --2- 172.21.15.122:0/513319391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f782415ac70 0x7f78240cf2a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7818009900 tx=0x7f781801c820 comp rx=0 tx=0).stop 2024-09-17T05:35:14.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.334+0000 7f782c800640 1 -- 172.21.15.122:0/513319391 shutdown_connections 2024-09-17T05:35:14.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.334+0000 7f782c800640 1 --2- 172.21.15.122:0/513319391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f782415ac70 0x7f78240cf2a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.334+0000 7f782c800640 1 -- 172.21.15.122:0/513319391 >> 172.21.15.122:0/513319391 conn(0x7f78240cd740 msgr2=0x7f78240cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:14.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.338+0000 7f782c800640 1 -- 172.21.15.122:0/513319391 shutdown_connections 2024-09-17T05:35:14.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.338+0000 7f782c800640 1 -- 172.21.15.122:0/513319391 wait complete. 2024-09-17T05:35:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.342+0000 7f782c800640 1 Processor -- start 2024-09-17T05:35:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.342+0000 7f782c800640 1 -- start start 2024-09-17T05:35:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f782c800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78240b2e50 0x7f78240b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:14.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f782c800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78240cbdb0 0x7f78240c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:14.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f782c800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782415ac70 0x7f78240c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f782c800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7824169080 con 0x7f78240cbdb0 2024-09-17T05:35:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f7829400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78240cbdb0 0x7f78240c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f7829400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78240cbdb0 0x7f78240c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53914/0 (socket says 172.21.15.122:53914) 2024-09-17T05:35:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f7829400640 1 -- 172.21.15.122:0/2677995288 learned_addr learned my addr 172.21.15.122:0/2677995288 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f782c800640 1 -- 172.21.15.122:0/2677995288 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7824168f00 con 0x7f78240b2e50 2024-09-17T05:35:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f782c800640 1 -- 172.21.15.122:0/2677995288 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7824169200 con 0x7f782415ac70 2024-09-17T05:35:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f7829400640 1 -- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782415ac70 msgr2=0x7f78240c6a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:35:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f7829400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782415ac70 0x7f78240c6a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f7829400640 1 -- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78240b2e50 msgr2=0x7f78240b5550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:35:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f7829400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78240b2e50 0x7f78240b5550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.346+0000 7f7829400640 1 -- 172.21.15.122:0/2677995288 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78240c62f0 con 0x7f78240cbdb0 2024-09-17T05:35:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.350+0000 7f7829400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78240cbdb0 0x7f78240c5db0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f781400c450 tx=0x7f781400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.354+0000 7f781e000640 1 -- 172.21.15.122:0/2677995288 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814012830 con 0x7f78240cbdb0 2024-09-17T05:35:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.354+0000 7f781e000640 1 -- 172.21.15.122:0/2677995288 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7814017070 con 0x7f78240cbdb0 2024-09-17T05:35:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.354+0000 7f781e000640 1 -- 172.21.15.122:0/2677995288 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814012d60 con 0x7f78240cbdb0 2024-09-17T05:35:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.354+0000 7f782c800640 1 -- 172.21.15.122:0/2677995288 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78241fc0a0 con 0x7f78240cbdb0 2024-09-17T05:35:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.354+0000 7f782c800640 1 -- 172.21.15.122:0/2677995288 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78241fc660 con 0x7f78240cbdb0 2024-09-17T05:35:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.358+0000 7f7813400640 1 -- 172.21.15.122:0/2677995288 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78240cf2a0 con 0x7f78240cbdb0 2024-09-17T05:35:14.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.362+0000 7f781e000640 1 -- 172.21.15.122:0/2677995288 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f781401c460 con 0x7f78240cbdb0 2024-09-17T05:35:14.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.366+0000 7f781e000640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78080747b0 0x7f7808076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:14.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.366+0000 7f781e000640 1 -- 172.21.15.122:0/2677995288 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f78140937f0 con 0x7f78240cbdb0 2024-09-17T05:35:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.370+0000 7f781e000640 1 -- 172.21.15.122:0/2677995288 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f78140602f0 con 0x7f78240cbdb0 2024-09-17T05:35:14.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.398+0000 7f7829e00640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78080747b0 0x7f7808076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:14.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.426+0000 7f7829e00640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78080747b0 0x7f7808076c70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f78180af000 tx=0x7f781801b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:14.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:14.722+0000 7f7813400640 1 -- 172.21.15.122:0/2677995288 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone281413", "target": ["mon-mgr", ""]}) -- 0x7f7824159900 con 0x7f78080747b0 2024-09-17T05:35:15.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.546+0000 7f781e000640 1 -- 172.21.15.122:0/2677995288 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7824159900 con 0x7f78080747b0 2024-09-17T05:35:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 -- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78080747b0 msgr2=0x7f7808076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78080747b0 0x7f7808076c70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f78180af000 tx=0x7f781801b060 comp rx=0 tx=0).stop 2024-09-17T05:35:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 -- 172.21.15.122:0/2677995288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78240cbdb0 msgr2=0x7f78240c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78240cbdb0 0x7f78240c5db0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f781400c450 tx=0x7f781400c910 comp rx=0 tx=0).stop 2024-09-17T05:35:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 -- 172.21.15.122:0/2677995288 shutdown_connections 2024-09-17T05:35:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78080747b0 0x7f7808076c70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782415ac70 0x7f78240c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78240cbdb0 0x7f78240c5db0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 --2- 172.21.15.122:0/2677995288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78240b2e50 0x7f78240b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 -- 172.21.15.122:0/2677995288 >> 172.21.15.122:0/2677995288 conn(0x7f78240cd740 msgr2=0x7f782406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:15.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 -- 172.21.15.122:0/2677995288 shutdown_connections 2024-09-17T05:35:15.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.550+0000 7f7813400640 1 -- 172.21.15.122:0/2677995288 wait complete. 2024-09-17T05:35:15.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone479614 2024-09-17T05:35:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.858+0000 7fdaa9c00640 1 -- 172.21.15.122:0/3155196214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 msgr2=0x7fda88004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.858+0000 7fdaa9c00640 1 --2- 172.21.15.122:0/3155196214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 0x7fda88004c90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fda90007570 tx=0x7fda9001d630 comp rx=0 tx=0).stop 2024-09-17T05:35:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.858+0000 7fdaa9c00640 1 -- 172.21.15.122:0/3155196214 shutdown_connections 2024-09-17T05:35:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.858+0000 7fdaa9c00640 1 --2- 172.21.15.122:0/3155196214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 0x7fda88004c90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.858+0000 7fdaa9c00640 1 -- 172.21.15.122:0/3155196214 >> 172.21.15.122:0/3155196214 conn(0x7fdaa40b18f0 msgr2=0x7fdaa40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.858+0000 7fdaa9c00640 1 -- 172.21.15.122:0/3155196214 shutdown_connections 2024-09-17T05:35:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.858+0000 7fdaa9c00640 1 -- 172.21.15.122:0/3155196214 wait complete. 2024-09-17T05:35:15.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa9c00640 1 Processor -- start 2024-09-17T05:35:15.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa9c00640 1 -- start start 2024-09-17T05:35:15.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 0x7fdaa41ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:15.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa9c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa41ffe30 0x7fdaa409ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:15.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa9c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdaa409b180 0x7fdaa409b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:15.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa9c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa406ef10 con 0x7fda88004810 2024-09-17T05:35:15.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa9c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa406ed90 con 0x7fdaa41ffe30 2024-09-17T05:35:15.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa9c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa406f090 con 0x7fdaa409b180 2024-09-17T05:35:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 0x7fdaa41ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 0x7fdaa41ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53934/0 (socket says 172.21.15.122:53934) 2024-09-17T05:35:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.862+0000 7fdaa2a00640 1 -- 172.21.15.122:0/1777716549 learned_addr learned my addr 172.21.15.122:0/1777716549 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa2a00640 1 -- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdaa409b180 msgr2=0x7fdaa409b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa2a00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdaa409b180 0x7fdaa409b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa2a00640 1 -- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa41ffe30 msgr2=0x7fdaa409ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa2a00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa41ffe30 0x7fdaa409ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa2a00640 1 -- 172.21.15.122:0/1777716549 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa409bd90 con 0x7fda88004810 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa2a00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 0x7fdaa41ff8f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fda9400b580 tx=0x7fda9400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fda9aa00640 1 -- 172.21.15.122:0/1777716549 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9400f850 con 0x7fda88004810 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fda9aa00640 1 -- 172.21.15.122:0/1777716549 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda9400fae0 con 0x7fda88004810 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fda9aa00640 1 -- 172.21.15.122:0/1777716549 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94017400 con 0x7fda88004810 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa409c070 con 0x7fda88004810 2024-09-17T05:35:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa40ad150 con 0x7fda88004810 2024-09-17T05:35:15.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.866+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa4069410 con 0x7fda88004810 2024-09-17T05:35:15.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.870+0000 7fda9aa00640 1 -- 172.21.15.122:0/1777716549 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fda940175a0 con 0x7fda88004810 2024-09-17T05:35:15.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.870+0000 7fda9aa00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fda840747e0 0x7fda84076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:15.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.870+0000 7fda9aa00640 1 -- 172.21.15.122:0/1777716549 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fda94093400 con 0x7fda88004810 2024-09-17T05:35:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.878+0000 7fda9aa00640 1 -- 172.21.15.122:0/1777716549 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fda94013030 con 0x7fda88004810 2024-09-17T05:35:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.882+0000 7fdaa2000640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fda840747e0 0x7fda84076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:15.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:15.886+0000 7fdaa2000640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fda840747e0 0x7fda84076ca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fda900b1000 tx=0x7fda88005840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:16.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:16.258+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone479614", "target": ["mon-mgr", ""]}) -- 0x7fdaa40701e0 con 0x7fda840747e0 2024-09-17T05:35:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.606+0000 7fda9aa00640 1 -- 172.21.15.122:0/1777716549 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdaa40701e0 con 0x7fda840747e0 2024-09-17T05:35:17.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fda840747e0 msgr2=0x7fda84076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fda840747e0 0x7fda84076ca0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fda900b1000 tx=0x7fda88005840 comp rx=0 tx=0).stop 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 msgr2=0x7fdaa41ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 0x7fdaa41ff8f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fda9400b580 tx=0x7fda9400ba40 comp rx=0 tx=0).stop 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 shutdown_connections 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fda840747e0 0x7fda84076ca0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdaa409b180 0x7fdaa409b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdaa41ffe30 0x7fdaa409ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 --2- 172.21.15.122:0/1777716549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda88004810 0x7fdaa41ff8f0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 >> 172.21.15.122:0/1777716549 conn(0x7fdaa40b18f0 msgr2=0x7fdaa415c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 shutdown_connections 2024-09-17T05:35:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.614+0000 7fdaa9c00640 1 -- 172.21.15.122:0/1777716549 wait complete. 2024-09-17T05:35:17.644 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone321815 2024-09-17T05:35:17.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.882+0000 7f89cc800640 1 -- 172.21.15.122:0/360223567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b0004810 msgr2=0x7f89b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:17.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.882+0000 7f89cc800640 1 --2- 172.21.15.122:0/360223567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b0004810 0x7f89b0004c90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c4a0 tx=0x7f89c001cf60 comp rx=0 tx=0).stop 2024-09-17T05:35:17.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.882+0000 7f89cc800640 1 -- 172.21.15.122:0/360223567 shutdown_connections 2024-09-17T05:35:17.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.882+0000 7f89cc800640 1 --2- 172.21.15.122:0/360223567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b0004810 0x7f89b0004c90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.882+0000 7f89cc800640 1 -- 172.21.15.122:0/360223567 >> 172.21.15.122:0/360223567 conn(0x7f89c40cd740 msgr2=0x7f89c40cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:17.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.882+0000 7f89cc800640 1 -- 172.21.15.122:0/360223567 shutdown_connections 2024-09-17T05:35:17.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.882+0000 7f89cc800640 1 -- 172.21.15.122:0/360223567 wait complete. 2024-09-17T05:35:17.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.886+0000 7f89cc800640 1 Processor -- start 2024-09-17T05:35:17.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.886+0000 7f89cc800640 1 -- start start 2024-09-17T05:35:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89cc800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0004810 0x7f89c41f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89cc800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89c40c7610 0x7f89c40b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89cc800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c40b5080 0x7f89c40b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89cc800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89c415f1e0 con 0x7f89c40b5080 2024-09-17T05:35:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89cc800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89c415f060 con 0x7f89b0004810 2024-09-17T05:35:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89cc800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f89c40d0060 con 0x7f89c40c7610 2024-09-17T05:35:17.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89c9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0004810 0x7f89c41f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:17.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89ca800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c40b5080 0x7f89c40b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:17.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89c9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0004810 0x7f89c41f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:38010/0 (socket says 172.21.15.122:38010) 2024-09-17T05:35:17.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.890+0000 7f89c9e00640 1 -- 172.21.15.122:0/1775549433 learned_addr learned my addr 172.21.15.122:0/1775549433 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:17.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.894+0000 7f89c9400640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89c40c7610 0x7f89c40b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:17.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.894+0000 7f89ca800640 1 -- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89c40c7610 msgr2=0x7f89c40b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:17.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.894+0000 7f89ca800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89c40c7610 0x7f89c40b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.894+0000 7f89ca800640 1 -- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0004810 msgr2=0x7f89c41f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:17.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.894+0000 7f89ca800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0004810 0x7f89c41f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:17.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.894+0000 7f89ca800640 1 -- 172.21.15.122:0/1775549433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c40c7a90 con 0x7f89c40b5080 2024-09-17T05:35:17.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.894+0000 7f89c9e00640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0004810 0x7f89c41f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:35:17.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.898+0000 7f89ca800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c40b5080 0x7f89c40b5500 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f89c40b2740 tx=0x7f89b4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:17.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.902+0000 7f89ba000640 1 -- 172.21.15.122:0/1775549433 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4011600 con 0x7f89c40b5080 2024-09-17T05:35:17.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.902+0000 7f89ba000640 1 -- 172.21.15.122:0/1775549433 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89b40117a0 con 0x7f89c40b5080 2024-09-17T05:35:17.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.902+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c40c6c60 con 0x7f89c40b5080 2024-09-17T05:35:17.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.902+0000 7f89ba000640 1 -- 172.21.15.122:0/1775549433 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4011bf0 con 0x7f89c40b5080 2024-09-17T05:35:17.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.902+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c40c7d90 con 0x7f89c40b5080 2024-09-17T05:35:17.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.902+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c406cc20 con 0x7f89c40b5080 2024-09-17T05:35:17.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.906+0000 7f89ba000640 1 -- 172.21.15.122:0/1775549433 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f89b401a400 con 0x7f89c40b5080 2024-09-17T05:35:17.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.906+0000 7f89ba000640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89a8074760 0x7f89a8076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:17.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.910+0000 7f89ba000640 1 -- 172.21.15.122:0/1775549433 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f89b405a460 con 0x7f89c40b5080 2024-09-17T05:35:17.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.914+0000 7f89ba000640 1 -- 172.21.15.122:0/1775549433 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f89b405e8f0 con 0x7f89c40b5080 2024-09-17T05:35:17.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.926+0000 7f89c9e00640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89a8074760 0x7f89a8076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:17.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:17.942+0000 7f89c9e00640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89a8074760 0x7f89a8076c20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f89c40c7220 tx=0x7f89bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:18.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:18.270+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone321815", "target": ["mon-mgr", ""]}) -- 0x7f89c40b6c40 con 0x7f89a8074760 2024-09-17T05:35:19.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.266+0000 7f89ba000640 1 -- 172.21.15.122:0/1775549433 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f89c40b6c40 con 0x7f89a8074760 2024-09-17T05:35:19.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89a8074760 msgr2=0x7f89a8076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:19.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89a8074760 0x7f89a8076c20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f89c40c7220 tx=0x7f89bc009040 comp rx=0 tx=0).stop 2024-09-17T05:35:19.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c40b5080 msgr2=0x7f89c40b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:19.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c40b5080 0x7f89c40b5500 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f89c40b2740 tx=0x7f89b4009670 comp rx=0 tx=0).stop 2024-09-17T05:35:19.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 shutdown_connections 2024-09-17T05:35:19.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89a8074760 0x7f89a8076c20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:19.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c40b5080 0x7f89c40b5500 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:19.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89c40c7610 0x7f89c40b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:19.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 --2- 172.21.15.122:0/1775549433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0004810 0x7f89c41f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:19.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 >> 172.21.15.122:0/1775549433 conn(0x7f89c40cd740 msgr2=0x7f89c40ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:19.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 shutdown_connections 2024-09-17T05:35:19.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.274+0000 7f89cc800640 1 -- 172.21.15.122:0/1775549433 wait complete. 2024-09-17T05:35:19.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone125516 2024-09-17T05:35:19.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.566+0000 7fd641c00640 1 -- 172.21.15.122:0/1077988122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c06e810 msgr2=0x7fd63c06f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:19.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.566+0000 7fd641c00640 1 --2- 172.21.15.122:0/1077988122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c06e810 0x7fd63c06f3c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0cc770 tx=0x7fd63401cba0 comp rx=0 tx=0).stop 2024-09-17T05:35:19.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.566+0000 7fd641c00640 1 -- 172.21.15.122:0/1077988122 shutdown_connections 2024-09-17T05:35:19.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.566+0000 7fd641c00640 1 --2- 172.21.15.122:0/1077988122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c06e810 0x7fd63c06f3c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:19.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.566+0000 7fd641c00640 1 -- 172.21.15.122:0/1077988122 >> 172.21.15.122:0/1077988122 conn(0x7fd63c0b18f0 msgr2=0x7fd63c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:19.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.570+0000 7fd641c00640 1 -- 172.21.15.122:0/1077988122 shutdown_connections 2024-09-17T05:35:19.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.570+0000 7fd641c00640 1 -- 172.21.15.122:0/1077988122 wait complete. 2024-09-17T05:35:19.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.570+0000 7fd641c00640 1 Processor -- start 2024-09-17T05:35:19.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.570+0000 7fd641c00640 1 -- start start 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd641c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd63c06e810 0x7fd63c09b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd641c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c0cf300 0x7fd63c073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd641c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd63c15ade0 0x7fd63c14c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd641c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c0ce1a0 con 0x7fd63c0cf300 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd641c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c0ce020 con 0x7fd63c15ade0 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd641c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c0ce320 con 0x7fd63c06e810 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c0cf300 0x7fd63c073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c0cf300 0x7fd63c073b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53976/0 (socket says 172.21.15.122:53976) 2024-09-17T05:35:19.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 -- 172.21.15.122:0/1578439450 learned_addr learned my addr 172.21.15.122:0/1578439450 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:19.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 -- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd63c06e810 msgr2=0x7fd63c09b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd63c06e810 0x7fd63c09b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 -- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd63c15ade0 msgr2=0x7fd63c14c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd63c15ade0 0x7fd63c14c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 -- 172.21.15.122:0/1578439450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c14f4f0 con 0x7fd63c0cf300 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd63a000640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c0cf300 0x7fd63c073b20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c4c0 tx=0x7fd628001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd632a00640 1 -- 172.21.15.122:0/1578439450 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628017070 con 0x7fd63c0cf300 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd632a00640 1 -- 172.21.15.122:0/1578439450 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd628003600 con 0x7fd63c0cf300 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c09aab0 con 0x7fd63c0cf300 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd632a00640 1 -- 172.21.15.122:0/1578439450 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628003a50 con 0x7fd63c0cf300 2024-09-17T05:35:19.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.574+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c09b070 con 0x7fd63c0cf300 2024-09-17T05:35:19.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.578+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6140052a0 con 0x7fd63c0cf300 2024-09-17T05:35:19.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.578+0000 7fd632a00640 1 -- 172.21.15.122:0/1578439450 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fd62801a070 con 0x7fd63c0cf300 2024-09-17T05:35:19.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.578+0000 7fd632a00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd61c074760 0x7fd61c076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:19.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.578+0000 7fd632a00640 1 -- 172.21.15.122:0/1578439450 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fd628092620 con 0x7fd63c0cf300 2024-09-17T05:35:19.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.590+0000 7fd632a00640 1 -- 172.21.15.122:0/1578439450 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd62805ed80 con 0x7fd63c0cf300 2024-09-17T05:35:19.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.614+0000 7fd63aa00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd61c074760 0x7fd61c076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:19.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:19.682+0000 7fd63aa00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd61c074760 0x7fd61c076c20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0046c0 tx=0x7fd62c007910 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:20.010+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone125516", "target": ["mon-mgr", ""]}) -- 0x7fd614000c90 con 0x7fd61c074760 2024-09-17T05:35:21.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.470+0000 7fd632a00640 1 -- 172.21.15.122:0/1578439450 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd614000c90 con 0x7fd61c074760 2024-09-17T05:35:21.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd61c074760 msgr2=0x7fd61c076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd61c074760 0x7fd61c076c20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0046c0 tx=0x7fd62c007910 comp rx=0 tx=0).stop 2024-09-17T05:35:21.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c0cf300 msgr2=0x7fd63c073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c0cf300 0x7fd63c073b20 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c4c0 tx=0x7fd628001e80 comp rx=0 tx=0).stop 2024-09-17T05:35:21.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 shutdown_connections 2024-09-17T05:35:21.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd61c074760 0x7fd61c076c20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd63c15ade0 0x7fd63c14c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd63c0cf300 0x7fd63c073b20 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 --2- 172.21.15.122:0/1578439450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd63c06e810 0x7fd63c09b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 >> 172.21.15.122:0/1578439450 conn(0x7fd63c0b18f0 msgr2=0x7fd63c158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 shutdown_connections 2024-09-17T05:35:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.474+0000 7fd641c00640 1 -- 172.21.15.122:0/1578439450 wait complete. 2024-09-17T05:35:21.531 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone523517 2024-09-17T05:35:21.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.830+0000 7f1bf9200640 1 -- 172.21.15.122:0/158215298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd8004030 msgr2=0x7f1bd8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.830+0000 7f1bf9200640 1 --2- 172.21.15.122:0/158215298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd8004030 0x7f1bd8004810 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009b60 tx=0x7f1be001d2b0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.830+0000 7f1bf3e00640 1 -- 172.21.15.122:0/158215298 reap_dead start 2024-09-17T05:35:21.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.834+0000 7f1bf9200640 1 -- 172.21.15.122:0/158215298 shutdown_connections 2024-09-17T05:35:21.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.834+0000 7f1bf9200640 1 -- 172.21.15.122:0/158215298 >> 172.21.15.122:0/158215298 conn(0x7f1bf40d4210 msgr2=0x7f1bf40d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:21.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.834+0000 7f1bf9200640 1 -- 172.21.15.122:0/158215298 shutdown_connections 2024-09-17T05:35:21.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.834+0000 7f1bf9200640 1 -- 172.21.15.122:0/158215298 wait complete. 2024-09-17T05:35:21.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf9200640 1 Processor -- start 2024-09-17T05:35:21.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf9200640 1 -- start start 2024-09-17T05:35:21.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bd8004030 0x7f1bf40c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:21.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf41fb650 0x7f1bf40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:21.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf9200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bf41fbab0 0x7f1bf40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:21.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf9200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4169240 con 0x7f1bf41fb650 2024-09-17T05:35:21.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf9200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41690c0 con 0x7f1bd8004030 2024-09-17T05:35:21.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf9200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1bf41693c0 con 0x7f1bf41fbab0 2024-09-17T05:35:21.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf41fb650 0x7f1bf40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:21.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf41fb650 0x7f1bf40b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40920/0 (socket says 172.21.15.122:40920) 2024-09-17T05:35:21.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 -- 172.21.15.122:0/904144941 learned_addr learned my addr 172.21.15.122:0/904144941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:21.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf3400640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bd8004030 0x7f1bf40c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:21.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 -- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bf41fbab0 msgr2=0x7f1bf40b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:35:21.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bf41fbab0 0x7f1bf40b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 -- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bd8004030 msgr2=0x7f1bf40c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:21.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bd8004030 0x7f1bf40c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:21.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 -- 172.21.15.122:0/904144941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf4204e50 con 0x7f1bf41fb650 2024-09-17T05:35:21.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.838+0000 7f1bf2a00640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf41fb650 0x7f1bf40b47b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1be001cd20 tx=0x7f1be00045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:21.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.842+0000 7f1beb400640 1 -- 172.21.15.122:0/904144941 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be00b1040 con 0x7f1bf41fb650 2024-09-17T05:35:21.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.842+0000 7f1beb400640 1 -- 172.21.15.122:0/904144941 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1be00bc070 con 0x7f1bf41fb650 2024-09-17T05:35:21.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.842+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf40b4570 con 0x7f1bf41fb650 2024-09-17T05:35:21.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.842+0000 7f1beb400640 1 -- 172.21.15.122:0/904144941 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0009730 con 0x7f1bf41fb650 2024-09-17T05:35:21.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.842+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf42053d0 con 0x7f1bf41fb650 2024-09-17T05:35:21.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.842+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf40b5890 con 0x7f1bf41fb650 2024-09-17T05:35:21.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.846+0000 7f1beb400640 1 -- 172.21.15.122:0/904144941 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f1be00ac4c0 con 0x7f1bf41fb650 2024-09-17T05:35:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.846+0000 7f1beb400640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1bd4074760 0x7f1bd4076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:21.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.846+0000 7f1beb400640 1 -- 172.21.15.122:0/904144941 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f1be012eea0 con 0x7f1bf41fb650 2024-09-17T05:35:21.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.850+0000 7f1beb400640 1 -- 172.21.15.122:0/904144941 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1be0016fc0 con 0x7f1bf41fb650 2024-09-17T05:35:21.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.882+0000 7f1bf3400640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1bd4074760 0x7f1bd4076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:21.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:21.990+0000 7f1bf3400640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1bd4074760 0x7f1bd4076c20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1be4006fd0 tx=0x7f1be4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:22.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:22.194+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone523517", "target": ["mon-mgr", ""]}) -- 0x7f1bf40c7320 con 0x7f1bd4074760 2024-09-17T05:35:23.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.350+0000 7f1beb400640 1 -- 172.21.15.122:0/904144941 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1bf40c7320 con 0x7f1bd4074760 2024-09-17T05:35:23.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.366+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1bd4074760 msgr2=0x7f1bd4076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.366+0000 7f1bf9200640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1bd4074760 0x7f1bd4076c20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1be4006fd0 tx=0x7f1be4008040 comp rx=0 tx=0).stop 2024-09-17T05:35:23.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.366+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf41fb650 msgr2=0x7f1bf40b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.366+0000 7f1bf9200640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf41fb650 0x7f1bf40b47b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1be001cd20 tx=0x7f1be00045d0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.366+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 shutdown_connections 2024-09-17T05:35:23.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.366+0000 7f1bf9200640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1bd4074760 0x7f1bd4076c20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.366+0000 7f1bf9200640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bf41fbab0 0x7f1bf40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.370+0000 7f1bf9200640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bf41fb650 0x7f1bf40b47b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.370+0000 7f1bf9200640 1 --2- 172.21.15.122:0/904144941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bd8004030 0x7f1bf40c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.370+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 >> 172.21.15.122:0/904144941 conn(0x7f1bf40d4210 msgr2=0x7f1bf4164450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.370+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 shutdown_connections 2024-09-17T05:35:23.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.370+0000 7f1bf9200640 1 -- 172.21.15.122:0/904144941 wait complete. 2024-09-17T05:35:23.407 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone608518 2024-09-17T05:35:23.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 -- 172.21.15.122:0/94152210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0d0110 msgr2=0x7fb02c1606c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 --2- 172.21.15.122:0/94152210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0d0110 0x7fb02c1606c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fb02c074150 tx=0x7fb02401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 -- 172.21.15.122:0/94152210 shutdown_connections 2024-09-17T05:35:23.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 --2- 172.21.15.122:0/94152210 >> v2:172.21.15.173:3300/0 conn(0x7fb02c0d2810 0x7fb02c160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 --2- 172.21.15.122:0/94152210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0d0110 0x7fb02c1606c0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 --2- 172.21.15.122:0/94152210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb02c0cfd30 0x7fb02c15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 -- 172.21.15.122:0/94152210 >> 172.21.15.122:0/94152210 conn(0x7fb02c0d1cf0 msgr2=0x7fb02c0d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:23.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 -- 172.21.15.122:0/94152210 shutdown_connections 2024-09-17T05:35:23.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.614+0000 7fb031000640 1 -- 172.21.15.122:0/94152210 wait complete. 2024-09-17T05:35:23.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 Processor -- start 2024-09-17T05:35:23.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 -- start start 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0cfd30 0x7fb02c147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb02c0d0110 0x7fb02c1fb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb02c0d2810 0x7fb02c06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0b2770 con 0x7fb02c0cfd30 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0b25f0 con 0x7fb02c0d2810 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c0b28f0 con 0x7fb02c0d0110 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0cfd30 0x7fb02c147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0cfd30 0x7fb02c147ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40936/0 (socket says 172.21.15.122:40936) 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 -- 172.21.15.122:0/369316799 learned_addr learned my addr 172.21.15.122:0/369316799 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 -- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb02c0d0110 msgr2=0x7fb02c1fb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb02c0d0110 0x7fb02c1fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 -- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb02c0d2810 msgr2=0x7fb02c06dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:23.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb02c0d2810 0x7fb02c06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:23.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 -- 172.21.15.122:0/369316799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c0cf0b0 con 0x7fb02c0cfd30 2024-09-17T05:35:23.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb02b400640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0cfd30 0x7fb02c147ea0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00ba50 tx=0x7fb01c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb023e00640 1 -- 172.21.15.122:0/369316799 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c019070 con 0x7fb02c0cfd30 2024-09-17T05:35:23.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb023e00640 1 -- 172.21.15.122:0/369316799 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb01c00cd00 con 0x7fb02c0cfd30 2024-09-17T05:35:23.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c0cf390 con 0x7fb02c0cfd30 2024-09-17T05:35:23.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb023e00640 1 -- 172.21.15.122:0/369316799 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0146a0 con 0x7fb02c0cfd30 2024-09-17T05:35:23.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.618+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c155720 con 0x7fb02c0cfd30 2024-09-17T05:35:23.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.622+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02c15fc40 con 0x7fb02c0cfd30 2024-09-17T05:35:23.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.622+0000 7fb023e00640 1 -- 172.21.15.122:0/369316799 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fb01c01c070 con 0x7fb02c0cfd30 2024-09-17T05:35:23.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.622+0000 7fb023e00640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb008074760 0x7fb008076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:23.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.622+0000 7fb023e00640 1 -- 172.21.15.122:0/369316799 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fb01c093150 con 0x7fb02c0cfd30 2024-09-17T05:35:23.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.626+0000 7fb023e00640 1 -- 172.21.15.122:0/369316799 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb01c05f850 con 0x7fb02c0cfd30 2024-09-17T05:35:23.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.646+0000 7fb022a00640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb008074760 0x7fb008076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:23.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.666+0000 7fb022a00640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb008074760 0x7fb008076c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb014009690 tx=0x7fb014006c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:23.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:23.966+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone608518", "target": ["mon-mgr", ""]}) -- 0x7fb02c0cf560 con 0x7fb008074760 2024-09-17T05:35:25.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.042+0000 7fb023e00640 1 -- 172.21.15.122:0/369316799 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb02c0cf560 con 0x7fb008074760 2024-09-17T05:35:25.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb008074760 msgr2=0x7fb008076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb008074760 0x7fb008076c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb014009690 tx=0x7fb014006c00 comp rx=0 tx=0).stop 2024-09-17T05:35:25.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0cfd30 msgr2=0x7fb02c147ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0cfd30 0x7fb02c147ea0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00ba50 tx=0x7fb01c00bf10 comp rx=0 tx=0).stop 2024-09-17T05:35:25.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 shutdown_connections 2024-09-17T05:35:25.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb008074760 0x7fb008076c20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb02c0d2810 0x7fb02c06dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb02c0d0110 0x7fb02c1fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 --2- 172.21.15.122:0/369316799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb02c0cfd30 0x7fb02c147ea0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 >> 172.21.15.122:0/369316799 conn(0x7fb02c0d1cf0 msgr2=0x7fb02c06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:25.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 shutdown_connections 2024-09-17T05:35:25.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.046+0000 7fb031000640 1 -- 172.21.15.122:0/369316799 wait complete. 2024-09-17T05:35:25.082 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone446019 2024-09-17T05:35:25.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.378+0000 7f4e02800640 1 -- 172.21.15.122:0/3950984747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc06cc20 msgr2=0x7f4dfc06d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.378+0000 7f4e02800640 1 --2- 172.21.15.122:0/3950984747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc06cc20 0x7f4dfc06d0d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009bc0 tx=0x7f4df401cba0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.382+0000 7f4e02800640 1 -- 172.21.15.122:0/3950984747 shutdown_connections 2024-09-17T05:35:25.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.382+0000 7f4e02800640 1 --2- 172.21.15.122:0/3950984747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc06cc20 0x7f4dfc06d0d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.382+0000 7f4e02800640 1 --2- 172.21.15.122:0/3950984747 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dfc15be10 0x7f4dfc1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.382+0000 7f4e02800640 1 -- 172.21.15.122:0/3950984747 >> 172.21.15.122:0/3950984747 conn(0x7f4dfc0cd740 msgr2=0x7f4dfc0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:25.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.382+0000 7f4e02800640 1 -- 172.21.15.122:0/3950984747 shutdown_connections 2024-09-17T05:35:25.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 -- 172.21.15.122:0/3950984747 wait complete. 2024-09-17T05:35:25.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 Processor -- start 2024-09-17T05:35:25.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 -- start start 2024-09-17T05:35:25.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dfc06cc20 0x7f4dfc0c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc0cbeb0 0x7f4dfc152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dfc15be10 0x7f4dfc0c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc166f30 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc166db0 con 0x7f4dfc15be10 2024-09-17T05:35:25.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4e02800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4dfc1670b0 con 0x7f4dfc06cc20 2024-09-17T05:35:25.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4dfaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc0cbeb0 0x7f4dfc152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4dfaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc0cbeb0 0x7f4dfc152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40956/0 (socket says 172.21.15.122:40956) 2024-09-17T05:35:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.386+0000 7f4dfaa00640 1 -- 172.21.15.122:0/1026026621 learned_addr learned my addr 172.21.15.122:0/1026026621 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4dfaa00640 1 -- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dfc06cc20 msgr2=0x7f4dfc0c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4dfaa00640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dfc06cc20 0x7f4dfc0c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4dfaa00640 1 -- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dfc15be10 msgr2=0x7f4dfc0c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4dfaa00640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dfc15be10 0x7f4dfc0c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4dfaa00640 1 -- 172.21.15.122:0/1026026621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc0c7100 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4dfaa00640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc0cbeb0 0x7f4dfc152910 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c450 tx=0x7f4de800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4df3400640 1 -- 172.21.15.122:0/1026026621 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8017070 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4df3400640 1 -- 172.21.15.122:0/1026026621 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4de8003600 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4df3400640 1 -- 172.21.15.122:0/1026026621 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8003a50 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc0c8eb0 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.390+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc0c9470 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.394+0000 7f4df3400640 1 -- 172.21.15.122:0/1026026621 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f4de801a070 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.394+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc06d0d0 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.394+0000 7f4df3400640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ddc074790 0x7f4ddc076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.398+0000 7f4df3400640 1 -- 172.21.15.122:0/1026026621 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f4de8092600 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.402+0000 7f4dfb400640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ddc074790 0x7f4ddc076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:25.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.402+0000 7f4df3400640 1 -- 172.21.15.122:0/1026026621 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4dfc0c7100 con 0x7f4dfc0cbeb0 2024-09-17T05:35:25.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.486+0000 7f4dfb400640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ddc074790 0x7f4ddc076c50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0058d0 tx=0x7f4dec005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:25.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:25.754+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone446019", "target": ["mon-mgr", ""]}) -- 0x7f4dfc0c5bb0 con 0x7f4ddc074790 2024-09-17T05:35:27.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.138+0000 7f4df3400640 1 -- 172.21.15.122:0/1026026621 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4dfc0c5bb0 con 0x7f4ddc074790 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ddc074790 msgr2=0x7f4ddc076c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ddc074790 0x7f4ddc076c50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0058d0 tx=0x7f4dec005860 comp rx=0 tx=0).stop 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc0cbeb0 msgr2=0x7f4dfc152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc0cbeb0 0x7f4dfc152910 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4de800c450 tx=0x7f4de800c910 comp rx=0 tx=0).stop 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 shutdown_connections 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ddc074790 0x7f4ddc076c50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4dfc15be10 0x7f4dfc0c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc0cbeb0 0x7f4dfc152910 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 --2- 172.21.15.122:0/1026026621 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4dfc06cc20 0x7f4dfc0c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 >> 172.21.15.122:0/1026026621 conn(0x7f4dfc0cd740 msgr2=0x7f4dfc155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 shutdown_connections 2024-09-17T05:35:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:27.142+0000 7f4e02800640 1 -- 172.21.15.122:0/1026026621 wait complete. 2024-09-17T05:35:29.187 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:35:29.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.418+0000 7fa16e600640 1 -- 172.21.15.122:0/2289429286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa150003a60 msgr2=0x7fa150003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:29.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.418+0000 7fa16e600640 1 --2- 172.21.15.122:0/2289429286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa150003a60 0x7fa150003e60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0098a0 tx=0x7fa15c01c5e0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.418+0000 7fa16e600640 1 -- 172.21.15.122:0/2289429286 shutdown_connections 2024-09-17T05:35:29.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.418+0000 7fa16e600640 1 --2- 172.21.15.122:0/2289429286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa150003a60 0x7fa150003e60 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.418+0000 7fa16e600640 1 -- 172.21.15.122:0/2289429286 >> 172.21.15.122:0/2289429286 conn(0x7fa1680d1ce0 msgr2=0x7fa1680d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:29.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.418+0000 7fa16e600640 1 -- 172.21.15.122:0/2289429286 shutdown_connections 2024-09-17T05:35:29.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.418+0000 7fa16e600640 1 -- 172.21.15.122:0/2289429286 wait complete. 2024-09-17T05:35:29.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa16e600640 1 Processor -- start 2024-09-17T05:35:29.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa16e600640 1 -- start start 2024-09-17T05:35:29.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa16e600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa150003a60 0x7fa168157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:29.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa16e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa16806fa90 0x7fa16814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:29.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa16e600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1681560a0 0x7fa1681eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:29.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa16e600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa1680b1a00 con 0x7fa16806fa90 2024-09-17T05:35:29.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa16e600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1680b1880 con 0x7fa150003a60 2024-09-17T05:35:29.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa16e600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa1680b1b80 con 0x7fa1681560a0 2024-09-17T05:35:29.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa167e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa16806fa90 0x7fa16814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:29.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa167e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa16806fa90 0x7fa16814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40978/0 (socket says 172.21.15.122:40978) 2024-09-17T05:35:29.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa167e00640 1 -- 172.21.15.122:0/2198935016 learned_addr learned my addr 172.21.15.122:0/2198935016 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:29.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa167e00640 1 -- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1681560a0 msgr2=0x7fa1681eb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:29.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa167e00640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1681560a0 0x7fa1681eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa167e00640 1 -- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa150003a60 msgr2=0x7fa168157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:29.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa167e00640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa150003a60 0x7fa168157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.422+0000 7fa167e00640 1 -- 172.21.15.122:0/2198935016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16815c620 con 0x7fa16806fa90 2024-09-17T05:35:29.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa167e00640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa16806fa90 0x7fa16814afd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa15400b500 tx=0x7fa15400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:29.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa164c00640 1 -- 172.21.15.122:0/2198935016 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154014070 con 0x7fa16806fa90 2024-09-17T05:35:29.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa164c00640 1 -- 172.21.15.122:0/2198935016 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa154010960 con 0x7fa16806fa90 2024-09-17T05:35:29.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa164c00640 1 -- 172.21.15.122:0/2198935016 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154010db0 con 0x7fa16806fa90 2024-09-17T05:35:29.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1681f8110 con 0x7fa16806fa90 2024-09-17T05:35:29.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1681f86d0 con 0x7fa16806fa90 2024-09-17T05:35:29.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1680b20e0 con 0x7fa16806fa90 2024-09-17T05:35:29.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa164c00640 1 -- 172.21.15.122:0/2198935016 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7fa154017070 con 0x7fa16806fa90 2024-09-17T05:35:29.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.426+0000 7fa164c00640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa148074790 0x7fa148076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:29.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.430+0000 7fa164c00640 1 -- 172.21.15.122:0/2198935016 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fa154093550 con 0x7fa16806fa90 2024-09-17T05:35:29.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.430+0000 7fa164c00640 1 -- 172.21.15.122:0/2198935016 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa15405fc50 con 0x7fa16806fa90 2024-09-17T05:35:29.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.430+0000 7fa16d200640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa148074790 0x7fa148076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:29.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.442+0000 7fa16d200640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa148074790 0x7fa148076c50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009870 tx=0x7fa150005840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:29.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.814+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa16815ae10 con 0x7fa16806fa90 2024-09-17T05:35:29.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.814+0000 7fa164c00640 1 -- 172.21.15.122:0/2198935016 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2443 (secure 0 0 0) 0x7fa15405f550 con 0x7fa16806fa90 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:35:29.813 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 650, 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:35:29.814 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 90, 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:35:29.815 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 6040, 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 25134949069, 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 50296856576, 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 722797256704, 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:35:29.816 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 605421837, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 605467100, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 145, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 158 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 57, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:34:18:421377+0000", 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 6, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:35:29.817 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 6, 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "c", 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 7029 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:35:29.818 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 16, 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:35:16.639960+0000", 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:35:29.819 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "4 ongoing clones - average progress is 19.727% (33s)\n [=====.......................] (remaining: 2m)", 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.19727000594139099, 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-total-clones": { 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "Total 11 clones - average progress is 28.078% (25s)\n [=======.....................] (remaining: 66s)", 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0.28077998757362366, 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:35:29.820 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:35:29.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa148074790 msgr2=0x7fa148076c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:29.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa148074790 0x7fa148076c50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009870 tx=0x7fa150005840 comp rx=0 tx=0).stop 2024-09-17T05:35:29.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa16806fa90 msgr2=0x7fa16814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:29.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa16806fa90 0x7fa16814afd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa15400b500 tx=0x7fa15400b9c0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 shutdown_connections 2024-09-17T05:35:29.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa148074790 0x7fa148076c50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1681560a0 0x7fa1681eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa16806fa90 0x7fa16814afd0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 --2- 172.21.15.122:0/2198935016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa150003a60 0x7fa168157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:29.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 >> 172.21.15.122:0/2198935016 conn(0x7fa1680d1ce0 msgr2=0x7fa168159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:29.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 shutdown_connections 2024-09-17T05:35:29.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:29.818+0000 7fa16e600640 1 -- 172.21.15.122:0/2198935016 wait complete. 2024-09-17T05:35:29.859 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 19.727% (33s)\n [=====.......................] (remaining: 2m)', 'progress': 0.197270005941391, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 11 clones - average progress is 28.078% (25s)\n [=======.....................] (remaining: 66s)', 'progress': 0.28077998757362366, 'add_to_ceph_s': True}} 2024-09-17T05:35:29.859 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-09-17T05:35:29.859 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.28077998757362366; onpen_p = 0.3072899878025055 2024-09-17T05:35:29.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone84910 2024-09-17T05:35:30.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- 172.21.15.122:0/3780604266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc00cac60 msgr2=0x7f8bc00744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:30.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 --2- 172.21.15.122:0/3780604266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc00cac60 0x7f8bc00744e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009840 tx=0x7f8bac01c620 comp rx=0 tx=0).stop 2024-09-17T05:35:30.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- 172.21.15.122:0/3780604266 shutdown_connections 2024-09-17T05:35:30.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 --2- 172.21.15.122:0/3780604266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc00cac60 0x7f8bc00744e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:30.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 --2- 172.21.15.122:0/3780604266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc00aecb0 0x7f8bc0073fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:30.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- 172.21.15.122:0/3780604266 >> 172.21.15.122:0/3780604266 conn(0x7f8bc00cfa00 msgr2=0x7f8bc00cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:30.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- 172.21.15.122:0/3780604266 shutdown_connections 2024-09-17T05:35:30.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- 172.21.15.122:0/3780604266 wait complete. 2024-09-17T05:35:30.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 Processor -- start 2024-09-17T05:35:30.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- start start 2024-09-17T05:35:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc0072610 0x7f8bc014de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc00aecb0 0x7f8bc01e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc00cac60 0x7f8bc01f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:30.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc00ca050 con 0x7f8bc0072610 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc006ef70 con 0x7f8bc00cac60 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc6600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc00ca1d0 con 0x7f8bc00aecb0 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc0072610 0x7f8bc014de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc0072610 0x7f8bc014de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41006/0 (socket says 172.21.15.122:41006) 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.118+0000 7f8bc5200640 1 -- 172.21.15.122:0/889944737 learned_addr learned my addr 172.21.15.122:0/889944737 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bbfe00640 1 --2- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc00aecb0 0x7f8bc01e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc5c00640 1 --2- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc00cac60 0x7f8bc01f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc5200640 1 -- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc00aecb0 msgr2=0x7f8bc01e60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc5200640 1 --2- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8bc00aecb0 0x7f8bc01e60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc5200640 1 -- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc00cac60 msgr2=0x7f8bc01f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc5200640 1 --2- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bc00cac60 0x7f8bc01f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc5200640 1 -- 172.21.15.122:0/889944737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc01e5d30 con 0x7f8bc0072610 2024-09-17T05:35:30.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc5200640 1 --2- 172.21.15.122:0/889944737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8bc0072610 0x7f8bc014de70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400d950 tx=0x7f8bb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bbcc00640 1 -- 172.21.15.122:0/889944737 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4017070 con 0x7f8bc0072610 2024-09-17T05:35:30.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bbcc00640 1 -- 172.21.15.122:0/889944737 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bb4009d20 con 0x7f8bc0072610 2024-09-17T05:35:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bbcc00640 1 -- 172.21.15.122:0/889944737 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40106c0 con 0x7f8bc0072610 2024-09-17T05:35:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc6600640 1 -- 172.21.15.122:0/889944737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc01e5770 con 0x7f8bc0072610 2024-09-17T05:35:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc6600640 1 -- 172.21.15.122:0/889944737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc00c78c0 con 0x7f8bc0072610 2024-09-17T05:35:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bc6600640 1 -- 172.21.15.122:0/889944737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc00ca700 con 0x7f8bc0072610 2024-09-17T05:35:30.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.122+0000 7f8bbcc00640 1 -- 172.21.15.122:0/889944737 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f8bb4010860 con 0x7f8bc0072610 2024-09-17T05:35:30.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.126+0000 7f8bbcc00640 1 --2- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba0074790 0x7f8ba0076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:30.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.126+0000 7f8bbcc00640 1 -- 172.21.15.122:0/889944737 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f8bb4093660 con 0x7f8bc0072610 2024-09-17T05:35:30.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.126+0000 7f8bbfe00640 1 --2- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba0074790 0x7f8ba0076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:30.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.126+0000 7f8bbfe00640 1 --2- 172.21.15.122:0/889944737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ba0074790 0x7f8ba0076c50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0ae000 tx=0x7f8bac01ae60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:30.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.130+0000 7f8bbcc00640 1 -- 172.21.15.122:0/889944737 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8bb405f710 con 0x7f8bc0072610 2024-09-17T05:35:30.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.334+0000 7f8bc6600640 1 -- 172.21.15.122:0/889944737 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone84910", "target": ["mon-mgr", ""]}) -- 0x7f8bc015d290 con 0x7f8ba0074790 2024-09-17T05:35:30.859 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:30.866+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:30.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:30.862+0000 7f8bbcc00640 1 -- 172.21.15.122:0/889944737 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f8bc015d290 con 0x7f8ba0074790 2024-09-17T05:35:30.860 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:30.869 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:35:30.870 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92681 2024-09-17T05:35:31.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.042+0000 7f3278000640 1 -- 172.21.15.122:0/2033655265 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32700b1650 msgr2=0x7f32700b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:31.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.042+0000 7f3278000640 1 --2- 172.21.15.122:0/2033655265 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32700b1650 0x7f32700b1a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3264009840 tx=0x7f326401c820 comp rx=0 tx=0).stop 2024-09-17T05:35:31.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.042+0000 7f3278000640 1 -- 172.21.15.122:0/2033655265 shutdown_connections 2024-09-17T05:35:31.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.042+0000 7f3278000640 1 --2- 172.21.15.122:0/2033655265 >> v2:172.21.15.173:3300/0 conn(0x7f32700afb10 0x7f32700affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:31.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.042+0000 7f3278000640 1 --2- 172.21.15.122:0/2033655265 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32700b1650 0x7f32700b1a30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:31.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.042+0000 7f3278000640 1 -- 172.21.15.122:0/2033655265 >> 172.21.15.122:0/2033655265 conn(0x7f32700d1dd0 msgr2=0x7f32700d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:31.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.042+0000 7f3278000640 1 -- 172.21.15.122:0/2033655265 shutdown_connections 2024-09-17T05:35:31.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.042+0000 7f3278000640 1 -- 172.21.15.122:0/2033655265 wait complete. 2024-09-17T05:35:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 Processor -- start 2024-09-17T05:35:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 -- start start 2024-09-17T05:35:31.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32700afb10 0x7f327014a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:31.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32700b1650 0x7f32700cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:31.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32701605d0 0x7f327015be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:31.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3270169540 con 0x7f32701605d0 2024-09-17T05:35:31.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f32700b1430 con 0x7f32700b1650 2024-09-17T05:35:31.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f32701696c0 con 0x7f32700afb10 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32701605d0 0x7f327015be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32701605d0 0x7f327015be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41020/0 (socket says 172.21.15.122:41020) 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 -- 172.21.15.122:0/451656071 learned_addr learned my addr 172.21.15.122:0/451656071 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3275600640 1 --2- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32700afb10 0x7f327014a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 -- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32700afb10 msgr2=0x7f327014a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 --2- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32700afb10 0x7f327014a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 -- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32700b1650 msgr2=0x7f32700cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 --2- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32700b1650 0x7f32700cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 -- 172.21.15.122:0/451656071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32700b5000 con 0x7f32701605d0 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3275600640 1 --2- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32700afb10 0x7f327014a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:31.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3276000640 1 --2- 172.21.15.122:0/451656071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32701605d0 0x7f327015be80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f326800b9a0 tx=0x7f326800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:31.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f326d600640 1 -- 172.21.15.122:0/451656071 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268019070 con 0x7f32701605d0 2024-09-17T05:35:31.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f326d600640 1 -- 172.21.15.122:0/451656071 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f326800ccb0 con 0x7f32701605d0 2024-09-17T05:35:31.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 -- 172.21.15.122:0/451656071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32700b52e0 con 0x7f32701605d0 2024-09-17T05:35:31.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f326d600640 1 -- 172.21.15.122:0/451656071 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32680146a0 con 0x7f32701605d0 2024-09-17T05:35:31.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.046+0000 7f3278000640 1 -- 172.21.15.122:0/451656071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32700b58a0 con 0x7f32701605d0 2024-09-17T05:35:31.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.050+0000 7f326d600640 1 -- 172.21.15.122:0/451656071 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f326801c070 con 0x7f32701605d0 2024-09-17T05:35:31.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.050+0000 7f326d600640 1 --2- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3254074760 0x7f3254076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:31.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.050+0000 7f326d600640 1 -- 172.21.15.122:0/451656071 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f3268093080 con 0x7f32701605d0 2024-09-17T05:35:31.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.054+0000 7f3278000640 1 -- 172.21.15.122:0/451656071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32700affc0 con 0x7f32701605d0 2024-09-17T05:35:31.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.062+0000 7f326d600640 1 -- 172.21.15.122:0/451656071 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f326805f7c0 con 0x7f32701605d0 2024-09-17T05:35:31.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.062+0000 7f3275600640 1 --2- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3254074760 0x7f3254076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:31.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.070+0000 7f3275600640 1 --2- 172.21.15.122:0/451656071 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3254074760 0x7f3254076c20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3260001630 tx=0x7f3260009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:31.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.450+0000 7f3278000640 1 -- 172.21.15.122:0/451656071 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92681", "target": ["mon-mgr", ""]}) -- 0x7f3270158390 con 0x7f3254074760 2024-09-17T05:35:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.606+0000 7f326d600640 1 -- 172.21.15.122:0/451656071 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f326000d4a0 con 0x7f3254074760 2024-09-17T05:35:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:31.604 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:31.602+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:31.619 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:35:31.619 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone67922 2024-09-17T05:35:31.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.874+0000 7faacf600640 1 -- 172.21.15.122:0/2584550400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac815a5f0 msgr2=0x7faac815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:31.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.874+0000 7faacf600640 1 --2- 172.21.15.122:0/2584550400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac815a5f0 0x7faac815aa80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faac000ad90 tx=0x7faac001dc80 comp rx=0 tx=0).stop 2024-09-17T05:35:31.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.874+0000 7faacf600640 1 -- 172.21.15.122:0/2584550400 shutdown_connections 2024-09-17T05:35:31.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.874+0000 7faacf600640 1 --2- 172.21.15.122:0/2584550400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac815a5f0 0x7faac815aa80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:31.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.874+0000 7faacf600640 1 --2- 172.21.15.122:0/2584550400 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faac80cd330 0x7faac815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:31.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.874+0000 7faacf600640 1 -- 172.21.15.122:0/2584550400 >> 172.21.15.122:0/2584550400 conn(0x7faac80b2060 msgr2=0x7faac80b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:31.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.874+0000 7faacf600640 1 -- 172.21.15.122:0/2584550400 shutdown_connections 2024-09-17T05:35:31.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.874+0000 7faacf600640 1 -- 172.21.15.122:0/2584550400 wait complete. 2024-09-17T05:35:31.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.878+0000 7faacf600640 1 Processor -- start 2024-09-17T05:35:31.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.878+0000 7faacf600640 1 -- start start 2024-09-17T05:35:31.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faacf600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faac80cd330 0x7faac814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:31.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faacf600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac815a210 0x7faac81fe720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:31.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faacf600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac815a5f0 0x7faac8205160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:31.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faacf600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faac8070100 con 0x7faac815a210 2024-09-17T05:35:31.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faacf600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faac806ff80 con 0x7faac815a5f0 2024-09-17T05:35:31.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faacf600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faac8070280 con 0x7faac80cd330 2024-09-17T05:35:31.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faac7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac815a210 0x7faac81fe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:31.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faac7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac815a210 0x7faac81fe720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49856/0 (socket says 172.21.15.122:49856) 2024-09-17T05:35:31.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.882+0000 7faac7e00640 1 -- 172.21.15.122:0/451167777 learned_addr learned my addr 172.21.15.122:0/451167777 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:31.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac7e00640 1 -- 172.21.15.122:0/451167777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faac80cd330 msgr2=0x7faac814a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:31.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac7e00640 1 --2- 172.21.15.122:0/451167777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faac80cd330 0x7faac814a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:31.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac7e00640 1 -- 172.21.15.122:0/451167777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac815a5f0 msgr2=0x7faac8205160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac7e00640 1 --2- 172.21.15.122:0/451167777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac815a5f0 0x7faac8205160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac7e00640 1 -- 172.21.15.122:0/451167777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac82056a0 con 0x7faac815a210 2024-09-17T05:35:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac7e00640 1 --2- 172.21.15.122:0/451167777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faac815a210 0x7faac81fe720 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7faab400b440 tx=0x7faab400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac4c00640 1 -- 172.21.15.122:0/451167777 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4015030 con 0x7faac815a210 2024-09-17T05:35:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac4c00640 1 -- 172.21.15.122:0/451167777 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faab4005980 con 0x7faac815a210 2024-09-17T05:35:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faac4c00640 1 -- 172.21.15.122:0/451167777 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4005dd0 con 0x7faac815a210 2024-09-17T05:35:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.886+0000 7faacf600640 1 -- 172.21.15.122:0/451167777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac814ac80 con 0x7faac815a210 2024-09-17T05:35:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.890+0000 7faacf600640 1 -- 172.21.15.122:0/451167777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac8149230 con 0x7faac815a210 2024-09-17T05:35:31.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.894+0000 7faacf600640 1 -- 172.21.15.122:0/451167777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac80b6490 con 0x7faac815a210 2024-09-17T05:35:31.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.902+0000 7faac4c00640 1 -- 172.21.15.122:0/451167777 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7faab4018070 con 0x7faac815a210 2024-09-17T05:35:31.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.902+0000 7faac4c00640 1 --2- 172.21.15.122:0/451167777 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faaa8074830 0x7faaa8076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:31.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.902+0000 7faac4c00640 1 -- 172.21.15.122:0/451167777 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7faab4093040 con 0x7faac815a210 2024-09-17T05:35:31.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.910+0000 7faac4c00640 1 -- 172.21.15.122:0/451167777 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7faac82056a0 con 0x7faac815a210 2024-09-17T05:35:31.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.914+0000 7faaccc00640 1 --2- 172.21.15.122:0/451167777 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faaa8074830 0x7faaa8076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:31.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:31.918+0000 7faaccc00640 1 --2- 172.21.15.122:0/451167777 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faaa8074830 0x7faaa8076cf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7faac806a590 tx=0x7faabc0079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:32.302+0000 7faacf600640 1 -- 172.21.15.122:0/451167777 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone67922", "target": ["mon-mgr", ""]}) -- 0x7faac8069dc0 con 0x7faaa8074830 2024-09-17T05:35:32.499 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:32.502+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:35:32.499 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:32.502+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:35:32.499 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:32.502+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:35:32.499 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:32.502+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:35:32.500 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:32.502+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:35:32.917 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:32.922+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:32.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:32.934+0000 7faac4c00640 1 -- 172.21.15.122:0/451167777 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7faac8069dc0 con 0x7faaa8074830 2024-09-17T05:35:32.930 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:32.944 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:35:32.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone84053 2024-09-17T05:35:33.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.182+0000 7f65c5200640 1 -- 172.21.15.122:0/3961804184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65a4004810 msgr2=0x7f65a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:33.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.182+0000 7f65c5200640 1 --2- 172.21.15.122:0/3961804184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65a4004810 0x7f65a4004c90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006f20 tx=0x7f65ac027a10 comp rx=0 tx=0).stop 2024-09-17T05:35:33.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.182+0000 7f65c5200640 1 -- 172.21.15.122:0/3961804184 shutdown_connections 2024-09-17T05:35:33.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.182+0000 7f65c5200640 1 --2- 172.21.15.122:0/3961804184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65a4004810 0x7f65a4004c90 secure :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006f20 tx=0x7f65ac027a10 comp rx=0 tx=0).stop 2024-09-17T05:35:33.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.182+0000 7f65c5200640 1 -- 172.21.15.122:0/3961804184 >> 172.21.15.122:0/3961804184 conn(0x7f65c006cc20 msgr2=0x7f65c006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:33.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.182+0000 7f65c5200640 1 -- 172.21.15.122:0/3961804184 shutdown_connections 2024-09-17T05:35:33.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.182+0000 7f65c5200640 1 -- 172.21.15.122:0/3961804184 wait complete. 2024-09-17T05:35:33.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65c5200640 1 Processor -- start 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65c5200640 1 -- start start 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65c5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65c014a280 0x7f65c014e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65c5200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65c014a660 0x7f65c01f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65c5200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65c0157720 0x7f65c014d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65c5200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65c00d0070 con 0x7f65c014a280 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65c5200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f65c00cfef0 con 0x7f65c0157720 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65c5200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f65c00d01f0 con 0x7f65c014a660 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65bf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65c014a280 0x7f65c014e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:33.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65bf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65c014a280 0x7f65c014e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49874/0 (socket says 172.21.15.122:49874) 2024-09-17T05:35:33.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65bf400640 1 -- 172.21.15.122:0/1317034897 learned_addr learned my addr 172.21.15.122:0/1317034897 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:33.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65bf400640 1 -- 172.21.15.122:0/1317034897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65c014a660 msgr2=0x7f65c01f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:33.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65bf400640 1 --2- 172.21.15.122:0/1317034897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65c014a660 0x7f65c01f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:33.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65bf400640 1 -- 172.21.15.122:0/1317034897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65c0157720 msgr2=0x7f65c014d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:33.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65bf400640 1 --2- 172.21.15.122:0/1317034897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65c0157720 0x7f65c014d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:33.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.186+0000 7f65bf400640 1 -- 172.21.15.122:0/1317034897 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c01f1550 con 0x7f65c014a280 2024-09-17T05:35:33.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.190+0000 7f65bf400640 1 --2- 172.21.15.122:0/1317034897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65c014a280 0x7f65c014e780 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f65b0006fd0 tx=0x7f65b000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:33.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.190+0000 7f65b7400640 1 -- 172.21.15.122:0/1317034897 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0015020 con 0x7f65c014a280 2024-09-17T05:35:33.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.190+0000 7f65b7400640 1 -- 172.21.15.122:0/1317034897 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65b0011920 con 0x7f65c014a280 2024-09-17T05:35:33.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.190+0000 7f65b7400640 1 -- 172.21.15.122:0/1317034897 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0011d90 con 0x7f65c014a280 2024-09-17T05:35:33.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.190+0000 7f65c5200640 1 -- 172.21.15.122:0/1317034897 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c014ed60 con 0x7f65c014a280 2024-09-17T05:35:33.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.190+0000 7f65c5200640 1 -- 172.21.15.122:0/1317034897 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c014f320 con 0x7f65c014a280 2024-09-17T05:35:33.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.194+0000 7f65c5200640 1 -- 172.21.15.122:0/1317034897 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c006ebf0 con 0x7f65c014a280 2024-09-17T05:35:33.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.194+0000 7f65b7400640 1 -- 172.21.15.122:0/1317034897 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 35) ==== 99172+0+0 (secure 0 0 0) 0x7f65b0018020 con 0x7f65c014a280 2024-09-17T05:35:33.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.198+0000 7f65b7400640 1 --2- 172.21.15.122:0/1317034897 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65a0074760 0x7f65a0076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:33.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.198+0000 7f65b7400640 1 -- 172.21.15.122:0/1317034897 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f65b00931e0 con 0x7f65c014a280 2024-09-17T05:35:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.206+0000 7f65b7400640 1 -- 172.21.15.122:0/1317034897 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f65b001f020 con 0x7f65c014a280 2024-09-17T05:35:33.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.234+0000 7f65bea00640 1 --2- 172.21.15.122:0/1317034897 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65a0074760 0x7f65a0076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:33.237 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:33.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:35:33.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.282+0000 7f65bea00640 1 --2- 172.21.15.122:0/1317034897 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65a0074760 0x7f65a0076c20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0b5000 tx=0x7f65a4005840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:33.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.322+0000 7f65b7400640 1 -- 172.21.15.122:0/1317034897 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 36) ==== 99223+0+0 (secure 0 0 0) 0x7f65b005b450 con 0x7f65c014a280 2024-09-17T05:35:33.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.558+0000 7f65c5200640 1 -- 172.21.15.122:0/1317034897 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone84053", "target": ["mon-mgr", ""]}) -- 0x7f65c00cdd50 con 0x7f65a0074760 2024-09-17T05:35:33.937 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:33.942+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:33.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:33.942+0000 7f65b7400640 1 -- 172.21.15.122:0/1317034897 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f65c00cdd50 con 0x7f65a0074760 2024-09-17T05:35:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:33.959 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:35:33.960 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone9004 2024-09-17T05:35:34.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 -- 172.21.15.122:0/721271152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05d80703d0 msgr2=0x7f05d80b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:34.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 --2- 172.21.15.122:0/721271152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05d80703d0 0x7f05d80b42a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f05d80b2950 tx=0x7f05c401c570 comp rx=0 tx=0).stop 2024-09-17T05:35:34.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 -- 172.21.15.122:0/721271152 shutdown_connections 2024-09-17T05:35:34.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 --2- 172.21.15.122:0/721271152 >> v2:172.21.15.173:3300/0 conn(0x7f05d80b47e0 0x7f05d806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 --2- 172.21.15.122:0/721271152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05d80703d0 0x7f05d80b42a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 --2- 172.21.15.122:0/721271152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05d815a4d0 0x7f05d815ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 -- 172.21.15.122:0/721271152 >> 172.21.15.122:0/721271152 conn(0x7f05d80b18f0 msgr2=0x7f05d80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 -- 172.21.15.122:0/721271152 shutdown_connections 2024-09-17T05:35:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.242+0000 7f05df800640 1 -- 172.21.15.122:0/721271152 wait complete. 2024-09-17T05:35:34.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05df800640 1 Processor -- start 2024-09-17T05:35:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05df800640 1 -- start start 2024-09-17T05:35:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05df800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05d80703d0 0x7f05d814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05df800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05d80b47e0 0x7f05d8165b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05df800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05d815a4d0 0x7f05d81fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05df800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05d80ccda0 con 0x7f05d80703d0 2024-09-17T05:35:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05df800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05d80ccc20 con 0x7f05d80b47e0 2024-09-17T05:35:34.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05df800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f05d80ccf20 con 0x7f05d815a4d0 2024-09-17T05:35:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05d80703d0 0x7f05d814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05d80703d0 0x7f05d814d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49902/0 (socket says 172.21.15.122:49902) 2024-09-17T05:35:34.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 -- 172.21.15.122:0/1997432142 learned_addr learned my addr 172.21.15.122:0/1997432142 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:34.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 -- 172.21.15.122:0/1997432142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05d815a4d0 msgr2=0x7f05d81fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:34.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 --2- 172.21.15.122:0/1997432142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05d815a4d0 0x7f05d81fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:34.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 -- 172.21.15.122:0/1997432142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05d80b47e0 msgr2=0x7f05d8165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:34.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 --2- 172.21.15.122:0/1997432142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05d80b47e0 0x7f05d8165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:34.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 -- 172.21.15.122:0/1997432142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d81563f0 con 0x7f05d80703d0 2024-09-17T05:35:34.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.246+0000 7f05dce00640 1 --2- 172.21.15.122:0/1997432142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05d80703d0 0x7f05d814d300 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00b410 tx=0x7f05cc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:34.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.250+0000 7f05d4c00640 1 -- 172.21.15.122:0/1997432142 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc015020 con 0x7f05d80703d0 2024-09-17T05:35:34.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.250+0000 7f05d4c00640 1 -- 172.21.15.122:0/1997432142 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05cc011920 con 0x7f05d80703d0 2024-09-17T05:35:34.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.250+0000 7f05df800640 1 -- 172.21.15.122:0/1997432142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d81566d0 con 0x7f05d80703d0 2024-09-17T05:35:34.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.250+0000 7f05d4c00640 1 -- 172.21.15.122:0/1997432142 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05cc011d70 con 0x7f05d80703d0 2024-09-17T05:35:34.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.250+0000 7f05df800640 1 -- 172.21.15.122:0/1997432142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d81fb350 con 0x7f05d80703d0 2024-09-17T05:35:34.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.250+0000 7f05df800640 1 -- 172.21.15.122:0/1997432142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d8069410 con 0x7f05d80703d0 2024-09-17T05:35:34.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.258+0000 7f05d4c00640 1 -- 172.21.15.122:0/1997432142 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 36) ==== 99223+0+0 (secure 0 0 0) 0x7f05cc018020 con 0x7f05d80703d0 2024-09-17T05:35:34.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.262+0000 7f05d4c00640 1 --2- 172.21.15.122:0/1997432142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f05b8074870 0x7f05b8076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:34.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.262+0000 7f05d4c00640 1 -- 172.21.15.122:0/1997432142 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f05cc093360 con 0x7f05d80703d0 2024-09-17T05:35:34.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.266+0000 7f05d4c00640 1 -- 172.21.15.122:0/1997432142 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f05cc01f020 con 0x7f05d80703d0 2024-09-17T05:35:34.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.290+0000 7f05d7e00640 1 --2- 172.21.15.122:0/1997432142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f05b8074870 0x7f05b8076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:34.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.322+0000 7f05d7e00640 1 --2- 172.21.15.122:0/1997432142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f05b8074870 0x7f05b8076d30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f05c40af000 tx=0x7f05c4019310 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:34.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:34.646+0000 7f05df800640 1 -- 172.21.15.122:0/1997432142 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone9004", "target": ["mon-mgr", ""]}) -- 0x7f05d8164f40 con 0x7f05b8074870 2024-09-17T05:35:35.409 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:35.414+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:35.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.450+0000 7f05d4c00640 1 -- 172.21.15.122:0/1997432142 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f05c4004140 con 0x7f05b8074870 2024-09-17T05:35:35.445 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:35.456 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:35:35.457 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone25105 2024-09-17T05:35:35.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 -- 172.21.15.122:0/1981360220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d815db50 msgr2=0x7f96d80acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:35.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 --2- 172.21.15.122:0/1981360220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d815db50 0x7f96d80acd40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f96d8074c10 tx=0x7f96c801c830 comp rx=0 tx=0).stop 2024-09-17T05:35:35.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 -- 172.21.15.122:0/1981360220 shutdown_connections 2024-09-17T05:35:35.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 --2- 172.21.15.122:0/1981360220 >> v2:172.21.15.173:3300/0 conn(0x7f96d80b4a50 0x7f96d80ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:35.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 --2- 172.21.15.122:0/1981360220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d815db50 0x7f96d80acd40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:35.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 -- 172.21.15.122:0/1981360220 >> 172.21.15.122:0/1981360220 conn(0x7f96d80b18f0 msgr2=0x7f96d80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:35.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 -- 172.21.15.122:0/1981360220 shutdown_connections 2024-09-17T05:35:35.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 -- 172.21.15.122:0/1981360220 wait complete. 2024-09-17T05:35:35.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.726+0000 7f96dce00640 1 Processor -- start 2024-09-17T05:35:35.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 -- start start 2024-09-17T05:35:35.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d80b4a50 0x7f96d8159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:35.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96d80cd100 0x7f96d81f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:35.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96d815db50 0x7f96d809b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:35.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8167ac0 con 0x7f96d80b4a50 2024-09-17T05:35:35.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8167940 con 0x7f96d815db50 2024-09-17T05:35:35.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f96d8167c40 con 0x7f96d80cd100 2024-09-17T05:35:35.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d80b4a50 0x7f96d8159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:35.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d80b4a50 0x7f96d8159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49926/0 (socket says 172.21.15.122:49926) 2024-09-17T05:35:35.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 -- 172.21.15.122:0/1801840600 learned_addr learned my addr 172.21.15.122:0/1801840600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 -- 172.21.15.122:0/1801840600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96d80cd100 msgr2=0x7f96d81f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 --2- 172.21.15.122:0/1801840600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96d80cd100 0x7f96d81f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 -- 172.21.15.122:0/1801840600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96d815db50 msgr2=0x7f96d809b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 --2- 172.21.15.122:0/1801840600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96d815db50 0x7f96d809b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 -- 172.21.15.122:0/1801840600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d81f5810 con 0x7f96d80b4a50 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96d5e00640 1 --2- 172.21.15.122:0/1801840600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96d80b4a50 0x7f96d8159cc0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f96c80b0040 tx=0x7f96c8005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96ce000640 1 -- 172.21.15.122:0/1801840600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c80b2040 con 0x7f96d80b4a50 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96ce000640 1 -- 172.21.15.122:0/1801840600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96c80a6d60 con 0x7f96d80b4a50 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 -- 172.21.15.122:0/1801840600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d815a2a0 con 0x7f96d80b4a50 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96ce000640 1 -- 172.21.15.122:0/1801840600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8003860 con 0x7f96d80b4a50 2024-09-17T05:35:35.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.730+0000 7f96dce00640 1 -- 172.21.15.122:0/1801840600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d81f5ad0 con 0x7f96d80b4a50 2024-09-17T05:35:35.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.734+0000 7f96dce00640 1 -- 172.21.15.122:0/1801840600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d8069410 con 0x7f96d80b4a50 2024-09-17T05:35:35.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.734+0000 7f96ce000640 1 -- 172.21.15.122:0/1801840600 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 36) ==== 99223+0+0 (secure 0 0 0) 0x7f96c80bd060 con 0x7f96d80b4a50 2024-09-17T05:35:35.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.734+0000 7f96ce000640 1 --2- 172.21.15.122:0/1801840600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96b4078c00 0x7f96b407b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:35.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.734+0000 7f96ce000640 1 -- 172.21.15.122:0/1801840600 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f96c812c370 con 0x7f96d80b4a50 2024-09-17T05:35:35.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.742+0000 7f96ce000640 1 -- 172.21.15.122:0/1801840600 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f96c80164b0 con 0x7f96d80b4a50 2024-09-17T05:35:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.778+0000 7f96cfe00640 1 --2- 172.21.15.122:0/1801840600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96b4078c00 0x7f96b407b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:35.790+0000 7f96cfe00640 1 --2- 172.21.15.122:0/1801840600 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96b4078c00 0x7f96b407b0c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f96c0007fd0 tx=0x7f96c00079a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:36.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.066+0000 7f96dce00640 1 -- 172.21.15.122:0/1801840600 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone25105", "target": ["mon-mgr", ""]}) -- 0x7f96d80ce810 con 0x7f96b4078c00 2024-09-17T05:35:36.446 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:36.438+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:36.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.462+0000 7f96ce000640 1 -- 172.21.15.122:0/1801840600 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f96d80ce810 con 0x7f96b4078c00 2024-09-17T05:35:36.459 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:36.480 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:35:36.481 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone67336 2024-09-17T05:35:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 -- 172.21.15.122:0/4007306609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b00cf300 msgr2=0x7fc0b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 --2- 172.21.15.122:0/4007306609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b00cf300 0x7fc0b006ede0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00b28f0 tx=0x7fc0a801ca80 comp rx=0 tx=0).stop 2024-09-17T05:35:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 -- 172.21.15.122:0/4007306609 shutdown_connections 2024-09-17T05:35:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 --2- 172.21.15.122:0/4007306609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b00cf300 0x7fc0b006ede0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 --2- 172.21.15.122:0/4007306609 >> v2:172.21.15.173:3300/0 conn(0x7fc0b00cc5f0 0x7fc0b006e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 -- 172.21.15.122:0/4007306609 >> 172.21.15.122:0/4007306609 conn(0x7fc0b00b18f0 msgr2=0x7fc0b00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 -- 172.21.15.122:0/4007306609 shutdown_connections 2024-09-17T05:35:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 -- 172.21.15.122:0/4007306609 wait complete. 2024-09-17T05:35:36.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.774+0000 7fc0b7e00640 1 Processor -- start 2024-09-17T05:35:36.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b7e00640 1 -- start start 2024-09-17T05:35:36.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b0070420 0x7fc0b014c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0b00cf300 0x7fc0b01fbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:36.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b0070420 0x7fc0b014c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b0070420 0x7fc0b014c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49942/0 (socket says 172.21.15.122:49942) 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0b01fc330 0x7fc0b00cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b7e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0168620 con 0x7fc0b0070420 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b7e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01684a0 con 0x7fc0b00cf300 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b7e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b01687a0 con 0x7fc0b01fc330 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.778+0000 7fc0b5400640 1 -- 172.21.15.122:0/2182234179 learned_addr learned my addr 172.21.15.122:0/2182234179 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b5400640 1 -- 172.21.15.122:0/2182234179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0b01fc330 msgr2=0x7fc0b00cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b5400640 1 --2- 172.21.15.122:0/2182234179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0b01fc330 0x7fc0b00cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b5400640 1 -- 172.21.15.122:0/2182234179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0b00cf300 msgr2=0x7fc0b01fbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b5400640 1 --2- 172.21.15.122:0/2182234179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0b00cf300 0x7fc0b01fbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:36.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b5400640 1 -- 172.21.15.122:0/2182234179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b00acec0 con 0x7fc0b0070420 2024-09-17T05:35:36.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b5400640 1 --2- 172.21.15.122:0/2182234179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b0070420 0x7fc0b014c320 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400c450 tx=0x7fc0a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:36.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0ad600640 1 -- 172.21.15.122:0/2182234179 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4016020 con 0x7fc0b0070420 2024-09-17T05:35:36.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0ad600640 1 -- 172.21.15.122:0/2182234179 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a4012920 con 0x7fc0b0070420 2024-09-17T05:35:36.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0ad600640 1 -- 172.21.15.122:0/2182234179 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4012d70 con 0x7fc0b0070420 2024-09-17T05:35:36.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b7e00640 1 -- 172.21.15.122:0/2182234179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b01f95a0 con 0x7fc0b0070420 2024-09-17T05:35:36.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b7e00640 1 -- 172.21.15.122:0/2182234179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b00cb710 con 0x7fc0b0070420 2024-09-17T05:35:36.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0b7e00640 1 -- 172.21.15.122:0/2182234179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b00cebf0 con 0x7fc0b0070420 2024-09-17T05:35:36.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.782+0000 7fc0ad600640 1 -- 172.21.15.122:0/2182234179 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 36) ==== 99223+0+0 (secure 0 0 0) 0x7fc0a4019020 con 0x7fc0b0070420 2024-09-17T05:35:36.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.786+0000 7fc0ad600640 1 --2- 172.21.15.122:0/2182234179 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0900747f0 0x7fc090076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:36.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.786+0000 7fc0ad600640 1 -- 172.21.15.122:0/2182234179 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fc0a4093030 con 0x7fc0b0070420 2024-09-17T05:35:36.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.794+0000 7fc0ad600640 1 -- 172.21.15.122:0/2182234179 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc0a4020030 con 0x7fc0b0070420 2024-09-17T05:35:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.834+0000 7fc0b4a00640 1 --2- 172.21.15.122:0/2182234179 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0900747f0 0x7fc090076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:36.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:36.866+0000 7fc0b4a00640 1 --2- 172.21.15.122:0/2182234179 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0900747f0 0x7fc090076cb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0143d90 tx=0x7fc09c009640 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:37.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:37.194+0000 7fc0b7e00640 1 -- 172.21.15.122:0/2182234179 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone67336", "target": ["mon-mgr", ""]}) -- 0x7fc0b00cf720 con 0x7fc0900747f0 2024-09-17T05:35:38.218 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:38.222+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:38.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.222+0000 7fc0ad600640 1 -- 172.21.15.122:0/2182234179 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fc0b00cf720 con 0x7fc0900747f0 2024-09-17T05:35:38.219 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:38.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:35:38.245 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone21727 2024-09-17T05:35:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.594+0000 7f98d5400640 1 -- 172.21.15.122:0/844618313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98b4004810 msgr2=0x7f98b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:38.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.594+0000 7f98d5400640 1 --2- 172.21.15.122:0/844618313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98b4004810 0x7f98b4004c90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f98b4005620 tx=0x7f98bc01ce60 comp rx=0 tx=0).stop 2024-09-17T05:35:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.594+0000 7f98d5400640 1 -- 172.21.15.122:0/844618313 shutdown_connections 2024-09-17T05:35:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.594+0000 7f98d5400640 1 --2- 172.21.15.122:0/844618313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98b4004810 0x7f98b4004c90 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:38.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.594+0000 7f98d5400640 1 -- 172.21.15.122:0/844618313 >> 172.21.15.122:0/844618313 conn(0x7f98d00d3dd0 msgr2=0x7f98d00d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.594+0000 7f98d5400640 1 -- 172.21.15.122:0/844618313 shutdown_connections 2024-09-17T05:35:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.594+0000 7f98d5400640 1 -- 172.21.15.122:0/844618313 wait complete. 2024-09-17T05:35:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.598+0000 7f98d5400640 1 Processor -- start 2024-09-17T05:35:38.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.598+0000 7f98d5400640 1 -- start start 2024-09-17T05:35:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.598+0000 7f98d5400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98b4004810 0x7f98d00c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.598+0000 7f98d5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d01fc750 0x7f98d00c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.598+0000 7f98d5400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d00c8af0 0x7f98d00b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.598+0000 7f98d5400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98d00ced60 con 0x7f98b4004810 2024-09-17T05:35:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.598+0000 7f98d5400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f98d00cebe0 con 0x7f98d01fc750 2024-09-17T05:35:38.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.598+0000 7f98d5400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f98d00ceee0 con 0x7f98d00c8af0 2024-09-17T05:35:38.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98b4004810 0x7f98d00c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:38.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98b4004810 0x7f98d00c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49948/0 (socket says 172.21.15.122:49948) 2024-09-17T05:35:38.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 -- 172.21.15.122:0/3921295553 learned_addr learned my addr 172.21.15.122:0/3921295553 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:38.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cf400640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d01fc750 0x7f98d00c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:38.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98d4a00640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d00c8af0 0x7f98d00b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:38.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 -- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d00c8af0 msgr2=0x7f98d00b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:38.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98d00c8af0 0x7f98d00b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:38.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 -- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d01fc750 msgr2=0x7f98d00c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:38.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d01fc750 0x7f98d00c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:38.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 -- 172.21.15.122:0/3921295553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d00c9350 con 0x7f98b4004810 2024-09-17T05:35:38.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cf400640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98d01fc750 0x7f98d00c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:38.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98cfe00640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98b4004810 0x7f98d00c7d70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c4d0 tx=0x7f98c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:38.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98c7e00640 1 -- 172.21.15.122:0/3921295553 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00052d0 con 0x7f98b4004810 2024-09-17T05:35:38.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98c7e00640 1 -- 172.21.15.122:0/3921295553 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98c0005490 con 0x7f98b4004810 2024-09-17T05:35:38.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98c7e00640 1 -- 172.21.15.122:0/3921295553 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0012870 con 0x7f98b4004810 2024-09-17T05:35:38.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98d5400640 1 -- 172.21.15.122:0/3921295553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d00c9070 con 0x7f98b4004810 2024-09-17T05:35:38.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.602+0000 7f98d5400640 1 -- 172.21.15.122:0/3921295553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d01fbf50 con 0x7f98b4004810 2024-09-17T05:35:38.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.606+0000 7f98d5400640 1 -- 172.21.15.122:0/3921295553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d00cf440 con 0x7f98b4004810 2024-09-17T05:35:38.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.606+0000 7f98c7e00640 1 -- 172.21.15.122:0/3921295553 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 36) ==== 99223+0+0 (secure 0 0 0) 0x7f98c0003600 con 0x7f98b4004810 2024-09-17T05:35:38.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.610+0000 7f98c7e00640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b0074820 0x7f98b0076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:38.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.610+0000 7f98c7e00640 1 -- 172.21.15.122:0/3921295553 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f98c0093400 con 0x7f98b4004810 2024-09-17T05:35:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.618+0000 7f98c7e00640 1 -- 172.21.15.122:0/3921295553 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f98c0020060 con 0x7f98b4004810 2024-09-17T05:35:38.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.670+0000 7f98cf400640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b0074820 0x7f98b0076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:38.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.686+0000 7f98cf400640 1 --2- 172.21.15.122:0/3921295553 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f98b0074820 0x7f98b0076ce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0ac000 tx=0x7f98bc019ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:38.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:38.942+0000 7f98d5400640 1 -- 172.21.15.122:0/3921295553 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone21727", "target": ["mon-mgr", ""]}) -- 0x7f98d01fbac0 con 0x7f98b0074820 2024-09-17T05:35:39.080 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:35:39.078+0000 7fc098c00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.086+0000 7f98c7e00640 1 -- 172.21.15.122:0/3921295553 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f98d01fbac0 con 0x7f98b0074820 2024-09-17T05:35:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-17T05:35:39.102 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T05:35:39.102 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone59558 2024-09-17T05:35:39.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 -- 172.21.15.122:0/858673495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f4070960 msgr2=0x7fd5f40cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:39.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 --2- 172.21.15.122:0/858673495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f4070960 0x7fd5f40cc740 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009900 tx=0x7fd5dc01c850 comp rx=0 tx=0).stop 2024-09-17T05:35:39.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 -- 172.21.15.122:0/858673495 shutdown_connections 2024-09-17T05:35:39.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 --2- 172.21.15.122:0/858673495 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5f40ccc80 0x7fd5f406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:39.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 --2- 172.21.15.122:0/858673495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f4070960 0x7fd5f40cc740 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:39.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 -- 172.21.15.122:0/858673495 >> 172.21.15.122:0/858673495 conn(0x7fd5f40b18f0 msgr2=0x7fd5f40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:39.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 -- 172.21.15.122:0/858673495 shutdown_connections 2024-09-17T05:35:39.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 -- 172.21.15.122:0/858673495 wait complete. 2024-09-17T05:35:39.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.346+0000 7fd5fb400640 1 Processor -- start 2024-09-17T05:35:39.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5fb400640 1 -- start start 2024-09-17T05:35:39.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5fb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f406ece0 0x7fd5f4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:39.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5fb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5f4070960 0x7fd5f41f8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:39.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5fb400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5f40ccc80 0x7fd5f4148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:39.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5fb400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f40b47d0 con 0x7fd5f406ece0 2024-09-17T05:35:39.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5fb400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f40b4650 con 0x7fd5f4070960 2024-09-17T05:35:39.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5fb400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f40b4950 con 0x7fd5f40ccc80 2024-09-17T05:35:39.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5f8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f406ece0 0x7fd5f4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:39.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5f8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f406ece0 0x7fd5f4072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49962/0 (socket says 172.21.15.122:49962) 2024-09-17T05:35:39.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.350+0000 7fd5f8a00640 1 -- 172.21.15.122:0/2874620776 learned_addr learned my addr 172.21.15.122:0/2874620776 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:39.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.354+0000 7fd5ebe00640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5f4070960 0x7fd5f41f8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:39.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.354+0000 7fd5f8a00640 1 -- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5f40ccc80 msgr2=0x7fd5f4148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:39.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.354+0000 7fd5f8a00640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5f40ccc80 0x7fd5f4148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:39.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.354+0000 7fd5f8a00640 1 -- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5f4070960 msgr2=0x7fd5f41f8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:39.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.354+0000 7fd5f8a00640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5f4070960 0x7fd5f41f8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:39.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.354+0000 7fd5f8a00640 1 -- 172.21.15.122:0/2874620776 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f4149250 con 0x7fd5f406ece0 2024-09-17T05:35:39.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.354+0000 7fd5f8a00640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f406ece0 0x7fd5f4072be0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc01cd30 tx=0x7fd5dc0a6610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:39.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.358+0000 7fd5f1600640 1 -- 172.21.15.122:0/2874620776 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0ac070 con 0x7fd5f406ece0 2024-09-17T05:35:39.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.358+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f40732b0 con 0x7fd5f406ece0 2024-09-17T05:35:39.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.358+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f414f860 con 0x7fd5f406ece0 2024-09-17T05:35:39.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.358+0000 7fd5f1600640 1 -- 172.21.15.122:0/2874620776 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5dc0a66a0 con 0x7fd5f406ece0 2024-09-17T05:35:39.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.358+0000 7fd5f1600640 1 -- 172.21.15.122:0/2874620776 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0bf060 con 0x7fd5f406ece0 2024-09-17T05:35:39.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.358+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f4069410 con 0x7fd5f406ece0 2024-09-17T05:35:39.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.362+0000 7fd5f1600640 1 -- 172.21.15.122:0/2874620776 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 36) ==== 99223+0+0 (secure 0 0 0) 0x7fd5dc0af070 con 0x7fd5f406ece0 2024-09-17T05:35:39.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.366+0000 7fd5f1600640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5d00747f0 0x7fd5d0076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:39.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.366+0000 7fd5f1600640 1 -- 172.21.15.122:0/2874620776 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fd5dc12d260 con 0x7fd5f406ece0 2024-09-17T05:35:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.374+0000 7fd5ebe00640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5d00747f0 0x7fd5d0076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:39.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.374+0000 7fd5f1600640 1 -- 172.21.15.122:0/2874620776 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd5dc0b6070 con 0x7fd5f406ece0 2024-09-17T05:35:39.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.402+0000 7fd5ebe00640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5d00747f0 0x7fd5d0076cb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:39.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:39.782+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone59558", "target": ["mon-mgr", ""]}) -- 0x7fd5f40cc740 con 0x7fd5d00747f0 2024-09-17T05:35:41.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.178+0000 7fd5f1600640 1 -- 172.21.15.122:0/2874620776 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5f40cc740 con 0x7fd5d00747f0 2024-09-17T05:35:41.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5d00747f0 msgr2=0x7fd5d0076cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:41.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5d00747f0 0x7fd5d0076cb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4008040 comp rx=0 tx=0).stop 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f406ece0 msgr2=0x7fd5f4072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f406ece0 0x7fd5f4072be0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc01cd30 tx=0x7fd5dc0a6610 comp rx=0 tx=0).stop 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 shutdown_connections 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd5d00747f0 0x7fd5d0076cb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5f40ccc80 0x7fd5f4148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5f4070960 0x7fd5f41f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 --2- 172.21.15.122:0/2874620776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5f406ece0 0x7fd5f4072be0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 >> 172.21.15.122:0/2874620776 conn(0x7fd5f40b18f0 msgr2=0x7fd5f40b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 shutdown_connections 2024-09-17T05:35:41.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.182+0000 7fd5fb400640 1 -- 172.21.15.122:0/2874620776 wait complete. 2024-09-17T05:35:41.214 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone44559 2024-09-17T05:35:41.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.482+0000 7fd6d4000640 1 -- 172.21.15.122:0/3169190739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc0cc740 msgr2=0x7fd6cc0ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:41.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.482+0000 7fd6d4000640 1 --2- 172.21.15.122:0/3169190739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc0cc740 0x7fd6cc0ccb80 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc076940 tx=0x7fd6c001c830 comp rx=0 tx=0).stop 2024-09-17T05:35:41.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.482+0000 7fd6d4000640 1 -- 172.21.15.122:0/3169190739 shutdown_connections 2024-09-17T05:35:41.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.482+0000 7fd6d4000640 1 --2- 172.21.15.122:0/3169190739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc0cc740 0x7fd6cc0ccb80 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:41.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.482+0000 7fd6d4000640 1 --2- 172.21.15.122:0/3169190739 >> v2:172.21.15.173:3300/0 conn(0x7fd6cc06f8c0 0x7fd6cc06fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:41.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.482+0000 7fd6d4000640 1 -- 172.21.15.122:0/3169190739 >> 172.21.15.122:0/3169190739 conn(0x7fd6cc0b18f0 msgr2=0x7fd6cc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:41.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.482+0000 7fd6d4000640 1 -- 172.21.15.122:0/3169190739 shutdown_connections 2024-09-17T05:35:41.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.486+0000 7fd6d4000640 1 -- 172.21.15.122:0/3169190739 wait complete. 2024-09-17T05:35:41.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d4000640 1 Processor -- start 2024-09-17T05:35:41.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d4000640 1 -- start start 2024-09-17T05:35:41.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d4000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6cc06e7a0 0x7fd6cc1ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:41.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d4000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc06f8c0 0x7fd6cc13d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:41.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d4000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6cc0cc740 0x7fd6cc158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:41.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d4000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc165960 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d4000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1657e0 con 0x7fd6cc06e7a0 2024-09-17T05:35:41.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d4000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd6cc165ae0 con 0x7fd6cc0cc740 2024-09-17T05:35:41.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc06f8c0 0x7fd6cc13d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:41.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc06f8c0 0x7fd6cc13d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58986/0 (socket says 172.21.15.122:58986) 2024-09-17T05:35:41.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 -- 172.21.15.122:0/226625292 learned_addr learned my addr 172.21.15.122:0/226625292 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 -- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6cc0cc740 msgr2=0x7fd6cc158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6cc0cc740 0x7fd6cc158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 -- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6cc06e7a0 msgr2=0x7fd6cc1ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6cc06e7a0 0x7fd6cc1ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 -- 172.21.15.122:0/226625292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc072870 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.490+0000 7fd6d0c00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc06f8c0 0x7fd6cc13d530 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00b0040 tx=0x7fd6c00a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.494+0000 7fd6c9600640 1 -- 172.21.15.122:0/226625292 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00b2040 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.494+0000 7fd6c9600640 1 -- 172.21.15.122:0/226625292 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6c0004430 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.494+0000 7fd6c9600640 1 -- 172.21.15.122:0/226625292 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0003860 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.494+0000 7fd6d4000640 1 -- 172.21.15.122:0/226625292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc2005f0 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.494+0000 7fd6d4000640 1 -- 172.21.15.122:0/226625292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc200bb0 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.494+0000 7fd6bea00640 1 -- 172.21.15.122:0/226625292 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a00052a0 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.498+0000 7fd6c9600640 1 -- 172.21.15.122:0/226625292 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 36) ==== 99223+0+0 (secure 0 0 0) 0x7fd6c0003a00 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.498+0000 7fd6c9600640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd6ac0747f0 0x7fd6ac076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:41.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.498+0000 7fd6c9600640 1 -- 172.21.15.122:0/226625292 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fd6c012d550 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.502+0000 7fd6c9600640 1 -- 172.21.15.122:0/226625292 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd6c00164b0 con 0x7fd6cc06f8c0 2024-09-17T05:35:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.534+0000 7fd6d1600640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd6ac0747f0 0x7fd6ac076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:41.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.546+0000 7fd6d1600640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd6ac0747f0 0x7fd6ac076cb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0742b0 tx=0x7fd6c4009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:41.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:41.930+0000 7fd6bea00640 1 -- 172.21.15.122:0/226625292 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone44559", "target": ["mon-mgr", ""]}) -- 0x7fd6a0000bd0 con 0x7fd6ac0747f0 2024-09-17T05:35:43.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.186+0000 7fd6c9600640 1 -- 172.21.15.122:0/226625292 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd6a0000bd0 con 0x7fd6ac0747f0 2024-09-17T05:35:43.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 -- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd6ac0747f0 msgr2=0x7fd6ac076cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:43.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd6ac0747f0 0x7fd6ac076cb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0742b0 tx=0x7fd6c4009250 comp rx=0 tx=0).stop 2024-09-17T05:35:43.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 -- 172.21.15.122:0/226625292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc06f8c0 msgr2=0x7fd6cc13d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:43.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc06f8c0 0x7fd6cc13d530 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00b0040 tx=0x7fd6c00a5710 comp rx=0 tx=0).stop 2024-09-17T05:35:43.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 -- 172.21.15.122:0/226625292 shutdown_connections 2024-09-17T05:35:43.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd6ac0747f0 0x7fd6ac076cb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6cc0cc740 0x7fd6cc158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6cc06f8c0 0x7fd6cc13d530 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 --2- 172.21.15.122:0/226625292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6cc06e7a0 0x7fd6cc1ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 -- 172.21.15.122:0/226625292 >> 172.21.15.122:0/226625292 conn(0x7fd6cc0b18f0 msgr2=0x7fd6cc141c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 -- 172.21.15.122:0/226625292 shutdown_connections 2024-09-17T05:35:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.190+0000 7fd6bea00640 1 -- 172.21.15.122:0/226625292 wait complete. 2024-09-17T05:35:43.223 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone8110 2024-09-17T05:35:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- 172.21.15.122:0/3711442899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5aac003a10 msgr2=0x7f5aac003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 --2- 172.21.15.122:0/3711442899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5aac003a10 0x7f5aac003e30 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40d1970 tx=0x7f5ab801c5f0 comp rx=0 tx=0).stop 2024-09-17T05:35:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5aca600640 1 -- 172.21.15.122:0/3711442899 reap_dead start 2024-09-17T05:35:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- 172.21.15.122:0/3711442899 shutdown_connections 2024-09-17T05:35:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- 172.21.15.122:0/3711442899 >> 172.21.15.122:0/3711442899 conn(0x7f5ac40d0cd0 msgr2=0x7f5ac40d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- 172.21.15.122:0/3711442899 shutdown_connections 2024-09-17T05:35:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- 172.21.15.122:0/3711442899 wait complete. 2024-09-17T05:35:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 Processor -- start 2024-09-17T05:35:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- start start 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aac003a10 0x7f5ac41fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac4160c60 0x7f5ac41610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ac415e6e0 0x7f5ac415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac40cfe60 con 0x7f5ac415e6e0 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac40cfce0 con 0x7f5aac003a10 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5acc600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac40cb530 con 0x7f5ac4160c60 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5aca600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ac415e6e0 0x7f5ac415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5aca600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ac415e6e0 0x7f5ac415eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59004/0 (socket says 172.21.15.122:59004) 2024-09-17T05:35:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.502+0000 7f5aca600640 1 -- 172.21.15.122:0/261421838 learned_addr learned my addr 172.21.15.122:0/261421838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5aca600640 1 -- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac4160c60 msgr2=0x7f5ac41610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5aca600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac4160c60 0x7f5ac41610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5aca600640 1 -- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aac003a10 msgr2=0x7f5ac41fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5aca600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aac003a10 0x7f5ac41fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5aca600640 1 -- 172.21.15.122:0/261421838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac415f450 con 0x7f5ac415e6e0 2024-09-17T05:35:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5aca600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ac415e6e0 0x7f5ac415eb60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004a60 tx=0x7f5abc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5ac1600640 1 -- 172.21.15.122:0/261421838 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc017070 con 0x7f5ac415e6e0 2024-09-17T05:35:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5ac1600640 1 -- 172.21.15.122:0/261421838 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5abc00ab10 con 0x7f5ac415e6e0 2024-09-17T05:35:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac415f730 con 0x7f5ac415e6e0 2024-09-17T05:35:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5ac1600640 1 -- 172.21.15.122:0/261421838 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc007070 con 0x7f5ac415e6e0 2024-09-17T05:35:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac415fcf0 con 0x7f5ac415e6e0 2024-09-17T05:35:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.506+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac40ad8d0 con 0x7f5ac415e6e0 2024-09-17T05:35:43.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.510+0000 7f5ac1600640 1 -- 172.21.15.122:0/261421838 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 36) ==== 99223+0+0 (secure 0 0 0) 0x7f5abc01a070 con 0x7f5ac415e6e0 2024-09-17T05:35:43.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.510+0000 7f5ac1600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5aa4074870 0x7f5aa4076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:43.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.510+0000 7f5ac1600640 1 -- 172.21.15.122:0/261421838 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f5abc092c80 con 0x7f5ac415e6e0 2024-09-17T05:35:43.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.514+0000 7f5ac9c00640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5aa4074870 0x7f5aa4076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:43.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.518+0000 7f5ac1600640 1 -- 172.21.15.122:0/261421838 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5abc05f3b0 con 0x7f5ac415e6e0 2024-09-17T05:35:43.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.558+0000 7f5ac9c00640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5aa4074870 0x7f5aa4076d30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40ce850 tx=0x7f5aac005840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:43.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:43.974+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone8110", "target": ["mon-mgr", ""]}) -- 0x7f5ac40cdb30 con 0x7f5aa4074870 2024-09-17T05:35:44.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:44.934+0000 7f5ac1600640 1 -- 172.21.15.122:0/261421838 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f5abc05af20 con 0x7f5ac415e6e0 2024-09-17T05:35:45.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.010+0000 7f5ac1600640 1 -- 172.21.15.122:0/261421838 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ac40cdb30 con 0x7f5aa4074870 2024-09-17T05:35:45.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5aa4074870 msgr2=0x7f5aa4076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:45.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5aa4074870 0x7f5aa4076d30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40ce850 tx=0x7f5aac005840 comp rx=0 tx=0).stop 2024-09-17T05:35:45.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ac415e6e0 msgr2=0x7f5ac415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:45.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ac415e6e0 0x7f5ac415eb60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004a60 tx=0x7f5abc00ccc0 comp rx=0 tx=0).stop 2024-09-17T05:35:45.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 shutdown_connections 2024-09-17T05:35:45.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5aa4074870 0x7f5aa4076d30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:45.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ac415e6e0 0x7f5ac415eb60 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:45.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac4160c60 0x7f5ac41610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:45.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 --2- 172.21.15.122:0/261421838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aac003a10 0x7f5ac41fa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:45.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 >> 172.21.15.122:0/261421838 conn(0x7f5ac40d0cd0 msgr2=0x7f5ac40b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:45.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 shutdown_connections 2024-09-17T05:35:45.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.014+0000 7f5acc600640 1 -- 172.21.15.122:0/261421838 wait complete. 2024-09-17T05:35:45.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone610611 2024-09-17T05:35:45.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.346+0000 7f900a600640 1 -- 172.21.15.122:0/1355099571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe8004d50 msgr2=0x7f8fe80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:45.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.346+0000 7f900a600640 1 --2- 172.21.15.122:0/1355099571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fe8004d50 0x7f8fe80051d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f900415fb50 tx=0x7f8ffc01c930 comp rx=0 tx=0).stop 2024-09-17T05:35:45.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.346+0000 7f9009c00640 1 -- 172.21.15.122:0/1355099571 reap_dead start 2024-09-17T05:35:45.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.346+0000 7f900a600640 1 -- 172.21.15.122:0/1355099571 shutdown_connections 2024-09-17T05:35:45.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.346+0000 7f900a600640 1 -- 172.21.15.122:0/1355099571 >> 172.21.15.122:0/1355099571 conn(0x7f90040cfa00 msgr2=0x7f90040cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:45.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.350+0000 7f900a600640 1 -- 172.21.15.122:0/1355099571 shutdown_connections 2024-09-17T05:35:45.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.350+0000 7f900a600640 1 -- 172.21.15.122:0/1355099571 wait complete. 2024-09-17T05:35:45.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.350+0000 7f900a600640 1 Processor -- start 2024-09-17T05:35:45.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.354+0000 7f900a600640 1 -- start start 2024-09-17T05:35:45.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.354+0000 7f900a600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe8004d50 0x7f9004149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:45.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.354+0000 7f900a600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90041f8780 0x7f90041f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:45.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.354+0000 7f900a600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90041f9120 0x7f90040c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:45.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.354+0000 7f900a600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f90040ca1d0 con 0x7f90041f9120 2024-09-17T05:35:45.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.354+0000 7f900a600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90040ca050 con 0x7f90041f8780 2024-09-17T05:35:45.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.354+0000 7f900a600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f90040ca350 con 0x7f8fe8004d50 2024-09-17T05:35:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9003e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90041f8780 0x7f90041f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90041f9120 0x7f90040c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90041f9120 0x7f90040c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59026/0 (socket says 172.21.15.122:59026) 2024-09-17T05:35:45.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009c00640 1 -- 172.21.15.122:0/325039700 learned_addr learned my addr 172.21.15.122:0/325039700 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:45.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009200640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe8004d50 0x7f9004149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:45.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009c00640 1 -- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe8004d50 msgr2=0x7f9004149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:45.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009c00640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe8004d50 0x7f9004149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:45.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009c00640 1 -- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90041f8780 msgr2=0x7f90041f8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:45.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009c00640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90041f8780 0x7f90041f8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:45.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.358+0000 7f9009c00640 1 -- 172.21.15.122:0/325039700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90041f95a0 con 0x7f90041f9120 2024-09-17T05:35:45.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.362+0000 7f9009c00640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90041f9120 0x7f90040c3530 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc002240 tx=0x7f8ffc005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:45.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.366+0000 7f9000c00640 1 -- 172.21.15.122:0/325039700 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0b2040 con 0x7f90041f9120 2024-09-17T05:35:45.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.366+0000 7f9000c00640 1 -- 172.21.15.122:0/325039700 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ffc0ad550 con 0x7f90041f9120 2024-09-17T05:35:45.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.366+0000 7f9000c00640 1 -- 172.21.15.122:0/325039700 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0ad9a0 con 0x7f90041f9120 2024-09-17T05:35:45.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.366+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90040b5530 con 0x7f90041f9120 2024-09-17T05:35:45.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.366+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9004153c40 con 0x7f90041f9120 2024-09-17T05:35:45.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.366+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900415e300 con 0x7f90041f9120 2024-09-17T05:35:45.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.378+0000 7f9000c00640 1 -- 172.21.15.122:0/325039700 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f8ffc0adb40 con 0x7f90041f9120 2024-09-17T05:35:45.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.382+0000 7f9000c00640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fe4074860 0x7f8fe4076d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:45.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.382+0000 7f9000c00640 1 -- 172.21.15.122:0/325039700 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f8ffc12ca80 con 0x7f90041f9120 2024-09-17T05:35:45.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.382+0000 7f9000c00640 1 -- 172.21.15.122:0/325039700 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8ffc0bf060 con 0x7f90041f9120 2024-09-17T05:35:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.386+0000 7f9009200640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fe4074860 0x7f8fe4076d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:45.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.398+0000 7f9009200640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fe4074860 0x7f8fe4076d20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009d00 tx=0x7f8ff40091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:45.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:45.810+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone610611", "target": ["mon-mgr", ""]}) -- 0x7f90040b5810 con 0x7f8fe4074860 2024-09-17T05:35:47.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.882+0000 7f9000c00640 1 -- 172.21.15.122:0/325039700 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90040b5810 con 0x7f8fe4074860 2024-09-17T05:35:47.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fe4074860 msgr2=0x7f8fe4076d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fe4074860 0x7f8fe4076d20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009d00 tx=0x7f8ff40091e0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90041f9120 msgr2=0x7f90040c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90041f9120 0x7f90040c3530 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc002240 tx=0x7f8ffc005b70 comp rx=0 tx=0).stop 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 shutdown_connections 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8fe4074860 0x7f8fe4076d20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90041f9120 0x7f90040c3530 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90041f8780 0x7f90041f8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 --2- 172.21.15.122:0/325039700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8fe8004d50 0x7f9004149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:47.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 >> 172.21.15.122:0/325039700 conn(0x7f90040cfa00 msgr2=0x7f90040cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:47.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 shutdown_connections 2024-09-17T05:35:47.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:47.890+0000 7f900a600640 1 -- 172.21.15.122:0/325039700 wait complete. 2024-09-17T05:35:47.946 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone82612 2024-09-17T05:35:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.246+0000 7f7d7f200640 1 -- 172.21.15.122:0/1727941796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d80159c10 msgr2=0x7f7d8006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.246+0000 7f7d7f200640 1 --2- 172.21.15.122:0/1727941796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d80159c10 0x7f7d8006ebf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7d800b4aa0 tx=0x7f7d7001c450 comp rx=0 tx=0).stop 2024-09-17T05:35:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.246+0000 7f7d7f200640 1 -- 172.21.15.122:0/1727941796 shutdown_connections 2024-09-17T05:35:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.246+0000 7f7d7f200640 1 --2- 172.21.15.122:0/1727941796 >> v2:172.21.15.173:3300/0 conn(0x7f7d8006aba0 0x7f7d8015bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.246+0000 7f7d7f200640 1 --2- 172.21.15.122:0/1727941796 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d80159c10 0x7f7d8006ebf0 secure :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7d800b4aa0 tx=0x7f7d7001c450 comp rx=0 tx=0).stop 2024-09-17T05:35:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.246+0000 7f7d7f200640 1 -- 172.21.15.122:0/1727941796 >> 172.21.15.122:0/1727941796 conn(0x7f7d8006cc20 msgr2=0x7f7d8006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.246+0000 7f7d7f200640 1 -- 172.21.15.122:0/1727941796 shutdown_connections 2024-09-17T05:35:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.246+0000 7f7d7f200640 1 -- 172.21.15.122:0/1727941796 wait complete. 2024-09-17T05:35:48.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.254+0000 7f7d7f200640 1 Processor -- start 2024-09-17T05:35:48.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.254+0000 7f7d7f200640 1 -- start start 2024-09-17T05:35:48.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8006aba0 0x7f7d801f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d8014c280 0x7f7d8014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7f200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d801f3ea0 0x7f7d801f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8006aba0 0x7f7d801f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8006aba0 0x7f7d801f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59048/0 (socket says 172.21.15.122:59048) 2024-09-17T05:35:48.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7de00640 1 -- 172.21.15.122:0/3747872337 learned_addr learned my addr 172.21.15.122:0/3747872337 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:48.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801670d0 con 0x7f7d8006aba0 2024-09-17T05:35:48.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80166f50 con 0x7f7d8014c280 2024-09-17T05:35:48.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7d80167250 con 0x7f7d801f3ea0 2024-09-17T05:35:48.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7e800640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d801f3ea0 0x7f7d801f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.258+0000 7f7d7d400640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d8014c280 0x7f7d8014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.266+0000 7f7d7de00640 1 -- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d801f3ea0 msgr2=0x7f7d801f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.266+0000 7f7d7de00640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d801f3ea0 0x7f7d801f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.266+0000 7f7d7de00640 1 -- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d8014c280 msgr2=0x7f7d8014c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:48.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.266+0000 7f7d7de00640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d8014c280 0x7f7d8014c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:48.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.266+0000 7f7d7de00640 1 -- 172.21.15.122:0/3747872337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d801f4860 con 0x7f7d8006aba0 2024-09-17T05:35:48.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.270+0000 7f7d7de00640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8006aba0 0x7f7d801f6ca0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f7d7001a080 tx=0x7f7d70005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:48.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.274+0000 7f7d76000640 1 -- 172.21.15.122:0/3747872337 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700b8020 con 0x7f7d8006aba0 2024-09-17T05:35:48.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.274+0000 7f7d76000640 1 -- 172.21.15.122:0/3747872337 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d700042b0 con 0x7f7d8006aba0 2024-09-17T05:35:48.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.274+0000 7f7d76000640 1 -- 172.21.15.122:0/3747872337 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700b0040 con 0x7f7d8006aba0 2024-09-17T05:35:48.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.278+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d801f71e0 con 0x7f7d8006aba0 2024-09-17T05:35:48.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.278+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8014cdd0 con 0x7f7d8006aba0 2024-09-17T05:35:48.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.278+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8015bad0 con 0x7f7d8006aba0 2024-09-17T05:35:48.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.286+0000 7f7d76000640 1 -- 172.21.15.122:0/3747872337 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f7d700bb070 con 0x7f7d8006aba0 2024-09-17T05:35:48.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.290+0000 7f7d76000640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7d60074830 0x7f7d60076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:48.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.290+0000 7f7d76000640 1 -- 172.21.15.122:0/3747872337 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f7d7012bcb0 con 0x7f7d8006aba0 2024-09-17T05:35:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.290+0000 7f7d7d400640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7d60074830 0x7f7d60076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.294+0000 7f7d76000640 1 -- 172.21.15.122:0/3747872337 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7d801f4860 con 0x7f7d8006aba0 2024-09-17T05:35:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.318+0000 7f7d7d400640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7d60074830 0x7f7d60076cf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000c10 tx=0x7f7d6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:48.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:48.678+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone82612", "target": ["mon-mgr", ""]}) -- 0x7f7d80050fb0 con 0x7f7d60074830 2024-09-17T05:35:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d76000640 1 -- 172.21.15.122:0/3747872337 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7d80050fb0 con 0x7f7d60074830 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7d60074830 msgr2=0x7f7d60076cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7d60074830 0x7f7d60076cf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000c10 tx=0x7f7d6c009040 comp rx=0 tx=0).stop 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8006aba0 msgr2=0x7f7d801f6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8006aba0 0x7f7d801f6ca0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f7d7001a080 tx=0x7f7d70005e70 comp rx=0 tx=0).stop 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 shutdown_connections 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7d60074830 0x7f7d60076cf0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d801f3ea0 0x7f7d801f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d8014c280 0x7f7d8014c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 --2- 172.21.15.122:0/3747872337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8006aba0 0x7f7d801f6ca0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.538+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 >> 172.21.15.122:0/3747872337 conn(0x7f7d8006cc20 msgr2=0x7f7d800733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:49.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.542+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 shutdown_connections 2024-09-17T05:35:49.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.542+0000 7f7d7f200640 1 -- 172.21.15.122:0/3747872337 wait complete. 2024-09-17T05:35:49.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone281413 2024-09-17T05:35:49.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.854+0000 7f151e400640 1 -- 172.21.15.122:0/3570364102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f151806f520 msgr2=0x7f15180ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:49.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.854+0000 7f151e400640 1 --2- 172.21.15.122:0/3570364102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f151806f520 0x7f15180ce6f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f1518076400 tx=0x7f151001f370 comp rx=0 tx=0).stop 2024-09-17T05:35:49.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.854+0000 7f151e400640 1 -- 172.21.15.122:0/3570364102 shutdown_connections 2024-09-17T05:35:49.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.854+0000 7f151e400640 1 --2- 172.21.15.122:0/3570364102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f151806f520 0x7f15180ce6f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:49.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.854+0000 7f151e400640 1 --2- 172.21.15.122:0/3570364102 >> v2:172.21.15.173:3300/0 conn(0x7f151806f140 0x7f15180738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:49.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.854+0000 7f151e400640 1 -- 172.21.15.122:0/3570364102 >> 172.21.15.122:0/3570364102 conn(0x7f15180cfa00 msgr2=0x7f15180cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:49.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.854+0000 7f151e400640 1 -- 172.21.15.122:0/3570364102 shutdown_connections 2024-09-17T05:35:49.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.854+0000 7f151e400640 1 -- 172.21.15.122:0/3570364102 wait complete. 2024-09-17T05:35:49.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.858+0000 7f151e400640 1 Processor -- start 2024-09-17T05:35:49.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.858+0000 7f151e400640 1 -- start start 2024-09-17T05:35:49.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151e400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151806f140 0x7f15180c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:49.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151e400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f151806f520 0x7f15180c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:49.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151e400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180d0640 0x7f15180b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:49.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151e400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15180aec20 con 0x7f15180d0640 2024-09-17T05:35:49.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151e400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15180aeaa0 con 0x7f151806f520 2024-09-17T05:35:49.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151e400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f151806e650 con 0x7f151806f140 2024-09-17T05:35:49.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180d0640 0x7f15180b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:49.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180d0640 0x7f15180b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59062/0 (socket says 172.21.15.122:59062) 2024-09-17T05:35:49.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 -- 172.21.15.122:0/2815803418 learned_addr learned my addr 172.21.15.122:0/2815803418 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:49.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 -- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151806f140 msgr2=0x7f15180c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:49.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151806f140 0x7f15180c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:49.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 -- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f151806f520 msgr2=0x7f15180c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:49.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f151806f520 0x7f15180c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:49.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 -- 172.21.15.122:0/2815803418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15180b3f10 con 0x7f15180d0640 2024-09-17T05:35:49.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151da00640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180d0640 0x7f15180b31c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f1510009b10 tx=0x7f15100af560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:49.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f1515600640 1 -- 172.21.15.122:0/2815803418 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15100b4040 con 0x7f15180d0640 2024-09-17T05:35:49.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f1515600640 1 -- 172.21.15.122:0/2815803418 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15100bf070 con 0x7f15180d0640 2024-09-17T05:35:49.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f1515600640 1 -- 172.21.15.122:0/2815803418 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15100b0c10 con 0x7f15180d0640 2024-09-17T05:35:49.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15180b41f0 con 0x7f15180d0640 2024-09-17T05:35:49.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.862+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15180b47b0 con 0x7f15180d0640 2024-09-17T05:35:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.866+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151806eb70 con 0x7f15180d0640 2024-09-17T05:35:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.870+0000 7f1515600640 1 -- 172.21.15.122:0/2815803418 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f1510003fe0 con 0x7f15180d0640 2024-09-17T05:35:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.870+0000 7f1515600640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f4074760 0x7f14f4076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.870+0000 7f1515600640 1 -- 172.21.15.122:0/2815803418 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f1510131d60 con 0x7f15180d0640 2024-09-17T05:35:49.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.874+0000 7f1515600640 1 -- 172.21.15.122:0/2815803418 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1510018ed0 con 0x7f15180d0640 2024-09-17T05:35:49.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.886+0000 7f151d000640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f4074760 0x7f14f4076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:49.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:49.930+0000 7f151d000640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f4074760 0x7f14f4076c20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1500009730 tx=0x7f1500006ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:50.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:50.286+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone281413", "target": ["mon-mgr", ""]}) -- 0x7f15180b2ae0 con 0x7f14f4074760 2024-09-17T05:35:51.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f1515600640 1 -- 172.21.15.122:0/2815803418 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15180b2ae0 con 0x7f14f4074760 2024-09-17T05:35:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f4074760 msgr2=0x7f14f4076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f4074760 0x7f14f4076c20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1500009730 tx=0x7f1500006ca0 comp rx=0 tx=0).stop 2024-09-17T05:35:51.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180d0640 msgr2=0x7f15180b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180d0640 0x7f15180b31c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f1510009b10 tx=0x7f15100af560 comp rx=0 tx=0).stop 2024-09-17T05:35:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 shutdown_connections 2024-09-17T05:35:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f4074760 0x7f14f4076c20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180d0640 0x7f15180b31c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f151806f520 0x7f15180c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:51.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 --2- 172.21.15.122:0/2815803418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151806f140 0x7f15180c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.902+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 >> 172.21.15.122:0/2815803418 conn(0x7f15180cfa00 msgr2=0x7f15181ea400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:51.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.906+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 shutdown_connections 2024-09-17T05:35:51.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:51.906+0000 7f151e400640 1 -- 172.21.15.122:0/2815803418 wait complete. 2024-09-17T05:35:51.942 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone479614 2024-09-17T05:35:52.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.210+0000 7ffabe400640 1 -- 172.21.15.122:0/1232688934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa9c004e70 msgr2=0x7ffa9c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:52.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.210+0000 7ffabe400640 1 --2- 172.21.15.122:0/1232688934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa9c004e70 0x7ffa9c00b9d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ffab0001dc0 tx=0x7ffab001cb30 comp rx=0 tx=0).stop 2024-09-17T05:35:52.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.210+0000 7ffab7e00640 1 -- 172.21.15.122:0/1232688934 reap_dead start 2024-09-17T05:35:52.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.210+0000 7ffabe400640 1 -- 172.21.15.122:0/1232688934 shutdown_connections 2024-09-17T05:35:52.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.210+0000 7ffabe400640 1 -- 172.21.15.122:0/1232688934 >> 172.21.15.122:0/1232688934 conn(0x7ffab80b18f0 msgr2=0x7ffab80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:52.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.210+0000 7ffabe400640 1 -- 172.21.15.122:0/1232688934 shutdown_connections 2024-09-17T05:35:52.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.210+0000 7ffabe400640 1 -- 172.21.15.122:0/1232688934 wait complete. 2024-09-17T05:35:52.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffabe400640 1 Processor -- start 2024-09-17T05:35:52.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffabe400640 1 -- start start 2024-09-17T05:35:52.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffabe400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffab81522f0 0x7ffab81f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:52.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffabe400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffab81526d0 0x7ffab81f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:52.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffabe400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffab81f44c0 0x7ffab80afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:52.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffabe400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffab806e920 con 0x7ffab81f44c0 2024-09-17T05:35:52.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffabe400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffab806e7a0 con 0x7ffab81522f0 2024-09-17T05:35:52.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffabe400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ffab806eaa0 con 0x7ffab81526d0 2024-09-17T05:35:52.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffab7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffab81f44c0 0x7ffab80afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:52.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffab7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffab81f44c0 0x7ffab80afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54146/0 (socket says 172.21.15.122:54146) 2024-09-17T05:35:52.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffab7e00640 1 -- 172.21.15.122:0/3977321822 learned_addr learned my addr 172.21.15.122:0/3977321822 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:52.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.214+0000 7ffaafe00640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffab81526d0 0x7ffab81f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:52.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffab7e00640 1 -- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffab81526d0 msgr2=0x7ffab81f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:52.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffab7e00640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffab81526d0 0x7ffab81f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:52.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffab7e00640 1 -- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffab81522f0 msgr2=0x7ffab81f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:52.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffab7e00640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffab81522f0 0x7ffab81f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:52.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffab7e00640 1 -- 172.21.15.122:0/3977321822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab8151b20 con 0x7ffab81f44c0 2024-09-17T05:35:52.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffab7e00640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffab81f44c0 0x7ffab80afe00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ffab00ab040 tx=0x7ffab00a5ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:52.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffaaf400640 1 -- 172.21.15.122:0/3977321822 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0006cb0 con 0x7ffab81f44c0 2024-09-17T05:35:52.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab81f4dd0 con 0x7ffab81f44c0 2024-09-17T05:35:52.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab81f0aa0 con 0x7ffab81f44c0 2024-09-17T05:35:52.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffaaf400640 1 -- 172.21.15.122:0/3977321822 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffab00019f0 con 0x7ffab81f44c0 2024-09-17T05:35:52.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffaaf400640 1 -- 172.21.15.122:0/3977321822 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0006310 con 0x7ffab81f44c0 2024-09-17T05:35:52.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.218+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa900052a0 con 0x7ffab81f44c0 2024-09-17T05:35:52.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.222+0000 7ffaaf400640 1 -- 172.21.15.122:0/3977321822 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7ffab00064b0 con 0x7ffab81f44c0 2024-09-17T05:35:52.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.226+0000 7ffaaf400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffa94074760 0x7ffa94076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:52.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.226+0000 7ffaaf400640 1 -- 172.21.15.122:0/3977321822 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ffab012d220 con 0x7ffab81f44c0 2024-09-17T05:35:52.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.226+0000 7ffab7400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffa94074760 0x7ffa94076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:52.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.230+0000 7ffab7400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffa94074760 0x7ffa94076c20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffab8154830 tx=0x7ffaa800d3f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:52.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.234+0000 7ffaaf400640 1 -- 172.21.15.122:0/3977321822 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ffab0016890 con 0x7ffab81f44c0 2024-09-17T05:35:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:52.594+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone479614", "target": ["mon-mgr", ""]}) -- 0x7ffa90000bd0 con 0x7ffa94074760 2024-09-17T05:35:53.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.706+0000 7ffaaf400640 1 -- 172.21.15.122:0/3977321822 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa90000bd0 con 0x7ffa94074760 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffa94074760 msgr2=0x7ffa94076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffa94074760 0x7ffa94076c20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffab8154830 tx=0x7ffaa800d3f0 comp rx=0 tx=0).stop 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffab81f44c0 msgr2=0x7ffab80afe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffab81f44c0 0x7ffab80afe00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ffab00ab040 tx=0x7ffab00a5ef0 comp rx=0 tx=0).stop 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 shutdown_connections 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffa94074760 0x7ffa94076c20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffab81f44c0 0x7ffab80afe00 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffab81526d0 0x7ffab81f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:53.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 --2- 172.21.15.122:0/3977321822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffab81522f0 0x7ffab81f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:53.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 >> 172.21.15.122:0/3977321822 conn(0x7ffab80b18f0 msgr2=0x7ffab8073bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:53.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 shutdown_connections 2024-09-17T05:35:53.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:53.714+0000 7ffabe400640 1 -- 172.21.15.122:0/3977321822 wait complete. 2024-09-17T05:35:53.745 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone321815 2024-09-17T05:35:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.094+0000 7fcb32600640 1 -- 172.21.15.122:0/3078234598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c06f300 msgr2=0x7fcb2c15b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.094+0000 7fcb32600640 1 --2- 172.21.15.122:0/3078234598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c06f300 0x7fcb2c15b240 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0d0af0 tx=0x7fcb2401f370 comp rx=0 tx=0).stop 2024-09-17T05:35:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.094+0000 7fcb32600640 1 -- 172.21.15.122:0/3078234598 shutdown_connections 2024-09-17T05:35:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.094+0000 7fcb32600640 1 --2- 172.21.15.122:0/3078234598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c06f300 0x7fcb2c15b240 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.094+0000 7fcb32600640 1 --2- 172.21.15.122:0/3078234598 >> v2:172.21.15.173:3300/0 conn(0x7fcb2c06aba0 0x7fcb2c159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.094+0000 7fcb32600640 1 -- 172.21.15.122:0/3078234598 >> 172.21.15.122:0/3078234598 conn(0x7fcb2c06cc20 msgr2=0x7fcb2c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.094+0000 7fcb32600640 1 -- 172.21.15.122:0/3078234598 shutdown_connections 2024-09-17T05:35:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.094+0000 7fcb32600640 1 -- 172.21.15.122:0/3078234598 wait complete. 2024-09-17T05:35:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb32600640 1 Processor -- start 2024-09-17T05:35:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb32600640 1 -- start start 2024-09-17T05:35:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb32600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb2c06aba0 0x7fcb2c150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb32600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb2c06f300 0x7fcb2c146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb32600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c0705d0 0x7fcb2c148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb32600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c15b900 con 0x7fcb2c0705d0 2024-09-17T05:35:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb32600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c15b780 con 0x7fcb2c06f300 2024-09-17T05:35:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb32600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2c15ba80 con 0x7fcb2c06aba0 2024-09-17T05:35:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.098+0000 7fcb31c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c0705d0 0x7fcb2c148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb31c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c0705d0 0x7fcb2c148ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54152/0 (socket says 172.21.15.122:54152) 2024-09-17T05:35:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb31c00640 1 -- 172.21.15.122:0/3656043542 learned_addr learned my addr 172.21.15.122:0/3656043542 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb2be00640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb2c06f300 0x7fcb2c146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb31c00640 1 -- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb2c06aba0 msgr2=0x7fcb2c150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb31c00640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb2c06aba0 0x7fcb2c150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb31c00640 1 -- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb2c06f300 msgr2=0x7fcb2c146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb31c00640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb2c06f300 0x7fcb2c146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb31c00640 1 -- 172.21.15.122:0/3656043542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c14e2d0 con 0x7fcb2c0705d0 2024-09-17T05:35:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb2be00640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb2c06f300 0x7fcb2c146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:35:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.102+0000 7fcb31c00640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c0705d0 0x7fcb2c148ed0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401fc10 tx=0x7fcb240af420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:54.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.122+0000 7fcb28c00640 1 -- 172.21.15.122:0/3656043542 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240b4040 con 0x7fcb2c0705d0 2024-09-17T05:35:54.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.122+0000 7fcb28c00640 1 -- 172.21.15.122:0/3656043542 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb240bf070 con 0x7fcb2c0705d0 2024-09-17T05:35:54.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.122+0000 7fcb28c00640 1 -- 172.21.15.122:0/3656043542 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240b0930 con 0x7fcb2c0705d0 2024-09-17T05:35:54.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.122+0000 7fcb32600640 1 -- 172.21.15.122:0/3656043542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c14e5b0 con 0x7fcb2c0705d0 2024-09-17T05:35:54.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.122+0000 7fcb32600640 1 -- 172.21.15.122:0/3656043542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c14eb70 con 0x7fcb2c0705d0 2024-09-17T05:35:54.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.122+0000 7fcb1e000640 1 -- 172.21.15.122:0/3656043542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb000052a0 con 0x7fcb2c0705d0 2024-09-17T05:35:54.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.134+0000 7fcb28c00640 1 -- 172.21.15.122:0/3656043542 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7fcb240b0ad0 con 0x7fcb2c0705d0 2024-09-17T05:35:54.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.138+0000 7fcb28c00640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb0c074760 0x7fcb0c076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:54.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.138+0000 7fcb28c00640 1 -- 172.21.15.122:0/3656043542 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fcb240b0dc0 con 0x7fcb2c0705d0 2024-09-17T05:35:54.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.146+0000 7fcb31200640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb0c074760 0x7fcb0c076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:54.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.146+0000 7fcb28c00640 1 -- 172.21.15.122:0/3656043542 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcb24018ed0 con 0x7fcb2c0705d0 2024-09-17T05:35:54.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.162+0000 7fcb31200640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb0c074760 0x7fcb0c076c20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcb200058d0 tx=0x7fcb20005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:54.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:54.558+0000 7fcb1e000640 1 -- 172.21.15.122:0/3656043542 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone321815", "target": ["mon-mgr", ""]}) -- 0x7fcb00000bd0 con 0x7fcb0c074760 2024-09-17T05:35:55.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.506+0000 7fcb28c00640 1 -- 172.21.15.122:0/3656043542 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb00000bd0 con 0x7fcb0c074760 2024-09-17T05:35:55.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 -- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb0c074760 msgr2=0x7fcb0c076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:55.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb0c074760 0x7fcb0c076c20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcb200058d0 tx=0x7fcb20005860 comp rx=0 tx=0).stop 2024-09-17T05:35:55.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 -- 172.21.15.122:0/3656043542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c0705d0 msgr2=0x7fcb2c148ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:55.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c0705d0 0x7fcb2c148ed0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401fc10 tx=0x7fcb240af420 comp rx=0 tx=0).stop 2024-09-17T05:35:55.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 -- 172.21.15.122:0/3656043542 shutdown_connections 2024-09-17T05:35:55.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcb0c074760 0x7fcb0c076c20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb2c0705d0 0x7fcb2c148ed0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb2c06f300 0x7fcb2c146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 --2- 172.21.15.122:0/3656043542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb2c06aba0 0x7fcb2c150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 -- 172.21.15.122:0/3656043542 >> 172.21.15.122:0/3656043542 conn(0x7fcb2c06cc20 msgr2=0x7fcb2c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:55.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 -- 172.21.15.122:0/3656043542 shutdown_connections 2024-09-17T05:35:55.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.510+0000 7fcb1e000640 1 -- 172.21.15.122:0/3656043542 wait complete. 2024-09-17T05:35:55.538 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone125516 2024-09-17T05:35:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.802+0000 7f59a0800640 1 -- 172.21.15.122:0/3953729416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f599815b580 msgr2=0x7f599815b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:55.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.802+0000 7f59a0800640 1 --2- 172.21.15.122:0/3953729416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f599815b580 0x7f599815b960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f598c009900 tx=0x7f598c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.802+0000 7f59a0800640 1 -- 172.21.15.122:0/3953729416 shutdown_connections 2024-09-17T05:35:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.802+0000 7f59a0800640 1 --2- 172.21.15.122:0/3953729416 >> v2:172.21.15.173:3300/0 conn(0x7f59980b3300 0x7f59980cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.802+0000 7f59a0800640 1 --2- 172.21.15.122:0/3953729416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f599815b580 0x7f599815b960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.802+0000 7f59a0800640 1 -- 172.21.15.122:0/3953729416 >> 172.21.15.122:0/3953729416 conn(0x7f59980b18f0 msgr2=0x7f59980b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.802+0000 7f59a0800640 1 -- 172.21.15.122:0/3953729416 shutdown_connections 2024-09-17T05:35:55.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.802+0000 7f59a0800640 1 -- 172.21.15.122:0/3953729416 wait complete. 2024-09-17T05:35:55.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 Processor -- start 2024-09-17T05:35:55.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 -- start start 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59980b3300 0x7f59981543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59980b4bd0 0x7f59980746e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f599815b580 0x7f59981482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59980cc920 con 0x7f599815b580 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59980cc7a0 con 0x7f59980b3300 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f59980ccaa0 con 0x7f59980b4bd0 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f599815b580 0x7f59981482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f599815b580 0x7f59981482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54162/0 (socket says 172.21.15.122:54162) 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 -- 172.21.15.122:0/437695797 learned_addr learned my addr 172.21.15.122:0/437695797 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:55.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599d400640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59980b4bd0 0x7f59980746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 -- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59980b4bd0 msgr2=0x7f59980746e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59980b4bd0 0x7f59980746e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 -- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59980b3300 msgr2=0x7f59981543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59980b3300 0x7f59981543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:55.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 -- 172.21.15.122:0/437695797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59981549b0 con 0x7f599815b580 2024-09-17T05:35:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f599e800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f599815b580 0x7f59981482a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f599400b9e0 tx=0x7f599400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f5992000640 1 -- 172.21.15.122:0/437695797 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994019070 con 0x7f599815b580 2024-09-17T05:35:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f5992000640 1 -- 172.21.15.122:0/437695797 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599400cc10 con 0x7f599815b580 2024-09-17T05:35:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5998154c90 con 0x7f599815b580 2024-09-17T05:35:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.806+0000 7f5992000640 1 -- 172.21.15.122:0/437695797 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940146a0 con 0x7f599815b580 2024-09-17T05:35:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.810+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5998155250 con 0x7f599815b580 2024-09-17T05:35:55.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.810+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59980cdae0 con 0x7f599815b580 2024-09-17T05:35:55.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.810+0000 7f5992000640 1 -- 172.21.15.122:0/437695797 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f599401c070 con 0x7f599815b580 2024-09-17T05:35:55.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.810+0000 7f5992000640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f597c074760 0x7f597c076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:55.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.810+0000 7f5992000640 1 -- 172.21.15.122:0/437695797 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f5994092fe0 con 0x7f599815b580 2024-09-17T05:35:55.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.814+0000 7f599de00640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f597c074760 0x7f597c076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:55.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.818+0000 7f5992000640 1 -- 172.21.15.122:0/437695797 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f599405f090 con 0x7f599815b580 2024-09-17T05:35:55.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:55.834+0000 7f599de00640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f597c074760 0x7f597c076c20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f598c0af000 tx=0x7f598c019680 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:56.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:56.306+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone125516", "target": ["mon-mgr", ""]}) -- 0x7f59980b1570 con 0x7f597c074760 2024-09-17T05:35:57.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.526+0000 7f5992000640 1 -- 172.21.15.122:0/437695797 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59980b1570 con 0x7f597c074760 2024-09-17T05:35:57.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f597c074760 msgr2=0x7f597c076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:57.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f597c074760 0x7f597c076c20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f598c0af000 tx=0x7f598c019680 comp rx=0 tx=0).stop 2024-09-17T05:35:57.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f599815b580 msgr2=0x7f59981482a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f599815b580 0x7f59981482a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f599400b9e0 tx=0x7f599400bea0 comp rx=0 tx=0).stop 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 shutdown_connections 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f597c074760 0x7f597c076c20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f599815b580 0x7f59981482a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59980b4bd0 0x7f59980746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 --2- 172.21.15.122:0/437695797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59980b3300 0x7f59981543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 >> 172.21.15.122:0/437695797 conn(0x7f59980b18f0 msgr2=0x7f599806fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 shutdown_connections 2024-09-17T05:35:57.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.530+0000 7f59a0800640 1 -- 172.21.15.122:0/437695797 wait complete. 2024-09-17T05:35:57.550 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone523517 2024-09-17T05:35:57.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.790+0000 7f4c99a00640 1 -- 172.21.15.122:0/358679678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c940cac60 msgr2=0x7f4c9406e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:57.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.790+0000 7f4c99a00640 1 --2- 172.21.15.122:0/358679678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c940cac60 0x7f4c9406e650 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009900 tx=0x7f4c8401c850 comp rx=0 tx=0).stop 2024-09-17T05:35:57.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.794+0000 7f4c99a00640 1 -- 172.21.15.122:0/358679678 shutdown_connections 2024-09-17T05:35:57.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.794+0000 7f4c99a00640 1 --2- 172.21.15.122:0/358679678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c940cac60 0x7f4c9406e650 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:57.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.794+0000 7f4c99a00640 1 -- 172.21.15.122:0/358679678 >> 172.21.15.122:0/358679678 conn(0x7f4c940cfa00 msgr2=0x7f4c940cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:57.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.794+0000 7f4c99a00640 1 -- 172.21.15.122:0/358679678 shutdown_connections 2024-09-17T05:35:57.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.794+0000 7f4c99a00640 1 -- 172.21.15.122:0/358679678 wait complete. 2024-09-17T05:35:57.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.794+0000 7f4c99a00640 1 Processor -- start 2024-09-17T05:35:57.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c99a00640 1 -- start start 2024-09-17T05:35:57.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c99a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c94072610 0x7f4c94157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:57.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c99a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c940aecb0 0x7f4c9415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:57.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c99a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c940cac60 0x7f4c9415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:57.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c99a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9415d010 con 0x7f4c94072610 2024-09-17T05:35:57.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c99a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9415ce90 con 0x7f4c940cac60 2024-09-17T05:35:57.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c99a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9415d190 con 0x7f4c940aecb0 2024-09-17T05:35:57.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c93e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c94072610 0x7f4c94157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:57.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c93e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c94072610 0x7f4c94157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54180/0 (socket says 172.21.15.122:54180) 2024-09-17T05:35:57.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c93e00640 1 -- 172.21.15.122:0/3943368465 learned_addr learned my addr 172.21.15.122:0/3943368465 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:57.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c8be00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c940aecb0 0x7f4c9415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c8be00640 1 -- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c940cac60 msgr2=0x7f4c9415e4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:35:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c8be00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c940cac60 0x7f4c9415e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c8be00640 1 -- 172.21.15.122:0/3943368465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c94072610 msgr2=0x7f4c94157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c8be00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c94072610 0x7f4c94157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c8be00640 1 -- 172.21.15.122:0/3943368465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9415b2a0 con 0x7f4c940aecb0 2024-09-17T05:35:57.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.798+0000 7f4c93e00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c94072610 0x7f4c94157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:35:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.802+0000 7f4c8be00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c940aecb0 0x7f4c9415ad60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c840ac000 tx=0x7f4c840a56c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.802+0000 7f4c90c00640 1 -- 172.21.15.122:0/3943368465 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c840b2040 con 0x7f4c940aecb0 2024-09-17T05:35:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.802+0000 7f4c90c00640 1 -- 172.21.15.122:0/3943368465 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c84004450 con 0x7f4c940aecb0 2024-09-17T05:35:57.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.802+0000 7f4c90c00640 1 -- 172.21.15.122:0/3943368465 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84003860 con 0x7f4c940aecb0 2024-09-17T05:35:57.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.802+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c94157e80 con 0x7f4c940aecb0 2024-09-17T05:35:57.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.802+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c94158440 con 0x7f4c940aecb0 2024-09-17T05:35:57.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.806+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9406ee10 con 0x7f4c940aecb0 2024-09-17T05:35:57.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.810+0000 7f4c90c00640 1 -- 172.21.15.122:0/3943368465 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f4c84003a00 con 0x7f4c940aecb0 2024-09-17T05:35:57.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.814+0000 7f4c90c00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c70074830 0x7f4c70076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:57.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.814+0000 7f4c90c00640 1 -- 172.21.15.122:0/3943368465 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f4c8412cc90 con 0x7f4c940aecb0 2024-09-17T05:35:57.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.814+0000 7f4c93e00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c70074830 0x7f4c70076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:57.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.818+0000 7f4c93e00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c70074830 0x7f4c70076cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c001810 tx=0x7f4c7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:57.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:57.818+0000 7f4c90c00640 1 -- 172.21.15.122:0/3943368465 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4c840bd060 con 0x7f4c940aecb0 2024-09-17T05:35:58.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:58.146+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone523517", "target": ["mon-mgr", ""]}) -- 0x7f4c94069e00 con 0x7f4c70074830 2024-09-17T05:35:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.070+0000 7f4c90c00640 1 -- 172.21.15.122:0/3943368465 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c94069e00 con 0x7f4c70074830 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c70074830 msgr2=0x7f4c70076cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c70074830 0x7f4c70076cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c001810 tx=0x7f4c7c009040 comp rx=0 tx=0).stop 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c940aecb0 msgr2=0x7f4c9415ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c940aecb0 0x7f4c9415ad60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c840ac000 tx=0x7f4c840a56c0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 shutdown_connections 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c70074830 0x7f4c70076cf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c940cac60 0x7f4c9415e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c940aecb0 0x7f4c9415ad60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 --2- 172.21.15.122:0/3943368465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c94072610 0x7f4c94157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 >> 172.21.15.122:0/3943368465 conn(0x7f4c940cfa00 msgr2=0x7f4c940b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:59.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 shutdown_connections 2024-09-17T05:35:59.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.074+0000 7f4c99a00640 1 -- 172.21.15.122:0/3943368465 wait complete. 2024-09-17T05:35:59.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone608518 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- 172.21.15.122:0/3880191275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6380d05f0 msgr2=0x7ff6380d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 --2- 172.21.15.122:0/3880191275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6380d05f0 0x7ff6380d1e80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff6380b4770 tx=0x7ff63001caf0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- 172.21.15.122:0/3880191275 shutdown_connections 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 --2- 172.21.15.122:0/3880191275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6380d05f0 0x7ff6380d1e80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 --2- 172.21.15.122:0/3880191275 >> v2:172.21.15.122:3300/0 conn(0x7ff6380d0190 0x7ff6380d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 --2- 172.21.15.122:0/3880191275 >> v2:172.21.15.173:3300/0 conn(0x7ff63815ba60 0x7ff6381602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- 172.21.15.122:0/3880191275 >> 172.21.15.122:0/3880191275 conn(0x7ff63809aa10 msgr2=0x7ff63809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- 172.21.15.122:0/3880191275 shutdown_connections 2024-09-17T05:35:59.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- 172.21.15.122:0/3880191275 wait complete. 2024-09-17T05:35:59.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 Processor -- start 2024-09-17T05:35:59.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- start start 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6380d0190 0x7ff638155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6380d05f0 0x7ff638166e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff63815ba60 0x7ff6381673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380cca60 con 0x7ff6380d0190 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380cc8e0 con 0x7ff63815ba60 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.442+0000 7ff63ee00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff6380ccbe0 con 0x7ff6380d05f0 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6380d0190 0x7ff638155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6380d0190 0x7ff638155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54192/0 (socket says 172.21.15.122:54192) 2024-09-17T05:35:59.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 -- 172.21.15.122:0/2622172091 learned_addr learned my addr 172.21.15.122:0/2622172091 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:35:59.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63d000640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6380d05f0 0x7ff638166e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:59.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63e400640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff63815ba60 0x7ff6381673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:59.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 -- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6380d05f0 msgr2=0x7ff638166e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:59.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6380d05f0 0x7ff638166e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 -- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff63815ba60 msgr2=0x7ff6381673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff63815ba60 0x7ff6381673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 -- 172.21.15.122:0/2622172091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63815eb70 con 0x7ff6380d0190 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63da00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6380d0190 0x7ff638155630 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00d720 tx=0x7ff62c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff635600640 1 -- 172.21.15.122:0/2622172091 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c017070 con 0x7ff6380d0190 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff635600640 1 -- 172.21.15.122:0/2622172091 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff62c009d20 con 0x7ff6380d0190 2024-09-17T05:35:59.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff635600640 1 -- 172.21.15.122:0/2622172091 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c005280 con 0x7ff6380d0190 2024-09-17T05:35:59.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6380b5c40 con 0x7ff6380d0190 2024-09-17T05:35:59.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.446+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6380c54d0 con 0x7ff6380d0190 2024-09-17T05:35:59.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.450+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6381602b0 con 0x7ff6380d0190 2024-09-17T05:35:59.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.450+0000 7ff635600640 1 -- 172.21.15.122:0/2622172091 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7ff62c005420 con 0x7ff6380d0190 2024-09-17T05:35:59.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.454+0000 7ff635600640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff61c074760 0x7ff61c076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:35:59.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.454+0000 7ff635600640 1 -- 172.21.15.122:0/2622172091 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff62c093410 con 0x7ff6380d0190 2024-09-17T05:35:59.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.458+0000 7ff635600640 1 -- 172.21.15.122:0/2622172091 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff62c05fb10 con 0x7ff6380d0190 2024-09-17T05:35:59.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.458+0000 7ff63d000640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff61c074760 0x7ff61c076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:35:59.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.478+0000 7ff63d000640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff61c074760 0x7ff61c076c20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff6380cc200 tx=0x7ff6280091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:35:59.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:35:59.814+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone608518", "target": ["mon-mgr", ""]}) -- 0x7ff6380d2a60 con 0x7ff61c074760 2024-09-17T05:36:00.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.214+0000 7ff635600640 1 -- 172.21.15.122:0/2622172091 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff6380d2a60 con 0x7ff61c074760 2024-09-17T05:36:00.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff61c074760 msgr2=0x7ff61c076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:00.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff61c074760 0x7ff61c076c20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff6380cc200 tx=0x7ff6280091e0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6380d0190 msgr2=0x7ff638155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:00.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6380d0190 0x7ff638155630 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00d720 tx=0x7ff62c00dbe0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 shutdown_connections 2024-09-17T05:36:00.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff61c074760 0x7ff61c076c20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff63815ba60 0x7ff6381673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6380d05f0 0x7ff638166e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 --2- 172.21.15.122:0/2622172091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6380d0190 0x7ff638155630 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 >> 172.21.15.122:0/2622172091 conn(0x7ff63809aa10 msgr2=0x7ff6380b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:00.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 shutdown_connections 2024-09-17T05:36:00.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.222+0000 7ff63ee00640 1 -- 172.21.15.122:0/2622172091 wait complete. 2024-09-17T05:36:00.256 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone446019 2024-09-17T05:36:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 -- 172.21.15.122:0/416046150 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c380cbb90 msgr2=0x7f4c380cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 --2- 172.21.15.122:0/416046150 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c380cbb90 0x7f4c380cfaa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009840 tx=0x7f4c2401c700 comp rx=0 tx=0).stop 2024-09-17T05:36:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 -- 172.21.15.122:0/416046150 shutdown_connections 2024-09-17T05:36:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 --2- 172.21.15.122:0/416046150 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c380cbb90 0x7f4c380cfaa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 --2- 172.21.15.122:0/416046150 >> v2:172.21.15.173:3300/0 conn(0x7f4c380739f0 0x7f4c380cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 -- 172.21.15.122:0/416046150 >> 172.21.15.122:0/416046150 conn(0x7f4c380d0b30 msgr2=0x7f4c380d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 -- 172.21.15.122:0/416046150 shutdown_connections 2024-09-17T05:36:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 -- 172.21.15.122:0/416046150 wait complete. 2024-09-17T05:36:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 Processor -- start 2024-09-17T05:36:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 -- start start 2024-09-17T05:36:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.542+0000 7f4c37e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c380739f0 0x7f4c380b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c380cbb90 0x7f4c3815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c380d1650 0x7f4c38151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38168380 con 0x7f4c380cbb90 2024-09-17T05:36:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38168200 con 0x7f4c380739f0 2024-09-17T05:36:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4c38168500 con 0x7f4c380d1650 2024-09-17T05:36:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c380739f0 0x7f4c380b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c380cbb90 0x7f4c3815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c380d1650 0x7f4c38151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c380cbb90 0x7f4c3815a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54202/0 (socket says 172.21.15.122:54202) 2024-09-17T05:36:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36000640 1 -- 172.21.15.122:0/3666919552 learned_addr learned my addr 172.21.15.122:0/3666919552 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36a00640 1 -- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c380d1650 msgr2=0x7f4c38151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36a00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c380d1650 0x7f4c38151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36a00640 1 -- 172.21.15.122:0/3666919552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c380cbb90 msgr2=0x7f4c3815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36a00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c380cbb90 0x7f4c3815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36a00640 1 -- 172.21.15.122:0/3666919552 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3815b5b0 con 0x7f4c380739f0 2024-09-17T05:36:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c36a00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c380739f0 0x7f4c380b5fc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800e9f0 tx=0x7f4c2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c2ea00640 1 -- 172.21.15.122:0/3666919552 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800cd80 con 0x7f4c380739f0 2024-09-17T05:36:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c2ea00640 1 -- 172.21.15.122:0/3666919552 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c28004020 con 0x7f4c380739f0 2024-09-17T05:36:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c2ea00640 1 -- 172.21.15.122:0/3666919552 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28004450 con 0x7f4c380739f0 2024-09-17T05:36:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3815aff0 con 0x7f4c380739f0 2024-09-17T05:36:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3814f7d0 con 0x7f4c380739f0 2024-09-17T05:36:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.546+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3815df20 con 0x7f4c380739f0 2024-09-17T05:36:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.550+0000 7f4c2ea00640 1 -- 172.21.15.122:0/3666919552 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f4c28005030 con 0x7f4c380739f0 2024-09-17T05:36:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.550+0000 7f4c2ea00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c18074760 0x7f4c18076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.550+0000 7f4c2ea00640 1 -- 172.21.15.122:0/3666919552 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f4c28093c00 con 0x7f4c380739f0 2024-09-17T05:36:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.550+0000 7f4c36000640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c18074760 0x7f4c18076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.550+0000 7f4c36000640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c18074760 0x7f4c18076c20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4c2401adb0 tx=0x7f4c2401af40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:00.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.554+0000 7f4c2ea00640 1 -- 172.21.15.122:0/3666919552 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4c280600b0 con 0x7f4c380739f0 2024-09-17T05:36:00.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:00.882+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone446019", "target": ["mon-mgr", ""]}) -- 0x7f4c380b6ab0 con 0x7f4c18074760 2024-09-17T05:36:01.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c2ea00640 1 -- 172.21.15.122:0/3666919552 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c380b6ab0 con 0x7f4c18074760 2024-09-17T05:36:01.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c18074760 msgr2=0x7f4c18076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c18074760 0x7f4c18076c20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4c2401adb0 tx=0x7f4c2401af40 comp rx=0 tx=0).stop 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c380739f0 msgr2=0x7f4c380b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c380739f0 0x7f4c380b5fc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800e9f0 tx=0x7f4c2800eeb0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 shutdown_connections 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4c18074760 0x7f4c18076c20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c380d1650 0x7f4c38151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c380cbb90 0x7f4c3815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 --2- 172.21.15.122:0/3666919552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c380739f0 0x7f4c380b5fc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 >> 172.21.15.122:0/3666919552 conn(0x7f4c380d0b30 msgr2=0x7f4c38070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 shutdown_connections 2024-09-17T05:36:01.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.086+0000 7f4c37e00640 1 -- 172.21.15.122:0/3666919552 wait complete. 2024-09-17T05:36:01.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:36:01.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.250+0000 7ff4fbe00640 1 -- 172.21.15.122:0/313941491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc15f000 msgr2=0x7ff4fc0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.250+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/313941491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc15f000 0x7ff4fc0d01b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009c80 tx=0x7ff4ec01cd10 comp rx=0 tx=0).stop 2024-09-17T05:36:01.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.250+0000 7ff4fbe00640 1 -- 172.21.15.122:0/313941491 shutdown_connections 2024-09-17T05:36:01.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.250+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/313941491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4fc0b2590 0x7ff4fc0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.250+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/313941491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc15f000 0x7ff4fc0d01b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.250+0000 7ff4fbe00640 1 -- 172.21.15.122:0/313941491 >> 172.21.15.122:0/313941491 conn(0x7ff4fc09aa10 msgr2=0x7ff4fc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:01.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.250+0000 7ff4fbe00640 1 -- 172.21.15.122:0/313941491 shutdown_connections 2024-09-17T05:36:01.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.250+0000 7ff4fbe00640 1 -- 172.21.15.122:0/313941491 wait complete. 2024-09-17T05:36:01.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fbe00640 1 Processor -- start 2024-09-17T05:36:01.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fbe00640 1 -- start start 2024-09-17T05:36:01.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc0b2590 0x7ff4fc1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:01.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fbe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4fc0cefb0 0x7ff4fc151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:01.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4fc15f000 0x7ff4fc1fb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fbe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc168f10 con 0x7ff4fc0b2590 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fbe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc168d90 con 0x7ff4fc15f000 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fbe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff4fc169090 con 0x7ff4fc0cefb0 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc0b2590 0x7ff4fc1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc0b2590 0x7ff4fc1502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54214/0 (socket says 172.21.15.122:54214) 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 -- 172.21.15.122:0/4131355232 learned_addr learned my addr 172.21.15.122:0/4131355232 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4f2000640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4fc0cefb0 0x7ff4fc151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4fb400640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4fc15f000 0x7ff4fc1fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 -- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4fc0cefb0 msgr2=0x7ff4fc151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4fc0cefb0 0x7ff4fc151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 -- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4fc15f000 msgr2=0x7ff4fc1fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4fc15f000 0x7ff4fc1fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 -- 172.21.15.122:0/4131355232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc06de60 con 0x7ff4fc0b2590 2024-09-17T05:36:01.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.254+0000 7ff4faa00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc0b2590 0x7ff4fc1502d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec000f90 tx=0x7ff4ec0a59e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.258+0000 7ff4f3400640 1 -- 172.21.15.122:0/4131355232 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0bb070 con 0x7ff4fc0b2590 2024-09-17T05:36:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.258+0000 7ff4f3400640 1 -- 172.21.15.122:0/4131355232 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4ec004e00 con 0x7ff4fc0b2590 2024-09-17T05:36:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.258+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc06e100 con 0x7ff4fc0b2590 2024-09-17T05:36:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.258+0000 7ff4f3400640 1 -- 172.21.15.122:0/4131355232 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0b2040 con 0x7ff4fc0b2590 2024-09-17T05:36:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.258+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc06e6c0 con 0x7ff4fc0b2590 2024-09-17T05:36:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.258+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4fc0b29e0 con 0x7ff4fc0b2590 2024-09-17T05:36:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.258+0000 7ff4f3400640 1 -- 172.21.15.122:0/4131355232 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7ff4ec0be070 con 0x7ff4fc0b2590 2024-09-17T05:36:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.262+0000 7ff4f3400640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4d8074760 0x7ff4d8076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.262+0000 7ff4f3400640 1 -- 172.21.15.122:0/4131355232 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff4ec12d300 con 0x7ff4fc0b2590 2024-09-17T05:36:01.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.262+0000 7ff4f2000640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4d8074760 0x7ff4d8076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.262+0000 7ff4f2000640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4d8074760 0x7ff4d8076c20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc15e640 tx=0x7ff4e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:01.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.266+0000 7ff4f3400640 1 -- 172.21.15.122:0/4131355232 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff4ec134030 con 0x7ff4fc0b2590 2024-09-17T05:36:01.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.458+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff4fc0cbcd0 con 0x7ff4d8074760 2024-09-17T05:36:01.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.458+0000 7ff4f3400640 1 -- 172.21.15.122:0/4131355232 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff4fc0cbcd0 con 0x7ff4d8074760 2024-09-17T05:36:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:36:01.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.456 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:36:01.456 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:36:01.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4d8074760 msgr2=0x7ff4d8076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4d8074760 0x7ff4d8076c20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc15e640 tx=0x7ff4e4009040 comp rx=0 tx=0).stop 2024-09-17T05:36:01.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc0b2590 msgr2=0x7ff4fc1502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc0b2590 0x7ff4fc1502d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec000f90 tx=0x7ff4ec0a59e0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 shutdown_connections 2024-09-17T05:36:01.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4d8074760 0x7ff4d8076c20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4fc15f000 0x7ff4fc1fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4fc0cefb0 0x7ff4fc151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 --2- 172.21.15.122:0/4131355232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4fc0b2590 0x7ff4fc1502d0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 >> 172.21.15.122:0/4131355232 conn(0x7ff4fc09aa10 msgr2=0x7ff4fc157650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:01.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 shutdown_connections 2024-09-17T05:36:01.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.462+0000 7ff4fbe00640 1 -- 172.21.15.122:0/4131355232 wait complete. 2024-09-17T05:36:01.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:36:01.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.626+0000 7f5c21000640 1 -- 172.21.15.122:0/2230802327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0cfd80 msgr2=0x7f5c1c0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.626+0000 7f5c21000640 1 --2- 172.21.15.122:0/2230802327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0cfd80 0x7f5c1c0d1bf0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009900 tx=0x7f5c0401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.626+0000 7f5c21000640 1 -- 172.21.15.122:0/2230802327 shutdown_connections 2024-09-17T05:36:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.626+0000 7f5c21000640 1 --2- 172.21.15.122:0/2230802327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0cfd80 0x7f5c1c0d1bf0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.626+0000 7f5c21000640 1 --2- 172.21.15.122:0/2230802327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c1c15df10 0x7f5c1c06eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.626+0000 7f5c21000640 1 -- 172.21.15.122:0/2230802327 >> 172.21.15.122:0/2230802327 conn(0x7f5c1c09aa10 msgr2=0x7f5c1c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.626+0000 7f5c21000640 1 -- 172.21.15.122:0/2230802327 shutdown_connections 2024-09-17T05:36:01.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.626+0000 7f5c21000640 1 -- 172.21.15.122:0/2230802327 wait complete. 2024-09-17T05:36:01.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 Processor -- start 2024-09-17T05:36:01.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 -- start start 2024-09-17T05:36:01.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c1c0cfd80 0x7f5c1c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:01.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0d2130 0x7f5c1c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:01.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c1c15df10 0x7f5c1c156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:01.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c16b210 con 0x7f5c1c0d2130 2024-09-17T05:36:01.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c16b090 con 0x7f5c1c0cfd80 2024-09-17T05:36:01.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1c16b390 con 0x7f5c1c15df10 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0d2130 0x7f5c1c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0d2130 0x7f5c1c0b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36732/0 (socket says 172.21.15.122:36732) 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 -- 172.21.15.122:0/1052721506 learned_addr learned my addr 172.21.15.122:0/1052721506 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c1be00640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c1c15df10 0x7f5c1c156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c1b400640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c1c0cfd80 0x7f5c1c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 -- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c1c15df10 msgr2=0x7f5c1c156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c1c15df10 0x7f5c1c156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 -- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c1c0cfd80 msgr2=0x7f5c1c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c1c0cfd80 0x7f5c1c0b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 -- 172.21.15.122:0/1052721506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c0ca700 con 0x7f5c1c0d2130 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c12a00640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0d2130 0x7f5c1c0b4ab0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009850 tx=0x7f5c040a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c13e00640 1 -- 172.21.15.122:0/1052721506 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c040b2040 con 0x7f5c1c0d2130 2024-09-17T05:36:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c13e00640 1 -- 172.21.15.122:0/1052721506 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c04004470 con 0x7f5c1c0d2130 2024-09-17T05:36:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c13e00640 1 -- 172.21.15.122:0/1052721506 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c040052d0 con 0x7f5c1c0d2130 2024-09-17T05:36:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1c0b53e0 con 0x7f5c1c0d2130 2024-09-17T05:36:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1c154040 con 0x7f5c1c0d2130 2024-09-17T05:36:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.630+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1c0d1bf0 con 0x7f5c1c0d2130 2024-09-17T05:36:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.634+0000 7f5c13e00640 1 -- 172.21.15.122:0/1052721506 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f5c04005470 con 0x7f5c1c0d2130 2024-09-17T05:36:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.634+0000 7f5c13e00640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bf8074760 0x7f5bf8076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.634+0000 7f5c13e00640 1 -- 172.21.15.122:0/1052721506 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f5c0412c4f0 con 0x7f5c1c0d2130 2024-09-17T05:36:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.634+0000 7f5c1b400640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bf8074760 0x7f5bf8076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.634+0000 7f5c1b400640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bf8074760 0x7f5bf8076c20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009b40 tx=0x7f5c0c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.638+0000 7f5c13e00640 1 -- 172.21.15.122:0/1052721506 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5c1c0ca700 con 0x7f5c1c0d2130 2024-09-17T05:36:01.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.830+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c1c0ccf10 con 0x7f5bf8074760 2024-09-17T05:36:01.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.842+0000 7f5c13e00640 1 -- 172.21.15.122:0/1052721506 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1024 (secure 0 0 0) 0x7f5c1c0ccf10 con 0x7f5bf8074760 2024-09-17T05:36:01.839 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone608518" 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone321815" 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone9004" 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.840 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone84910" 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sv1" 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone523517" 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone59558" 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.841 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone479614" 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone67336" 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone446019" 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone610611" 2024-09-17T05:36:01.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone8110" 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone67922" 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone281413" 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone21727" 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone82612" 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone84053" 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone44559" 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone92681" 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.844 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone25105" 2024-09-17T05:36:01.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:01.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:01.845 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone125516" 2024-09-17T05:36:01.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:01.845 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bf8074760 msgr2=0x7f5bf8076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bf8074760 0x7f5bf8076c20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009b40 tx=0x7f5c0c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0d2130 msgr2=0x7f5c1c0b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0d2130 0x7f5c1c0b4ab0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009850 tx=0x7f5c040a5730 comp rx=0 tx=0).stop 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 shutdown_connections 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5bf8074760 0x7f5bf8076c20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c1c15df10 0x7f5c1c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c1c0d2130 0x7f5c1c0b4ab0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 --2- 172.21.15.122:0/1052721506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c1c0cfd80 0x7f5c1c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 >> 172.21.15.122:0/1052721506 conn(0x7f5c1c09aa10 msgr2=0x7f5c1c09adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 shutdown_connections 2024-09-17T05:36:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:01.846+0000 7f5c21000640 1 -- 172.21.15.122:0/1052721506 wait complete. 2024-09-17T05:36:01.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone608518 2024-09-17T05:36:02.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 -- 172.21.15.122:0/2727881783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 msgr2=0x7f34380d1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 --2- 172.21.15.122:0/2727881783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 0x7f34380d1ae0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f34380b6b00 tx=0x7f342c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:36:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 -- 172.21.15.122:0/2727881783 shutdown_connections 2024-09-17T05:36:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 --2- 172.21.15.122:0/2727881783 >> v2:172.21.15.173:3300/0 conn(0x7f34380d24d0 0x7f34380d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 --2- 172.21.15.122:0/2727881783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380cf950 0x7f34380d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 --2- 172.21.15.122:0/2727881783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 0x7f34380d1ae0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 -- 172.21.15.122:0/2727881783 >> 172.21.15.122:0/2727881783 conn(0x7f34380d3f10 msgr2=0x7f34380d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 -- 172.21.15.122:0/2727881783 shutdown_connections 2024-09-17T05:36:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 -- 172.21.15.122:0/2727881783 wait complete. 2024-09-17T05:36:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 Processor -- start 2024-09-17T05:36:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.010+0000 7f343ee00640 1 -- start start 2024-09-17T05:36:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cf950 0x7f343815f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380d24d0 0x7f343815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 0x7f34380b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:02.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34380b3fd0 con 0x7f343815dff0 2024-09-17T05:36:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3438161b70 con 0x7f34380cf950 2024-09-17T05:36:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f34381605a0 con 0x7f34380d24d0 2024-09-17T05:36:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 0x7f34380b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 0x7f34380b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36742/0 (socket says 172.21.15.122:36742) 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 -- 172.21.15.122:0/2763512416 learned_addr learned my addr 172.21.15.122:0/2763512416 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343d000640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380d24d0 0x7f343815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343da00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cf950 0x7f343815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 -- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380d24d0 msgr2=0x7f343815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380d24d0 0x7f343815fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 -- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cf950 msgr2=0x7f343815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cf950 0x7f343815f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 -- 172.21.15.122:0/2763512416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3438160880 con 0x7f343815dff0 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343e400640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 0x7f34380b2590 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f343000cce0 tx=0x7f343000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f3435600640 1 -- 172.21.15.122:0/2763512416 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343000f450 con 0x7f343815dff0 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f3435600640 1 -- 172.21.15.122:0/2763512416 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f343000f700 con 0x7f343815dff0 2024-09-17T05:36:02.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3438160b60 con 0x7f343815dff0 2024-09-17T05:36:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f3435600640 1 -- 172.21.15.122:0/2763512416 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343000fb50 con 0x7f343815dff0 2024-09-17T05:36:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343815f6b0 con 0x7f343815dff0 2024-09-17T05:36:02.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.014+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34380d2980 con 0x7f343815dff0 2024-09-17T05:36:02.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.018+0000 7f3435600640 1 -- 172.21.15.122:0/2763512416 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f343000fcf0 con 0x7f343815dff0 2024-09-17T05:36:02.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.018+0000 7f3435600640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3418074760 0x7f3418076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:02.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.018+0000 7f3435600640 1 -- 172.21.15.122:0/2763512416 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f3430093df0 con 0x7f343815dff0 2024-09-17T05:36:02.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.018+0000 7f343da00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3418074760 0x7f3418076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:02.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.018+0000 7f343da00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3418074760 0x7f3418076c20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f342c0af000 tx=0x7f342c01b100 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.022+0000 7f3435600640 1 -- 172.21.15.122:0/2763512416 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f343009d020 con 0x7f343815dff0 2024-09-17T05:36:02.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:02.218+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone608518", "target": ["mon-mgr", ""]}) -- 0x7f3438069e60 con 0x7f3418074760 2024-09-17T05:36:03.366 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:36:03.370 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:36:03.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.862+0000 7f3435600640 1 -- 172.21.15.122:0/2763512416 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f342c004320 con 0x7f3418074760 2024-09-17T05:36:03.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3418074760 msgr2=0x7f3418076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:03.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3418074760 0x7f3418076c20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f342c0af000 tx=0x7f342c01b100 comp rx=0 tx=0).stop 2024-09-17T05:36:03.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 msgr2=0x7f34380b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:03.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 0x7f34380b2590 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f343000cce0 tx=0x7f343000a620 comp rx=0 tx=0).stop 2024-09-17T05:36:03.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 shutdown_connections 2024-09-17T05:36:03.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3418074760 0x7f3418076c20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:03.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f343815dff0 0x7f34380b2590 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:03.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380d24d0 0x7f343815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:03.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 --2- 172.21.15.122:0/2763512416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cf950 0x7f343815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:03.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 >> 172.21.15.122:0/2763512416 conn(0x7f34380d3f10 msgr2=0x7f34380b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:03.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 shutdown_connections 2024-09-17T05:36:03.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:03.866+0000 7f343ee00640 1 -- 172.21.15.122:0/2763512416 wait complete. 2024-09-17T05:36:03.865 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.869+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.865 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.869+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.865 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.869+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.865 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.869+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.865 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.869+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.885 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.889+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.885 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.889+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.885 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.889+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.885 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.889+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.885 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:03.889+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:03.889 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone321815 2024-09-17T05:36:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 -- 172.21.15.122:0/840995762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e100b0b60 msgr2=0x7f5e100ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 --2- 172.21.15.122:0/840995762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e100b0b60 0x7f5e100ccbd0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5e10073e00 tx=0x7f5e0801f580 comp rx=0 tx=0).stop 2024-09-17T05:36:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 -- 172.21.15.122:0/840995762 shutdown_connections 2024-09-17T05:36:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 --2- 172.21.15.122:0/840995762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e100b0b60 0x7f5e100ccbd0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 --2- 172.21.15.122:0/840995762 >> v2:172.21.15.173:3300/0 conn(0x7f5e1015be10 0x7f5e100ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 -- 172.21.15.122:0/840995762 >> 172.21.15.122:0/840995762 conn(0x7f5e100d1da0 msgr2=0x7f5e100d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 -- 172.21.15.122:0/840995762 shutdown_connections 2024-09-17T05:36:04.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 -- 172.21.15.122:0/840995762 wait complete. 2024-09-17T05:36:04.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 Processor -- start 2024-09-17T05:36:04.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.046+0000 7f5e16400640 1 -- start start 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e100b0b60 0x7f5e100b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e1015be10 0x7f5e10151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e1015c1f0 0x7f5e100af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e100cd4a0 con 0x7f5e1015be10 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e100cd320 con 0x7f5e1015c1f0 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5e100cd620 con 0x7f5e100b0b60 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e1015be10 0x7f5e10151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e0f400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e100b0b60 0x7f5e100b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e1015be10 0x7f5e10151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36760/0 (socket says 172.21.15.122:36760) 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 -- 172.21.15.122:0/2575813958 learned_addr learned my addr 172.21.15.122:0/2575813958 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:04.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e0fe00640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e1015c1f0 0x7f5e100af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:04.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 -- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e100b0b60 msgr2=0x7f5e100b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:04.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e100b0b60 0x7f5e100b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:04.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 -- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e1015c1f0 msgr2=0x7f5e100af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:04.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e1015c1f0 0x7f5e100af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:04.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 -- 172.21.15.122:0/2575813958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e100b0800 con 0x7f5e1015be10 2024-09-17T05:36:04.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07e00640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e1015be10 0x7f5e10151bb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b440 tx=0x7f5df800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:04.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07400640 1 -- 172.21.15.122:0/2575813958 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8015020 con 0x7f5e1015be10 2024-09-17T05:36:04.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07400640 1 -- 172.21.15.122:0/2575813958 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5df8011980 con 0x7f5e1015be10 2024-09-17T05:36:04.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e07400640 1 -- 172.21.15.122:0/2575813958 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8011dd0 con 0x7f5e1015be10 2024-09-17T05:36:04.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1015d2e0 con 0x7f5e1015be10 2024-09-17T05:36:04.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1015d8a0 con 0x7f5e1015be10 2024-09-17T05:36:04.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.050+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e100cecc0 con 0x7f5e1015be10 2024-09-17T05:36:04.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.054+0000 7f5e07400640 1 -- 172.21.15.122:0/2575813958 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 37) ==== 99172+0+0 (secure 0 0 0) 0x7f5df8018020 con 0x7f5e1015be10 2024-09-17T05:36:04.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.054+0000 7f5e07400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5dec074760 0x7f5dec076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:04.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.054+0000 7f5e07400640 1 -- 172.21.15.122:0/2575813958 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f5df8093000 con 0x7f5e1015be10 2024-09-17T05:36:04.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.054+0000 7f5e0f400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5dec074760 0x7f5dec076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:04.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.054+0000 7f5e0f400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5dec074760 0x7f5dec076c20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5e00004630 tx=0x7f5e0000b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:04.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.058+0000 7f5e07400640 1 -- 172.21.15.122:0/2575813958 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5df801f030 con 0x7f5e1015be10 2024-09-17T05:36:04.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:04.250+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone321815", "target": ["mon-mgr", ""]}) -- 0x7f5e10070350 con 0x7f5dec074760 2024-09-17T05:36:05.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.562+0000 7f5e07400640 1 -- 172.21.15.122:0/2575813958 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f5df800f050 con 0x7f5e1015be10 2024-09-17T05:36:05.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.846+0000 7f5e07400640 1 -- 172.21.15.122:0/2575813958 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e10070350 con 0x7f5dec074760 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5dec074760 msgr2=0x7f5dec076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5dec074760 0x7f5dec076c20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5e00004630 tx=0x7f5e0000b390 comp rx=0 tx=0).stop 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e1015be10 msgr2=0x7f5e10151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e1015be10 0x7f5e10151bb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b440 tx=0x7f5df800b900 comp rx=0 tx=0).stop 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 shutdown_connections 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5dec074760 0x7f5dec076c20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e1015c1f0 0x7f5e100af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e1015be10 0x7f5e10151bb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 --2- 172.21.15.122:0/2575813958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5e100b0b60 0x7f5e100b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:05.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 >> 172.21.15.122:0/2575813958 conn(0x7f5e100d1da0 msgr2=0x7f5e100aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:05.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 shutdown_connections 2024-09-17T05:36:05.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:05.850+0000 7f5e16400640 1 -- 172.21.15.122:0/2575813958 wait complete. 2024-09-17T05:36:05.876 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone9004 2024-09-17T05:36:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- 172.21.15.122:0/2204540883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c0d28e0 msgr2=0x7f777c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 --2- 172.21.15.122:0/2204540883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c0d28e0 0x7f777c0d0170 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7764009900 tx=0x7f776401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- 172.21.15.122:0/2204540883 shutdown_connections 2024-09-17T05:36:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 --2- 172.21.15.122:0/2204540883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c0d28e0 0x7f777c0d0170 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 --2- 172.21.15.122:0/2204540883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f777c15f390 0x7f777c0cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- 172.21.15.122:0/2204540883 >> 172.21.15.122:0/2204540883 conn(0x7f777c0d1dc0 msgr2=0x7f777c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- 172.21.15.122:0/2204540883 shutdown_connections 2024-09-17T05:36:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- 172.21.15.122:0/2204540883 wait complete. 2024-09-17T05:36:06.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 Processor -- start 2024-09-17T05:36:06.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- start start 2024-09-17T05:36:06.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f777c0d06b0 0x7f777c0b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c0d28e0 0x7f777c155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c15f390 0x7f777c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f777c169110 con 0x7f777c15f390 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f777c168f90 con 0x7f777c0d28e0 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7782a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f777c169290 con 0x7f777c0d06b0 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7780a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c15f390 0x7f777c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7780a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c15f390 0x7f777c156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36768/0 (socket says 172.21.15.122:36768) 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f7780a00640 1 -- 172.21.15.122:0/3608046130 learned_addr learned my addr 172.21.15.122:0/3608046130 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7773400640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c0d28e0 0x7f777c155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.030+0000 7f777be00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f777c0d06b0 0x7f777c0b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7780a00640 1 -- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f777c0d06b0 msgr2=0x7f777c0b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7780a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f777c0d06b0 0x7f777c0b7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7780a00640 1 -- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c0d28e0 msgr2=0x7f777c155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7780a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c0d28e0 0x7f777c155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7780a00640 1 -- 172.21.15.122:0/3608046130 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c1f60f0 con 0x7f777c15f390 2024-09-17T05:36:06.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7780a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c15f390 0x7f777c156430 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f777400c7c0 tx=0x7f777400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:06.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7778c00640 1 -- 172.21.15.122:0/3608046130 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774003b80 con 0x7f777c15f390 2024-09-17T05:36:06.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7778c00640 1 -- 172.21.15.122:0/3608046130 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7774003d20 con 0x7f777c15f390 2024-09-17T05:36:06.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7778c00640 1 -- 172.21.15.122:0/3608046130 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777400aab0 con 0x7f777c15f390 2024-09-17T05:36:06.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777c1f63d0 con 0x7f777c15f390 2024-09-17T05:36:06.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777c1f6990 con 0x7f777c15f390 2024-09-17T05:36:06.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777c0b1350 con 0x7f777c15f390 2024-09-17T05:36:06.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.034+0000 7f7778c00640 1 -- 172.21.15.122:0/3608046130 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f777400ac50 con 0x7f777c15f390 2024-09-17T05:36:06.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.038+0000 7f7778c00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7758074880 0x7f7758076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.038+0000 7f7778c00640 1 -- 172.21.15.122:0/3608046130 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f7774092c00 con 0x7f777c15f390 2024-09-17T05:36:06.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.038+0000 7f777be00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7758074880 0x7f7758076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.038+0000 7f777be00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7758074880 0x7f7758076d40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f777c1f38d0 tx=0x7f776c00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:06.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.042+0000 7f7778c00640 1 -- 172.21.15.122:0/3608046130 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f777405f160 con 0x7f777c15f390 2024-09-17T05:36:06.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.230+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone9004", "target": ["mon-mgr", ""]}) -- 0x7f777c15cc80 con 0x7f7758074880 2024-09-17T05:36:06.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7778c00640 1 -- 172.21.15.122:0/3608046130 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f777c15cc80 con 0x7f7758074880 2024-09-17T05:36:06.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7758074880 msgr2=0x7f7758076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7758074880 0x7f7758076d40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f777c1f38d0 tx=0x7f776c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c15f390 msgr2=0x7f777c156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c15f390 0x7f777c156430 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f777400c7c0 tx=0x7f777400cc80 comp rx=0 tx=0).stop 2024-09-17T05:36:06.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 shutdown_connections 2024-09-17T05:36:06.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7758074880 0x7f7758076d40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f777c15f390 0x7f777c156430 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f777c0d28e0 0x7f777c155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 --2- 172.21.15.122:0/3608046130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f777c0d06b0 0x7f777c0b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 >> 172.21.15.122:0/3608046130 conn(0x7f777c0d1dc0 msgr2=0x7f777c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:06.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 shutdown_connections 2024-09-17T05:36:06.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.258+0000 7f7782a00640 1 -- 172.21.15.122:0/3608046130 wait complete. 2024-09-17T05:36:06.277 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone84910 2024-09-17T05:36:06.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a27200640 1 -- 172.21.15.122:0/3550746938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a040041d0 msgr2=0x7f7a04004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a27200640 1 --2- 172.21.15.122:0/3550746938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a040041d0 0x7f7a04004810 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7a08006fa0 tx=0x7f7a0801cee0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a25200640 1 -- 172.21.15.122:0/3550746938 reap_dead start 2024-09-17T05:36:06.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a27200640 1 -- 172.21.15.122:0/3550746938 shutdown_connections 2024-09-17T05:36:06.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a27200640 1 -- 172.21.15.122:0/3550746938 >> 172.21.15.122:0/3550746938 conn(0x7f7a200cfbc0 msgr2=0x7f7a200cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:06.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a27200640 1 -- 172.21.15.122:0/3550746938 shutdown_connections 2024-09-17T05:36:06.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a27200640 1 -- 172.21.15.122:0/3550746938 wait complete. 2024-09-17T05:36:06.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a27200640 1 Processor -- start 2024-09-17T05:36:06.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.430+0000 7f7a27200640 1 -- start start 2024-09-17T05:36:06.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040041d0 0x7f7a2006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a20150ab0 0x7f7a201f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a20148c10 0x7f7a20164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20168070 con 0x7f7a20148c10 2024-09-17T05:36:06.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20167ef0 con 0x7f7a20150ab0 2024-09-17T05:36:06.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7a201681f0 con 0x7f7a040041d0 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a20148c10 0x7f7a20164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a20148c10 0x7f7a20164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36784/0 (socket says 172.21.15.122:36784) 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 -- 172.21.15.122:0/2358144024 learned_addr learned my addr 172.21.15.122:0/2358144024 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a1fe00640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040041d0 0x7f7a2006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a17400640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a20150ab0 0x7f7a201f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 -- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040041d0 msgr2=0x7f7a2006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040041d0 0x7f7a2006e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 -- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a20150ab0 msgr2=0x7f7a201f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a20150ab0 0x7f7a201f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 -- 172.21.15.122:0/2358144024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2006ecb0 con 0x7f7a20148c10 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a25200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a20148c10 0x7f7a20164e80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7a200b2950 tx=0x7f7a18010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a1cc00640 1 -- 172.21.15.122:0/2358144024 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800ee10 con 0x7f7a20148c10 2024-09-17T05:36:06.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a1cc00640 1 -- 172.21.15.122:0/2358144024 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a18015070 con 0x7f7a20148c10 2024-09-17T05:36:06.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a1cc00640 1 -- 172.21.15.122:0/2358144024 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18003d10 con 0x7f7a20148c10 2024-09-17T05:36:06.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2006ef90 con 0x7f7a20148c10 2024-09-17T05:36:06.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2006f550 con 0x7f7a20148c10 2024-09-17T05:36:06.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.434+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a20069410 con 0x7f7a20148c10 2024-09-17T05:36:06.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.438+0000 7f7a1cc00640 1 -- 172.21.15.122:0/2358144024 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f7a18004030 con 0x7f7a20148c10 2024-09-17T05:36:06.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.438+0000 7f7a1cc00640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79fc074880 0x7f79fc076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.438+0000 7f7a1cc00640 1 -- 172.21.15.122:0/2358144024 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f7a18093450 con 0x7f7a20148c10 2024-09-17T05:36:06.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.438+0000 7f7a1fe00640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79fc074880 0x7f79fc076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.438+0000 7f7a1fe00640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79fc074880 0x7f79fc076d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a10000c10 tx=0x7f7a10006b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:06.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.442+0000 7f7a1cc00640 1 -- 172.21.15.122:0/2358144024 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7a1805f990 con 0x7f7a20148c10 2024-09-17T05:36:06.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.718+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone84910", "target": ["mon-mgr", ""]}) -- 0x7f7a2006f790 con 0x7f79fc074880 2024-09-17T05:36:06.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a1cc00640 1 -- 172.21.15.122:0/2358144024 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7a2006f790 con 0x7f79fc074880 2024-09-17T05:36:06.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79fc074880 msgr2=0x7f79fc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79fc074880 0x7f79fc076d40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7a10000c10 tx=0x7f7a10006b20 comp rx=0 tx=0).stop 2024-09-17T05:36:06.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a20148c10 msgr2=0x7f7a20164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a20148c10 0x7f7a20164e80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7a200b2950 tx=0x7f7a18010e80 comp rx=0 tx=0).stop 2024-09-17T05:36:06.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 shutdown_connections 2024-09-17T05:36:06.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f79fc074880 0x7f79fc076d40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a20148c10 0x7f7a20164e80 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a20150ab0 0x7f7a201f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 --2- 172.21.15.122:0/2358144024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a040041d0 0x7f7a2006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 >> 172.21.15.122:0/2358144024 conn(0x7f7a200cfbc0 msgr2=0x7f7a200b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:06.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 shutdown_connections 2024-09-17T05:36:06.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.738+0000 7f7a27200640 1 -- 172.21.15.122:0/2358144024 wait complete. 2024-09-17T05:36:06.756 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T05:36:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- 172.21.15.122:0/281273876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 msgr2=0x7f5d400adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 --2- 172.21.15.122:0/281273876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 0x7f5d400adf50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5d4006e170 tx=0x7f5d3001c570 comp rx=0 tx=0).stop 2024-09-17T05:36:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- 172.21.15.122:0/281273876 shutdown_connections 2024-09-17T05:36:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 --2- 172.21.15.122:0/281273876 >> v2:172.21.15.173:3300/0 conn(0x7f5d400aef60 0x7f5d400ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 --2- 172.21.15.122:0/281273876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d400ae9d0 0x7f5d40072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 --2- 172.21.15.122:0/281273876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 0x7f5d400adf50 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- 172.21.15.122:0/281273876 >> 172.21.15.122:0/281273876 conn(0x7f5d400cfbc0 msgr2=0x7f5d400cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- 172.21.15.122:0/281273876 shutdown_connections 2024-09-17T05:36:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- 172.21.15.122:0/281273876 wait complete. 2024-09-17T05:36:06.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 Processor -- start 2024-09-17T05:36:06.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- start start 2024-09-17T05:36:06.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d400ae9d0 0x7f5d40158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d400aef60 0x7f5d401faf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 0x7f5d40159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4015ec10 con 0x7f5d4015e360 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4015ea90 con 0x7f5d400ae9d0 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d47000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4015ed90 con 0x7f5d400aef60 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d45000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 0x7f5d40159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d45000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 0x7f5d40159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36790/0 (socket says 172.21.15.122:36790) 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d45000640 1 -- 172.21.15.122:0/309178810 learned_addr learned my addr 172.21.15.122:0/309178810 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d3fe00640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d400ae9d0 0x7f5d40158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.910+0000 7f5d37e00640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d400aef60 0x7f5d401faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d45000640 1 -- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d400aef60 msgr2=0x7f5d401faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d45000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d400aef60 0x7f5d401faf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d45000640 1 -- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d400ae9d0 msgr2=0x7f5d40158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d45000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d400ae9d0 0x7f5d40158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d45000640 1 -- 172.21.15.122:0/309178810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d401fb8c0 con 0x7f5d4015e360 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d3fe00640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d400ae9d0 0x7f5d40158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d45000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 0x7f5d40159840 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800eab0 tx=0x7f5d3800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d3cc00640 1 -- 172.21.15.122:0/309178810 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38004b30 con 0x7f5d4015e360 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d3cc00640 1 -- 172.21.15.122:0/309178810 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d38004dc0 con 0x7f5d4015e360 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d401fbba0 con 0x7f5d4015e360 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d3cc00640 1 -- 172.21.15.122:0/309178810 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38005280 con 0x7f5d4015e360 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d401fb4b0 con 0x7f5d4015e360 2024-09-17T05:36:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d40072870 con 0x7f5d4015e360 2024-09-17T05:36:06.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d3cc00640 1 -- 172.21.15.122:0/309178810 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f5d38005420 con 0x7f5d4015e360 2024-09-17T05:36:06.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d3cc00640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d1c074880 0x7f5d1c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:06.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.914+0000 7f5d3cc00640 1 -- 172.21.15.122:0/309178810 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f5d38093c20 con 0x7f5d4015e360 2024-09-17T05:36:06.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.918+0000 7f5d3fe00640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d1c074880 0x7f5d1c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:06.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.918+0000 7f5d3fe00640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d1c074880 0x7f5d1c076d40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5d300af000 tx=0x7f5d3001adb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:06.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:06.918+0000 7f5d3cc00640 1 -- 172.21.15.122:0/309178810 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5d38060180 con 0x7f5d4015e360 2024-09-17T05:36:07.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.114+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d400ae0f0 con 0x7f5d1c074880 2024-09-17T05:36:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d3cc00640 1 -- 172.21.15.122:0/309178810 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5d400ae0f0 con 0x7f5d1c074880 2024-09-17T05:36:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:07.125 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:36:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1" 2024-09-17T05:36:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:07.125 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:36:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d1c074880 msgr2=0x7f5d1c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d1c074880 0x7f5d1c076d40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5d300af000 tx=0x7f5d3001adb0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 msgr2=0x7f5d40159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 0x7f5d40159840 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800eab0 tx=0x7f5d3800ef70 comp rx=0 tx=0).stop 2024-09-17T05:36:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 shutdown_connections 2024-09-17T05:36:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d1c074880 0x7f5d1c076d40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d4015e360 0x7f5d40159840 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d400aef60 0x7f5d401faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 --2- 172.21.15.122:0/309178810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d400ae9d0 0x7f5d40158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 >> 172.21.15.122:0/309178810 conn(0x7f5d400cfbc0 msgr2=0x7f5d400b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 shutdown_connections 2024-09-17T05:36:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.130+0000 7f5d47000640 1 -- 172.21.15.122:0/309178810 wait complete. 2024-09-17T05:36:07.147 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T05:36:07.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- 172.21.15.122:0/3256800182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04641605d0 msgr2=0x7f04640d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 --2- 172.21.15.122:0/3256800182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04641605d0 0x7f04640d0690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0458009900 tx=0x7f045801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- 172.21.15.122:0/3256800182 shutdown_connections 2024-09-17T05:36:07.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 --2- 172.21.15.122:0/3256800182 >> v2:172.21.15.173:3300/0 conn(0x7f04640b1650 0x7f04640b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 --2- 172.21.15.122:0/3256800182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04641605d0 0x7f04640d0690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- 172.21.15.122:0/3256800182 >> 172.21.15.122:0/3256800182 conn(0x7f04640d1dd0 msgr2=0x7f04640d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:07.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- 172.21.15.122:0/3256800182 shutdown_connections 2024-09-17T05:36:07.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- 172.21.15.122:0/3256800182 wait complete. 2024-09-17T05:36:07.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 Processor -- start 2024-09-17T05:36:07.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- start start 2024-09-17T05:36:07.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04640aeb00 0x7f0464157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04640b1650 0x7f04641f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04641605d0 0x7f0464151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04640b0810 con 0x7f04641605d0 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04640b0ee0 con 0x7f04640b1650 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046c000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f04640b1220 con 0x7f04640aeb00 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04640b1650 0x7f04641f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f046a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04641605d0 0x7f0464151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04640b1650 0x7f04641f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:52194/0 (socket says 172.21.15.122:52194) 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 -- 172.21.15.122:0/3865707397 learned_addr learned my addr 172.21.15.122:0/3865707397 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0469600640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04640aeb00 0x7f0464157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 -- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04640aeb00 msgr2=0x7f0464157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04640aeb00 0x7f0464157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 -- 172.21.15.122:0/3865707397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04641605d0 msgr2=0x7f0464151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04641605d0 0x7f0464151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 -- 172.21.15.122:0/3865707397 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04640b76f0 con 0x7f04640b1650 2024-09-17T05:36:07.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0469600640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04640aeb00 0x7f0464157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:36:07.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.302+0000 7f0468c00640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04640b1650 0x7f04641f5d30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f045000ea40 tx=0x7f045000ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:07.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f0461600640 1 -- 172.21.15.122:0/3865707397 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045000cde0 con 0x7f04640b1650 2024-09-17T05:36:07.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f0461600640 1 -- 172.21.15.122:0/3865707397 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0450004020 con 0x7f04640b1650 2024-09-17T05:36:07.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f0461600640 1 -- 172.21.15.122:0/3865707397 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450004470 con 0x7f04640b1650 2024-09-17T05:36:07.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04641f67d0 con 0x7f04640b1650 2024-09-17T05:36:07.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0464157610 con 0x7f04640b1650 2024-09-17T05:36:07.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04640b1a30 con 0x7f04640b1650 2024-09-17T05:36:07.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f0461600640 1 -- 172.21.15.122:0/3865707397 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f0450003600 con 0x7f04640b1650 2024-09-17T05:36:07.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f0461600640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0444074880 0x7f0444076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:07.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f0461600640 1 -- 172.21.15.122:0/3865707397 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f04500937c0 con 0x7f04640b1650 2024-09-17T05:36:07.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.306+0000 7f0469600640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0444074880 0x7f0444076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:07.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.310+0000 7f0469600640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0444074880 0x7f0444076d40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f04580af000 tx=0x7f045801b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:07.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.310+0000 7f0461600640 1 -- 172.21.15.122:0/3865707397 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f045005fc80 con 0x7f04640b1650 2024-09-17T05:36:07.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.510+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0464157400 con 0x7f0444074880 2024-09-17T05:36:07.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f0461600640 1 -- 172.21.15.122:0/3865707397 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0464157400 con 0x7f0444074880 2024-09-17T05:36:07.538 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:07.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0444074880 msgr2=0x7f0444076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0444074880 0x7f0444076d40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f04580af000 tx=0x7f045801b120 comp rx=0 tx=0).stop 2024-09-17T05:36:07.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04640b1650 msgr2=0x7f04641f5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04640b1650 0x7f04641f5d30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f045000ea40 tx=0x7f045000ef00 comp rx=0 tx=0).stop 2024-09-17T05:36:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 shutdown_connections 2024-09-17T05:36:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0444074880 0x7f0444076d40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04641605d0 0x7f0464151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04640b1650 0x7f04641f5d30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 --2- 172.21.15.122:0/3865707397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04640aeb00 0x7f0464157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 >> 172.21.15.122:0/3865707397 conn(0x7f04640d1dd0 msgr2=0x7f04640b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:07.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 shutdown_connections 2024-09-17T05:36:07.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.542+0000 7f046c000640 1 -- 172.21.15.122:0/3865707397 wait complete. 2024-09-17T05:36:07.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T05:36:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.710+0000 7fc477000640 1 -- 172.21.15.122:0/3344168167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc454003c40 msgr2=0x7fc454004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.710+0000 7fc477000640 1 --2- 172.21.15.122:0/3344168167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc454003c40 0x7fc454004040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c3f0 tx=0x7fc46001f660 comp rx=0 tx=0).stop 2024-09-17T05:36:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.710+0000 7fc477000640 1 -- 172.21.15.122:0/3344168167 shutdown_connections 2024-09-17T05:36:07.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.710+0000 7fc477000640 1 --2- 172.21.15.122:0/3344168167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc454003c40 0x7fc454004040 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.710+0000 7fc477000640 1 -- 172.21.15.122:0/3344168167 >> 172.21.15.122:0/3344168167 conn(0x7fc4700d1e00 msgr2=0x7fc4700d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.710+0000 7fc477000640 1 -- 172.21.15.122:0/3344168167 shutdown_connections 2024-09-17T05:36:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.710+0000 7fc477000640 1 -- 172.21.15.122:0/3344168167 wait complete. 2024-09-17T05:36:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 Processor -- start 2024-09-17T05:36:07.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 -- start start 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc454003c40 0x7fc4701f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47015ad50 0x7fc4701583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4701594f0 0x7fc47015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4700b2d90 con 0x7fc47015ad50 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4700b2c10 con 0x7fc4701594f0 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc4700b34a0 con 0x7fc454003c40 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47015ad50 0x7fc4701583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47015ad50 0x7fc4701583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36804/0 (socket says 172.21.15.122:36804) 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 -- 172.21.15.122:0/4222659067 learned_addr learned my addr 172.21.15.122:0/4222659067 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc475000640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4701594f0 0x7fc47015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:07.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc46fe00640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc454003c40 0x7fc4701f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 -- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc454003c40 msgr2=0x7fc4701f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc454003c40 0x7fc4701f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 -- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4701594f0 msgr2=0x7fc47015b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4701594f0 0x7fc47015b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 -- 172.21.15.122:0/4222659067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47015b1d0 con 0x7fc47015ad50 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc467400640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47015ad50 0x7fc4701583f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc458011a70 tx=0x7fc458011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc46cc00640 1 -- 172.21.15.122:0/4222659067 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458004460 con 0x7fc47015ad50 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc46cc00640 1 -- 172.21.15.122:0/4222659067 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc45800f940 con 0x7fc47015ad50 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc46cc00640 1 -- 172.21.15.122:0/4222659067 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45800fd50 con 0x7fc47015ad50 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc470159cb0 con 0x7fc47015ad50 2024-09-17T05:36:07.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47015a270 con 0x7fc47015ad50 2024-09-17T05:36:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.714+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4700b3970 con 0x7fc47015ad50 2024-09-17T05:36:07.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.718+0000 7fc46cc00640 1 -- 172.21.15.122:0/4222659067 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fc458005020 con 0x7fc47015ad50 2024-09-17T05:36:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.718+0000 7fc46cc00640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc44c0748b0 0x7fc44c076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.718+0000 7fc46cc00640 1 -- 172.21.15.122:0/4222659067 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fc458093a70 con 0x7fc47015ad50 2024-09-17T05:36:07.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.718+0000 7fc46fe00640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc44c0748b0 0x7fc44c076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:07.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.718+0000 7fc46fe00640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc44c0748b0 0x7fc44c076d70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc4600b0000 tx=0x7fc46001c4a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:07.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.722+0000 7fc46cc00640 1 -- 172.21.15.122:0/4222659067 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc45805fff0 con 0x7fc47015ad50 2024-09-17T05:36:07.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.918+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc4700b7350 con 0x7fc44c0748b0 2024-09-17T05:36:07.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.934+0000 7fc46cc00640 1 -- 172.21.15.122:0/4222659067 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc4700b7350 con 0x7fc44c0748b0 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc44c0748b0 msgr2=0x7fc44c076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc44c0748b0 0x7fc44c076d70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc4600b0000 tx=0x7fc46001c4a0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47015ad50 msgr2=0x7fc4701583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47015ad50 0x7fc4701583f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc458011a70 tx=0x7fc458011f30 comp rx=0 tx=0).stop 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 shutdown_connections 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc44c0748b0 0x7fc44c076d70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4701594f0 0x7fc47015b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc47015ad50 0x7fc4701583f0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 --2- 172.21.15.122:0/4222659067 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc454003c40 0x7fc4701f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:07.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 >> 172.21.15.122:0/4222659067 conn(0x7fc4700d1e00 msgr2=0x7fc4700ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:07.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 shutdown_connections 2024-09-17T05:36:07.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:07.938+0000 7fc477000640 1 -- 172.21.15.122:0/4222659067 wait complete. 2024-09-17T05:36:07.955 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone523517 2024-09-17T05:36:08.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 -- 172.21.15.122:0/4017321426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815e6c0 msgr2=0x7f214815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 --2- 172.21.15.122:0/4017321426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815e6c0 0x7f214815a630 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2130009930 tx=0x7f213001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 -- 172.21.15.122:0/4017321426 shutdown_connections 2024-09-17T05:36:08.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 --2- 172.21.15.122:0/4017321426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2148070620 0x7f214815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 --2- 172.21.15.122:0/4017321426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815e6c0 0x7f214815a630 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 -- 172.21.15.122:0/4017321426 >> 172.21.15.122:0/4017321426 conn(0x7f21480cfa80 msgr2=0x7f21480cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:08.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 -- 172.21.15.122:0/4017321426 shutdown_connections 2024-09-17T05:36:08.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 -- 172.21.15.122:0/4017321426 wait complete. 2024-09-17T05:36:08.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 Processor -- start 2024-09-17T05:36:08.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.106+0000 7f214da00640 1 -- start start 2024-09-17T05:36:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2148070240 0x7f2148150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2148070620 0x7f21480c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815c170 0x7f21480b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2148167be0 con 0x7f2148070620 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2148167a60 con 0x7f214815c170 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2148167d60 con 0x7f2148070240 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2148070620 0x7f21480c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2148070620 0x7f21480c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36814/0 (socket says 172.21.15.122:36814) 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 -- 172.21.15.122:0/3433963426 learned_addr learned my addr 172.21.15.122:0/3433963426 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214d000640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815c170 0x7f21480b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f2147e00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2148070240 0x7f2148150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 -- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2148070240 msgr2=0x7f2148150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2148070240 0x7f2148150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 -- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815c170 msgr2=0x7f21480b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815c170 0x7f21480b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 -- 172.21.15.122:0/3433963426 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21481555c0 con 0x7f2148070620 2024-09-17T05:36:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214d000640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815c170 0x7f21480b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:36:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f213f400640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2148070620 0x7f21480c29f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f213001a510 tx=0x7f2130005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f2144c00640 1 -- 172.21.15.122:0/3433963426 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21300b2040 con 0x7f2148070620 2024-09-17T05:36:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f2144c00640 1 -- 172.21.15.122:0/3433963426 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2130005130 con 0x7f2148070620 2024-09-17T05:36:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f2144c00640 1 -- 172.21.15.122:0/3433963426 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130005580 con 0x7f2148070620 2024-09-17T05:36:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214815c600 con 0x7f2148070620 2024-09-17T05:36:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214815cb50 con 0x7f2148070620 2024-09-17T05:36:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.110+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214806e640 con 0x7f2148070620 2024-09-17T05:36:08.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.114+0000 7f2144c00640 1 -- 172.21.15.122:0/3433963426 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f21300046d0 con 0x7f2148070620 2024-09-17T05:36:08.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.114+0000 7f2144c00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2124074880 0x7f2124076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:08.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.114+0000 7f2144c00640 1 -- 172.21.15.122:0/3433963426 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f21300bd070 con 0x7f2148070620 2024-09-17T05:36:08.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.114+0000 7f2147e00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2124074880 0x7f2124076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:08.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.114+0000 7f2147e00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2124074880 0x7f2124076d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f21480b53d0 tx=0x7f2138009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:08.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.118+0000 7f2144c00640 1 -- 172.21.15.122:0/3433963426 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f21300164d0 con 0x7f2148070620 2024-09-17T05:36:08.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.310+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone523517", "target": ["mon-mgr", ""]}) -- 0x7f21480c9eb0 con 0x7f2124074880 2024-09-17T05:36:08.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.330+0000 7f2144c00640 1 -- 172.21.15.122:0/3433963426 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f21480c9eb0 con 0x7f2124074880 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2124074880 msgr2=0x7f2124076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2124074880 0x7f2124076d40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f21480b53d0 tx=0x7f2138009040 comp rx=0 tx=0).stop 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2148070620 msgr2=0x7f21480c29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2148070620 0x7f21480c29f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f213001a510 tx=0x7f2130005720 comp rx=0 tx=0).stop 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 shutdown_connections 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2124074880 0x7f2124076d40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f214815c170 0x7f21480b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2148070620 0x7f21480c29f0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 --2- 172.21.15.122:0/3433963426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2148070240 0x7f2148150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 >> 172.21.15.122:0/3433963426 conn(0x7f21480cfa80 msgr2=0x7f21481ece20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:08.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 shutdown_connections 2024-09-17T05:36:08.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.334+0000 7f214da00640 1 -- 172.21.15.122:0/3433963426 wait complete. 2024-09-17T05:36:08.355 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone59558 2024-09-17T05:36:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.510+0000 7f56bbe00640 1 -- 172.21.15.122:0/2391768457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0003d60 msgr2=0x7f56a00041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.510+0000 7f56bbe00640 1 --2- 172.21.15.122:0/2391768457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0003d60 0x7f56a00041e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f56bc15fa60 tx=0x7f56b401cc00 comp rx=0 tx=0).stop 2024-09-17T05:36:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.510+0000 7f56bb400640 1 -- 172.21.15.122:0/2391768457 reap_dead start 2024-09-17T05:36:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.510+0000 7f56bbe00640 1 -- 172.21.15.122:0/2391768457 shutdown_connections 2024-09-17T05:36:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.510+0000 7f56bbe00640 1 -- 172.21.15.122:0/2391768457 >> 172.21.15.122:0/2391768457 conn(0x7f56bc0d1c90 msgr2=0x7f56bc0d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.510+0000 7f56bbe00640 1 -- 172.21.15.122:0/2391768457 shutdown_connections 2024-09-17T05:36:08.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.510+0000 7f56bbe00640 1 -- 172.21.15.122:0/2391768457 wait complete. 2024-09-17T05:36:08.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bbe00640 1 Processor -- start 2024-09-17T05:36:08.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bbe00640 1 -- start start 2024-09-17T05:36:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0003d60 0x7f56bc0c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc151530 0x7f56bc0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc151000 0x7f56bc0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bbe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc06e0c0 con 0x7f56a0003d60 2024-09-17T05:36:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bbe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc06df40 con 0x7f56bc151530 2024-09-17T05:36:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bbe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc06e240 con 0x7f56bc151000 2024-09-17T05:36:08.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bb400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc151000 0x7f56bc0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:08.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bb400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc151000 0x7f56bc0c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:36484/0 (socket says 172.21.15.122:36484) 2024-09-17T05:36:08.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56bb400640 1 -- 172.21.15.122:0/3604346996 learned_addr learned my addr 172.21.15.122:0/3604346996 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:08.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56ba000640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc151530 0x7f56bc0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:08.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.514+0000 7f56baa00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0003d60 0x7f56bc0c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:08.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56bb400640 1 -- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc151530 msgr2=0x7f56bc0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56bb400640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc151530 0x7f56bc0cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56bb400640 1 -- 172.21.15.122:0/3604346996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0003d60 msgr2=0x7f56bc0c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56bb400640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0003d60 0x7f56bc0c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56bb400640 1 -- 172.21.15.122:0/3604346996 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc0c90b0 con 0x7f56bc151000 2024-09-17T05:36:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56baa00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0003d60 0x7f56bc0c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:36:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56ba000640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc151530 0x7f56bc0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:36:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56bb400640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc151000 0x7f56bc0c93a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f56b40985b0 tx=0x7f56b40a5e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56b2a00640 1 -- 172.21.15.122:0/3604346996 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4006c50 con 0x7f56bc151000 2024-09-17T05:36:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56b2a00640 1 -- 172.21.15.122:0/3604346996 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b4001b40 con 0x7f56bc151000 2024-09-17T05:36:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56b2a00640 1 -- 172.21.15.122:0/3604346996 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b40061b0 con 0x7f56bc151000 2024-09-17T05:36:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc0cb5c0 con 0x7f56bc151000 2024-09-17T05:36:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.518+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc0c8d60 con 0x7f56bc151000 2024-09-17T05:36:08.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.522+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56940052a0 con 0x7f56bc151000 2024-09-17T05:36:08.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.522+0000 7f56b2a00640 1 -- 172.21.15.122:0/3604346996 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f56b4006350 con 0x7f56bc151000 2024-09-17T05:36:08.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.526+0000 7f56b2a00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0748b0 0x7f569c076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:08.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.526+0000 7f56b2a00640 1 -- 172.21.15.122:0/3604346996 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f56b412d060 con 0x7f56bc151000 2024-09-17T05:36:08.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.526+0000 7f56baa00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0748b0 0x7f569c076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:08.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.526+0000 7f56baa00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0748b0 0x7f569c076d70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f56ac001130 tx=0x7f56ac006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:08.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.534+0000 7f56b2a00640 1 -- 172.21.15.122:0/3604346996 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f56bc0d05e0 con 0x7f56bc151000 2024-09-17T05:36:08.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.898+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone59558", "target": ["mon-mgr", ""]}) -- 0x7f5694000bd0 con 0x7f569c0748b0 2024-09-17T05:36:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.934+0000 7f56b2a00640 1 -- 172.21.15.122:0/3604346996 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5694000bd0 con 0x7f569c0748b0 2024-09-17T05:36:08.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0748b0 msgr2=0x7f569c076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0748b0 0x7f569c076d70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f56ac001130 tx=0x7f56ac006040 comp rx=0 tx=0).stop 2024-09-17T05:36:08.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc151000 msgr2=0x7f56bc0c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc151000 0x7f56bc0c93a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f56b40985b0 tx=0x7f56b40a5e50 comp rx=0 tx=0).stop 2024-09-17T05:36:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 shutdown_connections 2024-09-17T05:36:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0748b0 0x7f569c076d70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc151000 0x7f56bc0c93a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc151530 0x7f56bc0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 --2- 172.21.15.122:0/3604346996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56a0003d60 0x7f56bc0c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 >> 172.21.15.122:0/3604346996 conn(0x7f56bc0d1c90 msgr2=0x7f56bc15aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 shutdown_connections 2024-09-17T05:36:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:08.942+0000 7f56bbe00640 1 -- 172.21.15.122:0/3604346996 wait complete. 2024-09-17T05:36:08.974 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone479614 2024-09-17T05:36:09.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 -- 172.21.15.122:0/3955907976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 msgr2=0x7ff0ac0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 --2- 172.21.15.122:0/3955907976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 0x7ff0ac0ce700 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009900 tx=0x7ff0a001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 -- 172.21.15.122:0/3955907976 shutdown_connections 2024-09-17T05:36:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 --2- 172.21.15.122:0/3955907976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0ac0b0b60 0x7ff0ac0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 --2- 172.21.15.122:0/3955907976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 0x7ff0ac0ce700 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 -- 172.21.15.122:0/3955907976 >> 172.21.15.122:0/3955907976 conn(0x7ff0ac0d1da0 msgr2=0x7ff0ac0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 -- 172.21.15.122:0/3955907976 shutdown_connections 2024-09-17T05:36:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 -- 172.21.15.122:0/3955907976 wait complete. 2024-09-17T05:36:09.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 Processor -- start 2024-09-17T05:36:09.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.174+0000 7ff0b4000640 1 -- start start 2024-09-17T05:36:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0ac0b0b60 0x7ff0ac1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 0x7ff0ac158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0ac15c1f0 0x7ff0ac070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac169040 con 0x7ff0ac15be10 2024-09-17T05:36:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac168ec0 con 0x7ff0ac15c1f0 2024-09-17T05:36:09.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ac1691c0 con 0x7ff0ac0b0b60 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 0x7ff0ac158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 0x7ff0ac158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36842/0 (socket says 172.21.15.122:36842) 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 -- 172.21.15.122:0/575893110 learned_addr learned my addr 172.21.15.122:0/575893110 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b2000640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0ac15c1f0 0x7ff0ac070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b1600640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0ac0b0b60 0x7ff0ac1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 -- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0ac0b0b60 msgr2=0x7ff0ac1595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0ac0b0b60 0x7ff0ac1595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 -- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0ac15c1f0 msgr2=0x7ff0ac070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0ac15c1f0 0x7ff0ac070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 -- 172.21.15.122:0/575893110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac155480 con 0x7ff0ac15be10 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b0c00640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 0x7ff0ac158340 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff09800c3c0 tx=0x7ff09800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0a9600640 1 -- 172.21.15.122:0/575893110 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098017070 con 0x7ff0ac15be10 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0a9600640 1 -- 172.21.15.122:0/575893110 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0980053c0 con 0x7ff0ac15be10 2024-09-17T05:36:09.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac1fba10 con 0x7ff0ac15be10 2024-09-17T05:36:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0a9600640 1 -- 172.21.15.122:0/575893110 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098003860 con 0x7ff0ac15be10 2024-09-17T05:36:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac1fbcf0 con 0x7ff0ac15be10 2024-09-17T05:36:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac0ccbd0 con 0x7ff0ac15be10 2024-09-17T05:36:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.178+0000 7ff0a9600640 1 -- 172.21.15.122:0/575893110 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7ff09801a070 con 0x7ff0ac15be10 2024-09-17T05:36:09.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.182+0000 7ff0a9600640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff08c074880 0x7ff08c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:09.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.182+0000 7ff0a9600640 1 -- 172.21.15.122:0/575893110 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff0980921a0 con 0x7ff0ac15be10 2024-09-17T05:36:09.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.182+0000 7ff0b1600640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff08c074880 0x7ff08c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:09.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.182+0000 7ff0b1600640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff08c074880 0x7ff08c076d40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00af000 tx=0x7ff0a001b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:09.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.186+0000 7ff0a9600640 1 -- 172.21.15.122:0/575893110 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff09805e660 con 0x7ff0ac15be10 2024-09-17T05:36:09.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.378+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone479614", "target": ["mon-mgr", ""]}) -- 0x7ff0ac155960 con 0x7ff08c074880 2024-09-17T05:36:09.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.394+0000 7ff0a9600640 1 -- 172.21.15.122:0/575893110 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff0a00041c0 con 0x7ff08c074880 2024-09-17T05:36:09.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff08c074880 msgr2=0x7ff08c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff08c074880 0x7ff08c076d40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00af000 tx=0x7ff0a001b120 comp rx=0 tx=0).stop 2024-09-17T05:36:09.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 msgr2=0x7ff0ac158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 0x7ff0ac158340 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff09800c3c0 tx=0x7ff09800c880 comp rx=0 tx=0).stop 2024-09-17T05:36:09.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 shutdown_connections 2024-09-17T05:36:09.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff08c074880 0x7ff08c076d40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0ac15c1f0 0x7ff0ac070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0ac15be10 0x7ff0ac158340 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 --2- 172.21.15.122:0/575893110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0ac0b0b60 0x7ff0ac1595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 >> 172.21.15.122:0/575893110 conn(0x7ff0ac0d1da0 msgr2=0x7ff0ac0b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:09.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 shutdown_connections 2024-09-17T05:36:09.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.398+0000 7ff0b4000640 1 -- 172.21.15.122:0/575893110 wait complete. 2024-09-17T05:36:09.417 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone67336 2024-09-17T05:36:09.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 -- 172.21.15.122:0/1552054584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faedc0ce180 msgr2=0x7faedc0739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 --2- 172.21.15.122:0/1552054584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faedc0ce180 0x7faedc0739e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faed400afd0 tx=0x7faed401d7a0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 -- 172.21.15.122:0/1552054584 shutdown_connections 2024-09-17T05:36:09.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 --2- 172.21.15.122:0/1552054584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faedc0ce180 0x7faedc0739e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 --2- 172.21.15.122:0/1552054584 >> v2:172.21.15.122:3300/0 conn(0x7faedc0cdd60 0x7faedc0734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 -- 172.21.15.122:0/1552054584 >> 172.21.15.122:0/1552054584 conn(0x7faedc0cfa80 msgr2=0x7faedc0cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:09.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 -- 172.21.15.122:0/1552054584 shutdown_connections 2024-09-17T05:36:09.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 -- 172.21.15.122:0/1552054584 wait complete. 2024-09-17T05:36:09.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 Processor -- start 2024-09-17T05:36:09.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.610+0000 7faee2c00640 1 -- start start 2024-09-17T05:36:09.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee2c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faedc0aee40 0x7faedc1f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:09.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee2c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faedc0cdd60 0x7faedc1f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:09.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee2c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faedc0ce180 0x7faedc0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:09.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee2c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faedc0744f0 con 0x7faedc0ce180 2024-09-17T05:36:09.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee2c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faedc0cf510 con 0x7faedc0cdd60 2024-09-17T05:36:09.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee2c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faedc167610 con 0x7faedc0aee40 2024-09-17T05:36:09.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee0e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faedc0cdd60 0x7faedc1f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:09.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee0e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faedc0cdd60 0x7faedc1f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:52264/0 (socket says 172.21.15.122:52264) 2024-09-17T05:36:09.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee0e00640 1 -- 172.21.15.122:0/1017834943 learned_addr learned my addr 172.21.15.122:0/1017834943 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:09.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee1800640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faedc0aee40 0x7faedc1f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:09.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.614+0000 7faee2200640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faedc0ce180 0x7faedc0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:09.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee0e00640 1 -- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faedc0aee40 msgr2=0x7faedc1f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee0e00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faedc0aee40 0x7faedc1f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee0e00640 1 -- 172.21.15.122:0/1017834943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faedc0ce180 msgr2=0x7faedc0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee0e00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faedc0ce180 0x7faedc0c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee0e00640 1 -- 172.21.15.122:0/1017834943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc1f8ff0 con 0x7faedc0cdd60 2024-09-17T05:36:09.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee2200640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faedc0ce180 0x7faedc0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee0e00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faedc0cdd60 0x7faedc1f4e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faec800ea40 tx=0x7faec800ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faed9600640 1 -- 172.21.15.122:0/1017834943 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec800ce40 con 0x7faedc0cdd60 2024-09-17T05:36:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faed9600640 1 -- 172.21.15.122:0/1017834943 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faec8004020 con 0x7faedc0cdd60 2024-09-17T05:36:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faed9600640 1 -- 172.21.15.122:0/1017834943 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8004470 con 0x7faedc0cdd60 2024-09-17T05:36:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc1f92d0 con 0x7faedc0cdd60 2024-09-17T05:36:09.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc1f9890 con 0x7faedc0cdd60 2024-09-17T05:36:09.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.618+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faedc0cd650 con 0x7faedc0cdd60 2024-09-17T05:36:09.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.622+0000 7faed9600640 1 -- 172.21.15.122:0/1017834943 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7faec8003600 con 0x7faedc0cdd60 2024-09-17T05:36:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.622+0000 7faed9600640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faebc074880 0x7faebc076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.622+0000 7faed9600640 1 -- 172.21.15.122:0/1017834943 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7faec8093a80 con 0x7faedc0cdd60 2024-09-17T05:36:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.622+0000 7faee1800640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faebc074880 0x7faebc076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:09.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.626+0000 7faee1800640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faebc074880 0x7faebc076d40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faed0007ed0 tx=0x7faed0007e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:09.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.630+0000 7faed9600640 1 -- 172.21.15.122:0/1017834943 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7faec8060690 con 0x7faedc0cdd60 2024-09-17T05:36:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.918+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone67336", "target": ["mon-mgr", ""]}) -- 0x7faedc0b5a00 con 0x7faebc074880 2024-09-17T05:36:09.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.938+0000 7faed9600640 1 -- 172.21.15.122:0/1017834943 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faedc0b5a00 con 0x7faebc074880 2024-09-17T05:36:09.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faebc074880 msgr2=0x7faebc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faebc074880 0x7faebc076d40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7faed0007ed0 tx=0x7faed0007e60 comp rx=0 tx=0).stop 2024-09-17T05:36:09.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faedc0cdd60 msgr2=0x7faedc1f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:09.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faedc0cdd60 0x7faedc1f4e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7faec800ea40 tx=0x7faec800ef00 comp rx=0 tx=0).stop 2024-09-17T05:36:09.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 shutdown_connections 2024-09-17T05:36:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faebc074880 0x7faebc076d40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faedc0ce180 0x7faedc0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faedc0cdd60 0x7faedc1f4e50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 --2- 172.21.15.122:0/1017834943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faedc0aee40 0x7faedc1f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 >> 172.21.15.122:0/1017834943 conn(0x7faedc0cfa80 msgr2=0x7faedc0aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 shutdown_connections 2024-09-17T05:36:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:09.942+0000 7faee2c00640 1 -- 172.21.15.122:0/1017834943 wait complete. 2024-09-17T05:36:09.958 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone446019 2024-09-17T05:36:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.146+0000 7fbabc000640 1 -- 172.21.15.122:0/2197006262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab40ad3c0 msgr2=0x7fbab4072870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.146+0000 7fbabc000640 1 --2- 172.21.15.122:0/2197006262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab40ad3c0 0x7fbab4072870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbab406f550 tx=0x7fbaac01cda0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.146+0000 7fbabc000640 1 -- 172.21.15.122:0/2197006262 shutdown_connections 2024-09-17T05:36:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.146+0000 7fbabc000640 1 --2- 172.21.15.122:0/2197006262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab40ad3c0 0x7fbab4072870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.146+0000 7fbabc000640 1 --2- 172.21.15.122:0/2197006262 >> v2:172.21.15.122:3300/0 conn(0x7fbab415ade0 0x7fbab415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.146+0000 7fbabc000640 1 -- 172.21.15.122:0/2197006262 >> 172.21.15.122:0/2197006262 conn(0x7fbab40cfbc0 msgr2=0x7fbab40cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.146+0000 7fbabc000640 1 -- 172.21.15.122:0/2197006262 shutdown_connections 2024-09-17T05:36:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.146+0000 7fbabc000640 1 -- 172.21.15.122:0/2197006262 wait complete. 2024-09-17T05:36:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbabc000640 1 Processor -- start 2024-09-17T05:36:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbabc000640 1 -- start start 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbabc000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab409dad0 0x7fbab413a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbabc000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbab40ad3c0 0x7fbab40b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbabc000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab415ade0 0x7fbab40adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbabc000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4167e30 con 0x7fbab409dad0 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbabc000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4167cb0 con 0x7fbab40ad3c0 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbabc000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbab4167fb0 con 0x7fbab415ade0 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbab9600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab409dad0 0x7fbab413a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbab9600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab409dad0 0x7fbab413a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36886/0 (socket says 172.21.15.122:36886) 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbab9600640 1 -- 172.21.15.122:0/325036781 learned_addr learned my addr 172.21.15.122:0/325036781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:10.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbaba000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab415ade0 0x7fbab40adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:10.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbab8c00640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbab40ad3c0 0x7fbab40b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbaba000640 1 -- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbab40ad3c0 msgr2=0x7fbab40b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbaba000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbab40ad3c0 0x7fbab40b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbaba000640 1 -- 172.21.15.122:0/325036781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab409dad0 msgr2=0x7fbab413a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbaba000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab409dad0 0x7fbab413a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbaba000640 1 -- 172.21.15.122:0/325036781 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab409ca20 con 0x7fbab415ade0 2024-09-17T05:36:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.150+0000 7fbaba000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab415ade0 0x7fbab40adb30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0ac000 tx=0x7fbaac009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.154+0000 7fbab1600640 1 -- 172.21.15.122:0/325036781 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0bb070 con 0x7fbab415ade0 2024-09-17T05:36:10.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.154+0000 7fbab1600640 1 -- 172.21.15.122:0/325036781 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaac004d40 con 0x7fbab415ade0 2024-09-17T05:36:10.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.154+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab41549e0 con 0x7fbab415ade0 2024-09-17T05:36:10.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.154+0000 7fbab1600640 1 -- 172.21.15.122:0/325036781 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0b2040 con 0x7fbab415ade0 2024-09-17T05:36:10.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.154+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab40b5770 con 0x7fbab415ade0 2024-09-17T05:36:10.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.154+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab40b55a0 con 0x7fbab415ade0 2024-09-17T05:36:10.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.154+0000 7fbab1600640 1 -- 172.21.15.122:0/325036781 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fbaac0be070 con 0x7fbab415ade0 2024-09-17T05:36:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.158+0000 7fbab1600640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fba940748b0 0x7fba94076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:10.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.158+0000 7fbab1600640 1 -- 172.21.15.122:0/325036781 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fbaac12cbe0 con 0x7fbab415ade0 2024-09-17T05:36:10.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.158+0000 7fbab9600640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fba940748b0 0x7fba94076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:10.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.162+0000 7fbab9600640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fba940748b0 0x7fba94076d70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009b90 tx=0x7fbaa00091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.166+0000 7fbab1600640 1 -- 172.21.15.122:0/325036781 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbaac0169b0 con 0x7fbab415ade0 2024-09-17T05:36:10.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.362+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone446019", "target": ["mon-mgr", ""]}) -- 0x7fbab4072870 con 0x7fba940748b0 2024-09-17T05:36:10.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.378+0000 7fbab1600640 1 -- 172.21.15.122:0/325036781 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbab4072870 con 0x7fba940748b0 2024-09-17T05:36:10.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fba940748b0 msgr2=0x7fba94076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:10.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fba940748b0 0x7fba94076d70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009b90 tx=0x7fbaa00091e0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab415ade0 msgr2=0x7fbab40adb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:10.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab415ade0 0x7fbab40adb30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0ac000 tx=0x7fbaac009330 comp rx=0 tx=0).stop 2024-09-17T05:36:10.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 shutdown_connections 2024-09-17T05:36:10.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fba940748b0 0x7fba94076d70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbab415ade0 0x7fbab40adb30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbab40ad3c0 0x7fbab40b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 --2- 172.21.15.122:0/325036781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbab409dad0 0x7fbab413a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 >> 172.21.15.122:0/325036781 conn(0x7fbab40cfbc0 msgr2=0x7fbab40af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:10.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 shutdown_connections 2024-09-17T05:36:10.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.382+0000 7fbabc000640 1 -- 172.21.15.122:0/325036781 wait complete. 2024-09-17T05:36:10.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone610611 2024-09-17T05:36:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.598+0000 7ffb21a00640 1 -- 172.21.15.122:0/111605033 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 msgr2=0x7ffb1c0746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.598+0000 7ffb21a00640 1 --2- 172.21.15.122:0/111605033 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 0x7ffb1c0746c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009bc0 tx=0x7ffb1401cbc0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.598+0000 7ffb21a00640 1 -- 172.21.15.122:0/111605033 shutdown_connections 2024-09-17T05:36:10.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.598+0000 7ffb21a00640 1 --2- 172.21.15.122:0/111605033 >> v2:172.21.15.122:3300/0 conn(0x7ffb1c070f60 0x7ffb1c0ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.598+0000 7ffb21a00640 1 --2- 172.21.15.122:0/111605033 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 0x7ffb1c0746c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.598+0000 7ffb21a00640 1 -- 172.21.15.122:0/111605033 >> 172.21.15.122:0/111605033 conn(0x7ffb1c0b1b90 msgr2=0x7ffb1c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.598+0000 7ffb21a00640 1 -- 172.21.15.122:0/111605033 shutdown_connections 2024-09-17T05:36:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.598+0000 7ffb21a00640 1 -- 172.21.15.122:0/111605033 wait complete. 2024-09-17T05:36:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb21a00640 1 Processor -- start 2024-09-17T05:36:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb21a00640 1 -- start start 2024-09-17T05:36:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb21a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1c070f60 0x7ffb1c14aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb21a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 0x7ffb1c2079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb21a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb1c152c10 0x7ffb1c1fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb21a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c0b3320 con 0x7ffb1c152c10 2024-09-17T05:36:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb21a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c0b31a0 con 0x7ffb1c070f60 2024-09-17T05:36:10.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb21a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ffb1c0b34a0 con 0x7ffb1c152830 2024-09-17T05:36:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb1c152c10 0x7ffb1c1fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1a000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 0x7ffb1c2079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1a000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 0x7ffb1c2079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:36566/0 (socket says 172.21.15.122:36566) 2024-09-17T05:36:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1a000640 1 -- 172.21.15.122:0/349926018 learned_addr learned my addr 172.21.15.122:0/349926018 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb1c152c10 0x7ffb1c1fdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36896/0 (socket says 172.21.15.122:36896) 2024-09-17T05:36:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1aa00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1c070f60 0x7ffb1c14aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1a000640 1 -- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1c070f60 msgr2=0x7ffb1c14aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1a000640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1c070f60 0x7ffb1c14aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1a000640 1 -- 172.21.15.122:0/349926018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb1c152c10 msgr2=0x7ffb1c1fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1a000640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb1c152c10 0x7ffb1c1fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.602+0000 7ffb1a000640 1 -- 172.21.15.122:0/349926018 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c2085a0 con 0x7ffb1c152830 2024-09-17T05:36:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.606+0000 7ffb1a000640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 0x7ffb1c2079a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c3c0 tx=0x7ffb0c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.606+0000 7ffb12a00640 1 -- 172.21.15.122:0/349926018 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c017070 con 0x7ffb1c152830 2024-09-17T05:36:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.606+0000 7ffb12a00640 1 -- 172.21.15.122:0/349926018 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb0c0053c0 con 0x7ffb1c152830 2024-09-17T05:36:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.606+0000 7ffb12a00640 1 -- 172.21.15.122:0/349926018 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c003860 con 0x7ffb1c152830 2024-09-17T05:36:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.606+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c1557c0 con 0x7ffb1c152830 2024-09-17T05:36:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.606+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c156000 con 0x7ffb1c152830 2024-09-17T05:36:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.606+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c156340 con 0x7ffb1c152830 2024-09-17T05:36:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.610+0000 7ffb12a00640 1 -- 172.21.15.122:0/349926018 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7ffb0c01a070 con 0x7ffb1c152830 2024-09-17T05:36:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.610+0000 7ffb12a00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffafc074880 0x7ffafc076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.610+0000 7ffb12a00640 1 -- 172.21.15.122:0/349926018 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ffb0c092250 con 0x7ffb1c152830 2024-09-17T05:36:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.610+0000 7ffb1aa00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffafc074880 0x7ffafc076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:10.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.614+0000 7ffb1aa00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffafc074880 0x7ffafc076d40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb140af000 tx=0x7ffb1401b400 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:10.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:10.618+0000 7ffb12a00640 1 -- 172.21.15.122:0/349926018 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ffb1c156340 con 0x7ffb1c152830 2024-09-17T05:36:10.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.002+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone610611", "target": ["mon-mgr", ""]}) -- 0x7ffb1c074470 con 0x7ffafc074880 2024-09-17T05:36:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.034+0000 7ffb12a00640 1 -- 172.21.15.122:0/349926018 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffb1c074470 con 0x7ffafc074880 2024-09-17T05:36:11.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffafc074880 msgr2=0x7ffafc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffafc074880 0x7ffafc076d40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ffb140af000 tx=0x7ffb1401b400 comp rx=0 tx=0).stop 2024-09-17T05:36:11.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 msgr2=0x7ffb1c2079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 0x7ffb1c2079a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c3c0 tx=0x7ffb0c00c880 comp rx=0 tx=0).stop 2024-09-17T05:36:11.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 shutdown_connections 2024-09-17T05:36:11.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffafc074880 0x7ffafc076d40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb1c152c10 0x7ffb1c1fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb1c152830 0x7ffb1c2079a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 --2- 172.21.15.122:0/349926018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb1c070f60 0x7ffb1c14aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 >> 172.21.15.122:0/349926018 conn(0x7ffb1c0b1b90 msgr2=0x7ffb1c072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:11.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 shutdown_connections 2024-09-17T05:36:11.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.038+0000 7ffb21a00640 1 -- 172.21.15.122:0/349926018 wait complete. 2024-09-17T05:36:11.071 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone8110 2024-09-17T05:36:11.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.242+0000 7f0f62a00640 1 -- 172.21.15.122:0/4163405099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0734f0 msgr2=0x7f0f5c15a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.242+0000 7f0f62a00640 1 --2- 172.21.15.122:0/4163405099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0734f0 0x7f0f5c15a9d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00ad90 tx=0x7f0f4c01dbf0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.242+0000 7f0f62a00640 1 -- 172.21.15.122:0/4163405099 shutdown_connections 2024-09-17T05:36:11.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.242+0000 7f0f62a00640 1 --2- 172.21.15.122:0/4163405099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f5c0cea60 0x7f0f5c06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.242+0000 7f0f62a00640 1 --2- 172.21.15.122:0/4163405099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0734f0 0x7f0f5c15a9d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.242+0000 7f0f62a00640 1 -- 172.21.15.122:0/4163405099 >> 172.21.15.122:0/4163405099 conn(0x7f0f5c0cfbc0 msgr2=0x7f0f5c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:11.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.242+0000 7f0f62a00640 1 -- 172.21.15.122:0/4163405099 shutdown_connections 2024-09-17T05:36:11.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.242+0000 7f0f62a00640 1 -- 172.21.15.122:0/4163405099 wait complete. 2024-09-17T05:36:11.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 Processor -- start 2024-09-17T05:36:11.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 -- start start 2024-09-17T05:36:11.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f5c0734f0 0x7f0f5c1fb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:11.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0cea60 0x7f0f5c0b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5c0cf440 0x7f0f5c0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c06faf0 con 0x7f0f5c0cf440 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c06f970 con 0x7f0f5c0cea60 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c06fc70 con 0x7f0f5c0734f0 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5c0cf440 0x7f0f5c0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5c0cf440 0x7f0f5c0b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36904/0 (socket says 172.21.15.122:36904) 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 -- 172.21.15.122:0/506629885 learned_addr learned my addr 172.21.15.122:0/506629885 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f5b400640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0cea60 0x7f0f5c0b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f5be00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f5c0734f0 0x7f0f5c1fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 -- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f5c0734f0 msgr2=0x7f0f5c1fb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f5c0734f0 0x7f0f5c1fb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 -- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0cea60 msgr2=0x7f0f5c0b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0cea60 0x7f0f5c0b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 -- 172.21.15.122:0/506629885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c1facb0 con 0x7f0f5c0cf440 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f5b400640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0cea60 0x7f0f5c0b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f5be00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f5c0734f0 0x7f0f5c1fb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f60a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5c0cf440 0x7f0f5c0b4cc0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400e940 tx=0x7f0f5400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f53e00640 1 -- 172.21.15.122:0/506629885 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400ccc0 con 0x7f0f5c0cf440 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f53e00640 1 -- 172.21.15.122:0/506629885 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f54005020 con 0x7f0f5c0cf440 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c0b55f0 con 0x7f0f5c0cf440 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f53e00640 1 -- 172.21.15.122:0/506629885 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400ce60 con 0x7f0f5c0cf440 2024-09-17T05:36:11.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c1faf50 con 0x7f0f5c0cf440 2024-09-17T05:36:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.246+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c069410 con 0x7f0f5c0cf440 2024-09-17T05:36:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.250+0000 7f0f53e00640 1 -- 172.21.15.122:0/506629885 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f0f54005450 con 0x7f0f5c0cf440 2024-09-17T05:36:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.250+0000 7f0f53e00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f3c074880 0x7f0f3c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:11.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.250+0000 7f0f53e00640 1 -- 172.21.15.122:0/506629885 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f0f540939c0 con 0x7f0f5c0cf440 2024-09-17T05:36:11.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.250+0000 7f0f5be00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f3c074880 0x7f0f3c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:11.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.250+0000 7f0f5be00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f3c074880 0x7f0f3c076d40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0b0000 tx=0x7f0f4c01c430 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:11.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.254+0000 7f0f53e00640 1 -- 172.21.15.122:0/506629885 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0f540604d0 con 0x7f0f5c0cf440 2024-09-17T05:36:11.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.466+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone8110", "target": ["mon-mgr", ""]}) -- 0x7f0f5c0d0850 con 0x7f0f3c074880 2024-09-17T05:36:11.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.486+0000 7f0f53e00640 1 -- 172.21.15.122:0/506629885 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0f5c0d0850 con 0x7f0f3c074880 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f3c074880 msgr2=0x7f0f3c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f3c074880 0x7f0f3c076d40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0b0000 tx=0x7f0f4c01c430 comp rx=0 tx=0).stop 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5c0cf440 msgr2=0x7f0f5c0b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5c0cf440 0x7f0f5c0b4cc0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400e940 tx=0x7f0f5400ee00 comp rx=0 tx=0).stop 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 shutdown_connections 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0f3c074880 0x7f0f3c076d40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5c0cf440 0x7f0f5c0b4cc0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f5c0cea60 0x7f0f5c0b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 --2- 172.21.15.122:0/506629885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f5c0734f0 0x7f0f5c1fb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 >> 172.21.15.122:0/506629885 conn(0x7f0f5c0cfbc0 msgr2=0x7f0f5c072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 shutdown_connections 2024-09-17T05:36:11.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.490+0000 7f0f62a00640 1 -- 172.21.15.122:0/506629885 wait complete. 2024-09-17T05:36:11.514 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone67922 2024-09-17T05:36:11.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.766+0000 7fe661600640 1 -- 172.21.15.122:0/1005813958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c159d30 msgr2=0x7fe65c073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.766+0000 7fe661600640 1 --2- 172.21.15.122:0/1005813958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c159d30 0x7fe65c073970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe654009bf0 tx=0x7fe65401cbb0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.766+0000 7fe661600640 1 -- 172.21.15.122:0/1005813958 shutdown_connections 2024-09-17T05:36:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.766+0000 7fe661600640 1 --2- 172.21.15.122:0/1005813958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c159d30 0x7fe65c073970 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.766+0000 7fe661600640 1 -- 172.21.15.122:0/1005813958 >> 172.21.15.122:0/1005813958 conn(0x7fe65c06ec90 msgr2=0x7fe65c06f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.766+0000 7fe661600640 1 -- 172.21.15.122:0/1005813958 shutdown_connections 2024-09-17T05:36:11.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.770+0000 7fe661600640 1 -- 172.21.15.122:0/1005813958 wait complete. 2024-09-17T05:36:11.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.770+0000 7fe661600640 1 Processor -- start 2024-09-17T05:36:11.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe661600640 1 -- start start 2024-09-17T05:36:11.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe661600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65c06aba0 0x7fe65c150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:11.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe661600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c0b2fe0 0x7fe65c1fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:11.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe661600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe65c159d30 0x7fe65c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:11.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe661600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c1693e0 con 0x7fe65c159d30 2024-09-17T05:36:11.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe661600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c169260 con 0x7fe65c06aba0 2024-09-17T05:36:11.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe661600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe65c169560 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe660c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe65c159d30 0x7fe65c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:11.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c0b2fe0 0x7fe65c1fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:11.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c0b2fe0 0x7fe65c1fb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:48568/0 (socket says 172.21.15.122:48568) 2024-09-17T05:36:11.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65b400640 1 -- 172.21.15.122:0/2200794950 learned_addr learned my addr 172.21.15.122:0/2200794950 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:11.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65be00640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65c06aba0 0x7fe65c150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:11.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65b400640 1 -- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65c06aba0 msgr2=0x7fe65c150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65b400640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65c06aba0 0x7fe65c150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65b400640 1 -- 172.21.15.122:0/2200794950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe65c159d30 msgr2=0x7fe65c15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65b400640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe65c159d30 0x7fe65c15e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.774+0000 7fe65b400640 1 -- 172.21.15.122:0/2200794950 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c14f760 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.778+0000 7fe65b400640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c0b2fe0 0x7fe65c1fb1c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe64800c4c0 tx=0x7fe648001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.778+0000 7fe653e00640 1 -- 172.21.15.122:0/2200794950 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648017070 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.778+0000 7fe653e00640 1 -- 172.21.15.122:0/2200794950 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe648003600 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.778+0000 7fe653e00640 1 -- 172.21.15.122:0/2200794950 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648003a50 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.778+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c14fa40 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.778+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c150000 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.778+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65c169a80 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.782+0000 7fe653e00640 1 -- 172.21.15.122:0/2200794950 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fe648003bf0 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.782+0000 7fe653e00640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe63c074880 0x7fe63c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.782+0000 7fe653e00640 1 -- 172.21.15.122:0/2200794950 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7fe648092430 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.782+0000 7fe65be00640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe63c074880 0x7fe63c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.786+0000 7fe653e00640 1 -- 172.21.15.122:0/2200794950 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe64805e970 con 0x7fe65c0b2fe0 2024-09-17T05:36:11.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:11.786+0000 7fe65be00640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe63c074880 0x7fe63c076d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe64c001910 tx=0x7fe64c001420 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:12.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.038+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone67922", "target": ["mon-mgr", ""]}) -- 0x7fe65c0738e0 con 0x7fe63c074880 2024-09-17T05:36:12.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.058+0000 7fe653e00640 1 -- 172.21.15.122:0/2200794950 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe65c0738e0 con 0x7fe63c074880 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe63c074880 msgr2=0x7fe63c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe63c074880 0x7fe63c076d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe64c001910 tx=0x7fe64c001420 comp rx=0 tx=0).stop 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c0b2fe0 msgr2=0x7fe65c1fb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c0b2fe0 0x7fe65c1fb1c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe64800c4c0 tx=0x7fe648001e80 comp rx=0 tx=0).stop 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 shutdown_connections 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe63c074880 0x7fe63c076d40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe65c159d30 0x7fe65c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe65c0b2fe0 0x7fe65c1fb1c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 --2- 172.21.15.122:0/2200794950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe65c06aba0 0x7fe65c150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 >> 172.21.15.122:0/2200794950 conn(0x7fe65c06ec90 msgr2=0x7fe65c14d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:12.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 shutdown_connections 2024-09-17T05:36:12.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.062+0000 7fe661600640 1 -- 172.21.15.122:0/2200794950 wait complete. 2024-09-17T05:36:12.079 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone281413 2024-09-17T05:36:12.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- 172.21.15.122:0/1720812586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f339c0ce890 msgr2=0x7f339c06eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 --2- 172.21.15.122:0/1720812586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f339c0ce890 0x7f339c06eb40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3394009840 tx=0x7f339401c820 comp rx=0 tx=0).stop 2024-09-17T05:36:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- 172.21.15.122:0/1720812586 shutdown_connections 2024-09-17T05:36:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 --2- 172.21.15.122:0/1720812586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f339c0ce890 0x7f339c06eb40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 --2- 172.21.15.122:0/1720812586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f339c15a9d0 0x7f339c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- 172.21.15.122:0/1720812586 >> 172.21.15.122:0/1720812586 conn(0x7f339c0cfbc0 msgr2=0x7f339c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- 172.21.15.122:0/1720812586 shutdown_connections 2024-09-17T05:36:12.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- 172.21.15.122:0/1720812586 wait complete. 2024-09-17T05:36:12.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 Processor -- start 2024-09-17T05:36:12.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- start start 2024-09-17T05:36:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f339c0734f0 0x7f339c1648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f339c0ce890 0x7f339c09d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f339c15a9d0 0x7f339c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1674e0 con 0x7f339c0ce890 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f339c167360 con 0x7f339c15a9d0 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a4400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f339c167660 con 0x7f339c0734f0 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a1000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f339c0ce890 0x7f339c09d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a1000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f339c0ce890 0x7f339c09d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39860/0 (socket says 172.21.15.122:39860) 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.234+0000 7f33a1000640 1 -- 172.21.15.122:0/461671103 learned_addr learned my addr 172.21.15.122:0/461671103 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a2400640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f339c15a9d0 0x7f339c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a1a00640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f339c0734f0 0x7f339c1648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a1000640 1 -- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f339c0734f0 msgr2=0x7f339c1648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a1000640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f339c0734f0 0x7f339c1648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a1000640 1 -- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f339c15a9d0 msgr2=0x7f339c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a1000640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f339c15a9d0 0x7f339c159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a1000640 1 -- 172.21.15.122:0/461671103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c164e30 con 0x7f339c0ce890 2024-09-17T05:36:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a1000640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f339c0ce890 0x7f339c09d420 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f338800ea90 tx=0x7f338800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f3399600640 1 -- 172.21.15.122:0/461671103 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338801b070 con 0x7f339c0ce890 2024-09-17T05:36:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f3399600640 1 -- 172.21.15.122:0/461671103 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f338800ccb0 con 0x7f339c0ce890 2024-09-17T05:36:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f3399600640 1 -- 172.21.15.122:0/461671103 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388016670 con 0x7f339c0ce890 2024-09-17T05:36:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c0ad9a0 con 0x7f339c0ce890 2024-09-17T05:36:12.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c0adf60 con 0x7f339c0ce890 2024-09-17T05:36:12.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c069410 con 0x7f339c0ce890 2024-09-17T05:36:12.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.238+0000 7f3399600640 1 -- 172.21.15.122:0/461671103 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f338801e070 con 0x7f339c0ce890 2024-09-17T05:36:12.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.242+0000 7f3399600640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f337c074880 0x7f337c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:12.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.242+0000 7f3399600640 1 -- 172.21.15.122:0/461671103 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f33880930f0 con 0x7f339c0ce890 2024-09-17T05:36:12.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.242+0000 7f33a1a00640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f337c074880 0x7f337c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:12.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.242+0000 7f33a1a00640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f337c074880 0x7f337c076d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f339000f290 tx=0x7f3390001aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:12.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.246+0000 7f3399600640 1 -- 172.21.15.122:0/461671103 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f339c164e30 con 0x7f339c0ce890 2024-09-17T05:36:12.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.438+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone281413", "target": ["mon-mgr", ""]}) -- 0x7f339c0ad750 con 0x7f337c074880 2024-09-17T05:36:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.458+0000 7f3399600640 1 -- 172.21.15.122:0/461671103 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f339c0ad750 con 0x7f337c074880 2024-09-17T05:36:12.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.458+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f337c074880 msgr2=0x7f337c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.458+0000 7f33a4400640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f337c074880 0x7f337c076d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f339000f290 tx=0x7f3390001aa0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.458+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f339c0ce890 msgr2=0x7f339c09d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.458+0000 7f33a4400640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f339c0ce890 0x7f339c09d420 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f338800ea90 tx=0x7f338800ef50 comp rx=0 tx=0).stop 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.458+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 shutdown_connections 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.458+0000 7f33a4400640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f337c074880 0x7f337c076d40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.462+0000 7f33a4400640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f339c15a9d0 0x7f339c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.462+0000 7f33a4400640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f339c0ce890 0x7f339c09d420 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.462+0000 7f33a4400640 1 --2- 172.21.15.122:0/461671103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f339c0734f0 0x7f339c1648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.462+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 >> 172.21.15.122:0/461671103 conn(0x7f339c0cfbc0 msgr2=0x7f339c15bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.462+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 shutdown_connections 2024-09-17T05:36:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.462+0000 7f33a4400640 1 -- 172.21.15.122:0/461671103 wait complete. 2024-09-17T05:36:12.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone21727 2024-09-17T05:36:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.722+0000 7f0d8d200640 1 -- 172.21.15.122:0/3964556449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880aee40 msgr2=0x7f0d880cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.722+0000 7f0d8d200640 1 --2- 172.21.15.122:0/3964556449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880aee40 0x7f0d880cd650 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f0d740074e0 tx=0x7f0d7401d500 comp rx=0 tx=0).stop 2024-09-17T05:36:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.722+0000 7f0d8d200640 1 -- 172.21.15.122:0/3964556449 shutdown_connections 2024-09-17T05:36:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.722+0000 7f0d8d200640 1 --2- 172.21.15.122:0/3964556449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d880cdb90 0x7f0d880cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.722+0000 7f0d8d200640 1 --2- 172.21.15.122:0/3964556449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880aee40 0x7f0d880cd650 secure :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f0d740074e0 tx=0x7f0d7401d500 comp rx=0 tx=0).stop 2024-09-17T05:36:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.726+0000 7f0d8d200640 1 -- 172.21.15.122:0/3964556449 >> 172.21.15.122:0/3964556449 conn(0x7f0d880cfa80 msgr2=0x7f0d880cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.726+0000 7f0d8d200640 1 -- 172.21.15.122:0/3964556449 shutdown_connections 2024-09-17T05:36:12.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.726+0000 7f0d8d200640 1 -- 172.21.15.122:0/3964556449 wait complete. 2024-09-17T05:36:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d8d200640 1 Processor -- start 2024-09-17T05:36:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d8d200640 1 -- start start 2024-09-17T05:36:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d8d200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d880cdb90 0x7f0d881540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d8d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d8815da60 0x7f0d88156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d8d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d880c3590 0x7f0d8815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d8d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88167610 con 0x7f0d8815da60 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d8d200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d880cf510 con 0x7f0d880c3590 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d8d200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0d88167790 con 0x7f0d880cdb90 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d880c3590 0x7f0d8815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d880c3590 0x7f0d8815c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:37254/0 (socket says 172.21.15.122:37254) 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 -- 172.21.15.122:0/1541473571 learned_addr learned my addr 172.21.15.122:0/1541473571 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 -- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d880cdb90 msgr2=0x7f0d881540e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:36:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d880cdb90 0x7f0d881540e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 -- 172.21.15.122:0/1541473571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d8815da60 msgr2=0x7f0d88156530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:36:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d8815da60 0x7f0d88156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 -- 172.21.15.122:0/1541473571 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d880c3a50 con 0x7f0d880c3590 2024-09-17T05:36:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.730+0000 7f0d87e00640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d880c3590 0x7f0d8815c4d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c3c0 tx=0x7f0d7800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.734+0000 7f0d7f400640 1 -- 172.21.15.122:0/1541473571 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78017070 con 0x7f0d880c3590 2024-09-17T05:36:12.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.734+0000 7f0d7f400640 1 -- 172.21.15.122:0/1541473571 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d780053c0 con 0x7f0d880c3590 2024-09-17T05:36:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.734+0000 7f0d7f400640 1 -- 172.21.15.122:0/1541473571 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78003860 con 0x7f0d880c3590 2024-09-17T05:36:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.734+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d880c3d30 con 0x7f0d880c3590 2024-09-17T05:36:12.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.734+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d880c42f0 con 0x7f0d880c3590 2024-09-17T05:36:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.734+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d880cd650 con 0x7f0d880c3590 2024-09-17T05:36:12.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.738+0000 7f0d7f400640 1 -- 172.21.15.122:0/1541473571 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f0d7801a070 con 0x7f0d880c3590 2024-09-17T05:36:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.738+0000 7f0d7f400640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d680748d0 0x7f0d68076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.738+0000 7f0d87400640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d680748d0 0x7f0d68076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.738+0000 7f0d7f400640 1 -- 172.21.15.122:0/1541473571 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f0d7805ab40 con 0x7f0d880c3590 2024-09-17T05:36:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.742+0000 7f0d87400640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d680748d0 0x7f0d68076d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d740af000 tx=0x7f0d7401bd40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:12.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:12.742+0000 7f0d7f400640 1 -- 172.21.15.122:0/1541473571 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0d78021080 con 0x7f0d880c3590 2024-09-17T05:36:13.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.102+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone21727", "target": ["mon-mgr", ""]}) -- 0x7f0d880b5a00 con 0x7f0d680748d0 2024-09-17T05:36:13.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.118+0000 7f0d7f400640 1 -- 172.21.15.122:0/1541473571 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d880b5a00 con 0x7f0d680748d0 2024-09-17T05:36:13.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d680748d0 msgr2=0x7f0d68076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d680748d0 0x7f0d68076d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d740af000 tx=0x7f0d7401bd40 comp rx=0 tx=0).stop 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d880c3590 msgr2=0x7f0d8815c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d880c3590 0x7f0d8815c4d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c3c0 tx=0x7f0d7800c880 comp rx=0 tx=0).stop 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 shutdown_connections 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0d680748d0 0x7f0d68076d90 secure :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d740af000 tx=0x7f0d7401bd40 comp rx=0 tx=0).stop 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d880c3590 0x7f0d8815c4d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d8815da60 0x7f0d88156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 --2- 172.21.15.122:0/1541473571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d880cdb90 0x7f0d881540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 >> 172.21.15.122:0/1541473571 conn(0x7f0d880cfa80 msgr2=0x7f0d880af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 shutdown_connections 2024-09-17T05:36:13.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.122+0000 7f0d8d200640 1 -- 172.21.15.122:0/1541473571 wait complete. 2024-09-17T05:36:13.141 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone82612 2024-09-17T05:36:13.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 -- 172.21.15.122:0/1041286324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1488162720 msgr2=0x7f1488161b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 --2- 172.21.15.122:0/1041286324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1488162720 0x7f1488161b80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f148808e2f0 tx=0x7f148001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 -- 172.21.15.122:0/1041286324 shutdown_connections 2024-09-17T05:36:13.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 --2- 172.21.15.122:0/1041286324 >> v2:172.21.15.173:3300/0 conn(0x7f148815e220 0x7f14880d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 --2- 172.21.15.122:0/1041286324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1488162720 0x7f1488161b80 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 --2- 172.21.15.122:0/1041286324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148815de40 0x7f14880cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 -- 172.21.15.122:0/1041286324 >> 172.21.15.122:0/1041286324 conn(0x7f14880d3f90 msgr2=0x7f14880d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:13.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 -- 172.21.15.122:0/1041286324 shutdown_connections 2024-09-17T05:36:13.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.290+0000 7f148f400640 1 -- 172.21.15.122:0/1041286324 wait complete. 2024-09-17T05:36:13.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148f400640 1 Processor -- start 2024-09-17T05:36:13.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148f400640 1 -- start start 2024-09-17T05:36:13.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148f400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148815de40 0x7f1488157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:13.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148815e220 0x7f14880b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1488162720 0x7f14880b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148f400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14880d0b10 con 0x7f148815e220 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148f400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14880d0990 con 0x7f1488162720 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148f400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f14880d0c90 con 0x7f148815de40 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148815e220 0x7f14880b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148815e220 0x7f14880b87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39896/0 (socket says 172.21.15.122:39896) 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 -- 172.21.15.122:0/1165769791 learned_addr learned my addr 172.21.15.122:0/1165769791 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148ca00640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148815de40 0x7f1488157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f148d400640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1488162720 0x7f14880b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 -- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148815de40 msgr2=0x7f1488157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148815de40 0x7f1488157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 -- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1488162720 msgr2=0x7f14880b8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1488162720 0x7f14880b8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 -- 172.21.15.122:0/1165769791 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1488074720 con 0x7f148815e220 2024-09-17T05:36:13.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.294+0000 7f147fe00640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148815e220 0x7f14880b87c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f147000e9f0 tx=0x7f147000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:13.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.298+0000 7f1485600640 1 -- 172.21.15.122:0/1165769791 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147000ccc0 con 0x7f148815e220 2024-09-17T05:36:13.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.298+0000 7f1485600640 1 -- 172.21.15.122:0/1165769791 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1470004020 con 0x7f148815e220 2024-09-17T05:36:13.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.298+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14881fb5e0 con 0x7f148815e220 2024-09-17T05:36:13.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.298+0000 7f1485600640 1 -- 172.21.15.122:0/1165769791 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147000ce60 con 0x7f148815e220 2024-09-17T05:36:13.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.298+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881fbba0 con 0x7f148815e220 2024-09-17T05:36:13.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.298+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14880cf6f0 con 0x7f148815e220 2024-09-17T05:36:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.298+0000 7f1485600640 1 -- 172.21.15.122:0/1165769791 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f1470003600 con 0x7f148815e220 2024-09-17T05:36:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.298+0000 7f1485600640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1464074880 0x7f1464076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.302+0000 7f1485600640 1 -- 172.21.15.122:0/1165769791 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f1470093b10 con 0x7f148815e220 2024-09-17T05:36:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.302+0000 7f148ca00640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1464074880 0x7f1464076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.302+0000 7f148ca00640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1464074880 0x7f1464076d40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1478005e10 tx=0x7f1478005c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:13.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.302+0000 7f1485600640 1 -- 172.21.15.122:0/1165769791 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1470060620 con 0x7f148815e220 2024-09-17T05:36:13.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.494+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone82612", "target": ["mon-mgr", ""]}) -- 0x7f14881fb390 con 0x7f1464074880 2024-09-17T05:36:13.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.522+0000 7f1485600640 1 -- 172.21.15.122:0/1165769791 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14881fb390 con 0x7f1464074880 2024-09-17T05:36:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.526+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1464074880 msgr2=0x7f1464076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.526+0000 7f148f400640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1464074880 0x7f1464076d40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1478005e10 tx=0x7f1478005c60 comp rx=0 tx=0).stop 2024-09-17T05:36:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.526+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148815e220 msgr2=0x7f14880b87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.526+0000 7f148f400640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148815e220 0x7f14880b87c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f147000e9f0 tx=0x7f147000eeb0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.530+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 shutdown_connections 2024-09-17T05:36:13.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.530+0000 7f148f400640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1464074880 0x7f1464076d40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.530+0000 7f148f400640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1488162720 0x7f14880b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.530+0000 7f148f400640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148815e220 0x7f14880b87c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.530+0000 7f148f400640 1 --2- 172.21.15.122:0/1165769791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148815de40 0x7f1488157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.530+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 >> 172.21.15.122:0/1165769791 conn(0x7f14880d3f90 msgr2=0x7f14880b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.530+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 shutdown_connections 2024-09-17T05:36:13.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.530+0000 7f148f400640 1 -- 172.21.15.122:0/1165769791 wait complete. 2024-09-17T05:36:13.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone84053 2024-09-17T05:36:13.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.822+0000 7f5ef2800640 1 -- 172.21.15.122:0/1688480818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5eec0b5170 msgr2=0x7f5eec0aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.822+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1688480818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5eec0b5170 0x7f5eec0aedf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009840 tx=0x7f5ed801c790 comp rx=0 tx=0).stop 2024-09-17T05:36:13.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.822+0000 7f5ef2800640 1 -- 172.21.15.122:0/1688480818 shutdown_connections 2024-09-17T05:36:13.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.822+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1688480818 >> v2:172.21.15.173:3300/0 conn(0x7f5eec0af330 0x7f5eec0cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.822+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1688480818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5eec0b5170 0x7f5eec0aedf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.822+0000 7f5ef2800640 1 -- 172.21.15.122:0/1688480818 >> 172.21.15.122:0/1688480818 conn(0x7f5eec0cfbc0 msgr2=0x7f5eec0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:13.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.822+0000 7f5ef2800640 1 -- 172.21.15.122:0/1688480818 shutdown_connections 2024-09-17T05:36:13.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.822+0000 7f5ef2800640 1 -- 172.21.15.122:0/1688480818 wait complete. 2024-09-17T05:36:13.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5ef2800640 1 Processor -- start 2024-09-17T05:36:13.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5ef2800640 1 -- start start 2024-09-17T05:36:13.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5ef2800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5eec0af330 0x7f5eec153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:13.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5ef2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5eec0b5170 0x7f5eec153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:13.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5ef2800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eec15bf60 0x7f5eec1569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:13.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5ef2800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec169240 con 0x7f5eec0b5170 2024-09-17T05:36:13.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5ef2800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec1690c0 con 0x7f5eec15bf60 2024-09-17T05:36:13.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5ef2800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5eec1693c0 con 0x7f5eec0af330 2024-09-17T05:36:13.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eeaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5eec0b5170 0x7f5eec153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:13.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eeaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5eec0b5170 0x7f5eec153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39908/0 (socket says 172.21.15.122:39908) 2024-09-17T05:36:13.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eeaa00640 1 -- 172.21.15.122:0/1609362771 learned_addr learned my addr 172.21.15.122:0/1609362771 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:13.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eebe00640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eec15bf60 0x7f5eec1569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:13.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eeaa00640 1 -- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5eec0af330 msgr2=0x7f5eec153c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:36:13.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eeaa00640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5eec0af330 0x7f5eec153c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eeaa00640 1 -- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eec15bf60 msgr2=0x7f5eec1569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:13.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eeaa00640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eec15bf60 0x7f5eec1569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:13.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.826+0000 7f5eeaa00640 1 -- 172.21.15.122:0/1609362771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec164e80 con 0x7f5eec0b5170 2024-09-17T05:36:13.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.830+0000 7f5eeaa00640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5eec0b5170 0x7f5eec153040 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80193a0 tx=0x7f5ed8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:13.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.830+0000 7f5ee3400640 1 -- 172.21.15.122:0/1609362771 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed80b2040 con 0x7f5eec0b5170 2024-09-17T05:36:13.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.830+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec0ce880 con 0x7f5eec0b5170 2024-09-17T05:36:13.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.830+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec15b4b0 con 0x7f5eec0b5170 2024-09-17T05:36:13.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.830+0000 7f5ee3400640 1 -- 172.21.15.122:0/1609362771 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ed80053e0 con 0x7f5eec0b5170 2024-09-17T05:36:13.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.830+0000 7f5ee3400640 1 -- 172.21.15.122:0/1609362771 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed80b2040 con 0x7f5eec0b5170 2024-09-17T05:36:13.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.830+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec40052a0 con 0x7f5eec0b5170 2024-09-17T05:36:13.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.834+0000 7f5ee3400640 1 -- 172.21.15.122:0/1609362771 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f5ed80bd070 con 0x7f5eec0b5170 2024-09-17T05:36:13.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.834+0000 7f5ee3400640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ecc074a80 0x7f5ecc076f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:13.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.834+0000 7f5ee3400640 1 -- 172.21.15.122:0/1609362771 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f5ed812c1d0 con 0x7f5eec0b5170 2024-09-17T05:36:13.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.834+0000 7f5eeb400640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ecc074a80 0x7f5ecc076f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:13.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.834+0000 7f5eeb400640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ecc074a80 0x7f5ecc076f40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5edc000f70 tx=0x7f5edc0079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:13.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:13.842+0000 7f5ee3400640 1 -- 172.21.15.122:0/1609362771 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5ed80f2120 con 0x7f5eec0b5170 2024-09-17T05:36:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.182+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone84053", "target": ["mon-mgr", ""]}) -- 0x7f5ec4000bd0 con 0x7f5ecc074a80 2024-09-17T05:36:14.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.198+0000 7f5ee3400640 1 -- 172.21.15.122:0/1609362771 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ec4000bd0 con 0x7f5ecc074a80 2024-09-17T05:36:14.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ecc074a80 msgr2=0x7f5ecc076f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ecc074a80 0x7f5ecc076f40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5edc000f70 tx=0x7f5edc0079c0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5eec0b5170 msgr2=0x7f5eec153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5eec0b5170 0x7f5eec153040 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80193a0 tx=0x7f5ed8005040 comp rx=0 tx=0).stop 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 shutdown_connections 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ecc074a80 0x7f5ecc076f40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5eec15bf60 0x7f5eec1569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5eec0b5170 0x7f5eec153040 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 --2- 172.21.15.122:0/1609362771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5eec0af330 0x7f5eec153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 >> 172.21.15.122:0/1609362771 conn(0x7f5eec0cfbc0 msgr2=0x7f5eec158430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 shutdown_connections 2024-09-17T05:36:14.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.202+0000 7f5ef2800640 1 -- 172.21.15.122:0/1609362771 wait complete. 2024-09-17T05:36:14.218 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone44559 2024-09-17T05:36:14.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.370+0000 7fb599600640 1 -- 172.21.15.122:0/758279260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb59415e740 msgr2=0x7fb59415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.370+0000 7fb599600640 1 --2- 172.21.15.122:0/758279260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb59415e740 0x7fb59415ebf0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009bc0 tx=0x7fb58c01cb10 comp rx=0 tx=0).stop 2024-09-17T05:36:14.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.370+0000 7fb599600640 1 -- 172.21.15.122:0/758279260 shutdown_connections 2024-09-17T05:36:14.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.370+0000 7fb599600640 1 --2- 172.21.15.122:0/758279260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb59415e740 0x7fb59415ebf0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.370+0000 7fb599600640 1 --2- 172.21.15.122:0/758279260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb59415e360 0x7fb5940adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.370+0000 7fb599600640 1 -- 172.21.15.122:0/758279260 >> 172.21.15.122:0/758279260 conn(0x7fb5940cfbc0 msgr2=0x7fb5940cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:14.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.370+0000 7fb599600640 1 -- 172.21.15.122:0/758279260 shutdown_connections 2024-09-17T05:36:14.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.370+0000 7fb599600640 1 -- 172.21.15.122:0/758279260 wait complete. 2024-09-17T05:36:14.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 Processor -- start 2024-09-17T05:36:14.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 -- start start 2024-09-17T05:36:14.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5940aef60 0x7fb59414aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:14.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb59415e360 0x7fb594149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:14.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb59415e740 0x7fb5941f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5941680b0 con 0x7fb5940aef60 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb594167f30 con 0x7fb59415e360 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb594168230 con 0x7fb59415e740 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5940aef60 0x7fb59414aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5940aef60 0x7fb59414aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39932/0 (socket says 172.21.15.122:39932) 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 -- 172.21.15.122:0/69749072 learned_addr learned my addr 172.21.15.122:0/69749072 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb58a000640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb59415e360 0x7fb594149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb593400640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb59415e740 0x7fb5941f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 -- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb59415e740 msgr2=0x7fb5941f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb59415e740 0x7fb5941f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 -- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb59415e360 msgr2=0x7fb594149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb59415e360 0x7fb594149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 -- 172.21.15.122:0/69749072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5941fb8f0 con 0x7fb5940aef60 2024-09-17T05:36:14.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb592a00640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5940aef60 0x7fb59414aac0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d980 tx=0x7fb58400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:14.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb58b400640 1 -- 172.21.15.122:0/69749072 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584017070 con 0x7fb5940aef60 2024-09-17T05:36:14.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb58b400640 1 -- 172.21.15.122:0/69749072 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb584009d20 con 0x7fb5940aef60 2024-09-17T05:36:14.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb58b400640 1 -- 172.21.15.122:0/69749072 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584004280 con 0x7fb5940aef60 2024-09-17T05:36:14.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5941fbb90 con 0x7fb5940aef60 2024-09-17T05:36:14.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5941547a0 con 0x7fb5940aef60 2024-09-17T05:36:14.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.374+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb594069410 con 0x7fb5940aef60 2024-09-17T05:36:14.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.378+0000 7fb58b400640 1 -- 172.21.15.122:0/69749072 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fb584003600 con 0x7fb5940aef60 2024-09-17T05:36:14.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.378+0000 7fb58b400640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb570074880 0x7fb570076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:14.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.378+0000 7fb58b400640 1 -- 172.21.15.122:0/69749072 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7fb5840933a0 con 0x7fb5940aef60 2024-09-17T05:36:14.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.378+0000 7fb58a000640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb570074880 0x7fb570076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.378+0000 7fb58a000640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb570074880 0x7fb570076d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb5940b5cf0 tx=0x7fb57c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.382+0000 7fb58b400640 1 -- 172.21.15.122:0/69749072 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb58405f860 con 0x7fb5940aef60 2024-09-17T05:36:14.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.654+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone44559", "target": ["mon-mgr", ""]}) -- 0x7fb59415ebf0 con 0x7fb570074880 2024-09-17T05:36:14.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.694+0000 7fb58b400640 1 -- 172.21.15.122:0/69749072 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb59415ebf0 con 0x7fb570074880 2024-09-17T05:36:14.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb570074880 msgr2=0x7fb570076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb570074880 0x7fb570076d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb5940b5cf0 tx=0x7fb57c009040 comp rx=0 tx=0).stop 2024-09-17T05:36:14.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5940aef60 msgr2=0x7fb59414aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5940aef60 0x7fb59414aac0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d980 tx=0x7fb58400de40 comp rx=0 tx=0).stop 2024-09-17T05:36:14.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 shutdown_connections 2024-09-17T05:36:14.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb570074880 0x7fb570076d40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb59415e740 0x7fb5941f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb59415e360 0x7fb594149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 --2- 172.21.15.122:0/69749072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5940aef60 0x7fb59414aac0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 >> 172.21.15.122:0/69749072 conn(0x7fb5940cfbc0 msgr2=0x7fb5940ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:14.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 shutdown_connections 2024-09-17T05:36:14.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.698+0000 7fb599600640 1 -- 172.21.15.122:0/69749072 wait complete. 2024-09-17T05:36:14.731 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92681 2024-09-17T05:36:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.982+0000 7f2982800640 1 -- 172.21.15.122:0/1037851006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 msgr2=0x7f297c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.982+0000 7f2982800640 1 --2- 172.21.15.122:0/1037851006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 0x7f297c06fcc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2964007240 tx=0x7f296401c3f0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.982+0000 7f2982800640 1 -- 172.21.15.122:0/1037851006 shutdown_connections 2024-09-17T05:36:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.982+0000 7f2982800640 1 --2- 172.21.15.122:0/1037851006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 0x7f297c06fcc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.982+0000 7f2982800640 1 -- 172.21.15.122:0/1037851006 >> 172.21.15.122:0/1037851006 conn(0x7f297c0cfbc0 msgr2=0x7f297c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:14.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.982+0000 7f2982800640 1 -- 172.21.15.122:0/1037851006 shutdown_connections 2024-09-17T05:36:14.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.982+0000 7f2982800640 1 -- 172.21.15.122:0/1037851006 wait complete. 2024-09-17T05:36:14.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f2982800640 1 Processor -- start 2024-09-17T05:36:14.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f2982800640 1 -- start start 2024-09-17T05:36:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f2982800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 0x7f297c0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f2982800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cf2b0 0x7f297c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f2982800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f297c15b000 0x7f297c0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f2982800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1688b0 con 0x7f297c0cf2b0 2024-09-17T05:36:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f2982800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f297c168730 con 0x7f297c15b000 2024-09-17T05:36:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f2982800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f297c168a30 con 0x7f297c072f80 2024-09-17T05:36:14.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 0x7f297c0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:14.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 0x7f297c0b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:48682/0 (socket says 172.21.15.122:48682) 2024-09-17T05:36:14.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297b400640 1 -- 172.21.15.122:0/2578257814 learned_addr learned my addr 172.21.15.122:0/2578257814 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:14.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297aa00640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cf2b0 0x7f297c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297b400640 1 -- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f297c15b000 msgr2=0x7f297c0b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297be00640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f297c15b000 0x7f297c0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:14.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297b400640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f297c15b000 0x7f297c0b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297b400640 1 -- 172.21.15.122:0/2578257814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cf2b0 msgr2=0x7f297c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.986+0000 7f297b400640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cf2b0 0x7f297c152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.990+0000 7f297b400640 1 -- 172.21.15.122:0/2578257814 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c0b3b10 con 0x7f297c072f80 2024-09-17T05:36:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.990+0000 7f297be00640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f297c15b000 0x7f297c0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:14.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.990+0000 7f297b400640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 0x7f297c0b0910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f296c00c4c0 tx=0x7f296c00c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.990+0000 7f2973400640 1 -- 172.21.15.122:0/2578257814 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c016020 con 0x7f297c072f80 2024-09-17T05:36:14.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.990+0000 7f2973400640 1 -- 172.21.15.122:0/2578257814 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f296c012940 con 0x7f297c072f80 2024-09-17T05:36:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.990+0000 7f2973400640 1 -- 172.21.15.122:0/2578257814 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c012d90 con 0x7f297c072f80 2024-09-17T05:36:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.990+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c0b3db0 con 0x7f297c072f80 2024-09-17T05:36:14.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.990+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c156130 con 0x7f297c072f80 2024-09-17T05:36:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.994+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c069410 con 0x7f297c072f80 2024-09-17T05:36:14.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.994+0000 7f2973400640 1 -- 172.21.15.122:0/2578257814 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f296c019020 con 0x7f297c072f80 2024-09-17T05:36:14.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.998+0000 7f2973400640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2958074880 0x7f2958076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:14.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.998+0000 7f2973400640 1 -- 172.21.15.122:0/2578257814 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f296c093260 con 0x7f297c072f80 2024-09-17T05:36:14.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:14.998+0000 7f297aa00640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2958074880 0x7f2958076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:14.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.002+0000 7f297aa00640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2958074880 0x7f2958076d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f29640ae000 tx=0x7f296401ac30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:14.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.002+0000 7f2973400640 1 -- 172.21.15.122:0/2578257814 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f296c020020 con 0x7f297c072f80 2024-09-17T05:36:15.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.258+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92681", "target": ["mon-mgr", ""]}) -- 0x7f297c0d07b0 con 0x7f2958074880 2024-09-17T05:36:15.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.274+0000 7f2973400640 1 -- 172.21.15.122:0/2578257814 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f297c0d07b0 con 0x7f2958074880 2024-09-17T05:36:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2958074880 msgr2=0x7f2958076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2958074880 0x7f2958076d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f29640ae000 tx=0x7f296401ac30 comp rx=0 tx=0).stop 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 msgr2=0x7f297c0b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 0x7f297c0b0910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f296c00c4c0 tx=0x7f296c00c980 comp rx=0 tx=0).stop 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 shutdown_connections 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2958074880 0x7f2958076d40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f297c15b000 0x7f297c0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f297c0cf2b0 0x7f297c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 --2- 172.21.15.122:0/2578257814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f297c072f80 0x7f297c0b0910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 >> 172.21.15.122:0/2578257814 conn(0x7f297c0cfbc0 msgr2=0x7f297c15b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 shutdown_connections 2024-09-17T05:36:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.278+0000 7f2982800640 1 -- 172.21.15.122:0/2578257814 wait complete. 2024-09-17T05:36:15.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone25105 2024-09-17T05:36:15.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.446+0000 7f0c94800640 1 -- 172.21.15.122:0/1700681880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c0af330 msgr2=0x7f0c8c0cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:15.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.446+0000 7f0c94800640 1 --2- 172.21.15.122:0/1700681880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c0af330 0x7f0c8c0cdac0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009bc0 tx=0x7f0c8801cad0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.446+0000 7f0c94800640 1 -- 172.21.15.122:0/1700681880 shutdown_connections 2024-09-17T05:36:15.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.446+0000 7f0c94800640 1 --2- 172.21.15.122:0/1700681880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c0af330 0x7f0c8c0cdac0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.446+0000 7f0c94800640 1 --2- 172.21.15.122:0/1700681880 >> v2:172.21.15.173:3300/0 conn(0x7f0c8c0b5170 0x7f0c8c0aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.446+0000 7f0c94800640 1 -- 172.21.15.122:0/1700681880 >> 172.21.15.122:0/1700681880 conn(0x7f0c8c0cfbc0 msgr2=0x7f0c8c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:15.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.446+0000 7f0c94800640 1 -- 172.21.15.122:0/1700681880 shutdown_connections 2024-09-17T05:36:15.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 -- 172.21.15.122:0/1700681880 wait complete. 2024-09-17T05:36:15.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 Processor -- start 2024-09-17T05:36:15.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 -- start start 2024-09-17T05:36:15.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c8c0af330 0x7f0c8c147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c8c0b5170 0x7f0c8c1ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c15bf60 0x7f0c8c147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c169240 con 0x7f0c8c15bf60 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1690c0 con 0x7f0c8c0af330 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c1693c0 con 0x7f0c8c0b5170 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c15bf60 0x7f0c8c147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c15bf60 0x7f0c8c147e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39964/0 (socket says 172.21.15.122:39964) 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 -- 172.21.15.122:0/3338424408 learned_addr learned my addr 172.21.15.122:0/3338424408 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c91400640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c8c0b5170 0x7f0c8c1ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c91e00640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c8c0af330 0x7f0c8c147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 -- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c8c0b5170 msgr2=0x7f0c8c1ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:15.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c8c0b5170 0x7f0c8c1ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 -- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c8c0af330 msgr2=0x7f0c8c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:15.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c8c0af330 0x7f0c8c147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 -- 172.21.15.122:0/3338424408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c159560 con 0x7f0c8c15bf60 2024-09-17T05:36:15.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c92800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c15bf60 0x7f0c8c147e00 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f0c880196e0 tx=0x7f0c880a59e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:15.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c86000640 1 -- 172.21.15.122:0/3338424408 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880b2040 con 0x7f0c8c15bf60 2024-09-17T05:36:15.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c86000640 1 -- 172.21.15.122:0/3338424408 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c88004d40 con 0x7f0c8c15bf60 2024-09-17T05:36:15.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c86000640 1 -- 172.21.15.122:0/3338424408 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880a5e60 con 0x7f0c8c15bf60 2024-09-17T05:36:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.450+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c159820 con 0x7f0c8c15bf60 2024-09-17T05:36:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.454+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c159de0 con 0x7f0c8c15bf60 2024-09-17T05:36:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.454+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8c069410 con 0x7f0c8c15bf60 2024-09-17T05:36:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.454+0000 7f0c86000640 1 -- 172.21.15.122:0/3338424408 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f0c880be060 con 0x7f0c8c15bf60 2024-09-17T05:36:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.454+0000 7f0c86000640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0c70074880 0x7f0c70076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.454+0000 7f0c86000640 1 -- 172.21.15.122:0/3338424408 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f0c8812ca90 con 0x7f0c8c15bf60 2024-09-17T05:36:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.458+0000 7f0c91e00640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0c70074880 0x7f0c70076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.458+0000 7f0c91e00640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0c70074880 0x7f0c70076d40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0c80007fd0 tx=0x7f0c80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.458+0000 7f0c86000640 1 -- 172.21.15.122:0/3338424408 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0c880f3120 con 0x7f0c8c15bf60 2024-09-17T05:36:15.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.714+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone25105", "target": ["mon-mgr", ""]}) -- 0x7f0c8c0cdac0 con 0x7f0c70074880 2024-09-17T05:36:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.738+0000 7f0c86000640 1 -- 172.21.15.122:0/3338424408 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0c8c0cdac0 con 0x7f0c70074880 2024-09-17T05:36:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0c70074880 msgr2=0x7f0c70076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0c70074880 0x7f0c70076d40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0c80007fd0 tx=0x7f0c80009040 comp rx=0 tx=0).stop 2024-09-17T05:36:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c15bf60 msgr2=0x7f0c8c147e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c15bf60 0x7f0c8c147e00 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f0c880196e0 tx=0x7f0c880a59e0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 shutdown_connections 2024-09-17T05:36:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0c70074880 0x7f0c70076d40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c8c15bf60 0x7f0c8c147e00 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c8c0b5170 0x7f0c8c1ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 --2- 172.21.15.122:0/3338424408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c8c0af330 0x7f0c8c147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 >> 172.21.15.122:0/3338424408 conn(0x7f0c8c0cfbc0 msgr2=0x7f0c8c157c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 shutdown_connections 2024-09-17T05:36:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:15.742+0000 7f0c94800640 1 -- 172.21.15.122:0/3338424408 wait complete. 2024-09-17T05:36:15.780 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone125516 2024-09-17T05:36:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 -- 172.21.15.122:0/1311025759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15be10 msgr2=0x7fdfcc0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/1311025759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15be10 0x7fdfcc0ce700 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009bc0 tx=0x7fdfc401f3b0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 -- 172.21.15.122:0/1311025759 shutdown_connections 2024-09-17T05:36:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/1311025759 >> v2:172.21.15.173:3300/0 conn(0x7fdfcc0b0b60 0x7fdfcc0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/1311025759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15be10 0x7fdfcc0ce700 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 -- 172.21.15.122:0/1311025759 >> 172.21.15.122:0/1311025759 conn(0x7fdfcc0d1da0 msgr2=0x7fdfcc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 -- 172.21.15.122:0/1311025759 shutdown_connections 2024-09-17T05:36:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 -- 172.21.15.122:0/1311025759 wait complete. 2024-09-17T05:36:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 Processor -- start 2024-09-17T05:36:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.022+0000 7fdfd0a00640 1 -- start start 2024-09-17T05:36:16.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfcc0b0b60 0x7fdfcc0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:16.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfcc15be10 0x7fdfcc0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:16.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfd0a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15c1f0 0x7fdfcc0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:16.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfd0a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc0cd320 con 0x7fdfcc0b0b60 2024-09-17T05:36:16.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfd0a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc0cf590 con 0x7fdfcc15be10 2024-09-17T05:36:16.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfd0a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc0cd4a0 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfc9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfcc0b0b60 0x7fdfcc0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15c1f0 0x7fdfcc0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfc9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfcc15be10 0x7fdfcc0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15c1f0 0x7fdfcc0c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:48698/0 (socket says 172.21.15.122:48698) 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 -- 172.21.15.122:0/3128257638 learned_addr learned my addr 172.21.15.122:0/3128257638 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 -- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfcc15be10 msgr2=0x7fdfcc0c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfcc15be10 0x7fdfcc0c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 -- 172.21.15.122:0/3128257638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfcc0b0b60 msgr2=0x7fdfcc0c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfcc0b0b60 0x7fdfcc0c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 -- 172.21.15.122:0/3128257638 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc14f380 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfca800640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15c1f0 0x7fdfcc0c58d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e8c0 tx=0x7fdfbc00ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfc2000640 1 -- 172.21.15.122:0/3128257638 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00ccc0 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc14f660 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc14fc20 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfc2000640 1 -- 172.21.15.122:0/3128257638 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfbc010430 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.026+0000 7fdfc2000640 1 -- 172.21.15.122:0/3128257638 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc010800 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.030+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa40052a0 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.030+0000 7fdfc2000640 1 -- 172.21.15.122:0/3128257638 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fdfbc0109a0 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.030+0000 7fdfc2000640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfac074880 0x7fdfac076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.030+0000 7fdfc2000640 1 -- 172.21.15.122:0/3128257638 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7fdfbc093b20 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.030+0000 7fdfc9e00640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfac074880 0x7fdfac076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.034+0000 7fdfc9e00640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfac074880 0x7fdfac076d40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b1000 tx=0x7fdfc401dbf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:16.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.042+0000 7fdfc2000640 1 -- 172.21.15.122:0/3128257638 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdfbc0605a0 con 0x7fdfcc15c1f0 2024-09-17T05:36:16.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.314+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone125516", "target": ["mon-mgr", ""]}) -- 0x7fdfa4000bd0 con 0x7fdfac074880 2024-09-17T05:36:16.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfc2000640 1 -- 172.21.15.122:0/3128257638 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdfa4000bd0 con 0x7fdfac074880 2024-09-17T05:36:16.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfac074880 msgr2=0x7fdfac076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfac074880 0x7fdfac076d40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b1000 tx=0x7fdfc401dbf0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15c1f0 msgr2=0x7fdfcc0c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15c1f0 0x7fdfcc0c58d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00e8c0 tx=0x7fdfbc00ed80 comp rx=0 tx=0).stop 2024-09-17T05:36:16.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 shutdown_connections 2024-09-17T05:36:16.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdfac074880 0x7fdfac076d40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdfcc15c1f0 0x7fdfcc0c58d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfcc15be10 0x7fdfcc0c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 --2- 172.21.15.122:0/3128257638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfcc0b0b60 0x7fdfcc0c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 >> 172.21.15.122:0/3128257638 conn(0x7fdfcc0d1da0 msgr2=0x7fdfcc0cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:16.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 shutdown_connections 2024-09-17T05:36:16.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.330+0000 7fdfd0a00640 1 -- 172.21.15.122:0/3128257638 wait complete. 2024-09-17T05:36:16.349 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:36:16.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:36:16.403 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:36:16.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:36:16.411 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:36:16.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.610+0000 7fc5d1000640 1 -- 172.21.15.122:0/622944646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc161520 msgr2=0x7fc5cc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.610+0000 7fc5d1000640 1 --2- 172.21.15.122:0/622944646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc161520 0x7fc5cc06e7a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009bc0 tx=0x7fc5c401cba0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.610+0000 7fc5d1000640 1 -- 172.21.15.122:0/622944646 shutdown_connections 2024-09-17T05:36:16.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.610+0000 7fc5d1000640 1 --2- 172.21.15.122:0/622944646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc161520 0x7fc5cc06e7a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.610+0000 7fc5d1000640 1 -- 172.21.15.122:0/622944646 >> 172.21.15.122:0/622944646 conn(0x7fc5cc0b18f0 msgr2=0x7fc5cc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:16.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.610+0000 7fc5d1000640 1 -- 172.21.15.122:0/622944646 shutdown_connections 2024-09-17T05:36:16.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.610+0000 7fc5d1000640 1 -- 172.21.15.122:0/622944646 wait complete. 2024-09-17T05:36:16.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5d1000640 1 Processor -- start 2024-09-17T05:36:16.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5d1000640 1 -- start start 2024-09-17T05:36:16.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5d1000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5cc06ece0 0x7fc5cc09d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:16.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5d1000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc0b40a0 0x7fc5cc15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:16.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5d1000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5cc161520 0x7fc5cc15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:16.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5d1000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc16c260 con 0x7fc5cc0b40a0 2024-09-17T05:36:16.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5d1000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc16c0e0 con 0x7fc5cc06ece0 2024-09-17T05:36:16.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5d1000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc16c3e0 con 0x7fc5cc161520 2024-09-17T05:36:16.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5c9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc0b40a0 0x7fc5cc15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:16.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5c9400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc0b40a0 0x7fc5cc15e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40002/0 (socket says 172.21.15.122:40002) 2024-09-17T05:36:16.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5c9400640 1 -- 172.21.15.122:0/3246608729 learned_addr learned my addr 172.21.15.122:0/3246608729 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:16.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.614+0000 7fc5c9e00640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5cc06ece0 0x7fc5cc09d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:16.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c9e00640 1 -- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5cc161520 msgr2=0x7fc5cc15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c9e00640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5cc161520 0x7fc5cc15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c9e00640 1 -- 172.21.15.122:0/3246608729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc0b40a0 msgr2=0x7fc5cc15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c9e00640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc0b40a0 0x7fc5cc15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c9e00640 1 -- 172.21.15.122:0/3246608729 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc1fc150 con 0x7fc5cc06ece0 2024-09-17T05:36:16.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c9e00640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5cc06ece0 0x7fc5cc09d400 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40ac000 tx=0x7fc5c40a5960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:16.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c2000640 1 -- 172.21.15.122:0/3246608729 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40bd070 con 0x7fc5cc06ece0 2024-09-17T05:36:16.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c2000640 1 -- 172.21.15.122:0/3246608729 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5c40b3040 con 0x7fc5cc06ece0 2024-09-17T05:36:16.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5c2000640 1 -- 172.21.15.122:0/3246608729 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40b2070 con 0x7fc5cc06ece0 2024-09-17T05:36:16.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc1ffb80 con 0x7fc5cc06ece0 2024-09-17T05:36:16.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc200140 con 0x7fc5cc06ece0 2024-09-17T05:36:16.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.618+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc06e7a0 con 0x7fc5cc06ece0 2024-09-17T05:36:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.622+0000 7fc5c2000640 1 -- 172.21.15.122:0/3246608729 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fc5c40a5b60 con 0x7fc5cc06ece0 2024-09-17T05:36:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.622+0000 7fc5c2000640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc5ac074870 0x7fc5ac076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.622+0000 7fc5c2000640 1 -- 172.21.15.122:0/3246608729 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7fc5c412c8e0 con 0x7fc5cc06ece0 2024-09-17T05:36:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.622+0000 7fc5c9400640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc5ac074870 0x7fc5ac076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:16.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.626+0000 7fc5c9400640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc5ac074870 0x7fc5ac076d30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8007b40 tx=0x7fc5b8007a90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:16.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.626+0000 7fc5c2000640 1 -- 172.21.15.122:0/3246608729 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc5c4016790 con 0x7fc5cc06ece0 2024-09-17T05:36:16.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.934+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fc5cc09dc70 con 0x7fc5cc06ece0 2024-09-17T05:36:16.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.934+0000 7fc5c2000640 1 -- 172.21.15.122:0/3246608729 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v43)=0 v43) ==== 131+0+0 (secure 0 0 0) 0x7fc5c40f8f20 con 0x7fc5cc06ece0 2024-09-17T05:36:16.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc5ac074870 msgr2=0x7fc5ac076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc5ac074870 0x7fc5ac076d30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8007b40 tx=0x7fc5b8007a90 comp rx=0 tx=0).stop 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5cc06ece0 msgr2=0x7fc5cc09d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5cc06ece0 0x7fc5cc09d400 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40ac000 tx=0x7fc5c40a5960 comp rx=0 tx=0).stop 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 shutdown_connections 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc5ac074870 0x7fc5ac076d30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5cc161520 0x7fc5cc15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5cc0b40a0 0x7fc5cc15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 --2- 172.21.15.122:0/3246608729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5cc06ece0 0x7fc5cc09d400 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 >> 172.21.15.122:0/3246608729 conn(0x7fc5cc0b18f0 msgr2=0x7fc5cc157780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 shutdown_connections 2024-09-17T05:36:16.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:16.938+0000 7fc5d1000640 1 -- 172.21.15.122:0/3246608729 wait complete. 2024-09-17T05:36:16.953 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:36:16.953 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:36:17.055 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:36:17.178 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:36:17.178 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:36:17.178 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:36:17.179 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:17.187 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:36:17.187 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:17.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:36:17.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 -- 172.21.15.122:0/1970369650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc00b3d30 msgr2=0x7f1fc00b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:17.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1970369650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc00b3d30 0x7f1fc00b41e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009bc0 tx=0x7f1fb801cb10 comp rx=0 tx=0).stop 2024-09-17T05:36:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 -- 172.21.15.122:0/1970369650 shutdown_connections 2024-09-17T05:36:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1970369650 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc00b3d30 0x7f1fc00b41e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1970369650 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fc006f8c0 0x7f1fc006fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 -- 172.21.15.122:0/1970369650 >> 172.21.15.122:0/1970369650 conn(0x7f1fc00b18f0 msgr2=0x7f1fc00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 -- 172.21.15.122:0/1970369650 shutdown_connections 2024-09-17T05:36:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 -- 172.21.15.122:0/1970369650 wait complete. 2024-09-17T05:36:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 Processor -- start 2024-09-17T05:36:17.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.434+0000 7f1fc6600640 1 -- start start 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc006f8c0 0x7f1fc0141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc00b3d30 0x7f1fc01fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fc00cc740 0x7f1fc015df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0167580 con 0x7f1fc00b3d30 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0167400 con 0x7f1fc006f8c0 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc0167700 con 0x7f1fc00cc740 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc00b3d30 0x7f1fc01fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc00b3d30 0x7f1fc01fbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40008/0 (socket says 172.21.15.122:40008) 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 -- 172.21.15.122:0/1402500833 learned_addr learned my addr 172.21.15.122:0/1402500833 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbf400640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc006f8c0 0x7f1fc0141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbfe00640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fc00cc740 0x7f1fc015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:17.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 -- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fc00cc740 msgr2=0x7f1fc015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fc00cc740 0x7f1fc015df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 -- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc006f8c0 msgr2=0x7f1fc0141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc006f8c0 0x7f1fc0141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 -- 172.21.15.122:0/1402500833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc015d560 con 0x7f1fc00b3d30 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbf400640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc006f8c0 0x7f1fc0141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fbea00640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc00b3d30 0x7f1fc01fbb30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c390 tx=0x7f1fac00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fb7400640 1 -- 172.21.15.122:0/1402500833 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac017070 con 0x7f1fc00b3d30 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fb7400640 1 -- 172.21.15.122:0/1402500833 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fac003600 con 0x7f1fc00b3d30 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fb7400640 1 -- 172.21.15.122:0/1402500833 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac003a50 con 0x7f1fc00b3d30 2024-09-17T05:36:17.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc0157300 con 0x7f1fc00b3d30 2024-09-17T05:36:17.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01fb330 con 0x7f1fc00b3d30 2024-09-17T05:36:17.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.438+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc0069410 con 0x7f1fc00b3d30 2024-09-17T05:36:17.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.442+0000 7f1fb7400640 1 -- 172.21.15.122:0/1402500833 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f1fac01a070 con 0x7f1fc00b3d30 2024-09-17T05:36:17.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.442+0000 7f1fb7400640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa00748b0 0x7f1fa0076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:17.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.442+0000 7f1fb7400640 1 -- 172.21.15.122:0/1402500833 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f1fac092090 con 0x7f1fc00b3d30 2024-09-17T05:36:17.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.442+0000 7f1fbf400640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa00748b0 0x7f1fa0076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:17.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.442+0000 7f1fbf400640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa00748b0 0x7f1fa0076d70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000a7a0 tx=0x7f1fb000c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:17.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.446+0000 7f1fb7400640 1 -- 172.21.15.122:0/1402500833 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1fac05eba0 con 0x7f1fc00b3d30 2024-09-17T05:36:17.729 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:36:17.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.734+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1fc00ccdd0 con 0x7f1fc00b3d30 2024-09-17T05:36:17.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.734+0000 7f1fb7400640 1 -- 172.21.15.122:0/1402500833 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) ==== 80+0+0 (secure 0 0 0) 0x7f1fac05e4a0 con 0x7f1fc00b3d30 2024-09-17T05:36:17.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa00748b0 msgr2=0x7f1fa0076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:17.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa00748b0 0x7f1fa0076d70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000a7a0 tx=0x7f1fb000c040 comp rx=0 tx=0).stop 2024-09-17T05:36:17.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc00b3d30 msgr2=0x7f1fc01fbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:17.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc00b3d30 0x7f1fc01fbb30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c390 tx=0x7f1fac00c850 comp rx=0 tx=0).stop 2024-09-17T05:36:17.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 shutdown_connections 2024-09-17T05:36:17.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fa00748b0 0x7f1fa0076d70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:17.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fc00cc740 0x7f1fc015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:17.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc00b3d30 0x7f1fc01fbb30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:17.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 --2- 172.21.15.122:0/1402500833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fc006f8c0 0x7f1fc0141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:17.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.738+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 >> 172.21.15.122:0/1402500833 conn(0x7f1fc00b18f0 msgr2=0x7f1fc0143390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:17.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.742+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 shutdown_connections 2024-09-17T05:36:17.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:17.742+0000 7f1fc6600640 1 -- 172.21.15.122:0/1402500833 wait complete. 2024-09-17T05:36:17.769 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:36:17.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:17.769 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:36:17.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:36:17.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:17.862 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:36:17.877 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:36:17.878 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:36:17.878 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:17.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:36:17.891 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:36:17.892 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:36:17.892 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:17.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:36:17.936 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:36:17.936 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:36:17.936 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:17.952 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:36:17.952 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:36:18.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.210+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1374907587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 msgr2=0x7fd8d4073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:18.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.210+0000 7fd8d0a00640 1 -- 172.21.15.122:0/1374907587 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40a5db0 con 0x7fd8d40cffc0 2024-09-17T05:36:18.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.210+0000 7fd8d8e00640 1 --2- 172.21.15.122:0/1374907587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 0x7fd8d4073ed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c401c830 comp rx=0 tx=0).stop 2024-09-17T05:36:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.210+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1374907587 shutdown_connections 2024-09-17T05:36:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.210+0000 7fd8d8e00640 1 --2- 172.21.15.122:0/1374907587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 0x7fd8d4073ed0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.210+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1374907587 >> 172.21.15.122:0/1374907587 conn(0x7fd8d40d0c90 msgr2=0x7fd8d40d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.210+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1374907587 shutdown_connections 2024-09-17T05:36:18.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.210+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1374907587 wait complete. 2024-09-17T05:36:18.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d8e00640 1 Processor -- start 2024-09-17T05:36:18.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d8e00640 1 -- start start 2024-09-17T05:36:18.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d8e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8d406f9a0 0x7fd8d40c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:18.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d8e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 0x7fd8d40c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8d415af60 0x7fd8d40b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d8e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d4169560 con 0x7fd8d415af60 2024-09-17T05:36:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d8e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d4070b50 con 0x7fd8d406f9a0 2024-09-17T05:36:18.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d8e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d41696e0 con 0x7fd8d40cffc0 2024-09-17T05:36:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d1400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 0x7fd8d40c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8d406f9a0 0x7fd8d40c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d1400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 0x7fd8d40c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:48738/0 (socket says 172.21.15.122:48738) 2024-09-17T05:36:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d1400640 1 -- 172.21.15.122:0/1567445625 learned_addr learned my addr 172.21.15.122:0/1567445625 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.214+0000 7fd8d2800640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8d415af60 0x7fd8d40b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d1400640 1 -- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8d406f9a0 msgr2=0x7fd8d40c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d1400640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8d406f9a0 0x7fd8d40c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d1400640 1 -- 172.21.15.122:0/1567445625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8d415af60 msgr2=0x7fd8d40b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d1400640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8d415af60 0x7fd8d40b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d1400640 1 -- 172.21.15.122:0/1567445625 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d40c6f90 con 0x7fd8d40cffc0 2024-09-17T05:36:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d1400640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 0x7fd8d40c62c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c40a5740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8ca000640 1 -- 172.21.15.122:0/1567445625 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40bd070 con 0x7fd8d40cffc0 2024-09-17T05:36:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8ca000640 1 -- 172.21.15.122:0/1567445625 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8c4004470 con 0x7fd8d40cffc0 2024-09-17T05:36:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8ca000640 1 -- 172.21.15.122:0/1567445625 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40b2040 con 0x7fd8d40cffc0 2024-09-17T05:36:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d40b66f0 con 0x7fd8d40cffc0 2024-09-17T05:36:18.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d4157550 con 0x7fd8d40cffc0 2024-09-17T05:36:18.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.218+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d4071bf0 con 0x7fd8d40cffc0 2024-09-17T05:36:18.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.222+0000 7fd8ca000640 1 -- 172.21.15.122:0/1567445625 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fd8c40c0070 con 0x7fd8d40cffc0 2024-09-17T05:36:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.222+0000 7fd8ca000640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8b0074950 0x7fd8b0076e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:18.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.222+0000 7fd8ca000640 1 -- 172.21.15.122:0/1567445625 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7fd8c412cea0 con 0x7fd8d40cffc0 2024-09-17T05:36:18.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.222+0000 7fd8d1e00640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8b0074950 0x7fd8b0076e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:18.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.226+0000 7fd8d1e00640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8b0074950 0x7fd8b0076e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc004450 tx=0x7fd8cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:18.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.230+0000 7fd8ca000640 1 -- 172.21.15.122:0/1567445625 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd8c4016450 con 0x7fd8d40cffc0 2024-09-17T05:36:18.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.442+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8d40b6400 con 0x7fd8d40cffc0 2024-09-17T05:36:18.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.442+0000 7fd8ca000640 1 -- 172.21.15.122:0/1567445625 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5237 (secure 0 0 0) 0x7fd8c40f9a30 con 0x7fd8d40cffc0 2024-09-17T05:36:18.438 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:18.438 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":57,"btime":"2024-09-17T05:34:18:421377+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1117224890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1117224890},{"type":"v1","addr":"172.21.15.122:6835","nonce":1117224890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2271477440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2271477440},{"type":"v1","addr":"172.21.15.173:6837","nonce":2271477440}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3081382903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3081382903},{"type":"v1","addr":"172.21.15.173:6836","nonce":3081382903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:09.761813+0000","modified":"2024-09-17T05:34:12.779372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6834/3516694104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3516694104},{"type":"v1","addr":"172.21.15.122:6834","nonce":3516694104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-17T05:36:18.438 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 57 2024-09-17T05:36:18.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.442+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8b0074950 msgr2=0x7fd8b0076e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:18.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.442+0000 7fd8d8e00640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8b0074950 0x7fd8b0076e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc004450 tx=0x7fd8cc009040 comp rx=0 tx=0).stop 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.442+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 msgr2=0x7fd8d40c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.442+0000 7fd8d8e00640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 0x7fd8d40c62c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c40a5740 comp rx=0 tx=0).stop 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.442+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 shutdown_connections 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.442+0000 7fd8d8e00640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd8b0074950 0x7fd8b0076e10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.446+0000 7fd8d8e00640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8d415af60 0x7fd8d40b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.446+0000 7fd8d8e00640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8d40cffc0 0x7fd8d40c62c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.446+0000 7fd8d8e00640 1 --2- 172.21.15.122:0/1567445625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8d406f9a0 0x7fd8d40c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.446+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 >> 172.21.15.122:0/1567445625 conn(0x7fd8d40d0c90 msgr2=0x7fd8d40737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.446+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 shutdown_connections 2024-09-17T05:36:18.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.446+0000 7fd8d8e00640 1 -- 172.21.15.122:0/1567445625 wait complete. 2024-09-17T05:36:18.460 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:36:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.678+0000 7f5c66c00640 1 -- 172.21.15.122:0/321017941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 msgr2=0x7f5c600b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.678+0000 7f5c66c00640 1 --2- 172.21.15.122:0/321017941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 0x7f5c600b81c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009bf0 tx=0x7f5c5801cc00 comp rx=0 tx=0).stop 2024-09-17T05:36:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.678+0000 7f5c66c00640 1 -- 172.21.15.122:0/321017941 shutdown_connections 2024-09-17T05:36:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.678+0000 7f5c66c00640 1 --2- 172.21.15.122:0/321017941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 0x7f5c600b81c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.678+0000 7f5c66c00640 1 -- 172.21.15.122:0/321017941 >> 172.21.15.122:0/321017941 conn(0x7f5c600d3fe0 msgr2=0x7f5c600d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.678+0000 7f5c66c00640 1 -- 172.21.15.122:0/321017941 shutdown_connections 2024-09-17T05:36:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.678+0000 7f5c66c00640 1 -- 172.21.15.122:0/321017941 wait complete. 2024-09-17T05:36:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.682+0000 7f5c66c00640 1 Processor -- start 2024-09-17T05:36:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.682+0000 7f5c66c00640 1 -- start start 2024-09-17T05:36:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c66c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 0x7f5c601523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c66c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c600d2060 0x7f5c6015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c66c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c600d2b50 0x7f5c60151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c66c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6016c4c0 con 0x7f5c600d2060 2024-09-17T05:36:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c66c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6016c340 con 0x7f5c600d2b50 2024-09-17T05:36:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c66c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6016c640 con 0x7f5c600cf660 2024-09-17T05:36:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 0x7f5c601523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 0x7f5c601523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:48750/0 (socket says 172.21.15.122:48750) 2024-09-17T05:36:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 -- 172.21.15.122:0/2638141847 learned_addr learned my addr 172.21.15.122:0/2638141847 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c64c00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c600d2b50 0x7f5c60151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 -- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c600d2b50 msgr2=0x7f5c60151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c600d2b50 0x7f5c60151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 -- 172.21.15.122:0/2638141847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c600d2060 msgr2=0x7f5c6015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c600d2060 0x7f5c6015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 -- 172.21.15.122:0/2638141847 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c600c8350 con 0x7f5c600cf660 2024-09-17T05:36:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c64c00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c600d2b50 0x7f5c60151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:36:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c5fe00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 0x7f5c601523e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c410 tx=0x7f5c4800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c57e00640 1 -- 172.21.15.122:0/2638141847 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48017070 con 0x7f5c600cf660 2024-09-17T05:36:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c57e00640 1 -- 172.21.15.122:0/2638141847 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c48003600 con 0x7f5c600cf660 2024-09-17T05:36:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c57e00640 1 -- 172.21.15.122:0/2638141847 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48003a90 con 0x7f5c600cf660 2024-09-17T05:36:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.686+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6015e1f0 con 0x7f5c600cf660 2024-09-17T05:36:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.690+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c60152ba0 con 0x7f5c600cf660 2024-09-17T05:36:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.690+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6015df70 con 0x7f5c600cf660 2024-09-17T05:36:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.690+0000 7f5c57e00640 1 -- 172.21.15.122:0/2638141847 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f5c4801a070 con 0x7f5c600cf660 2024-09-17T05:36:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.694+0000 7f5c57e00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c3c0748d0 0x7f5c3c076d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.694+0000 7f5c57e00640 1 -- 172.21.15.122:0/2638141847 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5821+0+0 (secure 0 0 0) 0x7f5c48092130 con 0x7f5c600cf660 2024-09-17T05:36:18.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.694+0000 7f5c5f400640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c3c0748d0 0x7f5c3c076d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:18.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.694+0000 7f5c5f400640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c3c0748d0 0x7f5c3c076d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5c600c7480 tx=0x7f5c50006bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:18.698+0000 7f5c57e00640 1 -- 172.21.15.122:0/2638141847 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5c6015df70 con 0x7f5c600cf660 2024-09-17T05:36:19.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.094+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c600d07d0 con 0x7f5c600cf660 2024-09-17T05:36:19.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.094+0000 7f5c57e00640 1 -- 172.21.15.122:0/2638141847 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5237 (secure 0 0 0) 0x7f5c4805e6f0 con 0x7f5c600cf660 2024-09-17T05:36:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:19.091 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":57,"btime":"2024-09-17T05:34:18:421377+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1117224890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1117224890},{"type":"v1","addr":"172.21.15.122:6835","nonce":1117224890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2271477440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2271477440},{"type":"v1","addr":"172.21.15.173:6837","nonce":2271477440}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3081382903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3081382903},{"type":"v1","addr":"172.21.15.173:6836","nonce":3081382903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:09.761813+0000","modified":"2024-09-17T05:34:12.779372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6834/3516694104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3516694104},{"type":"v1","addr":"172.21.15.122:6834","nonce":3516694104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-17T05:36:19.092 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 57 2024-09-17T05:36:19.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c3c0748d0 msgr2=0x7f5c3c076d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c3c0748d0 0x7f5c3c076d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5c600c7480 tx=0x7f5c50006bd0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 msgr2=0x7f5c601523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 0x7f5c601523e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c410 tx=0x7f5c4800c8d0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 shutdown_connections 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5c3c0748d0 0x7f5c3c076d90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c600d2b50 0x7f5c60151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c600d2060 0x7f5c6015d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 --2- 172.21.15.122:0/2638141847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c600cf660 0x7f5c601523e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 >> 172.21.15.122:0/2638141847 conn(0x7f5c600d3fe0 msgr2=0x7f5c600d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 shutdown_connections 2024-09-17T05:36:19.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.098+0000 7f5c66c00640 1 -- 172.21.15.122:0/2638141847 wait complete. 2024-09-17T05:36:19.113 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:36:19.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.258+0000 7f3ed9a00640 1 -- 172.21.15.122:0/1298461138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ed415a7e0 msgr2=0x7f3ed40ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.258+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/1298461138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ed415a7e0 0x7f3ed40ae4d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009900 tx=0x7f3ec401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.258+0000 7f3ed9a00640 1 -- 172.21.15.122:0/1298461138 shutdown_connections 2024-09-17T05:36:19.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.258+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/1298461138 >> v2:172.21.15.173:3300/0 conn(0x7f3ed40aea10 0x7f3ed40d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.258+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/1298461138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ed415a7e0 0x7f3ed40ae4d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.258+0000 7f3ed9a00640 1 -- 172.21.15.122:0/1298461138 >> 172.21.15.122:0/1298461138 conn(0x7f3ed406dd30 msgr2=0x7f3ed406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:19.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.258+0000 7f3ed9a00640 1 -- 172.21.15.122:0/1298461138 shutdown_connections 2024-09-17T05:36:19.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.258+0000 7f3ed9a00640 1 -- 172.21.15.122:0/1298461138 wait complete. 2024-09-17T05:36:19.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 Processor -- start 2024-09-17T05:36:19.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 -- start start 2024-09-17T05:36:19.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ed40aea10 0x7f3ed4161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ed415a7e0 0x7f3ed41484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ed415abc0 0x7f3ed41450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4168000 con 0x7f3ed415a7e0 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4167e80 con 0x7f3ed415abc0 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed4168180 con 0x7f3ed40aea10 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ed415a7e0 0x7f3ed41484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ed415a7e0 0x7f3ed41484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40032/0 (socket says 172.21.15.122:40032) 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 -- 172.21.15.122:0/431570491 learned_addr learned my addr 172.21.15.122:0/431570491 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed2a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ed40aea10 0x7f3ed4161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed3400640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ed415abc0 0x7f3ed41450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 -- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ed40aea10 msgr2=0x7f3ed4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ed40aea10 0x7f3ed4161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 -- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ed415abc0 msgr2=0x7f3ed41450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ed415abc0 0x7f3ed41450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 -- 172.21.15.122:0/431570491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed415f880 con 0x7f3ed415a7e0 2024-09-17T05:36:19.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed3400640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ed415abc0 0x7f3ed41450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:19.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3eca000640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ed415a7e0 0x7f3ed41484e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009d10 tx=0x7f3ebc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:19.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ecb400640 1 -- 172.21.15.122:0/431570491 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc010630 con 0x7f3ed415a7e0 2024-09-17T05:36:19.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ecb400640 1 -- 172.21.15.122:0/431570491 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ebc0108c0 con 0x7f3ed415a7e0 2024-09-17T05:36:19.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ecb400640 1 -- 172.21.15.122:0/431570491 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc010d10 con 0x7f3ed415a7e0 2024-09-17T05:36:19.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed415fb60 con 0x7f3ed415a7e0 2024-09-17T05:36:19.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.262+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed4160120 con 0x7f3ed415a7e0 2024-09-17T05:36:19.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.266+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed4160320 con 0x7f3ed415a7e0 2024-09-17T05:36:19.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.266+0000 7f3ecb400640 1 -- 172.21.15.122:0/431570491 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f3ebc018420 con 0x7f3ed415a7e0 2024-09-17T05:36:19.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.266+0000 7f3ecb400640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3eb0074880 0x7f3eb0076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.266+0000 7f3ecb400640 1 -- 172.21.15.122:0/431570491 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5793+0+0 (secure 0 0 0) 0x7f3ebc093dc0 con 0x7f3ed415a7e0 2024-09-17T05:36:19.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.266+0000 7f3ed2a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3eb0074880 0x7f3eb0076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.270+0000 7f3ed2a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3eb0074880 0x7f3eb0076d40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40af000 tx=0x7f3ec401b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:19.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.270+0000 7f3ecb400640 1 -- 172.21.15.122:0/431570491 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3ed4160320 con 0x7f3ed415a7e0 2024-09-17T05:36:19.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.442+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ed40710e0 con 0x7f3ed415a7e0 2024-09-17T05:36:19.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.442+0000 7f3ecb400640 1 -- 172.21.15.122:0/431570491 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16945 (secure 0 0 0) 0x7f3ebc0608d0 con 0x7f3ed415a7e0 2024-09-17T05:36:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:19.439 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":92,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:36:19.045814+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:07.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:08.718769+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":91,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3eb0074880 msgr2=0x7f3eb0076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3eb0074880 0x7f3eb0076d40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40af000 tx=0x7f3ec401b120 comp rx=0 tx=0).stop 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ed415a7e0 msgr2=0x7f3ed41484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ed415a7e0 0x7f3ed41484e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009d10 tx=0x7f3ebc00c6a0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 shutdown_connections 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3eb0074880 0x7f3eb0076d40 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ed415abc0 0x7f3ed41450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ed415a7e0 0x7f3ed41484e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 --2- 172.21.15.122:0/431570491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ed40aea10 0x7f3ed4161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 >> 172.21.15.122:0/431570491 conn(0x7f3ed406dd30 msgr2=0x7f3ed41596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:19.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 shutdown_connections 2024-09-17T05:36:19.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.446+0000 7f3ed9a00640 1 -- 172.21.15.122:0/431570491 wait complete. 2024-09-17T05:36:19.460 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:36:19.460 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:36:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc84e00640 1 -- 172.21.15.122:0/480113806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc64004d50 msgr2=0x7fbc640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc84e00640 1 --2- 172.21.15.122:0/480113806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc64004d50 0x7fbc640051d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009b30 tx=0x7fbc7801d270 comp rx=0 tx=0).stop 2024-09-17T05:36:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc7e800640 1 -- 172.21.15.122:0/480113806 reap_dead start 2024-09-17T05:36:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc84e00640 1 -- 172.21.15.122:0/480113806 shutdown_connections 2024-09-17T05:36:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc84e00640 1 -- 172.21.15.122:0/480113806 >> 172.21.15.122:0/480113806 conn(0x7fbc800d1ea0 msgr2=0x7fbc800d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc84e00640 1 -- 172.21.15.122:0/480113806 shutdown_connections 2024-09-17T05:36:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc84e00640 1 -- 172.21.15.122:0/480113806 wait complete. 2024-09-17T05:36:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc84e00640 1 Processor -- start 2024-09-17T05:36:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.606+0000 7fbc84e00640 1 -- start start 2024-09-17T05:36:19.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc64004d50 0x7fbc800b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc8015a7e0 0x7fbc8015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc801e6a20 0x7fbc80071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc800cc940 con 0x7fbc801e6a20 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc800cf770 con 0x7fbc64004d50 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbc800ccac0 con 0x7fbc8015a7e0 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc801e6a20 0x7fbc80071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc801e6a20 0x7fbc80071e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40046/0 (socket says 172.21.15.122:40046) 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 -- 172.21.15.122:0/1109949386 learned_addr learned my addr 172.21.15.122:0/1109949386 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc75400640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc8015a7e0 0x7fbc8015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7de00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc64004d50 0x7fbc800b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 -- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc8015a7e0 msgr2=0x7fbc8015a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc8015a7e0 0x7fbc8015a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 -- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc64004d50 msgr2=0x7fbc800b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc64004d50 0x7fbc800b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 -- 172.21.15.122:0/1109949386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc801f45b0 con 0x7fbc801e6a20 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc75400640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc8015a7e0 0x7fbc8015a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7de00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc64004d50 0x7fbc800b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc7e800640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc801e6a20 0x7fbc80071e70 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbc780af040 tx=0x7fbc780addc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc76a00640 1 -- 172.21.15.122:0/1109949386 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc780b1040 con 0x7fbc801e6a20 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc76a00640 1 -- 172.21.15.122:0/1109949386 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc780bb070 con 0x7fbc801e6a20 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc76a00640 1 -- 172.21.15.122:0/1109949386 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78004290 con 0x7fbc801e6a20 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc801f89f0 con 0x7fbc801e6a20 2024-09-17T05:36:19.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc800b7890 con 0x7fbc801e6a20 2024-09-17T05:36:19.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.610+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc800d2c40 con 0x7fbc801e6a20 2024-09-17T05:36:19.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.614+0000 7fbc76a00640 1 -- 172.21.15.122:0/1109949386 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fbc78005070 con 0x7fbc801e6a20 2024-09-17T05:36:19.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.614+0000 7fbc76a00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc5c074880 0x7fbc5c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.614+0000 7fbc76a00640 1 -- 172.21.15.122:0/1109949386 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5793+0+0 (secure 0 0 0) 0x7fbc7812de10 con 0x7fbc801e6a20 2024-09-17T05:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.614+0000 7fbc7de00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc5c074880 0x7fbc5c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.614+0000 7fbc7de00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc5c074880 0x7fbc5c076d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbc70002240 tx=0x7fbc70006ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:19.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.618+0000 7fbc76a00640 1 -- 172.21.15.122:0/1109949386 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbc78016f80 con 0x7fbc801e6a20 2024-09-17T05:36:19.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.818+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc800d2500 con 0x7fbc801e6a20 2024-09-17T05:36:19.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.818+0000 7fbc76a00640 1 -- 172.21.15.122:0/1109949386 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5237 (secure 0 0 0) 0x7fbc780fa920 con 0x7fbc801e6a20 2024-09-17T05:36:19.816 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:19.816 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":57,"btime":"2024-09-17T05:34:18:421377+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1117224890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1117224890},{"type":"v1","addr":"172.21.15.122:6835","nonce":1117224890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2271477440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2271477440},{"type":"v1","addr":"172.21.15.173:6837","nonce":2271477440}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3081382903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3081382903},{"type":"v1","addr":"172.21.15.173:6836","nonce":3081382903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:09.761813+0000","modified":"2024-09-17T05:34:12.779372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6834/3516694104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3516694104},{"type":"v1","addr":"172.21.15.122:6834","nonce":3516694104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-17T05:36:19.816 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 57 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc5c074880 msgr2=0x7fbc5c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc5c074880 0x7fbc5c076d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbc70002240 tx=0x7fbc70006ab0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc801e6a20 msgr2=0x7fbc80071e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc801e6a20 0x7fbc80071e70 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbc780af040 tx=0x7fbc780addc0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 shutdown_connections 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc5c074880 0x7fbc5c076d40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc801e6a20 0x7fbc80071e70 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc8015a7e0 0x7fbc8015a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 --2- 172.21.15.122:0/1109949386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc64004d50 0x7fbc800b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 >> 172.21.15.122:0/1109949386 conn(0x7fbc800d1ea0 msgr2=0x7fbc800cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:19.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 shutdown_connections 2024-09-17T05:36:19.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.822+0000 7fbc84e00640 1 -- 172.21.15.122:0/1109949386 wait complete. 2024-09-17T05:36:19.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:36:19.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 -- 172.21.15.122:0/638394222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01200b2e80 msgr2=0x7f01200b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 --2- 172.21.15.122:0/638394222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01200b2e80 0x7f01200b1a00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f010c009840 tx=0x7f010c01c6d0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 -- 172.21.15.122:0/638394222 shutdown_connections 2024-09-17T05:36:19.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 --2- 172.21.15.122:0/638394222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01200b2e80 0x7f01200b1a00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 --2- 172.21.15.122:0/638394222 >> v2:172.21.15.173:3300/0 conn(0x7f012015f490 0x7f01200b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 -- 172.21.15.122:0/638394222 >> 172.21.15.122:0/638394222 conn(0x7f01200d1e30 msgr2=0x7f01200d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:19.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 -- 172.21.15.122:0/638394222 shutdown_connections 2024-09-17T05:36:19.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 -- 172.21.15.122:0/638394222 wait complete. 2024-09-17T05:36:19.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 Processor -- start 2024-09-17T05:36:19.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.982+0000 7f0125800640 1 -- start start 2024-09-17T05:36:19.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01200b1f40 0x7f0120150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01200b2e80 0x7f0120152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f012015f490 0x7f012006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0120168e90 con 0x7f01200b1f40 2024-09-17T05:36:19.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f012015f9e0 con 0x7f012015f490 2024-09-17T05:36:19.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0120169010 con 0x7f01200b2e80 2024-09-17T05:36:19.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01200b1f40 0x7f0120150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01200b1f40 0x7f0120150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40048/0 (socket says 172.21.15.122:40048) 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 -- 172.21.15.122:0/99111198 learned_addr learned my addr 172.21.15.122:0/99111198 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011e000640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01200b2e80 0x7f0120152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011f400640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f012015f490 0x7f012006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 -- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01200b2e80 msgr2=0x7f0120152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01200b2e80 0x7f0120152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 -- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f012015f490 msgr2=0x7f012006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f012015f490 0x7f012006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 -- 172.21.15.122:0/99111198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01200d0020 con 0x7f01200b1f40 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f011ea00640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01200b1f40 0x7f0120150ef0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f011000e9a0 tx=0x7f011000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0116a00640 1 -- 172.21.15.122:0/99111198 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011000cd80 con 0x7f01200b1f40 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0116a00640 1 -- 172.21.15.122:0/99111198 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0110004020 con 0x7f01200b1f40 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012015d9b0 con 0x7f01200b1f40 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0116a00640 1 -- 172.21.15.122:0/99111198 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110004450 con 0x7f01200b1f40 2024-09-17T05:36:19.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0120151d50 con 0x7f01200b1f40 2024-09-17T05:36:19.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.986+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01200cfda0 con 0x7f01200b1f40 2024-09-17T05:36:19.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.990+0000 7f0116a00640 1 -- 172.21.15.122:0/99111198 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f0110003600 con 0x7f01200b1f40 2024-09-17T05:36:19.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.990+0000 7f0116a00640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100074880 0x7f0100076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:19.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.990+0000 7f0116a00640 1 -- 172.21.15.122:0/99111198 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5793+0+0 (secure 0 0 0) 0x7f0110093850 con 0x7f01200b1f40 2024-09-17T05:36:19.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.990+0000 7f011e000640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100074880 0x7f0100076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:19.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.990+0000 7f011e000640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100074880 0x7f0100076d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f010c0af000 tx=0x7f010c01af10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:19.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:19.990+0000 7f0116a00640 1 -- 172.21.15.122:0/99111198 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f011005fe90 con 0x7f01200b1f40 2024-09-17T05:36:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.190+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0120151740 con 0x7f01200b1f40 2024-09-17T05:36:20.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0116a00640 1 -- 172.21.15.122:0/99111198 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5237 (secure 0 0 0) 0x7f01100648c0 con 0x7f01200b1f40 2024-09-17T05:36:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:20.189 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":57,"btime":"2024-09-17T05:34:18:421377+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1117224890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1117224890},{"type":"v1","addr":"172.21.15.122:6835","nonce":1117224890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2271477440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2271477440},{"type":"v1","addr":"172.21.15.173:6837","nonce":2271477440}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3081382903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3081382903},{"type":"v1","addr":"172.21.15.173:6836","nonce":3081382903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:34:09.761813+0000","modified":"2024-09-17T05:34:12.779372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7029":{"gid":7029,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6834/3516694104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3516694104},{"type":"v1","addr":"172.21.15.122:6834","nonce":3516694104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7029,"qdb_cluster":[7029]},"id":6}]} 2024-09-17T05:36:20.189 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 57 2024-09-17T05:36:20.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100074880 msgr2=0x7f0100076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100074880 0x7f0100076d40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f010c0af000 tx=0x7f010c01af10 comp rx=0 tx=0).stop 2024-09-17T05:36:20.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01200b1f40 msgr2=0x7f0120150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01200b1f40 0x7f0120150ef0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f011000e9a0 tx=0x7f011000ee60 comp rx=0 tx=0).stop 2024-09-17T05:36:20.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 shutdown_connections 2024-09-17T05:36:20.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0100074880 0x7f0100076d40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f012015f490 0x7f012006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01200b2e80 0x7f0120152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 --2- 172.21.15.122:0/99111198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01200b1f40 0x7f0120150ef0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 >> 172.21.15.122:0/99111198 conn(0x7f01200d1e30 msgr2=0x7f01200ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:20.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 shutdown_connections 2024-09-17T05:36:20.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.194+0000 7f0125800640 1 -- 172.21.15.122:0/99111198 wait complete. 2024-09-17T05:36:20.210 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:36:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- 172.21.15.122:0/2140999394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb64004810 msgr2=0x7fdb64004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 --2- 172.21.15.122:0/2140999394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb64004810 0x7fdb64004c90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c006fa0 tx=0x7fdb6c01ce80 comp rx=0 tx=0).stop 2024-09-17T05:36:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- 172.21.15.122:0/2140999394 shutdown_connections 2024-09-17T05:36:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 --2- 172.21.15.122:0/2140999394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb64004810 0x7fdb64004c90 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- 172.21.15.122:0/2140999394 >> 172.21.15.122:0/2140999394 conn(0x7fdb800b2240 msgr2=0x7fdb800b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- 172.21.15.122:0/2140999394 shutdown_connections 2024-09-17T05:36:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- 172.21.15.122:0/2140999394 wait complete. 2024-09-17T05:36:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 Processor -- start 2024-09-17T05:36:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- start start 2024-09-17T05:36:20.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb64004810 0x7fdb8013e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:20.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80139c30 0x7fdb80139380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8013b550 0x7fdb8015d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8006cc20 con 0x7fdb8013b550 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801441f0 con 0x7fdb80139c30 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb86200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8006cda0 con 0x7fdb64004810 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb7fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8013b550 0x7fdb8015d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb7fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8013b550 0x7fdb8015d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40070/0 (socket says 172.21.15.122:40070) 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.358+0000 7fdb7fe00640 1 -- 172.21.15.122:0/1360438705 learned_addr learned my addr 172.21.15.122:0/1360438705 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7ea00640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80139c30 0x7fdb80139380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7f400640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb64004810 0x7fdb8013e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7fe00640 1 -- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb64004810 msgr2=0x7fdb8013e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7fe00640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb64004810 0x7fdb8013e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7fe00640 1 -- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80139c30 msgr2=0x7fdb80139380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7fe00640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80139c30 0x7fdb80139380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7fe00640 1 -- 172.21.15.122:0/1360438705 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8013f380 con 0x7fdb8013b550 2024-09-17T05:36:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7ea00640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80139c30 0x7fdb80139380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:20.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7f400640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb64004810 0x7fdb8013e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb7fe00640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8013b550 0x7fdb8015d490 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fdb800ce020 tx=0x7fdb78011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb77400640 1 -- 172.21.15.122:0/1360438705 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78004230 con 0x7fdb8013b550 2024-09-17T05:36:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb77400640 1 -- 172.21.15.122:0/1360438705 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb78016070 con 0x7fdb8013b550 2024-09-17T05:36:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb77400640 1 -- 172.21.15.122:0/1360438705 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800faf0 con 0x7fdb8013b550 2024-09-17T05:36:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb80138bc0 con 0x7fdb8013b550 2024-09-17T05:36:20.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8013f0a0 con 0x7fdb8013b550 2024-09-17T05:36:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8013a010 con 0x7fdb8013b550 2024-09-17T05:36:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb77400640 1 -- 172.21.15.122:0/1360438705 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fdb78003600 con 0x7fdb8013b550 2024-09-17T05:36:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb77400640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60074880 0x7fdb60076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.362+0000 7fdb77400640 1 -- 172.21.15.122:0/1360438705 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5793+0+0 (secure 0 0 0) 0x7fdb78093120 con 0x7fdb8013b550 2024-09-17T05:36:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.366+0000 7fdb7f400640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60074880 0x7fdb60076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.366+0000 7fdb7f400640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60074880 0x7fdb60076d40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000afd0 tx=0x7fdb70001260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:20.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.366+0000 7fdb77400640 1 -- 172.21.15.122:0/1360438705 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdb78021070 con 0x7fdb8013b550 2024-09-17T05:36:20.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.538+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb8015e8b0 con 0x7fdb8013b550 2024-09-17T05:36:20.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.538+0000 7fdb77400640 1 -- 172.21.15.122:0/1360438705 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16945 (secure 0 0 0) 0x7fdb7805fc30 con 0x7fdb8013b550 2024-09-17T05:36:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:20.536 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":92,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:36:19.045814+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:34:07.443301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-17T05:34:08.718769+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":91,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60074880 msgr2=0x7fdb60076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60074880 0x7fdb60076d40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000afd0 tx=0x7fdb70001260 comp rx=0 tx=0).stop 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8013b550 msgr2=0x7fdb8015d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8013b550 0x7fdb8015d490 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fdb800ce020 tx=0x7fdb78011b10 comp rx=0 tx=0).stop 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 shutdown_connections 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb60074880 0x7fdb60076d40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb8013b550 0x7fdb8015d490 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80139c30 0x7fdb80139380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 --2- 172.21.15.122:0/1360438705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb64004810 0x7fdb8013e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 >> 172.21.15.122:0/1360438705 conn(0x7fdb800b2240 msgr2=0x7fdb8006f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:20.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 shutdown_connections 2024-09-17T05:36:20.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.542+0000 7fdb86200640 1 -- 172.21.15.122:0/1360438705 wait complete. 2024-09-17T05:36:20.557 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:36:20.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde34200640 1 -- 172.21.15.122:0/1756973952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde14004010 msgr2=0x7fde14004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde34200640 1 --2- 172.21.15.122:0/1756973952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde14004010 0x7fde14004810 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fde180016f0 tx=0x7fde1801c7b0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde32200640 1 -- 172.21.15.122:0/1756973952 reap_dead start 2024-09-17T05:36:20.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde34200640 1 -- 172.21.15.122:0/1756973952 shutdown_connections 2024-09-17T05:36:20.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde34200640 1 -- 172.21.15.122:0/1756973952 >> 172.21.15.122:0/1756973952 conn(0x7fde2c0d1dc0 msgr2=0x7fde2c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:20.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde34200640 1 -- 172.21.15.122:0/1756973952 shutdown_connections 2024-09-17T05:36:20.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde34200640 1 -- 172.21.15.122:0/1756973952 wait complete. 2024-09-17T05:36:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde34200640 1 Processor -- start 2024-09-17T05:36:20.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.706+0000 7fde34200640 1 -- start start 2024-09-17T05:36:20.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde14004010 0x7fde2c0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde2c152dd0 0x7fde2c153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c09c200 0x7fde2c09c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c15c400 con 0x7fde2c09c200 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c15c280 con 0x7fde2c152dd0 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fde2c0ce670 con 0x7fde14004010 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c09c200 0x7fde2c09c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c09c200 0x7fde2c09c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40084/0 (socket says 172.21.15.122:40084) 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde31800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde14004010 0x7fde2c0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 -- 172.21.15.122:0/3292495357 learned_addr learned my addr 172.21.15.122:0/3292495357 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde30e00640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde2c152dd0 0x7fde2c153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 -- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde14004010 msgr2=0x7fde2c0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde14004010 0x7fde2c0b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 -- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde2c152dd0 msgr2=0x7fde2c153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde2c152dd0 0x7fde2c153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 -- 172.21.15.122:0/3292495357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c153790 con 0x7fde2c09c200 2024-09-17T05:36:20.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde32200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c09c200 0x7fde2c09c680 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fde1400bf30 tx=0x7fde24012870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:20.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde29600640 1 -- 172.21.15.122:0/3292495357 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24013860 con 0x7fde2c09c200 2024-09-17T05:36:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde29600640 1 -- 172.21.15.122:0/3292495357 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde24013a40 con 0x7fde2c09c200 2024-09-17T05:36:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2c1f0790 con 0x7fde2c09c200 2024-09-17T05:36:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde29600640 1 -- 172.21.15.122:0/3292495357 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24010480 con 0x7fde2c09c200 2024-09-17T05:36:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2c1f0d30 con 0x7fde2c09c200 2024-09-17T05:36:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.710+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2c0ceb60 con 0x7fde2c09c200 2024-09-17T05:36:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.714+0000 7fde29600640 1 -- 172.21.15.122:0/3292495357 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fde24010620 con 0x7fde2c09c200 2024-09-17T05:36:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.714+0000 7fde29600640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde0c0748b0 0x7fde0c076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:20.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.714+0000 7fde29600640 1 -- 172.21.15.122:0/3292495357 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5793+0+0 (secure 0 0 0) 0x7fde24096c10 con 0x7fde2c09c200 2024-09-17T05:36:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.714+0000 7fde31800640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde0c0748b0 0x7fde0c076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.714+0000 7fde31800640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde0c0748b0 0x7fde0c076d70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fde2c06f510 tx=0x7fde200091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:20.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.718+0000 7fde29600640 1 -- 172.21.15.122:0/3292495357 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fde24063170 con 0x7fde2c09c200 2024-09-17T05:36:20.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.926+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fde2c0d2300 con 0x7fde2c09c200 2024-09-17T05:36:20.921 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:36:20.926+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:36:20.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.926+0000 7fde29600640 1 -- 172.21.15.122:0/3292495357 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7fde24067ba0 con 0x7fde2c09c200 2024-09-17T05:36:20.923 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde0c0748b0 msgr2=0x7fde0c076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde0c0748b0 0x7fde0c076d70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fde2c06f510 tx=0x7fde200091e0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c09c200 msgr2=0x7fde2c09c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c09c200 0x7fde2c09c680 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fde1400bf30 tx=0x7fde24012870 comp rx=0 tx=0).stop 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 shutdown_connections 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fde0c0748b0 0x7fde0c076d70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c09c200 0x7fde2c09c680 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde2c152dd0 0x7fde2c153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 --2- 172.21.15.122:0/3292495357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde14004010 0x7fde2c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:20.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 >> 172.21.15.122:0/3292495357 conn(0x7fde2c0d1dc0 msgr2=0x7fde2c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:20.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 shutdown_connections 2024-09-17T05:36:20.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:20.930+0000 7fde34200640 1 -- 172.21.15.122:0/3292495357 wait complete. 2024-09-17T05:36:20.950 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:36:20.997 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:36:21.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.102+0000 7f2c35e00640 1 -- 172.21.15.122:0/2454961065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 msgr2=0x7f2c300b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:21.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.102+0000 7f2c35e00640 1 --2- 172.21.15.122:0/2454961065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 0x7f2c300b1cf0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f2c30074d30 tx=0x7f2c1c01f0b0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.102+0000 7f2c35e00640 1 -- 172.21.15.122:0/2454961065 shutdown_connections 2024-09-17T05:36:21.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.102+0000 7f2c35e00640 1 --2- 172.21.15.122:0/2454961065 >> v2:172.21.15.173:3300/0 conn(0x7f2c300d04d0 0x7f2c300af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.102+0000 7f2c35e00640 1 --2- 172.21.15.122:0/2454961065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 0x7f2c300b1cf0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.102+0000 7f2c35e00640 1 -- 172.21.15.122:0/2454961065 >> 172.21.15.122:0/2454961065 conn(0x7f2c300d1e90 msgr2=0x7f2c300d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:21.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.102+0000 7f2c35e00640 1 -- 172.21.15.122:0/2454961065 shutdown_connections 2024-09-17T05:36:21.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.102+0000 7f2c35e00640 1 -- 172.21.15.122:0/2454961065 wait complete. 2024-09-17T05:36:21.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 Processor -- start 2024-09-17T05:36:21.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 -- start start 2024-09-17T05:36:21.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c300d00f0 0x7f2c300b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:21.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c300d04d0 0x7f2c300b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:21.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 0x7f2c300b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:21.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c300b0ed0 con 0x7f2c300d29b0 2024-09-17T05:36:21.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c300b0d50 con 0x7f2c300d00f0 2024-09-17T05:36:21.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2c300af740 con 0x7f2c300d04d0 2024-09-17T05:36:21.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 0x7f2c300b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:21.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 0x7f2c300b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40136/0 (socket says 172.21.15.122:40136) 2024-09-17T05:36:21.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 -- 172.21.15.122:0/871667389 learned_addr learned my addr 172.21.15.122:0/871667389 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:21.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2e000640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c300d04d0 0x7f2c300b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:21.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2ea00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c300d00f0 0x7f2c300b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:21.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 -- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c300d04d0 msgr2=0x7f2c300b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c300d04d0 0x7f2c300b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 -- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c300d00f0 msgr2=0x7f2c300b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c300d00f0 0x7f2c300b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 -- 172.21.15.122:0/871667389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c300c6620 con 0x7f2c300d29b0 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c2f400640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 0x7f2c300b5740 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800eb50 tx=0x7f2c2800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c27400640 1 -- 172.21.15.122:0/871667389 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28004d30 con 0x7f2c300d29b0 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c27400640 1 -- 172.21.15.122:0/871667389 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c2800ce70 con 0x7f2c300d29b0 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c300c6900 con 0x7f2c300d29b0 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c27400640 1 -- 172.21.15.122:0/871667389 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28005240 con 0x7f2c300d29b0 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.106+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c300c6ec0 con 0x7f2c300d29b0 2024-09-17T05:36:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.110+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c300af200 con 0x7f2c300d29b0 2024-09-17T05:36:21.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.110+0000 7f2c27400640 1 -- 172.21.15.122:0/871667389 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f2c280053e0 con 0x7f2c300d29b0 2024-09-17T05:36:21.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.110+0000 7f2c27400640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c10074880 0x7f2c10076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:21.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.110+0000 7f2c27400640 1 -- 172.21.15.122:0/871667389 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7f2c28093b00 con 0x7f2c300d29b0 2024-09-17T05:36:21.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.110+0000 7f2c2ea00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c10074880 0x7f2c10076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:21.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.110+0000 7f2c2ea00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c10074880 0x7f2c10076d40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2c20007fd0 tx=0x7f2c20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:21.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.114+0000 7f2c27400640 1 -- 172.21.15.122:0/871667389 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2c2805ffc0 con 0x7f2c300d29b0 2024-09-17T05:36:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.318+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2c30070cf0 con 0x7f2c300d29b0 2024-09-17T05:36:21.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.930+0000 7f2c27400640 1 -- 172.21.15.122:0/871667389 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7f2c280649f0 con 0x7f2c300d29b0 2024-09-17T05:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c10074880 msgr2=0x7f2c10076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c10074880 0x7f2c10076d40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2c20007fd0 tx=0x7f2c20009040 comp rx=0 tx=0).stop 2024-09-17T05:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 msgr2=0x7f2c300b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 0x7f2c300b5740 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800eb50 tx=0x7f2c2800c6a0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 shutdown_connections 2024-09-17T05:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c10074880 0x7f2c10076d40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c300d29b0 0x7f2c300b5740 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c300d04d0 0x7f2c300b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 --2- 172.21.15.122:0/871667389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c300d00f0 0x7f2c300b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:21.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 >> 172.21.15.122:0/871667389 conn(0x7f2c300d1e90 msgr2=0x7f2c3006e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:21.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 shutdown_connections 2024-09-17T05:36:21.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:21.934+0000 7f2c35e00640 1 -- 172.21.15.122:0/871667389 wait complete. 2024-09-17T05:36:21.952 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:36:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.146+0000 7fe64e200640 1 -- 172.21.15.122:0/1004262625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 msgr2=0x7fe62c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.146+0000 7fe64e200640 1 --2- 172.21.15.122:0/1004262625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 0x7fe62c003e30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe638009b60 tx=0x7fe63801d2b0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.146+0000 7fe647e00640 1 -- 172.21.15.122:0/1004262625 reap_dead start 2024-09-17T05:36:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.146+0000 7fe64e200640 1 -- 172.21.15.122:0/1004262625 shutdown_connections 2024-09-17T05:36:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.146+0000 7fe64e200640 1 -- 172.21.15.122:0/1004262625 >> 172.21.15.122:0/1004262625 conn(0x7fe6480d1da0 msgr2=0x7fe6480d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.146+0000 7fe64e200640 1 -- 172.21.15.122:0/1004262625 shutdown_connections 2024-09-17T05:36:22.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.146+0000 7fe64e200640 1 -- 172.21.15.122:0/1004262625 wait complete. 2024-09-17T05:36:22.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 Processor -- start 2024-09-17T05:36:22.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 -- start start 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 0x7fe6480b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6480b68d0 0x7fe64815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6481f8090 0x7fe6481f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe648160620 con 0x7fe62c003a10 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe6481604a0 con 0x7fe6481f8090 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe6481607a0 con 0x7fe6480b68d0 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 0x7fe6480b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 0x7fe6480b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55036/0 (socket says 172.21.15.122:55036) 2024-09-17T05:36:22.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 -- 172.21.15.122:0/1188692806 learned_addr learned my addr 172.21.15.122:0/1188692806 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:22.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647e00640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6481f8090 0x7fe6481f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:22.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe63ea00640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6480b68d0 0x7fe64815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:22.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 -- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6480b68d0 msgr2=0x7fe64815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:22.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6480b68d0 0x7fe64815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 -- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6481f8090 msgr2=0x7fe6481f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:22.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6481f8090 0x7fe6481f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 -- 172.21.15.122:0/1188692806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6481f5c60 con 0x7fe62c003a10 2024-09-17T05:36:22.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe647400640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 0x7fe6480b2fc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe63801d0e0 tx=0x7fe6380055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:22.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe63fe00640 1 -- 172.21.15.122:0/1188692806 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b1040 con 0x7fe62c003a10 2024-09-17T05:36:22.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe63fe00640 1 -- 172.21.15.122:0/1188692806 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6380ac400 con 0x7fe62c003a10 2024-09-17T05:36:22.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe63fe00640 1 -- 172.21.15.122:0/1188692806 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638009730 con 0x7fe62c003a10 2024-09-17T05:36:22.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6480b6d60 con 0x7fe62c003a10 2024-09-17T05:36:22.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.150+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6480c6b70 con 0x7fe62c003a10 2024-09-17T05:36:22.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.154+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64815fb80 con 0x7fe62c003a10 2024-09-17T05:36:22.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.154+0000 7fe63fe00640 1 -- 172.21.15.122:0/1188692806 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fe6380bd020 con 0x7fe62c003a10 2024-09-17T05:36:22.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.154+0000 7fe63fe00640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe624074880 0x7fe624076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:22.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.154+0000 7fe63fe00640 1 -- 172.21.15.122:0/1188692806 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5851+0+0 (secure 0 0 0) 0x7fe63812d430 con 0x7fe62c003a10 2024-09-17T05:36:22.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.154+0000 7fe63ea00640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe624074880 0x7fe624076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:22.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.154+0000 7fe63ea00640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe624074880 0x7fe624076d40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe6480b2f50 tx=0x7fe6300091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:22.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.158+0000 7fe63fe00640 1 -- 172.21.15.122:0/1188692806 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe6380c4020 con 0x7fe62c003a10 2024-09-17T05:36:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.326+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe6480d2460 con 0x7fe62c003a10 2024-09-17T05:36:22.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.930+0000 7fe63fe00640 1 -- 172.21.15.122:0/1188692806 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7fe638016fc0 con 0x7fe62c003a10 2024-09-17T05:36:22.928 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:36:22.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe624074880 msgr2=0x7fe624076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe624074880 0x7fe624076d40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe6480b2f50 tx=0x7fe6300091e0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 msgr2=0x7fe6480b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 0x7fe6480b2fc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe63801d0e0 tx=0x7fe6380055d0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 shutdown_connections 2024-09-17T05:36:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe624074880 0x7fe624076d40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6481f8090 0x7fe6481f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6480b68d0 0x7fe64815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 --2- 172.21.15.122:0/1188692806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe62c003a10 0x7fe6480b2fc0 secure :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fe63801d0e0 tx=0x7fe6380055d0 comp rx=0 tx=0).stop 2024-09-17T05:36:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.934+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 >> 172.21.15.122:0/1188692806 conn(0x7fe6480d1da0 msgr2=0x7fe648070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:22.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.938+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 shutdown_connections 2024-09-17T05:36:22.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:22.938+0000 7fe64e200640 1 -- 172.21.15.122:0/1188692806 wait complete. 2024-09-17T05:36:22.959 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:36:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.110+0000 7f7ef3c00640 1 -- 172.21.15.122:0/4086925982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 msgr2=0x7f7eec06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.110+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/4086925982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 0x7f7eec06fcc0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009840 tx=0x7f7edc01c820 comp rx=0 tx=0).stop 2024-09-17T05:36:23.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.110+0000 7f7ef3c00640 1 -- 172.21.15.122:0/4086925982 shutdown_connections 2024-09-17T05:36:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.110+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/4086925982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7eec072f80 0x7f7eec15b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.110+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/4086925982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 0x7f7eec06fcc0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.110+0000 7f7ef3c00640 1 -- 172.21.15.122:0/4086925982 >> 172.21.15.122:0/4086925982 conn(0x7f7eec0cfbc0 msgr2=0x7f7eec0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.110+0000 7f7ef3c00640 1 -- 172.21.15.122:0/4086925982 shutdown_connections 2024-09-17T05:36:23.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.110+0000 7f7ef3c00640 1 -- 172.21.15.122:0/4086925982 wait complete. 2024-09-17T05:36:23.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 Processor -- start 2024-09-17T05:36:23.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 -- start start 2024-09-17T05:36:23.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 0x7f7eec1f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:23.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7eec072f80 0x7f7eec159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7eec0cf2b0 0x7f7eec1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec1681a0 con 0x7f7eec070960 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec168020 con 0x7f7eec072f80 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7eec168320 con 0x7f7eec0cf2b0 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 0x7f7eec1f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 0x7f7eec1f89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55062/0 (socket says 172.21.15.122:55062) 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 -- 172.21.15.122:0/2372333841 learned_addr learned my addr 172.21.15.122:0/2372333841 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ee3e00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7eec072f80 0x7f7eec159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1c00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7eec0cf2b0 0x7f7eec1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 -- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7eec0cf2b0 msgr2=0x7f7eec1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:23.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7eec0cf2b0 0x7f7eec1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 -- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7eec072f80 msgr2=0x7f7eec159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7eec072f80 0x7f7eec159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 -- 172.21.15.122:0/2372333841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec0b3230 con 0x7f7eec070960 2024-09-17T05:36:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ee3e00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7eec072f80 0x7f7eec159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:23.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef1200640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 0x7f7eec1f89a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009790 tx=0x7f7edc005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:23.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ee9600640 1 -- 172.21.15.122:0/2372333841 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0b2040 con 0x7f7eec070960 2024-09-17T05:36:23.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ee9600640 1 -- 172.21.15.122:0/2372333841 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7edc004330 con 0x7f7eec070960 2024-09-17T05:36:23.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ee9600640 1 -- 172.21.15.122:0/2372333841 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc005220 con 0x7f7eec070960 2024-09-17T05:36:23.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.114+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec0b3510 con 0x7f7eec070960 2024-09-17T05:36:23.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.118+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eec0b3ad0 con 0x7f7eec070960 2024-09-17T05:36:23.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.118+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec069410 con 0x7f7eec070960 2024-09-17T05:36:23.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.118+0000 7f7ee9600640 1 -- 172.21.15.122:0/2372333841 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f7edc0a5750 con 0x7f7eec070960 2024-09-17T05:36:23.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.118+0000 7f7ee9600640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec8074880 0x7f7ec8076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:23.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.118+0000 7f7ee9600640 1 -- 172.21.15.122:0/2372333841 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5413+0+0 (secure 0 0 0) 0x7f7edc12bed0 con 0x7f7eec070960 2024-09-17T05:36:23.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.122+0000 7f7ee3e00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec8074880 0x7f7ec8076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:23.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.122+0000 7f7ee3e00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec8074880 0x7f7ec8076d40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0afc60 tx=0x7f7ed4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:23.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.122+0000 7f7ee9600640 1 -- 172.21.15.122:0/2372333841 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7edc016410 con 0x7f7eec070960 2024-09-17T05:36:23.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.290+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7eec06fcc0 con 0x7f7eec070960 2024-09-17T05:36:23.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.934+0000 7f7ee9600640 1 -- 172.21.15.122:0/2372333841 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f7edc0f89e0 con 0x7f7eec070960 2024-09-17T05:36:23.929 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:36:23.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.938+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec8074880 msgr2=0x7f7ec8076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:23.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.938+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec8074880 0x7f7ec8076d40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0afc60 tx=0x7f7ed4009040 comp rx=0 tx=0).stop 2024-09-17T05:36:23.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.938+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 msgr2=0x7f7eec1f89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:23.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.938+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 0x7f7eec1f89a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009790 tx=0x7f7edc005db0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.942+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 shutdown_connections 2024-09-17T05:36:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.942+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7ec8074880 0x7f7ec8076d40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.942+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7eec0cf2b0 0x7f7eec1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.942+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7eec072f80 0x7f7eec159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.942+0000 7f7ef3c00640 1 --2- 172.21.15.122:0/2372333841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eec070960 0x7f7eec1f89a0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.942+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 >> 172.21.15.122:0/2372333841 conn(0x7f7eec0cfbc0 msgr2=0x7f7eec159820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.942+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 shutdown_connections 2024-09-17T05:36:23.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:23.942+0000 7f7ef3c00640 1 -- 172.21.15.122:0/2372333841 wait complete. 2024-09-17T05:36:23.964 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:36:24.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.142+0000 7fc178400640 1 -- 172.21.15.122:0/1431391330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 msgr2=0x7fc158004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.142+0000 7fc178400640 1 --2- 172.21.15.122:0/1431391330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 0x7fc158004c90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00c470 tx=0x7fc15c01cea0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.142+0000 7fc176400640 1 -- 172.21.15.122:0/1431391330 reap_dead start 2024-09-17T05:36:24.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.142+0000 7fc178400640 1 -- 172.21.15.122:0/1431391330 shutdown_connections 2024-09-17T05:36:24.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.142+0000 7fc178400640 1 -- 172.21.15.122:0/1431391330 >> 172.21.15.122:0/1431391330 conn(0x7fc1700d1da0 msgr2=0x7fc1700d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:24.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.142+0000 7fc178400640 1 -- 172.21.15.122:0/1431391330 shutdown_connections 2024-09-17T05:36:24.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.142+0000 7fc178400640 1 -- 172.21.15.122:0/1431391330 wait complete. 2024-09-17T05:36:24.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 Processor -- start 2024-09-17T05:36:24.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 -- start start 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 0x7fc170156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17015a2d0 0x7fc17009f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17015a710 0x7fc170158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1700ccfc0 con 0x7fc158004810 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1700d2b80 con 0x7fc17015a2d0 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc1700cd140 con 0x7fc17015a710 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 0x7fc170156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 0x7fc170156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55084/0 (socket says 172.21.15.122:55084) 2024-09-17T05:36:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 -- 172.21.15.122:0/577582783 learned_addr learned my addr 172.21.15.122:0/577582783 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175000640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17015a2d0 0x7fc17009f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 -- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17015a710 msgr2=0x7fc170158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17015a710 0x7fc170158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 -- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17015a2d0 msgr2=0x7fc17009f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17015a2d0 0x7fc17009f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 -- 172.21.15.122:0/577582783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1700c8830 con 0x7fc158004810 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175000640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17015a2d0 0x7fc17009f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc175a00640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 0x7fc170156550 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc1640115d0 tx=0x7fc164011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc16d600640 1 -- 172.21.15.122:0/577582783 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164004020 con 0x7fc158004810 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc16d600640 1 -- 172.21.15.122:0/577582783 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1640041e0 con 0x7fc158004810 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1701f2d60 con 0x7fc158004810 2024-09-17T05:36:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc16d600640 1 -- 172.21.15.122:0/577582783 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164013720 con 0x7fc158004810 2024-09-17T05:36:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.146+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1701f3320 con 0x7fc158004810 2024-09-17T05:36:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.150+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1700cc8b0 con 0x7fc158004810 2024-09-17T05:36:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.150+0000 7fc16d600640 1 -- 172.21.15.122:0/577582783 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fc1640138c0 con 0x7fc158004810 2024-09-17T05:36:24.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.150+0000 7fc16d600640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc1500748b0 0x7fc150076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:24.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.150+0000 7fc16d600640 1 -- 172.21.15.122:0/577582783 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7fc164096570 con 0x7fc158004810 2024-09-17T05:36:24.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.150+0000 7fc175000640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc1500748b0 0x7fc150076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:24.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.150+0000 7fc175000640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc1500748b0 0x7fc150076d70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0ac000 tx=0x7fc15c019ce0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:24.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.154+0000 7fc16d600640 1 -- 172.21.15.122:0/577582783 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc164017070 con 0x7fc158004810 2024-09-17T05:36:24.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.322+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fc1700b2f50 con 0x7fc158004810 2024-09-17T05:36:24.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.326+0000 7fc16d600640 1 -- 172.21.15.122:0/577582783 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7fc164063080 con 0x7fc158004810 2024-09-17T05:36:24.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc1500748b0 msgr2=0x7fc150076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc1500748b0 0x7fc150076d70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0ac000 tx=0x7fc15c019ce0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 msgr2=0x7fc170156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 0x7fc170156550 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc1640115d0 tx=0x7fc164011a90 comp rx=0 tx=0).stop 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 shutdown_connections 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc1500748b0 0x7fc150076d70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17015a710 0x7fc170158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17015a2d0 0x7fc17009f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 --2- 172.21.15.122:0/577582783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc158004810 0x7fc170156550 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 >> 172.21.15.122:0/577582783 conn(0x7fc1700d1da0 msgr2=0x7fc170071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 shutdown_connections 2024-09-17T05:36:24.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.330+0000 7fc178400640 1 -- 172.21.15.122:0/577582783 wait complete. 2024-09-17T05:36:24.346 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-17T05:36:24.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- 172.21.15.122:0/3408539456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93e4004d50 msgr2=0x7f93e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 --2- 172.21.15.122:0/3408539456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93e4004d50 0x7f93e40051d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f93f800c640 tx=0x7f93f801fa60 comp rx=0 tx=0).stop 2024-09-17T05:36:24.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ffe00640 1 -- 172.21.15.122:0/3408539456 reap_dead start 2024-09-17T05:36:24.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- 172.21.15.122:0/3408539456 shutdown_connections 2024-09-17T05:36:24.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- 172.21.15.122:0/3408539456 >> 172.21.15.122:0/3408539456 conn(0x7f94000b18f0 msgr2=0x7f94000b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:24.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- 172.21.15.122:0/3408539456 shutdown_connections 2024-09-17T05:36:24.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- 172.21.15.122:0/3408539456 wait complete. 2024-09-17T05:36:24.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 Processor -- start 2024-09-17T05:36:24.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- start start 2024-09-17T05:36:24.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4004d50 0x7f94001f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9400155b70 0x7f940009d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f940009aa10 0x7f94000af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94000cef80 con 0x7f940009aa10 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f94000cee00 con 0x7f9400155b70 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f9406400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f94000b2bf0 con 0x7f93e4004d50 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ffe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f940009aa10 0x7f94000af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ff400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4004d50 0x7f94001f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ff400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4004d50 0x7f94001f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:57908/0 (socket says 172.21.15.122:57908) 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ff400640 1 -- 172.21.15.122:0/1762054798 learned_addr learned my addr 172.21.15.122:0/1762054798 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93f6a00640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9400155b70 0x7f940009d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ff400640 1 -- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9400155b70 msgr2=0x7f940009d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ff400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9400155b70 0x7f940009d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ff400640 1 -- 172.21.15.122:0/1762054798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f940009aa10 msgr2=0x7f94000af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ff400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f940009aa10 0x7f94000af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ff400640 1 -- 172.21.15.122:0/1762054798 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94001f2c10 con 0x7f93e4004d50 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.494+0000 7f93ffe00640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f940009aa10 0x7f94000af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f93ff400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4004d50 0x7f94001f2630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f93f0011ae0 tx=0x7f93f0011ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f93f7e00640 1 -- 172.21.15.122:0/1762054798 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0004460 con 0x7f93e4004d50 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f93f7e00640 1 -- 172.21.15.122:0/1762054798 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93f000fa00 con 0x7f93e4004d50 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94001f2ef0 con 0x7f93e4004d50 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f93f7e00640 1 -- 172.21.15.122:0/1762054798 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f000fe30 con 0x7f93e4004d50 2024-09-17T05:36:24.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94001f34b0 con 0x7f93e4004d50 2024-09-17T05:36:24.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94000b3130 con 0x7f93e4004d50 2024-09-17T05:36:24.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f93f7e00640 1 -- 172.21.15.122:0/1762054798 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f93f0003600 con 0x7f93e4004d50 2024-09-17T05:36:24.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f93f7e00640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93dc074880 0x7f93dc076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:24.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f93f7e00640 1 -- 172.21.15.122:0/1762054798 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7f93f0093eb0 con 0x7f93e4004d50 2024-09-17T05:36:24.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.498+0000 7f93f6a00640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93dc074880 0x7f93dc076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:24.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.502+0000 7f93f6a00640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93dc074880 0x7f93dc076d40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f94000b0690 tx=0x7f93e80091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:24.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.502+0000 7f93f7e00640 1 -- 172.21.15.122:0/1762054798 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f93f00603b0 con 0x7f93e4004d50 2024-09-17T05:36:24.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.730+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f940006ff10 con 0x7f93e4004d50 2024-09-17T05:36:24.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.946+0000 7f93f7e00640 1 -- 172.21.15.122:0/1762054798 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v515) ==== 174+0+0 (secure 0 0 0) 0x7f93f0064de0 con 0x7f93e4004d50 2024-09-17T05:36:24.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93dc074880 msgr2=0x7f93dc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93dc074880 0x7f93dc076d40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f94000b0690 tx=0x7f93e80091e0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4004d50 msgr2=0x7f94001f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4004d50 0x7f94001f2630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f93f0011ae0 tx=0x7f93f0011ee0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 shutdown_connections 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93dc074880 0x7f93dc076d40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f940009aa10 0x7f94000af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9400155b70 0x7f940009d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 --2- 172.21.15.122:0/1762054798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4004d50 0x7f94001f2630 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 >> 172.21.15.122:0/1762054798 conn(0x7f94000b18f0 msgr2=0x7f94001586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 shutdown_connections 2024-09-17T05:36:24.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:24.950+0000 7f9406400640 1 -- 172.21.15.122:0/1762054798 wait complete. 2024-09-17T05:36:24.964 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-09-17T05:36:24.964 INFO:tasks.cephfs_test_runner:Starting test: test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-09-17T05:36:24.965 INFO:tasks.cephfs_test_runner:test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:36:24.965 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:36:25.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.110+0000 7f290d000640 1 -- 172.21.15.122:0/3788375176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 msgr2=0x7f28ec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.110+0000 7f290d000640 1 --2- 172.21.15.122:0/3788375176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 0x7f28ec00b9d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2900001830 tx=0x7f290001c8a0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.110+0000 7f2906800640 1 -- 172.21.15.122:0/3788375176 reap_dead start 2024-09-17T05:36:25.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.110+0000 7f290d000640 1 -- 172.21.15.122:0/3788375176 shutdown_connections 2024-09-17T05:36:25.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.110+0000 7f290d000640 1 -- 172.21.15.122:0/3788375176 >> 172.21.15.122:0/3788375176 conn(0x7f29080d1dc0 msgr2=0x7f29080d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:25.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.110+0000 7f290d000640 1 -- 172.21.15.122:0/3788375176 shutdown_connections 2024-09-17T05:36:25.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.110+0000 7f290d000640 1 -- 172.21.15.122:0/3788375176 wait complete. 2024-09-17T05:36:25.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 Processor -- start 2024-09-17T05:36:25.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 -- start start 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 0x7f29080c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f290815aa00 0x7f290815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29080c7eb0 0x7f29080c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29080d2af0 con 0x7f28ec0051d0 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29080ae9b0 con 0x7f290815aa00 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f29080c9f60 con 0x7f29080c7eb0 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 0x7f29080c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 0x7f29080c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55126/0 (socket says 172.21.15.122:55126) 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 -- 172.21.15.122:0/837264893 learned_addr learned my addr 172.21.15.122:0/837264893 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2906800640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29080c7eb0 0x7f29080c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905400640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f290815aa00 0x7f290815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 -- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29080c7eb0 msgr2=0x7f29080c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29080c7eb0 0x7f29080c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 -- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f290815aa00 msgr2=0x7f290815ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f290815aa00 0x7f290815ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 -- 172.21.15.122:0/837264893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29080c9ce0 con 0x7f28ec0051d0 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f2905e00640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 0x7f29080c97a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f28f8012610 tx=0x7f28f8012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f28fe000640 1 -- 172.21.15.122:0/837264893 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8013920 con 0x7f28ec0051d0 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f28fe000640 1 -- 172.21.15.122:0/837264893 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28f8013ae0 con 0x7f28ec0051d0 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f28fe000640 1 -- 172.21.15.122:0/837264893 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8010480 con 0x7f28ec0051d0 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29081f8b20 con 0x7f28ec0051d0 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.114+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29081f8060 con 0x7f28ec0051d0 2024-09-17T05:36:25.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.118+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29080ce670 con 0x7f28ec0051d0 2024-09-17T05:36:25.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.118+0000 7f28fe000640 1 -- 172.21.15.122:0/837264893 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f28f8010620 con 0x7f28ec0051d0 2024-09-17T05:36:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.118+0000 7f28fe000640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f28e4074880 0x7f28e4076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.118+0000 7f28fe000640 1 -- 172.21.15.122:0/837264893 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7f28f8096e40 con 0x7f28ec0051d0 2024-09-17T05:36:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.118+0000 7f2905400640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f28e4074880 0x7f28e4076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.118+0000 7f2905400640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f28e4074880 0x7f28e4076d40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f290815fa90 tx=0x7f28f00091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:25.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.122+0000 7f28fe000640 1 -- 172.21.15.122:0/837264893 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f29080c9ce0 con 0x7f28ec0051d0 2024-09-17T05:36:25.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.290+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29080cc1e0 con 0x7f28ec0051d0 2024-09-17T05:36:25.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.290+0000 7f28fe000640 1 -- 172.21.15.122:0/837264893 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+13182 (secure 0 0 0) 0x7f28f8063300 con 0x7f28ec0051d0 2024-09-17T05:36:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:25.287 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":95,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:36:23.934806+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6834/3516694104":"2024-09-18T05:36:20.929072+0000","172.21.15.122:6832/3516694104":"2024-09-18T05:36:20.929072+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:36:25.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f28e4074880 msgr2=0x7f28e4076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f28e4074880 0x7f28e4076d40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f290815fa90 tx=0x7f28f00091e0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 msgr2=0x7f29080c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 0x7f29080c97a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f28f8012610 tx=0x7f28f8012a10 comp rx=0 tx=0).stop 2024-09-17T05:36:25.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 shutdown_connections 2024-09-17T05:36:25.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f28e4074880 0x7f28e4076d40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29080c7eb0 0x7f29080c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f290815aa00 0x7f290815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 --2- 172.21.15.122:0/837264893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28ec0051d0 0x7f29080c97a0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 >> 172.21.15.122:0/837264893 conn(0x7f29080d1dc0 msgr2=0x7f29080b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:25.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 shutdown_connections 2024-09-17T05:36:25.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.294+0000 7f290d000640 1 -- 172.21.15.122:0/837264893 wait complete. 2024-09-17T05:36:25.309 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:36:25.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:36:25.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 -- 172.21.15.122:0/3473265588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c15bd40 msgr2=0x7f8f5c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 --2- 172.21.15.122:0/3473265588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c15bd40 0x7f8f5c0d0240 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c073190 tx=0x7f8f5001c800 comp rx=0 tx=0).stop 2024-09-17T05:36:25.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 -- 172.21.15.122:0/3473265588 shutdown_connections 2024-09-17T05:36:25.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 --2- 172.21.15.122:0/3473265588 >> v2:172.21.15.173:3300/0 conn(0x7f8f5c0b0d10 0x7f8f5c0d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 --2- 172.21.15.122:0/3473265588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c15bd40 0x7f8f5c0d0240 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 -- 172.21.15.122:0/3473265588 >> 172.21.15.122:0/3473265588 conn(0x7f8f5c09aa10 msgr2=0x7f8f5c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:25.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 -- 172.21.15.122:0/3473265588 shutdown_connections 2024-09-17T05:36:25.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 -- 172.21.15.122:0/3473265588 wait complete. 2024-09-17T05:36:25.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 Processor -- start 2024-09-17T05:36:25.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.458+0000 7f8f62a00640 1 -- start start 2024-09-17T05:36:25.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f5c0b0d10 0x7f8f5c166130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c0d20f0 0x7f8f5c0b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5c15bd40 0x7f8f5c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c168db0 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c168c30 con 0x7f8f5c15bd40 2024-09-17T05:36:25.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5c168f30 con 0x7f8f5c0b0d10 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c0d20f0 0x7f8f5c0b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c0d20f0 0x7f8f5c0b7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55142/0 (socket says 172.21.15.122:55142) 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 -- 172.21.15.122:0/1432887514 learned_addr learned my addr 172.21.15.122:0/1432887514 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f61600640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f5c0b0d10 0x7f8f5c166130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62000640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5c15bd40 0x7f8f5c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 -- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f5c0b0d10 msgr2=0x7f8f5c166130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f5c0b0d10 0x7f8f5c166130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 -- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5c15bd40 msgr2=0x7f8f5c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5c15bd40 0x7f8f5c151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 -- 172.21.15.122:0/1432887514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c0cf810 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62000640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5c15bd40 0x7f8f5c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:25.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f60c00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c0d20f0 0x7f8f5c0b7640 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800d950 tx=0x7f8f4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:25.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f59600640 1 -- 172.21.15.122:0/1432887514 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48017070 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f59600640 1 -- 172.21.15.122:0/1432887514 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f48009d20 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5c1fb0e0 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f59600640 1 -- 172.21.15.122:0/1432887514 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f480106c0 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5c1fb6a0 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.462+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f5c0d1bb0 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.466+0000 7f8f59600640 1 -- 172.21.15.122:0/1432887514 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f8f48010860 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.466+0000 7f8f59600640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f3c074880 0x7f8f3c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.466+0000 7f8f59600640 1 -- 172.21.15.122:0/1432887514 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7f8f480932b0 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.466+0000 7f8f61600640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f3c074880 0x7f8f3c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.466+0000 7f8f61600640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f3c074880 0x7f8f3c076d40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8f500af000 tx=0x7f8f5001b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:25.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.470+0000 7f8f59600640 1 -- 172.21.15.122:0/1432887514 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8f4805f770 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.634+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f5c0b06e0 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.634+0000 7f8f59600640 1 -- 172.21.15.122:0/1432887514 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f8f480641a0 con 0x7f8f5c0d20f0 2024-09-17T05:36:25.632 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:36:25.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f3c074880 msgr2=0x7f8f3c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f3c074880 0x7f8f3c076d40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8f500af000 tx=0x7f8f5001b040 comp rx=0 tx=0).stop 2024-09-17T05:36:25.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c0d20f0 msgr2=0x7f8f5c0b7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c0d20f0 0x7f8f5c0b7640 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800d950 tx=0x7f8f4800de10 comp rx=0 tx=0).stop 2024-09-17T05:36:25.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 shutdown_connections 2024-09-17T05:36:25.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8f3c074880 0x7f8f3c076d40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f5c15bd40 0x7f8f5c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f5c0d20f0 0x7f8f5c0b7640 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 --2- 172.21.15.122:0/1432887514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f5c0b0d10 0x7f8f5c166130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 >> 172.21.15.122:0/1432887514 conn(0x7f8f5c09aa10 msgr2=0x7f8f5c0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:25.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 shutdown_connections 2024-09-17T05:36:25.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.638+0000 7f8f62a00640 1 -- 172.21.15.122:0/1432887514 wait complete. 2024-09-17T05:36:25.653 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-17T05:36:25.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.798+0000 7fe22d600640 1 -- 172.21.15.122:0/2942953275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe22806e810 msgr2=0x7fe22806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.798+0000 7fe22d600640 1 --2- 172.21.15.122:0/2942953275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe22806e810 0x7fe22806f1f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe228070d80 tx=0x7fe22001cd50 comp rx=0 tx=0).stop 2024-09-17T05:36:25.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.798+0000 7fe22d600640 1 -- 172.21.15.122:0/2942953275 shutdown_connections 2024-09-17T05:36:25.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.798+0000 7fe22d600640 1 --2- 172.21.15.122:0/2942953275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe22806e810 0x7fe22806f1f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.798+0000 7fe22d600640 1 --2- 172.21.15.122:0/2942953275 >> v2:172.21.15.173:3300/0 conn(0x7fe2280cf300 0x7fe22806ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.798+0000 7fe22d600640 1 -- 172.21.15.122:0/2942953275 >> 172.21.15.122:0/2942953275 conn(0x7fe2280b18f0 msgr2=0x7fe2280b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:25.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.798+0000 7fe22d600640 1 -- 172.21.15.122:0/2942953275 shutdown_connections 2024-09-17T05:36:25.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.798+0000 7fe22d600640 1 -- 172.21.15.122:0/2942953275 wait complete. 2024-09-17T05:36:25.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 Processor -- start 2024-09-17T05:36:25.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 -- start start 2024-09-17T05:36:25.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe22806e810 0x7fe22814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2280cf300 0x7fe228153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe22815ade0 0x7fe2280add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe228168840 con 0x7fe2280cf300 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe2281686c0 con 0x7fe22806e810 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe2281689c0 con 0x7fe22815ade0 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2280cf300 0x7fe228153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2280cf300 0x7fe228153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55158/0 (socket says 172.21.15.122:55158) 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 -- 172.21.15.122:0/584257230 learned_addr learned my addr 172.21.15.122:0/584257230 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe226a00640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe22806e810 0x7fe22814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe227400640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe22815ade0 0x7fe2280add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 -- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe22815ade0 msgr2=0x7fe2280add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe22815ade0 0x7fe2280add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 -- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe22806e810 msgr2=0x7fe22814d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:25.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe22806e810 0x7fe22814d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 -- 172.21.15.122:0/584257230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22814dca0 con 0x7fe2280cf300 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe227400640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe22815ade0 0x7fe2280add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe226a00640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe22806e810 0x7fe22814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21e000640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2280cf300 0x7fe228153270 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe21000c970 tx=0x7fe21000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21f400640 1 -- 172.21.15.122:0/584257230 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210007c70 con 0x7fe2280cf300 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21f400640 1 -- 172.21.15.122:0/584257230 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe21000f450 con 0x7fe2280cf300 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22814d840 con 0x7fe2280cf300 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe21f400640 1 -- 172.21.15.122:0/584257230 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000f8a0 con 0x7fe2280cf300 2024-09-17T05:36:25.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe228072b50 con 0x7fe2280cf300 2024-09-17T05:36:25.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.802+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2280cebf0 con 0x7fe2280cf300 2024-09-17T05:36:25.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.806+0000 7fe21f400640 1 -- 172.21.15.122:0/584257230 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fe21000fa40 con 0x7fe2280cf300 2024-09-17T05:36:25.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.806+0000 7fe21f400640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe2040748b0 0x7fe204076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:25.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.806+0000 7fe21f400640 1 -- 172.21.15.122:0/584257230 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7fe210093830 con 0x7fe2280cf300 2024-09-17T05:36:25.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.806+0000 7fe226a00640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe2040748b0 0x7fe204076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:25.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.810+0000 7fe226a00640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe2040748b0 0x7fe204076d70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe2180098b0 tx=0x7fe218008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:25.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:25.810+0000 7fe21f400640 1 -- 172.21.15.122:0/584257230 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe22814dca0 con 0x7fe2280cf300 2024-09-17T05:36:26.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.042+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7fe2280af7d0 con 0x7fe2280cf300 2024-09-17T05:36:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.946+0000 7fe21f400640 1 -- 172.21.15.122:0/584257230 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v517) ==== 181+0+0 (secure 0 0 0) 0x7fe210014030 con 0x7fe2280cf300 2024-09-17T05:36:26.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe2040748b0 msgr2=0x7fe204076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe2040748b0 0x7fe204076d70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe2180098b0 tx=0x7fe218008040 comp rx=0 tx=0).stop 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2280cf300 msgr2=0x7fe228153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2280cf300 0x7fe228153270 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe21000c970 tx=0x7fe21000ce30 comp rx=0 tx=0).stop 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 shutdown_connections 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe2040748b0 0x7fe204076d70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe22815ade0 0x7fe2280add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2280cf300 0x7fe228153270 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 --2- 172.21.15.122:0/584257230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe22806e810 0x7fe22814d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 >> 172.21.15.122:0/584257230 conn(0x7fe2280b18f0 msgr2=0x7fe2280b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 shutdown_connections 2024-09-17T05:36:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:26.950+0000 7fe22d600640 1 -- 172.21.15.122:0/584257230 wait complete. 2024-09-17T05:36:26.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:36:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 -- 172.21.15.122:0/1054640952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75780cdb30 msgr2=0x7f75780cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 --2- 172.21.15.122:0/1054640952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75780cdb30 0x7f75780cebb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7560009840 tx=0x7f756001c820 comp rx=0 tx=0).stop 2024-09-17T05:36:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 -- 172.21.15.122:0/1054640952 shutdown_connections 2024-09-17T05:36:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 --2- 172.21.15.122:0/1054640952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75780cdb30 0x7f75780cebb0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 --2- 172.21.15.122:0/1054640952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f757815bea0 0x7f75780ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 -- 172.21.15.122:0/1054640952 >> 172.21.15.122:0/1054640952 conn(0x7f75780d1dc0 msgr2=0x7f75780d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:27.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 -- 172.21.15.122:0/1054640952 shutdown_connections 2024-09-17T05:36:27.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 -- 172.21.15.122:0/1054640952 wait complete. 2024-09-17T05:36:27.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 Processor -- start 2024-09-17T05:36:27.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.118+0000 7f757ce00640 1 -- start start 2024-09-17T05:36:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75780cdb30 0x7f75780b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75780d28e0 0x7f757814bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f757815bea0 0x7f75780ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7578168fc0 con 0x7f757815bea0 2024-09-17T05:36:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7578168e40 con 0x7f75780cdb30 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7578169140 con 0x7f75780d28e0 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f757815bea0 0x7f75780ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f757815bea0 0x7f75780ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55180/0 (socket says 172.21.15.122:55180) 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 -- 172.21.15.122:0/1723150851 learned_addr learned my addr 172.21.15.122:0/1723150851 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f756fe00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75780d28e0 0x7f757814bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7575e00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75780cdb30 0x7f75780b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 -- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75780d28e0 msgr2=0x7f757814bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75780d28e0 0x7f757814bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 -- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75780cdb30 msgr2=0x7f75780b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75780cdb30 0x7f75780b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 -- 172.21.15.122:0/1723150851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75781fc020 con 0x7f757815bea0 2024-09-17T05:36:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f756fe00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75780d28e0 0x7f757814bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f7576800640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f757815bea0 0x7f75780ca010 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f757000c770 tx=0x7f757000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f756e000640 1 -- 172.21.15.122:0/1723150851 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570017070 con 0x7f757815bea0 2024-09-17T05:36:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f756e000640 1 -- 172.21.15.122:0/1723150851 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7570005070 con 0x7f757815bea0 2024-09-17T05:36:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75780b3b50 con 0x7f757815bea0 2024-09-17T05:36:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f756e000640 1 -- 172.21.15.122:0/1723150851 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700054c0 con 0x7f757815bea0 2024-09-17T05:36:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781fb540 con 0x7f757815bea0 2024-09-17T05:36:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75780cc9d0 con 0x7f757815bea0 2024-09-17T05:36:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.122+0000 7f756e000640 1 -- 172.21.15.122:0/1723150851 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f757001a070 con 0x7f757815bea0 2024-09-17T05:36:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.126+0000 7f756e000640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7554074880 0x7f7554076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.126+0000 7f756e000640 1 -- 172.21.15.122:0/1723150851 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7f7570092690 con 0x7f757815bea0 2024-09-17T05:36:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.126+0000 7f7575e00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7554074880 0x7f7554076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.126+0000 7f7575e00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7554074880 0x7f7554076d40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f75680058d0 tx=0x7f7568005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:27.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.130+0000 7f756e000640 1 -- 172.21.15.122:0/1723150851 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f757005f1a0 con 0x7f757815bea0 2024-09-17T05:36:27.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f757815a980 con 0x7f757815bea0 2024-09-17T05:36:27.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f756e000640 1 -- 172.21.15.122:0/1723150851 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+1998 (secure 0 0 0) 0x7f757005eaa0 con 0x7f757815bea0 2024-09-17T05:36:27.297 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:36:27.297 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:36:27.297 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:36:27.297 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:36:27.297 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:36:27.297 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:36:27.297 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:36:27.298 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:36:27.299 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:36:27.299 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:36:27.299 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:36:27.299 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:36:27.299 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:36:27.299 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:36:27.299 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:36:27.299 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:36:27.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7554074880 msgr2=0x7f7554076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7554074880 0x7f7554076d40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f75680058d0 tx=0x7f7568005860 comp rx=0 tx=0).stop 2024-09-17T05:36:27.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f757815bea0 msgr2=0x7f75780ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f757815bea0 0x7f75780ca010 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f757000c770 tx=0x7f757000cc30 comp rx=0 tx=0).stop 2024-09-17T05:36:27.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 shutdown_connections 2024-09-17T05:36:27.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7554074880 0x7f7554076d40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f757815bea0 0x7f75780ca010 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75780d28e0 0x7f757814bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 --2- 172.21.15.122:0/1723150851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75780cdb30 0x7f75780b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 >> 172.21.15.122:0/1723150851 conn(0x7f75780d1dc0 msgr2=0x7f7578165f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:27.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 shutdown_connections 2024-09-17T05:36:27.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.302+0000 7f757ce00640 1 -- 172.21.15.122:0/1723150851 wait complete. 2024-09-17T05:36:27.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:36:27.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- 172.21.15.122:0/1117272427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa86c004810 msgr2=0x7fa86c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 --2- 172.21.15.122:0/1117272427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa86c004810 0x7fa86c004c90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa870006f30 tx=0x7fa87001ce40 comp rx=0 tx=0).stop 2024-09-17T05:36:27.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- 172.21.15.122:0/1117272427 shutdown_connections 2024-09-17T05:36:27.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 --2- 172.21.15.122:0/1117272427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa86c004810 0x7fa86c004c90 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- 172.21.15.122:0/1117272427 >> 172.21.15.122:0/1117272427 conn(0x7fa8880d1dd0 msgr2=0x7fa8880d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:27.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- 172.21.15.122:0/1117272427 shutdown_connections 2024-09-17T05:36:27.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- 172.21.15.122:0/1117272427 wait complete. 2024-09-17T05:36:27.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 Processor -- start 2024-09-17T05:36:27.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- start start 2024-09-17T05:36:27.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa86c004810 0x7fa88806ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:27.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8881f8240 0x7fa8881f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:27.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8881f86c0 0x7fa8881f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:27.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880b0ee0 con 0x7fa8881f86c0 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880b2cf0 con 0x7fa86c004810 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa88dc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa8880b1220 con 0x7fa8881f8240 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8881f86c0 0x7fa8881f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8881f86c0 0x7fa8881f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55184/0 (socket says 172.21.15.122:55184) 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 -- 172.21.15.122:0/1298126453 learned_addr learned my addr 172.21.15.122:0/1298126453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa886a00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa86c004810 0x7fa88806ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa87fe00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8881f8240 0x7fa8881f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 -- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8881f8240 msgr2=0x7fa8881f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8881f8240 0x7fa8881f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 -- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa86c004810 msgr2=0x7fa88806ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa86c004810 0x7fa88806ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 -- 172.21.15.122:0/1298126453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88815eb20 con 0x7fa8881f86c0 2024-09-17T05:36:27.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.478+0000 7fa887400640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8881f86c0 0x7fa8881f4f30 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa8880744b0 tx=0x7fa88000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:27.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa87ea00640 1 -- 172.21.15.122:0/1298126453 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880017070 con 0x7fa8881f86c0 2024-09-17T05:36:27.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa87ea00640 1 -- 172.21.15.122:0/1298126453 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa880015070 con 0x7fa8881f86c0 2024-09-17T05:36:27.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa87ea00640 1 -- 172.21.15.122:0/1298126453 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880003be0 con 0x7fa8881f86c0 2024-09-17T05:36:27.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88815ee00 con 0x7fa8881f86c0 2024-09-17T05:36:27.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88815f3c0 con 0x7fa8881f86c0 2024-09-17T05:36:27.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8880b1bf0 con 0x7fa8881f86c0 2024-09-17T05:36:27.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa87ea00640 1 -- 172.21.15.122:0/1298126453 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fa880004030 con 0x7fa8881f86c0 2024-09-17T05:36:27.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa87ea00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa864074880 0x7fa864076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:27.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa87ea00640 1 -- 172.21.15.122:0/1298126453 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7fa8800934c0 con 0x7fa8881f86c0 2024-09-17T05:36:27.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.482+0000 7fa886a00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa864074880 0x7fa864076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:27.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.486+0000 7fa886a00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa864074880 0x7fa864076d40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa878006fd0 tx=0x7fa878006b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:27.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.486+0000 7fa87ea00640 1 -- 172.21.15.122:0/1298126453 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa88005fa20 con 0x7fa8881f86c0 2024-09-17T05:36:27.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.654+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa8880d1080 con 0x7fa8881f86c0 2024-09-17T05:36:27.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.658+0000 7fa87ea00640 1 -- 172.21.15.122:0/1298126453 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fa880064450 con 0x7fa8881f86c0 2024-09-17T05:36:27.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa864074880 msgr2=0x7fa864076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa864074880 0x7fa864076d40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa878006fd0 tx=0x7fa878006b20 comp rx=0 tx=0).stop 2024-09-17T05:36:27.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8881f86c0 msgr2=0x7fa8881f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:27.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8881f86c0 0x7fa8881f4f30 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa8880744b0 tx=0x7fa88000e720 comp rx=0 tx=0).stop 2024-09-17T05:36:27.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 shutdown_connections 2024-09-17T05:36:27.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa864074880 0x7fa864076d40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8881f86c0 0x7fa8881f4f30 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8881f8240 0x7fa8881f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 --2- 172.21.15.122:0/1298126453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa86c004810 0x7fa88806ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:27.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 >> 172.21.15.122:0/1298126453 conn(0x7fa8880d1dd0 msgr2=0x7fa8880d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:27.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 shutdown_connections 2024-09-17T05:36:27.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:27.662+0000 7fa88dc00640 1 -- 172.21.15.122:0/1298126453 wait complete. 2024-09-17T05:36:27.677 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:36:27.677 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:36:27.759 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:36:27.842 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:36:27.842 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:36:27.842 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:27.851 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:36:27.851 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:36:27.851 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:27.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:36:27.860 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:36:27.860 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:27.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:36:27.899 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:36:27.899 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:27.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:36:27.912 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:36:27.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:36:28.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- 172.21.15.122:0/2106731024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 msgr2=0x7fdca40cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 --2- 172.21.15.122:0/2106731024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 0x7fdca40cd000 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009840 tx=0x7fdc9001f010 comp rx=0 tx=0).stop 2024-09-17T05:36:28.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- 172.21.15.122:0/2106731024 shutdown_connections 2024-09-17T05:36:28.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 --2- 172.21.15.122:0/2106731024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 0x7fdca40cd000 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- 172.21.15.122:0/2106731024 >> 172.21.15.122:0/2106731024 conn(0x7fdca40b18f0 msgr2=0x7fdca40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:28.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- 172.21.15.122:0/2106731024 shutdown_connections 2024-09-17T05:36:28.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- 172.21.15.122:0/2106731024 wait complete. 2024-09-17T05:36:28.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 Processor -- start 2024-09-17T05:36:28.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- start start 2024-09-17T05:36:28.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdca40cd540 0x7fdca40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 0x7fdca41554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdca415c3e0 0x7fdca40ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdca406f2f0 con 0x7fdca415bfa0 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdca406f170 con 0x7fdca415c3e0 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdcaa400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdca406f470 con 0x7fdca40cd540 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 0x7fdca41554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 0x7fdca41554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55204/0 (socket says 172.21.15.122:55204) 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca2a00640 1 -- 172.21.15.122:0/3461183269 learned_addr learned my addr 172.21.15.122:0/3461183269 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca3400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdca40cd540 0x7fdca40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca3e00640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdca415c3e0 0x7fdca40ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca2a00640 1 -- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdca40cd540 msgr2=0x7fdca40b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca2a00640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdca40cd540 0x7fdca40b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca2a00640 1 -- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdca415c3e0 msgr2=0x7fdca40ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca2a00640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdca415c3e0 0x7fdca40ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca2a00640 1 -- 172.21.15.122:0/3461183269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca40b4ac0 con 0x7fdca415bfa0 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.086+0000 7fdca3e00640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdca415c3e0 0x7fdca40ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdca2a00640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 0x7fdca41554f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdc9001cc40 tx=0x7fdc90005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdc9b400640 1 -- 172.21.15.122:0/3461183269 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc900b5040 con 0x7fdca415bfa0 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdc9b400640 1 -- 172.21.15.122:0/3461183269 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc90005220 con 0x7fdca415bfa0 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdc9b400640 1 -- 172.21.15.122:0/3461183269 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc900046d0 con 0x7fdca415bfa0 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca40cdcc0 con 0x7fdca415bfa0 2024-09-17T05:36:28.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca40ce280 con 0x7fdca415bfa0 2024-09-17T05:36:28.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca4069410 con 0x7fdca415bfa0 2024-09-17T05:36:28.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdc9b400640 1 -- 172.21.15.122:0/3461183269 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fdc900c1060 con 0x7fdca415bfa0 2024-09-17T05:36:28.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdc9b400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc840748b0 0x7fdc84076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:28.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdc9b400640 1 -- 172.21.15.122:0/3461183269 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7fdc9012fc20 con 0x7fdca415bfa0 2024-09-17T05:36:28.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.090+0000 7fdca3400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc840748b0 0x7fdc84076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:28.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.094+0000 7fdca3400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc840748b0 0x7fdc84076d70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdc940012f0 tx=0x7fdc94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:28.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.094+0000 7fdc9b400640 1 -- 172.21.15.122:0/3461183269 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdc90018b70 con 0x7fdca415bfa0 2024-09-17T05:36:28.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.294+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdca40cd000 con 0x7fdca415bfa0 2024-09-17T05:36:28.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.294+0000 7fdc9b400640 1 -- 172.21.15.122:0/3461183269 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+3923 (secure 0 0 0) 0x7fdc900fc7d0 con 0x7fdca415bfa0 2024-09-17T05:36:28.292 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:28.292 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":59,"btime":"2024-09-17T05:36:21:931347+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/1117224890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":1117224890},{"type":"v1","addr":"172.21.15.122:6835","nonce":1117224890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2271477440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2271477440},{"type":"v1","addr":"172.21.15.173:6837","nonce":2271477440}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/3081382903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3081382903},{"type":"v1","addr":"172.21.15.173:6836","nonce":3081382903}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":7761,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/4074273244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":4074273244},{"type":"v1","addr":"172.21.15.122:6834","nonce":4074273244}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-09-17T05:36:28.292 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 59 2024-09-17T05:36:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc840748b0 msgr2=0x7fdc84076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc840748b0 0x7fdc84076d70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdc940012f0 tx=0x7fdc94009040 comp rx=0 tx=0).stop 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 msgr2=0x7fdca41554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 0x7fdca41554f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdc9001cc40 tx=0x7fdc90005dd0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 shutdown_connections 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc840748b0 0x7fdc84076d70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdca415c3e0 0x7fdca40ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdca415bfa0 0x7fdca41554f0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 --2- 172.21.15.122:0/3461183269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdca40cd540 0x7fdca40b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 >> 172.21.15.122:0/3461183269 conn(0x7fdca40b18f0 msgr2=0x7fdca415c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 shutdown_connections 2024-09-17T05:36:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.298+0000 7fdcaa400640 1 -- 172.21.15.122:0/3461183269 wait complete. 2024-09-17T05:36:28.314 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:36:28.314 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:36:28.314 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:36:28.314 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:36:28.314 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:36:28.314 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:36:28.315 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:36:28.315 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:36:28.315 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:36:28.315 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:36:28.315 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:36:28.315 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:36:28.315 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:36:28.315 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:36:28.316 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:36:28.316 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:36:28.326 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:36:28.326 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:36:28.327 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:36:28.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:36:28.331 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:36:28.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:36:28.331 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:36:28.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:36:28.370 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:36:28.371 DEBUG:teuthology.parallel:result is None 2024-09-17T05:36:28.372 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:36:28.372 DEBUG:teuthology.parallel:result is None 2024-09-17T05:36:28.374 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:36:28.374 DEBUG:teuthology.parallel:result is None 2024-09-17T05:36:28.376 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:36:28.376 DEBUG:teuthology.parallel:result is None 2024-09-17T05:36:28.376 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:36:28.458 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:36:28.458 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:36:28.466 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:36:28.467 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:36:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 -- 172.21.15.122:0/1046560449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc0b3300 msgr2=0x7f55fc15f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 --2- 172.21.15.122:0/1046560449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc0b3300 0x7f55fc15f4e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009840 tx=0x7f55e801c7f0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 -- 172.21.15.122:0/1046560449 shutdown_connections 2024-09-17T05:36:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 --2- 172.21.15.122:0/1046560449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc0b3300 0x7f55fc15f4e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 -- 172.21.15.122:0/1046560449 >> 172.21.15.122:0/1046560449 conn(0x7f55fc0b18f0 msgr2=0x7f55fc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 -- 172.21.15.122:0/1046560449 shutdown_connections 2024-09-17T05:36:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 -- 172.21.15.122:0/1046560449 wait complete. 2024-09-17T05:36:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 Processor -- start 2024-09-17T05:36:28.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 -- start start 2024-09-17T05:36:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc0b3300 0x7f55fc09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.530+0000 7f5601a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc0cd0d0 0x7f55fc141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f5601a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc15af90 0x7f55fc0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:28.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f5601a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1696f0 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f5601a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc169570 con 0x7f55fc15af90 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f5601a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc169870 con 0x7f55fc0b3300 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc0cd0d0 0x7f55fc141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc0cd0d0 0x7f55fc141d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55236/0 (socket says 172.21.15.122:55236) 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 -- 172.21.15.122:0/1470487559 learned_addr learned my addr 172.21.15.122:0/1470487559 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55faa00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc0b3300 0x7f55fc09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fb400640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc15af90 0x7f55fc0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 -- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc0b3300 msgr2=0x7f55fc09aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc0b3300 0x7f55fc09aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 -- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc15af90 msgr2=0x7f55fc0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc15af90 0x7f55fc0acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 -- 172.21.15.122:0/1470487559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc09c300 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55fa000640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc0cd0d0 0x7f55fc141d30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009790 tx=0x7f55e8005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:28.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55f2a00640 1 -- 172.21.15.122:0/1470487559 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e80b2040 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55f2a00640 1 -- 172.21.15.122:0/1470487559 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e80043f0 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc142270 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55f2a00640 1 -- 172.21.15.122:0/1470487559 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8005220 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc09c580 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55fc069410 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.534+0000 7f55f2a00640 1 -- 172.21.15.122:0/1470487559 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f55e80a5830 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.538+0000 7f55f2a00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55dc074880 0x7f55dc076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:28.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.538+0000 7f55f2a00640 1 -- 172.21.15.122:0/1470487559 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4999+0+0 (secure 0 0 0) 0x7f55e812bfe0 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.538+0000 7f55faa00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55dc074880 0x7f55dc076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:28.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.538+0000 7f55faa00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55dc074880 0x7f55dc076d40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f55ec001130 tx=0x7f55ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:28.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.542+0000 7f55f2a00640 1 -- 172.21.15.122:0/1470487559 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f55fc09c300 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.722+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f55fc06ec00 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.962+0000 7f55f2a00640 1 -- 172.21.15.122:0/1470487559 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) ==== 97+0+0 (secure 0 0 0) 0x7f55e8016410 con 0x7f55fc0cd0d0 2024-09-17T05:36:28.960 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:36:28.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55dc074880 msgr2=0x7f55dc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55dc074880 0x7f55dc076d40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f55ec001130 tx=0x7f55ec009040 comp rx=0 tx=0).stop 2024-09-17T05:36:28.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc0cd0d0 msgr2=0x7f55fc141d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:28.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc0cd0d0 0x7f55fc141d30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009790 tx=0x7f55e8005db0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 shutdown_connections 2024-09-17T05:36:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55dc074880 0x7f55dc076d40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55fc15af90 0x7f55fc0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55fc0cd0d0 0x7f55fc141d30 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 --2- 172.21.15.122:0/1470487559 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55fc0b3300 0x7f55fc09aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 >> 172.21.15.122:0/1470487559 conn(0x7f55fc0b18f0 msgr2=0x7f55fc0b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 shutdown_connections 2024-09-17T05:36:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:28.966+0000 7f5601a00640 1 -- 172.21.15.122:0/1470487559 wait complete. 2024-09-17T05:36:28.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:36:29.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.130+0000 7fc7d0200640 1 -- 172.21.15.122:0/3909224047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7b0004d50 msgr2=0x7fc7b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.130+0000 7fc7d0200640 1 --2- 172.21.15.122:0/3909224047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7b0004d50 0x7fc7b00051d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c640 tx=0x7fc7c001d260 comp rx=0 tx=0).stop 2024-09-17T05:36:29.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.130+0000 7fc7ce200640 1 -- 172.21.15.122:0/3909224047 reap_dead start 2024-09-17T05:36:29.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.130+0000 7fc7d0200640 1 -- 172.21.15.122:0/3909224047 shutdown_connections 2024-09-17T05:36:29.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.130+0000 7fc7d0200640 1 -- 172.21.15.122:0/3909224047 >> 172.21.15.122:0/3909224047 conn(0x7fc7c80b18f0 msgr2=0x7fc7c80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:29.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.130+0000 7fc7d0200640 1 -- 172.21.15.122:0/3909224047 shutdown_connections 2024-09-17T05:36:29.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.130+0000 7fc7d0200640 1 -- 172.21.15.122:0/3909224047 wait complete. 2024-09-17T05:36:29.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.130+0000 7fc7d0200640 1 Processor -- start 2024-09-17T05:36:29.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 -- start start 2024-09-17T05:36:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7b0004d50 0x7fc7c8158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c809d020 0x7fc7c815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7c81544d0 0x7fc7c8154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c80acec0 con 0x7fc7c81544d0 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c80acd40 con 0x7fc7c809d020 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c80ad040 con 0x7fc7b0004d50 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7c81544d0 0x7fc7c8154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7c81544d0 0x7fc7c8154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55254/0 (socket says 172.21.15.122:55254) 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 -- 172.21.15.122:0/3474065042 learned_addr learned my addr 172.21.15.122:0/3474065042 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7cce00640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c809d020 0x7fc7c815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7cd800640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7b0004d50 0x7fc7c8158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 -- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7b0004d50 msgr2=0x7fc7c8158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7b0004d50 0x7fc7c8158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 -- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c809d020 msgr2=0x7fc7c815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c809d020 0x7fc7c815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 -- 172.21.15.122:0/3474065042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c8154ea0 con 0x7fc7c81544d0 2024-09-17T05:36:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7cce00640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c809d020 0x7fc7c815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7ce200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7c81544d0 0x7fc7c8154960 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc7c001b960 tx=0x7fc7c0004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7c5600640 1 -- 172.21.15.122:0/3474065042 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0001fc0 con 0x7fc7c81544d0 2024-09-17T05:36:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7c5600640 1 -- 172.21.15.122:0/3474065042 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7c0003c70 con 0x7fc7c81544d0 2024-09-17T05:36:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c809d4e0 con 0x7fc7c81544d0 2024-09-17T05:36:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7c5600640 1 -- 172.21.15.122:0/3474065042 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00ba620 con 0x7fc7c81544d0 2024-09-17T05:36:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c809daa0 con 0x7fc7c81544d0 2024-09-17T05:36:29.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.134+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c8069410 con 0x7fc7c81544d0 2024-09-17T05:36:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.138+0000 7fc7c5600640 1 -- 172.21.15.122:0/3474065042 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fc7c00c1020 con 0x7fc7c81544d0 2024-09-17T05:36:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.138+0000 7fc7c5600640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7a8074880 0x7fc7a8076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.138+0000 7fc7c5600640 1 -- 172.21.15.122:0/3474065042 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4913+0+0 (secure 0 0 0) 0x7fc7c012d800 con 0x7fc7c81544d0 2024-09-17T05:36:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.138+0000 7fc7cd800640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7a8074880 0x7fc7a8076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.138+0000 7fc7cd800640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7a8074880 0x7fc7a8076d40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc7c815d330 tx=0x7fc7bc00c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:29.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.142+0000 7fc7c5600640 1 -- 172.21.15.122:0/3474065042 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc7c0017010 con 0x7fc7c81544d0 2024-09-17T05:36:29.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc7c809dce0 con 0x7fc7c81544d0 2024-09-17T05:36:29.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7c5600640 1 -- 172.21.15.122:0/3474065042 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fc7c809dce0 con 0x7fc7c81544d0 2024-09-17T05:36:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:29.366 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:36:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:36:29.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:36:29.367 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:36:29.368 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.370 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.371 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:36:29.372 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.373 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:36:29.374 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow r", 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.375 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.376 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:36:29.377 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.378 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:36:29.379 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:36:29.380 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:36:29.380 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:36:29.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.380 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:36:29.380 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:36:29.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7a8074880 msgr2=0x7fc7a8076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7a8074880 0x7fc7a8076d40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc7c815d330 tx=0x7fc7bc00c040 comp rx=0 tx=0).stop 2024-09-17T05:36:29.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7c81544d0 msgr2=0x7fc7c8154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7c81544d0 0x7fc7c8154960 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc7c001b960 tx=0x7fc7c0004e60 comp rx=0 tx=0).stop 2024-09-17T05:36:29.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 shutdown_connections 2024-09-17T05:36:29.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7a8074880 0x7fc7a8076d40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7c81544d0 0x7fc7c8154960 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7c809d020 0x7fc7c815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 --2- 172.21.15.122:0/3474065042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7b0004d50 0x7fc7c8158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.370+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 >> 172.21.15.122:0/3474065042 conn(0x7fc7c80b18f0 msgr2=0x7fc7c80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:29.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.374+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 shutdown_connections 2024-09-17T05:36:29.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.374+0000 7fc7d0200640 1 -- 172.21.15.122:0/3474065042 wait complete. 2024-09-17T05:36:29.388 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:36:29.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 -- 172.21.15.122:0/2661273566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad40d28e0 msgr2=0x7f4ad40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/2661273566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad40d28e0 0x7f4ad40cebb0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40745d0 tx=0x7f4acc01caf0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 -- 172.21.15.122:0/2661273566 shutdown_connections 2024-09-17T05:36:29.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/2661273566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad40d28e0 0x7f4ad40cebb0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/2661273566 >> v2:172.21.15.173:3300/0 conn(0x7f4ad415bea0 0x7f4ad40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 -- 172.21.15.122:0/2661273566 >> 172.21.15.122:0/2661273566 conn(0x7f4ad40d1dc0 msgr2=0x7f4ad40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:29.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 -- 172.21.15.122:0/2661273566 shutdown_connections 2024-09-17T05:36:29.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 -- 172.21.15.122:0/2661273566 wait complete. 2024-09-17T05:36:29.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 Processor -- start 2024-09-17T05:36:29.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.534+0000 7f4ad8e00640 1 -- start start 2024-09-17T05:36:29.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ad40cdb30 0x7f4ad4159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad40d28e0 0x7f4ad41597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad415bea0 0x7f4ad40c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4168fc0 con 0x7f4ad415bea0 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4168e40 con 0x7f4ad40d28e0 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad4169140 con 0x7f4ad40cdb30 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad415bea0 0x7f4ad40c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad415bea0 0x7f4ad40c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55274/0 (socket says 172.21.15.122:55274) 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 -- 172.21.15.122:0/3034594250 learned_addr learned my addr 172.21.15.122:0/3034594250 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ac9400640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad40d28e0 0x7f4ad41597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad1e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ad40cdb30 0x7f4ad4159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 -- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ad40cdb30 msgr2=0x7f4ad4159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ad40cdb30 0x7f4ad4159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 -- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad40d28e0 msgr2=0x7f4ad41597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad40d28e0 0x7f4ad41597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 -- 172.21.15.122:0/3034594250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad4159d30 con 0x7f4ad415bea0 2024-09-17T05:36:29.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad1e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ad40cdb30 0x7f4ad4159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:29.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad2800640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad415bea0 0x7f4ad40c5f30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4acc019700 tx=0x7f4acc0a58f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:29.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4acaa00640 1 -- 172.21.15.122:0/3034594250 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0ba020 con 0x7f4ad415bea0 2024-09-17T05:36:29.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4acaa00640 1 -- 172.21.15.122:0/3034594250 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4acc004d40 con 0x7f4ad415bea0 2024-09-17T05:36:29.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad40c56c0 con 0x7f4ad415bea0 2024-09-17T05:36:29.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4acaa00640 1 -- 172.21.15.122:0/3034594250 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0b2040 con 0x7f4ad415bea0 2024-09-17T05:36:29.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad40c8970 con 0x7f4ad415bea0 2024-09-17T05:36:29.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.538+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad40cc9d0 con 0x7f4ad415bea0 2024-09-17T05:36:29.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.542+0000 7f4acaa00640 1 -- 172.21.15.122:0/3034594250 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f4acc004300 con 0x7f4ad415bea0 2024-09-17T05:36:29.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.542+0000 7f4acaa00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ab0074880 0x7f4ab0076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.542+0000 7f4acaa00640 1 -- 172.21.15.122:0/3034594250 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4913+0+0 (secure 0 0 0) 0x7f4acc12d6d0 con 0x7f4ad415bea0 2024-09-17T05:36:29.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.542+0000 7f4ad1e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ab0074880 0x7f4ab0076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.542+0000 7f4ad1e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ab0074880 0x7f4ab0076d40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4005fd0 tx=0x7f4ac4005f20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.546+0000 7f4acaa00640 1 -- 172.21.15.122:0/3034594250 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4acc0c4020 con 0x7f4ad415bea0 2024-09-17T05:36:29.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.714+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4ad40b50b0 con 0x7f4ad415bea0 2024-09-17T05:36:29.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.714+0000 7f4acaa00640 1 -- 172.21.15.122:0/3034594250 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) ==== 84+0+65 (secure 0 0 0) 0x7f4acc016770 con 0x7f4ad415bea0 2024-09-17T05:36:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:29.710 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:36:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:36:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:36:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:29.711 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:36:29.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.714+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ab0074880 msgr2=0x7f4ab0076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.714+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ab0074880 0x7f4ab0076d40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4005fd0 tx=0x7f4ac4005f20 comp rx=0 tx=0).stop 2024-09-17T05:36:29.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.714+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad415bea0 msgr2=0x7f4ad40c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.714+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad415bea0 0x7f4ad40c5f30 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4acc019700 tx=0x7f4acc0a58f0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.718+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 shutdown_connections 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.718+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4ab0074880 0x7f4ab0076d40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.718+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ad415bea0 0x7f4ad40c5f30 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.718+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ad40d28e0 0x7f4ad41597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.718+0000 7f4ad8e00640 1 --2- 172.21.15.122:0/3034594250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ad40cdb30 0x7f4ad4159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.718+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 >> 172.21.15.122:0/3034594250 conn(0x7f4ad40d1dc0 msgr2=0x7f4ad40b71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.718+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 shutdown_connections 2024-09-17T05:36:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.718+0000 7f4ad8e00640 1 -- 172.21.15.122:0/3034594250 wait complete. 2024-09-17T05:36:29.731 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:36:29.731 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:36:29.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.882+0000 7f131a200640 1 -- 172.21.15.122:0/2823555053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f80053d0 msgr2=0x7f12f800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.882+0000 7f131a200640 1 --2- 172.21.15.122:0/2823555053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f80053d0 0x7f12f800b9d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f130c001dc0 tx=0x7f130c01cb30 comp rx=0 tx=0).stop 2024-09-17T05:36:29.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.882+0000 7f131a200640 1 -- 172.21.15.122:0/2823555053 shutdown_connections 2024-09-17T05:36:29.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.882+0000 7f131a200640 1 --2- 172.21.15.122:0/2823555053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f80053d0 0x7f12f800b9d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.882+0000 7f131a200640 1 -- 172.21.15.122:0/2823555053 >> 172.21.15.122:0/2823555053 conn(0x7f13140cfbc0 msgr2=0x7f13140cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:29.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.882+0000 7f131a200640 1 -- 172.21.15.122:0/2823555053 shutdown_connections 2024-09-17T05:36:29.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.882+0000 7f131a200640 1 -- 172.21.15.122:0/2823555053 wait complete. 2024-09-17T05:36:29.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 Processor -- start 2024-09-17T05:36:29.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 -- start start 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80053d0 0x7f1314150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131414eaa0 0x7f13141f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13141590e0 0x7f1314159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f131415ed50 con 0x7f13141590e0 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f131415ebd0 con 0x7f12f80053d0 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f131415eed0 con 0x7f131414eaa0 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13141590e0 0x7f1314159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13141590e0 0x7f1314159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55278/0 (socket says 172.21.15.122:55278) 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 -- 172.21.15.122:0/3725387644 learned_addr learned my addr 172.21.15.122:0/3725387644 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:29.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f130aa00640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131414eaa0 0x7f13141f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313400640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80053d0 0x7f1314150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 -- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131414eaa0 msgr2=0x7f13141f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131414eaa0 0x7f13141f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 -- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80053d0 msgr2=0x7f1314150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:29.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80053d0 0x7f1314150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:29.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 -- 172.21.15.122:0/3725387644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13141f5ae0 con 0x7f13141590e0 2024-09-17T05:36:29.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f130aa00640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131414eaa0 0x7f13141f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:29.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f1313e00640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13141590e0 0x7f1314159590 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f130c01d3d0 tx=0x7f130c004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:29.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f130be00640 1 -- 172.21.15.122:0/3725387644 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0019f0 con 0x7f13141590e0 2024-09-17T05:36:29.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f130be00640 1 -- 172.21.15.122:0/3725387644 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f130c0b5070 con 0x7f13141590e0 2024-09-17T05:36:29.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f130be00640 1 -- 172.21.15.122:0/3725387644 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0063e0 con 0x7f13141590e0 2024-09-17T05:36:29.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13141f2b80 con 0x7f13141590e0 2024-09-17T05:36:29.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13141f3140 con 0x7f13141590e0 2024-09-17T05:36:29.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.886+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1314069410 con 0x7f13141590e0 2024-09-17T05:36:29.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.890+0000 7f130be00640 1 -- 172.21.15.122:0/3725387644 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f130c006580 con 0x7f13141590e0 2024-09-17T05:36:29.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.890+0000 7f130be00640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12f0074880 0x7f12f0076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:29.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.890+0000 7f130be00640 1 -- 172.21.15.122:0/3725387644 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4913+0+0 (secure 0 0 0) 0x7f130c12d750 con 0x7f13141590e0 2024-09-17T05:36:29.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.890+0000 7f1313400640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12f0074880 0x7f12f0076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:29.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.890+0000 7f1313400640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12f0074880 0x7f12f0076d40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1304001780 tx=0x7f13040067b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:29.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:29.894+0000 7f130be00640 1 -- 172.21.15.122:0/3725387644 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f130c0c1030 con 0x7f13141590e0 2024-09-17T05:36:30.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.066+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f131415f430 con 0x7f13141590e0 2024-09-17T05:36:30.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.974+0000 7f130be00640 1 -- 172.21.15.122:0/3725387644 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) ==== 138+0+0 (secure 0 0 0) 0x7f130c016890 con 0x7f13141590e0 2024-09-17T05:36:30.970 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:36:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12f0074880 msgr2=0x7f12f0076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12f0074880 0x7f12f0076d40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1304001780 tx=0x7f13040067b0 comp rx=0 tx=0).stop 2024-09-17T05:36:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13141590e0 msgr2=0x7f1314159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13141590e0 0x7f1314159590 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f130c01d3d0 tx=0x7f130c004f60 comp rx=0 tx=0).stop 2024-09-17T05:36:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 shutdown_connections 2024-09-17T05:36:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12f0074880 0x7f12f0076d40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13141590e0 0x7f1314159590 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:30.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131414eaa0 0x7f13141f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 --2- 172.21.15.122:0/3725387644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80053d0 0x7f1314150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 >> 172.21.15.122:0/3725387644 conn(0x7f13140cfbc0 msgr2=0x7f13140b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 shutdown_connections 2024-09-17T05:36:30.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:30.978+0000 7f131a200640 1 -- 172.21.15.122:0/3725387644 wait complete. 2024-09-17T05:36:30.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:36:31.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.154+0000 7efd05c00640 1 -- 172.21.15.122:0/3004437321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd000b1650 msgr2=0x7efd000b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:31.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.154+0000 7efd05c00640 1 --2- 172.21.15.122:0/3004437321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd000b1650 0x7efd000b1a30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7efd00073810 tx=0x7efcec01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.154+0000 7efd05c00640 1 -- 172.21.15.122:0/3004437321 shutdown_connections 2024-09-17T05:36:31.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.154+0000 7efd05c00640 1 --2- 172.21.15.122:0/3004437321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd000b1650 0x7efd000b1a30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.154+0000 7efd05c00640 1 --2- 172.21.15.122:0/3004437321 >> v2:172.21.15.173:3300/0 conn(0x7efd001605d0 0x7efd000d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.154+0000 7efd05c00640 1 -- 172.21.15.122:0/3004437321 >> 172.21.15.122:0/3004437321 conn(0x7efd000d1dd0 msgr2=0x7efd000d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:31.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.154+0000 7efd05c00640 1 -- 172.21.15.122:0/3004437321 shutdown_connections 2024-09-17T05:36:31.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.154+0000 7efd05c00640 1 -- 172.21.15.122:0/3004437321 wait complete. 2024-09-17T05:36:31.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 Processor -- start 2024-09-17T05:36:31.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 -- start start 2024-09-17T05:36:31.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd000aeb00 0x7efd0015f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:31.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd000b1650 0x7efd0014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:31.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd001605d0 0x7efd0014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:31.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd000b1220 con 0x7efd001605d0 2024-09-17T05:36:31.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd000b0ee0 con 0x7efd000aeb00 2024-09-17T05:36:31.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efd000b13a0 con 0x7efd000b1650 2024-09-17T05:36:31.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcff400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd001605d0 0x7efd0014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:31.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcfea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd000aeb00 0x7efd0015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:31.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcfea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd000aeb00 0x7efd0015f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:54750/0 (socket says 172.21.15.122:54750) 2024-09-17T05:36:31.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcfea00640 1 -- 172.21.15.122:0/843819876 learned_addr learned my addr 172.21.15.122:0/843819876 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:31.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcfe000640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd000b1650 0x7efd0014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:31.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcff400640 1 -- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd000b1650 msgr2=0x7efd0014ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:31.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcff400640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd000b1650 0x7efd0014ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcff400640 1 -- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd000aeb00 msgr2=0x7efd0015f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:31.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcff400640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd000aeb00 0x7efd0015f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcff400640 1 -- 172.21.15.122:0/843819876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd00153cf0 con 0x7efd001605d0 2024-09-17T05:36:31.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcff400640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd001605d0 0x7efd0014eb00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c440 tx=0x7efcf800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcf6a00640 1 -- 172.21.15.122:0/843819876 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8011790 con 0x7efd001605d0 2024-09-17T05:36:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcf6a00640 1 -- 172.21.15.122:0/843819876 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcf8017070 con 0x7efd001605d0 2024-09-17T05:36:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efcf6a00640 1 -- 172.21.15.122:0/843819876 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8011cc0 con 0x7efd001605d0 2024-09-17T05:36:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd00153770 con 0x7efd001605d0 2024-09-17T05:36:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd00153a10 con 0x7efd001605d0 2024-09-17T05:36:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.158+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd000b1a30 con 0x7efd001605d0 2024-09-17T05:36:31.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.162+0000 7efcf6a00640 1 -- 172.21.15.122:0/843819876 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7efcf8026430 con 0x7efd001605d0 2024-09-17T05:36:31.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.162+0000 7efcf6a00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efce0074880 0x7efce0076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:31.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.162+0000 7efcf6a00640 1 -- 172.21.15.122:0/843819876 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5283+0+0 (secure 0 0 0) 0x7efcf8093640 con 0x7efd001605d0 2024-09-17T05:36:31.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.162+0000 7efcfea00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efce0074880 0x7efce0076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:31.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.162+0000 7efcfea00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efce0074880 0x7efce0076d40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009f10 tx=0x7efcf0009660 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:31.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.166+0000 7efcf6a00640 1 -- 172.21.15.122:0/843819876 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efcf805fb80 con 0x7efd001605d0 2024-09-17T05:36:31.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.338+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efd000d1080 con 0x7efd001605d0 2024-09-17T05:36:31.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.978+0000 7efcf6a00640 1 -- 172.21.15.122:0/843819876 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) ==== 170+0+0 (secure 0 0 0) 0x7efcf80645b0 con 0x7efd001605d0 2024-09-17T05:36:31.973 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:36:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.978+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efce0074880 msgr2=0x7efce0076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.978+0000 7efd05c00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efce0074880 0x7efce0076d40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009f10 tx=0x7efcf0009660 comp rx=0 tx=0).stop 2024-09-17T05:36:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.978+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd001605d0 msgr2=0x7efd0014eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:31.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.978+0000 7efd05c00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd001605d0 0x7efd0014eb00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c440 tx=0x7efcf800c900 comp rx=0 tx=0).stop 2024-09-17T05:36:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.982+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 shutdown_connections 2024-09-17T05:36:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.982+0000 7efd05c00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efce0074880 0x7efce0076d40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.982+0000 7efd05c00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd001605d0 0x7efd0014eb00 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.982+0000 7efd05c00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd000b1650 0x7efd0014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.982+0000 7efd05c00640 1 --2- 172.21.15.122:0/843819876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd000aeb00 0x7efd0015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.982+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 >> 172.21.15.122:0/843819876 conn(0x7efd000d1dd0 msgr2=0x7efd000afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.982+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 shutdown_connections 2024-09-17T05:36:31.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:31.982+0000 7efd05c00640 1 -- 172.21.15.122:0/843819876 wait complete. 2024-09-17T05:36:32.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:36:32.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 -- 172.21.15.122:0/4126536174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4af00d0640 msgr2=0x7f4af00cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:32.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 --2- 172.21.15.122:0/4126536174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4af00d0640 0x7f4af00cdfe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009870 tx=0x7f4ad801c860 comp rx=0 tx=0).stop 2024-09-17T05:36:32.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 -- 172.21.15.122:0/4126536174 shutdown_connections 2024-09-17T05:36:32.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 --2- 172.21.15.122:0/4126536174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4af00d0640 0x7f4af00cdfe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:32.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 -- 172.21.15.122:0/4126536174 >> 172.21.15.122:0/4126536174 conn(0x7f4af00cfa00 msgr2=0x7f4af00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:32.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 -- 172.21.15.122:0/4126536174 shutdown_connections 2024-09-17T05:36:32.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 -- 172.21.15.122:0/4126536174 wait complete. 2024-09-17T05:36:32.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 Processor -- start 2024-09-17T05:36:32.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.166+0000 7f4af5200640 1 -- start start 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4af006f140 0x7f4af00b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4af006f520 0x7f4af00c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4af00d0640 0x7f4af00c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4af0168320 con 0x7f4af00d0640 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4af01681a0 con 0x7f4af006f140 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4af01684a0 con 0x7f4af006f520 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4af00d0640 0x7f4af00c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4af00d0640 0x7f4af00c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44506/0 (socket says 172.21.15.122:44506) 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 -- 172.21.15.122:0/2180753418 learned_addr learned my addr 172.21.15.122:0/2180753418 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aef400640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4af006f140 0x7f4af00b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:32.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4ae7e00640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4af006f520 0x7f4af00c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:32.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 -- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4af006f520 msgr2=0x7f4af00c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4af006f520 0x7f4af00c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 -- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4af006f140 msgr2=0x7f4af00b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4af006f140 0x7f4af00b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 -- 172.21.15.122:0/2180753418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af00b5790 con 0x7f4af00d0640 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4aefe00640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4af00d0640 0x7f4af00c9640 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c790 tx=0x7f4ae800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4ae7400640 1 -- 172.21.15.122:0/2180753418 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8003b80 con 0x7f4af00d0640 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4ae7400640 1 -- 172.21.15.122:0/2180753418 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ae8003d20 con 0x7f4af00d0640 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4ae7400640 1 -- 172.21.15.122:0/2180753418 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800aab0 con 0x7f4af00d0640 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af00b2e90 con 0x7f4af00d0640 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af00b3450 con 0x7f4af00d0640 2024-09-17T05:36:32.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.170+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af00738b0 con 0x7f4af00d0640 2024-09-17T05:36:32.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.174+0000 7f4ae7400640 1 -- 172.21.15.122:0/2180753418 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f4ae800ac50 con 0x7f4af00d0640 2024-09-17T05:36:32.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.174+0000 7f4ae7400640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4acc074880 0x7f4acc076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:32.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.174+0000 7f4ae7400640 1 -- 172.21.15.122:0/2180753418 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5665+0+0 (secure 0 0 0) 0x7f4ae8093700 con 0x7f4af00d0640 2024-09-17T05:36:32.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.174+0000 7f4aef400640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4acc074880 0x7f4acc076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:32.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.174+0000 7f4aef400640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4acc074880 0x7f4acc076d40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0007f30 tx=0x7f4ae0007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:32.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.178+0000 7f4ae7400640 1 -- 172.21.15.122:0/2180753418 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4ae805fc40 con 0x7f4af00d0640 2024-09-17T05:36:32.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:32.382+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4af00c65f0 con 0x7f4af00d0640 2024-09-17T05:36:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.010+0000 7f4ae7400640 1 -- 172.21.15.122:0/2180753418 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v62) ==== 174+0+0 (secure 0 0 0) 0x7f4ae8064670 con 0x7f4af00d0640 2024-09-17T05:36:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 14 and data pool 15 2024-09-17T05:36:33.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4acc074880 msgr2=0x7f4acc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4acc074880 0x7f4acc076d40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0007f30 tx=0x7f4ae0007ec0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4af00d0640 msgr2=0x7f4af00c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4af00d0640 0x7f4af00c9640 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c790 tx=0x7f4ae800cc50 comp rx=0 tx=0).stop 2024-09-17T05:36:33.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 shutdown_connections 2024-09-17T05:36:33.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4acc074880 0x7f4acc076d40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4af00d0640 0x7f4af00c9640 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4af006f520 0x7f4af00c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 --2- 172.21.15.122:0/2180753418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4af006f140 0x7f4af00b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.022+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 >> 172.21.15.122:0/2180753418 conn(0x7f4af00cfa00 msgr2=0x7f4af01624a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:33.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.026+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 shutdown_connections 2024-09-17T05:36:33.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.030+0000 7f4af5200640 1 -- 172.21.15.122:0/2180753418 wait complete. 2024-09-17T05:36:33.055 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:36:33.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.202+0000 7f92c5800640 1 -- 172.21.15.122:0/2467446483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92c00cffc0 msgr2=0x7f92c0073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.202+0000 7f92c5800640 1 --2- 172.21.15.122:0/2467446483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92c00cffc0 0x7f92c0073ed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009840 tx=0x7f92a801c7f0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.202+0000 7f92c5800640 1 -- 172.21.15.122:0/2467446483 shutdown_connections 2024-09-17T05:36:33.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.202+0000 7f92c5800640 1 --2- 172.21.15.122:0/2467446483 >> v2:172.21.15.173:3300/0 conn(0x7f92c006f9a0 0x7f92c006fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.202+0000 7f92c5800640 1 --2- 172.21.15.122:0/2467446483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92c00cffc0 0x7f92c0073ed0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.202+0000 7f92c5800640 1 -- 172.21.15.122:0/2467446483 >> 172.21.15.122:0/2467446483 conn(0x7f92c00d0c90 msgr2=0x7f92c00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:33.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.202+0000 7f92c5800640 1 -- 172.21.15.122:0/2467446483 shutdown_connections 2024-09-17T05:36:33.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.202+0000 7f92c5800640 1 -- 172.21.15.122:0/2467446483 wait complete. 2024-09-17T05:36:33.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 Processor -- start 2024-09-17T05:36:33.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 -- start start 2024-09-17T05:36:33.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c006f9a0 0x7f92c00c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92c00cffc0 0x7f92c01fb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c015af60 0x7f92c01fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0167b40 con 0x7f92c006f9a0 2024-09-17T05:36:33.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0167cc0 con 0x7f92c015af60 2024-09-17T05:36:33.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f92c0167e40 con 0x7f92c00cffc0 2024-09-17T05:36:33.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c006f9a0 0x7f92c00c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c006f9a0 0x7f92c00c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44522/0 (socket says 172.21.15.122:44522) 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 -- 172.21.15.122:0/1019105624 learned_addr learned my addr 172.21.15.122:0/1019105624 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bf400640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c015af60 0x7f92c01fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92b6000640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92c00cffc0 0x7f92c01fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 -- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92c00cffc0 msgr2=0x7f92c01fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92c00cffc0 0x7f92c01fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 -- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c015af60 msgr2=0x7f92c01fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c015af60 0x7f92c01fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 -- 172.21.15.122:0/1019105624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c009f4c0 con 0x7f92c006f9a0 2024-09-17T05:36:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92bea00640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c006f9a0 0x7f92c00c5b90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c400 tx=0x7f92b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92b7400640 1 -- 172.21.15.122:0/1019105624 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0011790 con 0x7f92c006f9a0 2024-09-17T05:36:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92b7400640 1 -- 172.21.15.122:0/1019105624 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92b0017070 con 0x7f92c006f9a0 2024-09-17T05:36:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92b7400640 1 -- 172.21.15.122:0/1019105624 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0011cc0 con 0x7f92c006f9a0 2024-09-17T05:36:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c009f7d0 con 0x7f92c006f9a0 2024-09-17T05:36:33.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c009fd90 con 0x7f92c006f9a0 2024-09-17T05:36:33.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.206+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c00c6130 con 0x7f92c006f9a0 2024-09-17T05:36:33.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.210+0000 7f92b7400640 1 -- 172.21.15.122:0/1019105624 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f92b001c460 con 0x7f92c006f9a0 2024-09-17T05:36:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.210+0000 7f92b7400640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f929c074880 0x7f929c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.210+0000 7f92b7400640 1 -- 172.21.15.122:0/1019105624 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5765+0+0 (secure 0 0 0) 0x7f92b0092c40 con 0x7f92c006f9a0 2024-09-17T05:36:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.210+0000 7f92b6000640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f929c074880 0x7f929c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.210+0000 7f92b6000640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f929c074880 0x7f929c076d40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f92a80af000 tx=0x7f92a801b030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:33.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.214+0000 7f92b7400640 1 -- 172.21.15.122:0/1019105624 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f92b005f080 con 0x7f92c006f9a0 2024-09-17T05:36:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.386+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92c006fe20 con 0x7f92c006f9a0 2024-09-17T05:36:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.386+0000 7f92b7400640 1 -- 172.21.15.122:0/1019105624 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16839 (secure 0 0 0) 0x7f92b0063ab0 con 0x7f92c006f9a0 2024-09-17T05:36:33.382 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:33.383 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":102,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:36:32.388490+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:36:30.080602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:36:31.349633+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f929c074880 msgr2=0x7f929c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f929c074880 0x7f929c076d40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f92a80af000 tx=0x7f92a801b030 comp rx=0 tx=0).stop 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c006f9a0 msgr2=0x7f92c00c5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c006f9a0 0x7f92c00c5b90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c400 tx=0x7f92b000c8c0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 shutdown_connections 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f929c074880 0x7f929c076d40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92c015af60 0x7f92c01fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92c00cffc0 0x7f92c01fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 --2- 172.21.15.122:0/1019105624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c006f9a0 0x7f92c00c5b90 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 >> 172.21.15.122:0/1019105624 conn(0x7f92c00d0c90 msgr2=0x7f92c01597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 shutdown_connections 2024-09-17T05:36:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.390+0000 7f92c5800640 1 -- 172.21.15.122:0/1019105624 wait complete. 2024-09-17T05:36:33.405 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:36:33.431 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:36:33.454 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:36:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- 172.21.15.122:0/59846076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 msgr2=0x7fc9b40b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 --2- 172.21.15.122:0/59846076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 0x7fc9b40b1a00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009840 tx=0x7fc9a001c6d0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- 172.21.15.122:0/59846076 shutdown_connections 2024-09-17T05:36:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 --2- 172.21.15.122:0/59846076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 0x7fc9b40b1a00 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 --2- 172.21.15.122:0/59846076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9b415f490 0x7fc9b40b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- 172.21.15.122:0/59846076 >> 172.21.15.122:0/59846076 conn(0x7fc9b40d1e30 msgr2=0x7fc9b40d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- 172.21.15.122:0/59846076 shutdown_connections 2024-09-17T05:36:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- 172.21.15.122:0/59846076 wait complete. 2024-09-17T05:36:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 Processor -- start 2024-09-17T05:36:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- start start 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9b40b1f40 0x7fc9b40c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 0x7fc9b40c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9b415f490 0x7fc9b40b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4168e00 con 0x7fc9b40b2e80 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4168c80 con 0x7fc9b415f490 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9bc600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b4168f80 con 0x7fc9b40b1f40 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9b9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 0x7fc9b40c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9b9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 0x7fc9b40c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44550/0 (socket says 172.21.15.122:44550) 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9b9200640 1 -- 172.21.15.122:0/3731554671 learned_addr learned my addr 172.21.15.122:0/3731554671 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9ba600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9b415f490 0x7fc9b40b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.554+0000 7fc9b9c00640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9b40b1f40 0x7fc9b40c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b9200640 1 -- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9b40b1f40 msgr2=0x7fc9b40c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b9200640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9b40b1f40 0x7fc9b40c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b9200640 1 -- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9b415f490 msgr2=0x7fc9b40b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b9200640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9b415f490 0x7fc9b40b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b9200640 1 -- 172.21.15.122:0/3731554671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b40c8350 con 0x7fc9b40b2e80 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9ba600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9b415f490 0x7fc9b40b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b9200640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 0x7fc9b40c7a70 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009790 tx=0x7fc9a00a7b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b1600640 1 -- 172.21.15.122:0/3731554671 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00bc030 con 0x7fc9b40b2e80 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b1600640 1 -- 172.21.15.122:0/3731554671 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9a00a7e60 con 0x7fc9b40b2e80 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b1600640 1 -- 172.21.15.122:0/3731554671 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0004290 con 0x7fc9b40b2e80 2024-09-17T05:36:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b40c8630 con 0x7fc9b40b2e80 2024-09-17T05:36:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b40cd5b0 con 0x7fc9b40b2e80 2024-09-17T05:36:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b40cfda0 con 0x7fc9b40b2e80 2024-09-17T05:36:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b1600640 1 -- 172.21.15.122:0/3731554671 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fc9a00bf070 con 0x7fc9b40b2e80 2024-09-17T05:36:33.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.558+0000 7fc9b1600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc994074880 0x7fc994076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.562+0000 7fc9b1600640 1 -- 172.21.15.122:0/3731554671 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5765+0+0 (secure 0 0 0) 0x7fc9a012d230 con 0x7fc9b40b2e80 2024-09-17T05:36:33.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.562+0000 7fc9b9c00640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc994074880 0x7fc994076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.562+0000 7fc9b9c00640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc994074880 0x7fc994076d40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009ef0 tx=0x7fc9a80095e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:33.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.562+0000 7fc9b1600640 1 -- 172.21.15.122:0/3731554671 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc9a00c7020 con 0x7fc9b40b2e80 2024-09-17T05:36:33.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.734+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9b40c64c0 con 0x7fc9b40b2e80 2024-09-17T05:36:33.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.734+0000 7fc9b1600640 1 -- 172.21.15.122:0/3731554671 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16839 (secure 0 0 0) 0x7fc9a0016410 con 0x7fc9b40b2e80 2024-09-17T05:36:33.730 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:33.730 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":102,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:36:32.388490+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:36:30.080602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:36:31.349633+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc994074880 msgr2=0x7fc994076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc994074880 0x7fc994076d40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009ef0 tx=0x7fc9a80095e0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 msgr2=0x7fc9b40c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 0x7fc9b40c7a70 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009790 tx=0x7fc9a00a7b80 comp rx=0 tx=0).stop 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 shutdown_connections 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc994074880 0x7fc994076d40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9b415f490 0x7fc9b40b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9b40b2e80 0x7fc9b40c7a70 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 --2- 172.21.15.122:0/3731554671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9b40b1f40 0x7fc9b40c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 >> 172.21.15.122:0/3731554671 conn(0x7fc9b40d1e30 msgr2=0x7fc9b415ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 shutdown_connections 2024-09-17T05:36:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.738+0000 7fc9bc600640 1 -- 172.21.15.122:0/3731554671 wait complete. 2024-09-17T05:36:33.751 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:36:33.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 -- 172.21.15.122:0/759973821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580cff20 msgr2=0x7fa8580d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 --2- 172.21.15.122:0/759973821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580cff20 0x7fa8580d0300 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa8580d0d80 tx=0x7fa84c01f0b0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 -- 172.21.15.122:0/759973821 shutdown_connections 2024-09-17T05:36:33.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 --2- 172.21.15.122:0/759973821 >> v2:172.21.15.173:3300/0 conn(0x7fa8580b14c0 0x7fa8580b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 --2- 172.21.15.122:0/759973821 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8580b2d80 0x7fa8580d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 --2- 172.21.15.122:0/759973821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580cff20 0x7fa8580d0300 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 -- 172.21.15.122:0/759973821 >> 172.21.15.122:0/759973821 conn(0x7fa8580d1dc0 msgr2=0x7fa8580d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:33.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 -- 172.21.15.122:0/759973821 shutdown_connections 2024-09-17T05:36:33.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.938+0000 7fa860000640 1 -- 172.21.15.122:0/759973821 wait complete. 2024-09-17T05:36:33.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 Processor -- start 2024-09-17T05:36:33.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 -- start start 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580b14c0 0x7fa8580cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8580b2d80 0x7fa8581f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8580cff20 0x7fa8580b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa858160110 con 0x7fa8580b14c0 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa85815ff90 con 0x7fa8580b2d80 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa858160290 con 0x7fa8580cff20 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580b14c0 0x7fa8580cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580b14c0 0x7fa8580cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44554/0 (socket says 172.21.15.122:44554) 2024-09-17T05:36:33.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 -- 172.21.15.122:0/3456760501 learned_addr learned my addr 172.21.15.122:0/3456760501 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85cc00640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8580b2d80 0x7fa8581f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85e000640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8580cff20 0x7fa8580b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 -- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8580cff20 msgr2=0x7fa8580b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8580cff20 0x7fa8580b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 -- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8580b2d80 msgr2=0x7fa8581f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8580b2d80 0x7fa8581f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 -- 172.21.15.122:0/3456760501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581f4fa0 con 0x7fa8580b14c0 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa85d600640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580b14c0 0x7fa8580cb540 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0b3040 tx=0x7fa84c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa855600640 1 -- 172.21.15.122:0/3456760501 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c0b5040 con 0x7fa8580b14c0 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa855600640 1 -- 172.21.15.122:0/3456760501 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa84c005230 con 0x7fa8580b14c0 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa855600640 1 -- 172.21.15.122:0/3456760501 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c0046d0 con 0x7fa8580b14c0 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581f5280 con 0x7fa8580b14c0 2024-09-17T05:36:33.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8580cba80 con 0x7fa8580b14c0 2024-09-17T05:36:33.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.942+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8580b1940 con 0x7fa8580b14c0 2024-09-17T05:36:33.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.946+0000 7fa855600640 1 -- 172.21.15.122:0/3456760501 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fa84c004870 con 0x7fa8580b14c0 2024-09-17T05:36:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.946+0000 7fa855600640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa838074880 0x7fa838076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.946+0000 7fa855600640 1 -- 172.21.15.122:0/3456760501 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5765+0+0 (secure 0 0 0) 0x7fa84c12fff0 con 0x7fa8580b14c0 2024-09-17T05:36:33.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.946+0000 7fa85cc00640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa838074880 0x7fa838076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:33.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.946+0000 7fa85cc00640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa838074880 0x7fa838076d40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa858073ac0 tx=0x7fa84400b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:33.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:33.950+0000 7fa855600640 1 -- 172.21.15.122:0/3456760501 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa84c018c10 con 0x7fa8580b14c0 2024-09-17T05:36:34.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:34.166+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa8580d0a50 con 0x7fa8580b14c0 2024-09-17T05:36:34.969 INFO:tasks.ceph.mds.c.smithi122.stderr:2024-09-17T05:36:34.974+0000 7fbcf0200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:36:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.030+0000 7fa855600640 1 -- 172.21.15.122:0/3456760501 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) ==== 118+0+0 (secure 0 0 0) 0x7fa84c0fc430 con 0x7fa8580b14c0 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa838074880 msgr2=0x7fa838076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa838074880 0x7fa838076d40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa858073ac0 tx=0x7fa84400b390 comp rx=0 tx=0).stop 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580b14c0 msgr2=0x7fa8580cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580b14c0 0x7fa8580cb540 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0b3040 tx=0x7fa84c005780 comp rx=0 tx=0).stop 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 shutdown_connections 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa838074880 0x7fa838076d40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8580cff20 0x7fa8580b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8580b2d80 0x7fa8581f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 --2- 172.21.15.122:0/3456760501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8580b14c0 0x7fa8580cb540 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 >> 172.21.15.122:0/3456760501 conn(0x7fa8580d1dc0 msgr2=0x7fa8580aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 shutdown_connections 2024-09-17T05:36:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.034+0000 7fa860000640 1 -- 172.21.15.122:0/3456760501 wait complete. 2024-09-17T05:36:35.053 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:36:35.053 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:36:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.206+0000 7f25bf000640 1 -- 172.21.15.122:0/156043402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b80b42d0 msgr2=0x7f25b80b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.206+0000 7f25bf000640 1 --2- 172.21.15.122:0/156043402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b80b42d0 0x7f25b80b4760 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f25b8069800 tx=0x7f25b001cb80 comp rx=0 tx=0).stop 2024-09-17T05:36:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.206+0000 7f25bf000640 1 -- 172.21.15.122:0/156043402 shutdown_connections 2024-09-17T05:36:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.206+0000 7f25bf000640 1 --2- 172.21.15.122:0/156043402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b80b42d0 0x7f25b80b4760 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.206+0000 7f25bf000640 1 --2- 172.21.15.122:0/156043402 >> v2:172.21.15.173:3300/0 conn(0x7f25b815a720 0x7f25b806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.206+0000 7f25bf000640 1 -- 172.21.15.122:0/156043402 >> 172.21.15.122:0/156043402 conn(0x7f25b80b18f0 msgr2=0x7f25b80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.206+0000 7f25bf000640 1 -- 172.21.15.122:0/156043402 shutdown_connections 2024-09-17T05:36:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.206+0000 7f25bf000640 1 -- 172.21.15.122:0/156043402 wait complete. 2024-09-17T05:36:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 Processor -- start 2024-09-17T05:36:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 -- start start 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b80b42d0 0x7f25b80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b80cd3c0 0x7f25b8143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b815a720 0x7f25b80acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25b8167c40 con 0x7f25b815a720 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f25b80cee70 con 0x7f25b80cd3c0 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f25b8167dc0 con 0x7f25b80b42d0 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b815a720 0x7f25b80acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b815a720 0x7f25b80acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44570/0 (socket says 172.21.15.122:44570) 2024-09-17T05:36:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 -- 172.21.15.122:0/1001037868 learned_addr learned my addr 172.21.15.122:0/1001037868 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25afe00640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b80cd3c0 0x7f25b8143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 -- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b80b42d0 msgr2=0x7f25b80aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25b7e00640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b80b42d0 0x7f25b80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b80b42d0 0x7f25b80aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 -- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b80cd3c0 msgr2=0x7f25b8143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b80cd3c0 0x7f25b8143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 -- 172.21.15.122:0/1001037868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b8143b60 con 0x7f25b815a720 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25afe00640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b80cd3c0 0x7f25b8143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bd000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b815a720 0x7f25b80acd40 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f25b001c5f0 tx=0x7f25b0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25b4c00640 1 -- 172.21.15.122:0/1001037868 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b00b2040 con 0x7f25b815a720 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25b4c00640 1 -- 172.21.15.122:0/1001037868 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b00bb070 con 0x7f25b815a720 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b80ad280 con 0x7f25b815a720 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25b4c00640 1 -- 172.21.15.122:0/1001037868 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0004330 con 0x7f25b815a720 2024-09-17T05:36:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b80ad800 con 0x7f25b815a720 2024-09-17T05:36:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.210+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b8069410 con 0x7f25b815a720 2024-09-17T05:36:35.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.214+0000 7f25b4c00640 1 -- 172.21.15.122:0/1001037868 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f25b00044d0 con 0x7f25b815a720 2024-09-17T05:36:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.214+0000 7f25b4c00640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2594074880 0x7f2594076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.214+0000 7f25b4c00640 1 -- 172.21.15.122:0/1001037868 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f25b012d8f0 con 0x7f25b815a720 2024-09-17T05:36:35.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.214+0000 7f25b7e00640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2594074880 0x7f2594076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:35.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.214+0000 7f25b7e00640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2594074880 0x7f2594076d40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f25a80058d0 tx=0x7f25a8005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:35.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.218+0000 7f25b4c00640 1 -- 172.21.15.122:0/1001037868 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f25b0016770 con 0x7f25b815a720 2024-09-17T05:36:35.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:35.418+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f25b80b4760 con 0x7f25b815a720 2024-09-17T05:36:36.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.030+0000 7f25b4c00640 1 -- 172.21.15.122:0/1001037868 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) ==== 122+0+0 (secure 0 0 0) 0x7f25b00fa4a0 con 0x7f25b815a720 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2594074880 msgr2=0x7f2594076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2594074880 0x7f2594076d40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f25a80058d0 tx=0x7f25a8005860 comp rx=0 tx=0).stop 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b815a720 msgr2=0x7f25b80acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b815a720 0x7f25b80acd40 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f25b001c5f0 tx=0x7f25b0005b00 comp rx=0 tx=0).stop 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 shutdown_connections 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2594074880 0x7f2594076d40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25b815a720 0x7f25b80acd40 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25b80cd3c0 0x7f25b8143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 --2- 172.21.15.122:0/1001037868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25b80b42d0 0x7f25b80aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 >> 172.21.15.122:0/1001037868 conn(0x7f25b80b18f0 msgr2=0x7f25b80b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:36.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 shutdown_connections 2024-09-17T05:36:36.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.034+0000 7f25bf000640 1 -- 172.21.15.122:0/1001037868 wait complete. 2024-09-17T05:36:36.053 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:36:36.053 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:36:36.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- 172.21.15.122:0/1796495120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64980051d0 msgr2=0x7f649800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 --2- 172.21.15.122:0/1796495120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64980051d0 0x7f649800b9d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009ee0 tx=0x7f64ac01d630 comp rx=0 tx=0).stop 2024-09-17T05:36:36.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- 172.21.15.122:0/1796495120 shutdown_connections 2024-09-17T05:36:36.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 --2- 172.21.15.122:0/1796495120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64980051d0 0x7f649800b9d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- 172.21.15.122:0/1796495120 >> 172.21.15.122:0/1796495120 conn(0x7f64b406dd30 msgr2=0x7f64b406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:36.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- 172.21.15.122:0/1796495120 shutdown_connections 2024-09-17T05:36:36.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- 172.21.15.122:0/1796495120 wait complete. 2024-09-17T05:36:36.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 Processor -- start 2024-09-17T05:36:36.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- start start 2024-09-17T05:36:36.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64980051d0 0x7f64b4145170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b41448a0 0x7f64b40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64b40b19d0 0x7f64b409d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f64b40ae860 con 0x7f64b41448a0 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f64b40ae6e0 con 0x7f64980051d0 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64ba400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f64b40ae9e0 con 0x7f64b40b19d0 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64aaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b41448a0 0x7f64b40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64aaa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b41448a0 0x7f64b40b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44594/0 (socket says 172.21.15.122:44594) 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64aaa00640 1 -- 172.21.15.122:0/2987000238 learned_addr learned my addr 172.21.15.122:0/2987000238 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64b3400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64980051d0 0x7f64b4145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.202+0000 7f64b3e00640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64b40b19d0 0x7f64b409d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64aaa00640 1 -- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64b40b19d0 msgr2=0x7f64b409d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64aaa00640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64b40b19d0 0x7f64b409d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64aaa00640 1 -- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64980051d0 msgr2=0x7f64b4145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64aaa00640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64980051d0 0x7f64b4145170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64aaa00640 1 -- 172.21.15.122:0/2987000238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b41f0570 con 0x7f64b41448a0 2024-09-17T05:36:36.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64b3400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64980051d0 0x7f64b4145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:36.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64aaa00640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b41448a0 0x7f64b40b1490 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f649c004e00 tx=0x7f649c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:36.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64abe00640 1 -- 172.21.15.122:0/2987000238 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c013b70 con 0x7f64b41448a0 2024-09-17T05:36:36.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64abe00640 1 -- 172.21.15.122:0/2987000238 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f649c017030 con 0x7f64b41448a0 2024-09-17T05:36:36.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b41f0850 con 0x7f64b41448a0 2024-09-17T05:36:36.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64abe00640 1 -- 172.21.15.122:0/2987000238 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c01c680 con 0x7f64b41448a0 2024-09-17T05:36:36.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b41f0e10 con 0x7f64b41448a0 2024-09-17T05:36:36.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b40b4940 con 0x7f64b41448a0 2024-09-17T05:36:36.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64abe00640 1 -- 172.21.15.122:0/2987000238 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f649c01b3d0 con 0x7f64b41448a0 2024-09-17T05:36:36.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.206+0000 7f64abe00640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6490074880 0x7f6490076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.210+0000 7f64abe00640 1 -- 172.21.15.122:0/2987000238 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f649c097510 con 0x7f64b41448a0 2024-09-17T05:36:36.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.210+0000 7f64b3400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6490074880 0x7f6490076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.210+0000 7f64b3400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6490074880 0x7f6490076d40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f64a4001870 tx=0x7f64a40011c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:36.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.210+0000 7f64abe00640 1 -- 172.21.15.122:0/2987000238 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f649c013e20 con 0x7f64b41448a0 2024-09-17T05:36:36.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.414+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64b406d150 con 0x7f64b41448a0 2024-09-17T05:36:36.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.414+0000 7f64abe00640 1 -- 172.21.15.122:0/2987000238 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5237 (secure 0 0 0) 0x7f649c0639d0 con 0x7f64b41448a0 2024-09-17T05:36:36.411 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:36.412 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":66,"btime":"2024-09-17T05:36:36:034083+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1758247367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1758247367},{"type":"v1","addr":"172.21.15.173:6836","nonce":1758247367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2422841556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2422841556},{"type":"v1","addr":"172.21.15.173:6837","nonce":2422841556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/4290616350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":4290616350},{"type":"v1","addr":"172.21.15.122:6834","nonce":4290616350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:36:32.388428+0000","modified":"2024-09-17T05:36:35.423704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2223381402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":2223381402},{"type":"v1","addr":"172.21.15.122:6835","nonce":2223381402}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T05:36:36.412 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 66 2024-09-17T05:36:36.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6490074880 msgr2=0x7f6490076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6490074880 0x7f6490076d40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f64a4001870 tx=0x7f64a40011c0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b41448a0 msgr2=0x7f64b40b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b41448a0 0x7f64b40b1490 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f649c004e00 tx=0x7f649c001640 comp rx=0 tx=0).stop 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 shutdown_connections 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6490074880 0x7f6490076d40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64b40b19d0 0x7f64b409d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b41448a0 0x7f64b40b1490 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 --2- 172.21.15.122:0/2987000238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64980051d0 0x7f64b4145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 >> 172.21.15.122:0/2987000238 conn(0x7f64b406dd30 msgr2=0x7f64b40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 shutdown_connections 2024-09-17T05:36:36.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.418+0000 7f64ba400640 1 -- 172.21.15.122:0/2987000238 wait complete. 2024-09-17T05:36:36.432 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:36:36.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 -- 172.21.15.122:0/1454399458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf815af60 msgr2=0x7f2cf80cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 --2- 172.21.15.122:0/1454399458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf815af60 0x7f2cf80cffc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009840 tx=0x7f2ce801c7f0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 -- 172.21.15.122:0/1454399458 shutdown_connections 2024-09-17T05:36:36.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 --2- 172.21.15.122:0/1454399458 >> v2:172.21.15.173:3300/0 conn(0x7f2cf8074330 0x7f2cf806f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 --2- 172.21.15.122:0/1454399458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf815af60 0x7f2cf80cffc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 -- 172.21.15.122:0/1454399458 >> 172.21.15.122:0/1454399458 conn(0x7f2cf80d0c90 msgr2=0x7f2cf80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:36.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 -- 172.21.15.122:0/1454399458 shutdown_connections 2024-09-17T05:36:36.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 -- 172.21.15.122:0/1454399458 wait complete. 2024-09-17T05:36:36.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 Processor -- start 2024-09-17T05:36:36.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.578+0000 7f2cfca00640 1 -- start start 2024-09-17T05:36:36.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf80737c0 0x7f2cf80c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cf8074330 0x7f2cf80a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cf815af60 0x7f2cf80b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf8167ab0 con 0x7f2cf8074330 2024-09-17T05:36:36.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf8167930 con 0x7f2cf815af60 2024-09-17T05:36:36.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf8167c30 con 0x7f2cf80737c0 2024-09-17T05:36:36.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cf8074330 0x7f2cf80a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cf8074330 0x7f2cf80a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44608/0 (socket says 172.21.15.122:44608) 2024-09-17T05:36:36.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 -- 172.21.15.122:0/89977282 learned_addr learned my addr 172.21.15.122:0/89977282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:36.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cf6800640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cf815af60 0x7f2cf80b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cf5e00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf80737c0 0x7f2cf80c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 -- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf80737c0 msgr2=0x7f2cf80c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf80737c0 0x7f2cf80c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 -- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cf815af60 msgr2=0x7f2cf80b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cf815af60 0x7f2cf80b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 -- 172.21.15.122:0/89977282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf80c86b0 con 0x7f2cf8074330 2024-09-17T05:36:36.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cf5e00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf80737c0 0x7f2cf80c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:36.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ced400640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cf8074330 0x7f2cf80a1800 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000cbb0 tx=0x7f2ce000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:36.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ceea00640 1 -- 172.21.15.122:0/89977282 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0016020 con 0x7f2cf8074330 2024-09-17T05:36:36.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ceea00640 1 -- 172.21.15.122:0/89977282 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ce000ab40 con 0x7f2cf8074330 2024-09-17T05:36:36.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf80c8990 con 0x7f2cf8074330 2024-09-17T05:36:36.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2ceea00640 1 -- 172.21.15.122:0/89977282 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0012420 con 0x7f2cf8074330 2024-09-17T05:36:36.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf80c8f50 con 0x7f2cf8074330 2024-09-17T05:36:36.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.582+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf806f9a0 con 0x7f2cf8074330 2024-09-17T05:36:36.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.586+0000 7f2ceea00640 1 -- 172.21.15.122:0/89977282 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f2ce0019020 con 0x7f2cf8074330 2024-09-17T05:36:36.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.586+0000 7f2ceea00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cd4074880 0x7f2cd4076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.586+0000 7f2ceea00640 1 -- 172.21.15.122:0/89977282 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f2ce0093150 con 0x7f2cf8074330 2024-09-17T05:36:36.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.586+0000 7f2cf5e00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cd4074880 0x7f2cd4076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.586+0000 7f2cf5e00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cd4074880 0x7f2cd4076d40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80af000 tx=0x7f2ce801b030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:36.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.590+0000 7f2ceea00640 1 -- 172.21.15.122:0/89977282 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2ce0020020 con 0x7f2cf8074330 2024-09-17T05:36:36.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.758+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cf80740a0 con 0x7f2cf8074330 2024-09-17T05:36:36.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.758+0000 7f2ceea00640 1 -- 172.21.15.122:0/89977282 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16826 (secure 0 0 0) 0x7f2ce005fd00 con 0x7f2cf8074330 2024-09-17T05:36:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:36.754 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":103,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:36:34.004572+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:36:30.080602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:36:31.349633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:36:36.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.758+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cd4074880 msgr2=0x7f2cd4076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.758+0000 7f2cfca00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cd4074880 0x7f2cd4076d40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80af000 tx=0x7f2ce801b030 comp rx=0 tx=0).stop 2024-09-17T05:36:36.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cf8074330 msgr2=0x7f2cf80a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cf8074330 0x7f2cf80a1800 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000cbb0 tx=0x7f2ce000cca0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 shutdown_connections 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2cd4074880 0x7f2cd4076d40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2cf815af60 0x7f2cf80b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cf8074330 0x7f2cf80a1800 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 --2- 172.21.15.122:0/89977282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2cf80737c0 0x7f2cf80c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 >> 172.21.15.122:0/89977282 conn(0x7f2cf80d0c90 msgr2=0x7f2cf80c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 shutdown_connections 2024-09-17T05:36:36.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.762+0000 7f2cfca00640 1 -- 172.21.15.122:0/89977282 wait complete. 2024-09-17T05:36:36.776 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:36:36.776 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:36:36.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 -- 172.21.15.122:0/3471347875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 msgr2=0x7fdcd806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 --2- 172.21.15.122:0/3471347875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 0x7fdcd806e7a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009c80 tx=0x7fdcc801cd70 comp rx=0 tx=0).stop 2024-09-17T05:36:36.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 -- 172.21.15.122:0/3471347875 shutdown_connections 2024-09-17T05:36:36.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 --2- 172.21.15.122:0/3471347875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdcd80cf1d0 0x7fdcd815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 --2- 172.21.15.122:0/3471347875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 0x7fdcd806e7a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 -- 172.21.15.122:0/3471347875 >> 172.21.15.122:0/3471347875 conn(0x7fdcd80b18f0 msgr2=0x7fdcd80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:36.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 -- 172.21.15.122:0/3471347875 shutdown_connections 2024-09-17T05:36:36.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 -- 172.21.15.122:0/3471347875 wait complete. 2024-09-17T05:36:36.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 Processor -- start 2024-09-17T05:36:36.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.922+0000 7fdcdce00640 1 -- start start 2024-09-17T05:36:36.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcd80cf1d0 0x7fdcd80b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 0x7fdcd8158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdcd81602c0 0x7fdcd8153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd8169720 con 0x7fdcd815f9a0 2024-09-17T05:36:36.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd81695a0 con 0x7fdcd80cf1d0 2024-09-17T05:36:36.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd81698a0 con 0x7fdcd81602c0 2024-09-17T05:36:36.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 0x7fdcd8158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 0x7fdcd8158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44624/0 (socket says 172.21.15.122:44624) 2024-09-17T05:36:36.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 -- 172.21.15.122:0/539408545 learned_addr learned my addr 172.21.15.122:0/539408545 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:36.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcd5e00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcd80cf1d0 0x7fdcd80b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcd6800640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdcd81602c0 0x7fdcd8153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 -- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdcd81602c0 msgr2=0x7fdcd8153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdcd81602c0 0x7fdcd8153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 -- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcd80cf1d0 msgr2=0x7fdcd80b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:36.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcd80cf1d0 0x7fdcd80b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:36.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 -- 172.21.15.122:0/539408545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd81f9240 con 0x7fdcd815f9a0 2024-09-17T05:36:36.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccd400640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 0x7fdcd8158c90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000d980 tx=0x7fdcc000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:36.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccea00640 1 -- 172.21.15.122:0/539408545 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0017070 con 0x7fdcd815f9a0 2024-09-17T05:36:36.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccea00640 1 -- 172.21.15.122:0/539408545 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcc0009d60 con 0x7fdcd815f9a0 2024-09-17T05:36:36.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd8153d50 con 0x7fdcd815f9a0 2024-09-17T05:36:36.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccea00640 1 -- 172.21.15.122:0/539408545 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0005280 con 0x7fdcd815f9a0 2024-09-17T05:36:36.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd81f88a0 con 0x7fdcd815f9a0 2024-09-17T05:36:36.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd8069410 con 0x7fdcd815f9a0 2024-09-17T05:36:36.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.926+0000 7fdccea00640 1 -- 172.21.15.122:0/539408545 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fdcc0005420 con 0x7fdcd815f9a0 2024-09-17T05:36:36.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.930+0000 7fdccea00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcb4074880 0x7fdcb4076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:36.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.930+0000 7fdccea00640 1 -- 172.21.15.122:0/539408545 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7fdcc0093140 con 0x7fdcd815f9a0 2024-09-17T05:36:36.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.930+0000 7fdcd5e00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcb4074880 0x7fdcb4076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:36.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.930+0000 7fdcd5e00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcb4074880 0x7fdcb4076d40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80af000 tx=0x7fdcc801b5b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:36.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:36.934+0000 7fdccea00640 1 -- 172.21.15.122:0/539408545 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdcc005f600 con 0x7fdcd815f9a0 2024-09-17T05:36:37.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.102+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdcd815fd80 con 0x7fdcb4074880 2024-09-17T05:36:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.110+0000 7fdccea00640 1 -- 172.21.15.122:0/539408545 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7fdcd815fd80 con 0x7fdcb4074880 2024-09-17T05:36:37.107 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:37.109 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:36:37.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.110+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcb4074880 msgr2=0x7fdcb4076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.110+0000 7fdcdce00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcb4074880 0x7fdcb4076d40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80af000 tx=0x7fdcc801b5b0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.110+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 msgr2=0x7fdcd8158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.110+0000 7fdcdce00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 0x7fdcd8158c90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000d980 tx=0x7fdcc000de40 comp rx=0 tx=0).stop 2024-09-17T05:36:37.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.114+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 shutdown_connections 2024-09-17T05:36:37.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.114+0000 7fdcdce00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdcb4074880 0x7fdcb4076d40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.114+0000 7fdcdce00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdcd81602c0 0x7fdcd8153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.114+0000 7fdcdce00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcd815f9a0 0x7fdcd8158c90 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.114+0000 7fdcdce00640 1 --2- 172.21.15.122:0/539408545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdcd80cf1d0 0x7fdcd80b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.114+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 >> 172.21.15.122:0/539408545 conn(0x7fdcd80b18f0 msgr2=0x7fdcd815a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:37.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.114+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 shutdown_connections 2024-09-17T05:36:37.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.114+0000 7fdcdce00640 1 -- 172.21.15.122:0/539408545 wait complete. 2024-09-17T05:36:37.128 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-17T05:36:36.690053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":833440,"kb_used_data":11040,"kb_used_omap":12,"kb_used_meta":822195,"kb_avail":754141280,"statfs":{"total":773094113280,"available":772240670720,"internally_reserved":0,"allocated":11304960,"data_stored":5553428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":841928246},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.710718"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364008+0000","last_change":"2024-09-17T05:36:33.025998+0000","last_active":"2024-09-17T05:36:34.364008+0000","last_peered":"2024-09-17T05:36:34.364008+0000","last_clean":"2024-09-17T05:36:34.364008+0000","last_became_active":"2024-09-17T05:36:33.025635+0000","last_became_peered":"2024-09-17T05:36:33.025635+0000","last_unstale":"2024-09-17T05:36:34.364008+0000","last_undegraded":"2024-09-17T05:36:34.364008+0000","last_fullsized":"2024-09-17T05:36:34.364008+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:11.430701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364039+0000","last_change":"2024-09-17T05:36:33.032232+0000","last_active":"2024-09-17T05:36:34.364039+0000","last_peered":"2024-09-17T05:36:34.364039+0000","last_clean":"2024-09-17T05:36:34.364039+0000","last_became_active":"2024-09-17T05:36:33.031848+0000","last_became_peered":"2024-09-17T05:36:33.031848+0000","last_unstale":"2024-09-17T05:36:34.364039+0000","last_undegraded":"2024-09-17T05:36:34.364039+0000","last_fullsized":"2024-09-17T05:36:34.364039+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:53.501709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028954+0000","last_change":"2024-09-17T05:36:33.037607+0000","last_active":"2024-09-17T05:36:34.028954+0000","last_peered":"2024-09-17T05:36:34.028954+0000","last_clean":"2024-09-17T05:36:34.028954+0000","last_became_active":"2024-09-17T05:36:33.037249+0000","last_became_peered":"2024-09-17T05:36:33.037249+0000","last_unstale":"2024-09-17T05:36:34.028954+0000","last_undegraded":"2024-09-17T05:36:34.028954+0000","last_fullsized":"2024-09-17T05:36:34.028954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:25.395395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363775+0000","last_change":"2024-09-17T05:36:33.034989+0000","last_active":"2024-09-17T05:36:34.363775+0000","last_peered":"2024-09-17T05:36:34.363775+0000","last_clean":"2024-09-17T05:36:34.363775+0000","last_became_active":"2024-09-17T05:36:33.034328+0000","last_became_peered":"2024-09-17T05:36:33.034328+0000","last_unstale":"2024-09-17T05:36:34.363775+0000","last_undegraded":"2024-09-17T05:36:34.363775+0000","last_fullsized":"2024-09-17T05:36:34.363775+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:47.756883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.030874+0000","last_change":"2024-09-17T05:36:33.022107+0000","last_active":"2024-09-17T05:36:33.030874+0000","last_peered":"2024-09-17T05:36:33.030874+0000","last_clean":"2024-09-17T05:36:33.030874+0000","last_became_active":"2024-09-17T05:36:33.021819+0000","last_became_peered":"2024-09-17T05:36:33.021819+0000","last_unstale":"2024-09-17T05:36:33.030874+0000","last_undegraded":"2024-09-17T05:36:33.030874+0000","last_fullsized":"2024-09-17T05:36:33.030874+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:47.934927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.030294+0000","last_change":"2024-09-17T05:36:33.017608+0000","last_active":"2024-09-17T05:36:33.030294+0000","last_peered":"2024-09-17T05:36:33.030294+0000","last_clean":"2024-09-17T05:36:33.030294+0000","last_became_active":"2024-09-17T05:36:33.017377+0000","last_became_peered":"2024-09-17T05:36:33.017377+0000","last_unstale":"2024-09-17T05:36:33.030294+0000","last_undegraded":"2024-09-17T05:36:33.030294+0000","last_fullsized":"2024-09-17T05:36:33.030294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:23.705153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027927+0000","last_change":"2024-09-17T05:36:33.016812+0000","last_active":"2024-09-17T05:36:33.027927+0000","last_peered":"2024-09-17T05:36:33.027927+0000","last_clean":"2024-09-17T05:36:33.027927+0000","last_became_active":"2024-09-17T05:36:33.016491+0000","last_became_peered":"2024-09-17T05:36:33.016491+0000","last_unstale":"2024-09-17T05:36:33.027927+0000","last_undegraded":"2024-09-17T05:36:33.027927+0000","last_fullsized":"2024-09-17T05:36:33.027927+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:39.487804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028228+0000","last_change":"2024-09-17T05:36:33.038714+0000","last_active":"2024-09-17T05:36:34.028228+0000","last_peered":"2024-09-17T05:36:34.028228+0000","last_clean":"2024-09-17T05:36:34.028228+0000","last_became_active":"2024-09-17T05:36:33.038440+0000","last_became_peered":"2024-09-17T05:36:33.038440+0000","last_unstale":"2024-09-17T05:36:34.028228+0000","last_undegraded":"2024-09-17T05:36:34.028228+0000","last_fullsized":"2024-09-17T05:36:34.028228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:22.355517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.047934+0000","last_change":"2024-09-17T05:36:33.047158+0000","last_active":"2024-09-17T05:36:33.047934+0000","last_peered":"2024-09-17T05:36:33.047934+0000","last_clean":"2024-09-17T05:36:33.047934+0000","last_became_active":"2024-09-17T05:36:33.046847+0000","last_became_peered":"2024-09-17T05:36:33.046847+0000","last_unstale":"2024-09-17T05:36:33.047934+0000","last_undegraded":"2024-09-17T05:36:33.047934+0000","last_fullsized":"2024-09-17T05:36:33.047934+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:58.063492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.008726+0000","last_change":"2024-09-17T05:36:33.034774+0000","last_active":"2024-09-17T05:36:34.008726+0000","last_peered":"2024-09-17T05:36:34.008726+0000","last_clean":"2024-09-17T05:36:34.008726+0000","last_became_active":"2024-09-17T05:36:33.034521+0000","last_became_peered":"2024-09-17T05:36:33.034521+0000","last_unstale":"2024-09-17T05:36:34.008726+0000","last_undegraded":"2024-09-17T05:36:34.008726+0000","last_fullsized":"2024-09-17T05:36:34.008726+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:18.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027962+0000","last_change":"2024-09-17T05:36:33.017053+0000","last_active":"2024-09-17T05:36:33.027962+0000","last_peered":"2024-09-17T05:36:33.027962+0000","last_clean":"2024-09-17T05:36:33.027962+0000","last_became_active":"2024-09-17T05:36:33.016787+0000","last_became_peered":"2024-09-17T05:36:33.016787+0000","last_unstale":"2024-09-17T05:36:33.027962+0000","last_undegraded":"2024-09-17T05:36:33.027962+0000","last_fullsized":"2024-09-17T05:36:33.027962+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:38.740525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.037722+0000","last_change":"2024-09-17T05:36:33.036410+0000","last_active":"2024-09-17T05:36:33.037722+0000","last_peered":"2024-09-17T05:36:33.037722+0000","last_clean":"2024-09-17T05:36:33.037722+0000","last_became_active":"2024-09-17T05:36:33.036111+0000","last_became_peered":"2024-09-17T05:36:33.036111+0000","last_unstale":"2024-09-17T05:36:33.037722+0000","last_undegraded":"2024-09-17T05:36:33.037722+0000","last_fullsized":"2024-09-17T05:36:33.037722+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:31.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010497+0000","last_change":"2024-09-17T05:36:33.017813+0000","last_active":"2024-09-17T05:36:34.010497+0000","last_peered":"2024-09-17T05:36:34.010497+0000","last_clean":"2024-09-17T05:36:34.010497+0000","last_became_active":"2024-09-17T05:36:33.016888+0000","last_became_peered":"2024-09-17T05:36:33.016888+0000","last_unstale":"2024-09-17T05:36:34.010497+0000","last_undegraded":"2024-09-17T05:36:34.010497+0000","last_fullsized":"2024-09-17T05:36:34.010497+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:25.030272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363646+0000","last_change":"2024-09-17T05:36:33.030560+0000","last_active":"2024-09-17T05:36:34.363646+0000","last_peered":"2024-09-17T05:36:34.363646+0000","last_clean":"2024-09-17T05:36:34.363646+0000","last_became_active":"2024-09-17T05:36:33.026357+0000","last_became_peered":"2024-09-17T05:36:33.026357+0000","last_unstale":"2024-09-17T05:36:34.363646+0000","last_undegraded":"2024-09-17T05:36:34.363646+0000","last_fullsized":"2024-09-17T05:36:34.363646+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:18.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363350+0000","last_change":"2024-09-17T05:36:33.045288+0000","last_active":"2024-09-17T05:36:34.363350+0000","last_peered":"2024-09-17T05:36:34.363350+0000","last_clean":"2024-09-17T05:36:34.363350+0000","last_became_active":"2024-09-17T05:36:33.045008+0000","last_became_peered":"2024-09-17T05:36:33.045008+0000","last_unstale":"2024-09-17T05:36:34.363350+0000","last_undegraded":"2024-09-17T05:36:34.363350+0000","last_fullsized":"2024-09-17T05:36:34.363350+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:39.180523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.030464+0000","last_change":"2024-09-17T05:36:33.027755+0000","last_active":"2024-09-17T05:36:33.030464+0000","last_peered":"2024-09-17T05:36:33.030464+0000","last_clean":"2024-09-17T05:36:33.030464+0000","last_became_active":"2024-09-17T05:36:33.024885+0000","last_became_peered":"2024-09-17T05:36:33.024885+0000","last_unstale":"2024-09-17T05:36:33.030464+0000","last_undegraded":"2024-09-17T05:36:33.030464+0000","last_fullsized":"2024-09-17T05:36:33.030464+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:50.481517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.044133+0000","last_change":"2024-09-17T05:36:33.041006+0000","last_active":"2024-09-17T05:36:33.044133+0000","last_peered":"2024-09-17T05:36:33.044133+0000","last_clean":"2024-09-17T05:36:33.044133+0000","last_became_active":"2024-09-17T05:36:33.040735+0000","last_became_peered":"2024-09-17T05:36:33.040735+0000","last_unstale":"2024-09-17T05:36:33.044133+0000","last_undegraded":"2024-09-17T05:36:33.044133+0000","last_fullsized":"2024-09-17T05:36:33.044133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:29.952722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364043+0000","last_change":"2024-09-17T05:36:33.046502+0000","last_active":"2024-09-17T05:36:34.364043+0000","last_peered":"2024-09-17T05:36:34.364043+0000","last_clean":"2024-09-17T05:36:34.364043+0000","last_became_active":"2024-09-17T05:36:33.045916+0000","last_became_peered":"2024-09-17T05:36:33.045916+0000","last_unstale":"2024-09-17T05:36:34.364043+0000","last_undegraded":"2024-09-17T05:36:34.364043+0000","last_fullsized":"2024-09-17T05:36:34.364043+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:31.764744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.043970+0000","last_change":"2024-09-17T05:36:33.039401+0000","last_active":"2024-09-17T05:36:33.043970+0000","last_peered":"2024-09-17T05:36:33.043970+0000","last_clean":"2024-09-17T05:36:33.043970+0000","last_became_active":"2024-09-17T05:36:33.039103+0000","last_became_peered":"2024-09-17T05:36:33.039103+0000","last_unstale":"2024-09-17T05:36:33.043970+0000","last_undegraded":"2024-09-17T05:36:33.043970+0000","last_fullsized":"2024-09-17T05:36:33.043970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:04.543809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.018152+0000","last_change":"2024-09-17T05:36:33.015241+0000","last_active":"2024-09-17T05:36:33.018152+0000","last_peered":"2024-09-17T05:36:33.018152+0000","last_clean":"2024-09-17T05:36:33.018152+0000","last_became_active":"2024-09-17T05:36:33.014894+0000","last_became_peered":"2024-09-17T05:36:33.014894+0000","last_unstale":"2024-09-17T05:36:33.018152+0000","last_undegraded":"2024-09-17T05:36:33.018152+0000","last_fullsized":"2024-09-17T05:36:33.018152+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:28.250794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028667+0000","last_change":"2024-09-17T05:36:33.041468+0000","last_active":"2024-09-17T05:36:34.028667+0000","last_peered":"2024-09-17T05:36:34.028667+0000","last_clean":"2024-09-17T05:36:34.028667+0000","last_became_active":"2024-09-17T05:36:33.041122+0000","last_became_peered":"2024-09-17T05:36:33.041122+0000","last_unstale":"2024-09-17T05:36:34.028667+0000","last_undegraded":"2024-09-17T05:36:34.028667+0000","last_fullsized":"2024-09-17T05:36:34.028667+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:23.082151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028669+0000","last_change":"2024-09-17T05:36:33.037082+0000","last_active":"2024-09-17T05:36:34.028669+0000","last_peered":"2024-09-17T05:36:34.028669+0000","last_clean":"2024-09-17T05:36:34.028669+0000","last_became_active":"2024-09-17T05:36:33.036668+0000","last_became_peered":"2024-09-17T05:36:33.036668+0000","last_unstale":"2024-09-17T05:36:34.028669+0000","last_undegraded":"2024-09-17T05:36:34.028669+0000","last_fullsized":"2024-09-17T05:36:34.028669+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:34.458277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363380+0000","last_change":"2024-09-17T05:36:33.024065+0000","last_active":"2024-09-17T05:36:34.363380+0000","last_peered":"2024-09-17T05:36:34.363380+0000","last_clean":"2024-09-17T05:36:34.363380+0000","last_became_active":"2024-09-17T05:36:33.023695+0000","last_became_peered":"2024-09-17T05:36:33.023695+0000","last_unstale":"2024-09-17T05:36:34.363380+0000","last_undegraded":"2024-09-17T05:36:34.363380+0000","last_fullsized":"2024-09-17T05:36:34.363380+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:43.515827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027730+0000","last_change":"2024-09-17T05:36:33.025699+0000","last_active":"2024-09-17T05:36:33.027730+0000","last_peered":"2024-09-17T05:36:33.027730+0000","last_clean":"2024-09-17T05:36:33.027730+0000","last_became_active":"2024-09-17T05:36:33.025447+0000","last_became_peered":"2024-09-17T05:36:33.025447+0000","last_unstale":"2024-09-17T05:36:33.027730+0000","last_undegraded":"2024-09-17T05:36:33.027730+0000","last_fullsized":"2024-09-17T05:36:33.027730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:35:10.636607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.008767+0000","last_change":"2024-09-17T05:36:33.034033+0000","last_active":"2024-09-17T05:36:34.008767+0000","last_peered":"2024-09-17T05:36:34.008767+0000","last_clean":"2024-09-17T05:36:34.008767+0000","last_became_active":"2024-09-17T05:36:33.033750+0000","last_became_peered":"2024-09-17T05:36:33.033750+0000","last_unstale":"2024-09-17T05:36:34.008767+0000","last_undegraded":"2024-09-17T05:36:34.008767+0000","last_fullsized":"2024-09-17T05:36:34.008767+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:27.635660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010651+0000","last_change":"2024-09-17T05:36:33.017598+0000","last_active":"2024-09-17T05:36:34.010651+0000","last_peered":"2024-09-17T05:36:34.010651+0000","last_clean":"2024-09-17T05:36:34.010651+0000","last_became_active":"2024-09-17T05:36:33.016883+0000","last_became_peered":"2024-09-17T05:36:33.016883+0000","last_unstale":"2024-09-17T05:36:34.010651+0000","last_undegraded":"2024-09-17T05:36:34.010651+0000","last_fullsized":"2024-09-17T05:36:34.010651+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:52.311207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.023555+0000","last_change":"2024-09-17T05:36:33.017635+0000","last_active":"2024-09-17T05:36:33.023555+0000","last_peered":"2024-09-17T05:36:33.023555+0000","last_clean":"2024-09-17T05:36:33.023555+0000","last_became_active":"2024-09-17T05:36:33.017336+0000","last_became_peered":"2024-09-17T05:36:33.017336+0000","last_unstale":"2024-09-17T05:36:33.023555+0000","last_undegraded":"2024-09-17T05:36:33.023555+0000","last_fullsized":"2024-09-17T05:36:33.023555+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:46.993290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.040622+0000","last_change":"2024-09-17T05:36:33.039251+0000","last_active":"2024-09-17T05:36:33.040622+0000","last_peered":"2024-09-17T05:36:33.040622+0000","last_clean":"2024-09-17T05:36:33.040622+0000","last_became_active":"2024-09-17T05:36:33.038875+0000","last_became_peered":"2024-09-17T05:36:33.038875+0000","last_unstale":"2024-09-17T05:36:33.040622+0000","last_undegraded":"2024-09-17T05:36:33.040622+0000","last_fullsized":"2024-09-17T05:36:33.040622+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:51.232824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010492+0000","last_change":"2024-09-17T05:36:33.018822+0000","last_active":"2024-09-17T05:36:34.010492+0000","last_peered":"2024-09-17T05:36:34.010492+0000","last_clean":"2024-09-17T05:36:34.010492+0000","last_became_active":"2024-09-17T05:36:33.018480+0000","last_became_peered":"2024-09-17T05:36:33.018480+0000","last_unstale":"2024-09-17T05:36:34.010492+0000","last_undegraded":"2024-09-17T05:36:34.010492+0000","last_fullsized":"2024-09-17T05:36:34.010492+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:26.901434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010564+0000","last_change":"2024-09-17T05:36:33.033972+0000","last_active":"2024-09-17T05:36:34.010564+0000","last_peered":"2024-09-17T05:36:34.010564+0000","last_clean":"2024-09-17T05:36:34.010564+0000","last_became_active":"2024-09-17T05:36:33.033712+0000","last_became_peered":"2024-09-17T05:36:33.033712+0000","last_unstale":"2024-09-17T05:36:34.010564+0000","last_undegraded":"2024-09-17T05:36:34.010564+0000","last_fullsized":"2024-09-17T05:36:34.010564+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:04.060279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028355+0000","last_change":"2024-09-17T05:36:33.032413+0000","last_active":"2024-09-17T05:36:34.028355+0000","last_peered":"2024-09-17T05:36:34.028355+0000","last_clean":"2024-09-17T05:36:34.028355+0000","last_became_active":"2024-09-17T05:36:33.031075+0000","last_became_peered":"2024-09-17T05:36:33.031075+0000","last_unstale":"2024-09-17T05:36:34.028355+0000","last_undegraded":"2024-09-17T05:36:34.028355+0000","last_fullsized":"2024-09-17T05:36:34.028355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:38.726170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363250+0000","last_change":"2024-09-17T05:36:33.029679+0000","last_active":"2024-09-17T05:36:34.363250+0000","last_peered":"2024-09-17T05:36:34.363250+0000","last_clean":"2024-09-17T05:36:34.363250+0000","last_became_active":"2024-09-17T05:36:33.029376+0000","last_became_peered":"2024-09-17T05:36:33.029376+0000","last_unstale":"2024-09-17T05:36:34.363250+0000","last_undegraded":"2024-09-17T05:36:34.363250+0000","last_fullsized":"2024-09-17T05:36:34.363250+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:06.167690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.039319+0000","last_change":"2024-09-17T05:36:33.033905+0000","last_active":"2024-09-17T05:36:33.039319+0000","last_peered":"2024-09-17T05:36:33.039319+0000","last_clean":"2024-09-17T05:36:33.039319+0000","last_became_active":"2024-09-17T05:36:33.033280+0000","last_became_peered":"2024-09-17T05:36:33.033280+0000","last_unstale":"2024-09-17T05:36:33.039319+0000","last_undegraded":"2024-09-17T05:36:33.039319+0000","last_fullsized":"2024-09-17T05:36:33.039319+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:06.896905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"102'4","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.011198+0000","last_change":"2024-09-17T05:36:31.995389+0000","last_active":"2024-09-17T05:36:34.011198+0000","last_peered":"2024-09-17T05:36:34.011198+0000","last_clean":"2024-09-17T05:36:34.011198+0000","last_became_active":"2024-09-17T05:36:31.994467+0000","last_became_peered":"2024-09-17T05:36:31.994467+0000","last_unstale":"2024-09-17T05:36:34.011198+0000","last_undegraded":"2024-09-17T05:36:34.011198+0000","last_fullsized":"2024-09-17T05:36:34.011198+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:05.885445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.011081+0000","last_change":"2024-09-17T05:36:33.019743+0000","last_active":"2024-09-17T05:36:34.011081+0000","last_peered":"2024-09-17T05:36:34.011081+0000","last_clean":"2024-09-17T05:36:34.011081+0000","last_became_active":"2024-09-17T05:36:33.019388+0000","last_became_peered":"2024-09-17T05:36:33.019388+0000","last_unstale":"2024-09-17T05:36:34.011081+0000","last_undegraded":"2024-09-17T05:36:34.011081+0000","last_fullsized":"2024-09-17T05:36:34.011081+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:25.468745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"102'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.039829+0000","last_change":"2024-09-17T05:36:32.001502+0000","last_active":"2024-09-17T05:36:33.039829+0000","last_peered":"2024-09-17T05:36:33.039829+0000","last_clean":"2024-09-17T05:36:33.039829+0000","last_became_active":"2024-09-17T05:36:32.000908+0000","last_became_peered":"2024-09-17T05:36:32.000908+0000","last_unstale":"2024-09-17T05:36:33.039829+0000","last_undegraded":"2024-09-17T05:36:33.039829+0000","last_fullsized":"2024-09-17T05:36:33.039829+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:24.664804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363894+0000","last_change":"2024-09-17T05:36:33.034691+0000","last_active":"2024-09-17T05:36:34.363894+0000","last_peered":"2024-09-17T05:36:34.363894+0000","last_clean":"2024-09-17T05:36:34.363894+0000","last_became_active":"2024-09-17T05:36:33.034015+0000","last_became_peered":"2024-09-17T05:36:33.034015+0000","last_unstale":"2024-09-17T05:36:34.363894+0000","last_undegraded":"2024-09-17T05:36:34.363894+0000","last_fullsized":"2024-09-17T05:36:34.363894+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:27.635660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.029079+0000","last_change":"2024-09-17T05:36:32.003003+0000","last_active":"2024-09-17T05:36:34.029079+0000","last_peered":"2024-09-17T05:36:34.029079+0000","last_clean":"2024-09-17T05:36:34.029079+0000","last_became_active":"2024-09-17T05:36:32.002474+0000","last_became_peered":"2024-09-17T05:36:32.002474+0000","last_unstale":"2024-09-17T05:36:34.029079+0000","last_undegraded":"2024-09-17T05:36:34.029079+0000","last_fullsized":"2024-09-17T05:36:34.029079+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:38.440356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.008736+0000","last_change":"2024-09-17T05:36:33.027677+0000","last_active":"2024-09-17T05:36:34.008736+0000","last_peered":"2024-09-17T05:36:34.008736+0000","last_clean":"2024-09-17T05:36:34.008736+0000","last_became_active":"2024-09-17T05:36:33.025497+0000","last_became_peered":"2024-09-17T05:36:33.025497+0000","last_unstale":"2024-09-17T05:36:34.008736+0000","last_undegraded":"2024-09-17T05:36:34.008736+0000","last_fullsized":"2024-09-17T05:36:34.008736+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:34.582552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363923+0000","last_change":"2024-09-17T05:36:32.006528+0000","last_active":"2024-09-17T05:36:34.363923+0000","last_peered":"2024-09-17T05:36:34.363923+0000","last_clean":"2024-09-17T05:36:34.363923+0000","last_became_active":"2024-09-17T05:36:32.006254+0000","last_became_peered":"2024-09-17T05:36:32.006254+0000","last_unstale":"2024-09-17T05:36:34.363923+0000","last_undegraded":"2024-09-17T05:36:34.363923+0000","last_fullsized":"2024-09-17T05:36:34.363923+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:31.804208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027595+0000","last_change":"2024-09-17T05:36:33.019304+0000","last_active":"2024-09-17T05:36:33.027595+0000","last_peered":"2024-09-17T05:36:33.027595+0000","last_clean":"2024-09-17T05:36:33.027595+0000","last_became_active":"2024-09-17T05:36:33.019012+0000","last_became_peered":"2024-09-17T05:36:33.019012+0000","last_unstale":"2024-09-17T05:36:33.027595+0000","last_undegraded":"2024-09-17T05:36:33.027595+0000","last_fullsized":"2024-09-17T05:36:33.027595+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:13.449104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363331+0000","last_change":"2024-09-17T05:36:32.008081+0000","last_active":"2024-09-17T05:36:34.363331+0000","last_peered":"2024-09-17T05:36:34.363331+0000","last_clean":"2024-09-17T05:36:34.363331+0000","last_became_active":"2024-09-17T05:36:32.007783+0000","last_became_peered":"2024-09-17T05:36:32.007783+0000","last_unstale":"2024-09-17T05:36:34.363331+0000","last_undegraded":"2024-09-17T05:36:34.363331+0000","last_fullsized":"2024-09-17T05:36:34.363331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:03.981516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033082+0000","last_change":"2024-09-17T05:36:33.031090+0000","last_active":"2024-09-17T05:36:33.033082+0000","last_peered":"2024-09-17T05:36:33.033082+0000","last_clean":"2024-09-17T05:36:33.033082+0000","last_became_active":"2024-09-17T05:36:33.030590+0000","last_became_peered":"2024-09-17T05:36:33.030590+0000","last_unstale":"2024-09-17T05:36:33.033082+0000","last_undegraded":"2024-09-17T05:36:33.033082+0000","last_fullsized":"2024-09-17T05:36:33.033082+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:16.656506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028798+0000","last_change":"2024-09-17T05:36:31.994407+0000","last_active":"2024-09-17T05:36:34.028798+0000","last_peered":"2024-09-17T05:36:34.028798+0000","last_clean":"2024-09-17T05:36:34.028798+0000","last_became_active":"2024-09-17T05:36:31.994090+0000","last_became_peered":"2024-09-17T05:36:31.994090+0000","last_unstale":"2024-09-17T05:36:34.028798+0000","last_undegraded":"2024-09-17T05:36:34.028798+0000","last_fullsized":"2024-09-17T05:36:34.028798+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:05.885445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010136+0000","last_change":"2024-09-17T05:36:33.039104+0000","last_active":"2024-09-17T05:36:34.010136+0000","last_peered":"2024-09-17T05:36:34.010136+0000","last_clean":"2024-09-17T05:36:34.010136+0000","last_became_active":"2024-09-17T05:36:33.038797+0000","last_became_peered":"2024-09-17T05:36:33.038797+0000","last_unstale":"2024-09-17T05:36:34.010136+0000","last_undegraded":"2024-09-17T05:36:34.010136+0000","last_fullsized":"2024-09-17T05:36:34.010136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:32.812669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.013521+0000","last_change":"2024-09-17T05:36:31.993191+0000","last_active":"2024-09-17T05:36:33.013521+0000","last_peered":"2024-09-17T05:36:33.013521+0000","last_clean":"2024-09-17T05:36:33.013521+0000","last_became_active":"2024-09-17T05:36:31.992802+0000","last_became_peered":"2024-09-17T05:36:31.992802+0000","last_unstale":"2024-09-17T05:36:33.013521+0000","last_undegraded":"2024-09-17T05:36:33.013521+0000","last_fullsized":"2024-09-17T05:36:33.013521+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:01.201626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027730+0000","last_change":"2024-09-17T05:36:33.021198+0000","last_active":"2024-09-17T05:36:33.027730+0000","last_peered":"2024-09-17T05:36:33.027730+0000","last_clean":"2024-09-17T05:36:33.027730+0000","last_became_active":"2024-09-17T05:36:33.019760+0000","last_became_peered":"2024-09-17T05:36:33.019760+0000","last_unstale":"2024-09-17T05:36:33.027730+0000","last_undegraded":"2024-09-17T05:36:33.027730+0000","last_fullsized":"2024-09-17T05:36:33.027730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:46.561978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.020926+0000","last_change":"2024-09-17T05:36:32.008293+0000","last_active":"2024-09-17T05:36:33.020926+0000","last_peered":"2024-09-17T05:36:33.020926+0000","last_clean":"2024-09-17T05:36:33.020926+0000","last_became_active":"2024-09-17T05:36:32.007997+0000","last_became_peered":"2024-09-17T05:36:32.007997+0000","last_unstale":"2024-09-17T05:36:33.020926+0000","last_undegraded":"2024-09-17T05:36:33.020926+0000","last_fullsized":"2024-09-17T05:36:33.020926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:12.542988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364239+0000","last_change":"2024-09-17T05:36:33.031240+0000","last_active":"2024-09-17T05:36:34.364239+0000","last_peered":"2024-09-17T05:36:34.364239+0000","last_clean":"2024-09-17T05:36:34.364239+0000","last_became_active":"2024-09-17T05:36:33.030592+0000","last_became_peered":"2024-09-17T05:36:33.030592+0000","last_unstale":"2024-09-17T05:36:34.364239+0000","last_undegraded":"2024-09-17T05:36:34.364239+0000","last_fullsized":"2024-09-17T05:36:34.364239+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:21.424105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.029228+0000","last_change":"2024-09-17T05:36:32.004372+0000","last_active":"2024-09-17T05:36:34.029228+0000","last_peered":"2024-09-17T05:36:34.029228+0000","last_clean":"2024-09-17T05:36:34.029228+0000","last_became_active":"2024-09-17T05:36:32.004091+0000","last_became_peered":"2024-09-17T05:36:32.004091+0000","last_unstale":"2024-09-17T05:36:34.029228+0000","last_undegraded":"2024-09-17T05:36:34.029228+0000","last_fullsized":"2024-09-17T05:36:34.029228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.950627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.023906+0000","last_change":"2024-09-17T05:36:33.017900+0000","last_active":"2024-09-17T05:36:33.023906+0000","last_peered":"2024-09-17T05:36:33.023906+0000","last_clean":"2024-09-17T05:36:33.023906+0000","last_became_active":"2024-09-17T05:36:33.017597+0000","last_became_peered":"2024-09-17T05:36:33.017597+0000","last_unstale":"2024-09-17T05:36:33.023906+0000","last_undegraded":"2024-09-17T05:36:33.023906+0000","last_fullsized":"2024-09-17T05:36:33.023906+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:44.314644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364154+0000","last_change":"2024-09-17T05:36:32.012238+0000","last_active":"2024-09-17T05:36:34.364154+0000","last_peered":"2024-09-17T05:36:34.364154+0000","last_clean":"2024-09-17T05:36:34.364154+0000","last_became_active":"2024-09-17T05:36:32.009314+0000","last_became_peered":"2024-09-17T05:36:32.009314+0000","last_unstale":"2024-09-17T05:36:34.364154+0000","last_undegraded":"2024-09-17T05:36:34.364154+0000","last_fullsized":"2024-09-17T05:36:34.364154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:03.051818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.037476+0000","last_change":"2024-09-17T05:36:33.035844+0000","last_active":"2024-09-17T05:36:33.037476+0000","last_peered":"2024-09-17T05:36:33.037476+0000","last_clean":"2024-09-17T05:36:33.037476+0000","last_became_active":"2024-09-17T05:36:33.035445+0000","last_became_peered":"2024-09-17T05:36:33.035445+0000","last_unstale":"2024-09-17T05:36:33.037476+0000","last_undegraded":"2024-09-17T05:36:33.037476+0000","last_fullsized":"2024-09-17T05:36:33.037476+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:57.304253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"102'3","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.037081+0000","last_change":"2024-09-17T05:36:32.004701+0000","last_active":"2024-09-17T05:36:33.037081+0000","last_peered":"2024-09-17T05:36:33.037081+0000","last_clean":"2024-09-17T05:36:33.037081+0000","last_became_active":"2024-09-17T05:36:32.004441+0000","last_became_peered":"2024-09-17T05:36:32.004441+0000","last_unstale":"2024-09-17T05:36:33.037081+0000","last_undegraded":"2024-09-17T05:36:33.037081+0000","last_fullsized":"2024-09-17T05:36:33.037081+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:36:20.424323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033669+0000","last_change":"2024-09-17T05:36:33.030406+0000","last_active":"2024-09-17T05:36:33.033669+0000","last_peered":"2024-09-17T05:36:33.033669+0000","last_clean":"2024-09-17T05:36:33.033669+0000","last_became_active":"2024-09-17T05:36:33.030132+0000","last_became_peered":"2024-09-17T05:36:33.030132+0000","last_unstale":"2024-09-17T05:36:33.033669+0000","last_undegraded":"2024-09-17T05:36:33.033669+0000","last_fullsized":"2024-09-17T05:36:33.033669+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:17.102365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"102'6","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.045329+0000","last_change":"2024-09-17T05:36:32.006369+0000","last_active":"2024-09-17T05:36:33.045329+0000","last_peered":"2024-09-17T05:36:33.045329+0000","last_clean":"2024-09-17T05:36:33.045329+0000","last_became_active":"2024-09-17T05:36:32.005971+0000","last_became_peered":"2024-09-17T05:36:32.005971+0000","last_unstale":"2024-09-17T05:36:33.045329+0000","last_undegraded":"2024-09-17T05:36:33.045329+0000","last_fullsized":"2024-09-17T05:36:33.045329+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:52.685453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363411+0000","last_change":"2024-09-17T05:36:33.021619+0000","last_active":"2024-09-17T05:36:34.363411+0000","last_peered":"2024-09-17T05:36:34.363411+0000","last_clean":"2024-09-17T05:36:34.363411+0000","last_became_active":"2024-09-17T05:36:33.021277+0000","last_became_peered":"2024-09-17T05:36:33.021277+0000","last_unstale":"2024-09-17T05:36:34.363411+0000","last_undegraded":"2024-09-17T05:36:34.363411+0000","last_fullsized":"2024-09-17T05:36:34.363411+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:19.131661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"102'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.045100+0000","last_change":"2024-09-17T05:36:32.000574+0000","last_active":"2024-09-17T05:36:33.045100+0000","last_peered":"2024-09-17T05:36:33.045100+0000","last_clean":"2024-09-17T05:36:33.045100+0000","last_became_active":"2024-09-17T05:36:32.000269+0000","last_became_peered":"2024-09-17T05:36:32.000269+0000","last_unstale":"2024-09-17T05:36:33.045100+0000","last_undegraded":"2024-09-17T05:36:33.045100+0000","last_fullsized":"2024-09-17T05:36:33.045100+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:11.550824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028716+0000","last_change":"2024-09-17T05:36:33.045516+0000","last_active":"2024-09-17T05:36:34.028716+0000","last_peered":"2024-09-17T05:36:34.028716+0000","last_clean":"2024-09-17T05:36:34.028716+0000","last_became_active":"2024-09-17T05:36:33.044505+0000","last_became_peered":"2024-09-17T05:36:33.044505+0000","last_unstale":"2024-09-17T05:36:34.028716+0000","last_undegraded":"2024-09-17T05:36:34.028716+0000","last_fullsized":"2024-09-17T05:36:34.028716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:19.355781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363567+0000","last_change":"2024-09-17T05:36:32.011939+0000","last_active":"2024-09-17T05:36:34.363567+0000","last_peered":"2024-09-17T05:36:34.363567+0000","last_clean":"2024-09-17T05:36:34.363567+0000","last_became_active":"2024-09-17T05:36:32.011652+0000","last_became_peered":"2024-09-17T05:36:32.011652+0000","last_unstale":"2024-09-17T05:36:34.363567+0000","last_undegraded":"2024-09-17T05:36:34.363567+0000","last_fullsized":"2024-09-17T05:36:34.363567+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:59.393647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033521+0000","last_change":"2024-09-17T05:36:33.031238+0000","last_active":"2024-09-17T05:36:33.033521+0000","last_peered":"2024-09-17T05:36:33.033521+0000","last_clean":"2024-09-17T05:36:33.033521+0000","last_became_active":"2024-09-17T05:36:33.030889+0000","last_became_peered":"2024-09-17T05:36:33.030889+0000","last_unstale":"2024-09-17T05:36:33.033521+0000","last_undegraded":"2024-09-17T05:36:33.033521+0000","last_fullsized":"2024-09-17T05:36:33.033521+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:12.051776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.008566+0000","last_change":"2024-09-17T05:36:32.003592+0000","last_active":"2024-09-17T05:36:33.008566+0000","last_peered":"2024-09-17T05:36:33.008566+0000","last_clean":"2024-09-17T05:36:33.008566+0000","last_became_active":"2024-09-17T05:36:32.003315+0000","last_became_peered":"2024-09-17T05:36:32.003315+0000","last_unstale":"2024-09-17T05:36:33.008566+0000","last_undegraded":"2024-09-17T05:36:33.008566+0000","last_fullsized":"2024-09-17T05:36:33.008566+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:15.089519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028573+0000","last_change":"2024-09-17T05:36:33.040736+0000","last_active":"2024-09-17T05:36:34.028573+0000","last_peered":"2024-09-17T05:36:34.028573+0000","last_clean":"2024-09-17T05:36:34.028573+0000","last_became_active":"2024-09-17T05:36:33.037647+0000","last_became_peered":"2024-09-17T05:36:33.037647+0000","last_unstale":"2024-09-17T05:36:34.028573+0000","last_undegraded":"2024-09-17T05:36:34.028573+0000","last_fullsized":"2024-09-17T05:36:34.028573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:54.657188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"102'8","reported_seq":23,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363476+0000","last_change":"2024-09-17T05:36:32.005582+0000","last_active":"2024-09-17T05:36:34.363476+0000","last_peered":"2024-09-17T05:36:34.363476+0000","last_clean":"2024-09-17T05:36:34.363476+0000","last_became_active":"2024-09-17T05:36:32.005285+0000","last_became_peered":"2024-09-17T05:36:32.005285+0000","last_unstale":"2024-09-17T05:36:34.363476+0000","last_undegraded":"2024-09-17T05:36:34.363476+0000","last_fullsized":"2024-09-17T05:36:34.363476+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.950627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.032939+0000","last_change":"2024-09-17T05:36:33.029109+0000","last_active":"2024-09-17T05:36:33.032939+0000","last_peered":"2024-09-17T05:36:33.032939+0000","last_clean":"2024-09-17T05:36:33.032939+0000","last_became_active":"2024-09-17T05:36:33.028829+0000","last_became_peered":"2024-09-17T05:36:33.028829+0000","last_unstale":"2024-09-17T05:36:33.032939+0000","last_undegraded":"2024-09-17T05:36:33.032939+0000","last_fullsized":"2024-09-17T05:36:33.032939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:01.741595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"102'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.040678+0000","last_change":"2024-09-17T05:36:32.002068+0000","last_active":"2024-09-17T05:36:33.040678+0000","last_peered":"2024-09-17T05:36:33.040678+0000","last_clean":"2024-09-17T05:36:33.040678+0000","last_became_active":"2024-09-17T05:36:32.001692+0000","last_became_peered":"2024-09-17T05:36:32.001692+0000","last_unstale":"2024-09-17T05:36:33.040678+0000","last_undegraded":"2024-09-17T05:36:33.040678+0000","last_fullsized":"2024-09-17T05:36:33.040678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:27.129580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.024632+0000","last_change":"2024-09-17T05:36:33.020613+0000","last_active":"2024-09-17T05:36:33.024632+0000","last_peered":"2024-09-17T05:36:33.024632+0000","last_clean":"2024-09-17T05:36:33.024632+0000","last_became_active":"2024-09-17T05:36:33.020327+0000","last_became_peered":"2024-09-17T05:36:33.020327+0000","last_unstale":"2024-09-17T05:36:33.024632+0000","last_undegraded":"2024-09-17T05:36:33.024632+0000","last_fullsized":"2024-09-17T05:36:33.024632+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:28.173152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"102'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.038148+0000","last_change":"2024-09-17T05:36:31.992413+0000","last_active":"2024-09-17T05:36:33.038148+0000","last_peered":"2024-09-17T05:36:33.038148+0000","last_clean":"2024-09-17T05:36:33.038148+0000","last_became_active":"2024-09-17T05:36:31.991962+0000","last_became_peered":"2024-09-17T05:36:31.991962+0000","last_unstale":"2024-09-17T05:36:33.038148+0000","last_undegraded":"2024-09-17T05:36:33.038148+0000","last_fullsized":"2024-09-17T05:36:33.038148+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:32:59.440061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.013940+0000","last_change":"2024-09-17T05:36:33.012396+0000","last_active":"2024-09-17T05:36:33.013940+0000","last_peered":"2024-09-17T05:36:33.013940+0000","last_clean":"2024-09-17T05:36:33.013940+0000","last_became_active":"2024-09-17T05:36:33.012169+0000","last_became_peered":"2024-09-17T05:36:33.012169+0000","last_unstale":"2024-09-17T05:36:33.013940+0000","last_undegraded":"2024-09-17T05:36:33.013940+0000","last_fullsized":"2024-09-17T05:36:33.013940+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:24.501901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.013407+0000","last_change":"2024-09-17T05:36:32.003677+0000","last_active":"2024-09-17T05:36:33.013407+0000","last_peered":"2024-09-17T05:36:33.013407+0000","last_clean":"2024-09-17T05:36:33.013407+0000","last_became_active":"2024-09-17T05:36:32.003406+0000","last_became_peered":"2024-09-17T05:36:32.003406+0000","last_unstale":"2024-09-17T05:36:33.013407+0000","last_undegraded":"2024-09-17T05:36:33.013407+0000","last_fullsized":"2024-09-17T05:36:33.013407+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:33.642857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.044098+0000","last_change":"2024-09-17T05:36:33.040193+0000","last_active":"2024-09-17T05:36:33.044098+0000","last_peered":"2024-09-17T05:36:33.044098+0000","last_clean":"2024-09-17T05:36:33.044098+0000","last_became_active":"2024-09-17T05:36:33.039510+0000","last_became_peered":"2024-09-17T05:36:33.039510+0000","last_unstale":"2024-09-17T05:36:33.044098+0000","last_undegraded":"2024-09-17T05:36:33.044098+0000","last_fullsized":"2024-09-17T05:36:33.044098+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:33.703862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363454+0000","last_change":"2024-09-17T05:36:32.009432+0000","last_active":"2024-09-17T05:36:34.363454+0000","last_peered":"2024-09-17T05:36:34.363454+0000","last_clean":"2024-09-17T05:36:34.363454+0000","last_became_active":"2024-09-17T05:36:32.009152+0000","last_became_peered":"2024-09-17T05:36:32.009152+0000","last_unstale":"2024-09-17T05:36:34.363454+0000","last_undegraded":"2024-09-17T05:36:34.363454+0000","last_fullsized":"2024-09-17T05:36:34.363454+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:39.496193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.048295+0000","last_change":"2024-09-17T05:36:33.047360+0000","last_active":"2024-09-17T05:36:33.048295+0000","last_peered":"2024-09-17T05:36:33.048295+0000","last_clean":"2024-09-17T05:36:33.048295+0000","last_became_active":"2024-09-17T05:36:33.047080+0000","last_became_peered":"2024-09-17T05:36:33.047080+0000","last_unstale":"2024-09-17T05:36:33.048295+0000","last_undegraded":"2024-09-17T05:36:33.048295+0000","last_fullsized":"2024-09-17T05:36:33.048295+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:49.016055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.010788+0000","last_change":"2024-09-17T05:36:31.991151+0000","last_active":"2024-09-17T05:36:33.010788+0000","last_peered":"2024-09-17T05:36:33.010788+0000","last_clean":"2024-09-17T05:36:33.010788+0000","last_became_active":"2024-09-17T05:36:31.990659+0000","last_became_peered":"2024-09-17T05:36:31.990659+0000","last_unstale":"2024-09-17T05:36:33.010788+0000","last_undegraded":"2024-09-17T05:36:33.010788+0000","last_fullsized":"2024-09-17T05:36:33.010788+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:12.582231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363568+0000","last_change":"2024-09-17T05:36:33.026731+0000","last_active":"2024-09-17T05:36:34.363568+0000","last_peered":"2024-09-17T05:36:34.363568+0000","last_clean":"2024-09-17T05:36:34.363568+0000","last_became_active":"2024-09-17T05:36:33.026438+0000","last_became_peered":"2024-09-17T05:36:33.026438+0000","last_unstale":"2024-09-17T05:36:34.363568+0000","last_undegraded":"2024-09-17T05:36:34.363568+0000","last_fullsized":"2024-09-17T05:36:34.363568+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:19.355781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.008518+0000","last_change":"2024-09-17T05:36:31.994567+0000","last_active":"2024-09-17T05:36:34.008518+0000","last_peered":"2024-09-17T05:36:34.008518+0000","last_clean":"2024-09-17T05:36:34.008518+0000","last_became_active":"2024-09-17T05:36:31.994238+0000","last_became_peered":"2024-09-17T05:36:31.994238+0000","last_unstale":"2024-09-17T05:36:34.008518+0000","last_undegraded":"2024-09-17T05:36:34.008518+0000","last_fullsized":"2024-09-17T05:36:34.008518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:18.347320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.040752+0000","last_change":"2024-09-17T05:36:33.039414+0000","last_active":"2024-09-17T05:36:33.040752+0000","last_peered":"2024-09-17T05:36:33.040752+0000","last_clean":"2024-09-17T05:36:33.040752+0000","last_became_active":"2024-09-17T05:36:33.039084+0000","last_became_peered":"2024-09-17T05:36:33.039084+0000","last_unstale":"2024-09-17T05:36:33.040752+0000","last_undegraded":"2024-09-17T05:36:33.040752+0000","last_fullsized":"2024-09-17T05:36:33.040752+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:00.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028199+0000","last_change":"2024-09-17T05:36:32.002615+0000","last_active":"2024-09-17T05:36:34.028199+0000","last_peered":"2024-09-17T05:36:34.028199+0000","last_clean":"2024-09-17T05:36:34.028199+0000","last_became_active":"2024-09-17T05:36:32.002027+0000","last_became_peered":"2024-09-17T05:36:32.002027+0000","last_unstale":"2024-09-17T05:36:34.028199+0000","last_undegraded":"2024-09-17T05:36:34.028199+0000","last_fullsized":"2024-09-17T05:36:34.028199+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:40.433841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033730+0000","last_change":"2024-09-17T05:36:33.030122+0000","last_active":"2024-09-17T05:36:33.033730+0000","last_peered":"2024-09-17T05:36:33.033730+0000","last_clean":"2024-09-17T05:36:33.033730+0000","last_became_active":"2024-09-17T05:36:33.029501+0000","last_became_peered":"2024-09-17T05:36:33.029501+0000","last_unstale":"2024-09-17T05:36:33.033730+0000","last_undegraded":"2024-09-17T05:36:33.033730+0000","last_fullsized":"2024-09-17T05:36:33.033730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:27.883776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"102'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.038619+0000","last_change":"2024-09-17T05:36:32.008486+0000","last_active":"2024-09-17T05:36:33.038619+0000","last_peered":"2024-09-17T05:36:33.038619+0000","last_clean":"2024-09-17T05:36:33.038619+0000","last_became_active":"2024-09-17T05:36:32.006776+0000","last_became_peered":"2024-09-17T05:36:32.006776+0000","last_unstale":"2024-09-17T05:36:33.038619+0000","last_undegraded":"2024-09-17T05:36:33.038619+0000","last_fullsized":"2024-09-17T05:36:33.038619+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:48.058349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.043912+0000","last_change":"2024-09-17T05:36:33.039816+0000","last_active":"2024-09-17T05:36:33.043912+0000","last_peered":"2024-09-17T05:36:33.043912+0000","last_clean":"2024-09-17T05:36:33.043912+0000","last_became_active":"2024-09-17T05:36:33.039524+0000","last_became_peered":"2024-09-17T05:36:33.039524+0000","last_unstale":"2024-09-17T05:36:33.043912+0000","last_undegraded":"2024-09-17T05:36:33.043912+0000","last_fullsized":"2024-09-17T05:36:33.043912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:54.978590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.019784+0000","last_change":"2024-09-17T05:36:32.006835+0000","last_active":"2024-09-17T05:36:33.019784+0000","last_peered":"2024-09-17T05:36:33.019784+0000","last_clean":"2024-09-17T05:36:33.019784+0000","last_became_active":"2024-09-17T05:36:32.006488+0000","last_became_peered":"2024-09-17T05:36:32.006488+0000","last_unstale":"2024-09-17T05:36:33.019784+0000","last_undegraded":"2024-09-17T05:36:33.019784+0000","last_fullsized":"2024-09-17T05:36:33.019784+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:02.697756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.029063+0000","last_change":"2024-09-17T05:36:33.038063+0000","last_active":"2024-09-17T05:36:34.029063+0000","last_peered":"2024-09-17T05:36:34.029063+0000","last_clean":"2024-09-17T05:36:34.029063+0000","last_became_active":"2024-09-17T05:36:33.037604+0000","last_became_peered":"2024-09-17T05:36:33.037604+0000","last_unstale":"2024-09-17T05:36:34.029063+0000","last_undegraded":"2024-09-17T05:36:34.029063+0000","last_fullsized":"2024-09-17T05:36:34.029063+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:33.747047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.013599+0000","last_change":"2024-09-17T05:36:32.006594+0000","last_active":"2024-09-17T05:36:33.013599+0000","last_peered":"2024-09-17T05:36:33.013599+0000","last_clean":"2024-09-17T05:36:33.013599+0000","last_became_active":"2024-09-17T05:36:32.006312+0000","last_became_peered":"2024-09-17T05:36:32.006312+0000","last_unstale":"2024-09-17T05:36:33.013599+0000","last_undegraded":"2024-09-17T05:36:33.013599+0000","last_fullsized":"2024-09-17T05:36:33.013599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:37:37.815565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.028490+0000","last_change":"2024-09-17T05:36:33.020933+0000","last_active":"2024-09-17T05:36:33.028490+0000","last_peered":"2024-09-17T05:36:33.028490+0000","last_clean":"2024-09-17T05:36:33.028490+0000","last_became_active":"2024-09-17T05:36:33.020637+0000","last_became_peered":"2024-09-17T05:36:33.020637+0000","last_unstale":"2024-09-17T05:36:33.028490+0000","last_undegraded":"2024-09-17T05:36:33.028490+0000","last_fullsized":"2024-09-17T05:36:33.028490+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:40.154175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.011490+0000","last_change":"2024-09-17T05:36:32.003577+0000","last_active":"2024-09-17T05:36:33.011490+0000","last_peered":"2024-09-17T05:36:33.011490+0000","last_clean":"2024-09-17T05:36:33.011490+0000","last_became_active":"2024-09-17T05:36:32.002589+0000","last_became_peered":"2024-09-17T05:36:32.002589+0000","last_unstale":"2024-09-17T05:36:33.011490+0000","last_undegraded":"2024-09-17T05:36:33.011490+0000","last_fullsized":"2024-09-17T05:36:33.011490+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:30.756283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033360+0000","last_change":"2024-09-17T05:36:33.030019+0000","last_active":"2024-09-17T05:36:33.033360+0000","last_peered":"2024-09-17T05:36:33.033360+0000","last_clean":"2024-09-17T05:36:33.033360+0000","last_became_active":"2024-09-17T05:36:33.029746+0000","last_became_peered":"2024-09-17T05:36:33.029746+0000","last_unstale":"2024-09-17T05:36:33.033360+0000","last_undegraded":"2024-09-17T05:36:33.033360+0000","last_fullsized":"2024-09-17T05:36:33.033360+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:42.869668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028934+0000","last_change":"2024-09-17T05:36:32.012434+0000","last_active":"2024-09-17T05:36:34.028934+0000","last_peered":"2024-09-17T05:36:34.028934+0000","last_clean":"2024-09-17T05:36:34.028934+0000","last_became_active":"2024-09-17T05:36:32.012180+0000","last_became_peered":"2024-09-17T05:36:32.012180+0000","last_unstale":"2024-09-17T05:36:34.028934+0000","last_undegraded":"2024-09-17T05:36:34.028934+0000","last_fullsized":"2024-09-17T05:36:34.028934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:59.393647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.047969+0000","last_change":"2024-09-17T05:36:33.047189+0000","last_active":"2024-09-17T05:36:33.047969+0000","last_peered":"2024-09-17T05:36:33.047969+0000","last_clean":"2024-09-17T05:36:33.047969+0000","last_became_active":"2024-09-17T05:36:33.046883+0000","last_became_peered":"2024-09-17T05:36:33.046883+0000","last_unstale":"2024-09-17T05:36:33.047969+0000","last_undegraded":"2024-09-17T05:36:33.047969+0000","last_fullsized":"2024-09-17T05:36:33.047969+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:38.306111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.010999+0000","last_change":"2024-09-17T05:36:32.001897+0000","last_active":"2024-09-17T05:36:33.010999+0000","last_peered":"2024-09-17T05:36:33.010999+0000","last_clean":"2024-09-17T05:36:33.010999+0000","last_became_active":"2024-09-17T05:36:32.001626+0000","last_became_peered":"2024-09-17T05:36:32.001626+0000","last_unstale":"2024-09-17T05:36:33.010999+0000","last_undegraded":"2024-09-17T05:36:33.010999+0000","last_fullsized":"2024-09-17T05:36:33.010999+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:58.710334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033579+0000","last_change":"2024-09-17T05:36:33.031152+0000","last_active":"2024-09-17T05:36:33.033579+0000","last_peered":"2024-09-17T05:36:33.033579+0000","last_clean":"2024-09-17T05:36:33.033579+0000","last_became_active":"2024-09-17T05:36:33.030868+0000","last_became_peered":"2024-09-17T05:36:33.030868+0000","last_unstale":"2024-09-17T05:36:33.033579+0000","last_undegraded":"2024-09-17T05:36:33.033579+0000","last_fullsized":"2024-09-17T05:36:33.033579+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:12.715902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010257+0000","last_change":"2024-09-17T05:36:32.009452+0000","last_active":"2024-09-17T05:36:34.010257+0000","last_peered":"2024-09-17T05:36:34.010257+0000","last_clean":"2024-09-17T05:36:34.010257+0000","last_became_active":"2024-09-17T05:36:32.009160+0000","last_became_peered":"2024-09-17T05:36:32.009160+0000","last_unstale":"2024-09-17T05:36:34.010257+0000","last_undegraded":"2024-09-17T05:36:34.010257+0000","last_fullsized":"2024-09-17T05:36:34.010257+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.950627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.028537+0000","last_change":"2024-09-17T05:36:33.025924+0000","last_active":"2024-09-17T05:36:33.028537+0000","last_peered":"2024-09-17T05:36:33.028537+0000","last_clean":"2024-09-17T05:36:33.028537+0000","last_became_active":"2024-09-17T05:36:33.025654+0000","last_became_peered":"2024-09-17T05:36:33.025654+0000","last_unstale":"2024-09-17T05:36:33.028537+0000","last_undegraded":"2024-09-17T05:36:33.028537+0000","last_fullsized":"2024-09-17T05:36:33.028537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:22.208384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028316+0000","last_change":"2024-09-17T05:36:32.002306+0000","last_active":"2024-09-17T05:36:34.028316+0000","last_peered":"2024-09-17T05:36:34.028316+0000","last_clean":"2024-09-17T05:36:34.028316+0000","last_became_active":"2024-09-17T05:36:32.001676+0000","last_became_peered":"2024-09-17T05:36:32.001676+0000","last_unstale":"2024-09-17T05:36:34.028316+0000","last_undegraded":"2024-09-17T05:36:34.028316+0000","last_fullsized":"2024-09-17T05:36:34.028316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:03.981516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.029033+0000","last_change":"2024-09-17T05:36:33.039006+0000","last_active":"2024-09-17T05:36:34.029033+0000","last_peered":"2024-09-17T05:36:34.029033+0000","last_clean":"2024-09-17T05:36:34.029033+0000","last_became_active":"2024-09-17T05:36:33.037887+0000","last_became_peered":"2024-09-17T05:36:33.037887+0000","last_unstale":"2024-09-17T05:36:34.029033+0000","last_undegraded":"2024-09-17T05:36:34.029033+0000","last_fullsized":"2024-09-17T05:36:34.029033+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:53.765357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"102'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.039833+0000","last_change":"2024-09-17T05:36:32.008404+0000","last_active":"2024-09-17T05:36:33.039833+0000","last_peered":"2024-09-17T05:36:33.039833+0000","last_clean":"2024-09-17T05:36:33.039833+0000","last_became_active":"2024-09-17T05:36:32.007744+0000","last_became_peered":"2024-09-17T05:36:32.007744+0000","last_unstale":"2024-09-17T05:36:33.039833+0000","last_undegraded":"2024-09-17T05:36:33.039833+0000","last_fullsized":"2024-09-17T05:36:33.039833+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:44.549268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":263,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.006626+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:36:33.006626+0000","last_peered":"2024-09-17T05:36:33.006626+0000","last_clean":"2024-09-17T05:36:33.006626+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:36:33.006626+0000","last_undegraded":"2024-09-17T05:36:33.006626+0000","last_fullsized":"2024-09-17T05:36:33.006626+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64288,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307552,"statfs":{"total":96636764160,"available":96570933248,"internally_reserved":0,"allocated":1257472,"data_stored":544977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144808,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227032,"statfs":{"total":96636764160,"available":96488480768,"internally_reserved":0,"allocated":1253376,"data_stored":544395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104560,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267280,"statfs":{"total":96636764160,"available":96529694720,"internally_reserved":0,"allocated":1269760,"data_stored":551333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122600,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249240,"statfs":{"total":96636764160,"available":96511221760,"internally_reserved":0,"allocated":1265664,"data_stored":544589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122592,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249248,"statfs":{"total":96636764160,"available":96511229952,"internally_reserved":0,"allocated":1257472,"data_stored":544465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509584,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64888,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306952,"statfs":{"total":96636764160,"available":96570318848,"internally_reserved":0,"allocated":1867776,"data_stored":1136487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82352,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289488,"statfs":{"total":96636764160,"available":96552435712,"internally_reserved":0,"allocated":1265664,"data_stored":550751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":83032523},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542288,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127352,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244488,"statfs":{"total":96636764160,"available":96506355712,"internally_reserved":0,"allocated":1867776,"data_stored":1136431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:36:37.134 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:36:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- 172.21.15.122:0/3855989433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500cdb30 msgr2=0x7f2b500cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 --2- 172.21.15.122:0/3855989433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500cdb30 0x7f2b500cebb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2b500745d0 tx=0x7f2b3c01c830 comp rx=0 tx=0).stop 2024-09-17T05:36:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- 172.21.15.122:0/3855989433 shutdown_connections 2024-09-17T05:36:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 --2- 172.21.15.122:0/3855989433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500cdb30 0x7f2b500cebb0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 --2- 172.21.15.122:0/3855989433 >> v2:172.21.15.173:3300/0 conn(0x7f2b5015bea0 0x7f2b500ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- 172.21.15.122:0/3855989433 >> 172.21.15.122:0/3855989433 conn(0x7f2b500d1dc0 msgr2=0x7f2b500d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- 172.21.15.122:0/3855989433 shutdown_connections 2024-09-17T05:36:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- 172.21.15.122:0/3855989433 wait complete. 2024-09-17T05:36:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 Processor -- start 2024-09-17T05:36:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- start start 2024-09-17T05:36:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b500cdb30 0x7f2b501491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500d28e0 0x7f2b500b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5015bea0 0x7f2b50158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50168f10 con 0x7f2b500d28e0 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50168d90 con 0x7f2b5015bea0 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b54c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2b50169090 con 0x7f2b500cdb30 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500d28e0 0x7f2b500b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500d28e0 0x7f2b500b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44642/0 (socket says 172.21.15.122:44642) 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4d400640 1 -- 172.21.15.122:0/2827634531 learned_addr learned my addr 172.21.15.122:0/2827634531 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4e800640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5015bea0 0x7f2b50158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4de00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b500cdb30 0x7f2b501491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4d400640 1 -- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b500cdb30 msgr2=0x7f2b501491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4d400640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b500cdb30 0x7f2b501491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4d400640 1 -- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5015bea0 msgr2=0x7f2b50158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4d400640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5015bea0 0x7f2b50158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.322+0000 7f2b4d400640 1 -- 172.21.15.122:0/2827634531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b50149700 con 0x7f2b500d28e0 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b4de00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b500cdb30 0x7f2b501491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b4d400640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500d28e0 0x7f2b500b2c20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0b0040 tx=0x7f2b3c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b46000640 1 -- 172.21.15.122:0/2827634531 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0b2040 con 0x7f2b500d28e0 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b46000640 1 -- 172.21.15.122:0/2827634531 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b3c005020 con 0x7f2b500d28e0 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b50157cd0 con 0x7f2b500d28e0 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b46000640 1 -- 172.21.15.122:0/2827634531 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c005470 con 0x7f2b500d28e0 2024-09-17T05:36:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b50158290 con 0x7f2b500d28e0 2024-09-17T05:36:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b500cc9d0 con 0x7f2b500d28e0 2024-09-17T05:36:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b46000640 1 -- 172.21.15.122:0/2827634531 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f2b3c0046d0 con 0x7f2b500d28e0 2024-09-17T05:36:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b46000640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b30074880 0x7f2b30076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:37.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b46000640 1 -- 172.21.15.122:0/2827634531 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f2b3c0bd070 con 0x7f2b500d28e0 2024-09-17T05:36:37.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.326+0000 7f2b4de00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b30074880 0x7f2b30076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:37.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.330+0000 7f2b4de00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b30074880 0x7f2b30076d40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2b40005fd0 tx=0x7f2b40005d00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:37.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.330+0000 7f2b46000640 1 -- 172.21.15.122:0/2827634531 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2b3c0164b0 con 0x7f2b500d28e0 2024-09-17T05:36:37.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.498+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b500b0530 con 0x7f2b30074880 2024-09-17T05:36:37.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.502+0000 7f2b46000640 1 -- 172.21.15.122:0/2827634531 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f2b500b0530 con 0x7f2b30074880 2024-09-17T05:36:37.500 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:37.501 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b30074880 msgr2=0x7f2b30076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b30074880 0x7f2b30076d40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2b40005fd0 tx=0x7f2b40005d00 comp rx=0 tx=0).stop 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500d28e0 msgr2=0x7f2b500b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500d28e0 0x7f2b500b2c20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0b0040 tx=0x7f2b3c004040 comp rx=0 tx=0).stop 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 shutdown_connections 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b30074880 0x7f2b30076d40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5015bea0 0x7f2b50158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b500d28e0 0x7f2b500b2c20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 --2- 172.21.15.122:0/2827634531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b500cdb30 0x7f2b501491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 >> 172.21.15.122:0/2827634531 conn(0x7f2b500d1dc0 msgr2=0x7f2b500b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 shutdown_connections 2024-09-17T05:36:37.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.506+0000 7f2b54c00640 1 -- 172.21.15.122:0/2827634531 wait complete. 2024-09-17T05:36:37.521 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-17T05:36:36.690053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":833440,"kb_used_data":11040,"kb_used_omap":12,"kb_used_meta":822195,"kb_avail":754141280,"statfs":{"total":773094113280,"available":772240670720,"internally_reserved":0,"allocated":11304960,"data_stored":5553428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":841928246},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.710718"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364008+0000","last_change":"2024-09-17T05:36:33.025998+0000","last_active":"2024-09-17T05:36:34.364008+0000","last_peered":"2024-09-17T05:36:34.364008+0000","last_clean":"2024-09-17T05:36:34.364008+0000","last_became_active":"2024-09-17T05:36:33.025635+0000","last_became_peered":"2024-09-17T05:36:33.025635+0000","last_unstale":"2024-09-17T05:36:34.364008+0000","last_undegraded":"2024-09-17T05:36:34.364008+0000","last_fullsized":"2024-09-17T05:36:34.364008+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:11.430701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364039+0000","last_change":"2024-09-17T05:36:33.032232+0000","last_active":"2024-09-17T05:36:34.364039+0000","last_peered":"2024-09-17T05:36:34.364039+0000","last_clean":"2024-09-17T05:36:34.364039+0000","last_became_active":"2024-09-17T05:36:33.031848+0000","last_became_peered":"2024-09-17T05:36:33.031848+0000","last_unstale":"2024-09-17T05:36:34.364039+0000","last_undegraded":"2024-09-17T05:36:34.364039+0000","last_fullsized":"2024-09-17T05:36:34.364039+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:53.501709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028954+0000","last_change":"2024-09-17T05:36:33.037607+0000","last_active":"2024-09-17T05:36:34.028954+0000","last_peered":"2024-09-17T05:36:34.028954+0000","last_clean":"2024-09-17T05:36:34.028954+0000","last_became_active":"2024-09-17T05:36:33.037249+0000","last_became_peered":"2024-09-17T05:36:33.037249+0000","last_unstale":"2024-09-17T05:36:34.028954+0000","last_undegraded":"2024-09-17T05:36:34.028954+0000","last_fullsized":"2024-09-17T05:36:34.028954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:25.395395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363775+0000","last_change":"2024-09-17T05:36:33.034989+0000","last_active":"2024-09-17T05:36:34.363775+0000","last_peered":"2024-09-17T05:36:34.363775+0000","last_clean":"2024-09-17T05:36:34.363775+0000","last_became_active":"2024-09-17T05:36:33.034328+0000","last_became_peered":"2024-09-17T05:36:33.034328+0000","last_unstale":"2024-09-17T05:36:34.363775+0000","last_undegraded":"2024-09-17T05:36:34.363775+0000","last_fullsized":"2024-09-17T05:36:34.363775+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:47.756883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.030874+0000","last_change":"2024-09-17T05:36:33.022107+0000","last_active":"2024-09-17T05:36:33.030874+0000","last_peered":"2024-09-17T05:36:33.030874+0000","last_clean":"2024-09-17T05:36:33.030874+0000","last_became_active":"2024-09-17T05:36:33.021819+0000","last_became_peered":"2024-09-17T05:36:33.021819+0000","last_unstale":"2024-09-17T05:36:33.030874+0000","last_undegraded":"2024-09-17T05:36:33.030874+0000","last_fullsized":"2024-09-17T05:36:33.030874+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:47.934927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.030294+0000","last_change":"2024-09-17T05:36:33.017608+0000","last_active":"2024-09-17T05:36:33.030294+0000","last_peered":"2024-09-17T05:36:33.030294+0000","last_clean":"2024-09-17T05:36:33.030294+0000","last_became_active":"2024-09-17T05:36:33.017377+0000","last_became_peered":"2024-09-17T05:36:33.017377+0000","last_unstale":"2024-09-17T05:36:33.030294+0000","last_undegraded":"2024-09-17T05:36:33.030294+0000","last_fullsized":"2024-09-17T05:36:33.030294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:23.705153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027927+0000","last_change":"2024-09-17T05:36:33.016812+0000","last_active":"2024-09-17T05:36:33.027927+0000","last_peered":"2024-09-17T05:36:33.027927+0000","last_clean":"2024-09-17T05:36:33.027927+0000","last_became_active":"2024-09-17T05:36:33.016491+0000","last_became_peered":"2024-09-17T05:36:33.016491+0000","last_unstale":"2024-09-17T05:36:33.027927+0000","last_undegraded":"2024-09-17T05:36:33.027927+0000","last_fullsized":"2024-09-17T05:36:33.027927+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:39.487804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028228+0000","last_change":"2024-09-17T05:36:33.038714+0000","last_active":"2024-09-17T05:36:34.028228+0000","last_peered":"2024-09-17T05:36:34.028228+0000","last_clean":"2024-09-17T05:36:34.028228+0000","last_became_active":"2024-09-17T05:36:33.038440+0000","last_became_peered":"2024-09-17T05:36:33.038440+0000","last_unstale":"2024-09-17T05:36:34.028228+0000","last_undegraded":"2024-09-17T05:36:34.028228+0000","last_fullsized":"2024-09-17T05:36:34.028228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:22.355517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.047934+0000","last_change":"2024-09-17T05:36:33.047158+0000","last_active":"2024-09-17T05:36:33.047934+0000","last_peered":"2024-09-17T05:36:33.047934+0000","last_clean":"2024-09-17T05:36:33.047934+0000","last_became_active":"2024-09-17T05:36:33.046847+0000","last_became_peered":"2024-09-17T05:36:33.046847+0000","last_unstale":"2024-09-17T05:36:33.047934+0000","last_undegraded":"2024-09-17T05:36:33.047934+0000","last_fullsized":"2024-09-17T05:36:33.047934+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:58.063492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.008726+0000","last_change":"2024-09-17T05:36:33.034774+0000","last_active":"2024-09-17T05:36:34.008726+0000","last_peered":"2024-09-17T05:36:34.008726+0000","last_clean":"2024-09-17T05:36:34.008726+0000","last_became_active":"2024-09-17T05:36:33.034521+0000","last_became_peered":"2024-09-17T05:36:33.034521+0000","last_unstale":"2024-09-17T05:36:34.008726+0000","last_undegraded":"2024-09-17T05:36:34.008726+0000","last_fullsized":"2024-09-17T05:36:34.008726+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:18.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027962+0000","last_change":"2024-09-17T05:36:33.017053+0000","last_active":"2024-09-17T05:36:33.027962+0000","last_peered":"2024-09-17T05:36:33.027962+0000","last_clean":"2024-09-17T05:36:33.027962+0000","last_became_active":"2024-09-17T05:36:33.016787+0000","last_became_peered":"2024-09-17T05:36:33.016787+0000","last_unstale":"2024-09-17T05:36:33.027962+0000","last_undegraded":"2024-09-17T05:36:33.027962+0000","last_fullsized":"2024-09-17T05:36:33.027962+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:38.740525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.037722+0000","last_change":"2024-09-17T05:36:33.036410+0000","last_active":"2024-09-17T05:36:33.037722+0000","last_peered":"2024-09-17T05:36:33.037722+0000","last_clean":"2024-09-17T05:36:33.037722+0000","last_became_active":"2024-09-17T05:36:33.036111+0000","last_became_peered":"2024-09-17T05:36:33.036111+0000","last_unstale":"2024-09-17T05:36:33.037722+0000","last_undegraded":"2024-09-17T05:36:33.037722+0000","last_fullsized":"2024-09-17T05:36:33.037722+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:31.796598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010497+0000","last_change":"2024-09-17T05:36:33.017813+0000","last_active":"2024-09-17T05:36:34.010497+0000","last_peered":"2024-09-17T05:36:34.010497+0000","last_clean":"2024-09-17T05:36:34.010497+0000","last_became_active":"2024-09-17T05:36:33.016888+0000","last_became_peered":"2024-09-17T05:36:33.016888+0000","last_unstale":"2024-09-17T05:36:34.010497+0000","last_undegraded":"2024-09-17T05:36:34.010497+0000","last_fullsized":"2024-09-17T05:36:34.010497+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:25.030272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363646+0000","last_change":"2024-09-17T05:36:33.030560+0000","last_active":"2024-09-17T05:36:34.363646+0000","last_peered":"2024-09-17T05:36:34.363646+0000","last_clean":"2024-09-17T05:36:34.363646+0000","last_became_active":"2024-09-17T05:36:33.026357+0000","last_became_peered":"2024-09-17T05:36:33.026357+0000","last_unstale":"2024-09-17T05:36:34.363646+0000","last_undegraded":"2024-09-17T05:36:34.363646+0000","last_fullsized":"2024-09-17T05:36:34.363646+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:18.951095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363350+0000","last_change":"2024-09-17T05:36:33.045288+0000","last_active":"2024-09-17T05:36:34.363350+0000","last_peered":"2024-09-17T05:36:34.363350+0000","last_clean":"2024-09-17T05:36:34.363350+0000","last_became_active":"2024-09-17T05:36:33.045008+0000","last_became_peered":"2024-09-17T05:36:33.045008+0000","last_unstale":"2024-09-17T05:36:34.363350+0000","last_undegraded":"2024-09-17T05:36:34.363350+0000","last_fullsized":"2024-09-17T05:36:34.363350+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:39.180523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.030464+0000","last_change":"2024-09-17T05:36:33.027755+0000","last_active":"2024-09-17T05:36:33.030464+0000","last_peered":"2024-09-17T05:36:33.030464+0000","last_clean":"2024-09-17T05:36:33.030464+0000","last_became_active":"2024-09-17T05:36:33.024885+0000","last_became_peered":"2024-09-17T05:36:33.024885+0000","last_unstale":"2024-09-17T05:36:33.030464+0000","last_undegraded":"2024-09-17T05:36:33.030464+0000","last_fullsized":"2024-09-17T05:36:33.030464+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:50.481517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.044133+0000","last_change":"2024-09-17T05:36:33.041006+0000","last_active":"2024-09-17T05:36:33.044133+0000","last_peered":"2024-09-17T05:36:33.044133+0000","last_clean":"2024-09-17T05:36:33.044133+0000","last_became_active":"2024-09-17T05:36:33.040735+0000","last_became_peered":"2024-09-17T05:36:33.040735+0000","last_unstale":"2024-09-17T05:36:33.044133+0000","last_undegraded":"2024-09-17T05:36:33.044133+0000","last_fullsized":"2024-09-17T05:36:33.044133+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:29.952722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364043+0000","last_change":"2024-09-17T05:36:33.046502+0000","last_active":"2024-09-17T05:36:34.364043+0000","last_peered":"2024-09-17T05:36:34.364043+0000","last_clean":"2024-09-17T05:36:34.364043+0000","last_became_active":"2024-09-17T05:36:33.045916+0000","last_became_peered":"2024-09-17T05:36:33.045916+0000","last_unstale":"2024-09-17T05:36:34.364043+0000","last_undegraded":"2024-09-17T05:36:34.364043+0000","last_fullsized":"2024-09-17T05:36:34.364043+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:31.764744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.043970+0000","last_change":"2024-09-17T05:36:33.039401+0000","last_active":"2024-09-17T05:36:33.043970+0000","last_peered":"2024-09-17T05:36:33.043970+0000","last_clean":"2024-09-17T05:36:33.043970+0000","last_became_active":"2024-09-17T05:36:33.039103+0000","last_became_peered":"2024-09-17T05:36:33.039103+0000","last_unstale":"2024-09-17T05:36:33.043970+0000","last_undegraded":"2024-09-17T05:36:33.043970+0000","last_fullsized":"2024-09-17T05:36:33.043970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:04.543809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.018152+0000","last_change":"2024-09-17T05:36:33.015241+0000","last_active":"2024-09-17T05:36:33.018152+0000","last_peered":"2024-09-17T05:36:33.018152+0000","last_clean":"2024-09-17T05:36:33.018152+0000","last_became_active":"2024-09-17T05:36:33.014894+0000","last_became_peered":"2024-09-17T05:36:33.014894+0000","last_unstale":"2024-09-17T05:36:33.018152+0000","last_undegraded":"2024-09-17T05:36:33.018152+0000","last_fullsized":"2024-09-17T05:36:33.018152+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:28.250794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028667+0000","last_change":"2024-09-17T05:36:33.041468+0000","last_active":"2024-09-17T05:36:34.028667+0000","last_peered":"2024-09-17T05:36:34.028667+0000","last_clean":"2024-09-17T05:36:34.028667+0000","last_became_active":"2024-09-17T05:36:33.041122+0000","last_became_peered":"2024-09-17T05:36:33.041122+0000","last_unstale":"2024-09-17T05:36:34.028667+0000","last_undegraded":"2024-09-17T05:36:34.028667+0000","last_fullsized":"2024-09-17T05:36:34.028667+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:23.082151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028669+0000","last_change":"2024-09-17T05:36:33.037082+0000","last_active":"2024-09-17T05:36:34.028669+0000","last_peered":"2024-09-17T05:36:34.028669+0000","last_clean":"2024-09-17T05:36:34.028669+0000","last_became_active":"2024-09-17T05:36:33.036668+0000","last_became_peered":"2024-09-17T05:36:33.036668+0000","last_unstale":"2024-09-17T05:36:34.028669+0000","last_undegraded":"2024-09-17T05:36:34.028669+0000","last_fullsized":"2024-09-17T05:36:34.028669+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:34.458277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363380+0000","last_change":"2024-09-17T05:36:33.024065+0000","last_active":"2024-09-17T05:36:34.363380+0000","last_peered":"2024-09-17T05:36:34.363380+0000","last_clean":"2024-09-17T05:36:34.363380+0000","last_became_active":"2024-09-17T05:36:33.023695+0000","last_became_peered":"2024-09-17T05:36:33.023695+0000","last_unstale":"2024-09-17T05:36:34.363380+0000","last_undegraded":"2024-09-17T05:36:34.363380+0000","last_fullsized":"2024-09-17T05:36:34.363380+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:43.515827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027730+0000","last_change":"2024-09-17T05:36:33.025699+0000","last_active":"2024-09-17T05:36:33.027730+0000","last_peered":"2024-09-17T05:36:33.027730+0000","last_clean":"2024-09-17T05:36:33.027730+0000","last_became_active":"2024-09-17T05:36:33.025447+0000","last_became_peered":"2024-09-17T05:36:33.025447+0000","last_unstale":"2024-09-17T05:36:33.027730+0000","last_undegraded":"2024-09-17T05:36:33.027730+0000","last_fullsized":"2024-09-17T05:36:33.027730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:35:10.636607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.008767+0000","last_change":"2024-09-17T05:36:33.034033+0000","last_active":"2024-09-17T05:36:34.008767+0000","last_peered":"2024-09-17T05:36:34.008767+0000","last_clean":"2024-09-17T05:36:34.008767+0000","last_became_active":"2024-09-17T05:36:33.033750+0000","last_became_peered":"2024-09-17T05:36:33.033750+0000","last_unstale":"2024-09-17T05:36:34.008767+0000","last_undegraded":"2024-09-17T05:36:34.008767+0000","last_fullsized":"2024-09-17T05:36:34.008767+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:27.635660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010651+0000","last_change":"2024-09-17T05:36:33.017598+0000","last_active":"2024-09-17T05:36:34.010651+0000","last_peered":"2024-09-17T05:36:34.010651+0000","last_clean":"2024-09-17T05:36:34.010651+0000","last_became_active":"2024-09-17T05:36:33.016883+0000","last_became_peered":"2024-09-17T05:36:33.016883+0000","last_unstale":"2024-09-17T05:36:34.010651+0000","last_undegraded":"2024-09-17T05:36:34.010651+0000","last_fullsized":"2024-09-17T05:36:34.010651+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:52.311207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.023555+0000","last_change":"2024-09-17T05:36:33.017635+0000","last_active":"2024-09-17T05:36:33.023555+0000","last_peered":"2024-09-17T05:36:33.023555+0000","last_clean":"2024-09-17T05:36:33.023555+0000","last_became_active":"2024-09-17T05:36:33.017336+0000","last_became_peered":"2024-09-17T05:36:33.017336+0000","last_unstale":"2024-09-17T05:36:33.023555+0000","last_undegraded":"2024-09-17T05:36:33.023555+0000","last_fullsized":"2024-09-17T05:36:33.023555+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:46.993290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.040622+0000","last_change":"2024-09-17T05:36:33.039251+0000","last_active":"2024-09-17T05:36:33.040622+0000","last_peered":"2024-09-17T05:36:33.040622+0000","last_clean":"2024-09-17T05:36:33.040622+0000","last_became_active":"2024-09-17T05:36:33.038875+0000","last_became_peered":"2024-09-17T05:36:33.038875+0000","last_unstale":"2024-09-17T05:36:33.040622+0000","last_undegraded":"2024-09-17T05:36:33.040622+0000","last_fullsized":"2024-09-17T05:36:33.040622+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:51.232824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010492+0000","last_change":"2024-09-17T05:36:33.018822+0000","last_active":"2024-09-17T05:36:34.010492+0000","last_peered":"2024-09-17T05:36:34.010492+0000","last_clean":"2024-09-17T05:36:34.010492+0000","last_became_active":"2024-09-17T05:36:33.018480+0000","last_became_peered":"2024-09-17T05:36:33.018480+0000","last_unstale":"2024-09-17T05:36:34.010492+0000","last_undegraded":"2024-09-17T05:36:34.010492+0000","last_fullsized":"2024-09-17T05:36:34.010492+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:26.901434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010564+0000","last_change":"2024-09-17T05:36:33.033972+0000","last_active":"2024-09-17T05:36:34.010564+0000","last_peered":"2024-09-17T05:36:34.010564+0000","last_clean":"2024-09-17T05:36:34.010564+0000","last_became_active":"2024-09-17T05:36:33.033712+0000","last_became_peered":"2024-09-17T05:36:33.033712+0000","last_unstale":"2024-09-17T05:36:34.010564+0000","last_undegraded":"2024-09-17T05:36:34.010564+0000","last_fullsized":"2024-09-17T05:36:34.010564+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:04.060279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028355+0000","last_change":"2024-09-17T05:36:33.032413+0000","last_active":"2024-09-17T05:36:34.028355+0000","last_peered":"2024-09-17T05:36:34.028355+0000","last_clean":"2024-09-17T05:36:34.028355+0000","last_became_active":"2024-09-17T05:36:33.031075+0000","last_became_peered":"2024-09-17T05:36:33.031075+0000","last_unstale":"2024-09-17T05:36:34.028355+0000","last_undegraded":"2024-09-17T05:36:34.028355+0000","last_fullsized":"2024-09-17T05:36:34.028355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:38.726170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363250+0000","last_change":"2024-09-17T05:36:33.029679+0000","last_active":"2024-09-17T05:36:34.363250+0000","last_peered":"2024-09-17T05:36:34.363250+0000","last_clean":"2024-09-17T05:36:34.363250+0000","last_became_active":"2024-09-17T05:36:33.029376+0000","last_became_peered":"2024-09-17T05:36:33.029376+0000","last_unstale":"2024-09-17T05:36:34.363250+0000","last_undegraded":"2024-09-17T05:36:34.363250+0000","last_fullsized":"2024-09-17T05:36:34.363250+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:06.167690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.039319+0000","last_change":"2024-09-17T05:36:33.033905+0000","last_active":"2024-09-17T05:36:33.039319+0000","last_peered":"2024-09-17T05:36:33.039319+0000","last_clean":"2024-09-17T05:36:33.039319+0000","last_became_active":"2024-09-17T05:36:33.033280+0000","last_became_peered":"2024-09-17T05:36:33.033280+0000","last_unstale":"2024-09-17T05:36:33.039319+0000","last_undegraded":"2024-09-17T05:36:33.039319+0000","last_fullsized":"2024-09-17T05:36:33.039319+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:06.896905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"102'4","reported_seq":19,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.011198+0000","last_change":"2024-09-17T05:36:31.995389+0000","last_active":"2024-09-17T05:36:34.011198+0000","last_peered":"2024-09-17T05:36:34.011198+0000","last_clean":"2024-09-17T05:36:34.011198+0000","last_became_active":"2024-09-17T05:36:31.994467+0000","last_became_peered":"2024-09-17T05:36:31.994467+0000","last_unstale":"2024-09-17T05:36:34.011198+0000","last_undegraded":"2024-09-17T05:36:34.011198+0000","last_fullsized":"2024-09-17T05:36:34.011198+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:05.885445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.011081+0000","last_change":"2024-09-17T05:36:33.019743+0000","last_active":"2024-09-17T05:36:34.011081+0000","last_peered":"2024-09-17T05:36:34.011081+0000","last_clean":"2024-09-17T05:36:34.011081+0000","last_became_active":"2024-09-17T05:36:33.019388+0000","last_became_peered":"2024-09-17T05:36:33.019388+0000","last_unstale":"2024-09-17T05:36:34.011081+0000","last_undegraded":"2024-09-17T05:36:34.011081+0000","last_fullsized":"2024-09-17T05:36:34.011081+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:25.468745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"102'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.039829+0000","last_change":"2024-09-17T05:36:32.001502+0000","last_active":"2024-09-17T05:36:33.039829+0000","last_peered":"2024-09-17T05:36:33.039829+0000","last_clean":"2024-09-17T05:36:33.039829+0000","last_became_active":"2024-09-17T05:36:32.000908+0000","last_became_peered":"2024-09-17T05:36:32.000908+0000","last_unstale":"2024-09-17T05:36:33.039829+0000","last_undegraded":"2024-09-17T05:36:33.039829+0000","last_fullsized":"2024-09-17T05:36:33.039829+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:24.664804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363894+0000","last_change":"2024-09-17T05:36:33.034691+0000","last_active":"2024-09-17T05:36:34.363894+0000","last_peered":"2024-09-17T05:36:34.363894+0000","last_clean":"2024-09-17T05:36:34.363894+0000","last_became_active":"2024-09-17T05:36:33.034015+0000","last_became_peered":"2024-09-17T05:36:33.034015+0000","last_unstale":"2024-09-17T05:36:34.363894+0000","last_undegraded":"2024-09-17T05:36:34.363894+0000","last_fullsized":"2024-09-17T05:36:34.363894+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:27.635660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.029079+0000","last_change":"2024-09-17T05:36:32.003003+0000","last_active":"2024-09-17T05:36:34.029079+0000","last_peered":"2024-09-17T05:36:34.029079+0000","last_clean":"2024-09-17T05:36:34.029079+0000","last_became_active":"2024-09-17T05:36:32.002474+0000","last_became_peered":"2024-09-17T05:36:32.002474+0000","last_unstale":"2024-09-17T05:36:34.029079+0000","last_undegraded":"2024-09-17T05:36:34.029079+0000","last_fullsized":"2024-09-17T05:36:34.029079+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:38.440356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.008736+0000","last_change":"2024-09-17T05:36:33.027677+0000","last_active":"2024-09-17T05:36:34.008736+0000","last_peered":"2024-09-17T05:36:34.008736+0000","last_clean":"2024-09-17T05:36:34.008736+0000","last_became_active":"2024-09-17T05:36:33.025497+0000","last_became_peered":"2024-09-17T05:36:33.025497+0000","last_unstale":"2024-09-17T05:36:34.008736+0000","last_undegraded":"2024-09-17T05:36:34.008736+0000","last_fullsized":"2024-09-17T05:36:34.008736+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:34.582552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363923+0000","last_change":"2024-09-17T05:36:32.006528+0000","last_active":"2024-09-17T05:36:34.363923+0000","last_peered":"2024-09-17T05:36:34.363923+0000","last_clean":"2024-09-17T05:36:34.363923+0000","last_became_active":"2024-09-17T05:36:32.006254+0000","last_became_peered":"2024-09-17T05:36:32.006254+0000","last_unstale":"2024-09-17T05:36:34.363923+0000","last_undegraded":"2024-09-17T05:36:34.363923+0000","last_fullsized":"2024-09-17T05:36:34.363923+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:31.804208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027595+0000","last_change":"2024-09-17T05:36:33.019304+0000","last_active":"2024-09-17T05:36:33.027595+0000","last_peered":"2024-09-17T05:36:33.027595+0000","last_clean":"2024-09-17T05:36:33.027595+0000","last_became_active":"2024-09-17T05:36:33.019012+0000","last_became_peered":"2024-09-17T05:36:33.019012+0000","last_unstale":"2024-09-17T05:36:33.027595+0000","last_undegraded":"2024-09-17T05:36:33.027595+0000","last_fullsized":"2024-09-17T05:36:33.027595+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:13.449104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363331+0000","last_change":"2024-09-17T05:36:32.008081+0000","last_active":"2024-09-17T05:36:34.363331+0000","last_peered":"2024-09-17T05:36:34.363331+0000","last_clean":"2024-09-17T05:36:34.363331+0000","last_became_active":"2024-09-17T05:36:32.007783+0000","last_became_peered":"2024-09-17T05:36:32.007783+0000","last_unstale":"2024-09-17T05:36:34.363331+0000","last_undegraded":"2024-09-17T05:36:34.363331+0000","last_fullsized":"2024-09-17T05:36:34.363331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:03.981516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033082+0000","last_change":"2024-09-17T05:36:33.031090+0000","last_active":"2024-09-17T05:36:33.033082+0000","last_peered":"2024-09-17T05:36:33.033082+0000","last_clean":"2024-09-17T05:36:33.033082+0000","last_became_active":"2024-09-17T05:36:33.030590+0000","last_became_peered":"2024-09-17T05:36:33.030590+0000","last_unstale":"2024-09-17T05:36:33.033082+0000","last_undegraded":"2024-09-17T05:36:33.033082+0000","last_fullsized":"2024-09-17T05:36:33.033082+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:16.656506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"102'1","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028798+0000","last_change":"2024-09-17T05:36:31.994407+0000","last_active":"2024-09-17T05:36:34.028798+0000","last_peered":"2024-09-17T05:36:34.028798+0000","last_clean":"2024-09-17T05:36:34.028798+0000","last_became_active":"2024-09-17T05:36:31.994090+0000","last_became_peered":"2024-09-17T05:36:31.994090+0000","last_unstale":"2024-09-17T05:36:34.028798+0000","last_undegraded":"2024-09-17T05:36:34.028798+0000","last_fullsized":"2024-09-17T05:36:34.028798+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:05.885445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010136+0000","last_change":"2024-09-17T05:36:33.039104+0000","last_active":"2024-09-17T05:36:34.010136+0000","last_peered":"2024-09-17T05:36:34.010136+0000","last_clean":"2024-09-17T05:36:34.010136+0000","last_became_active":"2024-09-17T05:36:33.038797+0000","last_became_peered":"2024-09-17T05:36:33.038797+0000","last_unstale":"2024-09-17T05:36:34.010136+0000","last_undegraded":"2024-09-17T05:36:34.010136+0000","last_fullsized":"2024-09-17T05:36:34.010136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:32.812669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.013521+0000","last_change":"2024-09-17T05:36:31.993191+0000","last_active":"2024-09-17T05:36:33.013521+0000","last_peered":"2024-09-17T05:36:33.013521+0000","last_clean":"2024-09-17T05:36:33.013521+0000","last_became_active":"2024-09-17T05:36:31.992802+0000","last_became_peered":"2024-09-17T05:36:31.992802+0000","last_unstale":"2024-09-17T05:36:33.013521+0000","last_undegraded":"2024-09-17T05:36:33.013521+0000","last_fullsized":"2024-09-17T05:36:33.013521+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:01.201626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.027730+0000","last_change":"2024-09-17T05:36:33.021198+0000","last_active":"2024-09-17T05:36:33.027730+0000","last_peered":"2024-09-17T05:36:33.027730+0000","last_clean":"2024-09-17T05:36:33.027730+0000","last_became_active":"2024-09-17T05:36:33.019760+0000","last_became_peered":"2024-09-17T05:36:33.019760+0000","last_unstale":"2024-09-17T05:36:33.027730+0000","last_undegraded":"2024-09-17T05:36:33.027730+0000","last_fullsized":"2024-09-17T05:36:33.027730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:46.561978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.020926+0000","last_change":"2024-09-17T05:36:32.008293+0000","last_active":"2024-09-17T05:36:33.020926+0000","last_peered":"2024-09-17T05:36:33.020926+0000","last_clean":"2024-09-17T05:36:33.020926+0000","last_became_active":"2024-09-17T05:36:32.007997+0000","last_became_peered":"2024-09-17T05:36:32.007997+0000","last_unstale":"2024-09-17T05:36:33.020926+0000","last_undegraded":"2024-09-17T05:36:33.020926+0000","last_fullsized":"2024-09-17T05:36:33.020926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:12.542988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364239+0000","last_change":"2024-09-17T05:36:33.031240+0000","last_active":"2024-09-17T05:36:34.364239+0000","last_peered":"2024-09-17T05:36:34.364239+0000","last_clean":"2024-09-17T05:36:34.364239+0000","last_became_active":"2024-09-17T05:36:33.030592+0000","last_became_peered":"2024-09-17T05:36:33.030592+0000","last_unstale":"2024-09-17T05:36:34.364239+0000","last_undegraded":"2024-09-17T05:36:34.364239+0000","last_fullsized":"2024-09-17T05:36:34.364239+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:21.424105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.029228+0000","last_change":"2024-09-17T05:36:32.004372+0000","last_active":"2024-09-17T05:36:34.029228+0000","last_peered":"2024-09-17T05:36:34.029228+0000","last_clean":"2024-09-17T05:36:34.029228+0000","last_became_active":"2024-09-17T05:36:32.004091+0000","last_became_peered":"2024-09-17T05:36:32.004091+0000","last_unstale":"2024-09-17T05:36:34.029228+0000","last_undegraded":"2024-09-17T05:36:34.029228+0000","last_fullsized":"2024-09-17T05:36:34.029228+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.950627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.023906+0000","last_change":"2024-09-17T05:36:33.017900+0000","last_active":"2024-09-17T05:36:33.023906+0000","last_peered":"2024-09-17T05:36:33.023906+0000","last_clean":"2024-09-17T05:36:33.023906+0000","last_became_active":"2024-09-17T05:36:33.017597+0000","last_became_peered":"2024-09-17T05:36:33.017597+0000","last_unstale":"2024-09-17T05:36:33.023906+0000","last_undegraded":"2024-09-17T05:36:33.023906+0000","last_fullsized":"2024-09-17T05:36:33.023906+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:44.314644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.364154+0000","last_change":"2024-09-17T05:36:32.012238+0000","last_active":"2024-09-17T05:36:34.364154+0000","last_peered":"2024-09-17T05:36:34.364154+0000","last_clean":"2024-09-17T05:36:34.364154+0000","last_became_active":"2024-09-17T05:36:32.009314+0000","last_became_peered":"2024-09-17T05:36:32.009314+0000","last_unstale":"2024-09-17T05:36:34.364154+0000","last_undegraded":"2024-09-17T05:36:34.364154+0000","last_fullsized":"2024-09-17T05:36:34.364154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:03.051818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.037476+0000","last_change":"2024-09-17T05:36:33.035844+0000","last_active":"2024-09-17T05:36:33.037476+0000","last_peered":"2024-09-17T05:36:33.037476+0000","last_clean":"2024-09-17T05:36:33.037476+0000","last_became_active":"2024-09-17T05:36:33.035445+0000","last_became_peered":"2024-09-17T05:36:33.035445+0000","last_unstale":"2024-09-17T05:36:33.037476+0000","last_undegraded":"2024-09-17T05:36:33.037476+0000","last_fullsized":"2024-09-17T05:36:33.037476+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:57.304253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"102'3","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.037081+0000","last_change":"2024-09-17T05:36:32.004701+0000","last_active":"2024-09-17T05:36:33.037081+0000","last_peered":"2024-09-17T05:36:33.037081+0000","last_clean":"2024-09-17T05:36:33.037081+0000","last_became_active":"2024-09-17T05:36:32.004441+0000","last_became_peered":"2024-09-17T05:36:32.004441+0000","last_unstale":"2024-09-17T05:36:33.037081+0000","last_undegraded":"2024-09-17T05:36:33.037081+0000","last_fullsized":"2024-09-17T05:36:33.037081+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:36:20.424323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033669+0000","last_change":"2024-09-17T05:36:33.030406+0000","last_active":"2024-09-17T05:36:33.033669+0000","last_peered":"2024-09-17T05:36:33.033669+0000","last_clean":"2024-09-17T05:36:33.033669+0000","last_became_active":"2024-09-17T05:36:33.030132+0000","last_became_peered":"2024-09-17T05:36:33.030132+0000","last_unstale":"2024-09-17T05:36:33.033669+0000","last_undegraded":"2024-09-17T05:36:33.033669+0000","last_fullsized":"2024-09-17T05:36:33.033669+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:17.102365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"102'6","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.045329+0000","last_change":"2024-09-17T05:36:32.006369+0000","last_active":"2024-09-17T05:36:33.045329+0000","last_peered":"2024-09-17T05:36:33.045329+0000","last_clean":"2024-09-17T05:36:33.045329+0000","last_became_active":"2024-09-17T05:36:32.005971+0000","last_became_peered":"2024-09-17T05:36:32.005971+0000","last_unstale":"2024-09-17T05:36:33.045329+0000","last_undegraded":"2024-09-17T05:36:33.045329+0000","last_fullsized":"2024-09-17T05:36:33.045329+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:11:52.685453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363411+0000","last_change":"2024-09-17T05:36:33.021619+0000","last_active":"2024-09-17T05:36:34.363411+0000","last_peered":"2024-09-17T05:36:34.363411+0000","last_clean":"2024-09-17T05:36:34.363411+0000","last_became_active":"2024-09-17T05:36:33.021277+0000","last_became_peered":"2024-09-17T05:36:33.021277+0000","last_unstale":"2024-09-17T05:36:34.363411+0000","last_undegraded":"2024-09-17T05:36:34.363411+0000","last_fullsized":"2024-09-17T05:36:34.363411+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:19.131661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"102'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.045100+0000","last_change":"2024-09-17T05:36:32.000574+0000","last_active":"2024-09-17T05:36:33.045100+0000","last_peered":"2024-09-17T05:36:33.045100+0000","last_clean":"2024-09-17T05:36:33.045100+0000","last_became_active":"2024-09-17T05:36:32.000269+0000","last_became_peered":"2024-09-17T05:36:32.000269+0000","last_unstale":"2024-09-17T05:36:33.045100+0000","last_undegraded":"2024-09-17T05:36:33.045100+0000","last_fullsized":"2024-09-17T05:36:33.045100+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:03:11.550824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028716+0000","last_change":"2024-09-17T05:36:33.045516+0000","last_active":"2024-09-17T05:36:34.028716+0000","last_peered":"2024-09-17T05:36:34.028716+0000","last_clean":"2024-09-17T05:36:34.028716+0000","last_became_active":"2024-09-17T05:36:33.044505+0000","last_became_peered":"2024-09-17T05:36:33.044505+0000","last_unstale":"2024-09-17T05:36:34.028716+0000","last_undegraded":"2024-09-17T05:36:34.028716+0000","last_fullsized":"2024-09-17T05:36:34.028716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:19.355781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363567+0000","last_change":"2024-09-17T05:36:32.011939+0000","last_active":"2024-09-17T05:36:34.363567+0000","last_peered":"2024-09-17T05:36:34.363567+0000","last_clean":"2024-09-17T05:36:34.363567+0000","last_became_active":"2024-09-17T05:36:32.011652+0000","last_became_peered":"2024-09-17T05:36:32.011652+0000","last_unstale":"2024-09-17T05:36:34.363567+0000","last_undegraded":"2024-09-17T05:36:34.363567+0000","last_fullsized":"2024-09-17T05:36:34.363567+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:59.393647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033521+0000","last_change":"2024-09-17T05:36:33.031238+0000","last_active":"2024-09-17T05:36:33.033521+0000","last_peered":"2024-09-17T05:36:33.033521+0000","last_clean":"2024-09-17T05:36:33.033521+0000","last_became_active":"2024-09-17T05:36:33.030889+0000","last_became_peered":"2024-09-17T05:36:33.030889+0000","last_unstale":"2024-09-17T05:36:33.033521+0000","last_undegraded":"2024-09-17T05:36:33.033521+0000","last_fullsized":"2024-09-17T05:36:33.033521+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:12.051776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.008566+0000","last_change":"2024-09-17T05:36:32.003592+0000","last_active":"2024-09-17T05:36:33.008566+0000","last_peered":"2024-09-17T05:36:33.008566+0000","last_clean":"2024-09-17T05:36:33.008566+0000","last_became_active":"2024-09-17T05:36:32.003315+0000","last_became_peered":"2024-09-17T05:36:32.003315+0000","last_unstale":"2024-09-17T05:36:33.008566+0000","last_undegraded":"2024-09-17T05:36:33.008566+0000","last_fullsized":"2024-09-17T05:36:33.008566+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:15.089519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028573+0000","last_change":"2024-09-17T05:36:33.040736+0000","last_active":"2024-09-17T05:36:34.028573+0000","last_peered":"2024-09-17T05:36:34.028573+0000","last_clean":"2024-09-17T05:36:34.028573+0000","last_became_active":"2024-09-17T05:36:33.037647+0000","last_became_peered":"2024-09-17T05:36:33.037647+0000","last_unstale":"2024-09-17T05:36:34.028573+0000","last_undegraded":"2024-09-17T05:36:34.028573+0000","last_fullsized":"2024-09-17T05:36:34.028573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:54.657188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"102'8","reported_seq":23,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363476+0000","last_change":"2024-09-17T05:36:32.005582+0000","last_active":"2024-09-17T05:36:34.363476+0000","last_peered":"2024-09-17T05:36:34.363476+0000","last_clean":"2024-09-17T05:36:34.363476+0000","last_became_active":"2024-09-17T05:36:32.005285+0000","last_became_peered":"2024-09-17T05:36:32.005285+0000","last_unstale":"2024-09-17T05:36:34.363476+0000","last_undegraded":"2024-09-17T05:36:34.363476+0000","last_fullsized":"2024-09-17T05:36:34.363476+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.950627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.032939+0000","last_change":"2024-09-17T05:36:33.029109+0000","last_active":"2024-09-17T05:36:33.032939+0000","last_peered":"2024-09-17T05:36:33.032939+0000","last_clean":"2024-09-17T05:36:33.032939+0000","last_became_active":"2024-09-17T05:36:33.028829+0000","last_became_peered":"2024-09-17T05:36:33.028829+0000","last_unstale":"2024-09-17T05:36:33.032939+0000","last_undegraded":"2024-09-17T05:36:33.032939+0000","last_fullsized":"2024-09-17T05:36:33.032939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:01.741595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"102'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.040678+0000","last_change":"2024-09-17T05:36:32.002068+0000","last_active":"2024-09-17T05:36:33.040678+0000","last_peered":"2024-09-17T05:36:33.040678+0000","last_clean":"2024-09-17T05:36:33.040678+0000","last_became_active":"2024-09-17T05:36:32.001692+0000","last_became_peered":"2024-09-17T05:36:32.001692+0000","last_unstale":"2024-09-17T05:36:33.040678+0000","last_undegraded":"2024-09-17T05:36:33.040678+0000","last_fullsized":"2024-09-17T05:36:33.040678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:27.129580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.024632+0000","last_change":"2024-09-17T05:36:33.020613+0000","last_active":"2024-09-17T05:36:33.024632+0000","last_peered":"2024-09-17T05:36:33.024632+0000","last_clean":"2024-09-17T05:36:33.024632+0000","last_became_active":"2024-09-17T05:36:33.020327+0000","last_became_peered":"2024-09-17T05:36:33.020327+0000","last_unstale":"2024-09-17T05:36:33.024632+0000","last_undegraded":"2024-09-17T05:36:33.024632+0000","last_fullsized":"2024-09-17T05:36:33.024632+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:28.173152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"102'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.038148+0000","last_change":"2024-09-17T05:36:31.992413+0000","last_active":"2024-09-17T05:36:33.038148+0000","last_peered":"2024-09-17T05:36:33.038148+0000","last_clean":"2024-09-17T05:36:33.038148+0000","last_became_active":"2024-09-17T05:36:31.991962+0000","last_became_peered":"2024-09-17T05:36:31.991962+0000","last_unstale":"2024-09-17T05:36:33.038148+0000","last_undegraded":"2024-09-17T05:36:33.038148+0000","last_fullsized":"2024-09-17T05:36:33.038148+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:32:59.440061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.013940+0000","last_change":"2024-09-17T05:36:33.012396+0000","last_active":"2024-09-17T05:36:33.013940+0000","last_peered":"2024-09-17T05:36:33.013940+0000","last_clean":"2024-09-17T05:36:33.013940+0000","last_became_active":"2024-09-17T05:36:33.012169+0000","last_became_peered":"2024-09-17T05:36:33.012169+0000","last_unstale":"2024-09-17T05:36:33.013940+0000","last_undegraded":"2024-09-17T05:36:33.013940+0000","last_fullsized":"2024-09-17T05:36:33.013940+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:24.501901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.013407+0000","last_change":"2024-09-17T05:36:32.003677+0000","last_active":"2024-09-17T05:36:33.013407+0000","last_peered":"2024-09-17T05:36:33.013407+0000","last_clean":"2024-09-17T05:36:33.013407+0000","last_became_active":"2024-09-17T05:36:32.003406+0000","last_became_peered":"2024-09-17T05:36:32.003406+0000","last_unstale":"2024-09-17T05:36:33.013407+0000","last_undegraded":"2024-09-17T05:36:33.013407+0000","last_fullsized":"2024-09-17T05:36:33.013407+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:33.642857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.044098+0000","last_change":"2024-09-17T05:36:33.040193+0000","last_active":"2024-09-17T05:36:33.044098+0000","last_peered":"2024-09-17T05:36:33.044098+0000","last_clean":"2024-09-17T05:36:33.044098+0000","last_became_active":"2024-09-17T05:36:33.039510+0000","last_became_peered":"2024-09-17T05:36:33.039510+0000","last_unstale":"2024-09-17T05:36:33.044098+0000","last_undegraded":"2024-09-17T05:36:33.044098+0000","last_fullsized":"2024-09-17T05:36:33.044098+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:33.703862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"102'2","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363454+0000","last_change":"2024-09-17T05:36:32.009432+0000","last_active":"2024-09-17T05:36:34.363454+0000","last_peered":"2024-09-17T05:36:34.363454+0000","last_clean":"2024-09-17T05:36:34.363454+0000","last_became_active":"2024-09-17T05:36:32.009152+0000","last_became_peered":"2024-09-17T05:36:32.009152+0000","last_unstale":"2024-09-17T05:36:34.363454+0000","last_undegraded":"2024-09-17T05:36:34.363454+0000","last_fullsized":"2024-09-17T05:36:34.363454+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:39.496193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.048295+0000","last_change":"2024-09-17T05:36:33.047360+0000","last_active":"2024-09-17T05:36:33.048295+0000","last_peered":"2024-09-17T05:36:33.048295+0000","last_clean":"2024-09-17T05:36:33.048295+0000","last_became_active":"2024-09-17T05:36:33.047080+0000","last_became_peered":"2024-09-17T05:36:33.047080+0000","last_unstale":"2024-09-17T05:36:33.048295+0000","last_undegraded":"2024-09-17T05:36:33.048295+0000","last_fullsized":"2024-09-17T05:36:33.048295+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:49.016055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.010788+0000","last_change":"2024-09-17T05:36:31.991151+0000","last_active":"2024-09-17T05:36:33.010788+0000","last_peered":"2024-09-17T05:36:33.010788+0000","last_clean":"2024-09-17T05:36:33.010788+0000","last_became_active":"2024-09-17T05:36:31.990659+0000","last_became_peered":"2024-09-17T05:36:31.990659+0000","last_unstale":"2024-09-17T05:36:33.010788+0000","last_undegraded":"2024-09-17T05:36:33.010788+0000","last_fullsized":"2024-09-17T05:36:33.010788+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:12.582231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.363568+0000","last_change":"2024-09-17T05:36:33.026731+0000","last_active":"2024-09-17T05:36:34.363568+0000","last_peered":"2024-09-17T05:36:34.363568+0000","last_clean":"2024-09-17T05:36:34.363568+0000","last_became_active":"2024-09-17T05:36:33.026438+0000","last_became_peered":"2024-09-17T05:36:33.026438+0000","last_unstale":"2024-09-17T05:36:34.363568+0000","last_undegraded":"2024-09-17T05:36:34.363568+0000","last_fullsized":"2024-09-17T05:36:34.363568+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:19.355781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.008518+0000","last_change":"2024-09-17T05:36:31.994567+0000","last_active":"2024-09-17T05:36:34.008518+0000","last_peered":"2024-09-17T05:36:34.008518+0000","last_clean":"2024-09-17T05:36:34.008518+0000","last_became_active":"2024-09-17T05:36:31.994238+0000","last_became_peered":"2024-09-17T05:36:31.994238+0000","last_unstale":"2024-09-17T05:36:34.008518+0000","last_undegraded":"2024-09-17T05:36:34.008518+0000","last_fullsized":"2024-09-17T05:36:34.008518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:18.347320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.040752+0000","last_change":"2024-09-17T05:36:33.039414+0000","last_active":"2024-09-17T05:36:33.040752+0000","last_peered":"2024-09-17T05:36:33.040752+0000","last_clean":"2024-09-17T05:36:33.040752+0000","last_became_active":"2024-09-17T05:36:33.039084+0000","last_became_peered":"2024-09-17T05:36:33.039084+0000","last_unstale":"2024-09-17T05:36:33.040752+0000","last_undegraded":"2024-09-17T05:36:33.040752+0000","last_fullsized":"2024-09-17T05:36:33.040752+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:00.658895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028199+0000","last_change":"2024-09-17T05:36:32.002615+0000","last_active":"2024-09-17T05:36:34.028199+0000","last_peered":"2024-09-17T05:36:34.028199+0000","last_clean":"2024-09-17T05:36:34.028199+0000","last_became_active":"2024-09-17T05:36:32.002027+0000","last_became_peered":"2024-09-17T05:36:32.002027+0000","last_unstale":"2024-09-17T05:36:34.028199+0000","last_undegraded":"2024-09-17T05:36:34.028199+0000","last_fullsized":"2024-09-17T05:36:34.028199+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:24:40.433841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033730+0000","last_change":"2024-09-17T05:36:33.030122+0000","last_active":"2024-09-17T05:36:33.033730+0000","last_peered":"2024-09-17T05:36:33.033730+0000","last_clean":"2024-09-17T05:36:33.033730+0000","last_became_active":"2024-09-17T05:36:33.029501+0000","last_became_peered":"2024-09-17T05:36:33.029501+0000","last_unstale":"2024-09-17T05:36:33.033730+0000","last_undegraded":"2024-09-17T05:36:33.033730+0000","last_fullsized":"2024-09-17T05:36:33.033730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:27.883776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"102'2","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.038619+0000","last_change":"2024-09-17T05:36:32.008486+0000","last_active":"2024-09-17T05:36:33.038619+0000","last_peered":"2024-09-17T05:36:33.038619+0000","last_clean":"2024-09-17T05:36:33.038619+0000","last_became_active":"2024-09-17T05:36:32.006776+0000","last_became_peered":"2024-09-17T05:36:32.006776+0000","last_unstale":"2024-09-17T05:36:33.038619+0000","last_undegraded":"2024-09-17T05:36:33.038619+0000","last_fullsized":"2024-09-17T05:36:33.038619+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:48.058349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.043912+0000","last_change":"2024-09-17T05:36:33.039816+0000","last_active":"2024-09-17T05:36:33.043912+0000","last_peered":"2024-09-17T05:36:33.043912+0000","last_clean":"2024-09-17T05:36:33.043912+0000","last_became_active":"2024-09-17T05:36:33.039524+0000","last_became_peered":"2024-09-17T05:36:33.039524+0000","last_unstale":"2024-09-17T05:36:33.043912+0000","last_undegraded":"2024-09-17T05:36:33.043912+0000","last_fullsized":"2024-09-17T05:36:33.043912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:54.978590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.019784+0000","last_change":"2024-09-17T05:36:32.006835+0000","last_active":"2024-09-17T05:36:33.019784+0000","last_peered":"2024-09-17T05:36:33.019784+0000","last_clean":"2024-09-17T05:36:33.019784+0000","last_became_active":"2024-09-17T05:36:32.006488+0000","last_became_peered":"2024-09-17T05:36:32.006488+0000","last_unstale":"2024-09-17T05:36:33.019784+0000","last_undegraded":"2024-09-17T05:36:33.019784+0000","last_fullsized":"2024-09-17T05:36:33.019784+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:02.697756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.029063+0000","last_change":"2024-09-17T05:36:33.038063+0000","last_active":"2024-09-17T05:36:34.029063+0000","last_peered":"2024-09-17T05:36:34.029063+0000","last_clean":"2024-09-17T05:36:34.029063+0000","last_became_active":"2024-09-17T05:36:33.037604+0000","last_became_peered":"2024-09-17T05:36:33.037604+0000","last_unstale":"2024-09-17T05:36:34.029063+0000","last_undegraded":"2024-09-17T05:36:34.029063+0000","last_fullsized":"2024-09-17T05:36:34.029063+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:33.747047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.013599+0000","last_change":"2024-09-17T05:36:32.006594+0000","last_active":"2024-09-17T05:36:33.013599+0000","last_peered":"2024-09-17T05:36:33.013599+0000","last_clean":"2024-09-17T05:36:33.013599+0000","last_became_active":"2024-09-17T05:36:32.006312+0000","last_became_peered":"2024-09-17T05:36:32.006312+0000","last_unstale":"2024-09-17T05:36:33.013599+0000","last_undegraded":"2024-09-17T05:36:33.013599+0000","last_fullsized":"2024-09-17T05:36:33.013599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:37:37.815565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.028490+0000","last_change":"2024-09-17T05:36:33.020933+0000","last_active":"2024-09-17T05:36:33.028490+0000","last_peered":"2024-09-17T05:36:33.028490+0000","last_clean":"2024-09-17T05:36:33.028490+0000","last_became_active":"2024-09-17T05:36:33.020637+0000","last_became_peered":"2024-09-17T05:36:33.020637+0000","last_unstale":"2024-09-17T05:36:33.028490+0000","last_undegraded":"2024-09-17T05:36:33.028490+0000","last_fullsized":"2024-09-17T05:36:33.028490+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:40.154175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.011490+0000","last_change":"2024-09-17T05:36:32.003577+0000","last_active":"2024-09-17T05:36:33.011490+0000","last_peered":"2024-09-17T05:36:33.011490+0000","last_clean":"2024-09-17T05:36:33.011490+0000","last_became_active":"2024-09-17T05:36:32.002589+0000","last_became_peered":"2024-09-17T05:36:32.002589+0000","last_unstale":"2024-09-17T05:36:33.011490+0000","last_undegraded":"2024-09-17T05:36:33.011490+0000","last_fullsized":"2024-09-17T05:36:33.011490+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:30.756283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033360+0000","last_change":"2024-09-17T05:36:33.030019+0000","last_active":"2024-09-17T05:36:33.033360+0000","last_peered":"2024-09-17T05:36:33.033360+0000","last_clean":"2024-09-17T05:36:33.033360+0000","last_became_active":"2024-09-17T05:36:33.029746+0000","last_became_peered":"2024-09-17T05:36:33.029746+0000","last_unstale":"2024-09-17T05:36:33.033360+0000","last_undegraded":"2024-09-17T05:36:33.033360+0000","last_fullsized":"2024-09-17T05:36:33.033360+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:42.869668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028934+0000","last_change":"2024-09-17T05:36:32.012434+0000","last_active":"2024-09-17T05:36:34.028934+0000","last_peered":"2024-09-17T05:36:34.028934+0000","last_clean":"2024-09-17T05:36:34.028934+0000","last_became_active":"2024-09-17T05:36:32.012180+0000","last_became_peered":"2024-09-17T05:36:32.012180+0000","last_unstale":"2024-09-17T05:36:34.028934+0000","last_undegraded":"2024-09-17T05:36:34.028934+0000","last_fullsized":"2024-09-17T05:36:34.028934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:59.393647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.047969+0000","last_change":"2024-09-17T05:36:33.047189+0000","last_active":"2024-09-17T05:36:33.047969+0000","last_peered":"2024-09-17T05:36:33.047969+0000","last_clean":"2024-09-17T05:36:33.047969+0000","last_became_active":"2024-09-17T05:36:33.046883+0000","last_became_peered":"2024-09-17T05:36:33.046883+0000","last_unstale":"2024-09-17T05:36:33.047969+0000","last_undegraded":"2024-09-17T05:36:33.047969+0000","last_fullsized":"2024-09-17T05:36:33.047969+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:38.306111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.010999+0000","last_change":"2024-09-17T05:36:32.001897+0000","last_active":"2024-09-17T05:36:33.010999+0000","last_peered":"2024-09-17T05:36:33.010999+0000","last_clean":"2024-09-17T05:36:33.010999+0000","last_became_active":"2024-09-17T05:36:32.001626+0000","last_became_peered":"2024-09-17T05:36:32.001626+0000","last_unstale":"2024-09-17T05:36:33.010999+0000","last_undegraded":"2024-09-17T05:36:33.010999+0000","last_fullsized":"2024-09-17T05:36:33.010999+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:58.710334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.033579+0000","last_change":"2024-09-17T05:36:33.031152+0000","last_active":"2024-09-17T05:36:33.033579+0000","last_peered":"2024-09-17T05:36:33.033579+0000","last_clean":"2024-09-17T05:36:33.033579+0000","last_became_active":"2024-09-17T05:36:33.030868+0000","last_became_peered":"2024-09-17T05:36:33.030868+0000","last_unstale":"2024-09-17T05:36:33.033579+0000","last_undegraded":"2024-09-17T05:36:33.033579+0000","last_fullsized":"2024-09-17T05:36:33.033579+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:12.715902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.010257+0000","last_change":"2024-09-17T05:36:32.009452+0000","last_active":"2024-09-17T05:36:34.010257+0000","last_peered":"2024-09-17T05:36:34.010257+0000","last_clean":"2024-09-17T05:36:34.010257+0000","last_became_active":"2024-09-17T05:36:32.009160+0000","last_became_peered":"2024-09-17T05:36:32.009160+0000","last_unstale":"2024-09-17T05:36:34.010257+0000","last_undegraded":"2024-09-17T05:36:34.010257+0000","last_fullsized":"2024-09-17T05:36:34.010257+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:58.950627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.028537+0000","last_change":"2024-09-17T05:36:33.025924+0000","last_active":"2024-09-17T05:36:33.028537+0000","last_peered":"2024-09-17T05:36:33.028537+0000","last_clean":"2024-09-17T05:36:33.028537+0000","last_became_active":"2024-09-17T05:36:33.025654+0000","last_became_peered":"2024-09-17T05:36:33.025654+0000","last_unstale":"2024-09-17T05:36:33.028537+0000","last_undegraded":"2024-09-17T05:36:33.028537+0000","last_fullsized":"2024-09-17T05:36:33.028537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:22.208384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.028316+0000","last_change":"2024-09-17T05:36:32.002306+0000","last_active":"2024-09-17T05:36:34.028316+0000","last_peered":"2024-09-17T05:36:34.028316+0000","last_clean":"2024-09-17T05:36:34.028316+0000","last_became_active":"2024-09-17T05:36:32.001676+0000","last_became_peered":"2024-09-17T05:36:32.001676+0000","last_unstale":"2024-09-17T05:36:34.028316+0000","last_undegraded":"2024-09-17T05:36:34.028316+0000","last_fullsized":"2024-09-17T05:36:34.028316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:03.981516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T05:36:34.029033+0000","last_change":"2024-09-17T05:36:33.039006+0000","last_active":"2024-09-17T05:36:34.029033+0000","last_peered":"2024-09-17T05:36:34.029033+0000","last_clean":"2024-09-17T05:36:34.029033+0000","last_became_active":"2024-09-17T05:36:33.037887+0000","last_became_peered":"2024-09-17T05:36:33.037887+0000","last_unstale":"2024-09-17T05:36:34.029033+0000","last_undegraded":"2024-09-17T05:36:34.029033+0000","last_fullsized":"2024-09-17T05:36:34.029033+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:31.977144+0000","last_clean_scrub_stamp":"2024-09-17T05:36:31.977144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:53.765357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"102'1","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.039833+0000","last_change":"2024-09-17T05:36:32.008404+0000","last_active":"2024-09-17T05:36:33.039833+0000","last_peered":"2024-09-17T05:36:33.039833+0000","last_clean":"2024-09-17T05:36:33.039833+0000","last_became_active":"2024-09-17T05:36:32.007744+0000","last_became_peered":"2024-09-17T05:36:32.007744+0000","last_unstale":"2024-09-17T05:36:33.039833+0000","last_undegraded":"2024-09-17T05:36:33.039833+0000","last_fullsized":"2024-09-17T05:36:33.039833+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:36:30.968683+0000","last_clean_scrub_stamp":"2024-09-17T05:36:30.968683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:44.549268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":263,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T05:36:33.006626+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:36:33.006626+0000","last_peered":"2024-09-17T05:36:33.006626+0000","last_clean":"2024-09-17T05:36:33.006626+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:36:33.006626+0000","last_undegraded":"2024-09-17T05:36:33.006626+0000","last_fullsized":"2024-09-17T05:36:33.006626+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64288,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307552,"statfs":{"total":96636764160,"available":96570933248,"internally_reserved":0,"allocated":1257472,"data_stored":544977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144808,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227032,"statfs":{"total":96636764160,"available":96488480768,"internally_reserved":0,"allocated":1253376,"data_stored":544395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":146995654},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104560,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267280,"statfs":{"total":96636764160,"available":96529694720,"internally_reserved":0,"allocated":1269760,"data_stored":551333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122600,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249240,"statfs":{"total":96636764160,"available":96511221760,"internally_reserved":0,"allocated":1265664,"data_stored":544589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122592,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249248,"statfs":{"total":96636764160,"available":96511229952,"internally_reserved":0,"allocated":1257472,"data_stored":544465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509584,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64888,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306952,"statfs":{"total":96636764160,"available":96570318848,"internally_reserved":0,"allocated":1867776,"data_stored":1136487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509582,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82352,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94289488,"statfs":{"total":96636764160,"available":96552435712,"internally_reserved":0,"allocated":1265664,"data_stored":550751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":83032523},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542288,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127352,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244488,"statfs":{"total":96636764160,"available":96506355712,"internally_reserved":0,"allocated":1867776,"data_stored":1136431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:36:37.525 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:36:37.528 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:36:37.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 -- 172.21.15.122:0/2396209398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8014003a40 msgr2=0x7f8014003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 --2- 172.21.15.122:0/2396209398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8014003a40 0x7f8014003e60 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f80200012d0 tx=0x7f802001e920 comp rx=0 tx=0).stop 2024-09-17T05:36:37.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f802fe00640 1 -- 172.21.15.122:0/2396209398 reap_dead start 2024-09-17T05:36:37.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 -- 172.21.15.122:0/2396209398 shutdown_connections 2024-09-17T05:36:37.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 -- 172.21.15.122:0/2396209398 >> 172.21.15.122:0/2396209398 conn(0x7f80300b18f0 msgr2=0x7f80300b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:37.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 -- 172.21.15.122:0/2396209398 shutdown_connections 2024-09-17T05:36:37.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 -- 172.21.15.122:0/2396209398 wait complete. 2024-09-17T05:36:37.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 Processor -- start 2024-09-17T05:36:37.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 -- start start 2024-09-17T05:36:37.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.722+0000 7f8036400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8014003a40 0x7f803009d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:37.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8036400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8030074220 0x7f803014f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:37.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8036400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80300c5960 0x7f80301faf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8036400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80300ce810 con 0x7f80300c5960 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8036400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80300ce690 con 0x7f8014003a40 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8036400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8030165950 con 0x7f8030074220 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80300c5960 0x7f80301faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80300c5960 0x7f80301faf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44654/0 (socket says 172.21.15.122:44654) 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8014003a40 0x7f803009d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 -- 172.21.15.122:0/1945329657 learned_addr learned my addr 172.21.15.122:0/1945329657 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8026a00640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8030074220 0x7f803014f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 -- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8030074220 msgr2=0x7f803014f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8030074220 0x7f803014f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 -- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8014003a40 msgr2=0x7f803009d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8014003a40 0x7f803009d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 -- 172.21.15.122:0/1945329657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803014fd90 con 0x7f80300c5960 2024-09-17T05:36:37.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f802fe00640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80300c5960 0x7f80301faf30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f80300af370 tx=0x7f802800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:37.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8027e00640 1 -- 172.21.15.122:0/1945329657 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028017070 con 0x7f80300c5960 2024-09-17T05:36:37.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8027e00640 1 -- 172.21.15.122:0/1945329657 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8028015070 con 0x7f80300c5960 2024-09-17T05:36:37.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80300af4d0 con 0x7f80300c5960 2024-09-17T05:36:37.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8027e00640 1 -- 172.21.15.122:0/1945329657 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802800eea0 con 0x7f80300c5960 2024-09-17T05:36:37.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80300afa50 con 0x7f80300c5960 2024-09-17T05:36:37.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80300b2bf0 con 0x7f80300c5960 2024-09-17T05:36:37.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.726+0000 7f8027e00640 1 -- 172.21.15.122:0/1945329657 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f8028011440 con 0x7f80300c5960 2024-09-17T05:36:37.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.730+0000 7f8027e00640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c074880 0x7f800c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:37.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.730+0000 7f8027e00640 1 -- 172.21.15.122:0/1945329657 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f80280935c0 con 0x7f80300c5960 2024-09-17T05:36:37.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.730+0000 7f802f400640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c074880 0x7f800c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:37.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.730+0000 7f802f400640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c074880 0x7f800c076d40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f80200016c0 tx=0x7f802001b710 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:37.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.734+0000 7f8027e00640 1 -- 172.21.15.122:0/1945329657 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f802805fa80 con 0x7f80300c5960 2024-09-17T05:36:37.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.958+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8030070090 con 0x7f80300c5960 2024-09-17T05:36:37.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.962+0000 7f8027e00640 1 -- 172.21.15.122:0/1945329657 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f80280644b0 con 0x7f80300c5960 2024-09-17T05:36:37.958 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:36:37.958 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:36:37.958 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:36:37.958 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:36:37.958 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:36:37.958 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:36:37.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.962+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c074880 msgr2=0x7f800c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.962+0000 7f8036400640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c074880 0x7f800c076d40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f80200016c0 tx=0x7f802001b710 comp rx=0 tx=0).stop 2024-09-17T05:36:37.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.962+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80300c5960 msgr2=0x7f80301faf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:37.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.962+0000 7f8036400640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80300c5960 0x7f80301faf30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f80300af370 tx=0x7f802800e720 comp rx=0 tx=0).stop 2024-09-17T05:36:37.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.966+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 shutdown_connections 2024-09-17T05:36:37.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.966+0000 7f8036400640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f800c074880 0x7f800c076d40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.966+0000 7f8036400640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80300c5960 0x7f80301faf30 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.966+0000 7f8036400640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8030074220 0x7f803014f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.966+0000 7f8036400640 1 --2- 172.21.15.122:0/1945329657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8014003a40 0x7f803009d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:37.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.966+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 >> 172.21.15.122:0/1945329657 conn(0x7f80300b18f0 msgr2=0x7f803015d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:37.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.966+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 shutdown_connections 2024-09-17T05:36:37.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:37.966+0000 7f8036400640 1 -- 172.21.15.122:0/1945329657 wait complete. 2024-09-17T05:36:37.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:36:38.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.126+0000 7f5253200640 1 -- 172.21.15.122:0/3114182656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f524c15f000 msgr2=0x7f524c0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.126+0000 7f5253200640 1 --2- 172.21.15.122:0/3114182656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f524c15f000 0x7f524c0d01b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5240009900 tx=0x7f524001c850 comp rx=0 tx=0).stop 2024-09-17T05:36:38.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.126+0000 7f5253200640 1 -- 172.21.15.122:0/3114182656 shutdown_connections 2024-09-17T05:36:38.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.126+0000 7f5253200640 1 --2- 172.21.15.122:0/3114182656 >> v2:172.21.15.173:3300/0 conn(0x7f524c0cefb0 0x7f524c0cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.126+0000 7f5253200640 1 --2- 172.21.15.122:0/3114182656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f524c15f000 0x7f524c0d01b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.126+0000 7f5253200640 1 -- 172.21.15.122:0/3114182656 >> 172.21.15.122:0/3114182656 conn(0x7f524c09aa10 msgr2=0x7f524c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:38.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.126+0000 7f5253200640 1 -- 172.21.15.122:0/3114182656 shutdown_connections 2024-09-17T05:36:38.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.126+0000 7f5253200640 1 -- 172.21.15.122:0/3114182656 wait complete. 2024-09-17T05:36:38.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 Processor -- start 2024-09-17T05:36:38.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 -- start start 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f524c0ccaf0 0x7f524c14cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c0cefb0 0x7f524c0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f524c15f000 0x7f524c0c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f524c1692e0 con 0x7f524c0cefb0 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f524c169160 con 0x7f524c15f000 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f524c169460 con 0x7f524c0ccaf0 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c0cefb0 0x7f524c0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c0cefb0 0x7f524c0c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44658/0 (socket says 172.21.15.122:44658) 2024-09-17T05:36:38.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 -- 172.21.15.122:0/3097751933 learned_addr learned my addr 172.21.15.122:0/3097751933 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:38.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251e00640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f524c0ccaf0 0x7f524c14cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5252800640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f524c15f000 0x7f524c0c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 -- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f524c0ccaf0 msgr2=0x7f524c14cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f524c0ccaf0 0x7f524c14cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 -- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f524c15f000 msgr2=0x7f524c0c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f524c15f000 0x7f524c0c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 -- 172.21.15.122:0/3097751933 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c14d420 con 0x7f524c0cefb0 2024-09-17T05:36:38.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5251400640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c0cefb0 0x7f524c0c8ca0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f523c00e9f0 tx=0x7f523c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:38.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5246000640 1 -- 172.21.15.122:0/3097751933 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c00cd80 con 0x7f524c0cefb0 2024-09-17T05:36:38.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5246000640 1 -- 172.21.15.122:0/3097751933 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f523c010430 con 0x7f524c0cefb0 2024-09-17T05:36:38.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c1645b0 con 0x7f524c0cefb0 2024-09-17T05:36:38.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5246000640 1 -- 172.21.15.122:0/3097751933 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c010860 con 0x7f524c0cefb0 2024-09-17T05:36:38.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c164b70 con 0x7f524c0cefb0 2024-09-17T05:36:38.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.130+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524c0cc5b0 con 0x7f524c0cefb0 2024-09-17T05:36:38.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.134+0000 7f5246000640 1 -- 172.21.15.122:0/3097751933 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f523c010a00 con 0x7f524c0cefb0 2024-09-17T05:36:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.134+0000 7f5246000640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5230074880 0x7f5230076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.134+0000 7f5246000640 1 -- 172.21.15.122:0/3097751933 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f523c093a40 con 0x7f524c0cefb0 2024-09-17T05:36:38.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.134+0000 7f5251e00640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5230074880 0x7f5230076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.134+0000 7f5251e00640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5230074880 0x7f5230076d40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f52400af000 tx=0x7f524001b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:38.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.138+0000 7f5246000640 1 -- 172.21.15.122:0/3097751933 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f523c05ffc0 con 0x7f524c0cefb0 2024-09-17T05:36:38.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f524c0b02f0 con 0x7f524c0cefb0 2024-09-17T05:36:38.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5246000640 1 -- 172.21.15.122:0/3097751933 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5237 (secure 0 0 0) 0x7f523c0649f0 con 0x7f524c0cefb0 2024-09-17T05:36:38.337 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:36:38.337 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":66,"btime":"2024-09-17T05:36:36:034083+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1758247367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1758247367},{"type":"v1","addr":"172.21.15.173:6836","nonce":1758247367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2422841556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2422841556},{"type":"v1","addr":"172.21.15.173:6837","nonce":2422841556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/4290616350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":4290616350},{"type":"v1","addr":"172.21.15.122:6834","nonce":4290616350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:36:32.388428+0000","modified":"2024-09-17T05:36:35.423704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2223381402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":2223381402},{"type":"v1","addr":"172.21.15.122:6835","nonce":2223381402}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T05:36:38.338 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 66 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5230074880 msgr2=0x7f5230076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5230074880 0x7f5230076d40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f52400af000 tx=0x7f524001b090 comp rx=0 tx=0).stop 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c0cefb0 msgr2=0x7f524c0c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c0cefb0 0x7f524c0c8ca0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f523c00e9f0 tx=0x7f523c00eeb0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 shutdown_connections 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5230074880 0x7f5230076d40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f524c15f000 0x7f524c0c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f524c0cefb0 0x7f524c0c8ca0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 --2- 172.21.15.122:0/3097751933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f524c0ccaf0 0x7f524c14cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 >> 172.21.15.122:0/3097751933 conn(0x7f524c09aa10 msgr2=0x7f524c15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 shutdown_connections 2024-09-17T05:36:38.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.342+0000 7f5253200640 1 -- 172.21.15.122:0/3097751933 wait complete. 2024-09-17T05:36:38.762 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:36:32.388428+0000', 'modified': '2024-09-17T05:36:35.423704+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7860}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7860': {'gid': 7860, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6835/2223381402', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6833', 'nonce': 2223381402}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 2223381402}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7860, 'qdb_cluster': [7860]} 2024-09-17T05:36:38.762 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:36:38.762 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:36:38.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.914+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3804496913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4406e810 msgr2=0x7f4b4406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.914+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3804496913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4406e810 0x7f4b4406f1f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4b44070d80 tx=0x7f4b3c01cd50 comp rx=0 tx=0).stop 2024-09-17T05:36:38.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.914+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3804496913 shutdown_connections 2024-09-17T05:36:38.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.914+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3804496913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4406e810 0x7f4b4406f1f0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.914+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3804496913 >> v2:172.21.15.173:3300/0 conn(0x7f4b440cf300 0x7f4b4406ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.914+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3804496913 >> 172.21.15.122:0/3804496913 conn(0x7f4b440b18f0 msgr2=0x7f4b440b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:38.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.914+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3804496913 shutdown_connections 2024-09-17T05:36:38.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.914+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3804496913 wait complete. 2024-09-17T05:36:38.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 Processor -- start 2024-09-17T05:36:38.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 -- start start 2024-09-17T05:36:38.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4406e810 0x7f4b4414b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b440cf300 0x7f4b441f9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4415ade0 0x7f4b441f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44168840 con 0x7f4b4415ade0 2024-09-17T05:36:38.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4b441686c0 con 0x7f4b4406e810 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4b441689c0 con 0x7f4b440cf300 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4415ade0 0x7f4b441f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4415ade0 0x7f4b441f9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44682/0 (socket says 172.21.15.122:44682) 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 -- 172.21.15.122:0/3061701270 learned_addr learned my addr 172.21.15.122:0/3061701270 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b3be00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b440cf300 0x7f4b441f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49200640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4406e810 0x7f4b4414b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 -- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b440cf300 msgr2=0x7f4b441f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b440cf300 0x7f4b441f9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 -- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4406e810 msgr2=0x7f4b4414b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4406e810 0x7f4b4414b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 -- 172.21.15.122:0/3061701270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b441fa630 con 0x7f4b4415ade0 2024-09-17T05:36:38.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b49c00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4415ade0 0x7f4b441f9d90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c019960 tx=0x7f4b3c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:38.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b41600640 1 -- 172.21.15.122:0/3061701270 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0bb070 con 0x7f4b4415ade0 2024-09-17T05:36:38.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b41600640 1 -- 172.21.15.122:0/3061701270 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b3c004e00 con 0x7f4b4415ade0 2024-09-17T05:36:38.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b41600640 1 -- 172.21.15.122:0/3061701270 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0b2040 con 0x7f4b4415ade0 2024-09-17T05:36:38.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3061701270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b44155620 con 0x7f4b4415ade0 2024-09-17T05:36:38.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.918+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3061701270 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b44155be0 con 0x7f4b4415ade0 2024-09-17T05:36:38.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b41600640 1 -- 172.21.15.122:0/3061701270 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f4b3c0a5cb0 con 0x7f4b4415ade0 2024-09-17T05:36:38.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b41600640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4b20074880 0x7f4b20076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b4f02c000 1 Processor -- start 2024-09-17T05:36:38.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b41600640 1 -- 172.21.15.122:0/3061701270 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f4b3c12cc30 con 0x7f4b4415ade0 2024-09-17T05:36:38.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b4f02c000 1 -- start start 2024-09-17T05:36:38.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49200640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4b20074880 0x7f4b20076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 0x55f06f887db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8882f0 0x55f06f83b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f83be10 0x55f06f83c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f06f75c3f0 con 0x55f06f83be10 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f06f699b20 con 0x55f06f8882f0 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55f06f688f50 con 0x55f06f8879d0 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49200640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4b20074880 0x7f4b20076d40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b34007fd0 tx=0x7f4b34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f83be10 0x55f06f83c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f83be10 0x55f06f83c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44696/0 (socket says 172.21.15.122:44696) 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 0x55f06f887db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49c00640 1 -- 172.21.15.122:0/175461279 learned_addr learned my addr 172.21.15.122:0/175461279 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b3be00640 1 --2- 172.21.15.122:0/175461279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8882f0 0x55f06f83b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49c00640 1 -- 172.21.15.122:0/175461279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 msgr2=0x55f06f887db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49c00640 1 --2- 172.21.15.122:0/175461279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 0x55f06f887db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49c00640 1 -- 172.21.15.122:0/175461279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8882f0 msgr2=0x55f06f83b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49c00640 1 --2- 172.21.15.122:0/175461279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8882f0 0x55f06f83b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.922+0000 7f4b49c00640 1 -- 172.21.15.122:0/175461279 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06f650590 con 0x55f06f83be10 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 --2- 172.21.15.122:0/175461279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f83be10 0x55f06f83c1f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01d230 tx=0x7f4b3c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9a4be66d71d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b39600640 1 -- 172.21.15.122:0/175461279 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c102740 con 0x55f06f83be10 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b39600640 1 -- 172.21.15.122:0/175461279 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b3c100070 con 0x55f06f83be10 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b39600640 1 -- 172.21.15.122:0/175461279 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c11ad70 con 0x55f06f83be10 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- 172.21.15.122:0/175461279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f83be10 msgr2=0x55f06f83c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/175461279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f83be10 0x55f06f83c1f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01d230 tx=0x7f4b3c009960 comp rx=0 tx=0).stop 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- 172.21.15.122:0/175461279 shutdown_connections 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/175461279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f83be10 0x55f06f83c1f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/175461279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8882f0 0x55f06f83b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/175461279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 0x55f06f887db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- 172.21.15.122:0/175461279 >> 172.21.15.122:0/175461279 conn(0x55f06f725130 msgr2=0x55f06f86ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- 172.21.15.122:0/175461279 shutdown_connections 2024-09-17T05:36:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- 172.21.15.122:0/175461279 wait complete. 2024-09-17T05:36:38.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- start start 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 Processor -- start 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 0x55f06f83d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 0x55f06f83d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f83dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f06f6235d0 con 0x55f06f8882f0 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f06f68cd60 con 0x55f06f83be10 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55f06f62f500 con 0x55f06f8879d0 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b4f02c000 10 client.0 _ll_register_callbacks cb 0x55f06f633810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f83dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f83dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44700/0 (socket says 172.21.15.122:44700) 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 -- 172.21.15.122:0/3592897152 learned_addr learned my addr 172.21.15.122:0/3592897152 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:38.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b3be00640 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 0x55f06f83d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49200640 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 0x55f06f83d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 -- 172.21.15.122:0/3592897152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 msgr2=0x55f06f83d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 0x55f06f83d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 -- 172.21.15.122:0/3592897152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 msgr2=0x55f06f83d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 0x55f06f83d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.926+0000 7f4b49c00640 1 -- 172.21.15.122:0/3592897152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f06f627170 con 0x55f06f8882f0 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b49c00640 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f83dcc0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0f6360 tx=0x7f4b3c0f8560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afed31c6bb0bb886 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 1 -- 172.21.15.122:0/3592897152 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0f90f0 con 0x55f06f8882f0 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 1 -- 172.21.15.122:0/3592897152 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f4b3c0f4a40 con 0x55f06f8882f0 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 -- 172.21.15.122:0/3592897152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f06f6210f0 con 0x55f06f8882f0 2024-09-17T05:36:38.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 1 -- 172.21.15.122:0/3592897152 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4b3c0fdb20 con 0x55f06f8882f0 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 10 client.7977 fetch_fsmap learned FSMap version 66 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 -- 172.21.15.122:0/3592897152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55f06f70def0 con 0x55f06f8882f0 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 1 -- 172.21.15.122:0/3592897152 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f06f6210f0 con 0x55f06f8882f0 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 1 -- 172.21.15.122:0/3592897152 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7f4b3c0b5030 con 0x55f06f8882f0 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 10 client.7977 fetch_fsmap finished waiting for FSMap version 66 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 10 client.7977 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 20 client.7977 populate_metadata read hostname 'smithi122' 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] conn(0x55f06f896c70 0x55f06f899060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 4 client.7977 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 -- 172.21.15.122:0/3592897152 --> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f06f69ba70 con 0x55f06f896c70 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b49200640 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] conn(0x55f06f896c70 0x55f06f899060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b49200640 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] conn(0x55f06f896c70 0x55f06f899060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8247e7447422f6d5 server_cookie=485f1150115841f5 in_seq=0 out_seq=0 2024-09-17T05:36:38.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 10 client.7977 ms_handle_connect on v2:172.21.15.122:6833/2223381402 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 1 -- 172.21.15.122:0/3592897152 <== mds.0 v2:172.21.15.122:6833/2223381402 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f06f69ba70 con 0x55f06f896c70 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 10 client.7977 handle_command_reply: tid=0 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 client.7977 shutdown 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 -- 172.21.15.122:0/3592897152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 msgr2=0x55f06f83dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f83dcc0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0f6360 tx=0x7f4b3c0f8560 comp rx=0 tx=0).stop 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 -- 172.21.15.122:0/3592897152 shutdown_connections 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] conn(0x55f06f896c70 0x55f06f899060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f83dcc0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8879d0 0x55f06f83d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/3592897152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 0x55f06f83d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b32000640 0 client.7977 ms_handle_reset on v2:172.21.15.122:6833/2223381402 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 -- 172.21.15.122:0/3592897152 >> 172.21.15.122:0/3592897152 conn(0x55f06f725130 msgr2=0x55f06f83afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 -- 172.21.15.122:0/3592897152 shutdown_connections 2024-09-17T05:36:38.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 1 -- 172.21.15.122:0/3592897152 wait complete. 2024-09-17T05:36:38.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.930+0000 7f4b4f02c000 20 client.7977 trim_cache size 0 max 16384 2024-09-17T05:36:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 Processor -- start 2024-09-17T05:36:38.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- start start 2024-09-17T05:36:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 0x55f06f8f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8879d0 0x55f06f8f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8882f0 0x55f06f8f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f06f691390 con 0x55f06f8879d0 2024-09-17T05:36:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f06f6235d0 con 0x55f06f83be10 2024-09-17T05:36:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55f06f68cd60 con 0x55f06f8882f0 2024-09-17T05:36:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8879d0 0x55f06f8f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8879d0 0x55f06f8f96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44706/0 (socket says 172.21.15.122:44706) 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 -- 172.21.15.122:0/4098023685 learned_addr learned my addr 172.21.15.122:0/4098023685 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b49200640 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 0x55f06f8f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b49c00640 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8882f0 0x55f06f8f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 -- 172.21.15.122:0/4098023685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8882f0 msgr2=0x55f06f8f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8882f0 0x55f06f8f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 -- 172.21.15.122:0/4098023685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 msgr2=0x55f06f8f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 0x55f06f8f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 -- 172.21.15.122:0/4098023685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06f6e4d90 con 0x55f06f8879d0 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b3be00640 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8879d0 0x55f06f8f96f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00b850 tx=0x7f4b2c00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92cfa454b1557d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b38c00640 1 -- 172.21.15.122:0/4098023685 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c003ba0 con 0x55f06f8879d0 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b38c00640 1 -- 172.21.15.122:0/4098023685 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2c003d40 con 0x55f06f8879d0 2024-09-17T05:36:38.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b38c00640 1 -- 172.21.15.122:0/4098023685 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c005020 con 0x55f06f8879d0 2024-09-17T05:36:38.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- 172.21.15.122:0/4098023685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8879d0 msgr2=0x55f06f8f96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8879d0 0x55f06f8f96f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00b850 tx=0x7f4b2c00bd10 comp rx=0 tx=0).stop 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- 172.21.15.122:0/4098023685 shutdown_connections 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f8882f0 0x55f06f8f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8879d0 0x55f06f8f96f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/4098023685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f83be10 0x55f06f8f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- 172.21.15.122:0/4098023685 >> 172.21.15.122:0/4098023685 conn(0x55f06f725130 msgr2=0x55f06f83b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- 172.21.15.122:0/4098023685 shutdown_connections 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.950+0000 7f4b4f02c000 1 -- 172.21.15.122:0/4098023685 wait complete. 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 -- start start 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 Processor -- start 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f83be10 0x55f06f89e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8879d0 0x55f06f8b9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f8b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f06f68c070 con 0x55f06f8882f0 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f06f62f500 con 0x55f06f8879d0 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55f06f6988f0 con 0x55f06f83be10 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 10 client.0 _ll_register_callbacks cb 0x7f4b2c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f8b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f8b9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44714/0 (socket says 172.21.15.122:44714) 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 -- 172.21.15.122:0/1718069584 learned_addr learned my addr 172.21.15.122:0/1718069584 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49200640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f83be10 0x55f06f89e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b3be00640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8879d0 0x55f06f8b9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 -- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f83be10 msgr2=0x55f06f89e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f83be10 0x55f06f89e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 -- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8879d0 msgr2=0x55f06f8b9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8879d0 0x55f06f8b9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 -- 172.21.15.122:0/1718069584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f06f64fa90 con 0x55f06f8882f0 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b3be00640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8879d0 0x55f06f8b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49200640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f83be10 0x55f06f89e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49c00640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f8b9ac0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0f9080 tx=0x7f4b3c12d780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caa2ef57f39e494 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b32000640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b32000640 1 -- 172.21.15.122:0/1718069584 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c0f8ec0 con 0x55f06f8882f0 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b32000640 1 -- 172.21.15.122:0/1718069584 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f4b3c0f7fc0 con 0x55f06f8882f0 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 -- 172.21.15.122:0/1718069584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c003d40 con 0x55f06f8882f0 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b32000640 1 -- 172.21.15.122:0/1718069584 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4b3c0f90f0 con 0x55f06f8882f0 2024-09-17T05:36:38.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 10 client.7983 fetch_fsmap learned FSMap version 66 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 -- 172.21.15.122:0/1718069584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55f06f62e050 con 0x55f06f8882f0 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b32000640 1 -- 172.21.15.122:0/1718069584 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2c003d40 con 0x55f06f8882f0 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b32000640 1 -- 172.21.15.122:0/1718069584 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7f4b3c0b5070 con 0x55f06f8882f0 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 10 client.7983 fetch_fsmap finished waiting for FSMap version 66 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 10 client.7983 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 20 client.7983 populate_metadata read hostname 'smithi122' 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] conn(0x55f06f8ba000 0x55f06f8fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 4 client.7983 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b4f02c000 1 -- 172.21.15.122:0/1718069584 --> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] -- command(tid 0: {"prefix": "status"}) -- 0x55f06f697520 con 0x55f06f8ba000 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49200640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] conn(0x55f06f8ba000 0x55f06f8fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:38.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b32000640 10 client.7983 ms_handle_connect on v2:172.21.15.122:6833/2223381402 2024-09-17T05:36:38.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.954+0000 7f4b49200640 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] conn(0x55f06f8ba000 0x55f06f8fd400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e475c9d2517602 server_cookie=5085ed224f108cab in_seq=0 out_seq=0 2024-09-17T05:36:38.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b32000640 1 -- 172.21.15.122:0/1718069584 <== mds.0 v2:172.21.15.122:6833/2223381402 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f06f697520 con 0x55f06f8ba000 2024-09-17T05:36:38.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b32000640 10 client.7983 handle_command_reply: tid=0 2024-09-17T05:36:38.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 client.7983 shutdown 2024-09-17T05:36:38.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 -- 172.21.15.122:0/1718069584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 msgr2=0x55f06f8b9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f8b9ac0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0f9080 tx=0x7f4b3c12d780 comp rx=0 tx=0).stop 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 -- 172.21.15.122:0/1718069584 shutdown_connections 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.122:6833/2223381402,v1:172.21.15.122:6835/2223381402] conn(0x55f06f8ba000 0x55f06f8fd400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f06f8882f0 0x55f06f8b9ac0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f06f8879d0 0x55f06f8b9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 --2- 172.21.15.122:0/1718069584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f06f83be10 0x55f06f89e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b32000640 0 client.7983 ms_handle_reset on v2:172.21.15.122:6833/2223381402 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 -- 172.21.15.122:0/1718069584 >> 172.21.15.122:0/1718069584 conn(0x55f06f725130 msgr2=0x55f06f899060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 -- 172.21.15.122:0/1718069584 shutdown_connections 2024-09-17T05:36:38.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 1 -- 172.21.15.122:0/1718069584 wait complete. 2024-09-17T05:36:38.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4f02c000 20 client.7983 trim_cache size 0 max 16384 2024-09-17T05:36:38.961 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:36:38.961 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:36:38.961 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:36:38.961 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 7860, 2024-09-17T05:36:38.961 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:36:38.961 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:36:38.961 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:36:38.962 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 5.9435241410000001, 2024-09-17T05:36:38.962 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 66, 2024-09-17T05:36:38.962 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 103, 2024-09-17T05:36:38.962 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 103, 2024-09-17T05:36:38.962 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 10.490648509 2024-09-17T05:36:38.962 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4b20074880 msgr2=0x7f4b20076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4b20074880 0x7f4b20076d40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b34007fd0 tx=0x7f4b34009040 comp rx=0 tx=0).stop 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3061701270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4415ade0 msgr2=0x7f4b441f9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4415ade0 0x7f4b441f9d90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c019960 tx=0x7f4b3c003f60 comp rx=0 tx=0).stop 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3061701270 shutdown_connections 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4b20074880 0x7f4b20076d40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4415ade0 0x7f4b441f9d90 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b440cf300 0x7f4b441f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 --2- 172.21.15.122:0/3061701270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b4406e810 0x7f4b4414b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3061701270 >> 172.21.15.122:0/3061701270 conn(0x7f4b440b18f0 msgr2=0x7f4b440aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:38.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3061701270 shutdown_connections 2024-09-17T05:36:38.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:38.958+0000 7f4b4bc00640 1 -- 172.21.15.122:0/3061701270 wait complete. 2024-09-17T05:36:38.972 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:36:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.118+0000 7f1bb6a00640 1 -- 172.21.15.122:0/3523153752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb015f000 msgr2=0x7f1bb00d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.118+0000 7f1bb6a00640 1 --2- 172.21.15.122:0/3523153752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb015f000 0x7f1bb00d01b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009840 tx=0x7f1ba401c700 comp rx=0 tx=0).stop 2024-09-17T05:36:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.118+0000 7f1bb6a00640 1 -- 172.21.15.122:0/3523153752 shutdown_connections 2024-09-17T05:36:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.118+0000 7f1bb6a00640 1 --2- 172.21.15.122:0/3523153752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb015f000 0x7f1bb00d01b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.118+0000 7f1bb6a00640 1 -- 172.21.15.122:0/3523153752 >> 172.21.15.122:0/3523153752 conn(0x7f1bb009aa10 msgr2=0x7f1bb009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.118+0000 7f1bb6a00640 1 -- 172.21.15.122:0/3523153752 shutdown_connections 2024-09-17T05:36:39.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.118+0000 7f1bb6a00640 1 -- 172.21.15.122:0/3523153752 wait complete. 2024-09-17T05:36:39.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 Processor -- start 2024-09-17T05:36:39.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 -- start start 2024-09-17T05:36:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bb00b2590 0x7f1bb0158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb00cefb0 0x7f1bb0157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bb015f000 0x7f1bb0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb016a110 con 0x7f1bb00cefb0 2024-09-17T05:36:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0169f90 con 0x7f1bb015f000 2024-09-17T05:36:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb016a290 con 0x7f1bb00b2590 2024-09-17T05:36:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb00cefb0 0x7f1bb0157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb00cefb0 0x7f1bb0157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44728/0 (socket says 172.21.15.122:44728) 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 -- 172.21.15.122:0/1408285911 learned_addr learned my addr 172.21.15.122:0/1408285911 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb5600640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bb00b2590 0x7f1bb0158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6000640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bb015f000 0x7f1bb0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 -- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bb00b2590 msgr2=0x7f1bb0158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bb00b2590 0x7f1bb0158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 -- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bb015f000 msgr2=0x7f1bb0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bb015f000 0x7f1bb0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 -- 172.21.15.122:0/1408285911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb0156a60 con 0x7f1bb00cefb0 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6000640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bb015f000 0x7f1bb0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb4c00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb00cefb0 0x7f1bb0157040 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00c730 tx=0x7f1b9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bad600640 1 -- 172.21.15.122:0/1408285911 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c005070 con 0x7f1bb00cefb0 2024-09-17T05:36:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bad600640 1 -- 172.21.15.122:0/1408285911 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b9c005230 con 0x7f1bb00cefb0 2024-09-17T05:36:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb00c9030 con 0x7f1bb00cefb0 2024-09-17T05:36:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bad600640 1 -- 172.21.15.122:0/1408285911 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00a640 con 0x7f1bb00cefb0 2024-09-17T05:36:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.122+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb00c98d0 con 0x7f1bb00cefb0 2024-09-17T05:36:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.126+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb0159af0 con 0x7f1bb00cefb0 2024-09-17T05:36:39.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.126+0000 7f1bad600640 1 -- 172.21.15.122:0/1408285911 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f1b9c00a7e0 con 0x7f1bb00cefb0 2024-09-17T05:36:39.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.126+0000 7f1bad600640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b90074880 0x7f1b90076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:39.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.126+0000 7f1bad600640 1 -- 172.21.15.122:0/1408285911 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f1b9c092a10 con 0x7f1bb00cefb0 2024-09-17T05:36:39.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.126+0000 7f1bb5600640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b90074880 0x7f1b90076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.130+0000 7f1bb5600640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b90074880 0x7f1b90076d40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40af000 tx=0x7f1ba401af40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.130+0000 7f1bad600640 1 -- 172.21.15.122:0/1408285911 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1bb00cf400 con 0x7f1bb00cefb0 2024-09-17T05:36:39.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1bb00c9200 con 0x7f1bb00cefb0 2024-09-17T05:36:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bad600640 1 -- 172.21.15.122:0/1408285911 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f1b9c05f520 con 0x7f1bb00cefb0 2024-09-17T05:36:39.330 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:36:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b90074880 msgr2=0x7f1b90076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b90074880 0x7f1b90076d40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40af000 tx=0x7f1ba401af40 comp rx=0 tx=0).stop 2024-09-17T05:36:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb00cefb0 msgr2=0x7f1bb0157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb00cefb0 0x7f1bb0157040 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00c730 tx=0x7f1b9c00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:36:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 shutdown_connections 2024-09-17T05:36:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b90074880 0x7f1b90076d40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1bb015f000 0x7f1bb0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bb00cefb0 0x7f1bb0157040 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.334+0000 7f1bb6a00640 1 --2- 172.21.15.122:0/1408285911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1bb00b2590 0x7f1bb0158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.338+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 >> 172.21.15.122:0/1408285911 conn(0x7f1bb009aa10 msgr2=0x7f1bb015e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.338+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 shutdown_connections 2024-09-17T05:36:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39.338+0000 7f1bb6a00640 1 -- 172.21.15.122:0/1408285911 wait complete. 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:36:39.352 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:36:39.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:39.353 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:36:39.362 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:36:39.362 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:36:39.362 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:36:39.362 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:36:39.362 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 41768sec preferred_lft 41768sec 2024-09-17T05:36:39.363 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:36:39.364 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:36:39.365 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:36:39.365 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:36:39.365 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:36:39.365 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:36:39.365 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:36:39.365 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:36:39.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:39.365 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:36:39.421 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:36:39.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:39.423 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:36:39.475 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:36:39.475 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:36:39.476 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:36:39.476 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:36:39.477 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:36:39.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:36:39.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:36:39.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:36:39.477 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:36:39.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:36:39.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:36:39.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:39.612 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:36:39.620 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:36:39.621 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:36:39.621 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:36:39.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:39.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:36:39.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:36:39.687 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:36:39.736 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:36:39.736 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:36:39.736 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:36:39.787 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:36:39.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:39.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:36:39.879 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:36:39.879 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:36:39.879 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:36:39.894 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:36:39.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:36:39.894 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:36:39.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:36:39.894 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:36:39.903 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:36:39.903 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:36:39.903 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:36:39.903 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:36:39.903 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:36:39.903 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:36:39.903 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:36:39.903 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:36:39.907 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:36:39.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:39.907 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:36:39.927 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:36:40.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:36:40.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:36:40.036 INFO:teuthology.orchestra.run.smithi122.stdout:client7995 2024-09-17T05:36:40.036 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:36:40.037 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:40.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:36:40.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.234+0000 7fe467e00640 1 -- 172.21.15.122:0/242316556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 msgr2=0x7fe4600cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:40.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.234+0000 7fe467e00640 1 --2- 172.21.15.122:0/242316556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 0x7fe4600cee10 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009900 tx=0x7fe44c01f0d0 comp rx=0 tx=0).stop 2024-09-17T05:36:40.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.234+0000 7fe467e00640 1 -- 172.21.15.122:0/242316556 shutdown_connections 2024-09-17T05:36:40.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.234+0000 7fe467e00640 1 --2- 172.21.15.122:0/242316556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 0x7fe4600cee10 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:40.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.234+0000 7fe467e00640 1 -- 172.21.15.122:0/242316556 >> 172.21.15.122:0/242316556 conn(0x7fe4600d1da0 msgr2=0x7fe4600d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:40.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.234+0000 7fe467e00640 1 -- 172.21.15.122:0/242316556 shutdown_connections 2024-09-17T05:36:40.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.234+0000 7fe467e00640 1 -- 172.21.15.122:0/242316556 wait complete. 2024-09-17T05:36:40.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 Processor -- start 2024-09-17T05:36:40.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 -- start start 2024-09-17T05:36:40.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4600b0b60 0x7fe46015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:40.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46015be10 0x7fe46015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:40.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 0x7fe4601f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:40.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4600cd670 con 0x7fe46015c1f0 2024-09-17T05:36:40.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4600cd4f0 con 0x7fe4600b0b60 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe4600cd7f0 con 0x7fe46015be10 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 0x7fe4601f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 0x7fe4601f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44746/0 (socket says 172.21.15.122:44746) 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 -- 172.21.15.122:0/3417815153 learned_addr learned my addr 172.21.15.122:0/3417815153 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe464a00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46015be10 0x7fe46015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465400640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4600b0b60 0x7fe46015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 -- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46015be10 msgr2=0x7fe46015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46015be10 0x7fe46015eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 -- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4600b0b60 msgr2=0x7fe46015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4600b0b60 0x7fe46015dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 -- 172.21.15.122:0/3417815153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4601f5210 con 0x7fe46015c1f0 2024-09-17T05:36:40.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe465e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 0x7fe4601f4cd0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe45800c7b0 tx=0x7fe45800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:40.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe45d600640 1 -- 172.21.15.122:0/3417815153 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458004b70 con 0x7fe46015c1f0 2024-09-17T05:36:40.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe45d600640 1 -- 172.21.15.122:0/3417815153 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe458004e00 con 0x7fe46015c1f0 2024-09-17T05:36:40.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe460158500 con 0x7fe46015c1f0 2024-09-17T05:36:40.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe45d600640 1 -- 172.21.15.122:0/3417815153 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45800aa50 con 0x7fe46015c1f0 2024-09-17T05:36:40.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.238+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe460158ac0 con 0x7fe46015c1f0 2024-09-17T05:36:40.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.242+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4600cdca0 con 0x7fe46015c1f0 2024-09-17T05:36:40.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.242+0000 7fe45d600640 1 -- 172.21.15.122:0/3417815153 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fe45800abf0 con 0x7fe46015c1f0 2024-09-17T05:36:40.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.242+0000 7fe45d600640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe440074880 0x7fe440076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:40.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.242+0000 7fe45d600640 1 -- 172.21.15.122:0/3417815153 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7fe458093220 con 0x7fe46015c1f0 2024-09-17T05:36:40.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.242+0000 7fe465400640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe440074880 0x7fe440076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:40.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.242+0000 7fe465400640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe440074880 0x7fe440076d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe4540016f0 tx=0x7fe454009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:40.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.246+0000 7fe45d600640 1 -- 172.21.15.122:0/3417815153 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe458020070 con 0x7fe46015c1f0 2024-09-17T05:36:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:40.450+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fe460070350 con 0x7fe46015c1f0 2024-09-17T05:36:41.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.054+0000 7fe45d600640 1 -- 172.21.15.122:0/3417815153 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v67) ==== 136+0+0 (secure 0 0 0) 0x7fe45805fd30 con 0x7fe46015c1f0 2024-09-17T05:36:41.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe440074880 msgr2=0x7fe440076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe440074880 0x7fe440076d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe4540016f0 tx=0x7fe454009040 comp rx=0 tx=0).stop 2024-09-17T05:36:41.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 msgr2=0x7fe4601f4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 0x7fe4601f4cd0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe45800c7b0 tx=0x7fe45800cc70 comp rx=0 tx=0).stop 2024-09-17T05:36:41.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 shutdown_connections 2024-09-17T05:36:41.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe440074880 0x7fe440076d40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe46015c1f0 0x7fe4601f4cd0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46015be10 0x7fe46015eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 --2- 172.21.15.122:0/3417815153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4600b0b60 0x7fe46015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 >> 172.21.15.122:0/3417815153 conn(0x7fe4600d1da0 msgr2=0x7fe4600712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:41.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 shutdown_connections 2024-09-17T05:36:41.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.058+0000 7fe467e00640 1 -- 172.21.15.122:0/3417815153 wait complete. 2024-09-17T05:36:41.077 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:36:41.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 -- 172.21.15.122:0/727967389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c0cecf0 msgr2=0x7f4e6c0cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 --2- 172.21.15.122:0/727967389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c0cecf0 0x7f4e6c0cc940 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0d0780 tx=0x7f4e5801c700 comp rx=0 tx=0).stop 2024-09-17T05:36:41.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 -- 172.21.15.122:0/727967389 shutdown_connections 2024-09-17T05:36:41.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 --2- 172.21.15.122:0/727967389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c0cecf0 0x7f4e6c0cc940 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 -- 172.21.15.122:0/727967389 >> 172.21.15.122:0/727967389 conn(0x7f4e6c0d1ea0 msgr2=0x7f4e6c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:41.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 -- 172.21.15.122:0/727967389 shutdown_connections 2024-09-17T05:36:41.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 -- 172.21.15.122:0/727967389 wait complete. 2024-09-17T05:36:41.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 Processor -- start 2024-09-17T05:36:41.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.226+0000 7f4e74200640 1 -- start start 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6c0cecf0 0x7f4e6c155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c15bd50 0x7f4e6c0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e6c15c130 0x7f4e6c0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c16a5b0 con 0x7f4e6c15bd50 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c16a430 con 0x7f4e6c15c130 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6c16a730 con 0x7f4e6c0cecf0 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c15bd50 0x7f4e6c0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c15bd50 0x7f4e6c0c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44766/0 (socket says 172.21.15.122:44766) 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 -- 172.21.15.122:0/528294131 learned_addr learned my addr 172.21.15.122:0/528294131 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:41.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e72200640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e6c15c130 0x7f4e6c0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e71800640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6c0cecf0 0x7f4e6c155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 -- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6c0cecf0 msgr2=0x7f4e6c155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6c0cecf0 0x7f4e6c155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 -- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e6c15c130 msgr2=0x7f4e6c0c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e6c15c130 0x7f4e6c0c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 -- 172.21.15.122:0/528294131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c15a9c0 con 0x7f4e6c15bd50 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e72200640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e6c15c130 0x7f4e6c0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e71800640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6c0cecf0 0x7f4e6c155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:36:41.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e70e00640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c15bd50 0x7f4e6c0c82b0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000c10 tx=0x7f4e580040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e69600640 1 -- 172.21.15.122:0/528294131 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e580bb070 con 0x7f4e6c15bd50 2024-09-17T05:36:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e69600640 1 -- 172.21.15.122:0/528294131 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e58005030 con 0x7f4e6c15bd50 2024-09-17T05:36:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c0c9180 con 0x7f4e6c15bd50 2024-09-17T05:36:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e69600640 1 -- 172.21.15.122:0/528294131 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e580b2040 con 0x7f4e6c15bd50 2024-09-17T05:36:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c0c9740 con 0x7f4e6c15bd50 2024-09-17T05:36:41.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e6c0ce5e0 con 0x7f4e6c15bd50 2024-09-17T05:36:41.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.230+0000 7f4e69600640 1 -- 172.21.15.122:0/528294131 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f4e580be070 con 0x7f4e6c15bd50 2024-09-17T05:36:41.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.234+0000 7f4e69600640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e4c074880 0x7f4e4c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.234+0000 7f4e69600640 1 -- 172.21.15.122:0/528294131 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f4e5812c840 con 0x7f4e6c15bd50 2024-09-17T05:36:41.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.234+0000 7f4e71800640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e4c074880 0x7f4e4c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.234+0000 7f4e71800640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e4c074880 0x7f4e4c076d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4e600021a0 tx=0x7f4e60009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:41.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.238+0000 7f4e69600640 1 -- 172.21.15.122:0/528294131 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4e58016410 con 0x7f4e6c15bd50 2024-09-17T05:36:41.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.422+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f4e6c0cd170 con 0x7f4e4c074880 2024-09-17T05:36:41.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e69600640 1 -- 172.21.15.122:0/528294131 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f4e6c0cd170 con 0x7f4e4c074880 2024-09-17T05:36:41.422 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:36:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:36:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:36:41.422 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:36:41.422 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:36:41.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e4c074880 msgr2=0x7f4e4c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e4c074880 0x7f4e4c076d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4e600021a0 tx=0x7f4e60009040 comp rx=0 tx=0).stop 2024-09-17T05:36:41.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c15bd50 msgr2=0x7f4e6c0c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c15bd50 0x7f4e6c0c82b0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000c10 tx=0x7f4e580040c0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 shutdown_connections 2024-09-17T05:36:41.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e4c074880 0x7f4e4c076d40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e6c15c130 0x7f4e6c0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e6c15bd50 0x7f4e6c0c82b0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 --2- 172.21.15.122:0/528294131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e6c0cecf0 0x7f4e6c155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 >> 172.21.15.122:0/528294131 conn(0x7f4e6c0d1ea0 msgr2=0x7f4e6c070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:41.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.426+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 shutdown_connections 2024-09-17T05:36:41.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.430+0000 7f4e74200640 1 -- 172.21.15.122:0/528294131 wait complete. 2024-09-17T05:36:41.443 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:36:41.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 -- 172.21.15.122:0/1228241326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc1605d0 msgr2=0x7ffbdc0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 --2- 172.21.15.122:0/1228241326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc1605d0 0x7ffbdc0d0690 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc074840 tx=0x7ffbcc01cd60 comp rx=0 tx=0).stop 2024-09-17T05:36:41.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 -- 172.21.15.122:0/1228241326 shutdown_connections 2024-09-17T05:36:41.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 --2- 172.21.15.122:0/1228241326 >> v2:172.21.15.173:3300/0 conn(0x7ffbdc0aeb00 0x7ffbdc0aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 --2- 172.21.15.122:0/1228241326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbdc0b1650 0x7ffbdc0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 --2- 172.21.15.122:0/1228241326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc1605d0 0x7ffbdc0d0690 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 -- 172.21.15.122:0/1228241326 >> 172.21.15.122:0/1228241326 conn(0x7ffbdc0d1dd0 msgr2=0x7ffbdc0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:41.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 -- 172.21.15.122:0/1228241326 shutdown_connections 2024-09-17T05:36:41.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.590+0000 7ffbe1000640 1 -- 172.21.15.122:0/1228241326 wait complete. 2024-09-17T05:36:41.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 Processor -- start 2024-09-17T05:36:41.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 -- start start 2024-09-17T05:36:41.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbdc0aeb00 0x7ffbdc0b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc0b1650 0x7ffbdc150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbdc1605d0 0x7ffbdc0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc0b13a0 con 0x7ffbdc0b1650 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc0b1220 con 0x7ffbdc0aeb00 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc0afd00 con 0x7ffbdc1605d0 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc0b1650 0x7ffbdc150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc0b1650 0x7ffbdc150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54188/0 (socket says 172.21.15.122:54188) 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 -- 172.21.15.122:0/235964480 learned_addr learned my addr 172.21.15.122:0/235964480 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd9e00640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbdc0aeb00 0x7ffbdc0b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbda800640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbdc1605d0 0x7ffbdc0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 -- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbdc1605d0 msgr2=0x7ffbdc0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbdc1605d0 0x7ffbdc0b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 -- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbdc0aeb00 msgr2=0x7ffbdc0b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbdc0aeb00 0x7ffbdc0b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 -- 172.21.15.122:0/235964480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc1fb8e0 con 0x7ffbdc0b1650 2024-09-17T05:36:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd1400640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc0b1650 0x7ffbdc150970 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400d980 tx=0x7ffbc400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd2a00640 1 -- 172.21.15.122:0/235964480 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4017070 con 0x7ffbdc0b1650 2024-09-17T05:36:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd2a00640 1 -- 172.21.15.122:0/235964480 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbc4009d60 con 0x7ffbdc0b1650 2024-09-17T05:36:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc1fbbc0 con 0x7ffbdc0b1650 2024-09-17T05:36:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbd2a00640 1 -- 172.21.15.122:0/235964480 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4005280 con 0x7ffbdc0b1650 2024-09-17T05:36:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.594+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc06e4e0 con 0x7ffbdc0b1650 2024-09-17T05:36:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.598+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbdc0b1a30 con 0x7ffbdc0b1650 2024-09-17T05:36:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.598+0000 7ffbd2a00640 1 -- 172.21.15.122:0/235964480 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7ffbc4005420 con 0x7ffbdc0b1650 2024-09-17T05:36:41.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.598+0000 7ffbd2a00640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbb8074880 0x7ffbb8076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.598+0000 7ffbd2a00640 1 -- 172.21.15.122:0/235964480 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7ffbc4093080 con 0x7ffbdc0b1650 2024-09-17T05:36:41.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.598+0000 7ffbd9e00640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbb8074880 0x7ffbb8076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.602+0000 7ffbd9e00640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbb8074880 0x7ffbb8076d40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0af000 tx=0x7ffbcc01b5a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:41.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.602+0000 7ffbd2a00640 1 -- 172.21.15.122:0/235964480 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ffbc405f540 con 0x7ffbdc0b1650 2024-09-17T05:36:41.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.774+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffbdc0d1080 con 0x7ffbdc0b1650 2024-09-17T05:36:41.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.774+0000 7ffbd2a00640 1 -- 172.21.15.122:0/235964480 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7ffbc4063f70 con 0x7ffbdc0b1650 2024-09-17T05:36:41.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbb8074880 msgr2=0x7ffbb8076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbb8074880 0x7ffbb8076d40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0af000 tx=0x7ffbcc01b5a0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc0b1650 msgr2=0x7ffbdc150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc0b1650 0x7ffbdc150970 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400d980 tx=0x7ffbc400de40 comp rx=0 tx=0).stop 2024-09-17T05:36:41.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 shutdown_connections 2024-09-17T05:36:41.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbb8074880 0x7ffbb8076d40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbdc1605d0 0x7ffbdc0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbdc0b1650 0x7ffbdc150970 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 --2- 172.21.15.122:0/235964480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbdc0aeb00 0x7ffbdc0b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 >> 172.21.15.122:0/235964480 conn(0x7ffbdc0d1dd0 msgr2=0x7ffbdc0cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:41.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 shutdown_connections 2024-09-17T05:36:41.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.778+0000 7ffbe1000640 1 -- 172.21.15.122:0/235964480 wait complete. 2024-09-17T05:36:41.793 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:36:41.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 -- 172.21.15.122:0/2660620594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 msgr2=0x7f9d48004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 --2- 172.21.15.122:0/2660620594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 0x7f9d48004c90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c001860 tx=0x7f9d4c01f030 comp rx=0 tx=0).stop 2024-09-17T05:36:41.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 -- 172.21.15.122:0/2660620594 shutdown_connections 2024-09-17T05:36:41.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 --2- 172.21.15.122:0/2660620594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 0x7f9d48004c90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 -- 172.21.15.122:0/2660620594 >> 172.21.15.122:0/2660620594 conn(0x7f9d640d0ab0 msgr2=0x7f9d640d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:41.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 -- 172.21.15.122:0/2660620594 shutdown_connections 2024-09-17T05:36:41.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 -- 172.21.15.122:0/2660620594 wait complete. 2024-09-17T05:36:41.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 Processor -- start 2024-09-17T05:36:41.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.942+0000 7f9d68e00640 1 -- start start 2024-09-17T05:36:41.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 0x7f9d6415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d640b0860 0x7f9d640c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d641f7f10 0x7f9d641f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64070490 con 0x7f9d48004810 2024-09-17T05:36:41.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d640d17e0 con 0x7f9d641f7f10 2024-09-17T05:36:41.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6406f650 con 0x7f9d640b0860 2024-09-17T05:36:41.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 0x7f9d6415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 0x7f9d6415f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54216/0 (socket says 172.21.15.122:54216) 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 -- 172.21.15.122:0/3986700638 learned_addr learned my addr 172.21.15.122:0/3986700638 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63e00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d641f7f10 0x7f9d641f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d5aa00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d640b0860 0x7f9d640c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 -- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d640b0860 msgr2=0x7f9d640c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d640b0860 0x7f9d640c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 -- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d641f7f10 msgr2=0x7f9d641f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d641f7f10 0x7f9d641f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 -- 172.21.15.122:0/3986700638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d64074fa0 con 0x7f9d48004810 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d5aa00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d640b0860 0x7f9d640c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d63400640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 0x7f9d6415f0e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9d54011a10 tx=0x7f9d54011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d5be00640 1 -- 172.21.15.122:0/3986700638 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5401e070 con 0x7f9d48004810 2024-09-17T05:36:41.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d5be00640 1 -- 172.21.15.122:0/3986700638 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d54004460 con 0x7f9d48004810 2024-09-17T05:36:41.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d5be00640 1 -- 172.21.15.122:0/3986700638 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400fc00 con 0x7f9d48004810 2024-09-17T05:36:41.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d640ca400 con 0x7f9d48004810 2024-09-17T05:36:41.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d640ca9c0 con 0x7f9d48004810 2024-09-17T05:36:41.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.946+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d640b0c40 con 0x7f9d48004810 2024-09-17T05:36:41.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.950+0000 7f9d5be00640 1 -- 172.21.15.122:0/3986700638 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f9d54021070 con 0x7f9d48004810 2024-09-17T05:36:41.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.950+0000 7f9d5be00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40074880 0x7f9d40076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:41.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.950+0000 7f9d5be00640 1 -- 172.21.15.122:0/3986700638 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f9d540938c0 con 0x7f9d48004810 2024-09-17T05:36:41.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.950+0000 7f9d5aa00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40074880 0x7f9d40076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:41.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.950+0000 7f9d5aa00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40074880 0x7f9d40076d40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c001830 tx=0x7f9d4c01be70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:41.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:41.954+0000 7f9d5be00640 1 -- 172.21.15.122:0/3986700638 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9d640b0c40 con 0x7f9d48004810 2024-09-17T05:36:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.122+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f9d6406fb60 con 0x7f9d48004810 2024-09-17T05:36:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.122+0000 7f9d5be00640 1 -- 172.21.15.122:0/3986700638 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v45) ==== 116+0+2 (secure 0 0 0) 0x7f9d5405fd80 con 0x7f9d48004810 2024-09-17T05:36:42.119 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:36:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40074880 msgr2=0x7f9d40076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40074880 0x7f9d40076d40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c001830 tx=0x7f9d4c01be70 comp rx=0 tx=0).stop 2024-09-17T05:36:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 msgr2=0x7f9d6415f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 0x7f9d6415f0e0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9d54011a10 tx=0x7f9d54011ed0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 shutdown_connections 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d40074880 0x7f9d40076d40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d641f7f10 0x7f9d641f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d640b0860 0x7f9d640c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 --2- 172.21.15.122:0/3986700638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d48004810 0x7f9d6415f0e0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 >> 172.21.15.122:0/3986700638 conn(0x7f9d640d0ab0 msgr2=0x7f9d640ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 shutdown_connections 2024-09-17T05:36:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.126+0000 7f9d68e00640 1 -- 172.21.15.122:0/3986700638 wait complete. 2024-09-17T05:36:42.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:36:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- 172.21.15.122:0/856044122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 msgr2=0x7f49d815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 --2- 172.21.15.122:0/856044122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 0x7f49d815c220 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009840 tx=0x7f49c801c820 comp rx=0 tx=0).stop 2024-09-17T05:36:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- 172.21.15.122:0/856044122 shutdown_connections 2024-09-17T05:36:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 --2- 172.21.15.122:0/856044122 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d80740c0 0x7f49d8074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 --2- 172.21.15.122:0/856044122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 0x7f49d815c220 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- 172.21.15.122:0/856044122 >> 172.21.15.122:0/856044122 conn(0x7f49d806ecb0 msgr2=0x7f49d806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- 172.21.15.122:0/856044122 shutdown_connections 2024-09-17T05:36:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- 172.21.15.122:0/856044122 wait complete. 2024-09-17T05:36:42.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 Processor -- start 2024-09-17T05:36:42.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- start start 2024-09-17T05:36:42.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d80740c0 0x7f49d81594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 0x7f49d814b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d815f220 0x7f49d80a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8167960 con 0x7f49d81584d0 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49d81677e0 con 0x7f49d80740c0 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49df600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f49d8167ae0 con 0x7f49d815f220 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49cfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 0x7f49d814b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49cfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 0x7f49d814b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54232/0 (socket says 172.21.15.122:54232) 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49dcc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d80740c0 0x7f49d81594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49cfe00640 1 -- 172.21.15.122:0/2326460091 learned_addr learned my addr 172.21.15.122:0/2326460091 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.290+0000 7f49dd600640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d815f220 0x7f49d80a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49cfe00640 1 -- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d815f220 msgr2=0x7f49d80a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49cfe00640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d815f220 0x7f49d80a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49cfe00640 1 -- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d80740c0 msgr2=0x7f49d81594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49cfe00640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d80740c0 0x7f49d81594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49cfe00640 1 -- 172.21.15.122:0/2326460091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d80ae1a0 con 0x7f49d81584d0 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49cfe00640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 0x7f49d814b6c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c3f0 tx=0x7f49c000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49d5600640 1 -- 172.21.15.122:0/2326460091 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00068a0 con 0x7f49d81584d0 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49d5600640 1 -- 172.21.15.122:0/2326460091 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c0006b50 con 0x7f49d81584d0 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49d5600640 1 -- 172.21.15.122:0/2326460091 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0007020 con 0x7f49d81584d0 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d80ae410 con 0x7f49d81584d0 2024-09-17T05:36:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81590f0 con 0x7f49d81584d0 2024-09-17T05:36:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d8074500 con 0x7f49d81584d0 2024-09-17T05:36:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49d5600640 1 -- 172.21.15.122:0/2326460091 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f49c0019020 con 0x7f49d81584d0 2024-09-17T05:36:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49d5600640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b40748b0 0x7f49b4076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:42.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.294+0000 7f49d5600640 1 -- 172.21.15.122:0/2326460091 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f49c0092450 con 0x7f49d81584d0 2024-09-17T05:36:42.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.298+0000 7f49dcc00640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b40748b0 0x7f49b4076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:42.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.298+0000 7f49dcc00640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b40748b0 0x7f49b4076d70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f49c80af000 tx=0x7f49c801b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:42.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.298+0000 7f49d5600640 1 -- 172.21.15.122:0/2326460091 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f49c005e910 con 0x7f49d81584d0 2024-09-17T05:36:42.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.470+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f49d80b30a0 con 0x7f49d81584d0 2024-09-17T05:36:42.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49d5600640 1 -- 172.21.15.122:0/2326460091 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7f49c0063340 con 0x7f49d81584d0 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b40748b0 msgr2=0x7f49b4076d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b40748b0 0x7f49b4076d70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f49c80af000 tx=0x7f49c801b060 comp rx=0 tx=0).stop 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 msgr2=0x7f49d814b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 0x7f49d814b6c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c3f0 tx=0x7f49c000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 shutdown_connections 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b40748b0 0x7f49b4076d70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d815f220 0x7f49d80a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d81584d0 0x7f49d814b6c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 --2- 172.21.15.122:0/2326460091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d80740c0 0x7f49d81594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 >> 172.21.15.122:0/2326460091 conn(0x7f49d806ecb0 msgr2=0x7f49d8073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 shutdown_connections 2024-09-17T05:36:42.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.474+0000 7f49df600640 1 -- 172.21.15.122:0/2326460091 wait complete. 2024-09-17T05:36:42.489 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T05:36:42.490 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.492+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.490 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.492+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.490 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.492+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.490 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.492+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.490 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.492+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.638+0000 7fe342e00640 1 -- 172.21.15.122:0/3633448017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0ce890 msgr2=0x7fe33c06eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.638+0000 7fe342e00640 1 --2- 172.21.15.122:0/3633448017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0ce890 0x7fe33c06eb40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe334009900 tx=0x7fe33401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.638+0000 7fe342e00640 1 -- 172.21.15.122:0/3633448017 shutdown_connections 2024-09-17T05:36:42.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.638+0000 7fe342e00640 1 --2- 172.21.15.122:0/3633448017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0ce890 0x7fe33c06eb40 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.638+0000 7fe342e00640 1 --2- 172.21.15.122:0/3633448017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe33c15a9d0 0x7fe33c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.638+0000 7fe342e00640 1 -- 172.21.15.122:0/3633448017 >> 172.21.15.122:0/3633448017 conn(0x7fe33c0cfbc0 msgr2=0x7fe33c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:42.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.638+0000 7fe342e00640 1 -- 172.21.15.122:0/3633448017 shutdown_connections 2024-09-17T05:36:42.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.638+0000 7fe342e00640 1 -- 172.21.15.122:0/3633448017 wait complete. 2024-09-17T05:36:42.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 Processor -- start 2024-09-17T05:36:42.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 -- start start 2024-09-17T05:36:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0734f0 0x7fe33c09ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33c0ce890 0x7fe33c164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe33c15a9d0 0x7fe33c1523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1674e0 con 0x7fe33c0734f0 2024-09-17T05:36:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c167360 con 0x7fe33c0ce890 2024-09-17T05:36:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe33c167660 con 0x7fe33c15a9d0 2024-09-17T05:36:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0734f0 0x7fe33c09ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:42.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0734f0 0x7fe33c09ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54260/0 (socket says 172.21.15.122:54260) 2024-09-17T05:36:42.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 -- 172.21.15.122:0/638604957 learned_addr learned my addr 172.21.15.122:0/638604957 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:42.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33b400640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33c0ce890 0x7fe33c164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:42.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe340e00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe33c15a9d0 0x7fe33c1523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:42.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 -- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe33c15a9d0 msgr2=0x7fe33c1523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe33c15a9d0 0x7fe33c1523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 -- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33c0ce890 msgr2=0x7fe33c164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33c0ce890 0x7fe33c164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 -- 172.21.15.122:0/638604957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c155210 con 0x7fe33c0734f0 2024-09-17T05:36:42.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe33be00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0734f0 0x7fe33c09ba60 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c870 tx=0x7fe32c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:42.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe333e00640 1 -- 172.21.15.122:0/638604957 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c003b80 con 0x7fe33c0734f0 2024-09-17T05:36:42.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe333e00640 1 -- 172.21.15.122:0/638604957 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe32c003d20 con 0x7fe33c0734f0 2024-09-17T05:36:42.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe333e00640 1 -- 172.21.15.122:0/638604957 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00aab0 con 0x7fe33c0734f0 2024-09-17T05:36:42.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c159960 con 0x7fe33c0734f0 2024-09-17T05:36:42.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.642+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c164cb0 con 0x7fe33c0734f0 2024-09-17T05:36:42.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.646+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c069410 con 0x7fe33c0734f0 2024-09-17T05:36:42.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.646+0000 7fe333e00640 1 -- 172.21.15.122:0/638604957 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7fe32c00ac50 con 0x7fe33c0734f0 2024-09-17T05:36:42.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.646+0000 7fe333e00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe31c074880 0x7fe31c076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:42.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.646+0000 7fe333e00640 1 -- 172.21.15.122:0/638604957 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7fe32c092ca0 con 0x7fe33c0734f0 2024-09-17T05:36:42.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.650+0000 7fe33b400640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe31c074880 0x7fe31c076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:42.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.650+0000 7fe33b400640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe31c074880 0x7fe31c076d40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0af410 tx=0x7fe328010040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:42.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.650+0000 7fe333e00640 1 -- 172.21.15.122:0/638604957 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe32c05f160 con 0x7fe33c0734f0 2024-09-17T05:36:42.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.842+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fe33c06eb40 con 0x7fe31c074880 2024-09-17T05:36:42.849 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.852+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.849 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.852+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.849 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.852+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.849 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.852+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.849 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:36:42.852+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:36:42.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe333e00640 1 -- 172.21.15.122:0/638604957 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe33c06eb40 con 0x7fe31c074880 2024-09-17T05:36:42.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe31c074880 msgr2=0x7fe31c076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe31c074880 0x7fe31c076d40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe33c0af410 tx=0x7fe328010040 comp rx=0 tx=0).stop 2024-09-17T05:36:42.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0734f0 msgr2=0x7fe33c09ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0734f0 0x7fe33c09ba60 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c870 tx=0x7fe32c00cd30 comp rx=0 tx=0).stop 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 shutdown_connections 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe31c074880 0x7fe31c076d40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe33c15a9d0 0x7fe33c1523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe33c0ce890 0x7fe33c164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 --2- 172.21.15.122:0/638604957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe33c0734f0 0x7fe33c09ba60 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 >> 172.21.15.122:0/638604957 conn(0x7fe33c0cfbc0 msgr2=0x7fe33c15bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 shutdown_connections 2024-09-17T05:36:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:42.890+0000 7fe342e00640 1 -- 172.21.15.122:0/638604957 wait complete. 2024-09-17T05:36:42.906 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:36:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 -- 172.21.15.122:0/87497947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 msgr2=0x7f6db40affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 --2- 172.21.15.122:0/87497947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 0x7f6db40affc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009840 tx=0x7f6dac01c820 comp rx=0 tx=0).stop 2024-09-17T05:36:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 -- 172.21.15.122:0/87497947 shutdown_connections 2024-09-17T05:36:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 --2- 172.21.15.122:0/87497947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 0x7f6db40affc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 --2- 172.21.15.122:0/87497947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6db40b1650 0x7f6db40b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 -- 172.21.15.122:0/87497947 >> 172.21.15.122:0/87497947 conn(0x7f6db40d1dd0 msgr2=0x7f6db40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 -- 172.21.15.122:0/87497947 shutdown_connections 2024-09-17T05:36:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 -- 172.21.15.122:0/87497947 wait complete. 2024-09-17T05:36:43.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.054+0000 7f6dba800640 1 Processor -- start 2024-09-17T05:36:43.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 -- start start 2024-09-17T05:36:43.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 0x7f6db40c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:43.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6db40b1650 0x7f6db415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:43.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6db41605d0 0x7f6db415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:43.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4169540 con 0x7f6db40afb10 2024-09-17T05:36:43.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6db40b1430 con 0x7f6db41605d0 2024-09-17T05:36:43.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6db41696c0 con 0x7f6db40b1650 2024-09-17T05:36:43.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 0x7f6db40c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 0x7f6db40c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54292/0 (socket says 172.21.15.122:54292) 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 -- 172.21.15.122:0/4033587874 learned_addr learned my addr 172.21.15.122:0/4033587874 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6daaa00640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6db40b1650 0x7f6db415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3e00640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6db41605d0 0x7f6db415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 -- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6db40b1650 msgr2=0x7f6db415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6db40b1650 0x7f6db415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 -- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6db41605d0 msgr2=0x7f6db415c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6db41605d0 0x7f6db415c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 -- 172.21.15.122:0/4033587874 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db40c8750 con 0x7f6db40afb10 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6db3400640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 0x7f6db40c8210 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6da400c400 tx=0x7f6da400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dabe00640 1 -- 172.21.15.122:0/4033587874 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4011790 con 0x7f6db40afb10 2024-09-17T05:36:43.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dabe00640 1 -- 172.21.15.122:0/4033587874 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6da4017070 con 0x7f6db40afb10 2024-09-17T05:36:43.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dabe00640 1 -- 172.21.15.122:0/4033587874 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4011cc0 con 0x7f6db40afb10 2024-09-17T05:36:43.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db40b72f0 con 0x7f6db40afb10 2024-09-17T05:36:43.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db40b78b0 con 0x7f6db40afb10 2024-09-17T05:36:43.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.058+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db40affc0 con 0x7f6db40afb10 2024-09-17T05:36:43.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.062+0000 7f6dabe00640 1 -- 172.21.15.122:0/4033587874 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 38) ==== 99274+0+0 (secure 0 0 0) 0x7f6da401c460 con 0x7f6db40afb10 2024-09-17T05:36:43.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.062+0000 7f6dabe00640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d90074880 0x7f6d90076d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:36:43.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.062+0000 7f6dabe00640 1 -- 172.21.15.122:0/4033587874 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f6da4093660 con 0x7f6db40afb10 2024-09-17T05:36:43.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.062+0000 7f6daaa00640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d90074880 0x7f6d90076d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:36:43.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.062+0000 7f6daaa00640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d90074880 0x7f6d90076d40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c005c50 tx=0x7f6d9c005be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:36:43.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.066+0000 7f6dabe00640 1 -- 172.21.15.122:0/4033587874 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6da40602f0 con 0x7f6db40afb10 2024-09-17T05:36:43.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.262+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f6db40b7cd0 con 0x7f6d90074880 2024-09-17T05:36:43.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.266+0000 7f6dabe00640 1 -- 172.21.15.122:0/4033587874 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f6db40b7cd0 con 0x7f6d90074880 2024-09-17T05:36:43.263 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d90074880 msgr2=0x7f6d90076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d90074880 0x7f6d90076d40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c005c50 tx=0x7f6d9c005be0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 msgr2=0x7f6db40c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 0x7f6db40c8210 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6da400c400 tx=0x7f6da400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 shutdown_connections 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d90074880 0x7f6d90076d40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6db41605d0 0x7f6db415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6db40b1650 0x7f6db415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 --2- 172.21.15.122:0/4033587874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6db40afb10 0x7f6db40c8210 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 >> 172.21.15.122:0/4033587874 conn(0x7f6db40d1dd0 msgr2=0x7f6db4071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:36:43.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 shutdown_connections 2024-09-17T05:36:43.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:36:43.270+0000 7f6dba800640 1 -- 172.21.15.122:0/4033587874 wait complete. 2024-09-17T05:36:43.284 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24 2024-09-17T05:36:43.284 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:36:43.284 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:43.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:43.293 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:36:52.134 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:36:52.135 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:36:52.135 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.78411 s, 122 MB/s 2024-09-17T05:36:52.136 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:36:52.136 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:36:52.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:36:52.145 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:37:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:37:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:37:01.496 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.2986 s, 115 MB/s 2024-09-17T05:37:01.497 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:37:01.497 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:01.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:01.507 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:37:03.532 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:37:03.536 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:37:10.874 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:37:10.874 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:37:10.875 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.31807 s, 115 MB/s 2024-09-17T05:37:10.876 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T05:37:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.025+0000 7f4455800640 1 -- 172.21.15.122:0/1102611813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 msgr2=0x7f44500d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.025+0000 7f4455800640 1 --2- 172.21.15.122:0/1102611813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 0x7f44500d28c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f445006dc00 tx=0x7f443801c890 comp rx=0 tx=0).stop 2024-09-17T05:37:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.025+0000 7f4455800640 1 -- 172.21.15.122:0/1102611813 shutdown_connections 2024-09-17T05:37:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.025+0000 7f4455800640 1 --2- 172.21.15.122:0/1102611813 >> v2:172.21.15.173:3300/0 conn(0x7f44500d02e0 0x7f44500d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.025+0000 7f4455800640 1 --2- 172.21.15.122:0/1102611813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 0x7f44500d28c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.025+0000 7f4455800640 1 -- 172.21.15.122:0/1102611813 >> 172.21.15.122:0/1102611813 conn(0x7f44500d1da0 msgr2=0x7f44500d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.025+0000 7f4455800640 1 -- 172.21.15.122:0/1102611813 shutdown_connections 2024-09-17T05:37:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.025+0000 7f4455800640 1 -- 172.21.15.122:0/1102611813 wait complete. 2024-09-17T05:37:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 Processor -- start 2024-09-17T05:37:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 -- start start 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 0x7f445014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44500d02e0 0x7f445015db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f445015eff0 0x7f44500cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44500b0b80 con 0x7f44500cff00 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f44500b0a00 con 0x7f44500d02e0 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f44500b0d00 con 0x7f445015eff0 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 0x7f445014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 0x7f445014c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54814/0 (socket says 172.21.15.122:54814) 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 -- 172.21.15.122:0/1517635462 learned_addr learned my addr 172.21.15.122:0/1517635462 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4447e00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44500d02e0 0x7f445015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4454e00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f445015eff0 0x7f44500cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 -- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f445015eff0 msgr2=0x7f44500cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f445015eff0 0x7f44500cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 -- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44500d02e0 msgr2=0x7f445015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44500d02e0 0x7f445015db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 -- 172.21.15.122:0/1517635462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44500b2c20 con 0x7f44500cff00 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4447e00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44500d02e0 0x7f445015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444fe00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 0x7f445014c2d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4438000c10 tx=0x7f44380a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444cc00640 1 -- 172.21.15.122:0/1517635462 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44380bd070 con 0x7f44500cff00 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444cc00640 1 -- 172.21.15.122:0/1517635462 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4438003970 con 0x7f44500cff00 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44500cc880 con 0x7f44500cff00 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f444cc00640 1 -- 172.21.15.122:0/1517635462 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44380b2040 con 0x7f44500cff00 2024-09-17T05:37:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.029+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44500cce00 con 0x7f44500cff00 2024-09-17T05:37:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.033+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44500d28c0 con 0x7f44500cff00 2024-09-17T05:37:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.033+0000 7f444cc00640 1 -- 172.21.15.122:0/1517635462 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 40) ==== 99376+0+0 (secure 0 0 0) 0x7f44380ad440 con 0x7f44500cff00 2024-09-17T05:37:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.033+0000 7f444cc00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f442c0749d0 0x7f442c076e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.033+0000 7f444cc00640 1 -- 172.21.15.122:0/1517635462 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f443812d2e0 con 0x7f44500cff00 2024-09-17T05:37:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.033+0000 7f4447e00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f442c0749d0 0x7f442c076e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:11.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.033+0000 7f4447e00640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f442c0749d0 0x7f442c076e90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f44500cf8a0 tx=0x7f44400091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:11.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.037+0000 7f444cc00640 1 -- 172.21.15.122:0/1517635462 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f44380164b0 con 0x7f44500cff00 2024-09-17T05:37:11.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.233+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f4450069e60 con 0x7f442c0749d0 2024-09-17T05:37:11.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.245+0000 7f444cc00640 1 -- 172.21.15.122:0/1517635462 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4440016070 con 0x7f442c0749d0 2024-09-17T05:37:11.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f442c0749d0 msgr2=0x7f442c076e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f442c0749d0 0x7f442c076e90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f44500cf8a0 tx=0x7f44400091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 msgr2=0x7f445014c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 0x7f445014c2d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f4438000c10 tx=0x7f44380a5750 comp rx=0 tx=0).stop 2024-09-17T05:37:11.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 shutdown_connections 2024-09-17T05:37:11.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f442c0749d0 0x7f442c076e90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f445015eff0 0x7f44500cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44500d02e0 0x7f445015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 --2- 172.21.15.122:0/1517635462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44500cff00 0x7f445014c2d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 >> 172.21.15.122:0/1517635462 conn(0x7f44500d1da0 msgr2=0x7f445006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:11.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 shutdown_connections 2024-09-17T05:37:11.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.249+0000 7f4455800640 1 -- 172.21.15.122:0/1517635462 wait complete. 2024-09-17T05:37:11.279 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:37:11.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- 172.21.15.122:0/497027515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 msgr2=0x7f9e1c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 --2- 172.21.15.122:0/497027515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 0x7f9e1c0051d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c310 tx=0x7f9e3001f720 comp rx=0 tx=0).stop 2024-09-17T05:37:11.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3da00640 1 -- 172.21.15.122:0/497027515 reap_dead start 2024-09-17T05:37:11.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- 172.21.15.122:0/497027515 shutdown_connections 2024-09-17T05:37:11.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- 172.21.15.122:0/497027515 >> 172.21.15.122:0/497027515 conn(0x7f9e380b18f0 msgr2=0x7f9e380b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:11.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- 172.21.15.122:0/497027515 shutdown_connections 2024-09-17T05:37:11.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- 172.21.15.122:0/497027515 wait complete. 2024-09-17T05:37:11.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 Processor -- start 2024-09-17T05:37:11.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- start start 2024-09-17T05:37:11.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 0x7f9e3815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e38074200 0x7f9e38152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e38156800 0x7f9e3809cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3806fb40 con 0x7f9e1c004d50 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3806f9c0 con 0x7f9e38156800 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.429+0000 7f9e3fa00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9e380acd40 con 0x7f9e38074200 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 0x7f9e3815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e38156800 0x7f9e3809cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 0x7f9e3815afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59330/0 (socket says 172.21.15.122:59330) 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 -- 172.21.15.122:0/2742963421 learned_addr learned my addr 172.21.15.122:0/2742963421 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e37e00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e38074200 0x7f9e38152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 -- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e38074200 msgr2=0x7f9e38152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e38074200 0x7f9e38152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 -- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e38156800 msgr2=0x7f9e3809cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e38156800 0x7f9e3809cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 -- 172.21.15.122:0/2742963421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e38156c80 con 0x7f9e1c004d50 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e37e00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e38074200 0x7f9e38152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3d000640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 0x7f9e3815afc0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800ed80 tx=0x7f9e2800d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e34c00640 1 -- 172.21.15.122:0/2742963421 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28003b90 con 0x7f9e1c004d50 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e34c00640 1 -- 172.21.15.122:0/2742963421 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e28003d30 con 0x7f9e1c004d50 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e34c00640 1 -- 172.21.15.122:0/2742963421 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28004290 con 0x7f9e1c004d50 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e381f48d0 con 0x7f9e1c004d50 2024-09-17T05:37:11.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e381f4e90 con 0x7f9e1c004d50 2024-09-17T05:37:11.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e380698f0 con 0x7f9e1c004d50 2024-09-17T05:37:11.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.433+0000 7f9e34c00640 1 -- 172.21.15.122:0/2742963421 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 40) ==== 99376+0+0 (secure 0 0 0) 0x7f9e28004430 con 0x7f9e1c004d50 2024-09-17T05:37:11.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.437+0000 7f9e34c00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9e180749d0 0x7f9e18076e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:11.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.437+0000 7f9e34c00640 1 -- 172.21.15.122:0/2742963421 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f9e28093ce0 con 0x7f9e1c004d50 2024-09-17T05:37:11.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.437+0000 7f9e37e00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9e180749d0 0x7f9e18076e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:11.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.437+0000 7f9e37e00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9e180749d0 0x7f9e18076e90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e3809d7a0 tx=0x7f9e240091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:11.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.441+0000 7f9e34c00640 1 -- 172.21.15.122:0/2742963421 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9e280607f0 con 0x7f9e1c004d50 2024-09-17T05:37:11.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.633+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9e381f54e0 con 0x7f9e180749d0 2024-09-17T05:37:11.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e34c00640 1 -- 172.21.15.122:0/2742963421 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f9e381f54e0 con 0x7f9e180749d0 2024-09-17T05:37:11.633 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24 2024-09-17T05:37:11.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9e180749d0 msgr2=0x7f9e18076e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9e180749d0 0x7f9e18076e90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e3809d7a0 tx=0x7f9e240091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 msgr2=0x7f9e3815afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:11.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 0x7f9e3815afc0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800ed80 tx=0x7f9e2800d7b0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 shutdown_connections 2024-09-17T05:37:11.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9e180749d0 0x7f9e18076e90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e38156800 0x7f9e3809cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e38074200 0x7f9e38152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 --2- 172.21.15.122:0/2742963421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e1c004d50 0x7f9e3815afc0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:11.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 >> 172.21.15.122:0/2742963421 conn(0x7f9e380b18f0 msgr2=0x7f9e380b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:11.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 shutdown_connections 2024-09-17T05:37:11.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:11.637+0000 7f9e3fa00640 1 -- 172.21.15.122:0/2742963421 wait complete. 2024-09-17T05:37:11.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:11.654 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24 2024-09-17T05:37:11.654 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:37:11.663 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24 2024-09-17T05:37:11.663 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="2972172288" 2024-09-17T05:37:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:11.664 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-17T05:37:13.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:13.665 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24 2024-09-17T05:37:13.665 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:37:13.676 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/4f169ef0-9686-4e02-8a6d-134ee0cc0a24 2024-09-17T05:37:13.676 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T05:37:13.676 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:13.677 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T05:37:13.678 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-17T05:37:13.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.893+0000 7f9ac6600640 1 -- 172.21.15.122:0/1835827554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 msgr2=0x7f9ac00b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:13.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.893+0000 7f9ac6600640 1 --2- 172.21.15.122:0/1835827554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 0x7f9ac00b42d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00af800 tx=0x7f9ab801cc60 comp rx=0 tx=0).stop 2024-09-17T05:37:13.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.893+0000 7f9ac6600640 1 -- 172.21.15.122:0/1835827554 shutdown_connections 2024-09-17T05:37:13.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.893+0000 7f9ac6600640 1 --2- 172.21.15.122:0/1835827554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 0x7f9ac00b42d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:13.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.893+0000 7f9ac6600640 1 --2- 172.21.15.122:0/1835827554 >> v2:172.21.15.173:3300/0 conn(0x7f9ac00cd100 0x7f9ac015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:13.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.893+0000 7f9ac6600640 1 -- 172.21.15.122:0/1835827554 >> 172.21.15.122:0/1835827554 conn(0x7f9ac00b18f0 msgr2=0x7f9ac00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:13.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.893+0000 7f9ac6600640 1 -- 172.21.15.122:0/1835827554 shutdown_connections 2024-09-17T05:37:13.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.893+0000 7f9ac6600640 1 -- 172.21.15.122:0/1835827554 wait complete. 2024-09-17T05:37:13.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 Processor -- start 2024-09-17T05:37:13.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 -- start start 2024-09-17T05:37:13.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac006f8b0 0x7f9ac01515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:13.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 0x7f9ac00b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:13.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac00cd100 0x7f9ac006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac0168130 con 0x7f9ac00acd40 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac00cf4d0 con 0x7f9ac00cd100 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac01682b0 con 0x7f9ac006f8b0 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 0x7f9ac00b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 0x7f9ac00b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59344/0 (socket says 172.21.15.122:59344) 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 -- 172.21.15.122:0/2459840525 learned_addr learned my addr 172.21.15.122:0/2459840525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abfe00640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac00cd100 0x7f9ac006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abf400640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac006f8b0 0x7f9ac01515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 -- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac006f8b0 msgr2=0x7f9ac01515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac006f8b0 0x7f9ac01515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 -- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac00cd100 msgr2=0x7f9ac006e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac00cd100 0x7f9ac006e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:13.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 -- 172.21.15.122:0/2459840525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac00b1480 con 0x7f9ac00acd40 2024-09-17T05:37:13.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9abea00640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 0x7f9ac00b0f40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00e7c0 tx=0x7f9aac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:13.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ab7400640 1 -- 172.21.15.122:0/2459840525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00cc60 con 0x7f9ac00acd40 2024-09-17T05:37:13.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ab7400640 1 -- 172.21.15.122:0/2459840525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aac00ce20 con 0x7f9ac00acd40 2024-09-17T05:37:13.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ab7400640 1 -- 172.21.15.122:0/2459840525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac005280 con 0x7f9ac00acd40 2024-09-17T05:37:13.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac01f7f70 con 0x7f9ac00acd40 2024-09-17T05:37:13.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.897+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac01f8530 con 0x7f9ac00acd40 2024-09-17T05:37:13.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.901+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac0069410 con 0x7f9ac00acd40 2024-09-17T05:37:13.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.901+0000 7f9ab7400640 1 -- 172.21.15.122:0/2459840525 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 40) ==== 99376+0+0 (secure 0 0 0) 0x7f9aac005420 con 0x7f9ac00acd40 2024-09-17T05:37:13.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.901+0000 7f9ab7400640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa00749a0 0x7f9aa0076e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:13.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.901+0000 7f9ab7400640 1 -- 172.21.15.122:0/2459840525 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f9aac093760 con 0x7f9ac00acd40 2024-09-17T05:37:13.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.901+0000 7f9abf400640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa00749a0 0x7f9aa0076e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:13.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.901+0000 7f9abf400640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa00749a0 0x7f9aa0076e60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0007fd0 tx=0x7f9ab0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:13.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:13.905+0000 7f9ab7400640 1 -- 172.21.15.122:0/2459840525 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9aac05fc20 con 0x7f9ac00acd40 2024-09-17T05:37:14.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.105+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f9ac00b42d0 con 0x7f9aa00749a0 2024-09-17T05:37:14.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ab7400640 1 -- 172.21.15.122:0/2459840525 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ac00b42d0 con 0x7f9aa00749a0 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa00749a0 msgr2=0x7f9aa0076e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa00749a0 0x7f9aa0076e60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0007fd0 tx=0x7f9ab0009040 comp rx=0 tx=0).stop 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 msgr2=0x7f9ac00b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 0x7f9ac00b0f40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00e7c0 tx=0x7f9aac00ec80 comp rx=0 tx=0).stop 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 shutdown_connections 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa00749a0 0x7f9aa0076e60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac00cd100 0x7f9ac006e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac00acd40 0x7f9ac00b0f40 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 --2- 172.21.15.122:0/2459840525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac006f8b0 0x7f9ac01515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 >> 172.21.15.122:0/2459840525 conn(0x7f9ac00b18f0 msgr2=0x7f9ac015a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 shutdown_connections 2024-09-17T05:37:14.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.169+0000 7f9ac6600640 1 -- 172.21.15.122:0/2459840525 wait complete. 2024-09-17T05:37:14.173 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.176+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.173 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.176+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.173 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.176+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.174 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.176+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.174 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.176+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.190 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T05:37:14.191 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.196+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.192 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.196+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.192 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.196+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.192 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.196+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.192 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:14.196+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05f800640 1 -- 172.21.15.122:0/3833558860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc03c004810 msgr2=0x7fc03c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05f800640 1 --2- 172.21.15.122:0/3833558860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc03c004810 0x7fc03c004c90 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fc040006f90 tx=0x7fc04001cec0 comp rx=0 tx=0).stop 2024-09-17T05:37:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05d800640 1 -- 172.21.15.122:0/3833558860 reap_dead start 2024-09-17T05:37:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05f800640 1 -- 172.21.15.122:0/3833558860 shutdown_connections 2024-09-17T05:37:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05f800640 1 -- 172.21.15.122:0/3833558860 >> 172.21.15.122:0/3833558860 conn(0x7fc0580b18f0 msgr2=0x7fc0580b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05f800640 1 -- 172.21.15.122:0/3833558860 shutdown_connections 2024-09-17T05:37:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05f800640 1 -- 172.21.15.122:0/3833558860 wait complete. 2024-09-17T05:37:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05f800640 1 Processor -- start 2024-09-17T05:37:14.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.345+0000 7fc05f800640 1 -- start start 2024-09-17T05:37:14.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc03c004810 0x7fc05806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:14.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc058155360 0x7fc05815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:14.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05815ca00 0x7fc05809c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:14.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0580cc660 con 0x7fc05815ca00 2024-09-17T05:37:14.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc0580cd5b0 con 0x7fc058155360 2024-09-17T05:37:14.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc0580cc7e0 con 0x7fc03c004810 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05815ca00 0x7fc05809c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05ce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc03c004810 0x7fc05806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05ce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc03c004810 0x7fc05806e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:46062/0 (socket says 172.21.15.122:46062) 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05ce00640 1 -- 172.21.15.122:0/1018420771 learned_addr learned my addr 172.21.15.122:0/1018420771 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc058155360 0x7fc05815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05d800640 1 -- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc03c004810 msgr2=0x7fc05806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05d800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc03c004810 0x7fc05806e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05d800640 1 -- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc058155360 msgr2=0x7fc05815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05d800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc058155360 0x7fc05815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05d800640 1 -- 172.21.15.122:0/1018420771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0580cbde0 con 0x7fc05815ca00 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05d800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05815ca00 0x7fc05809c1b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc0580697d0 tx=0x7fc05000d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc055600640 1 -- 172.21.15.122:0/1018420771 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050003980 con 0x7fc05815ca00 2024-09-17T05:37:14.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc055600640 1 -- 172.21.15.122:0/1018420771 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc050003b20 con 0x7fc05815ca00 2024-09-17T05:37:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0581e81d0 con 0x7fc05815ca00 2024-09-17T05:37:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc055600640 1 -- 172.21.15.122:0/1018420771 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050004020 con 0x7fc05815ca00 2024-09-17T05:37:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05806ec30 con 0x7fc05815ca00 2024-09-17T05:37:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc058069410 con 0x7fc05815ca00 2024-09-17T05:37:14.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.349+0000 7fc055600640 1 -- 172.21.15.122:0/1018420771 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 40) ==== 99376+0+0 (secure 0 0 0) 0x7fc05001e060 con 0x7fc05815ca00 2024-09-17T05:37:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.353+0000 7fc055600640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0340749a0 0x7fc034076e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.353+0000 7fc055600640 1 -- 172.21.15.122:0/1018420771 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fc050093480 con 0x7fc05815ca00 2024-09-17T05:37:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.353+0000 7fc05ce00640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0340749a0 0x7fc034076e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.353+0000 7fc05ce00640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0340749a0 0x7fc034076e60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc0580b0f10 tx=0x7fc048006b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:14.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.357+0000 7fc055600640 1 -- 172.21.15.122:0/1018420771 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc05005f940 con 0x7fc05815ca00 2024-09-17T05:37:14.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:14.557+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fc0580ccce0 con 0x7fc0340749a0 2024-09-17T05:37:15.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.169+0000 7fc055600640 1 -- 172.21.15.122:0/1018420771 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 41) ==== 99478+0+0 (secure 0 0 0) 0x7fc05005bbb0 con 0x7fc05815ca00 2024-09-17T05:37:15.174 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:15.176+0000 7fc03e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:15.174 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:15.176+0000 7fc03e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:15.174 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:15.176+0000 7fc03e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:15.174 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:15.176+0000 7fc03e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:15.174 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:15.176+0000 7fc03e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.205+0000 7fc055600640 1 -- 172.21.15.122:0/1018420771 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7fc0580ccce0 con 0x7fc0340749a0 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "in-progress", 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_report": { 2024-09-17T05:37:15.202 INFO:teuthology.orchestra.run.smithi122.stdout: "percentage cloned": "0.0%", 2024-09-17T05:37:15.203 INFO:teuthology.orchestra.run.smithi122.stdout: "amount cloned": "184/3.0G", 2024-09-17T05:37:15.203 INFO:teuthology.orchestra.run.smithi122.stdout: "files cloned": "3/4" 2024-09-17T05:37:15.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:15.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:15.203 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:15.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0340749a0 msgr2=0x7fc034076e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:15.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0340749a0 0x7fc034076e60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc0580b0f10 tx=0x7fc048006b20 comp rx=0 tx=0).stop 2024-09-17T05:37:15.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05815ca00 msgr2=0x7fc05809c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:15.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05815ca00 0x7fc05809c1b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc0580697d0 tx=0x7fc05000d7b0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 shutdown_connections 2024-09-17T05:37:15.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc0340749a0 0x7fc034076e60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05815ca00 0x7fc05809c1b0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc058155360 0x7fc05815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 --2- 172.21.15.122:0/1018420771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc03c004810 0x7fc05806e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 >> 172.21.15.122:0/1018420771 conn(0x7fc0580b18f0 msgr2=0x7fc0580ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:15.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 shutdown_connections 2024-09-17T05:37:15.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.209+0000 7fc05f800640 1 -- 172.21.15.122:0/1018420771 wait complete. 2024-09-17T05:37:15.225 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-17T05:37:15.225 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T05:37:15.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.425+0000 7f1c73e00640 1 -- 172.21.15.122:0/2253623438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c740cba70 msgr2=0x7f1c740cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:15.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.425+0000 7f1c73e00640 1 --2- 172.21.15.122:0/2253623438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c740cba70 0x7f1c740cea20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1c74074f50 tx=0x7f1c6c01cd80 comp rx=0 tx=0).stop 2024-09-17T05:37:15.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.425+0000 7f1c73e00640 1 -- 172.21.15.122:0/2253623438 shutdown_connections 2024-09-17T05:37:15.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.425+0000 7f1c73e00640 1 --2- 172.21.15.122:0/2253623438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c740cba70 0x7f1c740cea20 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.425+0000 7f1c73e00640 1 --2- 172.21.15.122:0/2253623438 >> v2:172.21.15.173:3300/0 conn(0x7f1c74155080 0x7f1c740cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.425+0000 7f1c73e00640 1 -- 172.21.15.122:0/2253623438 >> 172.21.15.122:0/2253623438 conn(0x7f1c7406dd30 msgr2=0x7f1c7406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:15.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.425+0000 7f1c73e00640 1 -- 172.21.15.122:0/2253623438 shutdown_connections 2024-09-17T05:37:15.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.425+0000 7f1c73e00640 1 -- 172.21.15.122:0/2253623438 wait complete. 2024-09-17T05:37:15.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 Processor -- start 2024-09-17T05:37:15.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 -- start start 2024-09-17T05:37:15.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c740cba70 0x7f1c7406f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:15.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c740cda50 0x7f1c740af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:15.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c74155080 0x7f1c740b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:15.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74167ae0 con 0x7f1c74155080 2024-09-17T05:37:15.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74167960 con 0x7f1c740cda50 2024-09-17T05:37:15.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1c74167c60 con 0x7f1c740cba70 2024-09-17T05:37:15.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c74155080 0x7f1c740b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:15.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c74155080 0x7f1c740b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59386/0 (socket says 172.21.15.122:59386) 2024-09-17T05:37:15.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 -- 172.21.15.122:0/3098038426 learned_addr learned my addr 172.21.15.122:0/3098038426 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:15.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c72000640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c740cda50 0x7f1c740af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:15.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 -- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c740cba70 msgr2=0x7f1c7406f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:15.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c740cba70 0x7f1c7406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 -- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c740cda50 msgr2=0x7f1c740af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:15.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c740cda50 0x7f1c740af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 -- 172.21.15.122:0/3098038426 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c740c3d20 con 0x7f1c74155080 2024-09-17T05:37:15.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73400640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c74155080 0x7f1c740b17c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009bd0 tx=0x7f1c6c003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:15.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c6aa00640 1 -- 172.21.15.122:0/3098038426 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0bc070 con 0x7f1c74155080 2024-09-17T05:37:15.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c6aa00640 1 -- 172.21.15.122:0/3098038426 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c6c0ad8e0 con 0x7f1c74155080 2024-09-17T05:37:15.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c6aa00640 1 -- 172.21.15.122:0/3098038426 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0b2040 con 0x7f1c74155080 2024-09-17T05:37:15.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.429+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c740c3590 con 0x7f1c74155080 2024-09-17T05:37:15.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.433+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c74164680 con 0x7f1c74155080 2024-09-17T05:37:15.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.433+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c740c9230 con 0x7f1c74155080 2024-09-17T05:37:15.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.433+0000 7f1c6aa00640 1 -- 172.21.15.122:0/3098038426 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 41) ==== 99478+0+0 (secure 0 0 0) 0x7f1c6c0add10 con 0x7f1c74155080 2024-09-17T05:37:15.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.433+0000 7f1c6aa00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1c54074ac0 0x7f1c54076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:15.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.433+0000 7f1c6aa00640 1 -- 172.21.15.122:0/3098038426 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f1c6c12cea0 con 0x7f1c74155080 2024-09-17T05:37:15.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.441+0000 7f1c72a00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1c54074ac0 0x7f1c54076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.441+0000 7f1c6aa00640 1 -- 172.21.15.122:0/3098038426 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1c6c0f3120 con 0x7f1c74155080 2024-09-17T05:37:15.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.441+0000 7f1c72a00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1c54074ac0 0x7f1c54076f80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1c740c8320 tx=0x7f1c64005c80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.741+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f1c740cea20 con 0x7f1c54074ac0 2024-09-17T05:37:15.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.801+0000 7f1c6aa00640 1 -- 172.21.15.122:0/3098038426 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f1c740cea20 con 0x7f1c54074ac0 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "in-progress", 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_report": { 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "percentage cloned": "0.0%", 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "amount cloned": "184/3.0G", 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "files cloned": "4/4" 2024-09-17T05:37:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:15.798 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:15.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1c54074ac0 msgr2=0x7f1c54076f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:15.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1c54074ac0 0x7f1c54076f80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1c740c8320 tx=0x7f1c64005c80 comp rx=0 tx=0).stop 2024-09-17T05:37:15.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c74155080 msgr2=0x7f1c740b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:15.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c74155080 0x7f1c740b17c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009bd0 tx=0x7f1c6c003f80 comp rx=0 tx=0).stop 2024-09-17T05:37:15.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 shutdown_connections 2024-09-17T05:37:15.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1c54074ac0 0x7f1c54076f80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c74155080 0x7f1c740b17c0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c740cda50 0x7f1c740af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 --2- 172.21.15.122:0/3098038426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c740cba70 0x7f1c7406f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:15.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.805+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 >> 172.21.15.122:0/3098038426 conn(0x7f1c7406dd30 msgr2=0x7f1c74154730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:15.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.809+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 shutdown_connections 2024-09-17T05:37:15.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:15.809+0000 7f1c73e00640 1 -- 172.21.15.122:0/3098038426 wait complete. 2024-09-17T05:37:15.839 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 0.0% 2024-09-17T05:37:16.840 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T05:37:17.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 -- 172.21.15.122:0/2393530096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc0aeb00 msgr2=0x7f94cc0aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 --2- 172.21.15.122:0/2393530096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc0aeb00 0x7f94cc0aefb0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f94cc073d00 tx=0x7f94c401cd60 comp rx=0 tx=0).stop 2024-09-17T05:37:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 -- 172.21.15.122:0/2393530096 shutdown_connections 2024-09-17T05:37:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 --2- 172.21.15.122:0/2393530096 >> v2:172.21.15.173:3300/0 conn(0x7f94cc0b1650 0x7f94cc0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 --2- 172.21.15.122:0/2393530096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc0aeb00 0x7f94cc0aefb0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 --2- 172.21.15.122:0/2393530096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94cc1605d0 0x7f94cc0d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 -- 172.21.15.122:0/2393530096 >> 172.21.15.122:0/2393530096 conn(0x7f94cc0d1dd0 msgr2=0x7f94cc0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 -- 172.21.15.122:0/2393530096 shutdown_connections 2024-09-17T05:37:17.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.045+0000 7f94d4200640 1 -- 172.21.15.122:0/2393530096 wait complete. 2024-09-17T05:37:17.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 Processor -- start 2024-09-17T05:37:17.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 -- start start 2024-09-17T05:37:17.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94cc0aeb00 0x7f94cc1f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:17.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94cc0b1650 0x7f94cc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:17.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc1605d0 0x7f94cc1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc0b13a0 con 0x7f94cc1605d0 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc0b1220 con 0x7f94cc0aeb00 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f94cc0afd00 con 0x7f94cc0b1650 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc1605d0 0x7f94cc1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc1605d0 0x7f94cc1f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59408/0 (socket says 172.21.15.122:59408) 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 -- 172.21.15.122:0/1751349344 learned_addr learned my addr 172.21.15.122:0/1751349344 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 -- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94cc0b1650 msgr2=0x7f94cc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94cc0b1650 0x7f94cc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 -- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94cc0aeb00 msgr2=0x7f94cc1f8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94cc0aeb00 0x7f94cc1f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 -- 172.21.15.122:0/1751349344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc0cd0e0 con 0x7f94cc1605d0 2024-09-17T05:37:17.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d2200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc1605d0 0x7f94cc1f71e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f94c401b410 tx=0x7f94c40a5b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:17.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94c9600640 1 -- 172.21.15.122:0/1751349344 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c40bc020 con 0x7f94cc1605d0 2024-09-17T05:37:17.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94c9600640 1 -- 172.21.15.122:0/1751349344 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94c4005470 con 0x7f94cc1605d0 2024-09-17T05:37:17.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94cc0cd3c0 con 0x7f94cc1605d0 2024-09-17T05:37:17.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94cc1f7ee0 con 0x7f94cc1605d0 2024-09-17T05:37:17.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.049+0000 7f94c9600640 1 -- 172.21.15.122:0/1751349344 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c40b2040 con 0x7f94cc1605d0 2024-09-17T05:37:17.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.053+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94cc0b1a30 con 0x7f94cc1605d0 2024-09-17T05:37:17.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.053+0000 7f94c9600640 1 -- 172.21.15.122:0/1751349344 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 41) ==== 99478+0+0 (secure 0 0 0) 0x7f94c40bf070 con 0x7f94cc1605d0 2024-09-17T05:37:17.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.053+0000 7f94c9600640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f94ac074ac0 0x7f94ac076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:17.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.053+0000 7f94c9600640 1 -- 172.21.15.122:0/1751349344 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f94c412d770 con 0x7f94cc1605d0 2024-09-17T05:37:17.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.057+0000 7f94d1800640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f94ac074ac0 0x7f94ac076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:17.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.061+0000 7f94c9600640 1 -- 172.21.15.122:0/1751349344 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f94c4016950 con 0x7f94cc1605d0 2024-09-17T05:37:17.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.065+0000 7f94d1800640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f94ac074ac0 0x7f94ac076f80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009690 tx=0x7f94c0006c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:17.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.217+0000 7f94c9600640 1 -- 172.21.15.122:0/1751349344 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7f94c40f5e10 con 0x7f94cc1605d0 2024-09-17T05:37:17.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.377+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f94cc0d1080 con 0x7f94ac074ac0 2024-09-17T05:37:17.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94c9600640 1 -- 172.21.15.122:0/1751349344 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f94cc0d1080 con 0x7f94ac074ac0 2024-09-17T05:37:17.449 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "in-progress", 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_report": { 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "percentage cloned": "23.958%", 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "amount cloned": "736M/3.0G", 2024-09-17T05:37:17.450 INFO:teuthology.orchestra.run.smithi122.stdout: "files cloned": "4/4" 2024-09-17T05:37:17.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:17.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:17.451 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f94ac074ac0 msgr2=0x7f94ac076f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f94ac074ac0 0x7f94ac076f80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009690 tx=0x7f94c0006c00 comp rx=0 tx=0).stop 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc1605d0 msgr2=0x7f94cc1f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc1605d0 0x7f94cc1f71e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f94c401b410 tx=0x7f94c40a5b50 comp rx=0 tx=0).stop 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 shutdown_connections 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f94ac074ac0 0x7f94ac076f80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc1605d0 0x7f94cc1f71e0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94cc0b1650 0x7f94cc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 --2- 172.21.15.122:0/1751349344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94cc0aeb00 0x7f94cc1f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 >> 172.21.15.122:0/1751349344 conn(0x7f94cc0d1dd0 msgr2=0x7f94cc072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:17.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 shutdown_connections 2024-09-17T05:37:17.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:17.453+0000 7f94d4200640 1 -- 172.21.15.122:0/1751349344 wait complete. 2024-09-17T05:37:17.479 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 23.958% 2024-09-17T05:37:18.480 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T05:37:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 -- 172.21.15.122:0/304015460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a806d330 msgr2=0x7fc2a80ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 --2- 172.21.15.122:0/304015460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a806d330 0x7fc2a80ce790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009bc0 tx=0x7fc2a001cba0 comp rx=0 tx=0).stop 2024-09-17T05:37:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 -- 172.21.15.122:0/304015460 shutdown_connections 2024-09-17T05:37:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 --2- 172.21.15.122:0/304015460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a806d330 0x7fc2a80ce790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 --2- 172.21.15.122:0/304015460 >> v2:172.21.15.173:3300/0 conn(0x7fc2a80d0740 0x7fc2a80cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 -- 172.21.15.122:0/304015460 >> 172.21.15.122:0/304015460 conn(0x7fc2a80d0bc0 msgr2=0x7fc2a80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 -- 172.21.15.122:0/304015460 shutdown_connections 2024-09-17T05:37:18.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 -- 172.21.15.122:0/304015460 wait complete. 2024-09-17T05:37:18.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 Processor -- start 2024-09-17T05:37:18.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.693+0000 7fc2aec00640 1 -- start start 2024-09-17T05:37:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2aec00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a806d330 0x7fc2a81f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2aec00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a80d0740 0x7fc2a81552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2aec00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a80d16e0 0x7fc2a8154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2aec00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a816af60 con 0x7fc2a80d16e0 2024-09-17T05:37:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2aec00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a816ade0 con 0x7fc2a80d0740 2024-09-17T05:37:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2aec00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a816b0e0 con 0x7fc2a806d330 2024-09-17T05:37:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a80d16e0 0x7fc2a8154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a80d16e0 0x7fc2a8154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59418/0 (socket says 172.21.15.122:59418) 2024-09-17T05:37:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 -- 172.21.15.122:0/2061336707 learned_addr learned my addr 172.21.15.122:0/2061336707 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 -- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a806d330 msgr2=0x7fc2a81f7870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:37:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a806d330 0x7fc2a81f7870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 -- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a80d0740 msgr2=0x7fc2a81552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a80d0740 0x7fc2a81552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 -- 172.21.15.122:0/2061336707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a81559c0 con 0x7fc2a80d16e0 2024-09-17T05:37:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2ae200640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a80d16e0 0x7fc2a8154c40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc2a001a7d0 tx=0x7fc2a00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2a5600640 1 -- 172.21.15.122:0/2061336707 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00bb070 con 0x7fc2a80d16e0 2024-09-17T05:37:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2a5600640 1 -- 172.21.15.122:0/2061336707 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2a0004c00 con 0x7fc2a80d16e0 2024-09-17T05:37:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2a5600640 1 -- 172.21.15.122:0/2061336707 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00b2040 con 0x7fc2a80d16e0 2024-09-17T05:37:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a8155ca0 con 0x7fc2a80d16e0 2024-09-17T05:37:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.697+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a8156260 con 0x7fc2a80d16e0 2024-09-17T05:37:18.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.701+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a81f8840 con 0x7fc2a80d16e0 2024-09-17T05:37:18.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.701+0000 7fc2a5600640 1 -- 172.21.15.122:0/2061336707 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7fc2a00a5ae0 con 0x7fc2a80d16e0 2024-09-17T05:37:18.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.705+0000 7fc2a5600640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc288074b40 0x7fc288077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:18.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.705+0000 7fc2a5600640 1 -- 172.21.15.122:0/2061336707 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fc2a012ccc0 con 0x7fc2a80d16e0 2024-09-17T05:37:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.709+0000 7fc2ad800640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc288074b40 0x7fc288077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:18.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.709+0000 7fc2a5600640 1 -- 172.21.15.122:0/2061336707 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc2a0016790 con 0x7fc2a80d16e0 2024-09-17T05:37:18.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:18.713+0000 7fc2ad800640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc288074b40 0x7fc288077000 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc294009a70 tx=0x7fc294009260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:18.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.001+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fc2a80ce790 con 0x7fc288074b40 2024-09-17T05:37:19.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.021+0000 7fc2a5600640 1 -- 172.21.15.122:0/2061336707 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7fc2a80ce790 con 0x7fc288074b40 2024-09-17T05:37:19.017 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "in-progress", 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_report": { 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "percentage cloned": "66.667%", 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "amount cloned": "2.0G/3.0G", 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: "files cloned": "5/4" 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:19.018 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc288074b40 msgr2=0x7fc288077000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc288074b40 0x7fc288077000 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc294009a70 tx=0x7fc294009260 comp rx=0 tx=0).stop 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a80d16e0 msgr2=0x7fc2a8154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a80d16e0 0x7fc2a8154c40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc2a001a7d0 tx=0x7fc2a00092c0 comp rx=0 tx=0).stop 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 shutdown_connections 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc288074b40 0x7fc288077000 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a80d16e0 0x7fc2a8154c40 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a80d0740 0x7fc2a81552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 --2- 172.21.15.122:0/2061336707 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a806d330 0x7fc2a81f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 >> 172.21.15.122:0/2061336707 conn(0x7fc2a80d0bc0 msgr2=0x7fc2a80cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 shutdown_connections 2024-09-17T05:37:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:19.025+0000 7fc2aec00640 1 -- 172.21.15.122:0/2061336707 wait complete. 2024-09-17T05:37:19.042 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 66.667% 2024-09-17T05:37:20.043 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T05:37:20.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.269+0000 7f50bbe00640 1 -- 172.21.15.122:0/3032244096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a0004810 msgr2=0x7f50a0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:20.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.269+0000 7f50bbe00640 1 --2- 172.21.15.122:0/3032244096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a0004810 0x7f50a0004c90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f50a8006c20 tx=0x7f50a801f1b0 comp rx=0 tx=0).stop 2024-09-17T05:37:20.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.269+0000 7f50bb400640 1 -- 172.21.15.122:0/3032244096 reap_dead start 2024-09-17T05:37:20.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.269+0000 7f50bbe00640 1 -- 172.21.15.122:0/3032244096 shutdown_connections 2024-09-17T05:37:20.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.269+0000 7f50bbe00640 1 -- 172.21.15.122:0/3032244096 >> 172.21.15.122:0/3032244096 conn(0x7f50bc0b2680 msgr2=0x7f50bc0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:20.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.269+0000 7f50bbe00640 1 -- 172.21.15.122:0/3032244096 shutdown_connections 2024-09-17T05:37:20.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.269+0000 7f50bbe00640 1 -- 172.21.15.122:0/3032244096 wait complete. 2024-09-17T05:37:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.273+0000 7f50bbe00640 1 Processor -- start 2024-09-17T05:37:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.273+0000 7f50bbe00640 1 -- start start 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.273+0000 7f50bbe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50a0004810 0x7f50bc0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.273+0000 7f50bbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50bc0ca220 0x7f50bc0ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.273+0000 7f50bbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50bc150870 0x7f50bc150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.273+0000 7f50bbe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc06cfb0 con 0x7f50bc150870 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.273+0000 7f50bbe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc06ce30 con 0x7f50bc0ca220 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.273+0000 7f50bbe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f50bc06d130 con 0x7f50a0004810 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50bc150870 0x7f50bc150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50bc150870 0x7f50bc150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59438/0 (socket says 172.21.15.122:59438) 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 -- 172.21.15.122:0/4005596666 learned_addr learned my addr 172.21.15.122:0/4005596666 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50baa00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50a0004810 0x7f50bc0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:20.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 -- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50a0004810 msgr2=0x7f50bc0aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:20.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50a0004810 0x7f50bc0aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:20.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 -- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50bc0ca220 msgr2=0x7f50bc0ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:20.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50bc0ca220 0x7f50bc0ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 -- 172.21.15.122:0/4005596666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc0cabe0 con 0x7f50bc150870 2024-09-17T05:37:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50baa00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50a0004810 0x7f50bc0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bb400640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50bc150870 0x7f50bc150d00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f50a0004790 tx=0x7f50b400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50b2a00640 1 -- 172.21.15.122:0/4005596666 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4016070 con 0x7f50bc150870 2024-09-17T05:37:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50b2a00640 1 -- 172.21.15.122:0/4005596666 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50b400bb50 con 0x7f50bc150870 2024-09-17T05:37:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc0af180 con 0x7f50bc150870 2024-09-17T05:37:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50b2a00640 1 -- 172.21.15.122:0/4005596666 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4010460 con 0x7f50bc150870 2024-09-17T05:37:20.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc0ade80 con 0x7f50bc150870 2024-09-17T05:37:20.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50bc15e880 con 0x7f50bc150870 2024-09-17T05:37:20.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.277+0000 7f50b2a00640 1 -- 172.21.15.122:0/4005596666 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7f50b4019070 con 0x7f50bc150870 2024-09-17T05:37:20.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.281+0000 7f50b2a00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f509c074b50 0x7f509c077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:20.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.281+0000 7f50b2a00640 1 -- 172.21.15.122:0/4005596666 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f50b40932e0 con 0x7f50bc150870 2024-09-17T05:37:20.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.281+0000 7f50baa00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f509c074b50 0x7f509c077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:20.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.289+0000 7f50baa00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f509c074b50 0x7f509c077010 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0b6bf0 tx=0x7f50ac005d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:20.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.289+0000 7f50b2a00640 1 -- 172.21.15.122:0/4005596666 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f50bc0cabe0 con 0x7f50bc150870 2024-09-17T05:37:20.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.573+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f50bc15eb90 con 0x7f509c074b50 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.645+0000 7f50b2a00640 1 -- 172.21.15.122:0/4005596666 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f50bc15eb90 con 0x7f509c074b50 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "in-progress", 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:37:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_report": { 2024-09-17T05:37:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "percentage cloned": "90.625%", 2024-09-17T05:37:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "amount cloned": "2.7G/3.0G", 2024-09-17T05:37:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "files cloned": "6/4" 2024-09-17T05:37:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:20.643 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f509c074b50 msgr2=0x7f509c077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f509c074b50 0x7f509c077010 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0b6bf0 tx=0x7f50ac005d10 comp rx=0 tx=0).stop 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50bc150870 msgr2=0x7f50bc150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50bc150870 0x7f50bc150d00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f50a0004790 tx=0x7f50b400db60 comp rx=0 tx=0).stop 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 shutdown_connections 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f509c074b50 0x7f509c077010 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50bc150870 0x7f50bc150d00 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50bc0ca220 0x7f50bc0ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 --2- 172.21.15.122:0/4005596666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50a0004810 0x7f50bc0aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 >> 172.21.15.122:0/4005596666 conn(0x7f50bc0b2680 msgr2=0x7f50bc15d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:20.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 shutdown_connections 2024-09-17T05:37:20.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:20.649+0000 7f50bbe00640 1 -- 172.21.15.122:0/4005596666 wait complete. 2024-09-17T05:37:20.673 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 90.625% 2024-09-17T05:37:21.674 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T05:37:21.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.825+0000 7fe7fa600640 1 -- 172.21.15.122:0/1773703165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f40b0d10 msgr2=0x7fe7f40d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:21.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.825+0000 7fe7fa600640 1 --2- 172.21.15.122:0/1773703165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f40b0d10 0x7fe7f40d1bb0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40cf780 tx=0x7fe7e001c800 comp rx=0 tx=0).stop 2024-09-17T05:37:21.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.825+0000 7fe7fa600640 1 -- 172.21.15.122:0/1773703165 shutdown_connections 2024-09-17T05:37:21.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.825+0000 7fe7fa600640 1 --2- 172.21.15.122:0/1773703165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f40b0d10 0x7fe7f40d1bb0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:21.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.825+0000 7fe7fa600640 1 --2- 172.21.15.122:0/1773703165 >> v2:172.21.15.173:3300/0 conn(0x7fe7f415bd40 0x7fe7f40d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:21.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.825+0000 7fe7fa600640 1 -- 172.21.15.122:0/1773703165 >> 172.21.15.122:0/1773703165 conn(0x7fe7f409aa10 msgr2=0x7fe7f409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:21.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.825+0000 7fe7fa600640 1 -- 172.21.15.122:0/1773703165 shutdown_connections 2024-09-17T05:37:21.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.825+0000 7fe7fa600640 1 -- 172.21.15.122:0/1773703165 wait complete. 2024-09-17T05:37:21.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 Processor -- start 2024-09-17T05:37:21.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 -- start start 2024-09-17T05:37:21.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7f40b0d10 0x7fe7f414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:21.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f40d20f0 0x7fe7f4158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:21.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f415bd40 0x7fe7f414ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:21.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4168cb0 con 0x7fe7f415bd40 2024-09-17T05:37:21.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4168b30 con 0x7fe7f40d20f0 2024-09-17T05:37:21.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f4168e30 con 0x7fe7f40b0d10 2024-09-17T05:37:21.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f415bd40 0x7fe7f414ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:21.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f415bd40 0x7fe7f414ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51248/0 (socket says 172.21.15.122:51248) 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 -- 172.21.15.122:0/2483600570 learned_addr learned my addr 172.21.15.122:0/2483600570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f3e00640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f40d20f0 0x7fe7f4158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9200640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7f40b0d10 0x7fe7f414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 -- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7f40b0d10 msgr2=0x7fe7f414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7f40b0d10 0x7fe7f414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 -- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f40d20f0 msgr2=0x7fe7f4158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f40d20f0 0x7fe7f4158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 -- 172.21.15.122:0/2483600570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f4155170 con 0x7fe7f415bd40 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f9c00640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f415bd40 0x7fe7f414ee40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c410 tx=0x7fe7ec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f0c00640 1 -- 172.21.15.122:0/2483600570 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec011790 con 0x7fe7f415bd40 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f0c00640 1 -- 172.21.15.122:0/2483600570 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7ec017070 con 0x7fe7f415bd40 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f4155430 con 0x7fe7f415bd40 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7f0c00640 1 -- 172.21.15.122:0/2483600570 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec011cc0 con 0x7fe7f415bd40 2024-09-17T05:37:21.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f41559f0 con 0x7fe7f415bd40 2024-09-17T05:37:21.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.829+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f40d1bb0 con 0x7fe7f415bd40 2024-09-17T05:37:21.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.833+0000 7fe7f0c00640 1 -- 172.21.15.122:0/2483600570 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7fe7ec01c460 con 0x7fe7f415bd40 2024-09-17T05:37:21.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.833+0000 7fe7f0c00640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7d4074b50 0x7fe7d4077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:21.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.833+0000 7fe7f0c00640 1 -- 172.21.15.122:0/2483600570 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fe7ec092d60 con 0x7fe7f415bd40 2024-09-17T05:37:21.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.833+0000 7fe7f9200640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7d4074b50 0x7fe7d4077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:21.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.837+0000 7fe7f9200640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7d4074b50 0x7fe7d4077010 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009eb0 tx=0x7fe7e4009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:21.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:21.837+0000 7fe7f0c00640 1 -- 172.21.15.122:0/2483600570 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe7ec05f910 con 0x7fe7f415bd40 2024-09-17T05:37:22.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:22.041+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fe7f4069e30 con 0x7fe7d4074b50 2024-09-17T05:37:23.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.477+0000 7fe7f0c00640 1 -- 172.21.15.122:0/2483600570 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fe7e4009b00 con 0x7fe7d4074b50 2024-09-17T05:37:23.475 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:23.475 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:37:23.475 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "complete" 2024-09-17T05:37:23.475 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:23.475 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7d4074b50 msgr2=0x7fe7d4077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7d4074b50 0x7fe7d4077010 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009eb0 tx=0x7fe7e4009500 comp rx=0 tx=0).stop 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f415bd40 msgr2=0x7fe7f414ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f415bd40 0x7fe7f414ee40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c410 tx=0x7fe7ec00c8d0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 shutdown_connections 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe7d4074b50 0x7fe7d4077010 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7f415bd40 0x7fe7f414ee40 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f40d20f0 0x7fe7f4158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 --2- 172.21.15.122:0/2483600570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7f40b0d10 0x7fe7f414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 >> 172.21.15.122:0/2483600570 conn(0x7fe7f409aa10 msgr2=0x7fe7f415b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:23.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 shutdown_connections 2024-09-17T05:37:23.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.481+0000 7fe7fa600640 1 -- 172.21.15.122:0/2483600570 wait complete. 2024-09-17T05:37:23.501 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T05:37:23.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.653+0000 7f6030a00640 1 -- 172.21.15.122:0/2533429654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6010004d50 msgr2=0x7f60100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:23.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.653+0000 7f6030a00640 1 --2- 172.21.15.122:0/2533429654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6010004d50 0x7f60100051d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f60240078d0 tx=0x7f602401db20 comp rx=0 tx=0).stop 2024-09-17T05:37:23.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.653+0000 7f602a800640 1 -- 172.21.15.122:0/2533429654 reap_dead start 2024-09-17T05:37:23.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.653+0000 7f6030a00640 1 -- 172.21.15.122:0/2533429654 shutdown_connections 2024-09-17T05:37:23.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.653+0000 7f6030a00640 1 -- 172.21.15.122:0/2533429654 >> 172.21.15.122:0/2533429654 conn(0x7f602c0b18f0 msgr2=0x7f602c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:23.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.653+0000 7f6030a00640 1 -- 172.21.15.122:0/2533429654 shutdown_connections 2024-09-17T05:37:23.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.653+0000 7f6030a00640 1 -- 172.21.15.122:0/2533429654 wait complete. 2024-09-17T05:37:23.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 Processor -- start 2024-09-17T05:37:23.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 -- start start 2024-09-17T05:37:23.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010004d50 0x7f602c1f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:23.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f602c0b0f90 0x7f602c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f602c152060 0x7f602c1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f602c0b4ec0 con 0x7f602c152060 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f602c0b4d40 con 0x7f6010004d50 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f602c070960 con 0x7f602c0b0f90 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f602c152060 0x7f602c1524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f602c152060 0x7f602c1524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51270/0 (socket says 172.21.15.122:51270) 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 -- 172.21.15.122:0/451566562 learned_addr learned my addr 172.21.15.122:0/451566562 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6029e00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010004d50 0x7f602c1f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6021400640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f602c0b0f90 0x7f602c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 -- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f602c0b0f90 msgr2=0x7f602c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f602c0b0f90 0x7f602c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 -- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010004d50 msgr2=0x7f602c1f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:23.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010004d50 0x7f602c1f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 -- 172.21.15.122:0/451566562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c15f720 con 0x7f602c152060 2024-09-17T05:37:23.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6021400640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f602c0b0f90 0x7f602c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:23.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f602a800640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f602c152060 0x7f602c1524f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f602400a040 tx=0x7f6024003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:23.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6022a00640 1 -- 172.21.15.122:0/451566562 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60240bc070 con 0x7f602c152060 2024-09-17T05:37:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6022a00640 1 -- 172.21.15.122:0/451566562 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60240a6810 con 0x7f602c152060 2024-09-17T05:37:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6022a00640 1 -- 172.21.15.122:0/451566562 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024007550 con 0x7f602c152060 2024-09-17T05:37:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602c15fa00 con 0x7f602c152060 2024-09-17T05:37:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602c0b1410 con 0x7f602c152060 2024-09-17T05:37:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.657+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602c069410 con 0x7f602c152060 2024-09-17T05:37:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.661+0000 7f6022a00640 1 -- 172.21.15.122:0/451566562 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7f60240a6c00 con 0x7f602c152060 2024-09-17T05:37:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.661+0000 7f6022a00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6008074b80 0x7f6008077040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:23.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.661+0000 7f6022a00640 1 -- 172.21.15.122:0/451566562 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f602412c860 con 0x7f602c152060 2024-09-17T05:37:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.661+0000 7f6029e00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6008074b80 0x7f6008077040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.661+0000 7f6029e00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6008074b80 0x7f6008077040 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f601c000c10 tx=0x7f601c006b80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:23.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.665+0000 7f6022a00640 1 -- 172.21.15.122:0/451566562 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6024017830 con 0x7f602c152060 2024-09-17T05:37:23.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.869+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f602c06fca0 con 0x7f6008074b80 2024-09-17T05:37:23.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6022a00640 1 -- 172.21.15.122:0/451566562 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f602c06fca0 con 0x7f6008074b80 2024-09-17T05:37:23.871 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:37:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "complete" 2024-09-17T05:37:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:23.872 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6008074b80 msgr2=0x7f6008077040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6008074b80 0x7f6008077040 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f601c000c10 tx=0x7f601c006b80 comp rx=0 tx=0).stop 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f602c152060 msgr2=0x7f602c1524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f602c152060 0x7f602c1524f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f602400a040 tx=0x7f6024003bb0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 shutdown_connections 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6008074b80 0x7f6008077040 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f602c152060 0x7f602c1524f0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f602c0b0f90 0x7f602c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 --2- 172.21.15.122:0/451566562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6010004d50 0x7f602c1f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:23.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 >> 172.21.15.122:0/451566562 conn(0x7f602c0b18f0 msgr2=0x7f602c1533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:23.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 shutdown_connections 2024-09-17T05:37:23.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:23.877+0000 7f6030a00640 1 -- 172.21.15.122:0/451566562 wait complete. 2024-09-17T05:37:23.892 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-17T05:37:23.893 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:37:24.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3796231659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 msgr2=0x7ffbe40d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3796231659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 0x7ffbe40d01b0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40c56a0 tx=0x7ffbd401ca60 comp rx=0 tx=0).stop 2024-09-17T05:37:24.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3796231659 shutdown_connections 2024-09-17T05:37:24.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3796231659 >> v2:172.21.15.173:3300/0 conn(0x7ffbe40cefb0 0x7ffbe40cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3796231659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 0x7ffbe40d01b0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3796231659 >> 172.21.15.122:0/3796231659 conn(0x7ffbe409aa10 msgr2=0x7ffbe409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:24.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3796231659 shutdown_connections 2024-09-17T05:37:24.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3796231659 wait complete. 2024-09-17T05:37:24.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 Processor -- start 2024-09-17T05:37:24.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- start start 2024-09-17T05:37:24.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbe40ccaf0 0x7ffbe414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe40cefb0 0x7ffbe40b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 0x7ffbe40ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41692e0 con 0x7ffbe415f000 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4169160 con 0x7ffbe40cefb0 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe8e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe4169460 con 0x7ffbe40ccaf0 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 0x7ffbe40ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 0x7ffbe40ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51282/0 (socket says 172.21.15.122:51282) 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3e00640 1 -- 172.21.15.122:0/3728021649 learned_addr learned my addr 172.21.15.122:0/3728021649 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbdaa00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe40cefb0 0x7ffbe40b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3400640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbe40ccaf0 0x7ffbe414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3e00640 1 -- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbe40ccaf0 msgr2=0x7ffbe414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbe40ccaf0 0x7ffbe414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3e00640 1 -- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe40cefb0 msgr2=0x7ffbe40b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe40cefb0 0x7ffbe40b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.041+0000 7ffbe3e00640 1 -- 172.21.15.122:0/3728021649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe40b5950 con 0x7ffbe415f000 2024-09-17T05:37:24.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbe3e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 0x7ffbe40ccf60 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00c3c0 tx=0x7ffbdc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:24.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbdbe00640 1 -- 172.21.15.122:0/3728021649 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc017070 con 0x7ffbe415f000 2024-09-17T05:37:24.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbdbe00640 1 -- 172.21.15.122:0/3728021649 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbdc006da0 con 0x7ffbe415f000 2024-09-17T05:37:24.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe40b5bf0 con 0x7ffbe415f000 2024-09-17T05:37:24.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbdbe00640 1 -- 172.21.15.122:0/3728021649 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0072b0 con 0x7ffbe415f000 2024-09-17T05:37:24.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe40b61b0 con 0x7ffbe415f000 2024-09-17T05:37:24.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe40b6970 con 0x7ffbe415f000 2024-09-17T05:37:24.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbdbe00640 1 -- 172.21.15.122:0/3728021649 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7ffbdc01a070 con 0x7ffbe415f000 2024-09-17T05:37:24.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbdbe00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbc0074b80 0x7ffbc0077040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbdbe00640 1 -- 172.21.15.122:0/3728021649 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7ffbdc092380 con 0x7ffbe415f000 2024-09-17T05:37:24.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.045+0000 7ffbe3400640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbc0074b80 0x7ffbc0077040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.049+0000 7ffbe3400640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbc0074b80 0x7ffbc0077040 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40af000 tx=0x7ffbd401b2a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:24.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.049+0000 7ffbdbe00640 1 -- 172.21.15.122:0/3728021649 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ffbe40b6970 con 0x7ffbe415f000 2024-09-17T05:37:24.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.241+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ffbe40cc5b0 con 0x7ffbc0074b80 2024-09-17T05:37:24.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.241+0000 7ffbdbe00640 1 -- 172.21.15.122:0/3728021649 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ffbe40cc5b0 con 0x7ffbc0074b80 2024-09-17T05:37:24.238 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:37:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:37:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:24.239 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbc0074b80 msgr2=0x7ffbc0077040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbc0074b80 0x7ffbc0077040 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40af000 tx=0x7ffbd401b2a0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 msgr2=0x7ffbe40ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 0x7ffbe40ccf60 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00c3c0 tx=0x7ffbdc00c880 comp rx=0 tx=0).stop 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 shutdown_connections 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ffbc0074b80 0x7ffbc0077040 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbe415f000 0x7ffbe40ccf60 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe40cefb0 0x7ffbe40b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 --2- 172.21.15.122:0/3728021649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbe40ccaf0 0x7ffbe414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 >> 172.21.15.122:0/3728021649 conn(0x7ffbe409aa10 msgr2=0x7ffbe406eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:24.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 shutdown_connections 2024-09-17T05:37:24.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.245+0000 7ffbe8e00640 1 -- 172.21.15.122:0/3728021649 wait complete. 2024-09-17T05:37:24.261 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:37:24.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 -- 172.21.15.122:0/2448607893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 msgr2=0x7f2c0c15e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 --2- 172.21.15.122:0/2448607893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 0x7f2c0c15e7f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009c80 tx=0x7f2bfc01cd10 comp rx=0 tx=0).stop 2024-09-17T05:37:24.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 -- 172.21.15.122:0/2448607893 shutdown_connections 2024-09-17T05:37:24.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 --2- 172.21.15.122:0/2448607893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0c0af220 0x7f2c0c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 --2- 172.21.15.122:0/2448607893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 0x7f2c0c15e7f0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 -- 172.21.15.122:0/2448607893 >> 172.21.15.122:0/2448607893 conn(0x7f2c0c0cfc20 msgr2=0x7f2c0c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:24.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 -- 172.21.15.122:0/2448607893 shutdown_connections 2024-09-17T05:37:24.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 -- 172.21.15.122:0/2448607893 wait complete. 2024-09-17T05:37:24.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 Processor -- start 2024-09-17T05:37:24.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.409+0000 7f2c12600640 1 -- start start 2024-09-17T05:37:24.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 0x7f2c0c147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0c0af220 0x7f2c0c09d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0c0cddd0 0x7f2c0c147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c1683c0 con 0x7f2c0c06fea0 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c168240 con 0x7f2c0c0af220 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0c168540 con 0x7f2c0c0cddd0 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 0x7f2c0c147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 0x7f2c0c147ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51300/0 (socket says 172.21.15.122:51300) 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 -- 172.21.15.122:0/1926604567 learned_addr learned my addr 172.21.15.122:0/1926604567 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c02a00640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0c0af220 0x7f2c0c09d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0be00640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0c0cddd0 0x7f2c0c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 -- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0c0cddd0 msgr2=0x7f2c0c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0c0cddd0 0x7f2c0c147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 -- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0c0af220 msgr2=0x7f2c0c09d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0c0af220 0x7f2c0c09d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 -- 172.21.15.122:0/1926604567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c147bc0 con 0x7f2c0c06fea0 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c02a00640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0c0af220 0x7f2c0c09d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c0b400640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 0x7f2c0c147ed0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc000f90 tx=0x7f2bfc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c03e00640 1 -- 172.21.15.122:0/1926604567 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0bb070 con 0x7f2c0c06fea0 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c03e00640 1 -- 172.21.15.122:0/1926604567 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bfc004c50 con 0x7f2c0c06fea0 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c15b6b0 con 0x7f2c0c06fea0 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c03e00640 1 -- 172.21.15.122:0/1926604567 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0b2040 con 0x7f2c0c06fea0 2024-09-17T05:37:24.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c15bc50 con 0x7f2c0c06fea0 2024-09-17T05:37:24.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c0ca270 con 0x7f2c0c06fea0 2024-09-17T05:37:24.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.413+0000 7f2c03e00640 1 -- 172.21.15.122:0/1926604567 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7f2bfc0be070 con 0x7f2c0c06fea0 2024-09-17T05:37:24.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.417+0000 7f2c03e00640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2be8074b50 0x7f2be8077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.417+0000 7f2c03e00640 1 -- 172.21.15.122:0/1926604567 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f2bfc12cfd0 con 0x7f2c0c06fea0 2024-09-17T05:37:24.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.417+0000 7f2c02a00640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2be8074b50 0x7f2be8077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.417+0000 7f2c02a00640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2be8074b50 0x7f2be8077010 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c14af80 tx=0x7f2bf4005ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:24.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.421+0000 7f2c03e00640 1 -- 172.21.15.122:0/1926604567 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2bfc016990 con 0x7f2c0c06fea0 2024-09-17T05:37:24.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.613+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c0c09ad60 con 0x7f2be8074b50 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c03e00640 1 -- 172.21.15.122:0/1926604567 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f2c0c09ad60 con 0x7f2be8074b50 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sv1" 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1clone1" 2024-09-17T05:37:24.617 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:24.618 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2be8074b50 msgr2=0x7f2be8077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2be8074b50 0x7f2be8077010 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c14af80 tx=0x7f2bf4005ed0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 msgr2=0x7f2c0c147ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 0x7f2c0c147ed0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc000f90 tx=0x7f2bfc009360 comp rx=0 tx=0).stop 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 shutdown_connections 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2be8074b50 0x7f2be8077010 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0c0cddd0 0x7f2c0c147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0c0af220 0x7f2c0c09d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 --2- 172.21.15.122:0/1926604567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c0c06fea0 0x7f2c0c147ed0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 >> 172.21.15.122:0/1926604567 conn(0x7f2c0c0cfc20 msgr2=0x7f2c0c0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:24.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 shutdown_connections 2024-09-17T05:37:24.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.621+0000 7f2c12600640 1 -- 172.21.15.122:0/1926604567 wait complete. 2024-09-17T05:37:24.638 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T05:37:24.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- 172.21.15.122:0/3017773457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd98c004d50 msgr2=0x7fd98c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 --2- 172.21.15.122:0/3017773457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd98c004d50 0x7fd98c0051d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00c640 tx=0x7fd99c01f9e0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 -- 172.21.15.122:0/3017773457 reap_dead start 2024-09-17T05:37:24.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- 172.21.15.122:0/3017773457 shutdown_connections 2024-09-17T05:37:24.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- 172.21.15.122:0/3017773457 >> 172.21.15.122:0/3017773457 conn(0x7fd9a406dd30 msgr2=0x7fd9a406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:24.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- 172.21.15.122:0/3017773457 shutdown_connections 2024-09-17T05:37:24.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- 172.21.15.122:0/3017773457 wait complete. 2024-09-17T05:37:24.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 Processor -- start 2024-09-17T05:37:24.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- start start 2024-09-17T05:37:24.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd98c004d50 0x7fd9a40b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9a40b0510 0x7fd9a41f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9a4144310 0x7fd9a4144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4074140 con 0x7fd9a4144310 2024-09-17T05:37:24.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4073fc0 con 0x7fd98c004d50 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9ac600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a4070330 con 0x7fd9a40b0510 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9a4144310 0x7fd9a4144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9a4144310 0x7fd9a4144790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51310/0 (socket says 172.21.15.122:51310) 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 -- 172.21.15.122:0/3671401915 learned_addr learned my addr 172.21.15.122:0/3671401915 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9a9c00640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd98c004d50 0x7fd9a40b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9a9200640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9a40b0510 0x7fd9a41f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 -- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9a40b0510 msgr2=0x7fd9a41f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9a40b0510 0x7fd9a41f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 -- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd98c004d50 msgr2=0x7fd9a40b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd98c004d50 0x7fd9a40b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.789+0000 7fd9aa600640 1 -- 172.21.15.122:0/3671401915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a40b2020 con 0x7fd9a4144310 2024-09-17T05:37:24.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9aa600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9a4144310 0x7fd9a4144790 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd99c020280 tx=0x7fd99c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:24.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9a1600640 1 -- 172.21.15.122:0/3671401915 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0011d0 con 0x7fd9a4144310 2024-09-17T05:37:24.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9a1600640 1 -- 172.21.15.122:0/3671401915 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd99c001880 con 0x7fd9a4144310 2024-09-17T05:37:24.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9a1600640 1 -- 172.21.15.122:0/3671401915 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c001fc0 con 0x7fd9a4144310 2024-09-17T05:37:24.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a40b0a40 con 0x7fd9a4144310 2024-09-17T05:37:24.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a40b1000 con 0x7fd9a4144310 2024-09-17T05:37:24.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a41f8c10 con 0x7fd9a4144310 2024-09-17T05:37:24.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9a1600640 1 -- 172.21.15.122:0/3671401915 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7fd99c0bb030 con 0x7fd9a4144310 2024-09-17T05:37:24.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9a1600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd984074b80 0x7fd984077040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:24.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9a1600640 1 -- 172.21.15.122:0/3671401915 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fd99c12f9a0 con 0x7fd9a4144310 2024-09-17T05:37:24.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.793+0000 7fd9a9c00640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd984074b80 0x7fd984077040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:24.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.797+0000 7fd9a9c00640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd984074b80 0x7fd984077040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd998000c10 tx=0x7fd998006b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:24.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.797+0000 7fd9a1600640 1 -- 172.21.15.122:0/3671401915 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd99c019790 con 0x7fd9a4144310 2024-09-17T05:37:24.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:24.993+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9a40d1310 con 0x7fd984074b80 2024-09-17T05:37:25.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.009+0000 7fd9a1600640 1 -- 172.21.15.122:0/3671401915 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd9a40d1310 con 0x7fd984074b80 2024-09-17T05:37:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:25.006 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:37:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1" 2024-09-17T05:37:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:25.007 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:37:25.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd984074b80 msgr2=0x7fd984077040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd984074b80 0x7fd984077040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd998000c10 tx=0x7fd998006b20 comp rx=0 tx=0).stop 2024-09-17T05:37:25.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9a4144310 msgr2=0x7fd9a4144790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9a4144310 0x7fd9a4144790 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd99c020280 tx=0x7fd99c004ef0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 shutdown_connections 2024-09-17T05:37:25.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd984074b80 0x7fd984077040 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9a4144310 0x7fd9a4144790 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9a40b0510 0x7fd9a41f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 --2- 172.21.15.122:0/3671401915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd98c004d50 0x7fd9a40b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 >> 172.21.15.122:0/3671401915 conn(0x7fd9a406dd30 msgr2=0x7fd9a415c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:25.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 shutdown_connections 2024-09-17T05:37:25.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.013+0000 7fd9ac600640 1 -- 172.21.15.122:0/3671401915 wait complete. 2024-09-17T05:37:25.028 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T05:37:25.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- 172.21.15.122:0/3244767880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c80b49e0 msgr2=0x7fb2c80acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 --2- 172.21.15.122:0/3244767880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c80b49e0 0x7fb2c80acd40 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80708e0 tx=0x7fb2c001ca60 comp rx=0 tx=0).stop 2024-09-17T05:37:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- 172.21.15.122:0/3244767880 shutdown_connections 2024-09-17T05:37:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 --2- 172.21.15.122:0/3244767880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c80b49e0 0x7fb2c80acd40 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 --2- 172.21.15.122:0/3244767880 >> v2:172.21.15.173:3300/0 conn(0x7fb2c80cd100 0x7fb2c806f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- 172.21.15.122:0/3244767880 >> 172.21.15.122:0/3244767880 conn(0x7fb2c80b18f0 msgr2=0x7fb2c80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- 172.21.15.122:0/3244767880 shutdown_connections 2024-09-17T05:37:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- 172.21.15.122:0/3244767880 wait complete. 2024-09-17T05:37:25.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 Processor -- start 2024-09-17T05:37:25.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- start start 2024-09-17T05:37:25.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2c80b49e0 0x7fb2c80b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2c80cd100 0x7fb2c81536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c815dae0 0x7fb2c8153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80cf170 con 0x7fb2c815dae0 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80ceff0 con 0x7fb2c80cd100 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cf400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c80cf2f0 con 0x7fb2c80b49e0 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c815dae0 0x7fb2c8153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cd400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c815dae0 0x7fb2c8153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51330/0 (socket says 172.21.15.122:51330) 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.181+0000 7fb2cd400640 1 -- 172.21.15.122:0/844119186 learned_addr learned my addr 172.21.15.122:0/844119186 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cd400640 1 -- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2c80b49e0 msgr2=0x7fb2c80b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cca00640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2c80b49e0 0x7fb2c80b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2c7e00640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2c80cd100 0x7fb2c81536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cd400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2c80b49e0 0x7fb2c80b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cd400640 1 -- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2c80cd100 msgr2=0x7fb2c81536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cd400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2c80cd100 0x7fb2c81536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cd400640 1 -- 172.21.15.122:0/844119186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c81f8d50 con 0x7fb2c815dae0 2024-09-17T05:37:25.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cca00640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2c80b49e0 0x7fb2c80b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:25.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cd400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c815dae0 0x7fb2c8153be0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009b10 tx=0x7fb2c00a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:25.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2c4c00640 1 -- 172.21.15.122:0/844119186 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00bb070 con 0x7fb2c815dae0 2024-09-17T05:37:25.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2c4c00640 1 -- 172.21.15.122:0/844119186 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2c0004d40 con 0x7fb2c815dae0 2024-09-17T05:37:25.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2c4c00640 1 -- 172.21.15.122:0/844119186 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00b2040 con 0x7fb2c815dae0 2024-09-17T05:37:25.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c81f9030 con 0x7fb2c815dae0 2024-09-17T05:37:25.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81f95f0 con 0x7fb2c815dae0 2024-09-17T05:37:25.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c8069410 con 0x7fb2c815dae0 2024-09-17T05:37:25.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2c4c00640 1 -- 172.21.15.122:0/844119186 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7fb2c00a5930 con 0x7fb2c815dae0 2024-09-17T05:37:25.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.185+0000 7fb2c4c00640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2a8074b50 0x7fb2a8077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.189+0000 7fb2c4c00640 1 -- 172.21.15.122:0/844119186 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fb2c012cfb0 con 0x7fb2c815dae0 2024-09-17T05:37:25.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.189+0000 7fb2cca00640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2a8074b50 0x7fb2a8077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.189+0000 7fb2cca00640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2a8074b50 0x7fb2a8077010 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8000f70 tx=0x7fb2b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:25.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.189+0000 7fb2c4c00640 1 -- 172.21.15.122:0/844119186 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb2c0016770 con 0x7fb2c815dae0 2024-09-17T05:37:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.389+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2c80ae7f0 con 0x7fb2a8074b50 2024-09-17T05:37:25.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.417+0000 7fb2c4c00640 1 -- 172.21.15.122:0/844119186 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb2c80ae7f0 con 0x7fb2a8074b50 2024-09-17T05:37:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2a8074b50 msgr2=0x7fb2a8077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2a8074b50 0x7fb2a8077010 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8000f70 tx=0x7fb2b8009040 comp rx=0 tx=0).stop 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c815dae0 msgr2=0x7fb2c8153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c815dae0 0x7fb2c8153be0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009b10 tx=0x7fb2c00a57c0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 shutdown_connections 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb2a8074b50 0x7fb2a8077010 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2c815dae0 0x7fb2c8153be0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2c80cd100 0x7fb2c81536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 --2- 172.21.15.122:0/844119186 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2c80b49e0 0x7fb2c80b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 >> 172.21.15.122:0/844119186 conn(0x7fb2c80b18f0 msgr2=0x7fb2c809d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 shutdown_connections 2024-09-17T05:37:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.421+0000 7fb2cf400640 1 -- 172.21.15.122:0/844119186 wait complete. 2024-09-17T05:37:25.437 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T05:37:25.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.585+0000 7f18cb800640 1 -- 172.21.15.122:0/3840306762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c40b1650 msgr2=0x7f18c40b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.585+0000 7f18cb800640 1 --2- 172.21.15.122:0/3840306762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c40b1650 0x7f18c40b1a30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009840 tx=0x7f18ac01c820 comp rx=0 tx=0).stop 2024-09-17T05:37:25.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.585+0000 7f18cb800640 1 -- 172.21.15.122:0/3840306762 shutdown_connections 2024-09-17T05:37:25.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.585+0000 7f18cb800640 1 --2- 172.21.15.122:0/3840306762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c40afb10 0x7f18c40affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.585+0000 7f18cb800640 1 --2- 172.21.15.122:0/3840306762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c40b1650 0x7f18c40b1a30 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.585+0000 7f18cb800640 1 -- 172.21.15.122:0/3840306762 >> 172.21.15.122:0/3840306762 conn(0x7f18c40d1dd0 msgr2=0x7f18c40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:25.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.585+0000 7f18cb800640 1 -- 172.21.15.122:0/3840306762 shutdown_connections 2024-09-17T05:37:25.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.585+0000 7f18cb800640 1 -- 172.21.15.122:0/3840306762 wait complete. 2024-09-17T05:37:25.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 Processor -- start 2024-09-17T05:37:25.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 -- start start 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c40afb10 0x7f18c414b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c40b1650 0x7f18c40c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c41605d0 0x7f18c40c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18c40b0e60 con 0x7f18c41605d0 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f18c40b1430 con 0x7f18c40afb10 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f18c4169540 con 0x7f18c40b1650 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c41605d0 0x7f18c40c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c41605d0 0x7f18c40c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51346/0 (socket says 172.21.15.122:51346) 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 -- 172.21.15.122:0/1893515888 learned_addr learned my addr 172.21.15.122:0/1893515888 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 -- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c40b1650 msgr2=0x7f18c40c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c40b1650 0x7f18c40c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 -- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c40afb10 msgr2=0x7f18c414b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c40afb10 0x7f18c414b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 -- 172.21.15.122:0/1893515888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c40c77d0 con 0x7f18c41605d0 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c9800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c41605d0 0x7f18c40c71c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00eb50 tx=0x7f18bc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c1600640 1 -- 172.21.15.122:0/1893515888 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc004d30 con 0x7f18c41605d0 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c1600640 1 -- 172.21.15.122:0/1893515888 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18bc00ee50 con 0x7f18c41605d0 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c40c7ab0 con 0x7f18c41605d0 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18c1600640 1 -- 172.21.15.122:0/1893515888 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc005280 con 0x7f18c41605d0 2024-09-17T05:37:25.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.589+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c40c8070 con 0x7f18c41605d0 2024-09-17T05:37:25.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.593+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c40affc0 con 0x7f18c41605d0 2024-09-17T05:37:25.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.593+0000 7f18c1600640 1 -- 172.21.15.122:0/1893515888 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7f18bc005420 con 0x7f18c41605d0 2024-09-17T05:37:25.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.593+0000 7f18c1600640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18a0074b50 0x7f18a0077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.593+0000 7f18c1600640 1 -- 172.21.15.122:0/1893515888 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f18bc093e00 con 0x7f18c41605d0 2024-09-17T05:37:25.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.593+0000 7f18c8e00640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18a0074b50 0x7f18a0077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.593+0000 7f18c8e00640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18a0074b50 0x7f18a0077010 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f18b40046c0 tx=0x7f18b4007910 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.597+0000 7f18c1600640 1 -- 172.21.15.122:0/1893515888 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f18bc060910 con 0x7f18c41605d0 2024-09-17T05:37:25.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.789+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f18c40d1080 con 0x7f18a0074b50 2024-09-17T05:37:25.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.809+0000 7f18c1600640 1 -- 172.21.15.122:0/1893515888 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f18c40d1080 con 0x7f18a0074b50 2024-09-17T05:37:25.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18a0074b50 msgr2=0x7f18a0077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18a0074b50 0x7f18a0077010 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f18b40046c0 tx=0x7f18b4007910 comp rx=0 tx=0).stop 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c41605d0 msgr2=0x7f18c40c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c41605d0 0x7f18c40c71c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00eb50 tx=0x7f18bc00c6a0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 shutdown_connections 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18a0074b50 0x7f18a0077010 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18c41605d0 0x7f18c40c71c0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c40b1650 0x7f18c40c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 --2- 172.21.15.122:0/1893515888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c40afb10 0x7f18c414b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 >> 172.21.15.122:0/1893515888 conn(0x7f18c40d1dd0 msgr2=0x7f18c4070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 shutdown_connections 2024-09-17T05:37:25.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.813+0000 7f18cb800640 1 -- 172.21.15.122:0/1893515888 wait complete. 2024-09-17T05:37:25.820 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:25.823+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:25.820 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:25.823+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:25.820 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:25.823+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:25.820 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:25.823+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:25.820 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:25.823+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:25.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-17T05:37:25.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 -- 172.21.15.122:0/3502932248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376806cc20 msgr2=0x7f376806d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 --2- 172.21.15.122:0/3502932248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376806cc20 0x7f376806d090 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f37680d1da0 tx=0x7f376001cd60 comp rx=0 tx=0).stop 2024-09-17T05:37:25.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 -- 172.21.15.122:0/3502932248 shutdown_connections 2024-09-17T05:37:25.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 --2- 172.21.15.122:0/3502932248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376806cc20 0x7f376806d090 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 --2- 172.21.15.122:0/3502932248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f376815c120 0x7f376815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 --2- 172.21.15.122:0/3502932248 >> v2:172.21.15.173:3300/0 conn(0x7f376815bd40 0x7f3768073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 -- 172.21.15.122:0/3502932248 >> 172.21.15.122:0/3502932248 conn(0x7f376806dd30 msgr2=0x7f376806e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:25.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 -- 172.21.15.122:0/3502932248 shutdown_connections 2024-09-17T05:37:25.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.981+0000 7f376f200640 1 -- 172.21.15.122:0/3502932248 wait complete. 2024-09-17T05:37:25.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 Processor -- start 2024-09-17T05:37:25.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 -- start start 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376806cc20 0x7f376814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f376815bd40 0x7f3768160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376815c120 0x7f376815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37680b41d0 con 0x7f376815c120 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37680b4050 con 0x7f376806cc20 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f37680b4350 con 0x7f376815bd40 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376815c120 0x7f376815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376815c120 0x7f376815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51352/0 (socket says 172.21.15.122:51352) 2024-09-17T05:37:25.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 -- 172.21.15.122:0/1214946384 learned_addr learned my addr 172.21.15.122:0/1214946384 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f3767e00640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376806cc20 0x7f376814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 -- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f376815bd40 msgr2=0x7f3768160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f376815bd40 0x7f3768160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 -- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376806cc20 msgr2=0x7f376814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376806cc20 0x7f376814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 -- 172.21.15.122:0/1214946384 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376814ac40 con 0x7f376815c120 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f3767e00640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376806cc20 0x7f376814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376d200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376815c120 0x7f376815a680 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f376001d600 tx=0x7f37600093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f3764c00640 1 -- 172.21.15.122:0/1214946384 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37600bb070 con 0x7f376815c120 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f3764c00640 1 -- 172.21.15.122:0/1214946384 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3760004d60 con 0x7f376815c120 2024-09-17T05:37:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3768160e50 con 0x7f376815c120 2024-09-17T05:37:25.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f3764c00640 1 -- 172.21.15.122:0/1214946384 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37600b2040 con 0x7f376815c120 2024-09-17T05:37:25.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.985+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681fb270 con 0x7f376815c120 2024-09-17T05:37:25.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.989+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37680b47d0 con 0x7f376815c120 2024-09-17T05:37:25.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.989+0000 7f3764c00640 1 -- 172.21.15.122:0/1214946384 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7f37600be070 con 0x7f376815c120 2024-09-17T05:37:25.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.989+0000 7f3764c00640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3744074b50 0x7f3744077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:25.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.989+0000 7f3764c00640 1 -- 172.21.15.122:0/1214946384 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f376012cf00 con 0x7f376815c120 2024-09-17T05:37:25.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.989+0000 7f3767e00640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3744074b50 0x7f3744077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:25.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.993+0000 7f3767e00640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3744074b50 0x7f3744077010 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3758005fd0 tx=0x7f3758005da0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:25.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:25.997+0000 7f3764c00640 1 -- 172.21.15.122:0/1214946384 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3760016950 con 0x7f376815c120 2024-09-17T05:37:26.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.201+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f37680b5620 con 0x7f3744074b50 2024-09-17T05:37:26.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f3764c00640 1 -- 172.21.15.122:0/1214946384 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f37680b5620 con 0x7f3744074b50 2024-09-17T05:37:26.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3744074b50 msgr2=0x7f3744077010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3744074b50 0x7f3744077010 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3758005fd0 tx=0x7f3758005da0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376815c120 msgr2=0x7f376815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376815c120 0x7f376815a680 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f376001d600 tx=0x7f37600093c0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 shutdown_connections 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3744074b50 0x7f3744077010 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f376815c120 0x7f376815a680 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f376815bd40 0x7f3768160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 --2- 172.21.15.122:0/1214946384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376806cc20 0x7f376814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 >> 172.21.15.122:0/1214946384 conn(0x7f376806dd30 msgr2=0x7f376815d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 shutdown_connections 2024-09-17T05:37:26.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.221+0000 7f376f200640 1 -- 172.21.15.122:0/1214946384 wait complete. 2024-09-17T05:37:26.229 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:26.231+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:26.229 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:26.231+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:26.229 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:26.231+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:26.230 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:26.231+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:26.230 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:26.231+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:26.241 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:37:26.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:37:26.543 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:37:26.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:37:26.550 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:37:26.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.697+0000 7f1310a00640 1 -- 172.21.15.122:0/2337372175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 msgr2=0x7f12f0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:26.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.697+0000 7f1310a00640 1 --2- 172.21.15.122:0/2337372175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 0x7f12f0004c90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fa0 tx=0x7f12f801cbc0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.697+0000 7f130a800640 1 -- 172.21.15.122:0/2337372175 reap_dead start 2024-09-17T05:37:26.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.697+0000 7f1310a00640 1 -- 172.21.15.122:0/2337372175 shutdown_connections 2024-09-17T05:37:26.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.697+0000 7f1310a00640 1 -- 172.21.15.122:0/2337372175 >> 172.21.15.122:0/2337372175 conn(0x7f130c0d1ea0 msgr2=0x7f130c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:26.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.697+0000 7f1310a00640 1 -- 172.21.15.122:0/2337372175 shutdown_connections 2024-09-17T05:37:26.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.697+0000 7f1310a00640 1 -- 172.21.15.122:0/2337372175 wait complete. 2024-09-17T05:37:26.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 Processor -- start 2024-09-17T05:37:26.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 -- start start 2024-09-17T05:37:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 0x7f130c1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c09f2c0 0x7f130c159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f130c164b70 0x7f130c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f130c167e50 con 0x7f12f0004810 2024-09-17T05:37:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f130c167cd0 con 0x7f130c09f2c0 2024-09-17T05:37:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f130c167fd0 con 0x7f130c164b70 2024-09-17T05:37:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 0x7f130c1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 0x7f130c1554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51372/0 (socket says 172.21.15.122:51372) 2024-09-17T05:37:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 -- 172.21.15.122:0/2295492625 learned_addr learned my addr 172.21.15.122:0/2295492625 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 -- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f130c164b70 msgr2=0x7f130c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309400640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c09f2c0 0x7f130c159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f130a800640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f130c164b70 0x7f130c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f130c164b70 0x7f130c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 -- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c09f2c0 msgr2=0x7f130c159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c09f2c0 0x7f130c159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 -- 172.21.15.122:0/2295492625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c155a30 con 0x7f12f0004810 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309400640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c09f2c0 0x7f130c159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f130a800640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f130c164b70 0x7f130c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1309e00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 0x7f130c1554f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f12fc003e90 tx=0x7f12fc00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1302000640 1 -- 172.21.15.122:0/2295492625 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc011640 con 0x7f12f0004810 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1302000640 1 -- 172.21.15.122:0/2295492625 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12fc016020 con 0x7f12f0004810 2024-09-17T05:37:26.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c0cb400 con 0x7f12f0004810 2024-09-17T05:37:26.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1302000640 1 -- 172.21.15.122:0/2295492625 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc011aa0 con 0x7f12f0004810 2024-09-17T05:37:26.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.701+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c0cb6a0 con 0x7f12f0004810 2024-09-17T05:37:26.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.705+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130c0cc940 con 0x7f12f0004810 2024-09-17T05:37:26.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.705+0000 7f1302000640 1 -- 172.21.15.122:0/2295492625 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 42) ==== 99529+0+0 (secure 0 0 0) 0x7f12fc011c40 con 0x7f12f0004810 2024-09-17T05:37:26.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.705+0000 7f1302000640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12ec074b80 0x7f12ec077040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:26.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.705+0000 7f1302000640 1 -- 172.21.15.122:0/2295492625 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f12fc0944a0 con 0x7f12f0004810 2024-09-17T05:37:26.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.705+0000 7f1309400640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12ec074b80 0x7f12ec077040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:26.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.705+0000 7f1309400640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12ec074b80 0x7f12ec077040 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f12f80ac000 tx=0x7f12f8019d20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:26.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.709+0000 7f1302000640 1 -- 172.21.15.122:0/2295492625 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f12fc020020 con 0x7f12f0004810 2024-09-17T05:37:26.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f130c0d2710 con 0x7f12f0004810 2024-09-17T05:37:26.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1302000640 1 -- 172.21.15.122:0/2295492625 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7f12fc061050 con 0x7f12f0004810 2024-09-17T05:37:26.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12ec074b80 msgr2=0x7f12ec077040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:26.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12ec074b80 0x7f12ec077040 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f12f80ac000 tx=0x7f12f8019d20 comp rx=0 tx=0).stop 2024-09-17T05:37:26.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 msgr2=0x7f130c1554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:26.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 0x7f130c1554f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f12fc003e90 tx=0x7f12fc00f720 comp rx=0 tx=0).stop 2024-09-17T05:37:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 shutdown_connections 2024-09-17T05:37:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12ec074b80 0x7f12ec077040 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f130c164b70 0x7f130c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f130c09f2c0 0x7f130c159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 --2- 172.21.15.122:0/2295492625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12f0004810 0x7f130c1554f0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 >> 172.21.15.122:0/2295492625 conn(0x7f130c0d1ea0 msgr2=0x7f130c0719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 shutdown_connections 2024-09-17T05:37:26.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:26.881+0000 7f1310a00640 1 -- 172.21.15.122:0/2295492625 wait complete. 2024-09-17T05:37:26.897 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:37:26.898 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:37:26.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:37:27.087 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:37:27.087 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:37:27.087 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:37:27.087 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:27.096 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:37:27.096 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:27.148 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:37:27.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.341+0000 7f1949a00640 1 -- 172.21.15.122:0/3770630614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 msgr2=0x7f19440acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:27.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.341+0000 7f1949a00640 1 --2- 172.21.15.122:0/3770630614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 0x7f19440acd40 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f193400a950 tx=0x7f193401da50 comp rx=0 tx=0).stop 2024-09-17T05:37:27.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.341+0000 7f1949a00640 1 -- 172.21.15.122:0/3770630614 shutdown_connections 2024-09-17T05:37:27.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.341+0000 7f1949a00640 1 --2- 172.21.15.122:0/3770630614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 0x7f19440acd40 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.341+0000 7f1949a00640 1 -- 172.21.15.122:0/3770630614 >> 172.21.15.122:0/3770630614 conn(0x7f19440b18f0 msgr2=0x7f19440b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:27.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.341+0000 7f1949a00640 1 -- 172.21.15.122:0/3770630614 shutdown_connections 2024-09-17T05:37:27.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.341+0000 7f1949a00640 1 -- 172.21.15.122:0/3770630614 wait complete. 2024-09-17T05:37:27.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.341+0000 7f1949a00640 1 Processor -- start 2024-09-17T05:37:27.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 -- start start 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19440b49e0 0x7f19440b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19440cd100 0x7f1944152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 0x7f19440cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1944168570 con 0x7f194415dae0 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f19441683f0 con 0x7f19440b49e0 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f19441686f0 con 0x7f19440cd100 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 0x7f19440cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 0x7f19440cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51386/0 (socket says 172.21.15.122:51386) 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 -- 172.21.15.122:0/2706674441 learned_addr learned my addr 172.21.15.122:0/2706674441 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1942a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19440b49e0 0x7f19440b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:27.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f193a000640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19440cd100 0x7f1944152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 -- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19440cd100 msgr2=0x7f1944152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19440cd100 0x7f1944152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 -- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19440b49e0 msgr2=0x7f19440b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19440b49e0 0x7f19440b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 -- 172.21.15.122:0/2706674441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19440c9bd0 con 0x7f194415dae0 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1942a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19440b49e0 0x7f19440b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f193a000640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19440cd100 0x7f1944152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1943400640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 0x7f19440cafe0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c730 tx=0x7f193c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f193b400640 1 -- 172.21.15.122:0/2706674441 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c003db0 con 0x7f194415dae0 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f193b400640 1 -- 172.21.15.122:0/2706674441 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f193c00ce40 con 0x7f194415dae0 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19440c9eb0 con 0x7f194415dae0 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f193b400640 1 -- 172.21.15.122:0/2706674441 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0052d0 con 0x7f194415dae0 2024-09-17T05:37:27.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1944152bc0 con 0x7f194415dae0 2024-09-17T05:37:27.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.345+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1944069410 con 0x7f194415dae0 2024-09-17T05:37:27.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.349+0000 7f193b400640 1 -- 172.21.15.122:0/2706674441 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f193c005470 con 0x7f194415dae0 2024-09-17T05:37:27.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.349+0000 7f193b400640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1920074c70 0x7f1920077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:27.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.349+0000 7f193b400640 1 -- 172.21.15.122:0/2706674441 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f193c092e40 con 0x7f194415dae0 2024-09-17T05:37:27.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.349+0000 7f1942a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1920074c70 0x7f1920077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:27.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.349+0000 7f1942a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1920074c70 0x7f1920077130 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f19340b0000 tx=0x7f193401c290 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:27.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.353+0000 7f193b400640 1 -- 172.21.15.122:0/2706674441 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f193c05f9f0 con 0x7f194415dae0 2024-09-17T05:37:27.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.533+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f19440acd40 con 0x7f194415dae0 2024-09-17T05:37:27.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.533+0000 7f193b400640 1 -- 172.21.15.122:0/2706674441 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7f193c05f2f0 con 0x7f194415dae0 2024-09-17T05:37:27.530 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:37:27.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1920074c70 msgr2=0x7f1920077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:27.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1920074c70 0x7f1920077130 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f19340b0000 tx=0x7f193401c290 comp rx=0 tx=0).stop 2024-09-17T05:37:27.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 msgr2=0x7f19440cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 0x7f19440cafe0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c730 tx=0x7f193c00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 shutdown_connections 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1920074c70 0x7f1920077130 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f194415dae0 0x7f19440cafe0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19440cd100 0x7f1944152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 --2- 172.21.15.122:0/2706674441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19440b49e0 0x7f19440b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 >> 172.21.15.122:0/2706674441 conn(0x7f19440b18f0 msgr2=0x7f19440ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 shutdown_connections 2024-09-17T05:37:27.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.537+0000 7f1949a00640 1 -- 172.21.15.122:0/2706674441 wait complete. 2024-09-17T05:37:27.551 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:37:27.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:27.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:37:27.598 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:37:27.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:27.598 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:37:27.648 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:37:27.648 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:37:27.648 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:27.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:37:27.657 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:37:27.657 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:37:27.657 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:27.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:37:27.699 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:37:27.699 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:37:27.699 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:27.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:37:27.708 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:37:27.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49f800640 1 -- 172.21.15.122:0/2371436984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa47c0039f0 msgr2=0x7fa47c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:27.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49f800640 1 --2- 172.21.15.122:0/2371436984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa47c0039f0 0x7fa47c003dd0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa488006f90 tx=0x7fa48801cf00 comp rx=0 tx=0).stop 2024-09-17T05:37:27.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49d800640 1 -- 172.21.15.122:0/2371436984 reap_dead start 2024-09-17T05:37:27.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49f800640 1 -- 172.21.15.122:0/2371436984 shutdown_connections 2024-09-17T05:37:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49f800640 1 -- 172.21.15.122:0/2371436984 >> 172.21.15.122:0/2371436984 conn(0x7fa4980b18f0 msgr2=0x7fa4980b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49f800640 1 -- 172.21.15.122:0/2371436984 shutdown_connections 2024-09-17T05:37:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49f800640 1 -- 172.21.15.122:0/2371436984 wait complete. 2024-09-17T05:37:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49f800640 1 Processor -- start 2024-09-17T05:37:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.893+0000 7fa49f800640 1 -- start start 2024-09-17T05:37:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa47c0039f0 0x7fa4981f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4981516e0 0x7fa49815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa49809cd50 0x7fa49809d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa498166370 con 0x7fa49809cd50 2024-09-17T05:37:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4981661f0 con 0x7fa4981516e0 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4981664f0 con 0x7fa47c0039f0 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa49809cd50 0x7fa49809d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa49809cd50 0x7fa49809d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51402/0 (socket says 172.21.15.122:51402) 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 -- 172.21.15.122:0/998245597 learned_addr learned my addr 172.21.15.122:0/998245597 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4981516e0 0x7fa49815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49ce00640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa47c0039f0 0x7fa4981f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 -- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa47c0039f0 msgr2=0x7fa4981f9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa47c0039f0 0x7fa4981f9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 -- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4981516e0 msgr2=0x7fa49815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4981516e0 0x7fa49815ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 -- 172.21.15.122:0/998245597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49809d740 con 0x7fa49809cd50 2024-09-17T05:37:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49d800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa49809cd50 0x7fa49809d200 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa498072470 tx=0x7fa490006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa495600640 1 -- 172.21.15.122:0/998245597 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490015070 con 0x7fa49809cd50 2024-09-17T05:37:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa495600640 1 -- 172.21.15.122:0/998245597 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa49000fa80 con 0x7fa49809cd50 2024-09-17T05:37:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4981f6150 con 0x7fa49809cd50 2024-09-17T05:37:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa495600640 1 -- 172.21.15.122:0/998245597 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490011430 con 0x7fa49809cd50 2024-09-17T05:37:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4981f6710 con 0x7fa49809cd50 2024-09-17T05:37:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa498069410 con 0x7fa49809cd50 2024-09-17T05:37:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.897+0000 7fa495600640 1 -- 172.21.15.122:0/998245597 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fa4900115d0 con 0x7fa49809cd50 2024-09-17T05:37:27.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.901+0000 7fa495600640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa474074c70 0x7fa474077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.901+0000 7fa495600640 1 -- 172.21.15.122:0/998245597 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fa490093d20 con 0x7fa49809cd50 2024-09-17T05:37:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.901+0000 7fa49ce00640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa474074c70 0x7fa474077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.901+0000 7fa49ce00640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa474074c70 0x7fa474077130 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa4880aa000 tx=0x7fa4880b1040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:27.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:27.905+0000 7fa495600640 1 -- 172.21.15.122:0/998245597 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa4900601e0 con 0x7fa49809cd50 2024-09-17T05:37:28.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.157+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4980d0580 con 0x7fa49809cd50 2024-09-17T05:37:28.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.157+0000 7fa495600640 1 -- 172.21.15.122:0/998245597 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5237 (secure 0 0 0) 0x7fa490064c10 con 0x7fa49809cd50 2024-09-17T05:37:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:28.154 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":67,"btime":"2024-09-17T05:36:41:054240+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1758247367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1758247367},{"type":"v1","addr":"172.21.15.173:6836","nonce":1758247367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2422841556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2422841556},{"type":"v1","addr":"172.21.15.173:6837","nonce":2422841556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/4290616350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":4290616350},{"type":"v1","addr":"172.21.15.122:6834","nonce":4290616350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:36:32.388428+0000","modified":"2024-09-17T05:36:35.423704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2223381402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":2223381402},{"type":"v1","addr":"172.21.15.122:6835","nonce":2223381402}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T05:37:28.157 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 67 2024-09-17T05:37:28.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa474074c70 msgr2=0x7fa474077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa474074c70 0x7fa474077130 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa4880aa000 tx=0x7fa4880b1040 comp rx=0 tx=0).stop 2024-09-17T05:37:28.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa49809cd50 msgr2=0x7fa49809d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa49809cd50 0x7fa49809d200 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa498072470 tx=0x7fa490006660 comp rx=0 tx=0).stop 2024-09-17T05:37:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 shutdown_connections 2024-09-17T05:37:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa474074c70 0x7fa474077130 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa49809cd50 0x7fa49809d200 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4981516e0 0x7fa49815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 --2- 172.21.15.122:0/998245597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa47c0039f0 0x7fa4981f9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 >> 172.21.15.122:0/998245597 conn(0x7fa4980b18f0 msgr2=0x7fa498158460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.161+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 shutdown_connections 2024-09-17T05:37:28.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.165+0000 7fa49f800640 1 -- 172.21.15.122:0/998245597 wait complete. 2024-09-17T05:37:28.191 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:37:28.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- 172.21.15.122:0/3742457826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc300ced80 msgr2=0x7fdc300ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 --2- 172.21.15.122:0/3742457826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc300ced80 0x7fdc300ae9b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009840 tx=0x7fdc1c01c820 comp rx=0 tx=0).stop 2024-09-17T05:37:28.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- 172.21.15.122:0/3742457826 shutdown_connections 2024-09-17T05:37:28.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 --2- 172.21.15.122:0/3742457826 >> v2:172.21.15.173:3300/0 conn(0x7fdc300cdb30 0x7fdc300aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 --2- 172.21.15.122:0/3742457826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc300ced80 0x7fdc300ae9b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- 172.21.15.122:0/3742457826 >> 172.21.15.122:0/3742457826 conn(0x7fdc300d1dc0 msgr2=0x7fdc300d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:28.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- 172.21.15.122:0/3742457826 shutdown_connections 2024-09-17T05:37:28.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- 172.21.15.122:0/3742457826 wait complete. 2024-09-17T05:37:28.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 Processor -- start 2024-09-17T05:37:28.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- start start 2024-09-17T05:37:28.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc300cdb30 0x7fdc300b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:28.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc300ced80 0x7fdc300b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc3015bea0 0x7fdc301ff580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc301692f0 con 0x7fdc300cdb30 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30169170 con 0x7fdc300ced80 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.441+0000 7fdc38200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdc30169470 con 0x7fdc3015bea0 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc35800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc300cdb30 0x7fdc300b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc36200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc3015bea0 0x7fdc301ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc34e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc300ced80 0x7fdc300b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc36200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc3015bea0 0x7fdc301ff580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:42482/0 (socket says 172.21.15.122:42482) 2024-09-17T05:37:28.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc36200640 1 -- 172.21.15.122:0/224186683 learned_addr learned my addr 172.21.15.122:0/224186683 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:28.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc34e00640 1 -- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc3015bea0 msgr2=0x7fdc301ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc34e00640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc3015bea0 0x7fdc301ff580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc34e00640 1 -- 172.21.15.122:0/224186683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc300cdb30 msgr2=0x7fdc300b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc34e00640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc300cdb30 0x7fdc300b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc34e00640 1 -- 172.21.15.122:0/224186683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc300c57c0 con 0x7fdc300ced80 2024-09-17T05:37:28.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc35800640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc300cdb30 0x7fdc300b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:37:28.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc34e00640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc300ced80 0x7fdc300b27b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0ac000 tx=0x7fdc1c005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:28.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc2d600640 1 -- 172.21.15.122:0/224186683 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0b2040 con 0x7fdc300ced80 2024-09-17T05:37:28.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc2d600640 1 -- 172.21.15.122:0/224186683 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc1c0042f0 con 0x7fdc300ced80 2024-09-17T05:37:28.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc300c5aa0 con 0x7fdc300ced80 2024-09-17T05:37:28.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc2d600640 1 -- 172.21.15.122:0/224186683 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0052d0 con 0x7fdc300ced80 2024-09-17T05:37:28.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc300c6060 con 0x7fdc300ced80 2024-09-17T05:37:28.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc300c5550 con 0x7fdc300ced80 2024-09-17T05:37:28.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.445+0000 7fdc2d600640 1 -- 172.21.15.122:0/224186683 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fdc1c0a56b0 con 0x7fdc300ced80 2024-09-17T05:37:28.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.449+0000 7fdc2d600640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc10074c70 0x7fdc10077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:28.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.449+0000 7fdc2d600640 1 -- 172.21.15.122:0/224186683 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fdc1c12ccb0 con 0x7fdc300ced80 2024-09-17T05:37:28.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.449+0000 7fdc35800640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc10074c70 0x7fdc10077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:28.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.449+0000 7fdc35800640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc10074c70 0x7fdc10077130 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdc24007f30 tx=0x7fdc24007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:28.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.453+0000 7fdc2d600640 1 -- 172.21.15.122:0/224186683 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdc1c016410 con 0x7fdc300ced80 2024-09-17T05:37:28.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.661+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc300ce670 con 0x7fdc300ced80 2024-09-17T05:37:28.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.661+0000 7fdc2d600640 1 -- 172.21.15.122:0/224186683 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5237 (secure 0 0 0) 0x7fdc1c0f9170 con 0x7fdc300ced80 2024-09-17T05:37:28.658 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:28.658 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":67,"btime":"2024-09-17T05:36:41:054240+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1758247367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1758247367},{"type":"v1","addr":"172.21.15.173:6836","nonce":1758247367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2422841556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2422841556},{"type":"v1","addr":"172.21.15.173:6837","nonce":2422841556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/4290616350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":4290616350},{"type":"v1","addr":"172.21.15.122:6834","nonce":4290616350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:36:32.388428+0000","modified":"2024-09-17T05:36:35.423704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2223381402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":2223381402},{"type":"v1","addr":"172.21.15.122:6835","nonce":2223381402}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T05:37:28.659 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 67 2024-09-17T05:37:28.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc10074c70 msgr2=0x7fdc10077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc10074c70 0x7fdc10077130 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdc24007f30 tx=0x7fdc24007ec0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc300ced80 msgr2=0x7fdc300b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc300ced80 0x7fdc300b27b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0ac000 tx=0x7fdc1c005d60 comp rx=0 tx=0).stop 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 shutdown_connections 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdc10074c70 0x7fdc10077130 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdc3015bea0 0x7fdc301ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdc300ced80 0x7fdc300b27b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 --2- 172.21.15.122:0/224186683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc300cdb30 0x7fdc300b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 >> 172.21.15.122:0/224186683 conn(0x7fdc300d1dc0 msgr2=0x7fdc300b7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 shutdown_connections 2024-09-17T05:37:28.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.665+0000 7fdc38200640 1 -- 172.21.15.122:0/224186683 wait complete. 2024-09-17T05:37:28.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:37:28.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.829+0000 7fd37f200640 1 -- 172.21.15.122:0/1815062852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3800b11a0 msgr2=0x7fd380165670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.829+0000 7fd37f200640 1 --2- 172.21.15.122:0/1815062852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3800b11a0 0x7fd380165670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd378009840 tx=0x7fd37801c740 comp rx=0 tx=0).stop 2024-09-17T05:37:28.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.829+0000 7fd37f200640 1 -- 172.21.15.122:0/1815062852 shutdown_connections 2024-09-17T05:37:28.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.829+0000 7fd37f200640 1 --2- 172.21.15.122:0/1815062852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3800b11a0 0x7fd380165670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.829+0000 7fd37f200640 1 --2- 172.21.15.122:0/1815062852 >> v2:172.21.15.173:3300/0 conn(0x7fd3800b0ce0 0x7fd3800b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.829+0000 7fd37f200640 1 -- 172.21.15.122:0/1815062852 >> 172.21.15.122:0/1815062852 conn(0x7fd38009aa10 msgr2=0x7fd38009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:28.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.829+0000 7fd37f200640 1 -- 172.21.15.122:0/1815062852 shutdown_connections 2024-09-17T05:37:28.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.829+0000 7fd37f200640 1 -- 172.21.15.122:0/1815062852 wait complete. 2024-09-17T05:37:28.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 Processor -- start 2024-09-17T05:37:28.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 -- start start 2024-09-17T05:37:28.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3800b0ce0 0x7fd3801436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:28.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3800b11a0 0x7fd3801533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:28.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3800b4d80 0x7fd3801538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:28.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd380168960 con 0x7fd3800b0ce0 2024-09-17T05:37:28.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801687e0 con 0x7fd3800b11a0 2024-09-17T05:37:28.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd380168ae0 con 0x7fd3800b4d80 2024-09-17T05:37:28.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3800b0ce0 0x7fd3801436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3800b0ce0 0x7fd3801436a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51422/0 (socket says 172.21.15.122:51422) 2024-09-17T05:37:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 -- 172.21.15.122:0/3341621958 learned_addr learned my addr 172.21.15.122:0/3341621958 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37e800640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3800b4d80 0x7fd3801538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37d400640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3800b11a0 0x7fd3801533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 -- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3800b4d80 msgr2=0x7fd3801538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3800b4d80 0x7fd3801538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 -- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3800b11a0 msgr2=0x7fd3801533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:28.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3800b11a0 0x7fd3801533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:28.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 -- 172.21.15.122:0/3341621958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd380151500 con 0x7fd3800b0ce0 2024-09-17T05:37:28.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37de00640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3800b0ce0 0x7fd3801436a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd37000cc50 tx=0x7fd37000cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:28.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd376000640 1 -- 172.21.15.122:0/3341621958 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370016020 con 0x7fd3800b0ce0 2024-09-17T05:37:28.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd376000640 1 -- 172.21.15.122:0/3341621958 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd37000ab60 con 0x7fd3800b0ce0 2024-09-17T05:37:28.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd376000640 1 -- 172.21.15.122:0/3341621958 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370012420 con 0x7fd3800b0ce0 2024-09-17T05:37:28.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3801517e0 con 0x7fd3800b0ce0 2024-09-17T05:37:28.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.833+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd380151da0 con 0x7fd3800b0ce0 2024-09-17T05:37:28.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.837+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3800b5160 con 0x7fd3800b0ce0 2024-09-17T05:37:28.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.837+0000 7fd376000640 1 -- 172.21.15.122:0/3341621958 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fd370019020 con 0x7fd3800b0ce0 2024-09-17T05:37:28.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.837+0000 7fd376000640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd360074c70 0x7fd360077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:28.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.837+0000 7fd376000640 1 -- 172.21.15.122:0/3341621958 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fd370093370 con 0x7fd3800b0ce0 2024-09-17T05:37:28.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.837+0000 7fd37d400640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd360074c70 0x7fd360077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:28.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.837+0000 7fd37d400640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd360074c70 0x7fd360077130 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd380159bd0 tx=0x7fd36c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:28.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:28.841+0000 7fd376000640 1 -- 172.21.15.122:0/3341621958 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd370020020 con 0x7fd3800b0ce0 2024-09-17T05:37:29.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.009+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3801656f0 con 0x7fd3800b0ce0 2024-09-17T05:37:29.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.009+0000 7fd376000640 1 -- 172.21.15.122:0/3341621958 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16922 (secure 0 0 0) 0x7fd37005f830 con 0x7fd3800b0ce0 2024-09-17T05:37:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:29.006 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":105,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:36:48.081624+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:36:30.080602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:36:31.349633+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:37:29.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd360074c70 msgr2=0x7fd360077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd360074c70 0x7fd360077130 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd380159bd0 tx=0x7fd36c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3800b0ce0 msgr2=0x7fd3801436a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3800b0ce0 0x7fd3801436a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd37000cc50 tx=0x7fd37000cd40 comp rx=0 tx=0).stop 2024-09-17T05:37:29.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 shutdown_connections 2024-09-17T05:37:29.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd360074c70 0x7fd360077130 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3800b4d80 0x7fd3801538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3800b11a0 0x7fd3801533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 --2- 172.21.15.122:0/3341621958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3800b0ce0 0x7fd3801436a0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 >> 172.21.15.122:0/3341621958 conn(0x7fd38009aa10 msgr2=0x7fd38015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:29.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 shutdown_connections 2024-09-17T05:37:29.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.013+0000 7fd37f200640 1 -- 172.21.15.122:0/3341621958 wait complete. 2024-09-17T05:37:29.030 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:37:29.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:37:29.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.253+0000 7f6e6f800640 1 -- 172.21.15.122:0/459430174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68073880 msgr2=0x7f6e68073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.253+0000 7f6e6f800640 1 --2- 172.21.15.122:0/459430174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68073880 0x7f6e68073ce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6e680ae900 tx=0x7f6e5801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.253+0000 7f6e6f800640 1 -- 172.21.15.122:0/459430174 shutdown_connections 2024-09-17T05:37:29.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.253+0000 7f6e6f800640 1 --2- 172.21.15.122:0/459430174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68073880 0x7f6e68073ce0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.253+0000 7f6e6f800640 1 --2- 172.21.15.122:0/459430174 >> v2:172.21.15.122:3300/0 conn(0x7f6e680cc4c0 0x7f6e680cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.253+0000 7f6e6f800640 1 -- 172.21.15.122:0/459430174 >> 172.21.15.122:0/459430174 conn(0x7f6e680d0d50 msgr2=0x7f6e680d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:29.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.253+0000 7f6e6f800640 1 -- 172.21.15.122:0/459430174 shutdown_connections 2024-09-17T05:37:29.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.253+0000 7f6e6f800640 1 -- 172.21.15.122:0/459430174 wait complete. 2024-09-17T05:37:29.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6f800640 1 Processor -- start 2024-09-17T05:37:29.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6f800640 1 -- start start 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6f800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e68073880 0x7f6e680b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6f800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e680cc4c0 0x7f6e681f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6f800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e680ce920 0x7f6e681672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6f800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e680d09f0 con 0x7f6e68073880 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6f800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e680d0870 con 0x7f6e680cc4c0 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6f800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6815ba10 con 0x7f6e680ce920 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e68073880 0x7f6e680b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e68073880 0x7f6e680b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51448/0 (socket says 172.21.15.122:51448) 2024-09-17T05:37:29.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.257+0000 7f6e6ce00640 1 -- 172.21.15.122:0/1923683330 learned_addr learned my addr 172.21.15.122:0/1923683330 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:29.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6ce00640 1 -- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e680ce920 msgr2=0x7f6e681672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6ce00640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e680ce920 0x7f6e681672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6ce00640 1 -- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e680cc4c0 msgr2=0x7f6e681f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6ce00640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e680cc4c0 0x7f6e681f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6ce00640 1 -- 172.21.15.122:0/1923683330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e68154c20 con 0x7f6e68073880 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6ce00640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e68073880 0x7f6e680b3d40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e9f0 tx=0x7f6e5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e65600640 1 -- 172.21.15.122:0/1923683330 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000cd80 con 0x7f6e68073880 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e65600640 1 -- 172.21.15.122:0/1923683330 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e50010430 con 0x7f6e68073880 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6814fcc0 con 0x7f6e68073880 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e65600640 1 -- 172.21.15.122:0/1923683330 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50010860 con 0x7f6e68073880 2024-09-17T05:37:29.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e68154940 con 0x7f6e68073880 2024-09-17T05:37:29.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.261+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6815be30 con 0x7f6e68073880 2024-09-17T05:37:29.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.265+0000 7f6e65600640 1 -- 172.21.15.122:0/1923683330 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f6e50010a00 con 0x7f6e68073880 2024-09-17T05:37:29.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.265+0000 7f6e65600640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e44074cc0 0x7f6e44077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:29.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.265+0000 7f6e65600640 1 -- 172.21.15.122:0/1923683330 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f6e50093c20 con 0x7f6e68073880 2024-09-17T05:37:29.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.269+0000 7f6e5fe00640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e44074cc0 0x7f6e44077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:29.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.269+0000 7f6e5fe00640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e44074cc0 0x7f6e44077180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6e580af000 tx=0x7f6e580b2040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:29.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.273+0000 7f6e65600640 1 -- 172.21.15.122:0/1923683330 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6e500608f0 con 0x7f6e68073880 2024-09-17T05:37:29.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.553+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e68074ad0 con 0x7f6e68073880 2024-09-17T05:37:29.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.553+0000 7f6e65600640 1 -- 172.21.15.122:0/1923683330 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5237 (secure 0 0 0) 0x7f6e500601f0 con 0x7f6e68073880 2024-09-17T05:37:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:29.551 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":67,"btime":"2024-09-17T05:36:41:054240+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1758247367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1758247367},{"type":"v1","addr":"172.21.15.173:6836","nonce":1758247367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2422841556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2422841556},{"type":"v1","addr":"172.21.15.173:6837","nonce":2422841556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/4290616350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":4290616350},{"type":"v1","addr":"172.21.15.122:6834","nonce":4290616350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:36:32.388428+0000","modified":"2024-09-17T05:36:35.423704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2223381402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":2223381402},{"type":"v1","addr":"172.21.15.122:6835","nonce":2223381402}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T05:37:29.552 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 67 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e44074cc0 msgr2=0x7f6e44077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e44074cc0 0x7f6e44077180 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6e580af000 tx=0x7f6e580b2040 comp rx=0 tx=0).stop 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e68073880 msgr2=0x7f6e680b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e68073880 0x7f6e680b3d40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e9f0 tx=0x7f6e5000eeb0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 shutdown_connections 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e44074cc0 0x7f6e44077180 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e680ce920 0x7f6e681672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e680cc4c0 0x7f6e681f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 --2- 172.21.15.122:0/1923683330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e68073880 0x7f6e680b3d40 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 >> 172.21.15.122:0/1923683330 conn(0x7f6e680d0d50 msgr2=0x7f6e680b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:29.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 shutdown_connections 2024-09-17T05:37:29.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.557+0000 7f6e6f800640 1 -- 172.21.15.122:0/1923683330 wait complete. 2024-09-17T05:37:29.572 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:37:29.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 -- 172.21.15.122:0/616675267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 msgr2=0x7fba180ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 --2- 172.21.15.122:0/616675267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 0x7fba180ae9b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fba04009840 tx=0x7fba0401c820 comp rx=0 tx=0).stop 2024-09-17T05:37:29.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 -- 172.21.15.122:0/616675267 shutdown_connections 2024-09-17T05:37:29.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 --2- 172.21.15.122:0/616675267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba180cdb30 0x7fba180aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 --2- 172.21.15.122:0/616675267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 0x7fba180ae9b0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 -- 172.21.15.122:0/616675267 >> 172.21.15.122:0/616675267 conn(0x7fba180d1dc0 msgr2=0x7fba180d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:29.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 -- 172.21.15.122:0/616675267 shutdown_connections 2024-09-17T05:37:29.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 -- 172.21.15.122:0/616675267 wait complete. 2024-09-17T05:37:29.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.717+0000 7fba1fe00640 1 Processor -- start 2024-09-17T05:37:29.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 -- start start 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba180cdb30 0x7fba18159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 0x7fba1814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba1815bea0 0x7fba1814b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba18169340 con 0x7fba180ced80 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba181691c0 con 0x7fba180cdb30 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fba181694c0 con 0x7fba1815bea0 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 0x7fba1814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 0x7fba1814afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51458/0 (socket says 172.21.15.122:51458) 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 -- 172.21.15.122:0/2266964969 learned_addr learned my addr 172.21.15.122:0/2266964969 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1d400640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba180cdb30 0x7fba18159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 -- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba1815bea0 msgr2=0x7fba1814b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1de00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba1815bea0 0x7fba1814b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:29.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba1815bea0 0x7fba1814b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 -- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba180cdb30 msgr2=0x7fba18159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba180cdb30 0x7fba18159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 -- 172.21.15.122:0/2266964969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1806f5d0 con 0x7fba180ced80 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1d400640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba180cdb30 0x7fba18159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1de00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba1815bea0 0x7fba1814b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1ca00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 0x7fba1814afd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fba04009790 tx=0x7fba040a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba15600640 1 -- 172.21.15.122:0/2266964969 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba040bb070 con 0x7fba180ced80 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba15600640 1 -- 172.21.15.122:0/2266964969 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba04005030 con 0x7fba180ced80 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba15600640 1 -- 172.21.15.122:0/2266964969 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba040b2040 con 0x7fba180ced80 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1806f8b0 con 0x7fba180ced80 2024-09-17T05:37:29.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba180ca120 con 0x7fba180ced80 2024-09-17T05:37:29.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.721+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba180ce670 con 0x7fba180ced80 2024-09-17T05:37:29.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.725+0000 7fba15600640 1 -- 172.21.15.122:0/2266964969 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fba04005460 con 0x7fba180ced80 2024-09-17T05:37:29.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.725+0000 7fba15600640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9f8074c70 0x7fb9f8077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:29.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.725+0000 7fba15600640 1 -- 172.21.15.122:0/2266964969 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5879+0+0 (secure 0 0 0) 0x7fba0412c960 con 0x7fba180ced80 2024-09-17T05:37:29.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.725+0000 7fba1d400640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9f8074c70 0x7fb9f8077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:29.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.725+0000 7fba1d400640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9f8074c70 0x7fb9f8077130 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fba0c001630 tx=0x7fba0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:29.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.729+0000 7fba15600640 1 -- 172.21.15.122:0/2266964969 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fba04016410 con 0x7fba180ced80 2024-09-17T05:37:29.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba1814ba50 con 0x7fba180ced80 2024-09-17T05:37:29.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba15600640 1 -- 172.21.15.122:0/2266964969 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5237 (secure 0 0 0) 0x7fba040f9510 con 0x7fba180ced80 2024-09-17T05:37:29.924 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:29.925 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":67,"btime":"2024-09-17T05:36:41:054240+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1758247367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1758247367},{"type":"v1","addr":"172.21.15.173:6836","nonce":1758247367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2422841556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2422841556},{"type":"v1","addr":"172.21.15.173:6837","nonce":2422841556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/4290616350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":4290616350},{"type":"v1","addr":"172.21.15.122:6834","nonce":4290616350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:36:32.388428+0000","modified":"2024-09-17T05:36:35.423704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2223381402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":2223381402},{"type":"v1","addr":"172.21.15.122:6835","nonce":2223381402}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T05:37:29.925 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 67 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9f8074c70 msgr2=0x7fb9f8077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9f8074c70 0x7fb9f8077130 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fba0c001630 tx=0x7fba0c009040 comp rx=0 tx=0).stop 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 msgr2=0x7fba1814afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 0x7fba1814afd0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fba04009790 tx=0x7fba040a56a0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 shutdown_connections 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9f8074c70 0x7fb9f8077130 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba1815bea0 0x7fba1814b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba180ced80 0x7fba1814afd0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 --2- 172.21.15.122:0/2266964969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba180cdb30 0x7fba18159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 >> 172.21.15.122:0/2266964969 conn(0x7fba180d1dc0 msgr2=0x7fba180b7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 shutdown_connections 2024-09-17T05:37:29.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:29.929+0000 7fba1fe00640 1 -- 172.21.15.122:0/2266964969 wait complete. 2024-09-17T05:37:29.946 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:37:30.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.089+0000 7fa4d6800640 1 -- 172.21.15.122:0/3196582964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00cebf0 msgr2=0x7fa4d00cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.089+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3196582964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00cebf0 0x7fa4d00cc8b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00b7e50 tx=0x7fa4bc01c800 comp rx=0 tx=0).stop 2024-09-17T05:37:30.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- 172.21.15.122:0/3196582964 shutdown_connections 2024-09-17T05:37:30.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3196582964 >> v2:172.21.15.173:3300/0 conn(0x7fa4d00b6400 0x7fa4d00ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3196582964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00cebf0 0x7fa4d00cc8b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- 172.21.15.122:0/3196582964 >> 172.21.15.122:0/3196582964 conn(0x7fa4d00d1da0 msgr2=0x7fa4d00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:30.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- 172.21.15.122:0/3196582964 shutdown_connections 2024-09-17T05:37:30.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- 172.21.15.122:0/3196582964 wait complete. 2024-09-17T05:37:30.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 Processor -- start 2024-09-17T05:37:30.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- start start 2024-09-17T05:37:30.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00b6400 0x7fa4d015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4d00cebf0 0x7fa4d00c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4d015be80 0x7fa4d015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d01692b0 con 0x7fa4d00b6400 2024-09-17T05:37:30.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0169130 con 0x7fa4d00cebf0 2024-09-17T05:37:30.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d0169430 con 0x7fa4d015be80 2024-09-17T05:37:30.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00b6400 0x7fa4d015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00b6400 0x7fa4d015e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51470/0 (socket says 172.21.15.122:51470) 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 -- 172.21.15.122:0/3989809421 learned_addr learned my addr 172.21.15.122:0/3989809421 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cea00640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4d00cebf0 0x7fa4d00c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cfe00640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4d015be80 0x7fa4d015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 -- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4d015be80 msgr2=0x7fa4d015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4d015be80 0x7fa4d015ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 -- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4d00cebf0 msgr2=0x7fa4d00c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4d00cebf0 0x7fa4d00c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 -- 172.21.15.122:0/3989809421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d015cea0 con 0x7fa4d00b6400 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4cf400640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00b6400 0x7fa4d015e1f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000cbc0 tx=0x7fa4c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4c7400640 1 -- 172.21.15.122:0/3989809421 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0016020 con 0x7fa4d00b6400 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4c7400640 1 -- 172.21.15.122:0/3989809421 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4c000ab40 con 0x7fa4d00b6400 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d015d400 con 0x7fa4d00b6400 2024-09-17T05:37:30.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4c7400640 1 -- 172.21.15.122:0/3989809421 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c000ce40 con 0x7fa4d00b6400 2024-09-17T05:37:30.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.093+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d015d120 con 0x7fa4d00b6400 2024-09-17T05:37:30.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.097+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d00ce4e0 con 0x7fa4d00b6400 2024-09-17T05:37:30.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.097+0000 7fa4c7400640 1 -- 172.21.15.122:0/3989809421 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fa4c0019020 con 0x7fa4d00b6400 2024-09-17T05:37:30.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.097+0000 7fa4c7400640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4b0074c70 0x7fa4b0077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.097+0000 7fa4c7400640 1 -- 172.21.15.122:0/3989809421 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5879+0+0 (secure 0 0 0) 0x7fa4c0092fe0 con 0x7fa4d00b6400 2024-09-17T05:37:30.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.097+0000 7fa4cea00640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4b0074c70 0x7fa4b0077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.101+0000 7fa4cea00640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4b0074c70 0x7fa4b0077130 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0af000 tx=0x7fa4bc01b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:30.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.101+0000 7fa4c7400640 1 -- 172.21.15.122:0/3989809421 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa4c0020030 con 0x7fa4d00b6400 2024-09-17T05:37:30.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.273+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4d015e730 con 0x7fa4d00b6400 2024-09-17T05:37:30.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.273+0000 7fa4c7400640 1 -- 172.21.15.122:0/3989809421 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17034 (secure 0 0 0) 0x7fa4d015cea0 con 0x7fa4d00b6400 2024-09-17T05:37:30.271 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:30.272 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":106,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:37:29.298340+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:36:30.080602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-17T05:36:31.349633+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":4,"to":0}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:37:30.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4b0074c70 msgr2=0x7fa4b0077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4b0074c70 0x7fa4b0077130 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0af000 tx=0x7fa4bc01b040 comp rx=0 tx=0).stop 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00b6400 msgr2=0x7fa4d015e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00b6400 0x7fa4d015e1f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000cbc0 tx=0x7fa4c000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 shutdown_connections 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4b0074c70 0x7fa4b0077130 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4d015be80 0x7fa4d015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4d00cebf0 0x7fa4d00c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 --2- 172.21.15.122:0/3989809421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4d00b6400 0x7fa4d015e1f0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 >> 172.21.15.122:0/3989809421 conn(0x7fa4d00d1da0 msgr2=0x7fa4d0071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 shutdown_connections 2024-09-17T05:37:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.277+0000 7fa4d6800640 1 -- 172.21.15.122:0/3989809421 wait complete. 2024-09-17T05:37:30.295 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:37:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 -- 172.21.15.122:0/2128025296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e8164c60 msgr2=0x7f96e8167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 --2- 172.21.15.122:0/2128025296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e8164c60 0x7f96e8167050 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009bc0 tx=0x7f96e001cae0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 -- 172.21.15.122:0/2128025296 shutdown_connections 2024-09-17T05:37:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 --2- 172.21.15.122:0/2128025296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e8164c60 0x7f96e8167050 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 -- 172.21.15.122:0/2128025296 >> 172.21.15.122:0/2128025296 conn(0x7f96e80d1e30 msgr2=0x7f96e80d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 -- 172.21.15.122:0/2128025296 shutdown_connections 2024-09-17T05:37:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 -- 172.21.15.122:0/2128025296 wait complete. 2024-09-17T05:37:30.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 Processor -- start 2024-09-17T05:37:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.441+0000 7f96ed800640 1 -- start start 2024-09-17T05:37:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e80b2e80 0x7f96e81514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96e815f490 0x7f96e80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96e8164c60 0x7f96e80b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8169f30 con 0x7f96e80b2e80 2024-09-17T05:37:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8169db0 con 0x7f96e815f490 2024-09-17T05:37:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f96e816a0b0 con 0x7f96e8164c60 2024-09-17T05:37:30.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e80b2e80 0x7f96e81514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e80b2e80 0x7f96e81514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51488/0 (socket says 172.21.15.122:51488) 2024-09-17T05:37:30.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 -- 172.21.15.122:0/1958613138 learned_addr learned my addr 172.21.15.122:0/1958613138 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:30.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e7400640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96e8164c60 0x7f96e80b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6000640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96e815f490 0x7f96e80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 -- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96e8164c60 msgr2=0x7f96e80b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96e8164c60 0x7f96e80b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 -- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96e815f490 msgr2=0x7f96e80ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96e815f490 0x7f96e80ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 -- 172.21.15.122:0/1958613138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e80cd2d0 con 0x7f96e80b2e80 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96e6a00640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e80b2e80 0x7f96e81514d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f96d800c500 tx=0x7f96d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96dea00640 1 -- 172.21.15.122:0/1958613138 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8017070 con 0x7f96e80b2e80 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96dea00640 1 -- 172.21.15.122:0/1958613138 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96d80053c0 con 0x7f96e80b2e80 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e80cd5b0 con 0x7f96e80b2e80 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96dea00640 1 -- 172.21.15.122:0/1958613138 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8012890 con 0x7f96e80b2e80 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e80cdb70 con 0x7f96e80b2e80 2024-09-17T05:37:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e80b1c80 con 0x7f96e80b2e80 2024-09-17T05:37:30.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.445+0000 7f96dea00640 1 -- 172.21.15.122:0/1958613138 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f96d801a070 con 0x7f96e80b2e80 2024-09-17T05:37:30.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.449+0000 7f96dea00640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96c8074c70 0x7f96c8077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.449+0000 7f96dea00640 1 -- 172.21.15.122:0/1958613138 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5879+0+0 (secure 0 0 0) 0x7f96d80923b0 con 0x7f96e80b2e80 2024-09-17T05:37:30.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.449+0000 7f96e6000640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96c8074c70 0x7f96c8077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.449+0000 7f96e6000640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96c8074c70 0x7f96c8077130 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f96e81fad60 tx=0x7f96d4007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:30.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.453+0000 7f96dea00640 1 -- 172.21.15.122:0/1958613138 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f96d805e7f0 con 0x7f96e80b2e80 2024-09-17T05:37:30.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.657+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f96e80d0660 con 0x7f96e80b2e80 2024-09-17T05:37:30.653 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:37:30.657+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:37:30.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96dea00640 1 -- 172.21.15.122:0/1958613138 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f96d8063220 con 0x7f96e80b2e80 2024-09-17T05:37:30.656 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:37:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96c8074c70 msgr2=0x7f96c8077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96c8074c70 0x7f96c8077130 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f96e81fad60 tx=0x7f96d4007ea0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e80b2e80 msgr2=0x7f96e81514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e80b2e80 0x7f96e81514d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f96d800c500 tx=0x7f96d800c9c0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 shutdown_connections 2024-09-17T05:37:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f96c8074c70 0x7f96c8077130 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f96e8164c60 0x7f96e80b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96e815f490 0x7f96e80ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 --2- 172.21.15.122:0/1958613138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96e80b2e80 0x7f96e81514d0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 >> 172.21.15.122:0/1958613138 conn(0x7f96e80d1e30 msgr2=0x7f96e8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:30.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 shutdown_connections 2024-09-17T05:37:30.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.661+0000 7f96ed800640 1 -- 172.21.15.122:0/1958613138 wait complete. 2024-09-17T05:37:30.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:37:30.728 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:37:30.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.833+0000 7f496a400640 1 -- 172.21.15.122:0/1025110786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4964164a30 msgr2=0x7f4964166e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.833+0000 7f496a400640 1 --2- 172.21.15.122:0/1025110786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4964164a30 0x7f4964166e20 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f495c009840 tx=0x7f495c01c790 comp rx=0 tx=0).stop 2024-09-17T05:37:30.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.833+0000 7f496a400640 1 -- 172.21.15.122:0/1025110786 shutdown_connections 2024-09-17T05:37:30.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.833+0000 7f496a400640 1 --2- 172.21.15.122:0/1025110786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4964164a30 0x7f4964166e20 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.833+0000 7f496a400640 1 -- 172.21.15.122:0/1025110786 >> 172.21.15.122:0/1025110786 conn(0x7f49640d0da0 msgr2=0x7f49640d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:30.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.833+0000 7f496a400640 1 -- 172.21.15.122:0/1025110786 shutdown_connections 2024-09-17T05:37:30.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.833+0000 7f496a400640 1 -- 172.21.15.122:0/1025110786 wait complete. 2024-09-17T05:37:30.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 Processor -- start 2024-09-17T05:37:30.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 -- start start 2024-09-17T05:37:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f496406f8e0 0x7f496415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49640ccc60 0x7f49640d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4964164a30 0x7f496414fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f496416a290 con 0x7f496406f8e0 2024-09-17T05:37:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f496416a110 con 0x7f49640ccc60 2024-09-17T05:37:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f496416a410 con 0x7f4964164a30 2024-09-17T05:37:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f496406f8e0 0x7f496415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f496406f8e0 0x7f496415acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51518/0 (socket says 172.21.15.122:51518) 2024-09-17T05:37:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 -- 172.21.15.122:0/445606778 learned_addr learned my addr 172.21.15.122:0/445606778 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f495aa00640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49640ccc60 0x7f49640d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963e00640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4964164a30 0x7f496414fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 -- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4964164a30 msgr2=0x7f496414fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4964164a30 0x7f496414fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 -- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49640ccc60 msgr2=0x7f49640d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49640ccc60 0x7f49640d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:30.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 -- 172.21.15.122:0/445606778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49640d09b0 con 0x7f496406f8e0 2024-09-17T05:37:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963e00640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4964164a30 0x7f496414fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f495aa00640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49640ccc60 0x7f49640d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f4963400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f496406f8e0 0x7f496415acc0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f495400c4b0 tx=0x7f495400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f495be00640 1 -- 172.21.15.122:0/445606778 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954016020 con 0x7f496406f8e0 2024-09-17T05:37:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f495be00640 1 -- 172.21.15.122:0/445606778 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4954012920 con 0x7f496406f8e0 2024-09-17T05:37:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f495be00640 1 -- 172.21.15.122:0/445606778 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954012d50 con 0x7f496406f8e0 2024-09-17T05:37:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49641518b0 con 0x7f496406f8e0 2024-09-17T05:37:30.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49641fbad0 con 0x7f496406f8e0 2024-09-17T05:37:30.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.837+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496406fd70 con 0x7f496406f8e0 2024-09-17T05:37:30.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.841+0000 7f495be00640 1 -- 172.21.15.122:0/445606778 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f4954019020 con 0x7f496406f8e0 2024-09-17T05:37:30.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.841+0000 7f495be00640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4940074c70 0x7f4940077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:30.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.841+0000 7f495be00640 1 -- 172.21.15.122:0/445606778 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5937+0+0 (secure 0 0 0) 0x7f4954093150 con 0x7f496406f8e0 2024-09-17T05:37:30.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.841+0000 7f495aa00640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4940074c70 0x7f4940077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:30.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.841+0000 7f495aa00640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4940074c70 0x7f4940077130 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f49641fb370 tx=0x7f494c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:30.845+0000 7f495be00640 1 -- 172.21.15.122:0/445606778 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4954020020 con 0x7f496406f8e0 2024-09-17T05:37:31.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.049+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f496414f3d0 con 0x7f496406f8e0 2024-09-17T05:37:31.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.665+0000 7f495be00640 1 -- 172.21.15.122:0/445606778 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7f495405fc60 con 0x7f496406f8e0 2024-09-17T05:37:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4940074c70 msgr2=0x7f4940077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4940074c70 0x7f4940077130 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f49641fb370 tx=0x7f494c009040 comp rx=0 tx=0).stop 2024-09-17T05:37:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f496406f8e0 msgr2=0x7f496415acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f496406f8e0 0x7f496415acc0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f495400c4b0 tx=0x7f495400c970 comp rx=0 tx=0).stop 2024-09-17T05:37:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 shutdown_connections 2024-09-17T05:37:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4940074c70 0x7f4940077130 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:31.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4964164a30 0x7f496414fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49640ccc60 0x7f49640d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 --2- 172.21.15.122:0/445606778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f496406f8e0 0x7f496415acc0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 >> 172.21.15.122:0/445606778 conn(0x7f49640d0da0 msgr2=0x7f4964166a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 shutdown_connections 2024-09-17T05:37:31.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.669+0000 7f496a400640 1 -- 172.21.15.122:0/445606778 wait complete. 2024-09-17T05:37:31.687 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:37:31.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.885+0000 7f7f28200640 1 -- 172.21.15.122:0/704557681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f2015c1f0 msgr2=0x7f7f200cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:31.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.885+0000 7f7f28200640 1 --2- 172.21.15.122:0/704557681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f2015c1f0 0x7f7f200cee10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009840 tx=0x7f7f0c01f010 comp rx=0 tx=0).stop 2024-09-17T05:37:31.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.885+0000 7f7f28200640 1 -- 172.21.15.122:0/704557681 shutdown_connections 2024-09-17T05:37:31.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.885+0000 7f7f28200640 1 --2- 172.21.15.122:0/704557681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f2015c1f0 0x7f7f200cee10 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.885+0000 7f7f28200640 1 -- 172.21.15.122:0/704557681 >> 172.21.15.122:0/704557681 conn(0x7f7f200d1da0 msgr2=0x7f7f200d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.885+0000 7f7f28200640 1 -- 172.21.15.122:0/704557681 shutdown_connections 2024-09-17T05:37:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.885+0000 7f7f28200640 1 -- 172.21.15.122:0/704557681 wait complete. 2024-09-17T05:37:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 Processor -- start 2024-09-17T05:37:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 -- start start 2024-09-17T05:37:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f200b0b60 0x7f7f201f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f2015be10 0x7f7f201f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f2015c1f0 0x7f7f20153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200cd670 con 0x7f7f200b0b60 2024-09-17T05:37:31.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f200cd4f0 con 0x7f7f2015c1f0 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7f200cd7f0 con 0x7f7f2015be10 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f200b0b60 0x7f7f201f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f200b0b60 0x7f7f201f8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43702/0 (socket says 172.21.15.122:43702) 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 -- 172.21.15.122:0/230405603 learned_addr learned my addr 172.21.15.122:0/230405603 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f26200640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f2015c1f0 0x7f7f20153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f24e00640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f2015be10 0x7f7f201f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 -- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f2015be10 msgr2=0x7f7f201f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f2015be10 0x7f7f201f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 -- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f2015c1f0 msgr2=0x7f7f20153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:31.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f2015c1f0 0x7f7f20153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:31.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 -- 172.21.15.122:0/230405603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f200cedd0 con 0x7f7f200b0b60 2024-09-17T05:37:31.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f25800640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f200b0b60 0x7f7f201f8730 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c4e0 tx=0x7f7f1400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:31.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f1d600640 1 -- 172.21.15.122:0/230405603 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140052d0 con 0x7f7f200b0b60 2024-09-17T05:37:31.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f1d600640 1 -- 172.21.15.122:0/230405603 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f14005470 con 0x7f7f200b0b60 2024-09-17T05:37:31.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f201f8c70 con 0x7f7f200b0b60 2024-09-17T05:37:31.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f1d600640 1 -- 172.21.15.122:0/230405603 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14012890 con 0x7f7f200b0b60 2024-09-17T05:37:31.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f200c7c60 con 0x7f7f200b0b60 2024-09-17T05:37:31.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.889+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f200cdca0 con 0x7f7f200b0b60 2024-09-17T05:37:31.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.893+0000 7f7f1d600640 1 -- 172.21.15.122:0/230405603 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f7f14003600 con 0x7f7f200b0b60 2024-09-17T05:37:31.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.893+0000 7f7f1d600640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f00074c70 0x7f7f00077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:31.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.893+0000 7f7f1d600640 1 -- 172.21.15.122:0/230405603 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5937+0+0 (secure 0 0 0) 0x7f7f14092d80 con 0x7f7f200b0b60 2024-09-17T05:37:31.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.893+0000 7f7f24e00640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f00074c70 0x7f7f00077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:31.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.893+0000 7f7f24e00640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f00074c70 0x7f7f00077130 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0b2000 tx=0x7f7f0c01d850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:31.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:31.897+0000 7f7f1d600640 1 -- 172.21.15.122:0/230405603 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7f201f8c70 con 0x7f7f200b0b60 2024-09-17T05:37:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.065+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f20076410 con 0x7f7f200b0b60 2024-09-17T05:37:32.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.669+0000 7f7f1d600640 1 -- 172.21.15.122:0/230405603 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f7f1405f810 con 0x7f7f200b0b60 2024-09-17T05:37:32.664 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:37:32.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f00074c70 msgr2=0x7f7f00077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:32.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f00074c70 0x7f7f00077130 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0b2000 tx=0x7f7f0c01d850 comp rx=0 tx=0).stop 2024-09-17T05:37:32.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f200b0b60 msgr2=0x7f7f201f8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f200b0b60 0x7f7f201f8730 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400c4e0 tx=0x7f7f1400c9a0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 shutdown_connections 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f00074c70 0x7f7f00077130 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f2015c1f0 0x7f7f20153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f2015be10 0x7f7f201f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 --2- 172.21.15.122:0/230405603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f200b0b60 0x7f7f201f8730 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 >> 172.21.15.122:0/230405603 conn(0x7f7f200d1da0 msgr2=0x7f7f20071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 shutdown_connections 2024-09-17T05:37:32.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.673+0000 7f7f28200640 1 -- 172.21.15.122:0/230405603 wait complete. 2024-09-17T05:37:32.693 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:37:32.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- 172.21.15.122:0/2512854612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4040cff00 msgr2=0x7fa4040d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 --2- 172.21.15.122:0/2512854612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4040cff00 0x7fa4040d28c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009900 tx=0x7fa3f401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- 172.21.15.122:0/2512854612 shutdown_connections 2024-09-17T05:37:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 --2- 172.21.15.122:0/2512854612 >> v2:172.21.15.173:3300/0 conn(0x7fa4040d02e0 0x7fa4040d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 --2- 172.21.15.122:0/2512854612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4040cff00 0x7fa4040d28c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- 172.21.15.122:0/2512854612 >> 172.21.15.122:0/2512854612 conn(0x7fa4040d1da0 msgr2=0x7fa4040d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- 172.21.15.122:0/2512854612 shutdown_connections 2024-09-17T05:37:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- 172.21.15.122:0/2512854612 wait complete. 2024-09-17T05:37:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 Processor -- start 2024-09-17T05:37:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- start start 2024-09-17T05:37:32.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4040cff00 0x7fa4040cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:32.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4040d02e0 0x7fa4040c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:32.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa40415eff0 0x7fa404151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4041690e0 con 0x7fa40415eff0 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa404168f60 con 0x7fa4040cff00 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa403200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa404169260 con 0x7fa4040d02e0 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa402800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa40415eff0 0x7fa404151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa401e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4040cff00 0x7fa4040cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa401e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4040cff00 0x7fa4040cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:37632/0 (socket says 172.21.15.122:37632) 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa402800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa40415eff0 0x7fa404151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43708/0 (socket says 172.21.15.122:43708) 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa401e00640 1 -- 172.21.15.122:0/945532664 learned_addr learned my addr 172.21.15.122:0/945532664 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.845+0000 7fa401400640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4040d02e0 0x7fa4040c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa401e00640 1 -- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4040d02e0 msgr2=0x7fa4040c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa401e00640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4040d02e0 0x7fa4040c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa401e00640 1 -- 172.21.15.122:0/945532664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa40415eff0 msgr2=0x7fa404151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa401e00640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa40415eff0 0x7fa404151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa401e00640 1 -- 172.21.15.122:0/945532664 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4040c77b0 con 0x7fa4040cff00 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa401e00640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4040cff00 0x7fa4040cc880 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009850 tx=0x7fa3f4005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa3fa000640 1 -- 172.21.15.122:0/945532664 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40b2040 con 0x7fa4040cff00 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa3fa000640 1 -- 172.21.15.122:0/945532664 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3f40053e0 con 0x7fa4040cff00 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4040c7a90 con 0x7fa4040cff00 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa3fa000640 1 -- 172.21.15.122:0/945532664 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40046d0 con 0x7fa4040cff00 2024-09-17T05:37:32.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4040c82b0 con 0x7fa4040cff00 2024-09-17T05:37:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4040d28c0 con 0x7fa4040cff00 2024-09-17T05:37:32.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.849+0000 7fa3fa000640 1 -- 172.21.15.122:0/945532664 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fa3f40be060 con 0x7fa4040cff00 2024-09-17T05:37:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.853+0000 7fa3fa000640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3e4074c70 0x7fa3e4077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.853+0000 7fa3fa000640 1 -- 172.21.15.122:0/945532664 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5441+0+0 (secure 0 0 0) 0x7fa3f412ccb0 con 0x7fa4040cff00 2024-09-17T05:37:32.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.853+0000 7fa401400640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3e4074c70 0x7fa3e4077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:32.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.853+0000 7fa401400640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3e4074c70 0x7fa3e4077130 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa404155010 tx=0x7fa3f00091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:32.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:32.853+0000 7fa3fa000640 1 -- 172.21.15.122:0/945532664 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa3f40164d0 con 0x7fa4040cff00 2024-09-17T05:37:33.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.021+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa40406f900 con 0x7fa4040cff00 2024-09-17T05:37:33.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:37:33.597 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:37:33.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.685+0000 7fa3fa000640 1 -- 172.21.15.122:0/945532664 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7fa3f40f9210 con 0x7fa4040cff00 2024-09-17T05:37:33.682 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:37:33.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3e4074c70 msgr2=0x7fa3e4077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:33.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3e4074c70 0x7fa3e4077130 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa404155010 tx=0x7fa3f00091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4040cff00 msgr2=0x7fa4040cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:33.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4040cff00 0x7fa4040cc880 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009850 tx=0x7fa3f4005e70 comp rx=0 tx=0).stop 2024-09-17T05:37:33.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 shutdown_connections 2024-09-17T05:37:33.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa3e4074c70 0x7fa3e4077130 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa40415eff0 0x7fa404151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4040d02e0 0x7fa4040c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 --2- 172.21.15.122:0/945532664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4040cff00 0x7fa4040cc880 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 >> 172.21.15.122:0/945532664 conn(0x7fa4040d1da0 msgr2=0x7fa40406ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:33.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 shutdown_connections 2024-09-17T05:37:33.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.689+0000 7fa403200640 1 -- 172.21.15.122:0/945532664 wait complete. 2024-09-17T05:37:33.706 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:37:33.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.893+0000 7f13f9200640 1 -- 172.21.15.122:0/1938131584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bf70 msgr2=0x7f13f40b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:33.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.893+0000 7f13f9200640 1 --2- 172.21.15.122:0/1938131584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bf70 0x7f13f40b53e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009bc0 tx=0x7f13ec01cba0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.893+0000 7f13f9200640 1 -- 172.21.15.122:0/1938131584 shutdown_connections 2024-09-17T05:37:33.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.893+0000 7f13f9200640 1 --2- 172.21.15.122:0/1938131584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bf70 0x7f13f40b53e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.893+0000 7f13f9200640 1 --2- 172.21.15.122:0/1938131584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f40d2940 0x7f13f4160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.893+0000 7f13f9200640 1 -- 172.21.15.122:0/1938131584 >> 172.21.15.122:0/1938131584 conn(0x7f13f40d1d90 msgr2=0x7f13f40d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:33.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.893+0000 7f13f9200640 1 -- 172.21.15.122:0/1938131584 shutdown_connections 2024-09-17T05:37:33.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.893+0000 7f13f9200640 1 -- 172.21.15.122:0/1938131584 wait complete. 2024-09-17T05:37:33.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 Processor -- start 2024-09-17T05:37:33.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 -- start start 2024-09-17T05:37:33.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f40d2940 0x7f13f4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:33.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bb90 0x7f13f41f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:33.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13f415bf70 0x7f13f406f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:33.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f13f40b66e0 con 0x7f13f415bf70 2024-09-17T05:37:33.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f13f40b6560 con 0x7f13f415bb90 2024-09-17T05:37:33.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f13f4168ca0 con 0x7f13f40d2940 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13f415bf70 0x7f13f406f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f1400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bb90 0x7f13f41f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f40d2940 0x7f13f4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f1400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bb90 0x7f13f41f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:37644/0 (socket says 172.21.15.122:37644) 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13f415bf70 0x7f13f406f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43712/0 (socket says 172.21.15.122:43712) 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f1400640 1 -- 172.21.15.122:0/2189533378 learned_addr learned my addr 172.21.15.122:0/2189533378 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f2800640 1 -- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f40d2940 msgr2=0x7f13f4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f2800640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f40d2940 0x7f13f4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f2800640 1 -- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bb90 msgr2=0x7f13f41f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f2800640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bb90 0x7f13f41f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:33.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f2800640 1 -- 172.21.15.122:0/2189533378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f41f78f0 con 0x7f13f415bf70 2024-09-17T05:37:33.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f2800640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13f415bf70 0x7f13f406f430 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0ac000 tx=0x7f13ec005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:33.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13ea000640 1 -- 172.21.15.122:0/2189533378 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0bb070 con 0x7f13f415bf70 2024-09-17T05:37:33.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13ea000640 1 -- 172.21.15.122:0/2189533378 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13ec004040 con 0x7f13f415bf70 2024-09-17T05:37:33.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13ea000640 1 -- 172.21.15.122:0/2189533378 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0b2040 con 0x7f13f415bf70 2024-09-17T05:37:33.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f41562f0 con 0x7f13f415bf70 2024-09-17T05:37:33.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41568b0 con 0x7f13f415bf70 2024-09-17T05:37:33.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.897+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f40c6a40 con 0x7f13f415bf70 2024-09-17T05:37:33.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.901+0000 7f13ea000640 1 -- 172.21.15.122:0/2189533378 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f13ec0be070 con 0x7f13f415bf70 2024-09-17T05:37:33.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.901+0000 7f13ea000640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d4074c70 0x7f13d4077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:33.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.901+0000 7f13ea000640 1 -- 172.21.15.122:0/2189533378 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7f13ec12cb60 con 0x7f13f415bf70 2024-09-17T05:37:33.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.901+0000 7f13f1e00640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d4074c70 0x7f13d4077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:33.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.901+0000 7f13f1e00640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d4074c70 0x7f13d4077130 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f13e4001910 tx=0x7f13e4001420 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:33.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:33.905+0000 7f13ea000640 1 -- 172.21.15.122:0/2189533378 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f13f41f78f0 con 0x7f13f415bf70 2024-09-17T05:37:34.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.077+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f13f40b53e0 con 0x7f13f415bf70 2024-09-17T05:37:34.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.089+0000 7f13ea000640 1 -- 172.21.15.122:0/2189533378 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7f13ec016790 con 0x7f13f415bf70 2024-09-17T05:37:34.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d4074c70 msgr2=0x7f13d4077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d4074c70 0x7f13d4077130 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f13e4001910 tx=0x7f13e4001420 comp rx=0 tx=0).stop 2024-09-17T05:37:34.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13f415bf70 msgr2=0x7f13f406f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13f415bf70 0x7f13f406f430 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0ac000 tx=0x7f13ec005c40 comp rx=0 tx=0).stop 2024-09-17T05:37:34.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 shutdown_connections 2024-09-17T05:37:34.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f13d4074c70 0x7f13d4077130 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13f415bf70 0x7f13f406f430 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13f415bb90 0x7f13f41f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 --2- 172.21.15.122:0/2189533378 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13f40d2940 0x7f13f4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 >> 172.21.15.122:0/2189533378 conn(0x7f13f40d1d90 msgr2=0x7f13f40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:34.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 shutdown_connections 2024-09-17T05:37:34.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.093+0000 7f13f9200640 1 -- 172.21.15.122:0/2189533378 wait complete. 2024-09-17T05:37:34.107 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-17T05:37:34.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.253+0000 7f5da6400640 1 -- 172.21.15.122:0/4082197089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da015f000 msgr2=0x7f5da00d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.253+0000 7f5da6400640 1 --2- 172.21.15.122:0/4082197089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da015f000 0x7f5da00d01b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5da00c56a0 tx=0x7f5d9401c6e0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.253+0000 7f5da6400640 1 -- 172.21.15.122:0/4082197089 shutdown_connections 2024-09-17T05:37:34.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.253+0000 7f5da6400640 1 --2- 172.21.15.122:0/4082197089 >> v2:172.21.15.173:3300/0 conn(0x7f5da00cefb0 0x7f5da00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.253+0000 7f5da6400640 1 --2- 172.21.15.122:0/4082197089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da015f000 0x7f5da00d01b0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.253+0000 7f5da6400640 1 -- 172.21.15.122:0/4082197089 >> 172.21.15.122:0/4082197089 conn(0x7f5da009aa10 msgr2=0x7f5da009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:34.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.253+0000 7f5da6400640 1 -- 172.21.15.122:0/4082197089 shutdown_connections 2024-09-17T05:37:34.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.253+0000 7f5da6400640 1 -- 172.21.15.122:0/4082197089 wait complete. 2024-09-17T05:37:34.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 Processor -- start 2024-09-17T05:37:34.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 -- start start 2024-09-17T05:37:34.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da00ccaf0 0x7f5da015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:34.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5da00cefb0 0x7f5da00c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:34.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5da015f000 0x7f5da015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:34.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01692e0 con 0x7f5da00ccaf0 2024-09-17T05:37:34.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0169160 con 0x7f5da00cefb0 2024-09-17T05:37:34.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5da0169460 con 0x7f5da015f000 2024-09-17T05:37:34.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da00ccaf0 0x7f5da015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da00ccaf0 0x7f5da015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43740/0 (socket says 172.21.15.122:43740) 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 -- 172.21.15.122:0/3597143442 learned_addr learned my addr 172.21.15.122:0/3597143442 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5d9fe00640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5da00cefb0 0x7f5da00c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 -- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5da015f000 msgr2=0x7f5da015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5a00640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5da015f000 0x7f5da015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5da015f000 0x7f5da015bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 -- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5da00cefb0 msgr2=0x7f5da00c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5da00cefb0 0x7f5da00c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 -- 172.21.15.122:0/3597143442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00c83c0 con 0x7f5da00ccaf0 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5d9fe00640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5da00cefb0 0x7f5da00c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5a00640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5da015f000 0x7f5da015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da5000640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da00ccaf0 0x7f5da015a390 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5d94000c10 tx=0x7f5d940a8600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:34.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5d9cc00640 1 -- 172.21.15.122:0/3597143442 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940a8a50 con 0x7f5da00ccaf0 2024-09-17T05:37:34.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5d9cc00640 1 -- 172.21.15.122:0/3597143442 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d940a8bf0 con 0x7f5da00ccaf0 2024-09-17T05:37:34.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5d9cc00640 1 -- 172.21.15.122:0/3597143442 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94004030 con 0x7f5da00ccaf0 2024-09-17T05:37:34.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da00b5af0 con 0x7f5da00ccaf0 2024-09-17T05:37:34.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da00b60b0 con 0x7f5da00ccaf0 2024-09-17T05:37:34.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.257+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da00cc5b0 con 0x7f5da00ccaf0 2024-09-17T05:37:34.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.261+0000 7f5d9cc00640 1 -- 172.21.15.122:0/3597143442 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f5d940b8030 con 0x7f5da00ccaf0 2024-09-17T05:37:34.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.261+0000 7f5d9cc00640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d80074c70 0x7f5d80077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:34.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.261+0000 7f5d9cc00640 1 -- 172.21.15.122:0/3597143442 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7f5d9412d020 con 0x7f5da00ccaf0 2024-09-17T05:37:34.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.261+0000 7f5d9fe00640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d80074c70 0x7f5d80077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:34.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.261+0000 7f5d9fe00640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d80074c70 0x7f5d80077130 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005fd0 tx=0x7f5d8c005e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:34.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.265+0000 7f5d9cc00640 1 -- 172.21.15.122:0/3597143442 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5d940ac030 con 0x7f5da00ccaf0 2024-09-17T05:37:34.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.493+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f5da00cdd50 con 0x7f5da00ccaf0 2024-09-17T05:37:34.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.677+0000 7f5d9cc00640 1 -- 172.21.15.122:0/3597143442 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v570) ==== 178+0+0 (secure 0 0 0) 0x7f5d940163f0 con 0x7f5da00ccaf0 2024-09-17T05:37:34.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d80074c70 msgr2=0x7f5d80077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d80074c70 0x7f5d80077130 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005fd0 tx=0x7f5d8c005e60 comp rx=0 tx=0).stop 2024-09-17T05:37:34.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da00ccaf0 msgr2=0x7f5da015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da00ccaf0 0x7f5da015a390 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5d94000c10 tx=0x7f5d940a8600 comp rx=0 tx=0).stop 2024-09-17T05:37:34.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 shutdown_connections 2024-09-17T05:37:34.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5d80074c70 0x7f5d80077130 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5da015f000 0x7f5da015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5da00cefb0 0x7f5da00c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 --2- 172.21.15.122:0/3597143442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5da00ccaf0 0x7f5da015a390 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 >> 172.21.15.122:0/3597143442 conn(0x7f5da009aa10 msgr2=0x7f5da00b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:34.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 shutdown_connections 2024-09-17T05:37:34.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.681+0000 7f5da6400640 1 -- 172.21.15.122:0/3597143442 wait complete. 2024-09-17T05:37:34.697 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-09-17T05:37:34.697 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-09-17T05:37:34.697 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:37:34.698 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:37:34.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- 172.21.15.122:0/3961240828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 msgr2=0x7fe1800b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 --2- 172.21.15.122:0/3961240828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 0x7fe1800b14c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe170009840 tx=0x7fe17001c760 comp rx=0 tx=0).stop 2024-09-17T05:37:34.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- 172.21.15.122:0/3961240828 shutdown_connections 2024-09-17T05:37:34.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 --2- 172.21.15.122:0/3961240828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1800afca0 0x7fe1800b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 --2- 172.21.15.122:0/3961240828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 0x7fe1800b14c0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- 172.21.15.122:0/3961240828 >> 172.21.15.122:0/3961240828 conn(0x7fe1800d1e30 msgr2=0x7fe1800d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:34.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- 172.21.15.122:0/3961240828 shutdown_connections 2024-09-17T05:37:34.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- 172.21.15.122:0/3961240828 wait complete. 2024-09-17T05:37:34.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 Processor -- start 2024-09-17T05:37:34.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- start start 2024-09-17T05:37:34.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1800afca0 0x7fe18015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:34.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1800b2e80 0x7fe180070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:34.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 0x7fe180152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:34.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe180168c80 con 0x7fe18015f490 2024-09-17T05:37:34.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1800b2320 con 0x7fe1800afca0 2024-09-17T05:37:34.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe186200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe180168e00 con 0x7fe1800b2e80 2024-09-17T05:37:34.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 0x7fe180152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:34.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 0x7fe180152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43764/0 (socket says 172.21.15.122:43764) 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 -- 172.21.15.122:0/189445904 learned_addr learned my addr 172.21.15.122:0/189445904 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17f400640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1800afca0 0x7fe18015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe176a00640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1800b2e80 0x7fe180070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 -- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1800b2e80 msgr2=0x7fe180070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1800b2e80 0x7fe180070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 -- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1800afca0 msgr2=0x7fe18015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1800afca0 0x7fe18015ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 -- 172.21.15.122:0/189445904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18015bba0 con 0x7fe18015f490 2024-09-17T05:37:34.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.845+0000 7fe17fe00640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 0x7fe180152c10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe17800eb20 tx=0x7fe17800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:34.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe177e00640 1 -- 172.21.15.122:0/189445904 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178004b30 con 0x7fe18015f490 2024-09-17T05:37:34.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe177e00640 1 -- 172.21.15.122:0/189445904 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe178004cf0 con 0x7fe18015f490 2024-09-17T05:37:34.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe177e00640 1 -- 172.21.15.122:0/189445904 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178005280 con 0x7fe18015f490 2024-09-17T05:37:34.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18015bee0 con 0x7fe18015f490 2024-09-17T05:37:34.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1800cdc90 con 0x7fe18015f490 2024-09-17T05:37:34.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1800cde90 con 0x7fe18015f490 2024-09-17T05:37:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe177e00640 1 -- 172.21.15.122:0/189445904 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fe178005420 con 0x7fe18015f490 2024-09-17T05:37:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe177e00640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe15c074ca0 0x7fe15c077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:34.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe177e00640 1 -- 172.21.15.122:0/189445904 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7fe178093ed0 con 0x7fe18015f490 2024-09-17T05:37:34.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.849+0000 7fe17f400640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe15c074ca0 0x7fe15c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:34.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.853+0000 7fe17f400640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe15c074ca0 0x7fe15c077160 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe1700af000 tx=0x7fe17001afa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:34.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:34.853+0000 7fe177e00640 1 -- 172.21.15.122:0/189445904 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe18015bee0 con 0x7fe18015f490 2024-09-17T05:37:35.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.025+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe18015a860 con 0x7fe18015f490 2024-09-17T05:37:35.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe177e00640 1 -- 172.21.15.122:0/189445904 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13236 (secure 0 0 0) 0x7fe1780609e0 con 0x7fe18015f490 2024-09-17T05:37:35.024 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:35.024 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":109,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:37:33.672901+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6835/2223381402":"2024-09-18T05:37:30.661293+0000","172.21.15.122:6833/2223381402":"2024-09-18T05:37:30.661293+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe15c074ca0 msgr2=0x7fe15c077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe15c074ca0 0x7fe15c077160 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe1700af000 tx=0x7fe17001afa0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 msgr2=0x7fe180152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 0x7fe180152c10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe17800eb20 tx=0x7fe17800c2d0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 shutdown_connections 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe15c074ca0 0x7fe15c077160 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe18015f490 0x7fe180152c10 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1800b2e80 0x7fe180070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 --2- 172.21.15.122:0/189445904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1800afca0 0x7fe18015ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 >> 172.21.15.122:0/189445904 conn(0x7fe1800d1e30 msgr2=0x7fe18015e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:35.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 shutdown_connections 2024-09-17T05:37:35.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.029+0000 7fe186200640 1 -- 172.21.15.122:0/189445904 wait complete. 2024-09-17T05:37:35.045 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:37:35.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:37:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.193+0000 7f5945600640 1 -- 172.21.15.122:0/2796768461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 msgr2=0x7f594015ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.193+0000 7f5945600640 1 --2- 172.21.15.122:0/2796768461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 0x7f594015ebe0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f5928005fd0 tx=0x7f592801c7c0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.193+0000 7f5945600640 1 -- 172.21.15.122:0/2796768461 shutdown_connections 2024-09-17T05:37:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.193+0000 7f5945600640 1 --2- 172.21.15.122:0/2796768461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 0x7f594015ebe0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.193+0000 7f5945600640 1 -- 172.21.15.122:0/2796768461 >> 172.21.15.122:0/2796768461 conn(0x7f59400cce90 msgr2=0x7f59400cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.193+0000 7f5945600640 1 -- 172.21.15.122:0/2796768461 shutdown_connections 2024-09-17T05:37:35.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.193+0000 7f5945600640 1 -- 172.21.15.122:0/2796768461 wait complete. 2024-09-17T05:37:35.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 Processor -- start 2024-09-17T05:37:35.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 -- start start 2024-09-17T05:37:35.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59400ce8a0 0x7f59400b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:35.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59401585f0 0x7f594014d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:35.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 0x7f5940157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:35.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5940168fb0 con 0x7f594015e790 2024-09-17T05:37:35.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5940168e30 con 0x7f59401585f0 2024-09-17T05:37:35.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5940169130 con 0x7f59400ce8a0 2024-09-17T05:37:35.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 0x7f5940157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 0x7f5940157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43778/0 (socket says 172.21.15.122:43778) 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 -- 172.21.15.122:0/3839731954 learned_addr learned my addr 172.21.15.122:0/3839731954 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593ea00640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59400ce8a0 0x7f59400b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5936000640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59401585f0 0x7f594014d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 -- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59400ce8a0 msgr2=0x7f59400b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59400ce8a0 0x7f59400b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 -- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59401585f0 msgr2=0x7f594014d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59401585f0 0x7f594014d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 -- 172.21.15.122:0/3839731954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5940157960 con 0x7f594015e790 2024-09-17T05:37:35.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f593f400640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 0x7f5940157420 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f593800cbc0 tx=0x7f593800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:35.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5937400640 1 -- 172.21.15.122:0/3839731954 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938016020 con 0x7f594015e790 2024-09-17T05:37:35.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5937400640 1 -- 172.21.15.122:0/3839731954 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f593800ab40 con 0x7f594015e790 2024-09-17T05:37:35.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5937400640 1 -- 172.21.15.122:0/3839731954 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938012420 con 0x7f594015e790 2024-09-17T05:37:35.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59400b3960 con 0x7f594015e790 2024-09-17T05:37:35.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59400b3f20 con 0x7f594015e790 2024-09-17T05:37:35.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.197+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f594015ebe0 con 0x7f594015e790 2024-09-17T05:37:35.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.201+0000 7f5937400640 1 -- 172.21.15.122:0/3839731954 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f5938019020 con 0x7f594015e790 2024-09-17T05:37:35.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.201+0000 7f5937400640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f591c074c70 0x7f591c077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:35.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.201+0000 7f5937400640 1 -- 172.21.15.122:0/3839731954 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7f5938092f20 con 0x7f594015e790 2024-09-17T05:37:35.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.201+0000 7f593ea00640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f591c074c70 0x7f591c077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:35.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.201+0000 7f593ea00640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f591c074c70 0x7f591c077130 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5930007fd0 tx=0x7f5930009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.205+0000 7f5937400640 1 -- 172.21.15.122:0/3839731954 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5938020030 con 0x7f594015e790 2024-09-17T05:37:35.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.373+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5940151650 con 0x7f594015e790 2024-09-17T05:37:35.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.373+0000 7f5937400640 1 -- 172.21.15.122:0/3839731954 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f593805f3e0 con 0x7f594015e790 2024-09-17T05:37:35.370 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f591c074c70 msgr2=0x7f591c077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f591c074c70 0x7f591c077130 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5930007fd0 tx=0x7f5930009040 comp rx=0 tx=0).stop 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 msgr2=0x7f5940157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 0x7f5940157420 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f593800cbc0 tx=0x7f593800ccb0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 shutdown_connections 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f591c074c70 0x7f591c077130 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594015e790 0x7f5940157420 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59401585f0 0x7f594014d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 --2- 172.21.15.122:0/3839731954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59400ce8a0 0x7f59400b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 >> 172.21.15.122:0/3839731954 conn(0x7f59400cce90 msgr2=0x7f594006cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 shutdown_connections 2024-09-17T05:37:35.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.377+0000 7f5945600640 1 -- 172.21.15.122:0/3839731954 wait complete. 2024-09-17T05:37:35.391 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-17T05:37:35.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.537+0000 7f6d5da00640 1 -- 172.21.15.122:0/1668613123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 msgr2=0x7f6d580ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.537+0000 7f6d5da00640 1 --2- 172.21.15.122:0/1668613123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 0x7f6d580ce670 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009bc0 tx=0x7f6d4801cb10 comp rx=0 tx=0).stop 2024-09-17T05:37:35.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.537+0000 7f6d5da00640 1 -- 172.21.15.122:0/1668613123 shutdown_connections 2024-09-17T05:37:35.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.537+0000 7f6d5da00640 1 --2- 172.21.15.122:0/1668613123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 0x7f6d580ce670 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.537+0000 7f6d5da00640 1 -- 172.21.15.122:0/1668613123 >> 172.21.15.122:0/1668613123 conn(0x7f6d580d1dc0 msgr2=0x7f6d580d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:35.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.537+0000 7f6d5da00640 1 -- 172.21.15.122:0/1668613123 shutdown_connections 2024-09-17T05:37:35.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.537+0000 7f6d5da00640 1 -- 172.21.15.122:0/1668613123 wait complete. 2024-09-17T05:37:35.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 Processor -- start 2024-09-17T05:37:35.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 -- start start 2024-09-17T05:37:35.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d580cdb30 0x7f6d580cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:35.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d580d28e0 0x7f6d580cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 0x7f6d58150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5816a930 con 0x7f6d5815bea0 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5816a7b0 con 0x7f6d580cdb30 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5816aab0 con 0x7f6d580d28e0 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 0x7f6d58150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 0x7f6d58150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43786/0 (socket says 172.21.15.122:43786) 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 -- 172.21.15.122:0/4009829543 learned_addr learned my addr 172.21.15.122:0/4009829543 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d56000640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d580d28e0 0x7f6d580cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d56a00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d580cdb30 0x7f6d580cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 -- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d580d28e0 msgr2=0x7f6d580cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d580d28e0 0x7f6d580cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 -- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d580cdb30 msgr2=0x7f6d580cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:35.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d580cdb30 0x7f6d580cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:35.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 -- 172.21.15.122:0/4009829543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5813cd60 con 0x7f6d5815bea0 2024-09-17T05:37:35.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d57400640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 0x7f6d58150b70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c3c0 tx=0x7f6d5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:35.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d4ea00640 1 -- 172.21.15.122:0/4009829543 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50011790 con 0x7f6d5815bea0 2024-09-17T05:37:35.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d4ea00640 1 -- 172.21.15.122:0/4009829543 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d50017070 con 0x7f6d5815bea0 2024-09-17T05:37:35.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d4ea00640 1 -- 172.21.15.122:0/4009829543 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50011cc0 con 0x7f6d5815bea0 2024-09-17T05:37:35.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5813d040 con 0x7f6d5815bea0 2024-09-17T05:37:35.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5813d600 con 0x7f6d5815bea0 2024-09-17T05:37:35.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.541+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d580ce670 con 0x7f6d5815bea0 2024-09-17T05:37:35.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.545+0000 7f6d4ea00640 1 -- 172.21.15.122:0/4009829543 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f6d5001c460 con 0x7f6d5815bea0 2024-09-17T05:37:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.545+0000 7f6d4ea00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d38074c70 0x7f6d38077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.545+0000 7f6d4ea00640 1 -- 172.21.15.122:0/4009829543 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7f6d50092dd0 con 0x7f6d5815bea0 2024-09-17T05:37:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.545+0000 7f6d56a00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d38074c70 0x7f6d38077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:35.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.545+0000 7f6d56a00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d38074c70 0x7f6d38077130 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6d480af000 tx=0x7f6d480b7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.549+0000 7f6d4ea00640 1 -- 172.21.15.122:0/4009829543 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6d5005f290 con 0x7f6d5815bea0 2024-09-17T05:37:35.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:35.777+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7f6d58075ae0 con 0x7f6d5815bea0 2024-09-17T05:37:36.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.685+0000 7f6d4ea00640 1 -- 172.21.15.122:0/4009829543 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v572) ==== 188+0+0 (secure 0 0 0) 0x7f6d50063cc0 con 0x7f6d5815bea0 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d38074c70 msgr2=0x7f6d38077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d38074c70 0x7f6d38077130 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6d480af000 tx=0x7f6d480b7040 comp rx=0 tx=0).stop 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 msgr2=0x7f6d58150b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 0x7f6d58150b70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c3c0 tx=0x7f6d5000c880 comp rx=0 tx=0).stop 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 shutdown_connections 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6d38074c70 0x7f6d38077130 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d5815bea0 0x7f6d58150b70 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d580d28e0 0x7f6d580cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 --2- 172.21.15.122:0/4009829543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d580cdb30 0x7f6d580cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 >> 172.21.15.122:0/4009829543 conn(0x7f6d580d1dc0 msgr2=0x7f6d580b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:36.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 shutdown_connections 2024-09-17T05:37:36.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.689+0000 7f6d5da00640 1 -- 172.21.15.122:0/4009829543 wait complete. 2024-09-17T05:37:36.709 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:37:36.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.857+0000 7f23b2400640 1 -- 172.21.15.122:0/3808697652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac0b2590 msgr2=0x7f23ac0b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:36.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.857+0000 7f23b2400640 1 --2- 172.21.15.122:0/3808697652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac0b2590 0x7f23ac0b29e0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009bc0 tx=0x7f23a401cb10 comp rx=0 tx=0).stop 2024-09-17T05:37:36.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.857+0000 7f23b2400640 1 -- 172.21.15.122:0/3808697652 shutdown_connections 2024-09-17T05:37:36.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.857+0000 7f23b2400640 1 --2- 172.21.15.122:0/3808697652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac0b2590 0x7f23ac0b29e0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:36.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.857+0000 7f23b2400640 1 --2- 172.21.15.122:0/3808697652 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23ac15f000 0x7f23ac0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:36.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.857+0000 7f23b2400640 1 -- 172.21.15.122:0/3808697652 >> 172.21.15.122:0/3808697652 conn(0x7f23ac09aa10 msgr2=0x7f23ac09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:36.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.857+0000 7f23b2400640 1 -- 172.21.15.122:0/3808697652 shutdown_connections 2024-09-17T05:37:36.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.857+0000 7f23b2400640 1 -- 172.21.15.122:0/3808697652 wait complete. 2024-09-17T05:37:36.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 Processor -- start 2024-09-17T05:37:36.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 -- start start 2024-09-17T05:37:36.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23ac0b2590 0x7f23ac0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:36.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23ac0cefb0 0x7f23ac1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:36.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac15f000 0x7f23ac0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:36.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac168e50 con 0x7f23ac15f000 2024-09-17T05:37:36.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac168cd0 con 0x7f23ac0b2590 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f23ac168fd0 con 0x7f23ac0cefb0 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac15f000 0x7f23ac0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac15f000 0x7f23ac0b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43806/0 (socket says 172.21.15.122:43806) 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 -- 172.21.15.122:0/276426235 learned_addr learned my addr 172.21.15.122:0/276426235 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23a3e00640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23ac0cefb0 0x7f23ac1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1000640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23ac0b2590 0x7f23ac0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 -- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23ac0cefb0 msgr2=0x7f23ac1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23ac0cefb0 0x7f23ac1519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 -- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23ac0b2590 msgr2=0x7f23ac0c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23ac0b2590 0x7f23ac0c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:36.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 -- 172.21.15.122:0/276426235 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac0b6c10 con 0x7f23ac15f000 2024-09-17T05:37:36.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b1a00640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac15f000 0x7f23ac0b5db0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f23a40b0040 tx=0x7f23a4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:36.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23a9600640 1 -- 172.21.15.122:0/276426235 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a40b2040 con 0x7f23ac15f000 2024-09-17T05:37:36.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23a9600640 1 -- 172.21.15.122:0/276426235 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23a40bb070 con 0x7f23ac15f000 2024-09-17T05:37:36.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23a9600640 1 -- 172.21.15.122:0/276426235 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4004370 con 0x7f23ac15f000 2024-09-17T05:37:36.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.861+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac158e10 con 0x7f23ac15f000 2024-09-17T05:37:36.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.865+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac0b4e80 con 0x7f23ac15f000 2024-09-17T05:37:36.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.865+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac0b29e0 con 0x7f23ac15f000 2024-09-17T05:37:36.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.865+0000 7f23a9600640 1 -- 172.21.15.122:0/276426235 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f23a4004510 con 0x7f23ac15f000 2024-09-17T05:37:36.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.865+0000 7f23a9600640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2388074ca0 0x7f2388077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:36.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.865+0000 7f23a9600640 1 -- 172.21.15.122:0/276426235 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7f23a412d8f0 con 0x7f23ac15f000 2024-09-17T05:37:36.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.865+0000 7f23b1000640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2388074ca0 0x7f2388077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:36.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.869+0000 7f23b1000640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2388074ca0 0x7f2388077160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f239c009b40 tx=0x7f239c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:36.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:36.869+0000 7f23a9600640 1 -- 172.21.15.122:0/276426235 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f23a4016790 con 0x7f23ac15f000 2024-09-17T05:37:37.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.037+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f23ac0cc1b0 con 0x7f23ac15f000 2024-09-17T05:37:37.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23a9600640 1 -- 172.21.15.122:0/276426235 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7f23a40f9d20 con 0x7f23ac15f000 2024-09-17T05:37:37.035 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:37:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:37:37.037 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:37:37.038 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:37:37.038 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2388074ca0 msgr2=0x7f2388077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2388074ca0 0x7f2388077160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f239c009b40 tx=0x7f239c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac15f000 msgr2=0x7f23ac0b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac15f000 0x7f23ac0b5db0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f23a40b0040 tx=0x7f23a4005b00 comp rx=0 tx=0).stop 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 shutdown_connections 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2388074ca0 0x7f2388077160 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23ac15f000 0x7f23ac0b5db0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23ac0cefb0 0x7f23ac1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 --2- 172.21.15.122:0/276426235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23ac0b2590 0x7f23ac0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 >> 172.21.15.122:0/276426235 conn(0x7f23ac09aa10 msgr2=0x7f23ac0ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 shutdown_connections 2024-09-17T05:37:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.041+0000 7f23b2400640 1 -- 172.21.15.122:0/276426235 wait complete. 2024-09-17T05:37:37.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:37:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.205+0000 7fe157200640 1 -- 172.21.15.122:0/771431997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe13c004810 msgr2=0x7fe13c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.205+0000 7fe157200640 1 --2- 172.21.15.122:0/771431997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe13c004810 0x7fe13c004c90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe140009b30 tx=0x7fe14001d230 comp rx=0 tx=0).stop 2024-09-17T05:37:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.205+0000 7fe157200640 1 -- 172.21.15.122:0/771431997 shutdown_connections 2024-09-17T05:37:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.205+0000 7fe157200640 1 --2- 172.21.15.122:0/771431997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe13c004810 0x7fe13c004c90 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.205+0000 7fe157200640 1 -- 172.21.15.122:0/771431997 >> 172.21.15.122:0/771431997 conn(0x7fe15809aa10 msgr2=0x7fe15809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.205+0000 7fe157200640 1 -- 172.21.15.122:0/771431997 shutdown_connections 2024-09-17T05:37:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.205+0000 7fe157200640 1 -- 172.21.15.122:0/771431997 wait complete. 2024-09-17T05:37:37.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 Processor -- start 2024-09-17T05:37:37.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 -- start start 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c004810 0x7fe158157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe158071880 0x7fe158071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1580b6290 0x7fe1580b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1580d03c0 con 0x7fe158071880 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe1580d0240 con 0x7fe13c004810 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe1580d0540 con 0x7fe1580b6290 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe158071880 0x7fe158071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe158071880 0x7fe158071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43838/0 (socket says 172.21.15.122:43838) 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 -- 172.21.15.122:0/4056892075 learned_addr learned my addr 172.21.15.122:0/4056892075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe155e00640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c004810 0x7fe158157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe156800640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1580b6290 0x7fe1580b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:37.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 -- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1580b6290 msgr2=0x7fe1580b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1580b6290 0x7fe1580b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 -- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c004810 msgr2=0x7fe158157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c004810 0x7fe158157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 -- 172.21.15.122:0/4056892075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1580b6c50 con 0x7fe158071880 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14d400640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe158071880 0x7fe158071d00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe140009a80 tx=0x7fe140005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14ea00640 1 -- 172.21.15.122:0/4056892075 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400b1040 con 0x7fe158071880 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14ea00640 1 -- 172.21.15.122:0/4056892075 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1400bc070 con 0x7fe158071880 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe14ea00640 1 -- 172.21.15.122:0/4056892075 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1400acb40 con 0x7fe158071880 2024-09-17T05:37:37.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1580c9f20 con 0x7fe158071880 2024-09-17T05:37:37.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1580ca4e0 con 0x7fe158071880 2024-09-17T05:37:37.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.209+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe158158120 con 0x7fe158071880 2024-09-17T05:37:37.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.213+0000 7fe14ea00640 1 -- 172.21.15.122:0/4056892075 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fe1400acce0 con 0x7fe158071880 2024-09-17T05:37:37.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.213+0000 7fe14ea00640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe134074ca0 0x7fe134077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:37.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.213+0000 7fe14ea00640 1 -- 172.21.15.122:0/4056892075 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7fe14012e8d0 con 0x7fe158071880 2024-09-17T05:37:37.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.213+0000 7fe155e00640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe134074ca0 0x7fe134077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:37.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.213+0000 7fe155e00640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe134074ca0 0x7fe134077160 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe148001ec0 tx=0x7fe148006730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:37.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.217+0000 7fe14ea00640 1 -- 172.21.15.122:0/4056892075 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe140016f40 con 0x7fe158071880 2024-09-17T05:37:37.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.381+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe1580b3f00 con 0x7fe158071880 2024-09-17T05:37:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.385+0000 7fe14ea00640 1 -- 172.21.15.122:0/4056892075 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7fe1400fae30 con 0x7fe158071880 2024-09-17T05:37:37.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe134074ca0 msgr2=0x7fe134077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe134074ca0 0x7fe134077160 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe148001ec0 tx=0x7fe148006730 comp rx=0 tx=0).stop 2024-09-17T05:37:37.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe158071880 msgr2=0x7fe158071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe158071880 0x7fe158071d00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe140009a80 tx=0x7fe140005640 comp rx=0 tx=0).stop 2024-09-17T05:37:37.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 shutdown_connections 2024-09-17T05:37:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe134074ca0 0x7fe134077160 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1580b6290 0x7fe1580b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe158071880 0x7fe158071d00 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 --2- 172.21.15.122:0/4056892075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe13c004810 0x7fe158157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 >> 172.21.15.122:0/4056892075 conn(0x7fe15809aa10 msgr2=0x7fe15806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 shutdown_connections 2024-09-17T05:37:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.389+0000 7fe157200640 1 -- 172.21.15.122:0/4056892075 wait complete. 2024-09-17T05:37:37.405 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:37:37.405 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:37:37.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:37:37.610 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:37:37.611 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:37:37.611 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:37.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:37:37.621 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:37:37.621 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:37.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:37:37.630 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:37:37.630 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:37.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:37:37.671 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:37:37.671 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:37.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:37:37.684 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:37:37.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:37:37.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- 172.21.15.122:0/2100610118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc18c15f4e0 msgr2=0x7fc18c0cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 --2- 172.21.15.122:0/2100610118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc18c15f4e0 0x7fc18c0cd0d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc184009bc0 tx=0x7fc18401cac0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- 172.21.15.122:0/2100610118 shutdown_connections 2024-09-17T05:37:37.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 --2- 172.21.15.122:0/2100610118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc18c15f4e0 0x7fc18c0cd0d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- 172.21.15.122:0/2100610118 >> 172.21.15.122:0/2100610118 conn(0x7fc18c0b18f0 msgr2=0x7fc18c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:37.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- 172.21.15.122:0/2100610118 shutdown_connections 2024-09-17T05:37:37.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- 172.21.15.122:0/2100610118 wait complete. 2024-09-17T05:37:37.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 Processor -- start 2024-09-17T05:37:37.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- start start 2024-09-17T05:37:37.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc18c0b3300 0x7fc18c15a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:37.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc18c15af90 0x7fc18c1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:37.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc18c15f4e0 0x7fc18c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:37.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c1698b0 con 0x7fc18c15af90 2024-09-17T05:37:37.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c169730 con 0x7fc18c0b3300 2024-09-17T05:37:37.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc190c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc18c169a30 con 0x7fc18c15f4e0 2024-09-17T05:37:37.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc181400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc18c15af90 0x7fc18c1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:37.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc181400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc18c15af90 0x7fc18c1542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43852/0 (socket says 172.21.15.122:43852) 2024-09-17T05:37:37.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.865+0000 7fc181400640 1 -- 172.21.15.122:0/991028379 learned_addr learned my addr 172.21.15.122:0/991028379 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:37.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc18a800640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc18c15f4e0 0x7fc18c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:37.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc189e00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc18c0b3300 0x7fc18c15a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:37.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc181400640 1 -- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc18c15f4e0 msgr2=0x7fc18c152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc181400640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc18c15f4e0 0x7fc18c152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc181400640 1 -- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc18c0b3300 msgr2=0x7fc18c15a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:37.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc181400640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc18c0b3300 0x7fc18c15a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:37.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc181400640 1 -- 172.21.15.122:0/991028379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c153560 con 0x7fc18c15af90 2024-09-17T05:37:37.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc181400640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc18c15af90 0x7fc18c1542e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc174001230 tx=0x7fc17400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:37.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc182a00640 1 -- 172.21.15.122:0/991028379 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174016020 con 0x7fc18c15af90 2024-09-17T05:37:37.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc182a00640 1 -- 172.21.15.122:0/991028379 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc17400a930 con 0x7fc18c15af90 2024-09-17T05:37:37.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18c153800 con 0x7fc18c15af90 2024-09-17T05:37:37.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc182a00640 1 -- 172.21.15.122:0/991028379 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17400ad80 con 0x7fc18c15af90 2024-09-17T05:37:37.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18c153dc0 con 0x7fc18c15af90 2024-09-17T05:37:37.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18c069410 con 0x7fc18c15af90 2024-09-17T05:37:37.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.869+0000 7fc182a00640 1 -- 172.21.15.122:0/991028379 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fc174019020 con 0x7fc18c15af90 2024-09-17T05:37:37.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.873+0000 7fc182a00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc168074c70 0x7fc168077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:37.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.873+0000 7fc182a00640 1 -- 172.21.15.122:0/991028379 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7fc174092df0 con 0x7fc18c15af90 2024-09-17T05:37:37.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.873+0000 7fc189e00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc168074c70 0x7fc168077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:37.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.873+0000 7fc189e00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc168074c70 0x7fc168077130 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007fd0 tx=0x7fc17c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:37.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:37.877+0000 7fc182a00640 1 -- 172.21.15.122:0/991028379 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc174020030 con 0x7fc18c15af90 2024-09-17T05:37:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.077+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc18c0cf9c0 con 0x7fc18c15af90 2024-09-17T05:37:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.077+0000 7fc182a00640 1 -- 172.21.15.122:0/991028379 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3923 (secure 0 0 0) 0x7fc17405f2b0 con 0x7fc18c15af90 2024-09-17T05:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:38.074 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":69,"btime":"2024-09-17T05:37:31:665197+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7237,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1758247367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1758247367},{"type":"v1","addr":"172.21.15.173:6836","nonce":1758247367}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7240,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2422841556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2422841556},{"type":"v1","addr":"172.21.15.173:6837","nonce":2422841556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7857,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/4290616350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":4290616350},{"type":"v1","addr":"172.21.15.122:6834","nonce":4290616350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":8274,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3498663365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":3498663365},{"type":"v1","addr":"172.21.15.122:6835","nonce":3498663365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-09-17T05:37:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 69 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc168074c70 msgr2=0x7fc168077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc168074c70 0x7fc168077130 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007fd0 tx=0x7fc17c009040 comp rx=0 tx=0).stop 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc18c15af90 msgr2=0x7fc18c1542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc18c15af90 0x7fc18c1542e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc174001230 tx=0x7fc17400cbb0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 shutdown_connections 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc168074c70 0x7fc168077130 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc18c15f4e0 0x7fc18c152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc18c15af90 0x7fc18c1542e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 --2- 172.21.15.122:0/991028379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc18c0b3300 0x7fc18c15a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 >> 172.21.15.122:0/991028379 conn(0x7fc18c0b18f0 msgr2=0x7fc18c0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 shutdown_connections 2024-09-17T05:37:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.081+0000 7fc190c00640 1 -- 172.21.15.122:0/991028379 wait complete. 2024-09-17T05:37:38.094 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:37:38.094 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:37:38.095 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:37:38.095 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:37:38.095 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:37:38.095 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:37:38.095 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:37:38.095 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:37:38.095 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:37:38.096 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:37:38.096 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:37:38.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:37:38.096 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:37:38.096 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:37:38.096 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:37:38.097 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:37:38.107 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:37:38.107 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:37:38.108 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:37:38.108 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:37:38.113 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:37:38.113 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:37:38.114 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:37:38.114 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:37:38.153 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:37:38.154 DEBUG:teuthology.parallel:result is None 2024-09-17T05:37:38.155 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:37:38.155 DEBUG:teuthology.parallel:result is None 2024-09-17T05:37:38.161 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:37:38.162 DEBUG:teuthology.parallel:result is None 2024-09-17T05:37:38.164 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:37:38.164 DEBUG:teuthology.parallel:result is None 2024-09-17T05:37:38.164 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:37:38.239 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:37:38.240 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:37:38.254 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:37:38.255 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:37:38.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 -- 172.21.15.122:0/3538478914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15b890 msgr2=0x7f51ac0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 --2- 172.21.15.122:0/3538478914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15b890 0x7f51ac0ce010 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0a21b0 tx=0x7f519c01c890 comp rx=0 tx=0).stop 2024-09-17T05:37:38.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 -- 172.21.15.122:0/3538478914 shutdown_connections 2024-09-17T05:37:38.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 --2- 172.21.15.122:0/3538478914 >> v2:172.21.15.173:3300/0 conn(0x7f51ac15bc70 0x7f51ac0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 --2- 172.21.15.122:0/3538478914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51ac06cc20 0x7f51ac06d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 --2- 172.21.15.122:0/3538478914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15b890 0x7f51ac0ce010 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 -- 172.21.15.122:0/3538478914 >> 172.21.15.122:0/3538478914 conn(0x7f51ac06dd30 msgr2=0x7f51ac06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:38.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 -- 172.21.15.122:0/3538478914 shutdown_connections 2024-09-17T05:37:38.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 -- 172.21.15.122:0/3538478914 wait complete. 2024-09-17T05:37:38.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 Processor -- start 2024-09-17T05:37:38.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.317+0000 7f51b1000640 1 -- start start 2024-09-17T05:37:38.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51ac06cc20 0x7f51ac15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:38.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51ac15b890 0x7f51ac0c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:38.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15bc70 0x7f51ac0c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:38.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac074620 con 0x7f51ac15bc70 2024-09-17T05:37:38.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac0744a0 con 0x7f51ac06cc20 2024-09-17T05:37:38.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f51ac0cb150 con 0x7f51ac15b890 2024-09-17T05:37:38.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51abe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15bc70 0x7f51ac0c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:38.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51ab400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51ac06cc20 0x7f51ac15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:38.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51aaa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51ac15b890 0x7f51ac0c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:38.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51abe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15bc70 0x7f51ac0c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43896/0 (socket says 172.21.15.122:43896) 2024-09-17T05:37:38.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51ab400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51ac06cc20 0x7f51ac15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:37778/0 (socket says 172.21.15.122:37778) 2024-09-17T05:37:38.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51abe00640 1 -- 172.21.15.122:0/2986837977 learned_addr learned my addr 172.21.15.122:0/2986837977 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:38.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51aaa00640 1 -- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51ac06cc20 msgr2=0x7f51ac15aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51aaa00640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51ac06cc20 0x7f51ac15aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51aaa00640 1 -- 172.21.15.122:0/2986837977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15bc70 msgr2=0x7f51ac0c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51aaa00640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15bc70 0x7f51ac0c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51aaa00640 1 -- 172.21.15.122:0/2986837977 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac15af60 con 0x7f51ac15b890 2024-09-17T05:37:38.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51abe00640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15bc70 0x7f51ac0c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:37:38.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51aaa00640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51ac15b890 0x7f51ac0c5420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f519800d660 tx=0x7f519800db20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:38.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51a3400640 1 -- 172.21.15.122:0/2986837977 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198017070 con 0x7f51ac15b890 2024-09-17T05:37:38.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51a3400640 1 -- 172.21.15.122:0/2986837977 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5198009b10 con 0x7f51ac15b890 2024-09-17T05:37:38.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51a3400640 1 -- 172.21.15.122:0/2986837977 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198010460 con 0x7f51ac15b890 2024-09-17T05:37:38.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ac0b2200 con 0x7f51ac15b890 2024-09-17T05:37:38.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ac0c6580 con 0x7f51ac15b890 2024-09-17T05:37:38.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.321+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ac0cdff0 con 0x7f51ac15b890 2024-09-17T05:37:38.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.325+0000 7f51a3400640 1 -- 172.21.15.122:0/2986837977 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f519801a070 con 0x7f51ac15b890 2024-09-17T05:37:38.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.325+0000 7f51a3400640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f518c074c70 0x7f518c077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:38.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.325+0000 7f51a3400640 1 -- 172.21.15.122:0/2986837977 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(109..109 src has 1..109) ==== 5027+0+0 (secure 0 0 0) 0x7f5198093230 con 0x7f51ac15b890 2024-09-17T05:37:38.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.325+0000 7f51ab400640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f518c074c70 0x7f518c077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:38.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.325+0000 7f51ab400640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f518c074c70 0x7f518c077130 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f519c0af000 tx=0x7f519c01b0d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:38.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.329+0000 7f51a3400640 1 -- 172.21.15.122:0/2986837977 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f519805f770 con 0x7f51ac15b890 2024-09-17T05:37:38.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.505+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f51ac0cbd80 con 0x7f51ac15b890 2024-09-17T05:37:38.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.709+0000 7f51a3400640 1 -- 172.21.15.122:0/2986837977 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) ==== 97+0+0 (secure 0 0 0) 0x7f51980641a0 con 0x7f51ac15b890 2024-09-17T05:37:38.705 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:37:38.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f518c074c70 msgr2=0x7f518c077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f518c074c70 0x7f518c077130 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f519c0af000 tx=0x7f519c01b0d0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51ac15b890 msgr2=0x7f51ac0c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51ac15b890 0x7f51ac0c5420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f519800d660 tx=0x7f519800db20 comp rx=0 tx=0).stop 2024-09-17T05:37:38.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 shutdown_connections 2024-09-17T05:37:38.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f518c074c70 0x7f518c077130 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51ac15bc70 0x7f51ac0c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51ac15b890 0x7f51ac0c5420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 --2- 172.21.15.122:0/2986837977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51ac06cc20 0x7f51ac15aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 >> 172.21.15.122:0/2986837977 conn(0x7f51ac06dd30 msgr2=0x7f51ac165720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:38.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 shutdown_connections 2024-09-17T05:37:38.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.713+0000 7f51b1000640 1 -- 172.21.15.122:0/2986837977 wait complete. 2024-09-17T05:37:38.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 -- 172.21.15.122:0/1473542883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0b53f0 msgr2=0x7feacc0b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 --2- 172.21.15.122:0/1473542883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0b53f0 0x7feacc0b5860 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7feacc073d80 tx=0x7feac401cd80 comp rx=0 tx=0).stop 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 -- 172.21.15.122:0/1473542883 shutdown_connections 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 --2- 172.21.15.122:0/1473542883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0b53f0 0x7feacc0b5860 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 --2- 172.21.15.122:0/1473542883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feacc0d28c0 0x7feacc0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 --2- 172.21.15.122:0/1473542883 >> v2:172.21.15.173:3300/0 conn(0x7feacc15be80 0x7feacc0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 -- 172.21.15.122:0/1473542883 >> 172.21.15.122:0/1473542883 conn(0x7feacc0d1da0 msgr2=0x7feacc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 -- 172.21.15.122:0/1473542883 shutdown_connections 2024-09-17T05:37:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.873+0000 7fead3400640 1 -- 172.21.15.122:0/1473542883 wait complete. 2024-09-17T05:37:38.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 Processor -- start 2024-09-17T05:37:38.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 -- start start 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feacc0b53f0 0x7feacc1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0d28c0 0x7feacc1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feacc15be80 0x7feacc0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feacc0cd000 con 0x7feacc0d28c0 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feacc0cce80 con 0x7feacc15be80 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7feacc0cd180 con 0x7feacc0b53f0 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0d28c0 0x7feacc1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0d28c0 0x7feacc1524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43914/0 (socket says 172.21.15.122:43914) 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 -- 172.21.15.122:0/2789449500 learned_addr learned my addr 172.21.15.122:0/2789449500 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:38.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead1400640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feacc15be80 0x7feacc0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead0a00640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feacc0b53f0 0x7feacc1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 -- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feacc0b53f0 msgr2=0x7feacc1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feacc0b53f0 0x7feacc1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 -- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feacc15be80 msgr2=0x7feacc0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feacc15be80 0x7feacc0c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 -- 172.21.15.122:0/2789449500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc15c5a0 con 0x7feacc0d28c0 2024-09-17T05:37:38.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac3e00640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0d28c0 0x7feacc1524e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7feab400c9a0 tx=0x7feab400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:38.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac9600640 1 -- 172.21.15.122:0/2789449500 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4007c70 con 0x7feacc0d28c0 2024-09-17T05:37:38.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac9600640 1 -- 172.21.15.122:0/2789449500 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feab400f450 con 0x7feacc0d28c0 2024-09-17T05:37:38.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7feac9600640 1 -- 172.21.15.122:0/2789449500 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab400f8a0 con 0x7feacc0d28c0 2024-09-17T05:37:38.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feacc152f00 con 0x7feacc0d28c0 2024-09-17T05:37:38.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.877+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feacc06ec50 con 0x7feacc0d28c0 2024-09-17T05:37:38.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.881+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feacc0b5860 con 0x7feacc0d28c0 2024-09-17T05:37:38.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.881+0000 7feac9600640 1 -- 172.21.15.122:0/2789449500 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7feab400fa40 con 0x7feacc0d28c0 2024-09-17T05:37:38.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.881+0000 7feac9600640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feaa8074c70 0x7feaa8077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:38.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.881+0000 7feac9600640 1 -- 172.21.15.122:0/2789449500 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4941+0+0 (secure 0 0 0) 0x7feab4093810 con 0x7feacc0d28c0 2024-09-17T05:37:38.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.881+0000 7fead0a00640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feaa8074c70 0x7feaa8077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:38.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.881+0000 7fead0a00640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feaa8074c70 0x7feaa8077130 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feacc0ca6b0 tx=0x7feabc00c440 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:38.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:38.885+0000 7feac9600640 1 -- 172.21.15.122:0/2789449500 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7feab4014030 con 0x7feacc0d28c0 2024-09-17T05:37:39.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.109+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7feacc06d0d0 con 0x7feacc0d28c0 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.109+0000 7feac9600640 1 -- 172.21.15.122:0/2789449500 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7feab40603a0 con 0x7feacc0d28c0 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:37:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:37:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:37:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:37:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:37:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow r", 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:37:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:37:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:37:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:37:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:37:39.121 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feaa8074c70 msgr2=0x7feaa8077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feaa8074c70 0x7feaa8077130 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feacc0ca6b0 tx=0x7feabc00c440 comp rx=0 tx=0).stop 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0d28c0 msgr2=0x7feacc1524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0d28c0 0x7feacc1524e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7feab400c9a0 tx=0x7feab400ce60 comp rx=0 tx=0).stop 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 shutdown_connections 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feaa8074c70 0x7feaa8077130 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feacc15be80 0x7feacc0c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feacc0d28c0 0x7feacc1524e0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 --2- 172.21.15.122:0/2789449500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feacc0b53f0 0x7feacc1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 >> 172.21.15.122:0/2789449500 conn(0x7feacc0d1da0 msgr2=0x7feacc0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 shutdown_connections 2024-09-17T05:37:39.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.113+0000 7fead3400640 1 -- 172.21.15.122:0/2789449500 wait complete. 2024-09-17T05:37:39.128 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:37:39.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 -- 172.21.15.122:0/2136259601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf480cdc20 msgr2=0x7fcf480cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 --2- 172.21.15.122:0/2136259601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf480cdc20 0x7fcf480cb360 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcf340098f0 tx=0x7fcf3401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 -- 172.21.15.122:0/2136259601 shutdown_connections 2024-09-17T05:37:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 --2- 172.21.15.122:0/2136259601 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf480cb8a0 0x7fcf480cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 --2- 172.21.15.122:0/2136259601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf480cdc20 0x7fcf480cb360 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 -- 172.21.15.122:0/2136259601 >> 172.21.15.122:0/2136259601 conn(0x7fcf4806dd30 msgr2=0x7fcf4806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 -- 172.21.15.122:0/2136259601 shutdown_connections 2024-09-17T05:37:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 -- 172.21.15.122:0/2136259601 wait complete. 2024-09-17T05:37:39.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.273+0000 7fcf47200640 1 Processor -- start 2024-09-17T05:37:39.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 -- start start 2024-09-17T05:37:39.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf480cb8a0 0x7fcf48154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:39.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf480cdc20 0x7fcf4815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:39.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48155080 0x7fcf4815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:39.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48167b70 con 0x7fcf48155080 2024-09-17T05:37:39.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf480cf9e0 con 0x7fcf480cb8a0 2024-09-17T05:37:39.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcf48167cf0 con 0x7fcf480cdc20 2024-09-17T05:37:39.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48155080 0x7fcf4815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:39.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48155080 0x7fcf4815dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43916/0 (socket says 172.21.15.122:43916) 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 -- 172.21.15.122:0/3804821971 learned_addr learned my addr 172.21.15.122:0/3804821971 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf45e00640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf480cb8a0 0x7fcf48154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf45400640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf480cdc20 0x7fcf4815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 -- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf480cdc20 msgr2=0x7fcf4815d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf480cdc20 0x7fcf4815d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 -- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf480cb8a0 msgr2=0x7fcf48154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf480cb8a0 0x7fcf48154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 -- 172.21.15.122:0/3804821971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf480b11f0 con 0x7fcf48155080 2024-09-17T05:37:39.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf46800640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48155080 0x7fcf4815dca0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000eab0 tx=0x7fcf4000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf3e000640 1 -- 172.21.15.122:0/3804821971 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000cea0 con 0x7fcf48155080 2024-09-17T05:37:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf3e000640 1 -- 172.21.15.122:0/3804821971 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf40004b30 con 0x7fcf48155080 2024-09-17T05:37:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf480b23c0 con 0x7fcf48155080 2024-09-17T05:37:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf3e000640 1 -- 172.21.15.122:0/3804821971 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40005020 con 0x7fcf48155080 2024-09-17T05:37:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf480b3cd0 con 0x7fcf48155080 2024-09-17T05:37:39.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.277+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf480b3f10 con 0x7fcf48155080 2024-09-17T05:37:39.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.281+0000 7fcf3e000640 1 -- 172.21.15.122:0/3804821971 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fcf400051c0 con 0x7fcf48155080 2024-09-17T05:37:39.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.281+0000 7fcf3e000640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf28074c70 0x7fcf28077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:39.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.281+0000 7fcf3e000640 1 -- 172.21.15.122:0/3804821971 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4941+0+0 (secure 0 0 0) 0x7fcf40093c50 con 0x7fcf48155080 2024-09-17T05:37:39.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.281+0000 7fcf45e00640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf28074c70 0x7fcf28077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:39.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.281+0000 7fcf45e00640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf28074c70 0x7fcf28077130 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcf38007fd0 tx=0x7fcf38009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:39.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.289+0000 7fcf3e000640 1 -- 172.21.15.122:0/3804821971 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcf480b3f10 con 0x7fcf48155080 2024-09-17T05:37:39.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.457+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcf480cea20 con 0x7fcf48155080 2024-09-17T05:37:39.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.457+0000 7fcf3e000640 1 -- 172.21.15.122:0/3804821971 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) ==== 84+0+65 (secure 0 0 0) 0x7fcf40060090 con 0x7fcf48155080 2024-09-17T05:37:39.455 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:39.455 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:37:39.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:39.455 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:37:39.455 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:37:39.455 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:39.455 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:37:39.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf28074c70 msgr2=0x7fcf28077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf28074c70 0x7fcf28077130 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcf38007fd0 tx=0x7fcf38009040 comp rx=0 tx=0).stop 2024-09-17T05:37:39.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48155080 msgr2=0x7fcf4815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48155080 0x7fcf4815dca0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000eab0 tx=0x7fcf4000ef70 comp rx=0 tx=0).stop 2024-09-17T05:37:39.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 shutdown_connections 2024-09-17T05:37:39.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf28074c70 0x7fcf28077130 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48155080 0x7fcf4815dca0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf480cdc20 0x7fcf4815d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 --2- 172.21.15.122:0/3804821971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf480cb8a0 0x7fcf48154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 >> 172.21.15.122:0/3804821971 conn(0x7fcf4806dd30 msgr2=0x7fcf48159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:39.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 shutdown_connections 2024-09-17T05:37:39.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.461+0000 7fcf47200640 1 -- 172.21.15.122:0/3804821971 wait complete. 2024-09-17T05:37:39.477 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:37:39.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:37:39.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- 172.21.15.122:0/3005752093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 msgr2=0x7f1a3c0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 --2- 172.21.15.122:0/3005752093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 0x7f1a3c0ce700 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c074e00 tx=0x7f1a2c01f580 comp rx=0 tx=0).stop 2024-09-17T05:37:39.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- 172.21.15.122:0/3005752093 shutdown_connections 2024-09-17T05:37:39.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 --2- 172.21.15.122:0/3005752093 >> v2:172.21.15.173:3300/0 conn(0x7f1a3c0b0b60 0x7f1a3c0cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 --2- 172.21.15.122:0/3005752093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3c15c1f0 0x7f1a3c0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 --2- 172.21.15.122:0/3005752093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 0x7f1a3c0ce700 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- 172.21.15.122:0/3005752093 >> 172.21.15.122:0/3005752093 conn(0x7f1a3c0d1da0 msgr2=0x7f1a3c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:39.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- 172.21.15.122:0/3005752093 shutdown_connections 2024-09-17T05:37:39.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- 172.21.15.122:0/3005752093 wait complete. 2024-09-17T05:37:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 Processor -- start 2024-09-17T05:37:39.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- start start 2024-09-17T05:37:39.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3c0b0b60 0x7f1a3c071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:39.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 0x7f1a3c15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:39.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a3c15c1f0 0x7f1a3c0c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:39.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1600a0 con 0x7f1a3c15be10 2024-09-17T05:37:39.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c15ff20 con 0x7f1a3c15c1f0 2024-09-17T05:37:39.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a43a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3c160220 con 0x7f1a3c0b0b60 2024-09-17T05:37:39.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 0x7f1a3c15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 0x7f1a3c15d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43930/0 (socket says 172.21.15.122:43930) 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a3be00640 1 -- 172.21.15.122:0/2964436301 learned_addr learned my addr 172.21.15.122:0/2964436301 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a41000640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3c0b0b60 0x7f1a3c071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.629+0000 7f1a41a00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a3c15c1f0 0x7f1a3c0c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a3be00640 1 -- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3c0b0b60 msgr2=0x7f1a3c071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a3be00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3c0b0b60 0x7f1a3c071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a3be00640 1 -- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a3c15c1f0 msgr2=0x7f1a3c0c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a3be00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a3c15c1f0 0x7f1a3c0c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a3be00640 1 -- 172.21.15.122:0/2964436301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c0c5f20 con 0x7f1a3c15be10 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a3be00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 0x7f1a3c15d790 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800d950 tx=0x7f1a2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a38c00640 1 -- 172.21.15.122:0/2964436301 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28017070 con 0x7f1a3c15be10 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a38c00640 1 -- 172.21.15.122:0/2964436301 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a28009d20 con 0x7f1a3c15be10 2024-09-17T05:37:39.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a38c00640 1 -- 172.21.15.122:0/2964436301 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a280106c0 con 0x7f1a3c15be10 2024-09-17T05:37:39.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c0c6200 con 0x7f1a3c15be10 2024-09-17T05:37:39.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c071a60 con 0x7f1a3c15be10 2024-09-17T05:37:39.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3c070350 con 0x7f1a3c15be10 2024-09-17T05:37:39.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a38c00640 1 -- 172.21.15.122:0/2964436301 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f1a2801a070 con 0x7f1a3c15be10 2024-09-17T05:37:39.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.633+0000 7f1a38c00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a1c074c70 0x7f1a1c077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:39.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.637+0000 7f1a38c00640 1 -- 172.21.15.122:0/2964436301 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4941+0+0 (secure 0 0 0) 0x7f1a280931b0 con 0x7f1a3c15be10 2024-09-17T05:37:39.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.637+0000 7f1a41000640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a1c074c70 0x7f1a1c077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:39.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.637+0000 7f1a41000640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a1c074c70 0x7f1a1c077130 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0b2000 tx=0x7f1a2c01ddc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:39.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.637+0000 7f1a38c00640 1 -- 172.21.15.122:0/2964436301 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1a2805f670 con 0x7f1a3c15be10 2024-09-17T05:37:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:39.809+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1a3c0aee70 con 0x7f1a3c15be10 2024-09-17T05:37:40.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.713+0000 7f1a38c00640 1 -- 172.21.15.122:0/2964436301 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) ==== 138+0+0 (secure 0 0 0) 0x7f1a280640a0 con 0x7f1a3c15be10 2024-09-17T05:37:40.711 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:37:40.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a1c074c70 msgr2=0x7f1a1c077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:40.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a1c074c70 0x7f1a1c077130 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0b2000 tx=0x7f1a2c01ddc0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 msgr2=0x7f1a3c15d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:40.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 0x7f1a3c15d790 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800d950 tx=0x7f1a2800de10 comp rx=0 tx=0).stop 2024-09-17T05:37:40.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 shutdown_connections 2024-09-17T05:37:40.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a1c074c70 0x7f1a1c077130 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a3c15c1f0 0x7f1a3c0c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a3c15be10 0x7f1a3c15d790 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 --2- 172.21.15.122:0/2964436301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3c0b0b60 0x7f1a3c071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 >> 172.21.15.122:0/2964436301 conn(0x7f1a3c0d1da0 msgr2=0x7f1a3c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:40.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 shutdown_connections 2024-09-17T05:37:40.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.717+0000 7f1a43a00640 1 -- 172.21.15.122:0/2964436301 wait complete. 2024-09-17T05:37:40.740 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:37:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.893+0000 7f2663600640 1 -- 172.21.15.122:0/2751321573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c1605d0 msgr2=0x7f265c0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.893+0000 7f2663600640 1 --2- 172.21.15.122:0/2751321573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c1605d0 0x7f265c0d0690 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f264c009840 tx=0x7f264c01c820 comp rx=0 tx=0).stop 2024-09-17T05:37:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.893+0000 7f2663600640 1 -- 172.21.15.122:0/2751321573 shutdown_connections 2024-09-17T05:37:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.893+0000 7f2663600640 1 --2- 172.21.15.122:0/2751321573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f265c0afb10 0x7f265c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.893+0000 7f2663600640 1 --2- 172.21.15.122:0/2751321573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c1605d0 0x7f265c0d0690 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.893+0000 7f2663600640 1 -- 172.21.15.122:0/2751321573 >> 172.21.15.122:0/2751321573 conn(0x7f265c0d1dd0 msgr2=0x7f265c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.893+0000 7f2663600640 1 -- 172.21.15.122:0/2751321573 shutdown_connections 2024-09-17T05:37:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.893+0000 7f2663600640 1 -- 172.21.15.122:0/2751321573 wait complete. 2024-09-17T05:37:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 Processor -- start 2024-09-17T05:37:40.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 -- start start 2024-09-17T05:37:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c0afb10 0x7f265c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f265c0b1650 0x7f265c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f265c1605d0 0x7f265c153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f265c0b13a0 con 0x7f265c0afb10 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f265c0b1220 con 0x7f265c1605d0 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f265c169540 con 0x7f265c0b1650 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c0afb10 0x7f265c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c0afb10 0x7f265c14f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43940/0 (socket says 172.21.15.122:43940) 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 -- 172.21.15.122:0/681078922 learned_addr learned my addr 172.21.15.122:0/681078922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 -- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f265c0b1650 msgr2=0x7f265c14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f265c0b1650 0x7f265c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 -- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f265c1605d0 msgr2=0x7f265c153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f265c1605d0 0x7f265c153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 -- 172.21.15.122:0/681078922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c14e3a0 con 0x7f265c0afb10 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2660c00640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c0afb10 0x7f265c14f790 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f264c009790 tx=0x7f264c0a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2659600640 1 -- 172.21.15.122:0/681078922 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0be070 con 0x7f265c0afb10 2024-09-17T05:37:40.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2659600640 1 -- 172.21.15.122:0/681078922 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f264c004470 con 0x7f265c0afb10 2024-09-17T05:37:40.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2659600640 1 -- 172.21.15.122:0/681078922 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0052d0 con 0x7f265c0afb10 2024-09-17T05:37:40.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c14ff30 con 0x7f265c0afb10 2024-09-17T05:37:40.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.897+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c152970 con 0x7f265c0afb10 2024-09-17T05:37:40.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.901+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c0affc0 con 0x7f265c0afb10 2024-09-17T05:37:40.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.901+0000 7f2659600640 1 -- 172.21.15.122:0/681078922 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f264c005470 con 0x7f265c0afb10 2024-09-17T05:37:40.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.901+0000 7f2659600640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2638074c70 0x7f2638077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:40.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.901+0000 7f2659600640 1 -- 172.21.15.122:0/681078922 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5311+0+0 (secure 0 0 0) 0x7f264c12d600 con 0x7f265c0afb10 2024-09-17T05:37:40.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.901+0000 7f2653e00640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2638074c70 0x7f2638077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:40.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.905+0000 7f2653e00640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2638074c70 0x7f2638077130 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2644007fd0 tx=0x7f26440078e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:40.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:40.905+0000 7f2659600640 1 -- 172.21.15.122:0/681078922 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f265c14e3a0 con 0x7f265c0afb10 2024-09-17T05:37:41.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.077+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f265c0d1080 con 0x7f265c0afb10 2024-09-17T05:37:41.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.717+0000 7f2659600640 1 -- 172.21.15.122:0/681078922 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) ==== 170+0+0 (secure 0 0 0) 0x7f264c016410 con 0x7f265c0afb10 2024-09-17T05:37:41.714 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:37:41.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2638074c70 msgr2=0x7f2638077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:41.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2638074c70 0x7f2638077130 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2644007fd0 tx=0x7f26440078e0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c0afb10 msgr2=0x7f265c14f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:41.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c0afb10 0x7f265c14f790 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f264c009790 tx=0x7f264c0a66f0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 shutdown_connections 2024-09-17T05:37:41.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2638074c70 0x7f2638077130 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f265c1605d0 0x7f265c153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f265c0b1650 0x7f265c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 --2- 172.21.15.122:0/681078922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f265c0afb10 0x7f265c14f790 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 >> 172.21.15.122:0/681078922 conn(0x7f265c0d1dd0 msgr2=0x7f265c0ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:41.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.721+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 shutdown_connections 2024-09-17T05:37:41.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.725+0000 7f2663600640 1 -- 172.21.15.122:0/681078922 wait complete. 2024-09-17T05:37:41.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:37:41.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 -- 172.21.15.122:0/3490835358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 msgr2=0x7f43540b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 --2- 172.21.15.122:0/3490835358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 0x7f43540b1a30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4354073810 tx=0x7f433c01c800 comp rx=0 tx=0).stop 2024-09-17T05:37:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 -- 172.21.15.122:0/3490835358 shutdown_connections 2024-09-17T05:37:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 --2- 172.21.15.122:0/3490835358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 0x7f43540b1a30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 --2- 172.21.15.122:0/3490835358 >> v2:172.21.15.173:3300/0 conn(0x7f43541605d0 0x7f43540d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 -- 172.21.15.122:0/3490835358 >> 172.21.15.122:0/3490835358 conn(0x7f43540d1dd0 msgr2=0x7f43540d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 -- 172.21.15.122:0/3490835358 shutdown_connections 2024-09-17T05:37:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 -- 172.21.15.122:0/3490835358 wait complete. 2024-09-17T05:37:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 Processor -- start 2024-09-17T05:37:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.901+0000 7f435a000640 1 -- start start 2024-09-17T05:37:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43540aeb00 0x7f435415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 0x7f43541faba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43541605d0 0x7f43541fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43540b0810 con 0x7f43540b1650 2024-09-17T05:37:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f43540b0ee0 con 0x7f43540aeb00 2024-09-17T05:37:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f43540b1220 con 0x7f43541605d0 2024-09-17T05:37:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 0x7f43541faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 0x7f43541faba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47834/0 (socket says 172.21.15.122:47834) 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 -- 172.21.15.122:0/2886519311 learned_addr learned my addr 172.21.15.122:0/2886519311 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4353400640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43540aeb00 0x7f435415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4353e00640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43541605d0 0x7f43541fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 -- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43541605d0 msgr2=0x7f43541fb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43541605d0 0x7f43541fb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 -- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43540aeb00 msgr2=0x7f435415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43540aeb00 0x7f435415a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 -- 172.21.15.122:0/2886519311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435415fce0 con 0x7f43540b1650 2024-09-17T05:37:41.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f4352a00640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 0x7f43541faba0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f433c000c10 tx=0x7f433c0a6690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:41.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f434b400640 1 -- 172.21.15.122:0/2886519311 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c004380 con 0x7f43540b1650 2024-09-17T05:37:41.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f434b400640 1 -- 172.21.15.122:0/2886519311 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f433c0be070 con 0x7f43540b1650 2024-09-17T05:37:41.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43540b6470 con 0x7f43540b1650 2024-09-17T05:37:41.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f434b400640 1 -- 172.21.15.122:0/2886519311 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c005230 con 0x7f43540b1650 2024-09-17T05:37:41.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43540b6a30 con 0x7f43540b1650 2024-09-17T05:37:41.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.905+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43540b1a30 con 0x7f43540b1650 2024-09-17T05:37:41.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.909+0000 7f434b400640 1 -- 172.21.15.122:0/2886519311 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f433c0053d0 con 0x7f43540b1650 2024-09-17T05:37:41.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.909+0000 7f434b400640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4330074c70 0x7f4330077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:41.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.909+0000 7f434b400640 1 -- 172.21.15.122:0/2886519311 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5693+0+0 (secure 0 0 0) 0x7f433c12e8e0 con 0x7f43540b1650 2024-09-17T05:37:41.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.909+0000 7f4353400640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4330074c70 0x7f4330077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:41.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.909+0000 7f4353400640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4330074c70 0x7f4330077130 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4344009b40 tx=0x7f43440091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:41.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:41.913+0000 7f434b400640 1 -- 172.21.15.122:0/2886519311 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f433c0163f0 con 0x7f43540b1650 2024-09-17T05:37:42.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.113+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4354076280 con 0x7f43540b1650 2024-09-17T05:37:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.737+0000 7f434b400640 1 -- 172.21.15.122:0/2886519311 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v72) ==== 174+0+0 (secure 0 0 0) 0x7f433c0fae20 con 0x7f43540b1650 2024-09-17T05:37:42.734 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 16 and data pool 17 2024-09-17T05:37:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4330074c70 msgr2=0x7f4330077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4330074c70 0x7f4330077130 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4344009b40 tx=0x7f43440091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 msgr2=0x7f43541faba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:42.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 0x7f43541faba0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f433c000c10 tx=0x7f433c0a6690 comp rx=0 tx=0).stop 2024-09-17T05:37:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 shutdown_connections 2024-09-17T05:37:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4330074c70 0x7f4330077130 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43541605d0 0x7f43541fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43540b1650 0x7f43541faba0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 --2- 172.21.15.122:0/2886519311 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43540aeb00 0x7f435415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.741+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 >> 172.21.15.122:0/2886519311 conn(0x7f43540d1dd0 msgr2=0x7f435415b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.745+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 shutdown_connections 2024-09-17T05:37:42.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.749+0000 7f435a000640 1 -- 172.21.15.122:0/2886519311 wait complete. 2024-09-17T05:37:42.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:37:42.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- 172.21.15.122:0/4064533895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c15b5e0 msgr2=0x7f462c1650a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:42.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 --2- 172.21.15.122:0/4064533895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c15b5e0 0x7f462c1650a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4624009840 tx=0x7f462401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- 172.21.15.122:0/4064533895 shutdown_connections 2024-09-17T05:37:42.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 --2- 172.21.15.122:0/4064533895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c15b5e0 0x7f462c1650a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 --2- 172.21.15.122:0/4064533895 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f462c0cd100 0x7f462c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- 172.21.15.122:0/4064533895 >> 172.21.15.122:0/4064533895 conn(0x7f462c0b18f0 msgr2=0x7f462c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:42.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- 172.21.15.122:0/4064533895 shutdown_connections 2024-09-17T05:37:42.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- 172.21.15.122:0/4064533895 wait complete. 2024-09-17T05:37:42.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 Processor -- start 2024-09-17T05:37:42.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- start start 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f462c0b2bf0 0x7f462c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f462c0cd100 0x7f462c155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c14f290 0x7f462c09bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f462c168410 con 0x7f462c14f290 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f462c168290 con 0x7f462c0cd100 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.933+0000 7f4632200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f462c168590 con 0x7f462c0b2bf0 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c14f290 0x7f462c09bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c14f290 0x7f462c09bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47852/0 (socket says 172.21.15.122:47852) 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 -- 172.21.15.122:0/604918374 learned_addr learned my addr 172.21.15.122:0/604918374 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:42.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462b400640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f462c0b2bf0 0x7f462c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:42.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462aa00640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f462c0cd100 0x7f462c155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:42.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 -- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f462c0b2bf0 msgr2=0x7f462c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:42.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f462c0b2bf0 0x7f462c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 -- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f462c0cd100 msgr2=0x7f462c155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:42.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f462c0cd100 0x7f462c155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:42.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 -- 172.21.15.122:0/604918374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c154de0 con 0x7f462c14f290 2024-09-17T05:37:42.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f462be00640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c14f290 0x7f462c09bee0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4624009790 tx=0x7f4624005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:42.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f4623400640 1 -- 172.21.15.122:0/604918374 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46240b2040 con 0x7f462c14f290 2024-09-17T05:37:42.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f4623400640 1 -- 172.21.15.122:0/604918374 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4624004330 con 0x7f462c14f290 2024-09-17T05:37:42.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1550c0 con 0x7f462c14f290 2024-09-17T05:37:42.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f4623400640 1 -- 172.21.15.122:0/604918374 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624003800 con 0x7f462c14f290 2024-09-17T05:37:42.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c155680 con 0x7f462c14f290 2024-09-17T05:37:42.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c069410 con 0x7f462c14f290 2024-09-17T05:37:42.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.937+0000 7f4623400640 1 -- 172.21.15.122:0/604918374 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f46240bd060 con 0x7f462c14f290 2024-09-17T05:37:42.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.941+0000 7f4623400640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f460c074ca0 0x7f460c077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:42.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.941+0000 7f4623400640 1 -- 172.21.15.122:0/604918374 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5793+0+0 (secure 0 0 0) 0x7f462412c900 con 0x7f462c14f290 2024-09-17T05:37:42.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.941+0000 7f462b400640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f460c074ca0 0x7f460c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:42.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.941+0000 7f462b400640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f460c074ca0 0x7f460c077160 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f462c0ce440 tx=0x7f461c009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:42.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:42.945+0000 7f4623400640 1 -- 172.21.15.122:0/604918374 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4624016410 con 0x7f462c14f290 2024-09-17T05:37:43.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.109+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f462c0b3610 con 0x7f462c14f290 2024-09-17T05:37:43.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4623400640 1 -- 172.21.15.122:0/604918374 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16948 (secure 0 0 0) 0x7f46240f9410 con 0x7f462c14f290 2024-09-17T05:37:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:43.109 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":116,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:37:42.118014+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:37:39.824175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T05:37:41.091437+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:37:43.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f460c074ca0 msgr2=0x7f460c077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f460c074ca0 0x7f460c077160 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f462c0ce440 tx=0x7f461c009500 comp rx=0 tx=0).stop 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c14f290 msgr2=0x7f462c09bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c14f290 0x7f462c09bee0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4624009790 tx=0x7f4624005db0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 shutdown_connections 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f460c074ca0 0x7f460c077160 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f462c14f290 0x7f462c09bee0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f462c0cd100 0x7f462c155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 --2- 172.21.15.122:0/604918374 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f462c0b2bf0 0x7f462c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 >> 172.21.15.122:0/604918374 conn(0x7f462c0b18f0 msgr2=0x7f462c073c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 shutdown_connections 2024-09-17T05:37:43.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.113+0000 7f4632200640 1 -- 172.21.15.122:0/604918374 wait complete. 2024-09-17T05:37:43.130 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:37:43.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- 172.21.15.122:0/3846065666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880b2d00 msgr2=0x7f40881605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 --2- 172.21.15.122:0/3846065666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880b2d00 0x7f40881605d0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f40880c5910 tx=0x7f408001caf0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- 172.21.15.122:0/3846065666 shutdown_connections 2024-09-17T05:37:43.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 --2- 172.21.15.122:0/3846065666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880b2d00 0x7f40881605d0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 --2- 172.21.15.122:0/3846065666 >> v2:172.21.15.173:3300/0 conn(0x7f40880b2760 0x7f40880d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- 172.21.15.122:0/3846065666 >> 172.21.15.122:0/3846065666 conn(0x7f40880d1e40 msgr2=0x7f40880d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:43.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- 172.21.15.122:0/3846065666 shutdown_connections 2024-09-17T05:37:43.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- 172.21.15.122:0/3846065666 wait complete. 2024-09-17T05:37:43.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 Processor -- start 2024-09-17T05:37:43.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- start start 2024-09-17T05:37:43.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40880b2760 0x7f4088153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:43.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40880b2d00 0x7f408815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:43.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880cfe80 0x7f408815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:43.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40880b0990 con 0x7f40880cfe80 2024-09-17T05:37:43.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f40880b0810 con 0x7f40880b2d00 2024-09-17T05:37:43.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.277+0000 7f408e200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f40880b0b10 con 0x7f40880b2760 2024-09-17T05:37:43.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880cfe80 0x7f408815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:43.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880cfe80 0x7f408815b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47868/0 (socket says 172.21.15.122:47868) 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 -- 172.21.15.122:0/922401154 learned_addr learned my addr 172.21.15.122:0/922401154 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f407ea00640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40880b2d00 0x7f408815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087400640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40880b2760 0x7f4088153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 -- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40880b2760 msgr2=0x7f4088153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40880b2760 0x7f4088153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 -- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40880b2d00 msgr2=0x7f408815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40880b2d00 0x7f408815f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 -- 172.21.15.122:0/922401154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408815e220 con 0x7f40880cfe80 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087400640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40880b2760 0x7f4088153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f407ea00640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40880b2d00 0x7f408815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:43.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f4087e00640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880cfe80 0x7f408815b7f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4080009b10 tx=0x7f4080009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:43.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f407fe00640 1 -- 172.21.15.122:0/922401154 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800bb070 con 0x7f40880cfe80 2024-09-17T05:37:43.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f407fe00640 1 -- 172.21.15.122:0/922401154 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4080004d40 con 0x7f40880cfe80 2024-09-17T05:37:43.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408815e500 con 0x7f40880cfe80 2024-09-17T05:37:43.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f407fe00640 1 -- 172.21.15.122:0/922401154 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800b2040 con 0x7f40880cfe80 2024-09-17T05:37:43.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408815eac0 con 0x7f40880cfe80 2024-09-17T05:37:43.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40880d0260 con 0x7f40880cfe80 2024-09-17T05:37:43.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.281+0000 7f407fe00640 1 -- 172.21.15.122:0/922401154 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f40800be070 con 0x7f40880cfe80 2024-09-17T05:37:43.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.285+0000 7f407fe00640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4064074c70 0x7f4064077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:43.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.285+0000 7f407fe00640 1 -- 172.21.15.122:0/922401154 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5793+0+0 (secure 0 0 0) 0x7f408012d0b0 con 0x7f40880cfe80 2024-09-17T05:37:43.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.285+0000 7f4087400640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4064074c70 0x7f4064077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:43.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.285+0000 7f4087400640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4064074c70 0x7f4064077130 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4078001400 tx=0x7f4078009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:43.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.285+0000 7f407fe00640 1 -- 172.21.15.122:0/922401154 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4080016770 con 0x7f40880cfe80 2024-09-17T05:37:43.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.457+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4088159710 con 0x7f40880cfe80 2024-09-17T05:37:43.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.457+0000 7f407fe00640 1 -- 172.21.15.122:0/922401154 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16948 (secure 0 0 0) 0x7f4080100030 con 0x7f40880cfe80 2024-09-17T05:37:43.455 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:43.455 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":116,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:37:42.118014+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:37:39.824175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T05:37:41.091437+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:37:43.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4064074c70 msgr2=0x7f4064077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4064074c70 0x7f4064077130 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4078001400 tx=0x7f4078009040 comp rx=0 tx=0).stop 2024-09-17T05:37:43.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880cfe80 msgr2=0x7f408815b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880cfe80 0x7f408815b7f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4080009b10 tx=0x7f4080009300 comp rx=0 tx=0).stop 2024-09-17T05:37:43.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 shutdown_connections 2024-09-17T05:37:43.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4064074c70 0x7f4064077130 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40880cfe80 0x7f408815b7f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40880b2d00 0x7f408815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 --2- 172.21.15.122:0/922401154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40880b2760 0x7f4088153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 >> 172.21.15.122:0/922401154 conn(0x7f40880d1e40 msgr2=0x7f40880cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:43.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 shutdown_connections 2024-09-17T05:37:43.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.461+0000 7f408e200640 1 -- 172.21.15.122:0/922401154 wait complete. 2024-09-17T05:37:43.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:37:43.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.625+0000 7f270c000640 1 -- 172.21.15.122:0/3077736424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 msgr2=0x7f270406f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.625+0000 7f270c000640 1 --2- 172.21.15.122:0/3077736424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 0x7f270406f2d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009c80 tx=0x7f26fc01f5a0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.625+0000 7f270c000640 1 -- 172.21.15.122:0/3077736424 shutdown_connections 2024-09-17T05:37:43.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.625+0000 7f270c000640 1 --2- 172.21.15.122:0/3077736424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 0x7f270406f2d0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.625+0000 7f270c000640 1 --2- 172.21.15.122:0/3077736424 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f270415bee0 0x7f27040cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.625+0000 7f270c000640 1 -- 172.21.15.122:0/3077736424 >> 172.21.15.122:0/3077736424 conn(0x7f27040b18f0 msgr2=0x7f27040b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:43.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.625+0000 7f270c000640 1 -- 172.21.15.122:0/3077736424 shutdown_connections 2024-09-17T05:37:43.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.625+0000 7f270c000640 1 -- 172.21.15.122:0/3077736424 wait complete. 2024-09-17T05:37:43.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 Processor -- start 2024-09-17T05:37:43.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 -- start start 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 0x7f2704151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27040cf400 0x7f2704154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f270415bee0 0x7f270409c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f270415c430 con 0x7f270406ee20 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f27040cd190 con 0x7f270415bee0 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f270406f810 con 0x7f27040cf400 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 0x7f2704151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 0x7f2704151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47882/0 (socket says 172.21.15.122:47882) 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 -- 172.21.15.122:0/4292115748 learned_addr learned my addr 172.21.15.122:0/4292115748 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2708c00640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27040cf400 0x7f2704154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270a000640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f270415bee0 0x7f270409c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 -- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27040cf400 msgr2=0x7f2704154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27040cf400 0x7f2704154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 -- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f270415bee0 msgr2=0x7f270409c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f270415bee0 0x7f270409c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 -- 172.21.15.122:0/4292115748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27041f6a50 con 0x7f270406ee20 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2709600640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 0x7f2704151980 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f26f800b440 tx=0x7f26f800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2701600640 1 -- 172.21.15.122:0/4292115748 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8015020 con 0x7f270406ee20 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2701600640 1 -- 172.21.15.122:0/4292115748 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26f80119a0 con 0x7f270406ee20 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27041f6d30 con 0x7f270406ee20 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f2701600640 1 -- 172.21.15.122:0/4292115748 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8011df0 con 0x7f270406ee20 2024-09-17T05:37:43.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041f72f0 con 0x7f270406ee20 2024-09-17T05:37:43.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.629+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2704069410 con 0x7f270406ee20 2024-09-17T05:37:43.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.633+0000 7f2701600640 1 -- 172.21.15.122:0/4292115748 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f26f8018020 con 0x7f270406ee20 2024-09-17T05:37:43.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.633+0000 7f2701600640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26e4074c70 0x7f26e4077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:43.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.633+0000 7f2701600640 1 -- 172.21.15.122:0/4292115748 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5793+0+0 (secure 0 0 0) 0x7f26f8093090 con 0x7f270406ee20 2024-09-17T05:37:43.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.633+0000 7f2708c00640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26e4074c70 0x7f26e4077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:43.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.633+0000 7f2708c00640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26e4074c70 0x7f26e4077130 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f27041522c0 tx=0x7f26f0007c40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.637+0000 7f2701600640 1 -- 172.21.15.122:0/4292115748 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f26f801f020 con 0x7f270406ee20 2024-09-17T05:37:43.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:43.841+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2704158880 con 0x7f270406ee20 2024-09-17T05:37:44.708 INFO:tasks.ceph.mds.a.smithi122.stderr:2024-09-17T05:37:44.713+0000 7f22e1800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:37:44.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.765+0000 7f2701600640 1 -- 172.21.15.122:0/4292115748 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) ==== 118+0+0 (secure 0 0 0) 0x7f26f800d040 con 0x7f270406ee20 2024-09-17T05:37:44.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26e4074c70 msgr2=0x7f26e4077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:44.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26e4074c70 0x7f26e4077130 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f27041522c0 tx=0x7f26f0007c40 comp rx=0 tx=0).stop 2024-09-17T05:37:44.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 msgr2=0x7f2704151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:44.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 0x7f2704151980 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f26f800b440 tx=0x7f26f800b900 comp rx=0 tx=0).stop 2024-09-17T05:37:44.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 shutdown_connections 2024-09-17T05:37:44.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26e4074c70 0x7f26e4077130 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:44.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f270415bee0 0x7f270409c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:44.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27040cf400 0x7f2704154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:44.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 --2- 172.21.15.122:0/4292115748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f270406ee20 0x7f2704151980 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:44.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 >> 172.21.15.122:0/4292115748 conn(0x7f27040b18f0 msgr2=0x7f27040b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:44.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 shutdown_connections 2024-09-17T05:37:44.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.769+0000 7f270c000640 1 -- 172.21.15.122:0/4292115748 wait complete. 2024-09-17T05:37:44.788 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:37:44.788 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:37:44.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 -- 172.21.15.122:0/3878928191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300cd100 msgr2=0x7f1a3006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:44.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 --2- 172.21.15.122:0/3878928191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300cd100 0x7f1a3006f7b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800a950 tx=0x7f1a1801da50 comp rx=0 tx=0).stop 2024-09-17T05:37:44.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 -- 172.21.15.122:0/3878928191 shutdown_connections 2024-09-17T05:37:44.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 --2- 172.21.15.122:0/3878928191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a300b49e0 0x7f1a300acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:44.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 --2- 172.21.15.122:0/3878928191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300cd100 0x7f1a3006f7b0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:44.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 -- 172.21.15.122:0/3878928191 >> 172.21.15.122:0/3878928191 conn(0x7f1a300b18f0 msgr2=0x7f1a300b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:44.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 -- 172.21.15.122:0/3878928191 shutdown_connections 2024-09-17T05:37:44.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 -- 172.21.15.122:0/3878928191 wait complete. 2024-09-17T05:37:44.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 Processor -- start 2024-09-17T05:37:44.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.937+0000 7f1a37400640 1 -- start start 2024-09-17T05:37:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300b49e0 0x7f1a3014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a300cd100 0x7f1a3014e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3015dae0 0x7f1a300af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a300cf120 con 0x7f1a300b49e0 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a300cefa0 con 0x7f1a300cd100 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1a300cf2a0 con 0x7f1a3015dae0 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300b49e0 0x7f1a3014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300b49e0 0x7f1a3014e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47894/0 (socket says 172.21.15.122:47894) 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 -- 172.21.15.122:0/1094209698 learned_addr learned my addr 172.21.15.122:0/1094209698 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a27e00640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a300cd100 0x7f1a3014e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a35400640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3015dae0 0x7f1a300af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 -- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3015dae0 msgr2=0x7f1a300af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3015dae0 0x7f1a300af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 -- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a300cd100 msgr2=0x7f1a3014e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a300cd100 0x7f1a3014e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 -- 172.21.15.122:0/1094209698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a301562f0 con 0x7f1a300b49e0 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a35400640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3015dae0 0x7f1a300af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a34a00640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300b49e0 0x7f1a3014e800 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c3f0 tx=0x7f1a2000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a2d600640 1 -- 172.21.15.122:0/1094209698 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20011790 con 0x7f1a300b49e0 2024-09-17T05:37:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a2d600640 1 -- 172.21.15.122:0/1094209698 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a20017070 con 0x7f1a300b49e0 2024-09-17T05:37:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301565d0 con 0x7f1a300b49e0 2024-09-17T05:37:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a2d600640 1 -- 172.21.15.122:0/1094209698 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20011cc0 con 0x7f1a300b49e0 2024-09-17T05:37:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3006e690 con 0x7f1a300b49e0 2024-09-17T05:37:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.941+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a300acd40 con 0x7f1a300b49e0 2024-09-17T05:37:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.945+0000 7f1a2d600640 1 -- 172.21.15.122:0/1094209698 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f1a2001c460 con 0x7f1a300b49e0 2024-09-17T05:37:44.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.945+0000 7f1a2d600640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a0c074c70 0x7f1a0c077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:44.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.945+0000 7f1a2d600640 1 -- 172.21.15.122:0/1094209698 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7f1a20092de0 con 0x7f1a300b49e0 2024-09-17T05:37:44.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.945+0000 7f1a27e00640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a0c074c70 0x7f1a0c077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:44.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.945+0000 7f1a27e00640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a0c074c70 0x7f1a0c077130 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1a180b0000 tx=0x7f1a1801c290 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:44.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:44.949+0000 7f1a2d600640 1 -- 172.21.15.122:0/1094209698 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1a2005f8f0 con 0x7f1a300b49e0 2024-09-17T05:37:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.153+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1a3009d2f0 con 0x7f1a300b49e0 2024-09-17T05:37:45.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.769+0000 7f1a2d600640 1 -- 172.21.15.122:0/1094209698 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) ==== 122+0+0 (secure 0 0 0) 0x7f1a2005f1f0 con 0x7f1a300b49e0 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a0c074c70 msgr2=0x7f1a0c077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a0c074c70 0x7f1a0c077130 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1a180b0000 tx=0x7f1a1801c290 comp rx=0 tx=0).stop 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300b49e0 msgr2=0x7f1a3014e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300b49e0 0x7f1a3014e800 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c3f0 tx=0x7f1a2000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 shutdown_connections 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1a0c074c70 0x7f1a0c077130 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a3015dae0 0x7f1a300af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a300cd100 0x7f1a3014e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 --2- 172.21.15.122:0/1094209698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a300b49e0 0x7f1a3014e800 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 >> 172.21.15.122:0/1094209698 conn(0x7f1a300b18f0 msgr2=0x7f1a300ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:45.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 shutdown_connections 2024-09-17T05:37:45.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.773+0000 7f1a37400640 1 -- 172.21.15.122:0/1094209698 wait complete. 2024-09-17T05:37:45.789 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:37:45.790 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:37:45.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- 172.21.15.122:0/959084714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec0b4bd0 msgr2=0x7f46ec0b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:45.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 --2- 172.21.15.122:0/959084714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec0b4bd0 0x7f46ec0b2bf0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f46ec069820 tx=0x7f46d401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- 172.21.15.122:0/959084714 shutdown_connections 2024-09-17T05:37:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 --2- 172.21.15.122:0/959084714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46ec15bea0 0x7f46ec0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 --2- 172.21.15.122:0/959084714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec0b4bd0 0x7f46ec0b2bf0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 --2- 172.21.15.122:0/959084714 >> v2:172.21.15.173:3300/0 conn(0x7f46ec15b580 0x7f46ec15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- 172.21.15.122:0/959084714 >> 172.21.15.122:0/959084714 conn(0x7f46ec0b18f0 msgr2=0x7f46ec0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- 172.21.15.122:0/959084714 shutdown_connections 2024-09-17T05:37:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- 172.21.15.122:0/959084714 wait complete. 2024-09-17T05:37:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 Processor -- start 2024-09-17T05:37:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- start start 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46ec0b4bd0 0x7f46ec09d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec15b580 0x7f46ec14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46ec15bea0 0x7f46ec14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0cc6d0 con 0x7f46ec15b580 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0cc550 con 0x7f46ec0b4bd0 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f3800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec0cc850 con 0x7f46ec15bea0 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46e3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec15b580 0x7f46ec14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46e3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec15b580 0x7f46ec14d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47920/0 (socket says 172.21.15.122:47920) 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46e3e00640 1 -- 172.21.15.122:0/1098292092 learned_addr learned my addr 172.21.15.122:0/1098292092 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f0e00640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46ec0b4bd0 0x7f46ec09d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f1800640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46ec15bea0 0x7f46ec14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46e3e00640 1 -- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46ec15bea0 msgr2=0x7f46ec14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46e3e00640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46ec15bea0 0x7f46ec14fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46e3e00640 1 -- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46ec0b4bd0 msgr2=0x7f46ec09d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46e3e00640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46ec0b4bd0 0x7f46ec09d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46e3e00640 1 -- 172.21.15.122:0/1098292092 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec1f7d80 con 0x7f46ec15b580 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f0e00640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46ec0b4bd0 0x7f46ec09d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.937+0000 7f46f1800640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46ec15bea0 0x7f46ec14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46e3e00640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec15b580 0x7f46ec14d7e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f46d401af70 tx=0x7f46d4004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46e9600640 1 -- 172.21.15.122:0/1098292092 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d40b2040 con 0x7f46ec15b580 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46e9600640 1 -- 172.21.15.122:0/1098292092 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46d4005160 con 0x7f46ec15b580 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec1f8060 con 0x7f46ec15b580 2024-09-17T05:37:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46e9600640 1 -- 172.21.15.122:0/1098292092 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d40055b0 con 0x7f46ec15b580 2024-09-17T05:37:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1500f0 con 0x7f46ec15b580 2024-09-17T05:37:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec069410 con 0x7f46ec15b580 2024-09-17T05:37:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46e9600640 1 -- 172.21.15.122:0/1098292092 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f46d40be060 con 0x7f46ec15b580 2024-09-17T05:37:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46e9600640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f46c8074c70 0x7f46c8077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.941+0000 7f46e9600640 1 -- 172.21.15.122:0/1098292092 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7f46d412c630 con 0x7f46ec15b580 2024-09-17T05:37:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.945+0000 7f46f0e00640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f46c8074c70 0x7f46c8077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.945+0000 7f46f0e00640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f46c8074c70 0x7f46c8077130 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f46dc005fd0 tx=0x7f46dc00a5e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:45.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:45.945+0000 7f46e9600640 1 -- 172.21.15.122:0/1098292092 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f46d40164b0 con 0x7f46ec15b580 2024-09-17T05:37:46.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46ec15b960 con 0x7f46ec15b580 2024-09-17T05:37:46.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46e9600640 1 -- 172.21.15.122:0/1098292092 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5234 (secure 0 0 0) 0x7f46d40f9140 con 0x7f46ec15b580 2024-09-17T05:37:46.145 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:46.145 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":76,"btime":"2024-09-17T05:37:45:771684+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/4034540416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":4034540416},{"type":"v1","addr":"172.21.15.173:6836","nonce":4034540416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7651,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4130637584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4130637584},{"type":"v1","addr":"172.21.15.173:6837","nonce":4130637584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/543665156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":543665156},{"type":"v1","addr":"172.21.15.122:6833","nonce":543665156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:37:42.117978+0000","modified":"2024-09-17T05:37:45.159422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2875934571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2875934571},{"type":"v1","addr":"172.21.15.122:6835","nonce":2875934571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8364,"qdb_cluster":[8364]},"id":8}]} 2024-09-17T05:37:46.145 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 76 2024-09-17T05:37:46.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f46c8074c70 msgr2=0x7f46c8077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f46c8074c70 0x7f46c8077130 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f46dc005fd0 tx=0x7f46dc00a5e0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec15b580 msgr2=0x7f46ec14d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec15b580 0x7f46ec14d7e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f46d401af70 tx=0x7f46d4004220 comp rx=0 tx=0).stop 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 shutdown_connections 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f46c8074c70 0x7f46c8077130 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46ec15bea0 0x7f46ec14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46ec15b580 0x7f46ec14d7e0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 --2- 172.21.15.122:0/1098292092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46ec0b4bd0 0x7f46ec09d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 >> 172.21.15.122:0/1098292092 conn(0x7f46ec0b18f0 msgr2=0x7f46ec0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.149+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 shutdown_connections 2024-09-17T05:37:46.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.153+0000 7f46f3800640 1 -- 172.21.15.122:0/1098292092 wait complete. 2024-09-17T05:37:46.168 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:37:46.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.313+0000 7f3699600640 1 -- 172.21.15.122:0/1383171319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3678004da0 msgr2=0x7f3678005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.313+0000 7f3699600640 1 --2- 172.21.15.122:0/1383171319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3678004da0 0x7f3678005220 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f368c009e00 tx=0x7f368c01d130 comp rx=0 tx=0).stop 2024-09-17T05:37:46.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.313+0000 7f3693400640 1 -- 172.21.15.122:0/1383171319 reap_dead start 2024-09-17T05:37:46.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.313+0000 7f3699600640 1 -- 172.21.15.122:0/1383171319 shutdown_connections 2024-09-17T05:37:46.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.313+0000 7f3699600640 1 -- 172.21.15.122:0/1383171319 >> 172.21.15.122:0/1383171319 conn(0x7f36940d3fe0 msgr2=0x7f36940d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:46.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.313+0000 7f3699600640 1 -- 172.21.15.122:0/1383171319 shutdown_connections 2024-09-17T05:37:46.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.313+0000 7f3699600640 1 -- 172.21.15.122:0/1383171319 wait complete. 2024-09-17T05:37:46.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 Processor -- start 2024-09-17T05:37:46.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 -- start start 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3678004da0 0x7f36940a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36940b3230 0x7f36940b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36940c83a0 0x7f36940c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36940b8340 con 0x7f36940c83a0 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f36940b81c0 con 0x7f3678004da0 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f36940b84c0 con 0x7f36940b3230 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36940c83a0 0x7f36940c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36940c83a0 0x7f36940c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47940/0 (socket says 172.21.15.122:47940) 2024-09-17T05:37:46.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 -- 172.21.15.122:0/3934760291 learned_addr learned my addr 172.21.15.122:0/3934760291 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3692a00640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3678004da0 0x7f36940a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3692000640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36940b3230 0x7f36940b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 -- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36940b3230 msgr2=0x7f36940b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36940b3230 0x7f36940b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 -- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3678004da0 msgr2=0x7f36940a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3678004da0 0x7f36940a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 -- 172.21.15.122:0/3934760291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36940a1e40 con 0x7f36940c83a0 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3692000640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36940b3230 0x7f36940b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3693400640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36940c83a0 0x7f36940c8820 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f368c009e00 tx=0x7f368c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f368aa00640 1 -- 172.21.15.122:0/3934760291 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0b1040 con 0x7f36940c83a0 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f368aa00640 1 -- 172.21.15.122:0/3934760291 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f368c0bb070 con 0x7f36940c83a0 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f368aa00640 1 -- 172.21.15.122:0/3934760291 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c009a00 con 0x7f36940c83a0 2024-09-17T05:37:46.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36940c90d0 con 0x7f36940c83a0 2024-09-17T05:37:46.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.317+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36940c9690 con 0x7f36940c83a0 2024-09-17T05:37:46.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.321+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369414fcc0 con 0x7f36940c83a0 2024-09-17T05:37:46.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.321+0000 7f368aa00640 1 -- 172.21.15.122:0/3934760291 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f368c004cb0 con 0x7f36940c83a0 2024-09-17T05:37:46.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.321+0000 7f368aa00640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3674074ca0 0x7f3674077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:46.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.321+0000 7f368aa00640 1 -- 172.21.15.122:0/3934760291 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7f368c12e660 con 0x7f36940c83a0 2024-09-17T05:37:46.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.321+0000 7f3692a00640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3674074ca0 0x7f3674077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:46.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.321+0000 7f3692a00640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3674074ca0 0x7f3674077160 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3684001870 tx=0x7f3684006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:46.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.325+0000 7f368aa00640 1 -- 172.21.15.122:0/3934760291 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f369414fcc0 con 0x7f36940c83a0 2024-09-17T05:37:46.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.493+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36940b89e0 con 0x7f36940c83a0 2024-09-17T05:37:46.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.493+0000 7f368aa00640 1 -- 172.21.15.122:0/3934760291 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16935 (secure 0 0 0) 0x7f368c0a5e20 con 0x7f36940c83a0 2024-09-17T05:37:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:46.491 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":117,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:37:43.752164+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:37:39.824175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T05:37:41.091437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:37:46.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3674074ca0 msgr2=0x7f3674077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3674074ca0 0x7f3674077160 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3684001870 tx=0x7f3684006040 comp rx=0 tx=0).stop 2024-09-17T05:37:46.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36940c83a0 msgr2=0x7f36940c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36940c83a0 0x7f36940c8820 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f368c009e00 tx=0x7f368c0092a0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 shutdown_connections 2024-09-17T05:37:46.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3674074ca0 0x7f3674077160 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36940c83a0 0x7f36940c8820 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36940b3230 0x7f36940b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 --2- 172.21.15.122:0/3934760291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3678004da0 0x7f36940a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 >> 172.21.15.122:0/3934760291 conn(0x7f36940d3fe0 msgr2=0x7f36940cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:46.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 shutdown_connections 2024-09-17T05:37:46.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.497+0000 7f3699600640 1 -- 172.21.15.122:0/3934760291 wait complete. 2024-09-17T05:37:46.513 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:37:46.513 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:37:46.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.657+0000 7ff269a00640 1 -- 172.21.15.122:0/1523433714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 msgr2=0x7ff2640cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.657+0000 7ff269a00640 1 --2- 172.21.15.122:0/1523433714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 0x7ff2640cebb0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff2640745d0 tx=0x7ff25c01c770 comp rx=0 tx=0).stop 2024-09-17T05:37:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.657+0000 7ff269a00640 1 -- 172.21.15.122:0/1523433714 shutdown_connections 2024-09-17T05:37:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.657+0000 7ff269a00640 1 --2- 172.21.15.122:0/1523433714 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 0x7ff2640cebb0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.657+0000 7ff269a00640 1 --2- 172.21.15.122:0/1523433714 >> v2:172.21.15.173:3300/0 conn(0x7ff26415bea0 0x7ff2640ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.657+0000 7ff269a00640 1 -- 172.21.15.122:0/1523433714 >> 172.21.15.122:0/1523433714 conn(0x7ff2640d1dc0 msgr2=0x7ff2640d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.657+0000 7ff269a00640 1 -- 172.21.15.122:0/1523433714 shutdown_connections 2024-09-17T05:37:46.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.657+0000 7ff269a00640 1 -- 172.21.15.122:0/1523433714 wait complete. 2024-09-17T05:37:46.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 Processor -- start 2024-09-17T05:37:46.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 -- start start 2024-09-17T05:37:46.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2640cdb30 0x7ff2640b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 0x7ff264150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff26415bea0 0x7ff26415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff264168fc0 con 0x7ff2640d28e0 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff264168e40 con 0x7ff26415bea0 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff264169140 con 0x7ff2640cdb30 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 0x7ff264150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 0x7ff264150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47956/0 (socket says 172.21.15.122:47956) 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 -- 172.21.15.122:0/1570393793 learned_addr learned my addr 172.21.15.122:0/1570393793 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff263400640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff26415bea0 0x7ff26415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 -- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2640cdb30 msgr2=0x7ff2640b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2640cdb30 0x7ff2640b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 -- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff26415bea0 msgr2=0x7ff26415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff26415bea0 0x7ff26415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 -- 172.21.15.122:0/1570393793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff264150cc0 con 0x7ff2640d28e0 2024-09-17T05:37:46.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff262000640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 0x7ff264150780 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c490 tx=0x7ff25000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:46.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff25aa00640 1 -- 172.21.15.122:0/1570393793 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250016020 con 0x7ff2640d28e0 2024-09-17T05:37:46.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff25aa00640 1 -- 172.21.15.122:0/1570393793 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff250012940 con 0x7ff2640d28e0 2024-09-17T05:37:46.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff25aa00640 1 -- 172.21.15.122:0/1570393793 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250012d90 con 0x7ff2640d28e0 2024-09-17T05:37:46.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2640c8a00 con 0x7ff2640d28e0 2024-09-17T05:37:46.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641f7a80 con 0x7ff2640d28e0 2024-09-17T05:37:46.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.661+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2641f7590 con 0x7ff2640d28e0 2024-09-17T05:37:46.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.665+0000 7ff25aa00640 1 -- 172.21.15.122:0/1570393793 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7ff250019020 con 0x7ff2640d28e0 2024-09-17T05:37:46.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.665+0000 7ff25aa00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff244074c70 0x7ff244077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:46.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.665+0000 7ff25aa00640 1 -- 172.21.15.122:0/1570393793 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7ff250092de0 con 0x7ff2640d28e0 2024-09-17T05:37:46.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.665+0000 7ff262a00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff244074c70 0x7ff244077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:46.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.665+0000 7ff262a00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff244074c70 0x7ff244077130 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff2540092f0 tx=0x7ff254009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:46.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.669+0000 7ff25aa00640 1 -- 172.21.15.122:0/1570393793 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff250020030 con 0x7ff2640d28e0 2024-09-17T05:37:46.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.833+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2640cc9d0 con 0x7ff244074c70 2024-09-17T05:37:46.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff25aa00640 1 -- 172.21.15.122:0/1570393793 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252501 (secure 0 0 0) 0x7ff2640cc9d0 con 0x7ff244074c70 2024-09-17T05:37:46.839 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:46.840 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:37:46.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff244074c70 msgr2=0x7ff244077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff244074c70 0x7ff244077130 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff2540092f0 tx=0x7ff254009240 comp rx=0 tx=0).stop 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 msgr2=0x7ff264150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 0x7ff264150780 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c490 tx=0x7ff25000c950 comp rx=0 tx=0).stop 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 shutdown_connections 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff244074c70 0x7ff244077130 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff26415bea0 0x7ff26415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2640d28e0 0x7ff264150780 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 --2- 172.21.15.122:0/1570393793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2640cdb30 0x7ff2640b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.841+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 >> 172.21.15.122:0/1570393793 conn(0x7ff2640d1dc0 msgr2=0x7ff26406eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.845+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 shutdown_connections 2024-09-17T05:37:46.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:46.845+0000 7ff269a00640 1 -- 172.21.15.122:0/1570393793 wait complete. 2024-09-17T05:37:46.858 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-17T05:37:46.713434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1008344,"kb_used_data":12436,"kb_used_omap":12,"kb_used_meta":995699,"kb_avail":753966376,"statfs":{"total":773094113280,"available":772061569024,"internally_reserved":0,"allocated":12734464,"data_stored":6209548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1019596342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13],"upper_bound":8},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.993589"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539532+0000","last_change":"2024-09-17T05:37:42.766185+0000","last_active":"2024-09-17T05:37:44.539532+0000","last_peered":"2024-09-17T05:37:44.539532+0000","last_clean":"2024-09-17T05:37:44.539532+0000","last_became_active":"2024-09-17T05:37:42.765836+0000","last_became_peered":"2024-09-17T05:37:42.765836+0000","last_unstale":"2024-09-17T05:37:44.539532+0000","last_undegraded":"2024-09-17T05:37:44.539532+0000","last_fullsized":"2024-09-17T05:37:44.539532+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:29.592795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757819+0000","last_change":"2024-09-17T05:37:42.777286+0000","last_active":"2024-09-17T05:37:43.757819+0000","last_peered":"2024-09-17T05:37:43.757819+0000","last_clean":"2024-09-17T05:37:43.757819+0000","last_became_active":"2024-09-17T05:37:42.776768+0000","last_became_peered":"2024-09-17T05:37:42.776768+0000","last_unstale":"2024-09-17T05:37:43.757819+0000","last_undegraded":"2024-09-17T05:37:43.757819+0000","last_fullsized":"2024-09-17T05:37:43.757819+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:05.473222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775882+0000","last_change":"2024-09-17T05:37:42.770832+0000","last_active":"2024-09-17T05:37:42.775882+0000","last_peered":"2024-09-17T05:37:42.775882+0000","last_clean":"2024-09-17T05:37:42.775882+0000","last_became_active":"2024-09-17T05:37:42.770621+0000","last_became_peered":"2024-09-17T05:37:42.770621+0000","last_unstale":"2024-09-17T05:37:42.775882+0000","last_undegraded":"2024-09-17T05:37:42.775882+0000","last_fullsized":"2024-09-17T05:37:42.775882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:15.743235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.768559+0000","last_change":"2024-09-17T05:37:42.758448+0000","last_active":"2024-09-17T05:37:42.768559+0000","last_peered":"2024-09-17T05:37:42.768559+0000","last_clean":"2024-09-17T05:37:42.768559+0000","last_became_active":"2024-09-17T05:37:42.757329+0000","last_became_peered":"2024-09-17T05:37:42.757329+0000","last_unstale":"2024-09-17T05:37:42.768559+0000","last_undegraded":"2024-09-17T05:37:42.768559+0000","last_fullsized":"2024-09-17T05:37:42.768559+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:10:00.822510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757923+0000","last_change":"2024-09-17T05:37:42.770121+0000","last_active":"2024-09-17T05:37:43.757923+0000","last_peered":"2024-09-17T05:37:43.757923+0000","last_clean":"2024-09-17T05:37:43.757923+0000","last_became_active":"2024-09-17T05:37:42.769807+0000","last_became_peered":"2024-09-17T05:37:42.769807+0000","last_unstale":"2024-09-17T05:37:43.757923+0000","last_undegraded":"2024-09-17T05:37:43.757923+0000","last_fullsized":"2024-09-17T05:37:43.757923+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:05.922727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.769828+0000","last_change":"2024-09-17T05:37:42.763331+0000","last_active":"2024-09-17T05:37:42.769828+0000","last_peered":"2024-09-17T05:37:42.769828+0000","last_clean":"2024-09-17T05:37:42.769828+0000","last_became_active":"2024-09-17T05:37:42.759385+0000","last_became_peered":"2024-09-17T05:37:42.759385+0000","last_unstale":"2024-09-17T05:37:42.769828+0000","last_undegraded":"2024-09-17T05:37:42.769828+0000","last_fullsized":"2024-09-17T05:37:42.769828+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:07.189895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757332+0000","last_change":"2024-09-17T05:37:42.756890+0000","last_active":"2024-09-17T05:37:43.757332+0000","last_peered":"2024-09-17T05:37:43.757332+0000","last_clean":"2024-09-17T05:37:43.757332+0000","last_became_active":"2024-09-17T05:37:42.756610+0000","last_became_peered":"2024-09-17T05:37:42.756610+0000","last_unstale":"2024-09-17T05:37:43.757332+0000","last_undegraded":"2024-09-17T05:37:43.757332+0000","last_fullsized":"2024-09-17T05:37:43.757332+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:02.262860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757360+0000","last_change":"2024-09-17T05:37:42.769385+0000","last_active":"2024-09-17T05:37:43.757360+0000","last_peered":"2024-09-17T05:37:43.757360+0000","last_clean":"2024-09-17T05:37:43.757360+0000","last_became_active":"2024-09-17T05:37:42.769114+0000","last_became_peered":"2024-09-17T05:37:42.769114+0000","last_unstale":"2024-09-17T05:37:43.757360+0000","last_undegraded":"2024-09-17T05:37:43.757360+0000","last_fullsized":"2024-09-17T05:37:43.757360+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:00.402139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539522+0000","last_change":"2024-09-17T05:37:42.778584+0000","last_active":"2024-09-17T05:37:44.539522+0000","last_peered":"2024-09-17T05:37:44.539522+0000","last_clean":"2024-09-17T05:37:44.539522+0000","last_became_active":"2024-09-17T05:37:42.778270+0000","last_became_peered":"2024-09-17T05:37:42.778270+0000","last_unstale":"2024-09-17T05:37:44.539522+0000","last_undegraded":"2024-09-17T05:37:44.539522+0000","last_fullsized":"2024-09-17T05:37:44.539522+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:20.420049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757630+0000","last_change":"2024-09-17T05:37:42.769992+0000","last_active":"2024-09-17T05:37:43.757630+0000","last_peered":"2024-09-17T05:37:43.757630+0000","last_clean":"2024-09-17T05:37:43.757630+0000","last_became_active":"2024-09-17T05:37:42.769449+0000","last_became_peered":"2024-09-17T05:37:42.769449+0000","last_unstale":"2024-09-17T05:37:43.757630+0000","last_undegraded":"2024-09-17T05:37:43.757630+0000","last_fullsized":"2024-09-17T05:37:43.757630+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:38.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758503+0000","last_change":"2024-09-17T05:37:42.756782+0000","last_active":"2024-09-17T05:37:43.758503+0000","last_peered":"2024-09-17T05:37:43.758503+0000","last_clean":"2024-09-17T05:37:43.758503+0000","last_became_active":"2024-09-17T05:37:42.756480+0000","last_became_peered":"2024-09-17T05:37:42.756480+0000","last_unstale":"2024-09-17T05:37:43.758503+0000","last_undegraded":"2024-09-17T05:37:43.758503+0000","last_fullsized":"2024-09-17T05:37:43.758503+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:16.265003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757591+0000","last_change":"2024-09-17T05:37:42.767086+0000","last_active":"2024-09-17T05:37:43.757591+0000","last_peered":"2024-09-17T05:37:43.757591+0000","last_clean":"2024-09-17T05:37:43.757591+0000","last_became_active":"2024-09-17T05:37:42.766717+0000","last_became_peered":"2024-09-17T05:37:42.766717+0000","last_unstale":"2024-09-17T05:37:43.757591+0000","last_undegraded":"2024-09-17T05:37:43.757591+0000","last_fullsized":"2024-09-17T05:37:43.757591+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:12.958824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757682+0000","last_change":"2024-09-17T05:37:42.772392+0000","last_active":"2024-09-17T05:37:43.757682+0000","last_peered":"2024-09-17T05:37:43.757682+0000","last_clean":"2024-09-17T05:37:43.757682+0000","last_became_active":"2024-09-17T05:37:42.772190+0000","last_became_peered":"2024-09-17T05:37:42.772190+0000","last_unstale":"2024-09-17T05:37:43.757682+0000","last_undegraded":"2024-09-17T05:37:43.757682+0000","last_fullsized":"2024-09-17T05:37:43.757682+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:24.427080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.764116+0000","last_change":"2024-09-17T05:37:42.759129+0000","last_active":"2024-09-17T05:37:42.764116+0000","last_peered":"2024-09-17T05:37:42.764116+0000","last_clean":"2024-09-17T05:37:42.764116+0000","last_became_active":"2024-09-17T05:37:42.758853+0000","last_became_peered":"2024-09-17T05:37:42.758853+0000","last_unstale":"2024-09-17T05:37:42.764116+0000","last_undegraded":"2024-09-17T05:37:42.764116+0000","last_fullsized":"2024-09-17T05:37:42.764116+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:37.241223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.538487+0000","last_change":"2024-09-17T05:37:42.768544+0000","last_active":"2024-09-17T05:37:44.538487+0000","last_peered":"2024-09-17T05:37:44.538487+0000","last_clean":"2024-09-17T05:37:44.538487+0000","last_became_active":"2024-09-17T05:37:42.768249+0000","last_became_peered":"2024-09-17T05:37:42.768249+0000","last_unstale":"2024-09-17T05:37:44.538487+0000","last_undegraded":"2024-09-17T05:37:44.538487+0000","last_fullsized":"2024-09-17T05:37:44.538487+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:21.135854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.756995+0000","last_change":"2024-09-17T05:37:42.760207+0000","last_active":"2024-09-17T05:37:43.756995+0000","last_peered":"2024-09-17T05:37:43.756995+0000","last_clean":"2024-09-17T05:37:43.756995+0000","last_became_active":"2024-09-17T05:37:42.759568+0000","last_became_peered":"2024-09-17T05:37:42.759568+0000","last_unstale":"2024-09-17T05:37:43.756995+0000","last_undegraded":"2024-09-17T05:37:43.756995+0000","last_fullsized":"2024-09-17T05:37:43.756995+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:37.241223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.764993+0000","last_change":"2024-09-17T05:37:42.758916+0000","last_active":"2024-09-17T05:37:42.764993+0000","last_peered":"2024-09-17T05:37:42.764993+0000","last_clean":"2024-09-17T05:37:42.764993+0000","last_became_active":"2024-09-17T05:37:42.758624+0000","last_became_peered":"2024-09-17T05:37:42.758624+0000","last_unstale":"2024-09-17T05:37:42.764993+0000","last_undegraded":"2024-09-17T05:37:42.764993+0000","last_fullsized":"2024-09-17T05:37:42.764993+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:08.858852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758431+0000","last_change":"2024-09-17T05:37:42.762250+0000","last_active":"2024-09-17T05:37:43.758431+0000","last_peered":"2024-09-17T05:37:43.758431+0000","last_clean":"2024-09-17T05:37:43.758431+0000","last_became_active":"2024-09-17T05:37:42.761958+0000","last_became_peered":"2024-09-17T05:37:42.761958+0000","last_unstale":"2024-09-17T05:37:43.758431+0000","last_undegraded":"2024-09-17T05:37:43.758431+0000","last_fullsized":"2024-09-17T05:37:43.758431+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.997427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.762840+0000","last_change":"2024-09-17T05:37:42.756219+0000","last_active":"2024-09-17T05:37:42.762840+0000","last_peered":"2024-09-17T05:37:42.762840+0000","last_clean":"2024-09-17T05:37:42.762840+0000","last_became_active":"2024-09-17T05:37:42.755916+0000","last_became_peered":"2024-09-17T05:37:42.755916+0000","last_unstale":"2024-09-17T05:37:42.762840+0000","last_undegraded":"2024-09-17T05:37:42.762840+0000","last_fullsized":"2024-09-17T05:37:42.762840+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:12.171151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765045+0000","last_change":"2024-09-17T05:37:42.760052+0000","last_active":"2024-09-17T05:37:42.765045+0000","last_peered":"2024-09-17T05:37:42.765045+0000","last_clean":"2024-09-17T05:37:42.765045+0000","last_became_active":"2024-09-17T05:37:42.759751+0000","last_became_peered":"2024-09-17T05:37:42.759751+0000","last_unstale":"2024-09-17T05:37:42.765045+0000","last_undegraded":"2024-09-17T05:37:42.765045+0000","last_fullsized":"2024-09-17T05:37:42.765045+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:10.435095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758429+0000","last_change":"2024-09-17T05:37:42.760830+0000","last_active":"2024-09-17T05:37:43.758429+0000","last_peered":"2024-09-17T05:37:43.758429+0000","last_clean":"2024-09-17T05:37:43.758429+0000","last_became_active":"2024-09-17T05:37:42.760410+0000","last_became_peered":"2024-09-17T05:37:42.760410+0000","last_unstale":"2024-09-17T05:37:43.758429+0000","last_undegraded":"2024-09-17T05:37:43.758429+0000","last_fullsized":"2024-09-17T05:37:43.758429+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:31.611419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.769798+0000","last_change":"2024-09-17T05:37:42.762213+0000","last_active":"2024-09-17T05:37:42.769798+0000","last_peered":"2024-09-17T05:37:42.769798+0000","last_clean":"2024-09-17T05:37:42.769798+0000","last_became_active":"2024-09-17T05:37:42.759087+0000","last_became_peered":"2024-09-17T05:37:42.759087+0000","last_unstale":"2024-09-17T05:37:42.769798+0000","last_undegraded":"2024-09-17T05:37:42.769798+0000","last_fullsized":"2024-09-17T05:37:42.769798+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:23.040553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775983+0000","last_change":"2024-09-17T05:37:42.774874+0000","last_active":"2024-09-17T05:37:42.775983+0000","last_peered":"2024-09-17T05:37:42.775983+0000","last_clean":"2024-09-17T05:37:42.775983+0000","last_became_active":"2024-09-17T05:37:42.774426+0000","last_became_peered":"2024-09-17T05:37:42.774426+0000","last_unstale":"2024-09-17T05:37:42.775983+0000","last_undegraded":"2024-09-17T05:37:42.775983+0000","last_fullsized":"2024-09-17T05:37:42.775983+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:05.922727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.770814+0000","last_change":"2024-09-17T05:37:42.763427+0000","last_active":"2024-09-17T05:37:42.770814+0000","last_peered":"2024-09-17T05:37:42.770814+0000","last_clean":"2024-09-17T05:37:42.770814+0000","last_became_active":"2024-09-17T05:37:42.763104+0000","last_became_peered":"2024-09-17T05:37:42.763104+0000","last_unstale":"2024-09-17T05:37:42.770814+0000","last_undegraded":"2024-09-17T05:37:42.770814+0000","last_fullsized":"2024-09-17T05:37:42.770814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:38.814960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758474+0000","last_change":"2024-09-17T05:37:42.779208+0000","last_active":"2024-09-17T05:37:43.758474+0000","last_peered":"2024-09-17T05:37:43.758474+0000","last_clean":"2024-09-17T05:37:43.758474+0000","last_became_active":"2024-09-17T05:37:42.778183+0000","last_became_peered":"2024-09-17T05:37:42.778183+0000","last_unstale":"2024-09-17T05:37:43.758474+0000","last_undegraded":"2024-09-17T05:37:43.758474+0000","last_fullsized":"2024-09-17T05:37:43.758474+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:18.487725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.778282+0000","last_change":"2024-09-17T05:37:42.767382+0000","last_active":"2024-09-17T05:37:42.778282+0000","last_peered":"2024-09-17T05:37:42.778282+0000","last_clean":"2024-09-17T05:37:42.778282+0000","last_became_active":"2024-09-17T05:37:42.767016+0000","last_became_peered":"2024-09-17T05:37:42.767016+0000","last_unstale":"2024-09-17T05:37:42.778282+0000","last_undegraded":"2024-09-17T05:37:42.778282+0000","last_fullsized":"2024-09-17T05:37:42.778282+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:12.111670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.769488+0000","last_change":"2024-09-17T05:37:42.763169+0000","last_active":"2024-09-17T05:37:42.769488+0000","last_peered":"2024-09-17T05:37:42.769488+0000","last_clean":"2024-09-17T05:37:42.769488+0000","last_became_active":"2024-09-17T05:37:42.762882+0000","last_became_peered":"2024-09-17T05:37:42.762882+0000","last_unstale":"2024-09-17T05:37:42.769488+0000","last_undegraded":"2024-09-17T05:37:42.769488+0000","last_fullsized":"2024-09-17T05:37:42.769488+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:50.930921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.766034+0000","last_change":"2024-09-17T05:37:42.761132+0000","last_active":"2024-09-17T05:37:42.766034+0000","last_peered":"2024-09-17T05:37:42.766034+0000","last_clean":"2024-09-17T05:37:42.766034+0000","last_became_active":"2024-09-17T05:37:42.760872+0000","last_became_peered":"2024-09-17T05:37:42.760872+0000","last_unstale":"2024-09-17T05:37:42.766034+0000","last_undegraded":"2024-09-17T05:37:42.766034+0000","last_fullsized":"2024-09-17T05:37:42.766034+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:57.017435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.761239+0000","last_change":"2024-09-17T05:37:42.758724+0000","last_active":"2024-09-17T05:37:42.761239+0000","last_peered":"2024-09-17T05:37:42.761239+0000","last_clean":"2024-09-17T05:37:42.761239+0000","last_became_active":"2024-09-17T05:37:42.758416+0000","last_became_peered":"2024-09-17T05:37:42.758416+0000","last_unstale":"2024-09-17T05:37:42.761239+0000","last_undegraded":"2024-09-17T05:37:42.761239+0000","last_fullsized":"2024-09-17T05:37:42.761239+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:10.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757870+0000","last_change":"2024-09-17T05:37:42.766909+0000","last_active":"2024-09-17T05:37:43.757870+0000","last_peered":"2024-09-17T05:37:43.757870+0000","last_clean":"2024-09-17T05:37:43.757870+0000","last_became_active":"2024-09-17T05:37:42.766498+0000","last_became_peered":"2024-09-17T05:37:42.766498+0000","last_unstale":"2024-09-17T05:37:43.757870+0000","last_undegraded":"2024-09-17T05:37:43.757870+0000","last_fullsized":"2024-09-17T05:37:43.757870+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:34.011921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.538781+0000","last_change":"2024-09-17T05:37:42.765441+0000","last_active":"2024-09-17T05:37:44.538781+0000","last_peered":"2024-09-17T05:37:44.538781+0000","last_clean":"2024-09-17T05:37:44.538781+0000","last_became_active":"2024-09-17T05:37:42.765183+0000","last_became_peered":"2024-09-17T05:37:42.765183+0000","last_unstale":"2024-09-17T05:37:44.538781+0000","last_undegraded":"2024-09-17T05:37:44.538781+0000","last_fullsized":"2024-09-17T05:37:44.538781+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:54.597011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.756876+0000","last_change":"2024-09-17T05:37:42.752922+0000","last_active":"2024-09-17T05:37:43.756876+0000","last_peered":"2024-09-17T05:37:43.756876+0000","last_clean":"2024-09-17T05:37:43.756876+0000","last_became_active":"2024-09-17T05:37:42.752568+0000","last_became_peered":"2024-09-17T05:37:42.752568+0000","last_unstale":"2024-09-17T05:37:43.756876+0000","last_undegraded":"2024-09-17T05:37:43.756876+0000","last_fullsized":"2024-09-17T05:37:43.756876+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:10.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539489+0000","last_change":"2024-09-17T05:37:42.775360+0000","last_active":"2024-09-17T05:37:44.539489+0000","last_peered":"2024-09-17T05:37:44.539489+0000","last_clean":"2024-09-17T05:37:44.539489+0000","last_became_active":"2024-09-17T05:37:42.773254+0000","last_became_peered":"2024-09-17T05:37:42.773254+0000","last_unstale":"2024-09-17T05:37:44.539489+0000","last_undegraded":"2024-09-17T05:37:44.539489+0000","last_fullsized":"2024-09-17T05:37:44.539489+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:05.473222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"116'4","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.779101+0000","last_change":"2024-09-17T05:37:41.743245+0000","last_active":"2024-09-17T05:37:42.779101+0000","last_peered":"2024-09-17T05:37:42.779101+0000","last_clean":"2024-09-17T05:37:42.779101+0000","last_became_active":"2024-09-17T05:37:41.742970+0000","last_became_peered":"2024-09-17T05:37:41.742970+0000","last_unstale":"2024-09-17T05:37:42.779101+0000","last_undegraded":"2024-09-17T05:37:42.779101+0000","last_fullsized":"2024-09-17T05:37:42.779101+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:50.414654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775966+0000","last_change":"2024-09-17T05:37:42.774711+0000","last_active":"2024-09-17T05:37:42.775966+0000","last_peered":"2024-09-17T05:37:42.775966+0000","last_clean":"2024-09-17T05:37:42.775966+0000","last_became_active":"2024-09-17T05:37:42.774439+0000","last_became_peered":"2024-09-17T05:37:42.774439+0000","last_unstale":"2024-09-17T05:37:42.775966+0000","last_undegraded":"2024-09-17T05:37:42.775966+0000","last_fullsized":"2024-09-17T05:37:42.775966+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:38.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"116'1","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757929+0000","last_change":"2024-09-17T05:37:41.737668+0000","last_active":"2024-09-17T05:37:43.757929+0000","last_peered":"2024-09-17T05:37:43.757929+0000","last_clean":"2024-09-17T05:37:43.757929+0000","last_became_active":"2024-09-17T05:37:41.737399+0000","last_became_peered":"2024-09-17T05:37:41.737399+0000","last_unstale":"2024-09-17T05:37:43.757929+0000","last_undegraded":"2024-09-17T05:37:43.757929+0000","last_fullsized":"2024-09-17T05:37:43.757929+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:06.472415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775979+0000","last_change":"2024-09-17T05:37:42.769997+0000","last_active":"2024-09-17T05:37:42.775979+0000","last_peered":"2024-09-17T05:37:42.775979+0000","last_clean":"2024-09-17T05:37:42.775979+0000","last_became_active":"2024-09-17T05:37:42.769726+0000","last_became_peered":"2024-09-17T05:37:42.769726+0000","last_unstale":"2024-09-17T05:37:42.775979+0000","last_undegraded":"2024-09-17T05:37:42.775979+0000","last_fullsized":"2024-09-17T05:37:42.775979+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:32.249486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.780364+0000","last_change":"2024-09-17T05:37:41.738946+0000","last_active":"2024-09-17T05:37:42.780364+0000","last_peered":"2024-09-17T05:37:42.780364+0000","last_clean":"2024-09-17T05:37:42.780364+0000","last_became_active":"2024-09-17T05:37:41.738516+0000","last_became_peered":"2024-09-17T05:37:41.738516+0000","last_unstale":"2024-09-17T05:37:42.780364+0000","last_undegraded":"2024-09-17T05:37:42.780364+0000","last_fullsized":"2024-09-17T05:37:42.780364+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:08.463728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.762814+0000","last_change":"2024-09-17T05:37:42.755908+0000","last_active":"2024-09-17T05:37:42.762814+0000","last_peered":"2024-09-17T05:37:42.762814+0000","last_clean":"2024-09-17T05:37:42.762814+0000","last_became_active":"2024-09-17T05:37:42.755646+0000","last_became_peered":"2024-09-17T05:37:42.755646+0000","last_unstale":"2024-09-17T05:37:42.762814+0000","last_undegraded":"2024-09-17T05:37:42.762814+0000","last_fullsized":"2024-09-17T05:37:42.762814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:22.763410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.781870+0000","last_change":"2024-09-17T05:37:41.739084+0000","last_active":"2024-09-17T05:37:42.781870+0000","last_peered":"2024-09-17T05:37:42.781870+0000","last_clean":"2024-09-17T05:37:42.781870+0000","last_became_active":"2024-09-17T05:37:41.738487+0000","last_became_peered":"2024-09-17T05:37:41.738487+0000","last_unstale":"2024-09-17T05:37:42.781870+0000","last_undegraded":"2024-09-17T05:37:42.781870+0000","last_fullsized":"2024-09-17T05:37:42.781870+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:23.337333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.774693+0000","last_change":"2024-09-17T05:37:42.767924+0000","last_active":"2024-09-17T05:37:42.774693+0000","last_peered":"2024-09-17T05:37:42.774693+0000","last_clean":"2024-09-17T05:37:42.774693+0000","last_became_active":"2024-09-17T05:37:42.767523+0000","last_became_peered":"2024-09-17T05:37:42.767523+0000","last_unstale":"2024-09-17T05:37:42.774693+0000","last_undegraded":"2024-09-17T05:37:42.774693+0000","last_fullsized":"2024-09-17T05:37:42.774693+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:07.474166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"116'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.781791+0000","last_change":"2024-09-17T05:37:41.741314+0000","last_active":"2024-09-17T05:37:42.781791+0000","last_peered":"2024-09-17T05:37:42.781791+0000","last_clean":"2024-09-17T05:37:42.781791+0000","last_became_active":"2024-09-17T05:37:41.741054+0000","last_became_peered":"2024-09-17T05:37:41.741054+0000","last_unstale":"2024-09-17T05:37:42.781791+0000","last_undegraded":"2024-09-17T05:37:42.781791+0000","last_fullsized":"2024-09-17T05:37:42.781791+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:57.860855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758373+0000","last_change":"2024-09-17T05:37:42.762175+0000","last_active":"2024-09-17T05:37:43.758373+0000","last_peered":"2024-09-17T05:37:43.758373+0000","last_clean":"2024-09-17T05:37:43.758373+0000","last_became_active":"2024-09-17T05:37:42.761869+0000","last_became_peered":"2024-09-17T05:37:42.761869+0000","last_unstale":"2024-09-17T05:37:43.758373+0000","last_undegraded":"2024-09-17T05:37:43.758373+0000","last_fullsized":"2024-09-17T05:37:43.758373+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:24.330507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"116'1","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.538978+0000","last_change":"2024-09-17T05:37:41.747733+0000","last_active":"2024-09-17T05:37:44.538978+0000","last_peered":"2024-09-17T05:37:44.538978+0000","last_clean":"2024-09-17T05:37:44.538978+0000","last_became_active":"2024-09-17T05:37:41.744934+0000","last_became_peered":"2024-09-17T05:37:41.744934+0000","last_unstale":"2024-09-17T05:37:44.538978+0000","last_undegraded":"2024-09-17T05:37:44.538978+0000","last_fullsized":"2024-09-17T05:37:44.538978+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:43.050410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757151+0000","last_change":"2024-09-17T05:37:42.759506+0000","last_active":"2024-09-17T05:37:43.757151+0000","last_peered":"2024-09-17T05:37:43.757151+0000","last_clean":"2024-09-17T05:37:43.757151+0000","last_became_active":"2024-09-17T05:37:42.758818+0000","last_became_peered":"2024-09-17T05:37:42.758818+0000","last_unstale":"2024-09-17T05:37:43.757151+0000","last_undegraded":"2024-09-17T05:37:43.757151+0000","last_fullsized":"2024-09-17T05:37:43.757151+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:40.749920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757871+0000","last_change":"2024-09-17T05:37:41.732557+0000","last_active":"2024-09-17T05:37:43.757871+0000","last_peered":"2024-09-17T05:37:43.757871+0000","last_clean":"2024-09-17T05:37:43.757871+0000","last_became_active":"2024-09-17T05:37:41.732294+0000","last_became_peered":"2024-09-17T05:37:41.732294+0000","last_unstale":"2024-09-17T05:37:43.757871+0000","last_undegraded":"2024-09-17T05:37:43.757871+0000","last_fullsized":"2024-09-17T05:37:43.757871+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:38.857046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757353+0000","last_change":"2024-09-17T05:37:42.774746+0000","last_active":"2024-09-17T05:37:43.757353+0000","last_peered":"2024-09-17T05:37:43.757353+0000","last_clean":"2024-09-17T05:37:43.757353+0000","last_became_active":"2024-09-17T05:37:42.774477+0000","last_became_peered":"2024-09-17T05:37:42.774477+0000","last_unstale":"2024-09-17T05:37:43.757353+0000","last_undegraded":"2024-09-17T05:37:43.757353+0000","last_fullsized":"2024-09-17T05:37:43.757353+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:26.385590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757359+0000","last_change":"2024-09-17T05:37:41.739046+0000","last_active":"2024-09-17T05:37:43.757359+0000","last_peered":"2024-09-17T05:37:43.757359+0000","last_clean":"2024-09-17T05:37:43.757359+0000","last_became_active":"2024-09-17T05:37:41.738751+0000","last_became_peered":"2024-09-17T05:37:41.738751+0000","last_unstale":"2024-09-17T05:37:43.757359+0000","last_undegraded":"2024-09-17T05:37:43.757359+0000","last_fullsized":"2024-09-17T05:37:43.757359+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:53.389707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758076+0000","last_change":"2024-09-17T05:37:42.772827+0000","last_active":"2024-09-17T05:37:43.758076+0000","last_peered":"2024-09-17T05:37:43.758076+0000","last_clean":"2024-09-17T05:37:43.758076+0000","last_became_active":"2024-09-17T05:37:42.771994+0000","last_became_peered":"2024-09-17T05:37:42.771994+0000","last_unstale":"2024-09-17T05:37:43.758076+0000","last_undegraded":"2024-09-17T05:37:43.758076+0000","last_fullsized":"2024-09-17T05:37:43.758076+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:34.284820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.754118+0000","last_change":"2024-09-17T05:37:41.739982+0000","last_active":"2024-09-17T05:37:42.754118+0000","last_peered":"2024-09-17T05:37:42.754118+0000","last_clean":"2024-09-17T05:37:42.754118+0000","last_became_active":"2024-09-17T05:37:41.738530+0000","last_became_peered":"2024-09-17T05:37:41.738530+0000","last_unstale":"2024-09-17T05:37:42.754118+0000","last_undegraded":"2024-09-17T05:37:42.754118+0000","last_fullsized":"2024-09-17T05:37:42.754118+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:28.884615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757453+0000","last_change":"2024-09-17T05:37:42.779592+0000","last_active":"2024-09-17T05:37:43.757453+0000","last_peered":"2024-09-17T05:37:43.757453+0000","last_clean":"2024-09-17T05:37:43.757453+0000","last_became_active":"2024-09-17T05:37:42.778827+0000","last_became_peered":"2024-09-17T05:37:42.778827+0000","last_unstale":"2024-09-17T05:37:43.757453+0000","last_undegraded":"2024-09-17T05:37:43.757453+0000","last_fullsized":"2024-09-17T05:37:43.757453+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:54.597011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757516+0000","last_change":"2024-09-17T05:37:41.734361+0000","last_active":"2024-09-17T05:37:43.757516+0000","last_peered":"2024-09-17T05:37:43.757516+0000","last_clean":"2024-09-17T05:37:43.757516+0000","last_became_active":"2024-09-17T05:37:41.734019+0000","last_became_peered":"2024-09-17T05:37:41.734019+0000","last_unstale":"2024-09-17T05:37:43.757516+0000","last_undegraded":"2024-09-17T05:37:43.757516+0000","last_fullsized":"2024-09-17T05:37:43.757516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:09.433345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.780225+0000","last_change":"2024-09-17T05:37:42.777269+0000","last_active":"2024-09-17T05:37:42.780225+0000","last_peered":"2024-09-17T05:37:42.780225+0000","last_clean":"2024-09-17T05:37:42.780225+0000","last_became_active":"2024-09-17T05:37:42.771127+0000","last_became_peered":"2024-09-17T05:37:42.771127+0000","last_unstale":"2024-09-17T05:37:42.780225+0000","last_undegraded":"2024-09-17T05:37:42.780225+0000","last_fullsized":"2024-09-17T05:37:42.780225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:29.909332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"116'3","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757526+0000","last_change":"2024-09-17T05:37:41.746444+0000","last_active":"2024-09-17T05:37:43.757526+0000","last_peered":"2024-09-17T05:37:43.757526+0000","last_clean":"2024-09-17T05:37:43.757526+0000","last_became_active":"2024-09-17T05:37:41.744658+0000","last_became_peered":"2024-09-17T05:37:41.744658+0000","last_unstale":"2024-09-17T05:37:43.757526+0000","last_undegraded":"2024-09-17T05:37:43.757526+0000","last_fullsized":"2024-09-17T05:37:43.757526+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:21.761660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758537+0000","last_change":"2024-09-17T05:37:42.768900+0000","last_active":"2024-09-17T05:37:43.758537+0000","last_peered":"2024-09-17T05:37:43.758537+0000","last_clean":"2024-09-17T05:37:43.758537+0000","last_became_active":"2024-09-17T05:37:42.768597+0000","last_became_peered":"2024-09-17T05:37:42.768597+0000","last_unstale":"2024-09-17T05:37:43.758537+0000","last_undegraded":"2024-09-17T05:37:43.758537+0000","last_fullsized":"2024-09-17T05:37:43.758537+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:09.681290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"116'6","reported_seq":19,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.783668+0000","last_change":"2024-09-17T05:37:41.738756+0000","last_active":"2024-09-17T05:37:42.783668+0000","last_peered":"2024-09-17T05:37:42.783668+0000","last_clean":"2024-09-17T05:37:42.783668+0000","last_became_active":"2024-09-17T05:37:41.738080+0000","last_became_peered":"2024-09-17T05:37:41.738080+0000","last_unstale":"2024-09-17T05:37:42.783668+0000","last_undegraded":"2024-09-17T05:37:42.783668+0000","last_fullsized":"2024-09-17T05:37:42.783668+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:43.050410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757230+0000","last_change":"2024-09-17T05:37:42.760090+0000","last_active":"2024-09-17T05:37:43.757230+0000","last_peered":"2024-09-17T05:37:43.757230+0000","last_clean":"2024-09-17T05:37:43.757230+0000","last_became_active":"2024-09-17T05:37:42.759830+0000","last_became_peered":"2024-09-17T05:37:42.759830+0000","last_unstale":"2024-09-17T05:37:43.757230+0000","last_undegraded":"2024-09-17T05:37:43.757230+0000","last_fullsized":"2024-09-17T05:37:43.757230+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:02.747051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"116'2","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758041+0000","last_change":"2024-09-17T05:37:41.743439+0000","last_active":"2024-09-17T05:37:43.758041+0000","last_peered":"2024-09-17T05:37:43.758041+0000","last_clean":"2024-09-17T05:37:43.758041+0000","last_became_active":"2024-09-17T05:37:41.742731+0000","last_became_peered":"2024-09-17T05:37:41.742731+0000","last_unstale":"2024-09-17T05:37:43.758041+0000","last_undegraded":"2024-09-17T05:37:43.758041+0000","last_fullsized":"2024-09-17T05:37:43.758041+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:11.169401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765715+0000","last_change":"2024-09-17T05:37:42.757193+0000","last_active":"2024-09-17T05:37:42.765715+0000","last_peered":"2024-09-17T05:37:42.765715+0000","last_clean":"2024-09-17T05:37:42.765715+0000","last_became_active":"2024-09-17T05:37:42.756882+0000","last_became_peered":"2024-09-17T05:37:42.756882+0000","last_unstale":"2024-09-17T05:37:42.765715+0000","last_undegraded":"2024-09-17T05:37:42.765715+0000","last_fullsized":"2024-09-17T05:37:42.765715+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:34:56.544593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.746187+0000","last_change":"2024-09-17T05:37:41.744578+0000","last_active":"2024-09-17T05:37:42.746187+0000","last_peered":"2024-09-17T05:37:42.746187+0000","last_clean":"2024-09-17T05:37:42.746187+0000","last_became_active":"2024-09-17T05:37:41.744295+0000","last_became_peered":"2024-09-17T05:37:41.744295+0000","last_unstale":"2024-09-17T05:37:42.746187+0000","last_undegraded":"2024-09-17T05:37:42.746187+0000","last_fullsized":"2024-09-17T05:37:42.746187+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:19.261239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.776363+0000","last_change":"2024-09-17T05:37:42.774436+0000","last_active":"2024-09-17T05:37:42.776363+0000","last_peered":"2024-09-17T05:37:42.776363+0000","last_clean":"2024-09-17T05:37:42.776363+0000","last_became_active":"2024-09-17T05:37:42.774152+0000","last_became_peered":"2024-09-17T05:37:42.774152+0000","last_unstale":"2024-09-17T05:37:42.776363+0000","last_undegraded":"2024-09-17T05:37:42.776363+0000","last_fullsized":"2024-09-17T05:37:42.776363+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:33.447526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":23,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.538224+0000","last_change":"2024-09-17T05:37:41.740922+0000","last_active":"2024-09-17T05:37:44.538224+0000","last_peered":"2024-09-17T05:37:44.538224+0000","last_clean":"2024-09-17T05:37:44.538224+0000","last_became_active":"2024-09-17T05:37:41.740326+0000","last_became_peered":"2024-09-17T05:37:41.740326+0000","last_unstale":"2024-09-17T05:37:44.538224+0000","last_undegraded":"2024-09-17T05:37:44.538224+0000","last_fullsized":"2024-09-17T05:37:44.538224+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:07.914885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539495+0000","last_change":"2024-09-17T05:37:42.779606+0000","last_active":"2024-09-17T05:37:44.539495+0000","last_peered":"2024-09-17T05:37:44.539495+0000","last_clean":"2024-09-17T05:37:44.539495+0000","last_became_active":"2024-09-17T05:37:42.779327+0000","last_became_peered":"2024-09-17T05:37:42.779327+0000","last_unstale":"2024-09-17T05:37:44.539495+0000","last_undegraded":"2024-09-17T05:37:44.539495+0000","last_fullsized":"2024-09-17T05:37:44.539495+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:24.427080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"116'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.780557+0000","last_change":"2024-09-17T05:37:41.746033+0000","last_active":"2024-09-17T05:37:42.780557+0000","last_peered":"2024-09-17T05:37:42.780557+0000","last_clean":"2024-09-17T05:37:42.780557+0000","last_became_active":"2024-09-17T05:37:41.745738+0000","last_became_peered":"2024-09-17T05:37:41.745738+0000","last_unstale":"2024-09-17T05:37:42.780557+0000","last_undegraded":"2024-09-17T05:37:42.780557+0000","last_fullsized":"2024-09-17T05:37:42.780557+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:36.686851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539720+0000","last_change":"2024-09-17T05:37:42.774744+0000","last_active":"2024-09-17T05:37:44.539720+0000","last_peered":"2024-09-17T05:37:44.539720+0000","last_clean":"2024-09-17T05:37:44.539720+0000","last_became_active":"2024-09-17T05:37:42.774480+0000","last_became_peered":"2024-09-17T05:37:42.774480+0000","last_unstale":"2024-09-17T05:37:44.539720+0000","last_undegraded":"2024-09-17T05:37:44.539720+0000","last_fullsized":"2024-09-17T05:37:44.539720+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:24.700744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757637+0000","last_change":"2024-09-17T05:37:41.746517+0000","last_active":"2024-09-17T05:37:43.757637+0000","last_peered":"2024-09-17T05:37:43.757637+0000","last_clean":"2024-09-17T05:37:43.757637+0000","last_became_active":"2024-09-17T05:37:41.742602+0000","last_became_peered":"2024-09-17T05:37:41.742602+0000","last_unstale":"2024-09-17T05:37:43.757637+0000","last_undegraded":"2024-09-17T05:37:43.757637+0000","last_fullsized":"2024-09-17T05:37:43.757637+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:14.741485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758499+0000","last_change":"2024-09-17T05:37:42.769889+0000","last_active":"2024-09-17T05:37:43.758499+0000","last_peered":"2024-09-17T05:37:43.758499+0000","last_clean":"2024-09-17T05:37:43.758499+0000","last_became_active":"2024-09-17T05:37:42.769630+0000","last_became_peered":"2024-09-17T05:37:42.769630+0000","last_unstale":"2024-09-17T05:37:43.758499+0000","last_undegraded":"2024-09-17T05:37:43.758499+0000","last_fullsized":"2024-09-17T05:37:43.758499+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:33.969613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"116'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.781862+0000","last_change":"2024-09-17T05:37:41.739784+0000","last_active":"2024-09-17T05:37:42.781862+0000","last_peered":"2024-09-17T05:37:42.781862+0000","last_clean":"2024-09-17T05:37:42.781862+0000","last_became_active":"2024-09-17T05:37:41.739044+0000","last_became_peered":"2024-09-17T05:37:41.739044+0000","last_unstale":"2024-09-17T05:37:42.781862+0000","last_undegraded":"2024-09-17T05:37:42.781862+0000","last_fullsized":"2024-09-17T05:37:42.781862+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:38.033830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765906+0000","last_change":"2024-09-17T05:37:42.764734+0000","last_active":"2024-09-17T05:37:42.765906+0000","last_peered":"2024-09-17T05:37:42.765906+0000","last_clean":"2024-09-17T05:37:42.765906+0000","last_became_active":"2024-09-17T05:37:42.764408+0000","last_became_peered":"2024-09-17T05:37:42.764408+0000","last_unstale":"2024-09-17T05:37:42.765906+0000","last_undegraded":"2024-09-17T05:37:42.765906+0000","last_fullsized":"2024-09-17T05:37:42.765906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:40.749920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.750543+0000","last_change":"2024-09-17T05:37:41.744912+0000","last_active":"2024-09-17T05:37:42.750543+0000","last_peered":"2024-09-17T05:37:42.750543+0000","last_clean":"2024-09-17T05:37:42.750543+0000","last_became_active":"2024-09-17T05:37:41.743871+0000","last_became_peered":"2024-09-17T05:37:41.743871+0000","last_unstale":"2024-09-17T05:37:42.750543+0000","last_undegraded":"2024-09-17T05:37:42.750543+0000","last_fullsized":"2024-09-17T05:37:42.750543+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:42.169420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.777585+0000","last_change":"2024-09-17T05:37:42.775482+0000","last_active":"2024-09-17T05:37:42.777585+0000","last_peered":"2024-09-17T05:37:42.777585+0000","last_clean":"2024-09-17T05:37:42.777585+0000","last_became_active":"2024-09-17T05:37:42.775213+0000","last_became_peered":"2024-09-17T05:37:42.775213+0000","last_unstale":"2024-09-17T05:37:42.777585+0000","last_undegraded":"2024-09-17T05:37:42.777585+0000","last_fullsized":"2024-09-17T05:37:42.777585+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:13:11.996924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"116'2","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758161+0000","last_change":"2024-09-17T05:37:41.733952+0000","last_active":"2024-09-17T05:37:43.758161+0000","last_peered":"2024-09-17T05:37:43.758161+0000","last_clean":"2024-09-17T05:37:43.758161+0000","last_became_active":"2024-09-17T05:37:41.733642+0000","last_became_peered":"2024-09-17T05:37:41.733642+0000","last_unstale":"2024-09-17T05:37:43.758161+0000","last_undegraded":"2024-09-17T05:37:43.758161+0000","last_fullsized":"2024-09-17T05:37:43.758161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:07.857101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539528+0000","last_change":"2024-09-17T05:37:42.773733+0000","last_active":"2024-09-17T05:37:44.539528+0000","last_peered":"2024-09-17T05:37:44.539528+0000","last_clean":"2024-09-17T05:37:44.539528+0000","last_became_active":"2024-09-17T05:37:42.773465+0000","last_became_peered":"2024-09-17T05:37:42.773465+0000","last_unstale":"2024-09-17T05:37:44.539528+0000","last_undegraded":"2024-09-17T05:37:44.539528+0000","last_fullsized":"2024-09-17T05:37:44.539528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:48.048484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539226+0000","last_change":"2024-09-17T05:37:41.746049+0000","last_active":"2024-09-17T05:37:44.539226+0000","last_peered":"2024-09-17T05:37:44.539226+0000","last_clean":"2024-09-17T05:37:44.539226+0000","last_became_active":"2024-09-17T05:37:41.745264+0000","last_became_peered":"2024-09-17T05:37:41.745264+0000","last_unstale":"2024-09-17T05:37:44.539226+0000","last_undegraded":"2024-09-17T05:37:44.539226+0000","last_fullsized":"2024-09-17T05:37:44.539226+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:54.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765478+0000","last_change":"2024-09-17T05:37:42.763362+0000","last_active":"2024-09-17T05:37:42.765478+0000","last_peered":"2024-09-17T05:37:42.765478+0000","last_clean":"2024-09-17T05:37:42.765478+0000","last_became_active":"2024-09-17T05:37:42.762740+0000","last_became_peered":"2024-09-17T05:37:42.762740+0000","last_unstale":"2024-09-17T05:37:42.765478+0000","last_undegraded":"2024-09-17T05:37:42.765478+0000","last_fullsized":"2024-09-17T05:37:42.765478+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:02.747051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.753141+0000","last_change":"2024-09-17T05:37:41.743622+0000","last_active":"2024-09-17T05:37:42.753141+0000","last_peered":"2024-09-17T05:37:42.753141+0000","last_clean":"2024-09-17T05:37:42.753141+0000","last_became_active":"2024-09-17T05:37:41.743165+0000","last_became_peered":"2024-09-17T05:37:41.743165+0000","last_unstale":"2024-09-17T05:37:42.753141+0000","last_undegraded":"2024-09-17T05:37:42.753141+0000","last_fullsized":"2024-09-17T05:37:42.753141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:06.691127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757054+0000","last_change":"2024-09-17T05:37:42.759126+0000","last_active":"2024-09-17T05:37:43.757054+0000","last_peered":"2024-09-17T05:37:43.757054+0000","last_clean":"2024-09-17T05:37:43.757054+0000","last_became_active":"2024-09-17T05:37:42.758843+0000","last_became_peered":"2024-09-17T05:37:42.758843+0000","last_unstale":"2024-09-17T05:37:43.757054+0000","last_undegraded":"2024-09-17T05:37:43.757054+0000","last_fullsized":"2024-09-17T05:37:43.757054+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:05.716742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.753725+0000","last_change":"2024-09-17T05:37:41.736249+0000","last_active":"2024-09-17T05:37:42.753725+0000","last_peered":"2024-09-17T05:37:42.753725+0000","last_clean":"2024-09-17T05:37:42.753725+0000","last_became_active":"2024-09-17T05:37:41.735343+0000","last_became_peered":"2024-09-17T05:37:41.735343+0000","last_unstale":"2024-09-17T05:37:42.753725+0000","last_undegraded":"2024-09-17T05:37:42.753725+0000","last_fullsized":"2024-09-17T05:37:42.753725+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:17.386932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765907+0000","last_change":"2024-09-17T05:37:42.764881+0000","last_active":"2024-09-17T05:37:42.765907+0000","last_peered":"2024-09-17T05:37:42.765907+0000","last_clean":"2024-09-17T05:37:42.765907+0000","last_became_active":"2024-09-17T05:37:42.763651+0000","last_became_peered":"2024-09-17T05:37:42.763651+0000","last_unstale":"2024-09-17T05:37:42.765907+0000","last_undegraded":"2024-09-17T05:37:42.765907+0000","last_fullsized":"2024-09-17T05:37:42.765907+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:12.958824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"116'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.781300+0000","last_change":"2024-09-17T05:37:41.740894+0000","last_active":"2024-09-17T05:37:42.781300+0000","last_peered":"2024-09-17T05:37:42.781300+0000","last_clean":"2024-09-17T05:37:42.781300+0000","last_became_active":"2024-09-17T05:37:41.739553+0000","last_became_peered":"2024-09-17T05:37:41.739553+0000","last_unstale":"2024-09-17T05:37:42.781300+0000","last_undegraded":"2024-09-17T05:37:42.781300+0000","last_fullsized":"2024-09-17T05:37:42.781300+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:45.980792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.780440+0000","last_change":"2024-09-17T05:37:42.776959+0000","last_active":"2024-09-17T05:37:42.780440+0000","last_peered":"2024-09-17T05:37:42.780440+0000","last_clean":"2024-09-17T05:37:42.780440+0000","last_became_active":"2024-09-17T05:37:42.776653+0000","last_became_peered":"2024-09-17T05:37:42.776653+0000","last_unstale":"2024-09-17T05:37:42.780440+0000","last_undegraded":"2024-09-17T05:37:42.780440+0000","last_fullsized":"2024-09-17T05:37:42.780440+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:06.870759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758365+0000","last_change":"2024-09-17T05:37:41.738484+0000","last_active":"2024-09-17T05:37:43.758365+0000","last_peered":"2024-09-17T05:37:43.758365+0000","last_clean":"2024-09-17T05:37:43.758365+0000","last_became_active":"2024-09-17T05:37:41.738220+0000","last_became_peered":"2024-09-17T05:37:41.738220+0000","last_unstale":"2024-09-17T05:37:43.758365+0000","last_undegraded":"2024-09-17T05:37:43.758365+0000","last_fullsized":"2024-09-17T05:37:43.758365+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:06.614037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539478+0000","last_change":"2024-09-17T05:37:42.767193+0000","last_active":"2024-09-17T05:37:44.539478+0000","last_peered":"2024-09-17T05:37:44.539478+0000","last_clean":"2024-09-17T05:37:44.539478+0000","last_became_active":"2024-09-17T05:37:42.766906+0000","last_became_peered":"2024-09-17T05:37:42.766906+0000","last_unstale":"2024-09-17T05:37:44.539478+0000","last_undegraded":"2024-09-17T05:37:44.539478+0000","last_fullsized":"2024-09-17T05:37:44.539478+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:21.794149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.754373+0000","last_change":"2024-09-17T05:37:41.742333+0000","last_active":"2024-09-17T05:37:42.754373+0000","last_peered":"2024-09-17T05:37:42.754373+0000","last_clean":"2024-09-17T05:37:42.754373+0000","last_became_active":"2024-09-17T05:37:41.741956+0000","last_became_peered":"2024-09-17T05:37:41.741956+0000","last_unstale":"2024-09-17T05:37:42.754373+0000","last_undegraded":"2024-09-17T05:37:42.754373+0000","last_fullsized":"2024-09-17T05:37:42.754373+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:07.914885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.760039+0000","last_change":"2024-09-17T05:37:42.757770+0000","last_active":"2024-09-17T05:37:42.760039+0000","last_peered":"2024-09-17T05:37:42.760039+0000","last_clean":"2024-09-17T05:37:42.760039+0000","last_became_active":"2024-09-17T05:37:42.757437+0000","last_became_peered":"2024-09-17T05:37:42.757437+0000","last_unstale":"2024-09-17T05:37:42.760039+0000","last_undegraded":"2024-09-17T05:37:42.760039+0000","last_fullsized":"2024-09-17T05:37:42.760039+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:01.475301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.755531+0000","last_change":"2024-09-17T05:37:41.740909+0000","last_active":"2024-09-17T05:37:42.755531+0000","last_peered":"2024-09-17T05:37:42.755531+0000","last_clean":"2024-09-17T05:37:42.755531+0000","last_became_active":"2024-09-17T05:37:41.740592+0000","last_became_peered":"2024-09-17T05:37:41.740592+0000","last_unstale":"2024-09-17T05:37:42.755531+0000","last_undegraded":"2024-09-17T05:37:42.755531+0000","last_fullsized":"2024-09-17T05:37:42.755531+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:54.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.763468+0000","last_change":"2024-09-17T05:37:42.757895+0000","last_active":"2024-09-17T05:37:42.763468+0000","last_peered":"2024-09-17T05:37:42.763468+0000","last_clean":"2024-09-17T05:37:42.763468+0000","last_became_active":"2024-09-17T05:37:42.757625+0000","last_became_peered":"2024-09-17T05:37:42.757625+0000","last_unstale":"2024-09-17T05:37:42.763468+0000","last_undegraded":"2024-09-17T05:37:42.763468+0000","last_fullsized":"2024-09-17T05:37:42.763468+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:54.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758547+0000","last_change":"2024-09-17T05:37:41.734730+0000","last_active":"2024-09-17T05:37:43.758547+0000","last_peered":"2024-09-17T05:37:43.758547+0000","last_clean":"2024-09-17T05:37:43.758547+0000","last_became_active":"2024-09-17T05:37:41.734476+0000","last_became_peered":"2024-09-17T05:37:41.734476+0000","last_unstale":"2024-09-17T05:37:43.758547+0000","last_undegraded":"2024-09-17T05:37:43.758547+0000","last_fullsized":"2024-09-17T05:37:43.758547+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:18.422760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.762517+0000","last_change":"2024-09-17T05:37:42.760688+0000","last_active":"2024-09-17T05:37:42.762517+0000","last_peered":"2024-09-17T05:37:42.762517+0000","last_clean":"2024-09-17T05:37:42.762517+0000","last_became_active":"2024-09-17T05:37:42.760438+0000","last_became_peered":"2024-09-17T05:37:42.760438+0000","last_unstale":"2024-09-17T05:37:42.762517+0000","last_undegraded":"2024-09-17T05:37:42.762517+0000","last_fullsized":"2024-09-17T05:37:42.762517+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:05.716742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.750106+0000","last_change":"2024-09-17T05:37:41.746893+0000","last_active":"2024-09-17T05:37:42.750106+0000","last_peered":"2024-09-17T05:37:42.750106+0000","last_clean":"2024-09-17T05:37:42.750106+0000","last_became_active":"2024-09-17T05:37:41.746686+0000","last_became_peered":"2024-09-17T05:37:41.746686+0000","last_unstale":"2024-09-17T05:37:42.750106+0000","last_undegraded":"2024-09-17T05:37:42.750106+0000","last_fullsized":"2024-09-17T05:37:42.750106+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:41.990740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.776689+0000","last_change":"2024-09-17T05:37:42.775654+0000","last_active":"2024-09-17T05:37:42.776689+0000","last_peered":"2024-09-17T05:37:42.776689+0000","last_clean":"2024-09-17T05:37:42.776689+0000","last_became_active":"2024-09-17T05:37:42.775441+0000","last_became_peered":"2024-09-17T05:37:42.775441+0000","last_unstale":"2024-09-17T05:37:42.776689+0000","last_undegraded":"2024-09-17T05:37:42.776689+0000","last_fullsized":"2024-09-17T05:37:42.776689+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:20.793860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539254+0000","last_change":"2024-09-17T05:37:41.743435+0000","last_active":"2024-09-17T05:37:44.539254+0000","last_peered":"2024-09-17T05:37:44.539254+0000","last_clean":"2024-09-17T05:37:44.539254+0000","last_became_active":"2024-09-17T05:37:41.741689+0000","last_became_peered":"2024-09-17T05:37:41.741689+0000","last_unstale":"2024-09-17T05:37:44.539254+0000","last_undegraded":"2024-09-17T05:37:44.539254+0000","last_fullsized":"2024-09-17T05:37:44.539254+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:23.337333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775961+0000","last_change":"2024-09-17T05:37:42.765783+0000","last_active":"2024-09-17T05:37:42.775961+0000","last_peered":"2024-09-17T05:37:42.775961+0000","last_clean":"2024-09-17T05:37:42.775961+0000","last_became_active":"2024-09-17T05:37:42.764580+0000","last_became_peered":"2024-09-17T05:37:42.764580+0000","last_unstale":"2024-09-17T05:37:42.775961+0000","last_undegraded":"2024-09-17T05:37:42.775961+0000","last_fullsized":"2024-09-17T05:37:42.775961+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:57.014526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.762958+0000","last_change":"2024-09-17T05:37:41.746689+0000","last_active":"2024-09-17T05:37:42.762958+0000","last_peered":"2024-09-17T05:37:42.762958+0000","last_clean":"2024-09-17T05:37:42.762958+0000","last_became_active":"2024-09-17T05:37:41.746420+0000","last_became_peered":"2024-09-17T05:37:41.746420+0000","last_unstale":"2024-09-17T05:37:42.762958+0000","last_undegraded":"2024-09-17T05:37:42.762958+0000","last_fullsized":"2024-09-17T05:37:42.762958+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:31.889083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.767985+0000","last_change":"2024-09-17T05:37:42.759191+0000","last_active":"2024-09-17T05:37:42.767985+0000","last_peered":"2024-09-17T05:37:42.767985+0000","last_clean":"2024-09-17T05:37:42.767985+0000","last_became_active":"2024-09-17T05:37:42.758287+0000","last_became_peered":"2024-09-17T05:37:42.758287+0000","last_unstale":"2024-09-17T05:37:42.767985+0000","last_undegraded":"2024-09-17T05:37:42.767985+0000","last_fullsized":"2024-09-17T05:37:42.767985+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:45.112505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757717+0000","last_change":"2024-09-17T05:37:41.743122+0000","last_active":"2024-09-17T05:37:43.757717+0000","last_peered":"2024-09-17T05:37:43.757717+0000","last_clean":"2024-09-17T05:37:43.757717+0000","last_became_active":"2024-09-17T05:37:41.742861+0000","last_became_peered":"2024-09-17T05:37:41.742861+0000","last_unstale":"2024-09-17T05:37:43.757717+0000","last_undegraded":"2024-09-17T05:37:43.757717+0000","last_fullsized":"2024-09-17T05:37:43.757717+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:37.813210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":290,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.745456+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:37:42.745456+0000","last_peered":"2024-09-17T05:37:42.745456+0000","last_clean":"2024-09-17T05:37:42.745456+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:37:42.745456+0000","last_undegraded":"2024-09-17T05:37:42.745456+0000","last_fullsized":"2024-09-17T05:37:42.745456+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104752,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267088,"statfs":{"total":96636764160,"available":96529498112,"internally_reserved":0,"allocated":1466368,"data_stored":635321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":105773514},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149132,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222708,"statfs":{"total":96636764160,"available":96484052992,"internally_reserved":0,"allocated":1433600,"data_stored":627201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509597,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126944,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244896,"statfs":{"total":96636764160,"available":96506773504,"internally_reserved":0,"allocated":1449984,"data_stored":628903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509596,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126928,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244912,"statfs":{"total":96636764160,"available":96506789888,"internally_reserved":0,"allocated":1433600,"data_stored":627201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509597,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122780,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249060,"statfs":{"total":96636764160,"available":96511037440,"internally_reserved":0,"allocated":1445888,"data_stored":627839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509598,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141400,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230440,"statfs":{"total":96636764160,"available":96491970560,"internally_reserved":0,"allocated":2027520,"data_stored":1217569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104732,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267108,"statfs":{"total":96636764160,"available":96529518592,"internally_reserved":0,"allocated":1441792,"data_stored":627853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":105773515},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542302,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131676,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240164,"statfs":{"total":96636764160,"available":96501927936,"internally_reserved":0,"allocated":2035712,"data_stored":1217661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:37:46.862 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:37:47.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- 172.21.15.122:0/1046947093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980b49e0 msgr2=0x7f6e980acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 --2- 172.21.15.122:0/1046947093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980b49e0 0x7f6e980acd40 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6e980708e0 tx=0x7f6e9001dd90 comp rx=0 tx=0).stop 2024-09-17T05:37:47.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- 172.21.15.122:0/1046947093 shutdown_connections 2024-09-17T05:37:47.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 --2- 172.21.15.122:0/1046947093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980b49e0 0x7f6e980acd40 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 --2- 172.21.15.122:0/1046947093 >> v2:172.21.15.173:3300/0 conn(0x7f6e980cd100 0x7f6e9806f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- 172.21.15.122:0/1046947093 >> 172.21.15.122:0/1046947093 conn(0x7f6e980b18f0 msgr2=0x7f6e980b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:47.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- 172.21.15.122:0/1046947093 shutdown_connections 2024-09-17T05:37:47.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- 172.21.15.122:0/1046947093 wait complete. 2024-09-17T05:37:47.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 Processor -- start 2024-09-17T05:37:47.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- start start 2024-09-17T05:37:47.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e980b49e0 0x7f6e98073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980cd100 0x7f6e981496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e9815dae0 0x7f6e9806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e980cf170 con 0x7f6e980cd100 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6e980ceff0 con 0x7f6e980b49e0 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e9d600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6e980cf2f0 con 0x7f6e9815dae0 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e8e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980cd100 0x7f6e981496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e8e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980cd100 0x7f6e981496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47968/0 (socket says 172.21.15.122:47968) 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.045+0000 7f6e8e000640 1 -- 172.21.15.122:0/3845016273 learned_addr learned my addr 172.21.15.122:0/3845016273 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e96a00640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e980b49e0 0x7f6e98073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e97400640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e9815dae0 0x7f6e9806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8e000640 1 -- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e9815dae0 msgr2=0x7f6e9806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8e000640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e9815dae0 0x7f6e9806f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8e000640 1 -- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e980b49e0 msgr2=0x7f6e98073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8e000640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e980b49e0 0x7f6e98073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8e000640 1 -- 172.21.15.122:0/3845016273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9809d6f0 con 0x7f6e980cd100 2024-09-17T05:37:47.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e96a00640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e980b49e0 0x7f6e98073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:47.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8e000640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980cd100 0x7f6e981496f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000d980 tx=0x7f6e8000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:47.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8f400640 1 -- 172.21.15.122:0/3845016273 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80017070 con 0x7f6e980cd100 2024-09-17T05:37:47.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8f400640 1 -- 172.21.15.122:0/3845016273 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e80009d20 con 0x7f6e980cd100 2024-09-17T05:37:47.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8f400640 1 -- 172.21.15.122:0/3845016273 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80004280 con 0x7f6e980cd100 2024-09-17T05:37:47.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9809aab0 con 0x7f6e980cd100 2024-09-17T05:37:47.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9809d9b0 con 0x7f6e980cd100 2024-09-17T05:37:47.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e98069410 con 0x7f6e980cd100 2024-09-17T05:37:47.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.049+0000 7f6e8f400640 1 -- 172.21.15.122:0/3845016273 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f6e80003600 con 0x7f6e980cd100 2024-09-17T05:37:47.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.053+0000 7f6e8f400640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e74074c70 0x7f6e74077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.053+0000 7f6e8f400640 1 -- 172.21.15.122:0/3845016273 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7f6e80093210 con 0x7f6e980cd100 2024-09-17T05:37:47.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.053+0000 7f6e96a00640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e74074c70 0x7f6e74077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.053+0000 7f6e96a00640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e74074c70 0x7f6e74077130 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6e88000f70 tx=0x7f6e88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:47.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.053+0000 7f6e8f400640 1 -- 172.21.15.122:0/3845016273 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6e8005fd20 con 0x7f6e980cd100 2024-09-17T05:37:47.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.221+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e980acd40 con 0x7f6e74074c70 2024-09-17T05:37:47.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.225+0000 7f6e8f400640 1 -- 172.21.15.122:0/3845016273 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252501 (secure 0 0 0) 0x7f6e980acd40 con 0x7f6e74074c70 2024-09-17T05:37:47.224 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:47.225 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:37:47.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e74074c70 msgr2=0x7f6e74077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e74074c70 0x7f6e74077130 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6e88000f70 tx=0x7f6e88009040 comp rx=0 tx=0).stop 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980cd100 msgr2=0x7f6e981496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980cd100 0x7f6e981496f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000d980 tx=0x7f6e8000de40 comp rx=0 tx=0).stop 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 shutdown_connections 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6e74074c70 0x7f6e74077130 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e9815dae0 0x7f6e9806f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e980cd100 0x7f6e981496f0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 --2- 172.21.15.122:0/3845016273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e980b49e0 0x7f6e98073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 >> 172.21.15.122:0/3845016273 conn(0x7f6e980b18f0 msgr2=0x7f6e9815a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 shutdown_connections 2024-09-17T05:37:47.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.229+0000 7f6e9d600640 1 -- 172.21.15.122:0/3845016273 wait complete. 2024-09-17T05:37:47.245 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-17T05:37:46.713434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1008344,"kb_used_data":12436,"kb_used_omap":12,"kb_used_meta":995699,"kb_avail":753966376,"statfs":{"total":773094113280,"available":772061569024,"internally_reserved":0,"allocated":12734464,"data_stored":6209548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1019596342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13],"upper_bound":8},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.993589"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539532+0000","last_change":"2024-09-17T05:37:42.766185+0000","last_active":"2024-09-17T05:37:44.539532+0000","last_peered":"2024-09-17T05:37:44.539532+0000","last_clean":"2024-09-17T05:37:44.539532+0000","last_became_active":"2024-09-17T05:37:42.765836+0000","last_became_peered":"2024-09-17T05:37:42.765836+0000","last_unstale":"2024-09-17T05:37:44.539532+0000","last_undegraded":"2024-09-17T05:37:44.539532+0000","last_fullsized":"2024-09-17T05:37:44.539532+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:29.592795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757819+0000","last_change":"2024-09-17T05:37:42.777286+0000","last_active":"2024-09-17T05:37:43.757819+0000","last_peered":"2024-09-17T05:37:43.757819+0000","last_clean":"2024-09-17T05:37:43.757819+0000","last_became_active":"2024-09-17T05:37:42.776768+0000","last_became_peered":"2024-09-17T05:37:42.776768+0000","last_unstale":"2024-09-17T05:37:43.757819+0000","last_undegraded":"2024-09-17T05:37:43.757819+0000","last_fullsized":"2024-09-17T05:37:43.757819+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:05.473222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775882+0000","last_change":"2024-09-17T05:37:42.770832+0000","last_active":"2024-09-17T05:37:42.775882+0000","last_peered":"2024-09-17T05:37:42.775882+0000","last_clean":"2024-09-17T05:37:42.775882+0000","last_became_active":"2024-09-17T05:37:42.770621+0000","last_became_peered":"2024-09-17T05:37:42.770621+0000","last_unstale":"2024-09-17T05:37:42.775882+0000","last_undegraded":"2024-09-17T05:37:42.775882+0000","last_fullsized":"2024-09-17T05:37:42.775882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:15.743235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.768559+0000","last_change":"2024-09-17T05:37:42.758448+0000","last_active":"2024-09-17T05:37:42.768559+0000","last_peered":"2024-09-17T05:37:42.768559+0000","last_clean":"2024-09-17T05:37:42.768559+0000","last_became_active":"2024-09-17T05:37:42.757329+0000","last_became_peered":"2024-09-17T05:37:42.757329+0000","last_unstale":"2024-09-17T05:37:42.768559+0000","last_undegraded":"2024-09-17T05:37:42.768559+0000","last_fullsized":"2024-09-17T05:37:42.768559+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:10:00.822510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757923+0000","last_change":"2024-09-17T05:37:42.770121+0000","last_active":"2024-09-17T05:37:43.757923+0000","last_peered":"2024-09-17T05:37:43.757923+0000","last_clean":"2024-09-17T05:37:43.757923+0000","last_became_active":"2024-09-17T05:37:42.769807+0000","last_became_peered":"2024-09-17T05:37:42.769807+0000","last_unstale":"2024-09-17T05:37:43.757923+0000","last_undegraded":"2024-09-17T05:37:43.757923+0000","last_fullsized":"2024-09-17T05:37:43.757923+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:05.922727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.769828+0000","last_change":"2024-09-17T05:37:42.763331+0000","last_active":"2024-09-17T05:37:42.769828+0000","last_peered":"2024-09-17T05:37:42.769828+0000","last_clean":"2024-09-17T05:37:42.769828+0000","last_became_active":"2024-09-17T05:37:42.759385+0000","last_became_peered":"2024-09-17T05:37:42.759385+0000","last_unstale":"2024-09-17T05:37:42.769828+0000","last_undegraded":"2024-09-17T05:37:42.769828+0000","last_fullsized":"2024-09-17T05:37:42.769828+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:07.189895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757332+0000","last_change":"2024-09-17T05:37:42.756890+0000","last_active":"2024-09-17T05:37:43.757332+0000","last_peered":"2024-09-17T05:37:43.757332+0000","last_clean":"2024-09-17T05:37:43.757332+0000","last_became_active":"2024-09-17T05:37:42.756610+0000","last_became_peered":"2024-09-17T05:37:42.756610+0000","last_unstale":"2024-09-17T05:37:43.757332+0000","last_undegraded":"2024-09-17T05:37:43.757332+0000","last_fullsized":"2024-09-17T05:37:43.757332+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:02.262860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757360+0000","last_change":"2024-09-17T05:37:42.769385+0000","last_active":"2024-09-17T05:37:43.757360+0000","last_peered":"2024-09-17T05:37:43.757360+0000","last_clean":"2024-09-17T05:37:43.757360+0000","last_became_active":"2024-09-17T05:37:42.769114+0000","last_became_peered":"2024-09-17T05:37:42.769114+0000","last_unstale":"2024-09-17T05:37:43.757360+0000","last_undegraded":"2024-09-17T05:37:43.757360+0000","last_fullsized":"2024-09-17T05:37:43.757360+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:00.402139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539522+0000","last_change":"2024-09-17T05:37:42.778584+0000","last_active":"2024-09-17T05:37:44.539522+0000","last_peered":"2024-09-17T05:37:44.539522+0000","last_clean":"2024-09-17T05:37:44.539522+0000","last_became_active":"2024-09-17T05:37:42.778270+0000","last_became_peered":"2024-09-17T05:37:42.778270+0000","last_unstale":"2024-09-17T05:37:44.539522+0000","last_undegraded":"2024-09-17T05:37:44.539522+0000","last_fullsized":"2024-09-17T05:37:44.539522+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:20.420049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757630+0000","last_change":"2024-09-17T05:37:42.769992+0000","last_active":"2024-09-17T05:37:43.757630+0000","last_peered":"2024-09-17T05:37:43.757630+0000","last_clean":"2024-09-17T05:37:43.757630+0000","last_became_active":"2024-09-17T05:37:42.769449+0000","last_became_peered":"2024-09-17T05:37:42.769449+0000","last_unstale":"2024-09-17T05:37:43.757630+0000","last_undegraded":"2024-09-17T05:37:43.757630+0000","last_fullsized":"2024-09-17T05:37:43.757630+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:38.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758503+0000","last_change":"2024-09-17T05:37:42.756782+0000","last_active":"2024-09-17T05:37:43.758503+0000","last_peered":"2024-09-17T05:37:43.758503+0000","last_clean":"2024-09-17T05:37:43.758503+0000","last_became_active":"2024-09-17T05:37:42.756480+0000","last_became_peered":"2024-09-17T05:37:42.756480+0000","last_unstale":"2024-09-17T05:37:43.758503+0000","last_undegraded":"2024-09-17T05:37:43.758503+0000","last_fullsized":"2024-09-17T05:37:43.758503+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:16.265003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757591+0000","last_change":"2024-09-17T05:37:42.767086+0000","last_active":"2024-09-17T05:37:43.757591+0000","last_peered":"2024-09-17T05:37:43.757591+0000","last_clean":"2024-09-17T05:37:43.757591+0000","last_became_active":"2024-09-17T05:37:42.766717+0000","last_became_peered":"2024-09-17T05:37:42.766717+0000","last_unstale":"2024-09-17T05:37:43.757591+0000","last_undegraded":"2024-09-17T05:37:43.757591+0000","last_fullsized":"2024-09-17T05:37:43.757591+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:12.958824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757682+0000","last_change":"2024-09-17T05:37:42.772392+0000","last_active":"2024-09-17T05:37:43.757682+0000","last_peered":"2024-09-17T05:37:43.757682+0000","last_clean":"2024-09-17T05:37:43.757682+0000","last_became_active":"2024-09-17T05:37:42.772190+0000","last_became_peered":"2024-09-17T05:37:42.772190+0000","last_unstale":"2024-09-17T05:37:43.757682+0000","last_undegraded":"2024-09-17T05:37:43.757682+0000","last_fullsized":"2024-09-17T05:37:43.757682+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:24.427080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.764116+0000","last_change":"2024-09-17T05:37:42.759129+0000","last_active":"2024-09-17T05:37:42.764116+0000","last_peered":"2024-09-17T05:37:42.764116+0000","last_clean":"2024-09-17T05:37:42.764116+0000","last_became_active":"2024-09-17T05:37:42.758853+0000","last_became_peered":"2024-09-17T05:37:42.758853+0000","last_unstale":"2024-09-17T05:37:42.764116+0000","last_undegraded":"2024-09-17T05:37:42.764116+0000","last_fullsized":"2024-09-17T05:37:42.764116+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:37.241223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.538487+0000","last_change":"2024-09-17T05:37:42.768544+0000","last_active":"2024-09-17T05:37:44.538487+0000","last_peered":"2024-09-17T05:37:44.538487+0000","last_clean":"2024-09-17T05:37:44.538487+0000","last_became_active":"2024-09-17T05:37:42.768249+0000","last_became_peered":"2024-09-17T05:37:42.768249+0000","last_unstale":"2024-09-17T05:37:44.538487+0000","last_undegraded":"2024-09-17T05:37:44.538487+0000","last_fullsized":"2024-09-17T05:37:44.538487+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:21.135854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.756995+0000","last_change":"2024-09-17T05:37:42.760207+0000","last_active":"2024-09-17T05:37:43.756995+0000","last_peered":"2024-09-17T05:37:43.756995+0000","last_clean":"2024-09-17T05:37:43.756995+0000","last_became_active":"2024-09-17T05:37:42.759568+0000","last_became_peered":"2024-09-17T05:37:42.759568+0000","last_unstale":"2024-09-17T05:37:43.756995+0000","last_undegraded":"2024-09-17T05:37:43.756995+0000","last_fullsized":"2024-09-17T05:37:43.756995+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:37.241223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.764993+0000","last_change":"2024-09-17T05:37:42.758916+0000","last_active":"2024-09-17T05:37:42.764993+0000","last_peered":"2024-09-17T05:37:42.764993+0000","last_clean":"2024-09-17T05:37:42.764993+0000","last_became_active":"2024-09-17T05:37:42.758624+0000","last_became_peered":"2024-09-17T05:37:42.758624+0000","last_unstale":"2024-09-17T05:37:42.764993+0000","last_undegraded":"2024-09-17T05:37:42.764993+0000","last_fullsized":"2024-09-17T05:37:42.764993+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:08.858852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758431+0000","last_change":"2024-09-17T05:37:42.762250+0000","last_active":"2024-09-17T05:37:43.758431+0000","last_peered":"2024-09-17T05:37:43.758431+0000","last_clean":"2024-09-17T05:37:43.758431+0000","last_became_active":"2024-09-17T05:37:42.761958+0000","last_became_peered":"2024-09-17T05:37:42.761958+0000","last_unstale":"2024-09-17T05:37:43.758431+0000","last_undegraded":"2024-09-17T05:37:43.758431+0000","last_fullsized":"2024-09-17T05:37:43.758431+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:08.997427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.762840+0000","last_change":"2024-09-17T05:37:42.756219+0000","last_active":"2024-09-17T05:37:42.762840+0000","last_peered":"2024-09-17T05:37:42.762840+0000","last_clean":"2024-09-17T05:37:42.762840+0000","last_became_active":"2024-09-17T05:37:42.755916+0000","last_became_peered":"2024-09-17T05:37:42.755916+0000","last_unstale":"2024-09-17T05:37:42.762840+0000","last_undegraded":"2024-09-17T05:37:42.762840+0000","last_fullsized":"2024-09-17T05:37:42.762840+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:12.171151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765045+0000","last_change":"2024-09-17T05:37:42.760052+0000","last_active":"2024-09-17T05:37:42.765045+0000","last_peered":"2024-09-17T05:37:42.765045+0000","last_clean":"2024-09-17T05:37:42.765045+0000","last_became_active":"2024-09-17T05:37:42.759751+0000","last_became_peered":"2024-09-17T05:37:42.759751+0000","last_unstale":"2024-09-17T05:37:42.765045+0000","last_undegraded":"2024-09-17T05:37:42.765045+0000","last_fullsized":"2024-09-17T05:37:42.765045+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:10.435095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758429+0000","last_change":"2024-09-17T05:37:42.760830+0000","last_active":"2024-09-17T05:37:43.758429+0000","last_peered":"2024-09-17T05:37:43.758429+0000","last_clean":"2024-09-17T05:37:43.758429+0000","last_became_active":"2024-09-17T05:37:42.760410+0000","last_became_peered":"2024-09-17T05:37:42.760410+0000","last_unstale":"2024-09-17T05:37:43.758429+0000","last_undegraded":"2024-09-17T05:37:43.758429+0000","last_fullsized":"2024-09-17T05:37:43.758429+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:31.611419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.769798+0000","last_change":"2024-09-17T05:37:42.762213+0000","last_active":"2024-09-17T05:37:42.769798+0000","last_peered":"2024-09-17T05:37:42.769798+0000","last_clean":"2024-09-17T05:37:42.769798+0000","last_became_active":"2024-09-17T05:37:42.759087+0000","last_became_peered":"2024-09-17T05:37:42.759087+0000","last_unstale":"2024-09-17T05:37:42.769798+0000","last_undegraded":"2024-09-17T05:37:42.769798+0000","last_fullsized":"2024-09-17T05:37:42.769798+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:23.040553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775983+0000","last_change":"2024-09-17T05:37:42.774874+0000","last_active":"2024-09-17T05:37:42.775983+0000","last_peered":"2024-09-17T05:37:42.775983+0000","last_clean":"2024-09-17T05:37:42.775983+0000","last_became_active":"2024-09-17T05:37:42.774426+0000","last_became_peered":"2024-09-17T05:37:42.774426+0000","last_unstale":"2024-09-17T05:37:42.775983+0000","last_undegraded":"2024-09-17T05:37:42.775983+0000","last_fullsized":"2024-09-17T05:37:42.775983+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:05.922727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.770814+0000","last_change":"2024-09-17T05:37:42.763427+0000","last_active":"2024-09-17T05:37:42.770814+0000","last_peered":"2024-09-17T05:37:42.770814+0000","last_clean":"2024-09-17T05:37:42.770814+0000","last_became_active":"2024-09-17T05:37:42.763104+0000","last_became_peered":"2024-09-17T05:37:42.763104+0000","last_unstale":"2024-09-17T05:37:42.770814+0000","last_undegraded":"2024-09-17T05:37:42.770814+0000","last_fullsized":"2024-09-17T05:37:42.770814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:38.814960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758474+0000","last_change":"2024-09-17T05:37:42.779208+0000","last_active":"2024-09-17T05:37:43.758474+0000","last_peered":"2024-09-17T05:37:43.758474+0000","last_clean":"2024-09-17T05:37:43.758474+0000","last_became_active":"2024-09-17T05:37:42.778183+0000","last_became_peered":"2024-09-17T05:37:42.778183+0000","last_unstale":"2024-09-17T05:37:43.758474+0000","last_undegraded":"2024-09-17T05:37:43.758474+0000","last_fullsized":"2024-09-17T05:37:43.758474+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:18.487725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.778282+0000","last_change":"2024-09-17T05:37:42.767382+0000","last_active":"2024-09-17T05:37:42.778282+0000","last_peered":"2024-09-17T05:37:42.778282+0000","last_clean":"2024-09-17T05:37:42.778282+0000","last_became_active":"2024-09-17T05:37:42.767016+0000","last_became_peered":"2024-09-17T05:37:42.767016+0000","last_unstale":"2024-09-17T05:37:42.778282+0000","last_undegraded":"2024-09-17T05:37:42.778282+0000","last_fullsized":"2024-09-17T05:37:42.778282+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:12.111670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.769488+0000","last_change":"2024-09-17T05:37:42.763169+0000","last_active":"2024-09-17T05:37:42.769488+0000","last_peered":"2024-09-17T05:37:42.769488+0000","last_clean":"2024-09-17T05:37:42.769488+0000","last_became_active":"2024-09-17T05:37:42.762882+0000","last_became_peered":"2024-09-17T05:37:42.762882+0000","last_unstale":"2024-09-17T05:37:42.769488+0000","last_undegraded":"2024-09-17T05:37:42.769488+0000","last_fullsized":"2024-09-17T05:37:42.769488+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:08:50.930921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.766034+0000","last_change":"2024-09-17T05:37:42.761132+0000","last_active":"2024-09-17T05:37:42.766034+0000","last_peered":"2024-09-17T05:37:42.766034+0000","last_clean":"2024-09-17T05:37:42.766034+0000","last_became_active":"2024-09-17T05:37:42.760872+0000","last_became_peered":"2024-09-17T05:37:42.760872+0000","last_unstale":"2024-09-17T05:37:42.766034+0000","last_undegraded":"2024-09-17T05:37:42.766034+0000","last_fullsized":"2024-09-17T05:37:42.766034+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:57.017435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.761239+0000","last_change":"2024-09-17T05:37:42.758724+0000","last_active":"2024-09-17T05:37:42.761239+0000","last_peered":"2024-09-17T05:37:42.761239+0000","last_clean":"2024-09-17T05:37:42.761239+0000","last_became_active":"2024-09-17T05:37:42.758416+0000","last_became_peered":"2024-09-17T05:37:42.758416+0000","last_unstale":"2024-09-17T05:37:42.761239+0000","last_undegraded":"2024-09-17T05:37:42.761239+0000","last_fullsized":"2024-09-17T05:37:42.761239+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:10.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757870+0000","last_change":"2024-09-17T05:37:42.766909+0000","last_active":"2024-09-17T05:37:43.757870+0000","last_peered":"2024-09-17T05:37:43.757870+0000","last_clean":"2024-09-17T05:37:43.757870+0000","last_became_active":"2024-09-17T05:37:42.766498+0000","last_became_peered":"2024-09-17T05:37:42.766498+0000","last_unstale":"2024-09-17T05:37:43.757870+0000","last_undegraded":"2024-09-17T05:37:43.757870+0000","last_fullsized":"2024-09-17T05:37:43.757870+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:34.011921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.538781+0000","last_change":"2024-09-17T05:37:42.765441+0000","last_active":"2024-09-17T05:37:44.538781+0000","last_peered":"2024-09-17T05:37:44.538781+0000","last_clean":"2024-09-17T05:37:44.538781+0000","last_became_active":"2024-09-17T05:37:42.765183+0000","last_became_peered":"2024-09-17T05:37:42.765183+0000","last_unstale":"2024-09-17T05:37:44.538781+0000","last_undegraded":"2024-09-17T05:37:44.538781+0000","last_fullsized":"2024-09-17T05:37:44.538781+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:54.597011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.756876+0000","last_change":"2024-09-17T05:37:42.752922+0000","last_active":"2024-09-17T05:37:43.756876+0000","last_peered":"2024-09-17T05:37:43.756876+0000","last_clean":"2024-09-17T05:37:43.756876+0000","last_became_active":"2024-09-17T05:37:42.752568+0000","last_became_peered":"2024-09-17T05:37:42.752568+0000","last_unstale":"2024-09-17T05:37:43.756876+0000","last_undegraded":"2024-09-17T05:37:43.756876+0000","last_fullsized":"2024-09-17T05:37:43.756876+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:10.343602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539489+0000","last_change":"2024-09-17T05:37:42.775360+0000","last_active":"2024-09-17T05:37:44.539489+0000","last_peered":"2024-09-17T05:37:44.539489+0000","last_clean":"2024-09-17T05:37:44.539489+0000","last_became_active":"2024-09-17T05:37:42.773254+0000","last_became_peered":"2024-09-17T05:37:42.773254+0000","last_unstale":"2024-09-17T05:37:44.539489+0000","last_undegraded":"2024-09-17T05:37:44.539489+0000","last_fullsized":"2024-09-17T05:37:44.539489+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:05.473222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"116'4","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.779101+0000","last_change":"2024-09-17T05:37:41.743245+0000","last_active":"2024-09-17T05:37:42.779101+0000","last_peered":"2024-09-17T05:37:42.779101+0000","last_clean":"2024-09-17T05:37:42.779101+0000","last_became_active":"2024-09-17T05:37:41.742970+0000","last_became_peered":"2024-09-17T05:37:41.742970+0000","last_unstale":"2024-09-17T05:37:42.779101+0000","last_undegraded":"2024-09-17T05:37:42.779101+0000","last_fullsized":"2024-09-17T05:37:42.779101+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:50.414654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775966+0000","last_change":"2024-09-17T05:37:42.774711+0000","last_active":"2024-09-17T05:37:42.775966+0000","last_peered":"2024-09-17T05:37:42.775966+0000","last_clean":"2024-09-17T05:37:42.775966+0000","last_became_active":"2024-09-17T05:37:42.774439+0000","last_became_peered":"2024-09-17T05:37:42.774439+0000","last_unstale":"2024-09-17T05:37:42.775966+0000","last_undegraded":"2024-09-17T05:37:42.775966+0000","last_fullsized":"2024-09-17T05:37:42.775966+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:38.639248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"116'1","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757929+0000","last_change":"2024-09-17T05:37:41.737668+0000","last_active":"2024-09-17T05:37:43.757929+0000","last_peered":"2024-09-17T05:37:43.757929+0000","last_clean":"2024-09-17T05:37:43.757929+0000","last_became_active":"2024-09-17T05:37:41.737399+0000","last_became_peered":"2024-09-17T05:37:41.737399+0000","last_unstale":"2024-09-17T05:37:43.757929+0000","last_undegraded":"2024-09-17T05:37:43.757929+0000","last_fullsized":"2024-09-17T05:37:43.757929+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:06.472415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775979+0000","last_change":"2024-09-17T05:37:42.769997+0000","last_active":"2024-09-17T05:37:42.775979+0000","last_peered":"2024-09-17T05:37:42.775979+0000","last_clean":"2024-09-17T05:37:42.775979+0000","last_became_active":"2024-09-17T05:37:42.769726+0000","last_became_peered":"2024-09-17T05:37:42.769726+0000","last_unstale":"2024-09-17T05:37:42.775979+0000","last_undegraded":"2024-09-17T05:37:42.775979+0000","last_fullsized":"2024-09-17T05:37:42.775979+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:32.249486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.780364+0000","last_change":"2024-09-17T05:37:41.738946+0000","last_active":"2024-09-17T05:37:42.780364+0000","last_peered":"2024-09-17T05:37:42.780364+0000","last_clean":"2024-09-17T05:37:42.780364+0000","last_became_active":"2024-09-17T05:37:41.738516+0000","last_became_peered":"2024-09-17T05:37:41.738516+0000","last_unstale":"2024-09-17T05:37:42.780364+0000","last_undegraded":"2024-09-17T05:37:42.780364+0000","last_fullsized":"2024-09-17T05:37:42.780364+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:08.463728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.762814+0000","last_change":"2024-09-17T05:37:42.755908+0000","last_active":"2024-09-17T05:37:42.762814+0000","last_peered":"2024-09-17T05:37:42.762814+0000","last_clean":"2024-09-17T05:37:42.762814+0000","last_became_active":"2024-09-17T05:37:42.755646+0000","last_became_peered":"2024-09-17T05:37:42.755646+0000","last_unstale":"2024-09-17T05:37:42.762814+0000","last_undegraded":"2024-09-17T05:37:42.762814+0000","last_fullsized":"2024-09-17T05:37:42.762814+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:22.763410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.781870+0000","last_change":"2024-09-17T05:37:41.739084+0000","last_active":"2024-09-17T05:37:42.781870+0000","last_peered":"2024-09-17T05:37:42.781870+0000","last_clean":"2024-09-17T05:37:42.781870+0000","last_became_active":"2024-09-17T05:37:41.738487+0000","last_became_peered":"2024-09-17T05:37:41.738487+0000","last_unstale":"2024-09-17T05:37:42.781870+0000","last_undegraded":"2024-09-17T05:37:42.781870+0000","last_fullsized":"2024-09-17T05:37:42.781870+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:23.337333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.774693+0000","last_change":"2024-09-17T05:37:42.767924+0000","last_active":"2024-09-17T05:37:42.774693+0000","last_peered":"2024-09-17T05:37:42.774693+0000","last_clean":"2024-09-17T05:37:42.774693+0000","last_became_active":"2024-09-17T05:37:42.767523+0000","last_became_peered":"2024-09-17T05:37:42.767523+0000","last_unstale":"2024-09-17T05:37:42.774693+0000","last_undegraded":"2024-09-17T05:37:42.774693+0000","last_fullsized":"2024-09-17T05:37:42.774693+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:07.474166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"116'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.781791+0000","last_change":"2024-09-17T05:37:41.741314+0000","last_active":"2024-09-17T05:37:42.781791+0000","last_peered":"2024-09-17T05:37:42.781791+0000","last_clean":"2024-09-17T05:37:42.781791+0000","last_became_active":"2024-09-17T05:37:41.741054+0000","last_became_peered":"2024-09-17T05:37:41.741054+0000","last_unstale":"2024-09-17T05:37:42.781791+0000","last_undegraded":"2024-09-17T05:37:42.781791+0000","last_fullsized":"2024-09-17T05:37:42.781791+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:57.860855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758373+0000","last_change":"2024-09-17T05:37:42.762175+0000","last_active":"2024-09-17T05:37:43.758373+0000","last_peered":"2024-09-17T05:37:43.758373+0000","last_clean":"2024-09-17T05:37:43.758373+0000","last_became_active":"2024-09-17T05:37:42.761869+0000","last_became_peered":"2024-09-17T05:37:42.761869+0000","last_unstale":"2024-09-17T05:37:43.758373+0000","last_undegraded":"2024-09-17T05:37:43.758373+0000","last_fullsized":"2024-09-17T05:37:43.758373+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:24.330507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"116'1","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.538978+0000","last_change":"2024-09-17T05:37:41.747733+0000","last_active":"2024-09-17T05:37:44.538978+0000","last_peered":"2024-09-17T05:37:44.538978+0000","last_clean":"2024-09-17T05:37:44.538978+0000","last_became_active":"2024-09-17T05:37:41.744934+0000","last_became_peered":"2024-09-17T05:37:41.744934+0000","last_unstale":"2024-09-17T05:37:44.538978+0000","last_undegraded":"2024-09-17T05:37:44.538978+0000","last_fullsized":"2024-09-17T05:37:44.538978+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:43.050410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757151+0000","last_change":"2024-09-17T05:37:42.759506+0000","last_active":"2024-09-17T05:37:43.757151+0000","last_peered":"2024-09-17T05:37:43.757151+0000","last_clean":"2024-09-17T05:37:43.757151+0000","last_became_active":"2024-09-17T05:37:42.758818+0000","last_became_peered":"2024-09-17T05:37:42.758818+0000","last_unstale":"2024-09-17T05:37:43.757151+0000","last_undegraded":"2024-09-17T05:37:43.757151+0000","last_fullsized":"2024-09-17T05:37:43.757151+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:40.749920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757871+0000","last_change":"2024-09-17T05:37:41.732557+0000","last_active":"2024-09-17T05:37:43.757871+0000","last_peered":"2024-09-17T05:37:43.757871+0000","last_clean":"2024-09-17T05:37:43.757871+0000","last_became_active":"2024-09-17T05:37:41.732294+0000","last_became_peered":"2024-09-17T05:37:41.732294+0000","last_unstale":"2024-09-17T05:37:43.757871+0000","last_undegraded":"2024-09-17T05:37:43.757871+0000","last_fullsized":"2024-09-17T05:37:43.757871+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:38.857046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757353+0000","last_change":"2024-09-17T05:37:42.774746+0000","last_active":"2024-09-17T05:37:43.757353+0000","last_peered":"2024-09-17T05:37:43.757353+0000","last_clean":"2024-09-17T05:37:43.757353+0000","last_became_active":"2024-09-17T05:37:42.774477+0000","last_became_peered":"2024-09-17T05:37:42.774477+0000","last_unstale":"2024-09-17T05:37:43.757353+0000","last_undegraded":"2024-09-17T05:37:43.757353+0000","last_fullsized":"2024-09-17T05:37:43.757353+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:26.385590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757359+0000","last_change":"2024-09-17T05:37:41.739046+0000","last_active":"2024-09-17T05:37:43.757359+0000","last_peered":"2024-09-17T05:37:43.757359+0000","last_clean":"2024-09-17T05:37:43.757359+0000","last_became_active":"2024-09-17T05:37:41.738751+0000","last_became_peered":"2024-09-17T05:37:41.738751+0000","last_unstale":"2024-09-17T05:37:43.757359+0000","last_undegraded":"2024-09-17T05:37:43.757359+0000","last_fullsized":"2024-09-17T05:37:43.757359+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:53.389707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758076+0000","last_change":"2024-09-17T05:37:42.772827+0000","last_active":"2024-09-17T05:37:43.758076+0000","last_peered":"2024-09-17T05:37:43.758076+0000","last_clean":"2024-09-17T05:37:43.758076+0000","last_became_active":"2024-09-17T05:37:42.771994+0000","last_became_peered":"2024-09-17T05:37:42.771994+0000","last_unstale":"2024-09-17T05:37:43.758076+0000","last_undegraded":"2024-09-17T05:37:43.758076+0000","last_fullsized":"2024-09-17T05:37:43.758076+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:34.284820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.754118+0000","last_change":"2024-09-17T05:37:41.739982+0000","last_active":"2024-09-17T05:37:42.754118+0000","last_peered":"2024-09-17T05:37:42.754118+0000","last_clean":"2024-09-17T05:37:42.754118+0000","last_became_active":"2024-09-17T05:37:41.738530+0000","last_became_peered":"2024-09-17T05:37:41.738530+0000","last_unstale":"2024-09-17T05:37:42.754118+0000","last_undegraded":"2024-09-17T05:37:42.754118+0000","last_fullsized":"2024-09-17T05:37:42.754118+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:38:28.884615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757453+0000","last_change":"2024-09-17T05:37:42.779592+0000","last_active":"2024-09-17T05:37:43.757453+0000","last_peered":"2024-09-17T05:37:43.757453+0000","last_clean":"2024-09-17T05:37:43.757453+0000","last_became_active":"2024-09-17T05:37:42.778827+0000","last_became_peered":"2024-09-17T05:37:42.778827+0000","last_unstale":"2024-09-17T05:37:43.757453+0000","last_undegraded":"2024-09-17T05:37:43.757453+0000","last_fullsized":"2024-09-17T05:37:43.757453+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:54.597011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757516+0000","last_change":"2024-09-17T05:37:41.734361+0000","last_active":"2024-09-17T05:37:43.757516+0000","last_peered":"2024-09-17T05:37:43.757516+0000","last_clean":"2024-09-17T05:37:43.757516+0000","last_became_active":"2024-09-17T05:37:41.734019+0000","last_became_peered":"2024-09-17T05:37:41.734019+0000","last_unstale":"2024-09-17T05:37:43.757516+0000","last_undegraded":"2024-09-17T05:37:43.757516+0000","last_fullsized":"2024-09-17T05:37:43.757516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:09.433345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.780225+0000","last_change":"2024-09-17T05:37:42.777269+0000","last_active":"2024-09-17T05:37:42.780225+0000","last_peered":"2024-09-17T05:37:42.780225+0000","last_clean":"2024-09-17T05:37:42.780225+0000","last_became_active":"2024-09-17T05:37:42.771127+0000","last_became_peered":"2024-09-17T05:37:42.771127+0000","last_unstale":"2024-09-17T05:37:42.780225+0000","last_undegraded":"2024-09-17T05:37:42.780225+0000","last_fullsized":"2024-09-17T05:37:42.780225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:00:29.909332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"116'3","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757526+0000","last_change":"2024-09-17T05:37:41.746444+0000","last_active":"2024-09-17T05:37:43.757526+0000","last_peered":"2024-09-17T05:37:43.757526+0000","last_clean":"2024-09-17T05:37:43.757526+0000","last_became_active":"2024-09-17T05:37:41.744658+0000","last_became_peered":"2024-09-17T05:37:41.744658+0000","last_unstale":"2024-09-17T05:37:43.757526+0000","last_undegraded":"2024-09-17T05:37:43.757526+0000","last_fullsized":"2024-09-17T05:37:43.757526+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:21.761660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758537+0000","last_change":"2024-09-17T05:37:42.768900+0000","last_active":"2024-09-17T05:37:43.758537+0000","last_peered":"2024-09-17T05:37:43.758537+0000","last_clean":"2024-09-17T05:37:43.758537+0000","last_became_active":"2024-09-17T05:37:42.768597+0000","last_became_peered":"2024-09-17T05:37:42.768597+0000","last_unstale":"2024-09-17T05:37:43.758537+0000","last_undegraded":"2024-09-17T05:37:43.758537+0000","last_fullsized":"2024-09-17T05:37:43.758537+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:09.681290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"116'6","reported_seq":19,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.783668+0000","last_change":"2024-09-17T05:37:41.738756+0000","last_active":"2024-09-17T05:37:42.783668+0000","last_peered":"2024-09-17T05:37:42.783668+0000","last_clean":"2024-09-17T05:37:42.783668+0000","last_became_active":"2024-09-17T05:37:41.738080+0000","last_became_peered":"2024-09-17T05:37:41.738080+0000","last_unstale":"2024-09-17T05:37:42.783668+0000","last_undegraded":"2024-09-17T05:37:42.783668+0000","last_fullsized":"2024-09-17T05:37:42.783668+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:43.050410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757230+0000","last_change":"2024-09-17T05:37:42.760090+0000","last_active":"2024-09-17T05:37:43.757230+0000","last_peered":"2024-09-17T05:37:43.757230+0000","last_clean":"2024-09-17T05:37:43.757230+0000","last_became_active":"2024-09-17T05:37:42.759830+0000","last_became_peered":"2024-09-17T05:37:42.759830+0000","last_unstale":"2024-09-17T05:37:43.757230+0000","last_undegraded":"2024-09-17T05:37:43.757230+0000","last_fullsized":"2024-09-17T05:37:43.757230+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:02.747051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"116'2","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758041+0000","last_change":"2024-09-17T05:37:41.743439+0000","last_active":"2024-09-17T05:37:43.758041+0000","last_peered":"2024-09-17T05:37:43.758041+0000","last_clean":"2024-09-17T05:37:43.758041+0000","last_became_active":"2024-09-17T05:37:41.742731+0000","last_became_peered":"2024-09-17T05:37:41.742731+0000","last_unstale":"2024-09-17T05:37:43.758041+0000","last_undegraded":"2024-09-17T05:37:43.758041+0000","last_fullsized":"2024-09-17T05:37:43.758041+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:11.169401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765715+0000","last_change":"2024-09-17T05:37:42.757193+0000","last_active":"2024-09-17T05:37:42.765715+0000","last_peered":"2024-09-17T05:37:42.765715+0000","last_clean":"2024-09-17T05:37:42.765715+0000","last_became_active":"2024-09-17T05:37:42.756882+0000","last_became_peered":"2024-09-17T05:37:42.756882+0000","last_unstale":"2024-09-17T05:37:42.765715+0000","last_undegraded":"2024-09-17T05:37:42.765715+0000","last_fullsized":"2024-09-17T05:37:42.765715+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:34:56.544593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.746187+0000","last_change":"2024-09-17T05:37:41.744578+0000","last_active":"2024-09-17T05:37:42.746187+0000","last_peered":"2024-09-17T05:37:42.746187+0000","last_clean":"2024-09-17T05:37:42.746187+0000","last_became_active":"2024-09-17T05:37:41.744295+0000","last_became_peered":"2024-09-17T05:37:41.744295+0000","last_unstale":"2024-09-17T05:37:42.746187+0000","last_undegraded":"2024-09-17T05:37:42.746187+0000","last_fullsized":"2024-09-17T05:37:42.746187+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:19.261239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.776363+0000","last_change":"2024-09-17T05:37:42.774436+0000","last_active":"2024-09-17T05:37:42.776363+0000","last_peered":"2024-09-17T05:37:42.776363+0000","last_clean":"2024-09-17T05:37:42.776363+0000","last_became_active":"2024-09-17T05:37:42.774152+0000","last_became_peered":"2024-09-17T05:37:42.774152+0000","last_unstale":"2024-09-17T05:37:42.776363+0000","last_undegraded":"2024-09-17T05:37:42.776363+0000","last_fullsized":"2024-09-17T05:37:42.776363+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:33.447526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":23,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.538224+0000","last_change":"2024-09-17T05:37:41.740922+0000","last_active":"2024-09-17T05:37:44.538224+0000","last_peered":"2024-09-17T05:37:44.538224+0000","last_clean":"2024-09-17T05:37:44.538224+0000","last_became_active":"2024-09-17T05:37:41.740326+0000","last_became_peered":"2024-09-17T05:37:41.740326+0000","last_unstale":"2024-09-17T05:37:44.538224+0000","last_undegraded":"2024-09-17T05:37:44.538224+0000","last_fullsized":"2024-09-17T05:37:44.538224+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:07.914885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539495+0000","last_change":"2024-09-17T05:37:42.779606+0000","last_active":"2024-09-17T05:37:44.539495+0000","last_peered":"2024-09-17T05:37:44.539495+0000","last_clean":"2024-09-17T05:37:44.539495+0000","last_became_active":"2024-09-17T05:37:42.779327+0000","last_became_peered":"2024-09-17T05:37:42.779327+0000","last_unstale":"2024-09-17T05:37:44.539495+0000","last_undegraded":"2024-09-17T05:37:44.539495+0000","last_fullsized":"2024-09-17T05:37:44.539495+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:24.427080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"116'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.780557+0000","last_change":"2024-09-17T05:37:41.746033+0000","last_active":"2024-09-17T05:37:42.780557+0000","last_peered":"2024-09-17T05:37:42.780557+0000","last_clean":"2024-09-17T05:37:42.780557+0000","last_became_active":"2024-09-17T05:37:41.745738+0000","last_became_peered":"2024-09-17T05:37:41.745738+0000","last_unstale":"2024-09-17T05:37:42.780557+0000","last_undegraded":"2024-09-17T05:37:42.780557+0000","last_fullsized":"2024-09-17T05:37:42.780557+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:36.686851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539720+0000","last_change":"2024-09-17T05:37:42.774744+0000","last_active":"2024-09-17T05:37:44.539720+0000","last_peered":"2024-09-17T05:37:44.539720+0000","last_clean":"2024-09-17T05:37:44.539720+0000","last_became_active":"2024-09-17T05:37:42.774480+0000","last_became_peered":"2024-09-17T05:37:42.774480+0000","last_unstale":"2024-09-17T05:37:44.539720+0000","last_undegraded":"2024-09-17T05:37:44.539720+0000","last_fullsized":"2024-09-17T05:37:44.539720+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:24.700744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757637+0000","last_change":"2024-09-17T05:37:41.746517+0000","last_active":"2024-09-17T05:37:43.757637+0000","last_peered":"2024-09-17T05:37:43.757637+0000","last_clean":"2024-09-17T05:37:43.757637+0000","last_became_active":"2024-09-17T05:37:41.742602+0000","last_became_peered":"2024-09-17T05:37:41.742602+0000","last_unstale":"2024-09-17T05:37:43.757637+0000","last_undegraded":"2024-09-17T05:37:43.757637+0000","last_fullsized":"2024-09-17T05:37:43.757637+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:14.741485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758499+0000","last_change":"2024-09-17T05:37:42.769889+0000","last_active":"2024-09-17T05:37:43.758499+0000","last_peered":"2024-09-17T05:37:43.758499+0000","last_clean":"2024-09-17T05:37:43.758499+0000","last_became_active":"2024-09-17T05:37:42.769630+0000","last_became_peered":"2024-09-17T05:37:42.769630+0000","last_unstale":"2024-09-17T05:37:43.758499+0000","last_undegraded":"2024-09-17T05:37:43.758499+0000","last_fullsized":"2024-09-17T05:37:43.758499+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:33.969613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"116'1","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.781862+0000","last_change":"2024-09-17T05:37:41.739784+0000","last_active":"2024-09-17T05:37:42.781862+0000","last_peered":"2024-09-17T05:37:42.781862+0000","last_clean":"2024-09-17T05:37:42.781862+0000","last_became_active":"2024-09-17T05:37:41.739044+0000","last_became_peered":"2024-09-17T05:37:41.739044+0000","last_unstale":"2024-09-17T05:37:42.781862+0000","last_undegraded":"2024-09-17T05:37:42.781862+0000","last_fullsized":"2024-09-17T05:37:42.781862+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:38.033830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765906+0000","last_change":"2024-09-17T05:37:42.764734+0000","last_active":"2024-09-17T05:37:42.765906+0000","last_peered":"2024-09-17T05:37:42.765906+0000","last_clean":"2024-09-17T05:37:42.765906+0000","last_became_active":"2024-09-17T05:37:42.764408+0000","last_became_peered":"2024-09-17T05:37:42.764408+0000","last_unstale":"2024-09-17T05:37:42.765906+0000","last_undegraded":"2024-09-17T05:37:42.765906+0000","last_fullsized":"2024-09-17T05:37:42.765906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:40.749920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.750543+0000","last_change":"2024-09-17T05:37:41.744912+0000","last_active":"2024-09-17T05:37:42.750543+0000","last_peered":"2024-09-17T05:37:42.750543+0000","last_clean":"2024-09-17T05:37:42.750543+0000","last_became_active":"2024-09-17T05:37:41.743871+0000","last_became_peered":"2024-09-17T05:37:41.743871+0000","last_unstale":"2024-09-17T05:37:42.750543+0000","last_undegraded":"2024-09-17T05:37:42.750543+0000","last_fullsized":"2024-09-17T05:37:42.750543+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:42.169420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.777585+0000","last_change":"2024-09-17T05:37:42.775482+0000","last_active":"2024-09-17T05:37:42.777585+0000","last_peered":"2024-09-17T05:37:42.777585+0000","last_clean":"2024-09-17T05:37:42.777585+0000","last_became_active":"2024-09-17T05:37:42.775213+0000","last_became_peered":"2024-09-17T05:37:42.775213+0000","last_unstale":"2024-09-17T05:37:42.777585+0000","last_undegraded":"2024-09-17T05:37:42.777585+0000","last_fullsized":"2024-09-17T05:37:42.777585+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:13:11.996924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"116'2","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758161+0000","last_change":"2024-09-17T05:37:41.733952+0000","last_active":"2024-09-17T05:37:43.758161+0000","last_peered":"2024-09-17T05:37:43.758161+0000","last_clean":"2024-09-17T05:37:43.758161+0000","last_became_active":"2024-09-17T05:37:41.733642+0000","last_became_peered":"2024-09-17T05:37:41.733642+0000","last_unstale":"2024-09-17T05:37:43.758161+0000","last_undegraded":"2024-09-17T05:37:43.758161+0000","last_fullsized":"2024-09-17T05:37:43.758161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:07.857101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539528+0000","last_change":"2024-09-17T05:37:42.773733+0000","last_active":"2024-09-17T05:37:44.539528+0000","last_peered":"2024-09-17T05:37:44.539528+0000","last_clean":"2024-09-17T05:37:44.539528+0000","last_became_active":"2024-09-17T05:37:42.773465+0000","last_became_peered":"2024-09-17T05:37:42.773465+0000","last_unstale":"2024-09-17T05:37:44.539528+0000","last_undegraded":"2024-09-17T05:37:44.539528+0000","last_fullsized":"2024-09-17T05:37:44.539528+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:48.048484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539226+0000","last_change":"2024-09-17T05:37:41.746049+0000","last_active":"2024-09-17T05:37:44.539226+0000","last_peered":"2024-09-17T05:37:44.539226+0000","last_clean":"2024-09-17T05:37:44.539226+0000","last_became_active":"2024-09-17T05:37:41.745264+0000","last_became_peered":"2024-09-17T05:37:41.745264+0000","last_unstale":"2024-09-17T05:37:44.539226+0000","last_undegraded":"2024-09-17T05:37:44.539226+0000","last_fullsized":"2024-09-17T05:37:44.539226+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:54.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765478+0000","last_change":"2024-09-17T05:37:42.763362+0000","last_active":"2024-09-17T05:37:42.765478+0000","last_peered":"2024-09-17T05:37:42.765478+0000","last_clean":"2024-09-17T05:37:42.765478+0000","last_became_active":"2024-09-17T05:37:42.762740+0000","last_became_peered":"2024-09-17T05:37:42.762740+0000","last_unstale":"2024-09-17T05:37:42.765478+0000","last_undegraded":"2024-09-17T05:37:42.765478+0000","last_fullsized":"2024-09-17T05:37:42.765478+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:02.747051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.753141+0000","last_change":"2024-09-17T05:37:41.743622+0000","last_active":"2024-09-17T05:37:42.753141+0000","last_peered":"2024-09-17T05:37:42.753141+0000","last_clean":"2024-09-17T05:37:42.753141+0000","last_became_active":"2024-09-17T05:37:41.743165+0000","last_became_peered":"2024-09-17T05:37:41.743165+0000","last_unstale":"2024-09-17T05:37:42.753141+0000","last_undegraded":"2024-09-17T05:37:42.753141+0000","last_fullsized":"2024-09-17T05:37:42.753141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:06.691127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757054+0000","last_change":"2024-09-17T05:37:42.759126+0000","last_active":"2024-09-17T05:37:43.757054+0000","last_peered":"2024-09-17T05:37:43.757054+0000","last_clean":"2024-09-17T05:37:43.757054+0000","last_became_active":"2024-09-17T05:37:42.758843+0000","last_became_peered":"2024-09-17T05:37:42.758843+0000","last_unstale":"2024-09-17T05:37:43.757054+0000","last_undegraded":"2024-09-17T05:37:43.757054+0000","last_fullsized":"2024-09-17T05:37:43.757054+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:05.716742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.753725+0000","last_change":"2024-09-17T05:37:41.736249+0000","last_active":"2024-09-17T05:37:42.753725+0000","last_peered":"2024-09-17T05:37:42.753725+0000","last_clean":"2024-09-17T05:37:42.753725+0000","last_became_active":"2024-09-17T05:37:41.735343+0000","last_became_peered":"2024-09-17T05:37:41.735343+0000","last_unstale":"2024-09-17T05:37:42.753725+0000","last_undegraded":"2024-09-17T05:37:42.753725+0000","last_fullsized":"2024-09-17T05:37:42.753725+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:17.386932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.765907+0000","last_change":"2024-09-17T05:37:42.764881+0000","last_active":"2024-09-17T05:37:42.765907+0000","last_peered":"2024-09-17T05:37:42.765907+0000","last_clean":"2024-09-17T05:37:42.765907+0000","last_became_active":"2024-09-17T05:37:42.763651+0000","last_became_peered":"2024-09-17T05:37:42.763651+0000","last_unstale":"2024-09-17T05:37:42.765907+0000","last_undegraded":"2024-09-17T05:37:42.765907+0000","last_fullsized":"2024-09-17T05:37:42.765907+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:12.958824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"116'2","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.781300+0000","last_change":"2024-09-17T05:37:41.740894+0000","last_active":"2024-09-17T05:37:42.781300+0000","last_peered":"2024-09-17T05:37:42.781300+0000","last_clean":"2024-09-17T05:37:42.781300+0000","last_became_active":"2024-09-17T05:37:41.739553+0000","last_became_peered":"2024-09-17T05:37:41.739553+0000","last_unstale":"2024-09-17T05:37:42.781300+0000","last_undegraded":"2024-09-17T05:37:42.781300+0000","last_fullsized":"2024-09-17T05:37:42.781300+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:45.980792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.780440+0000","last_change":"2024-09-17T05:37:42.776959+0000","last_active":"2024-09-17T05:37:42.780440+0000","last_peered":"2024-09-17T05:37:42.780440+0000","last_clean":"2024-09-17T05:37:42.780440+0000","last_became_active":"2024-09-17T05:37:42.776653+0000","last_became_peered":"2024-09-17T05:37:42.776653+0000","last_unstale":"2024-09-17T05:37:42.780440+0000","last_undegraded":"2024-09-17T05:37:42.780440+0000","last_fullsized":"2024-09-17T05:37:42.780440+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:06.870759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758365+0000","last_change":"2024-09-17T05:37:41.738484+0000","last_active":"2024-09-17T05:37:43.758365+0000","last_peered":"2024-09-17T05:37:43.758365+0000","last_clean":"2024-09-17T05:37:43.758365+0000","last_became_active":"2024-09-17T05:37:41.738220+0000","last_became_peered":"2024-09-17T05:37:41.738220+0000","last_unstale":"2024-09-17T05:37:43.758365+0000","last_undegraded":"2024-09-17T05:37:43.758365+0000","last_fullsized":"2024-09-17T05:37:43.758365+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:06.614037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539478+0000","last_change":"2024-09-17T05:37:42.767193+0000","last_active":"2024-09-17T05:37:44.539478+0000","last_peered":"2024-09-17T05:37:44.539478+0000","last_clean":"2024-09-17T05:37:44.539478+0000","last_became_active":"2024-09-17T05:37:42.766906+0000","last_became_peered":"2024-09-17T05:37:42.766906+0000","last_unstale":"2024-09-17T05:37:44.539478+0000","last_undegraded":"2024-09-17T05:37:44.539478+0000","last_fullsized":"2024-09-17T05:37:44.539478+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:21.794149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.754373+0000","last_change":"2024-09-17T05:37:41.742333+0000","last_active":"2024-09-17T05:37:42.754373+0000","last_peered":"2024-09-17T05:37:42.754373+0000","last_clean":"2024-09-17T05:37:42.754373+0000","last_became_active":"2024-09-17T05:37:41.741956+0000","last_became_peered":"2024-09-17T05:37:41.741956+0000","last_unstale":"2024-09-17T05:37:42.754373+0000","last_undegraded":"2024-09-17T05:37:42.754373+0000","last_fullsized":"2024-09-17T05:37:42.754373+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:07.914885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.760039+0000","last_change":"2024-09-17T05:37:42.757770+0000","last_active":"2024-09-17T05:37:42.760039+0000","last_peered":"2024-09-17T05:37:42.760039+0000","last_clean":"2024-09-17T05:37:42.760039+0000","last_became_active":"2024-09-17T05:37:42.757437+0000","last_became_peered":"2024-09-17T05:37:42.757437+0000","last_unstale":"2024-09-17T05:37:42.760039+0000","last_undegraded":"2024-09-17T05:37:42.760039+0000","last_fullsized":"2024-09-17T05:37:42.760039+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:01.475301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.755531+0000","last_change":"2024-09-17T05:37:41.740909+0000","last_active":"2024-09-17T05:37:42.755531+0000","last_peered":"2024-09-17T05:37:42.755531+0000","last_clean":"2024-09-17T05:37:42.755531+0000","last_became_active":"2024-09-17T05:37:41.740592+0000","last_became_peered":"2024-09-17T05:37:41.740592+0000","last_unstale":"2024-09-17T05:37:42.755531+0000","last_undegraded":"2024-09-17T05:37:42.755531+0000","last_fullsized":"2024-09-17T05:37:42.755531+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:54.481451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.763468+0000","last_change":"2024-09-17T05:37:42.757895+0000","last_active":"2024-09-17T05:37:42.763468+0000","last_peered":"2024-09-17T05:37:42.763468+0000","last_clean":"2024-09-17T05:37:42.763468+0000","last_became_active":"2024-09-17T05:37:42.757625+0000","last_became_peered":"2024-09-17T05:37:42.757625+0000","last_unstale":"2024-09-17T05:37:42.763468+0000","last_undegraded":"2024-09-17T05:37:42.763468+0000","last_fullsized":"2024-09-17T05:37:42.763468+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:54.391457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.758547+0000","last_change":"2024-09-17T05:37:41.734730+0000","last_active":"2024-09-17T05:37:43.758547+0000","last_peered":"2024-09-17T05:37:43.758547+0000","last_clean":"2024-09-17T05:37:43.758547+0000","last_became_active":"2024-09-17T05:37:41.734476+0000","last_became_peered":"2024-09-17T05:37:41.734476+0000","last_unstale":"2024-09-17T05:37:43.758547+0000","last_undegraded":"2024-09-17T05:37:43.758547+0000","last_fullsized":"2024-09-17T05:37:43.758547+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:18.422760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.762517+0000","last_change":"2024-09-17T05:37:42.760688+0000","last_active":"2024-09-17T05:37:42.762517+0000","last_peered":"2024-09-17T05:37:42.762517+0000","last_clean":"2024-09-17T05:37:42.762517+0000","last_became_active":"2024-09-17T05:37:42.760438+0000","last_became_peered":"2024-09-17T05:37:42.760438+0000","last_unstale":"2024-09-17T05:37:42.762517+0000","last_undegraded":"2024-09-17T05:37:42.762517+0000","last_fullsized":"2024-09-17T05:37:42.762517+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:05.716742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.750106+0000","last_change":"2024-09-17T05:37:41.746893+0000","last_active":"2024-09-17T05:37:42.750106+0000","last_peered":"2024-09-17T05:37:42.750106+0000","last_clean":"2024-09-17T05:37:42.750106+0000","last_became_active":"2024-09-17T05:37:41.746686+0000","last_became_peered":"2024-09-17T05:37:41.746686+0000","last_unstale":"2024-09-17T05:37:42.750106+0000","last_undegraded":"2024-09-17T05:37:42.750106+0000","last_fullsized":"2024-09-17T05:37:42.750106+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:29:41.990740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.776689+0000","last_change":"2024-09-17T05:37:42.775654+0000","last_active":"2024-09-17T05:37:42.776689+0000","last_peered":"2024-09-17T05:37:42.776689+0000","last_clean":"2024-09-17T05:37:42.776689+0000","last_became_active":"2024-09-17T05:37:42.775441+0000","last_became_peered":"2024-09-17T05:37:42.775441+0000","last_unstale":"2024-09-17T05:37:42.776689+0000","last_undegraded":"2024-09-17T05:37:42.776689+0000","last_fullsized":"2024-09-17T05:37:42.776689+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:20.793860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:44.539254+0000","last_change":"2024-09-17T05:37:41.743435+0000","last_active":"2024-09-17T05:37:44.539254+0000","last_peered":"2024-09-17T05:37:44.539254+0000","last_clean":"2024-09-17T05:37:44.539254+0000","last_became_active":"2024-09-17T05:37:41.741689+0000","last_became_peered":"2024-09-17T05:37:41.741689+0000","last_unstale":"2024-09-17T05:37:44.539254+0000","last_undegraded":"2024-09-17T05:37:44.539254+0000","last_fullsized":"2024-09-17T05:37:44.539254+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:23.337333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.775961+0000","last_change":"2024-09-17T05:37:42.765783+0000","last_active":"2024-09-17T05:37:42.775961+0000","last_peered":"2024-09-17T05:37:42.775961+0000","last_clean":"2024-09-17T05:37:42.775961+0000","last_became_active":"2024-09-17T05:37:42.764580+0000","last_became_peered":"2024-09-17T05:37:42.764580+0000","last_unstale":"2024-09-17T05:37:42.775961+0000","last_undegraded":"2024-09-17T05:37:42.775961+0000","last_fullsized":"2024-09-17T05:37:42.775961+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:57.014526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.762958+0000","last_change":"2024-09-17T05:37:41.746689+0000","last_active":"2024-09-17T05:37:42.762958+0000","last_peered":"2024-09-17T05:37:42.762958+0000","last_clean":"2024-09-17T05:37:42.762958+0000","last_became_active":"2024-09-17T05:37:41.746420+0000","last_became_peered":"2024-09-17T05:37:41.746420+0000","last_unstale":"2024-09-17T05:37:42.762958+0000","last_undegraded":"2024-09-17T05:37:42.762958+0000","last_fullsized":"2024-09-17T05:37:42.762958+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:28:31.889083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.767985+0000","last_change":"2024-09-17T05:37:42.759191+0000","last_active":"2024-09-17T05:37:42.767985+0000","last_peered":"2024-09-17T05:37:42.767985+0000","last_clean":"2024-09-17T05:37:42.767985+0000","last_became_active":"2024-09-17T05:37:42.758287+0000","last_became_peered":"2024-09-17T05:37:42.758287+0000","last_unstale":"2024-09-17T05:37:42.767985+0000","last_undegraded":"2024-09-17T05:37:42.767985+0000","last_fullsized":"2024-09-17T05:37:42.767985+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:41.719517+0000","last_clean_scrub_stamp":"2024-09-17T05:37:41.719517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:45.112505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T05:37:43.757717+0000","last_change":"2024-09-17T05:37:41.743122+0000","last_active":"2024-09-17T05:37:43.757717+0000","last_peered":"2024-09-17T05:37:43.757717+0000","last_clean":"2024-09-17T05:37:43.757717+0000","last_became_active":"2024-09-17T05:37:41.742861+0000","last_became_peered":"2024-09-17T05:37:41.742861+0000","last_unstale":"2024-09-17T05:37:43.757717+0000","last_undegraded":"2024-09-17T05:37:43.757717+0000","last_fullsized":"2024-09-17T05:37:43.757717+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:37:40.717767+0000","last_clean_scrub_stamp":"2024-09-17T05:37:40.717767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:17:37.813210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":290,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-17T05:37:42.745456+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:37:42.745456+0000","last_peered":"2024-09-17T05:37:42.745456+0000","last_clean":"2024-09-17T05:37:42.745456+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:37:42.745456+0000","last_undegraded":"2024-09-17T05:37:42.745456+0000","last_fullsized":"2024-09-17T05:37:42.745456+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104752,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267088,"statfs":{"total":96636764160,"available":96529498112,"internally_reserved":0,"allocated":1466368,"data_stored":635321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":105773514},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149132,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222708,"statfs":{"total":96636764160,"available":96484052992,"internally_reserved":0,"allocated":1433600,"data_stored":627201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509597,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126944,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244896,"statfs":{"total":96636764160,"available":96506773504,"internally_reserved":0,"allocated":1449984,"data_stored":628903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,13],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509596,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126928,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244912,"statfs":{"total":96636764160,"available":96506789888,"internally_reserved":0,"allocated":1433600,"data_stored":627201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509597,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122780,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249060,"statfs":{"total":96636764160,"available":96511037440,"internally_reserved":0,"allocated":1445888,"data_stored":627839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509598,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141400,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230440,"statfs":{"total":96636764160,"available":96491970560,"internally_reserved":0,"allocated":2027520,"data_stored":1217569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104732,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94267108,"statfs":{"total":96636764160,"available":96529518592,"internally_reserved":0,"allocated":1441792,"data_stored":627853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":105773515},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542302,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131676,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240164,"statfs":{"total":96636764160,"available":96501927936,"internally_reserved":0,"allocated":2035712,"data_stored":1217661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:37:47.249 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:37:47.249 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:37:47.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- 172.21.15.122:0/847359790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c88004d50 msgr2=0x7f2c880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 --2- 172.21.15.122:0/847359790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c88004d50 0x7f2c880051d0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00c640 tx=0x7f2c9c01d2c0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2ca9600640 1 -- 172.21.15.122:0/847359790 reap_dead start 2024-09-17T05:37:47.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- 172.21.15.122:0/847359790 shutdown_connections 2024-09-17T05:37:47.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- 172.21.15.122:0/847359790 >> 172.21.15.122:0/847359790 conn(0x7f2ca40d1dc0 msgr2=0x7f2ca40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:47.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- 172.21.15.122:0/847359790 shutdown_connections 2024-09-17T05:37:47.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- 172.21.15.122:0/847359790 wait complete. 2024-09-17T05:37:47.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 Processor -- start 2024-09-17T05:37:47.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- start start 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88004d50 0x7f2ca40c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca40cc830 0x7f2ca40cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca406f050 0x7f2ca41f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca40d03f0 con 0x7f2ca40cc830 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca40d0270 con 0x7f2c88004d50 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2cab600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca40d0570 con 0x7f2ca406f050 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2c9be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca40cc830 0x7f2ca40cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2c9be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca40cc830 0x7f2ca40cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47990/0 (socket says 172.21.15.122:47990) 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2c9be00640 1 -- 172.21.15.122:0/2872512348 learned_addr learned my addr 172.21.15.122:0/2872512348 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:47.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2ca8c00640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88004d50 0x7f2ca40c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2ca9600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca406f050 0x7f2ca41f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2c9be00640 1 -- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca406f050 msgr2=0x7f2ca41f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2c9be00640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca406f050 0x7f2ca41f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2c9be00640 1 -- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88004d50 msgr2=0x7f2ca40c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2c9be00640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88004d50 0x7f2ca40c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2c9be00640 1 -- 172.21.15.122:0/2872512348 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca406f5c0 con 0x7f2ca40cc830 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2ca9600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca406f050 0x7f2ca41f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.441+0000 7f2ca8c00640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88004d50 0x7f2ca40c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2c9be00640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca40cc830 0x7f2ca40cdeb0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c004ee0 tx=0x7f2c8c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2ca1600640 1 -- 172.21.15.122:0/2872512348 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c015020 con 0x7f2ca40cc830 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2ca1600640 1 -- 172.21.15.122:0/2872512348 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c8c011960 con 0x7f2ca40cc830 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca406f8a0 con 0x7f2ca40cc830 2024-09-17T05:37:47.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2ca1600640 1 -- 172.21.15.122:0/2872512348 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c011d90 con 0x7f2ca40cc830 2024-09-17T05:37:47.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca4159680 con 0x7f2ca40cc830 2024-09-17T05:37:47.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca415fc20 con 0x7f2ca40cc830 2024-09-17T05:37:47.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2ca1600640 1 -- 172.21.15.122:0/2872512348 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f2c8c018020 con 0x7f2ca40cc830 2024-09-17T05:37:47.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2ca1600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c80074c70 0x7f2c80077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.445+0000 7f2ca1600640 1 -- 172.21.15.122:0/2872512348 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7f2c8c093c80 con 0x7f2ca40cc830 2024-09-17T05:37:47.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.449+0000 7f2ca8c00640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c80074c70 0x7f2c80077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.449+0000 7f2ca8c00640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c80074c70 0x7f2c80077130 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009bf0 tx=0x7f2c940091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:47.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.449+0000 7f2ca1600640 1 -- 172.21.15.122:0/2872512348 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2c8c01f020 con 0x7f2ca40cc830 2024-09-17T05:37:47.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.681+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2ca4074c70 con 0x7f2ca40cc830 2024-09-17T05:37:47.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.685+0000 7f2ca1600640 1 -- 172.21.15.122:0/2872512348 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f2c8c060710 con 0x7f2ca40cc830 2024-09-17T05:37:47.683 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:37:47.683 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:37:47.683 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:37:47.683 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:37:47.683 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:37:47.683 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:37:47.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c80074c70 msgr2=0x7f2c80077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c80074c70 0x7f2c80077130 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009bf0 tx=0x7f2c940091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca40cc830 msgr2=0x7f2ca40cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca40cc830 0x7f2ca40cdeb0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c004ee0 tx=0x7f2c8c001640 comp rx=0 tx=0).stop 2024-09-17T05:37:47.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 shutdown_connections 2024-09-17T05:37:47.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c80074c70 0x7f2c80077130 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ca406f050 0x7f2ca41f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ca40cc830 0x7f2ca40cdeb0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 --2- 172.21.15.122:0/2872512348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c88004d50 0x7f2ca40c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 >> 172.21.15.122:0/2872512348 conn(0x7f2ca40d1dc0 msgr2=0x7f2ca4158400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 shutdown_connections 2024-09-17T05:37:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.689+0000 7f2cab600640 1 -- 172.21.15.122:0/2872512348 wait complete. 2024-09-17T05:37:47.705 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:37:47.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.853+0000 7f769ae00640 1 -- 172.21.15.122:0/3622567098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769406e7a0 msgr2=0x7f769406ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.853+0000 7f769ae00640 1 --2- 172.21.15.122:0/3622567098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769406e7a0 0x7f769406ec30 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f768c009c80 tx=0x7f768c01f5a0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.853+0000 7f769ae00640 1 -- 172.21.15.122:0/3622567098 shutdown_connections 2024-09-17T05:37:47.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.853+0000 7f769ae00640 1 --2- 172.21.15.122:0/3622567098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769406e7a0 0x7f769406ec30 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.853+0000 7f769ae00640 1 --2- 172.21.15.122:0/3622567098 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f769415c3e0 0x7f769415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.853+0000 7f769ae00640 1 -- 172.21.15.122:0/3622567098 >> 172.21.15.122:0/3622567098 conn(0x7f76940b18f0 msgr2=0x7f76940b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:47.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.853+0000 7f769ae00640 1 -- 172.21.15.122:0/3622567098 shutdown_connections 2024-09-17T05:37:47.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 -- 172.21.15.122:0/3622567098 wait complete. 2024-09-17T05:37:47.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 Processor -- start 2024-09-17T05:37:47.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 -- start start 2024-09-17T05:37:47.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f769406e7a0 0x7f769409adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f769415bfa0 0x7f76941e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769415c3e0 0x7f769409bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f769406f2f0 con 0x7f769415c3e0 2024-09-17T05:37:47.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f769406f170 con 0x7f769415bfa0 2024-09-17T05:37:47.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f769ae00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f769406f470 con 0x7f769406e7a0 2024-09-17T05:37:47.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769415c3e0 0x7f769409bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769415c3e0 0x7f769409bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48010/0 (socket says 172.21.15.122:48010) 2024-09-17T05:37:47.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 -- 172.21.15.122:0/988439096 learned_addr learned my addr 172.21.15.122:0/988439096 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:47.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7693e00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f769406e7a0 0x7f769409adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f768b400640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f769415bfa0 0x7f76941e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 -- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f769406e7a0 msgr2=0x7f769409adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f769406e7a0 0x7f769409adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 -- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f769415bfa0 msgr2=0x7f76941e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f769415bfa0 0x7f76941e8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 -- 172.21.15.122:0/988439096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76941f5b20 con 0x7f769415c3e0 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7698e00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769415c3e0 0x7f769409bbb0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f768c009bd0 tx=0x7f768c0af410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.857+0000 7f7690c00640 1 -- 172.21.15.122:0/988439096 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c004e60 con 0x7f769415c3e0 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f7690c00640 1 -- 172.21.15.122:0/988439096 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f768c0be030 con 0x7f769415c3e0 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76941f5840 con 0x7f769415c3e0 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f7690c00640 1 -- 172.21.15.122:0/988439096 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c0afae0 con 0x7f769415c3e0 2024-09-17T05:37:47.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76940b3a30 con 0x7f769415c3e0 2024-09-17T05:37:47.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7694069410 con 0x7f769415c3e0 2024-09-17T05:37:47.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f7690c00640 1 -- 172.21.15.122:0/988439096 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f768c0afc80 con 0x7f769415c3e0 2024-09-17T05:37:47.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f7690c00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7670074c70 0x7f7670077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:47.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f7690c00640 1 -- 172.21.15.122:0/988439096 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5851+0+0 (secure 0 0 0) 0x7f768c131b00 con 0x7f769415c3e0 2024-09-17T05:37:47.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.861+0000 7f7693e00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7670074c70 0x7f7670077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:47.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.865+0000 7f7693e00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7670074c70 0x7f7670077130 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f768400bb50 tx=0x7f768400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:47.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:47.865+0000 7f7690c00640 1 -- 172.21.15.122:0/988439096 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f768c0ac050 con 0x7f769415c3e0 2024-09-17T05:37:48.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.069+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76940cbf00 con 0x7f769415c3e0 2024-09-17T05:37:48.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.069+0000 7f7690c00640 1 -- 172.21.15.122:0/988439096 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5234 (secure 0 0 0) 0x7f768c019100 con 0x7f769415c3e0 2024-09-17T05:37:48.065 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:37:48.065 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":76,"btime":"2024-09-17T05:37:45:771684+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/4034540416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":4034540416},{"type":"v1","addr":"172.21.15.173:6836","nonce":4034540416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7651,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4130637584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4130637584},{"type":"v1","addr":"172.21.15.173:6837","nonce":4130637584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/543665156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":543665156},{"type":"v1","addr":"172.21.15.122:6833","nonce":543665156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:37:42.117978+0000","modified":"2024-09-17T05:37:45.159422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2875934571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2875934571},{"type":"v1","addr":"172.21.15.122:6835","nonce":2875934571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8364,"qdb_cluster":[8364]},"id":8}]} 2024-09-17T05:37:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 76 2024-09-17T05:37:48.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7670074c70 msgr2=0x7f7670077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7670074c70 0x7f7670077130 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f768400bb50 tx=0x7f768400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769415c3e0 msgr2=0x7f769409bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769415c3e0 0x7f769409bbb0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f768c009bd0 tx=0x7f768c0af410 comp rx=0 tx=0).stop 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 shutdown_connections 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7670074c70 0x7f7670077130 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f769415c3e0 0x7f769409bbb0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f769415bfa0 0x7f76941e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 --2- 172.21.15.122:0/988439096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f769406e7a0 0x7f769409adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 >> 172.21.15.122:0/988439096 conn(0x7f76940b18f0 msgr2=0x7f76940ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 shutdown_connections 2024-09-17T05:37:48.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.073+0000 7f769ae00640 1 -- 172.21.15.122:0/988439096 wait complete. 2024-09-17T05:37:48.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:37:42.117978+0000', 'modified': '2024-09-17T05:37:45.159422+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8364}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8364': {'gid': 8364, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6835/2875934571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 2875934571}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 2875934571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8364, 'qdb_cluster': [8364]} 2024-09-17T05:37:48.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:37:48.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T05:37:48.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- 172.21.15.122:0/3241178416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9880b2d00 msgr2=0x7ff98815fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 --2- 172.21.15.122:0/3241178416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9880b2d00 0x7ff98815fec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff970009bc0 tx=0x7ff97001cb10 comp rx=0 tx=0).stop 2024-09-17T05:37:48.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- 172.21.15.122:0/3241178416 shutdown_connections 2024-09-17T05:37:48.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 --2- 172.21.15.122:0/3241178416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9880b2d00 0x7ff98815fec0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 --2- 172.21.15.122:0/3241178416 >> v2:172.21.15.173:3300/0 conn(0x7ff9880cfe80 0x7ff9880d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- 172.21.15.122:0/3241178416 >> 172.21.15.122:0/3241178416 conn(0x7ff9880d1e40 msgr2=0x7ff9880d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:48.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- 172.21.15.122:0/3241178416 shutdown_connections 2024-09-17T05:37:48.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- 172.21.15.122:0/3241178416 wait complete. 2024-09-17T05:37:48.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 Processor -- start 2024-09-17T05:37:48.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- start start 2024-09-17T05:37:48.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9880b2d00 0x7ff98815b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9880cfe80 0x7ff9881f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff988160400 0x7ff9880c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff988169520 con 0x7ff9880b2d00 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881693a0 con 0x7ff988160400 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff98da00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff9881696a0 con 0x7ff9880cfe80 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff986a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9880b2d00 0x7ff98815b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff986a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9880b2d00 0x7ff98815b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48020/0 (socket says 172.21.15.122:48020) 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.641+0000 7ff986a00640 1 -- 172.21.15.122:0/2772071000 learned_addr learned my addr 172.21.15.122:0/2772071000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff97e000640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9880cfe80 0x7ff9881f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff987400640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff988160400 0x7ff9880c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff986a00640 1 -- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9880cfe80 msgr2=0x7ff9881f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff986a00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9880cfe80 0x7ff9881f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff986a00640 1 -- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff988160400 msgr2=0x7ff9880c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff986a00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff988160400 0x7ff9880c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff986a00640 1 -- 172.21.15.122:0/2772071000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9880c7530 con 0x7ff9880b2d00 2024-09-17T05:37:48.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff986a00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9880b2d00 0x7ff98815b2f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ff97800ea40 tx=0x7ff97800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:48.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff97f400640 1 -- 172.21.15.122:0/2772071000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97800ce40 con 0x7ff9880b2d00 2024-09-17T05:37:48.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff97f400640 1 -- 172.21.15.122:0/2772071000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff978004020 con 0x7ff9880b2d00 2024-09-17T05:37:48.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff97f400640 1 -- 172.21.15.122:0/2772071000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978004470 con 0x7ff9880b2d00 2024-09-17T05:37:48.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff98da00640 1 -- 172.21.15.122:0/2772071000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9880c7810 con 0x7ff9880b2d00 2024-09-17T05:37:48.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff98da00640 1 -- 172.21.15.122:0/2772071000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9880c5dd0 con 0x7ff9880b2d00 2024-09-17T05:37:48.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.645+0000 7ff97f400640 1 -- 172.21.15.122:0/2772071000 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7ff978003600 con 0x7ff9880b2d00 2024-09-17T05:37:48.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97f400640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff964074c70 0x7ff964077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 Processor -- start 2024-09-17T05:37:48.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97f400640 1 -- 172.21.15.122:0/2772071000 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5851+0+0 (secure 0 0 0) 0x7ff978093980 con 0x7ff9880b2d00 2024-09-17T05:37:48.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- start start 2024-09-17T05:37:48.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff964074c70 0x7ff964077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 0x563203e76db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203e2a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e2b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563203d4b3f0 con 0x563203e772f0 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563203c88b20 con 0x563203e2ae10 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563203c77f50 con 0x563203e769d0 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203e2a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203e2a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48028/0 (socket says 172.21.15.122:48028) 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 -- 172.21.15.122:0/4274581197 learned_addr learned my addr 172.21.15.122:0/4274581197 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:48.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff964074c70 0x7ff964077130 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff9700af000 tx=0x7ff97001b350 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff986a00640 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 0x563203e76db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff987400640 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e2b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 -- 172.21.15.122:0/4274581197 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 msgr2=0x563203e76db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 0x563203e76db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 -- 172.21.15.122:0/4274581197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 msgr2=0x563203e2b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e2b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 -- 172.21.15.122:0/4274581197 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563203c3f590 con 0x563203e772f0 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff97e000640 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203e2a8d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ff970000f90 tx=0x7ff9700a7030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a9f77aa8c8a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff977400640 1 -- 172.21.15.122:0/4274581197 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700b2040 con 0x563203e772f0 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff977400640 1 -- 172.21.15.122:0/4274581197 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff970004b40 con 0x563203e772f0 2024-09-17T05:37:48.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff977400640 1 -- 172.21.15.122:0/4274581197 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970004040 con 0x563203e772f0 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- 172.21.15.122:0/4274581197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 msgr2=0x563203e2a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203e2a8d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ff970000f90 tx=0x7ff9700a7030 comp rx=0 tx=0).stop 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- 172.21.15.122:0/4274581197 shutdown_connections 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e2b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203e2a8d0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 --2- 172.21.15.122:0/4274581197 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 0x563203e76db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- 172.21.15.122:0/4274581197 >> 172.21.15.122:0/4274581197 conn(0x563203d14130 msgr2=0x563203e59e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- 172.21.15.122:0/4274581197 shutdown_connections 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.649+0000 7ff990e76000 1 -- 172.21.15.122:0/4274581197 wait complete. 2024-09-17T05:37:48.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 -- start start 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 Processor -- start 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e2ae10 0x563203e576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e769d0 0x563203e57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 0x563203e583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563203c125d0 con 0x563203e2ae10 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563203c7bd60 con 0x563203e769d0 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563203c1e500 con 0x563203e772f0 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 10 client.0 _ll_register_callbacks cb 0x563203c22810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e2ae10 0x563203e576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e2ae10 0x563203e576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48030/0 (socket says 172.21.15.122:48030) 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 -- 172.21.15.122:0/1161255944 learned_addr learned my addr 172.21.15.122:0/1161255944 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 -- 172.21.15.122:0/1161255944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 msgr2=0x563203e583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 0x563203e583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 -- 172.21.15.122:0/1161255944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e769d0 msgr2=0x563203e57e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e769d0 0x563203e57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 -- 172.21.15.122:0/1161255944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563203c16170 con 0x563203e2ae10 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff986a00640 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e2ae10 0x563203e576a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c760 tx=0x7ff978069510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b83712c1720f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff96be00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff96be00640 1 -- 172.21.15.122:0/1161255944 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978079030 con 0x563203e2ae10 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff96be00640 1 -- 172.21.15.122:0/1161255944 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5851+0+0 (secure 0 0 0) 0x7ff978085150 con 0x563203e2ae10 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 -- 172.21.15.122:0/1161255944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563203c100f0 con 0x563203e2ae10 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff96be00640 1 -- 172.21.15.122:0/1161255944 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff97805f950 con 0x563203e2ae10 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 10 client.8502 fetch_fsmap learned FSMap version 76 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 -- 172.21.15.122:0/1161255944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x563203cfcef0 con 0x563203e2ae10 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff96be00640 1 -- 172.21.15.122:0/1161255944 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563203c100f0 con 0x563203e2ae10 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff96be00640 1 -- 172.21.15.122:0/1161255944 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7ff97807c020 con 0x563203e2ae10 2024-09-17T05:37:48.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 10 client.8502 fetch_fsmap finished waiting for FSMap version 76 2024-09-17T05:37:48.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 10 client.8502 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:37:48.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 20 client.8502 populate_metadata read hostname 'smithi122' 2024-09-17T05:37:48.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] conn(0x563203e590f0 0x563203e88c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 4 client.8502 mds_command: new command op to 8364 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:37:48.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff990e76000 1 -- 172.21.15.122:0/1161255944 --> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563203c8aa70 con 0x563203e590f0 2024-09-17T05:37:48.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff97e000640 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] conn(0x563203e590f0 0x563203e88c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff96be00640 10 client.8502 ms_handle_connect on v2:172.21.15.122:6834/2875934571 2024-09-17T05:37:48.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.653+0000 7ff97e000640 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] conn(0x563203e590f0 0x563203e88c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b970296fffac512 server_cookie=bb8bcc6b4006e724 in_seq=0 out_seq=0 2024-09-17T05:37:48.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff96be00640 1 -- 172.21.15.122:0/1161255944 <== mds.0 v2:172.21.15.122:6834/2875934571 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563203c8aa70 con 0x563203e590f0 2024-09-17T05:37:48.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff96be00640 10 client.8502 handle_command_reply: tid=0 2024-09-17T05:37:48.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 client.8502 shutdown 2024-09-17T05:37:48.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 -- 172.21.15.122:0/1161255944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e2ae10 msgr2=0x563203e576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e2ae10 0x563203e576a0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c760 tx=0x7ff978069510 comp rx=0 tx=0).stop 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 -- 172.21.15.122:0/1161255944 shutdown_connections 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 0x563203e583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] conn(0x563203e590f0 0x563203e88c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e769d0 0x563203e57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 --2- 172.21.15.122:0/1161255944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e2ae10 0x563203e576a0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 -- 172.21.15.122:0/1161255944 >> 172.21.15.122:0/1161255944 conn(0x563203d14130 msgr2=0x563203e29fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff96be00640 0 client.8502 ms_handle_reset on v2:172.21.15.122:6834/2875934571 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 -- 172.21.15.122:0/1161255944 shutdown_connections 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 1 -- 172.21.15.122:0/1161255944 wait complete. 2024-09-17T05:37:48.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.657+0000 7ff990e76000 20 client.8502 trim_cache size 0 max 16384 2024-09-17T05:37:48.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.673+0000 7ff990e76000 1 Processor -- start 2024-09-17T05:37:48.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.673+0000 7ff990e76000 1 -- start start 2024-09-17T05:37:48.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203ee8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 0x563203ee9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203ee9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff988204ac0 con 0x563203e772f0 2024-09-17T05:37:48.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff988203b30 con 0x563203e2ae10 2024-09-17T05:37:48.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff9881fbae0 con 0x563203e769d0 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203ee9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203ee9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48046/0 (socket says 172.21.15.122:48046) 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 -- 172.21.15.122:0/345668075 learned_addr learned my addr 172.21.15.122:0/345668075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff986a00640 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203ee8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff97e000640 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 0x563203ee9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 -- 172.21.15.122:0/345668075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 msgr2=0x563203ee9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 0x563203ee9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 -- 172.21.15.122:0/345668075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 msgr2=0x563203ee8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203ee8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 -- 172.21.15.122:0/345668075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563203cd3d90 con 0x563203e772f0 2024-09-17T05:37:48.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff987400640 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203ee9640 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff9800018e0 tx=0x7ff980001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79506e9f39dd5d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff976a00640 1 -- 172.21.15.122:0/345668075 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98001a020 con 0x563203e772f0 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff976a00640 1 -- 172.21.15.122:0/345668075 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff980012c40 con 0x563203e772f0 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff976a00640 1 -- 172.21.15.122:0/345668075 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980015440 con 0x563203e772f0 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- 172.21.15.122:0/345668075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 msgr2=0x563203ee9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203ee9640 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff9800018e0 tx=0x7ff980001da0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- 172.21.15.122:0/345668075 shutdown_connections 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e772f0 0x563203ee9640 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e769d0 0x563203ee9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- 172.21.15.122:0/345668075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203ee8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- 172.21.15.122:0/345668075 >> 172.21.15.122:0/345668075 conn(0x563203d14130 msgr2=0x563203e83fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- 172.21.15.122:0/345668075 shutdown_connections 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- 172.21.15.122:0/345668075 wait complete. 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- start start 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 Processor -- start 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e8a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e769d0 0x563203e8acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 0x563203e8b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563203c878f0 con 0x563203e769d0 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563203c7b070 con 0x563203e2ae10 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563203cfca60 con 0x563203e772f0 2024-09-17T05:37:48.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.677+0000 7ff990e76000 10 client.0 _ll_register_callbacks cb 0x7ff97805bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e769d0 0x563203e8acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e769d0 0x563203e8acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48058/0 (socket says 172.21.15.122:48058) 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 -- 172.21.15.122:0/3965468755 learned_addr learned my addr 172.21.15.122:0/3965468755 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff986a00640 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e8a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff987400640 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 0x563203e8b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 -- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 msgr2=0x563203e8b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 0x563203e8b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 -- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 msgr2=0x563203e8a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e8a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 -- 172.21.15.122:0/3965468755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563203c3ea90 con 0x563203e769d0 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff986a00640 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e8a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff97e000640 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e769d0 0x563203e8acd0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7ff970000f90 tx=0x7ff970004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95801581c247efc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:48.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:37:48.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 1 -- 172.21.15.122:0/3965468755 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700da060 con 0x563203e769d0 2024-09-17T05:37:48.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 1 -- 172.21.15.122:0/3965468755 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5851+0+0 (secure 0 0 0) 0x7ff970003f90 con 0x563203e769d0 2024-09-17T05:37:48.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 -- 172.21.15.122:0/3965468755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff980012c40 con 0x563203e769d0 2024-09-17T05:37:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 1 -- 172.21.15.122:0/3965468755 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff9700e0050 con 0x563203e769d0 2024-09-17T05:37:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 10 client.8508 fetch_fsmap learned FSMap version 76 2024-09-17T05:37:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 -- 172.21.15.122:0/3965468755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x563203c1d050 con 0x563203e769d0 2024-09-17T05:37:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 1 -- 172.21.15.122:0/3965468755 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff980012c40 con 0x563203e769d0 2024-09-17T05:37:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 1 -- 172.21.15.122:0/3965468755 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7ff9700e3020 con 0x563203e769d0 2024-09-17T05:37:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 10 client.8508 fetch_fsmap finished waiting for FSMap version 76 2024-09-17T05:37:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 10 client.8508 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:37:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 20 client.8508 populate_metadata read hostname 'smithi122' 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] conn(0x563203e8bbd0 0x563203ef4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 4 client.8508 mds_command: new command op to 8364 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 -- 172.21.15.122:0/3965468755 --> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] -- command(tid 0: {"prefix": "status"}) -- 0x563203c86520 con 0x563203e8bbd0 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff986a00640 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] conn(0x563203e8bbd0 0x563203ef4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 10 client.8508 ms_handle_connect on v2:172.21.15.122:6834/2875934571 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff986a00640 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] conn(0x563203e8bbd0 0x563203ef4c40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80e3b433c741a1f server_cookie=e845acf14ba7110d in_seq=0 out_seq=0 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 1 -- 172.21.15.122:0/3965468755 <== mds.0 v2:172.21.15.122:6834/2875934571 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563203c86520 con 0x563203e8bbd0 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 10 client.8508 handle_command_reply: tid=0 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 client.8508 shutdown 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 -- 172.21.15.122:0/3965468755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e769d0 msgr2=0x563203e8acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e769d0 0x563203e8acd0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7ff970000f90 tx=0x7ff970004a50 comp rx=0 tx=0).stop 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 -- 172.21.15.122:0/3965468755 shutdown_connections 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563203e772f0 0x563203e8b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563203e769d0 0x563203e8acd0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.122:6834/2875934571,v1:172.21.15.122:6835/2875934571] conn(0x563203e8bbd0 0x563203ef4c40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 --2- 172.21.15.122:0/3965468755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563203e2ae10 0x563203e8a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 -- 172.21.15.122:0/3965468755 >> 172.21.15.122:0/3965468755 conn(0x563203d14130 msgr2=0x563203e32a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff96be00640 0 client.8508 ms_handle_reset on v2:172.21.15.122:6834/2875934571 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 -- 172.21.15.122:0/3965468755 shutdown_connections 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 1 -- 172.21.15.122:0/3965468755 wait complete. 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff990e76000 20 client.8508 trim_cache size 0 max 16384 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:37:48.683 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 8364, 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 5.9432335209999998, 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 76, 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 118, 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 117, 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 10.433740189 2024-09-17T05:37:48.684 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:37:48.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 -- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff964074c70 msgr2=0x7ff964077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff964074c70 0x7ff964077130 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff9700af000 tx=0x7ff97001b350 comp rx=0 tx=0).stop 2024-09-17T05:37:48.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 -- 172.21.15.122:0/2772071000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9880b2d00 msgr2=0x7ff98815b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9880b2d00 0x7ff98815b2f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ff97800ea40 tx=0x7ff97800ef00 comp rx=0 tx=0).stop 2024-09-17T05:37:48.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 -- 172.21.15.122:0/2772071000 shutdown_connections 2024-09-17T05:37:48.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff964074c70 0x7ff964077130 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff988160400 0x7ff9880c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9880cfe80 0x7ff9881f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 --2- 172.21.15.122:0/2772071000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9880b2d00 0x7ff98815b2f0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 -- 172.21.15.122:0/2772071000 >> 172.21.15.122:0/2772071000 conn(0x7ff9880d1e40 msgr2=0x7ff98815f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:48.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 -- 172.21.15.122:0/2772071000 shutdown_connections 2024-09-17T05:37:48.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.681+0000 7ff98da00640 1 -- 172.21.15.122:0/2772071000 wait complete. 2024-09-17T05:37:48.697 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:37:48.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.845+0000 7fd2fb400640 1 -- 172.21.15.122:0/2819278979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 msgr2=0x7fd2f40cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.845+0000 7fd2fb400640 1 --2- 172.21.15.122:0/2819278979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 0x7fd2f40cd150 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4073a20 tx=0x7fd2ec01cb70 comp rx=0 tx=0).stop 2024-09-17T05:37:48.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 -- 172.21.15.122:0/2819278979 shutdown_connections 2024-09-17T05:37:48.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 --2- 172.21.15.122:0/2819278979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 0x7fd2f40cd150 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 --2- 172.21.15.122:0/2819278979 >> v2:172.21.15.173:3300/0 conn(0x7fd2f415e130 0x7fd2f406e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 -- 172.21.15.122:0/2819278979 >> 172.21.15.122:0/2819278979 conn(0x7fd2f40d1ea0 msgr2=0x7fd2f40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:48.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 -- 172.21.15.122:0/2819278979 shutdown_connections 2024-09-17T05:37:48.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 -- 172.21.15.122:0/2819278979 wait complete. 2024-09-17T05:37:48.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 Processor -- start 2024-09-17T05:37:48.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 -- start start 2024-09-17T05:37:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2f40ce720 0x7fd2f41f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 0x7fd2f40c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2f415e130 0x7fd2f40ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f416ae90 con 0x7fd2f40cf250 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f40ce2b0 con 0x7fd2f40ce720 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2fb400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f416b010 con 0x7fd2f415e130 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 0x7fd2f40c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 0x7fd2f40c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48066/0 (socket says 172.21.15.122:48066) 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 -- 172.21.15.122:0/419391029 learned_addr learned my addr 172.21.15.122:0/419391029 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2f8a00640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2f40ce720 0x7fd2f41f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2f9400640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2f415e130 0x7fd2f40ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 -- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2f415e130 msgr2=0x7fd2f40ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2f415e130 0x7fd2f40ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 -- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2f40ce720 msgr2=0x7fd2f41f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:48.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2f40ce720 0x7fd2f41f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 -- 172.21.15.122:0/419391029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f40c6050 con 0x7fd2f40cf250 2024-09-17T05:37:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2f9400640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2f415e130 0x7fd2f40ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2ebe00640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 0x7fd2f40c5900 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00c450 tx=0x7fd2dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.849+0000 7fd2f1600640 1 -- 172.21.15.122:0/419391029 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc012830 con 0x7fd2f40cf250 2024-09-17T05:37:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2f1600640 1 -- 172.21.15.122:0/419391029 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2dc017070 con 0x7fd2f40cf250 2024-09-17T05:37:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f40c6330 con 0x7fd2f40cf250 2024-09-17T05:37:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2f1600640 1 -- 172.21.15.122:0/419391029 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc012d80 con 0x7fd2f40cf250 2024-09-17T05:37:48.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f40c68f0 con 0x7fd2f40cf250 2024-09-17T05:37:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f40cd150 con 0x7fd2f40cf250 2024-09-17T05:37:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2f1600640 1 -- 172.21.15.122:0/419391029 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fd2dc01c460 con 0x7fd2f40cf250 2024-09-17T05:37:48.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2f1600640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd2d0074c70 0x7fd2d0077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2f1600640 1 -- 172.21.15.122:0/419391029 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fd2dc092d90 con 0x7fd2f40cf250 2024-09-17T05:37:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.853+0000 7fd2f8a00640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd2d0074c70 0x7fd2d0077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:48.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.857+0000 7fd2f8a00640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd2d0074c70 0x7fd2d0077130 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40058d0 tx=0x7fd2e4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:48.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:48.857+0000 7fd2f1600640 1 -- 172.21.15.122:0/419391029 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd2dc05f940 con 0x7fd2f40cf250 2024-09-17T05:37:49.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd2f40d2630 con 0x7fd2f40cf250 2024-09-17T05:37:49.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2f1600640 1 -- 172.21.15.122:0/419391029 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7fd2dc05f240 con 0x7fd2f40cf250 2024-09-17T05:37:49.060 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd2d0074c70 msgr2=0x7fd2d0077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd2d0074c70 0x7fd2d0077130 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40058d0 tx=0x7fd2e4005f90 comp rx=0 tx=0).stop 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 msgr2=0x7fd2f40c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 0x7fd2f40c5900 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00c450 tx=0x7fd2dc00c910 comp rx=0 tx=0).stop 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 shutdown_connections 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd2d0074c70 0x7fd2d0077130 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2f415e130 0x7fd2f40ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2f40cf250 0x7fd2f40c5900 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 --2- 172.21.15.122:0/419391029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2f40ce720 0x7fd2f41f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:49.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 >> 172.21.15.122:0/419391029 conn(0x7fd2f40d1ea0 msgr2=0x7fd2f40b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:49.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 shutdown_connections 2024-09-17T05:37:49.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49.065+0000 7fd2fb400640 1 -- 172.21.15.122:0/419391029 wait complete. 2024-09-17T05:37:49.082 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:37:49.082 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:37:49.082 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:37:49.082 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:37:49.082 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:37:49.082 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:37:49.083 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:37:49.083 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:37:49.083 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:37:49.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:49.083 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:37:49.092 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 41698sec preferred_lft 41698sec 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:37:49.093 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:37:49.094 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:37:49.095 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:37:49.095 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:37:49.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:49.096 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:37:49.156 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:37:49.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:49.159 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:37:49.215 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:37:49.215 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:37:49.215 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:37:49.215 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:37:49.215 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:37:49.215 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:37:49.215 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:37:49.216 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:37:49.216 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:37:49.216 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:37:49.216 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:37:49.216 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:37:49.216 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:37:49.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:37:49.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:37:49.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:37:49.217 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:37:49.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:37:49.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:37:49.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:49.353 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:37:49.403 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:37:49.403 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:37:49.404 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:37:49.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:49.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:37:49.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:37:49.467 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:37:49.519 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:37:49.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:37:49.519 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:37:49.574 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:37:49.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:49.575 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:37:49.666 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:37:49.666 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:37:49.667 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:37:49.682 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:37:49.682 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:37:49.682 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:37:49.682 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:37:49.682 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:37:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:37:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:37:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:37:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:37:49.691 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:37:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:37:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:37:49.692 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:37:49.696 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:37:49.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:49.696 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:37:49.716 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:37:49.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:37:49.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:37:49.829 INFO:teuthology.orchestra.run.smithi122.stdout:client8517 2024-09-17T05:37:49.829 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:37:49.829 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:49.838 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:37:50.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- 172.21.15.122:0/4014975101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415c1f0 msgr2=0x7f32240cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:50.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 --2- 172.21.15.122:0/4014975101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415c1f0 0x7f32240cec40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f320c009900 tx=0x7f320c01f0d0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- 172.21.15.122:0/4014975101 shutdown_connections 2024-09-17T05:37:50.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 --2- 172.21.15.122:0/4014975101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415c1f0 0x7f32240cec40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 --2- 172.21.15.122:0/4014975101 >> v2:172.21.15.173:3300/0 conn(0x7f322415be10 0x7f32240ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- 172.21.15.122:0/4014975101 >> 172.21.15.122:0/4014975101 conn(0x7f32240d1da0 msgr2=0x7f32240d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:50.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- 172.21.15.122:0/4014975101 shutdown_connections 2024-09-17T05:37:50.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- 172.21.15.122:0/4014975101 wait complete. 2024-09-17T05:37:50.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 Processor -- start 2024-09-17T05:37:50.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- start start 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32240b0b60 0x7f32240ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415be10 0x7f322414f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f322415c1f0 0x7f32240c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32240cf510 con 0x7f322415c1f0 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f32240cf390 con 0x7f32240b0b60 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3229000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3224160290 con 0x7f322415be10 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3222800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f322415c1f0 0x7f32240c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3221e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32240b0b60 0x7f32240ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:50.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3219400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415be10 0x7f322414f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:50.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3222800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f322415c1f0 0x7f32240c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48068/0 (socket says 172.21.15.122:48068) 2024-09-17T05:37:50.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3221e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32240b0b60 0x7f32240ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:45622/0 (socket says 172.21.15.122:45622) 2024-09-17T05:37:50.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.033+0000 7f3222800640 1 -- 172.21.15.122:0/1517102545 learned_addr learned my addr 172.21.15.122:0/1517102545 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:50.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3219400640 1 -- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32240b0b60 msgr2=0x7f32240ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:50.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3219400640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32240b0b60 0x7f32240ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3219400640 1 -- 172.21.15.122:0/1517102545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f322415c1f0 msgr2=0x7f32240c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:50.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3219400640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f322415c1f0 0x7f32240c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3219400640 1 -- 172.21.15.122:0/1517102545 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322414f960 con 0x7f322415be10 2024-09-17T05:37:50.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3219400640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415be10 0x7f322414f420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f320c0af000 tx=0x7f320c005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:50.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f321aa00640 1 -- 172.21.15.122:0/1517102545 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0be070 con 0x7f322415be10 2024-09-17T05:37:50.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f321aa00640 1 -- 172.21.15.122:0/1517102545 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f320c003930 con 0x7f322415be10 2024-09-17T05:37:50.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f321aa00640 1 -- 172.21.15.122:0/1517102545 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0b5040 con 0x7f322415be10 2024-09-17T05:37:50.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32240c8690 con 0x7f322415be10 2024-09-17T05:37:50.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f32240cb1a0 con 0x7f322415be10 2024-09-17T05:37:50.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3224160780 con 0x7f322415be10 2024-09-17T05:37:50.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.037+0000 7f321aa00640 1 -- 172.21.15.122:0/1517102545 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f320c0c1070 con 0x7f322415be10 2024-09-17T05:37:50.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.041+0000 7f321aa00640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3200074c70 0x7f3200077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:50.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.041+0000 7f321aa00640 1 -- 172.21.15.122:0/1517102545 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f320c1306c0 con 0x7f322415be10 2024-09-17T05:37:50.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.041+0000 7f3221e00640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3200074c70 0x7f3200077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:50.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.041+0000 7f3221e00640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3200074c70 0x7f3200077130 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3214009be0 tx=0x7f32140091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:50.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.041+0000 7f321aa00640 1 -- 172.21.15.122:0/1517102545 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f320c0c8070 con 0x7f322415be10 2024-09-17T05:37:50.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.249+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f3224160a20 con 0x7f322415be10 2024-09-17T05:37:50.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.797+0000 7f321aa00640 1 -- 172.21.15.122:0/1517102545 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v77) ==== 136+0+0 (secure 0 0 0) 0x7f320c018c30 con 0x7f322415be10 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3200074c70 msgr2=0x7f3200077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3200074c70 0x7f3200077130 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3214009be0 tx=0x7f32140091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415be10 msgr2=0x7f322414f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415be10 0x7f322414f420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f320c0af000 tx=0x7f320c005d60 comp rx=0 tx=0).stop 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 shutdown_connections 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3200074c70 0x7f3200077130 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f322415c1f0 0x7f32240c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f322415be10 0x7f322414f420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 --2- 172.21.15.122:0/1517102545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32240b0b60 0x7f32240ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 >> 172.21.15.122:0/1517102545 conn(0x7f32240d1da0 msgr2=0x7f3224070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:50.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 shutdown_connections 2024-09-17T05:37:50.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.801+0000 7f3229000640 1 -- 172.21.15.122:0/1517102545 wait complete. 2024-09-17T05:37:50.815 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:37:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.961+0000 7f92bac00640 1 -- 172.21.15.122:0/2532959274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 msgr2=0x7f92b40b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.961+0000 7f92bac00640 1 --2- 172.21.15.122:0/2532959274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 0x7f92b40b29e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009c80 tx=0x7f92ac01cd10 comp rx=0 tx=0).stop 2024-09-17T05:37:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.961+0000 7f92bac00640 1 -- 172.21.15.122:0/2532959274 shutdown_connections 2024-09-17T05:37:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.961+0000 7f92bac00640 1 --2- 172.21.15.122:0/2532959274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 0x7f92b40b29e0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.961+0000 7f92bac00640 1 --2- 172.21.15.122:0/2532959274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b415f000 0x7f92b40d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.961+0000 7f92bac00640 1 -- 172.21.15.122:0/2532959274 >> 172.21.15.122:0/2532959274 conn(0x7f92b409aa10 msgr2=0x7f92b409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.961+0000 7f92bac00640 1 -- 172.21.15.122:0/2532959274 shutdown_connections 2024-09-17T05:37:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.961+0000 7f92bac00640 1 -- 172.21.15.122:0/2532959274 wait complete. 2024-09-17T05:37:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 Processor -- start 2024-09-17T05:37:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 -- start start 2024-09-17T05:37:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 0x7f92b406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40cefb0 0x7f92b406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b415f000 0x7f92b4071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4168f00 con 0x7f92b40b2590 2024-09-17T05:37:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4168d80 con 0x7f92b415f000 2024-09-17T05:37:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f92b4169080 con 0x7f92b40cefb0 2024-09-17T05:37:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 0x7f92b406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:50.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 0x7f92b406ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48090/0 (socket says 172.21.15.122:48090) 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 -- 172.21.15.122:0/3769789099 learned_addr learned my addr 172.21.15.122:0/3769789099 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 -- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40cefb0 msgr2=0x7f92b406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40cefb0 0x7f92b406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 -- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b415f000 msgr2=0x7f92b4071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b415f000 0x7f92b4071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 -- 172.21.15.122:0/3769789099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b406eb10 con 0x7f92b40b2590 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b9800640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 0x7f92b406ed80 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f92a80058d0 tx=0x7f92a800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b1600640 1 -- 172.21.15.122:0/3769789099 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8017070 con 0x7f92b40b2590 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b1600640 1 -- 172.21.15.122:0/3769789099 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92a8009d20 con 0x7f92b40b2590 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b406f410 con 0x7f92b40b2590 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92b1600640 1 -- 172.21.15.122:0/3769789099 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a80106c0 con 0x7f92b40b2590 2024-09-17T05:37:50.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b406f9d0 con 0x7f92b40b2590 2024-09-17T05:37:50.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.965+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b406e200 con 0x7f92b40b2590 2024-09-17T05:37:50.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.969+0000 7f92b1600640 1 -- 172.21.15.122:0/3769789099 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f92a801a070 con 0x7f92b40b2590 2024-09-17T05:37:50.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.969+0000 7f92b1600640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9294074c70 0x7f9294077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:50.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.969+0000 7f92b1600640 1 -- 172.21.15.122:0/3769789099 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f92a8093250 con 0x7f92b40b2590 2024-09-17T05:37:50.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.969+0000 7f92b8e00640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9294074c70 0x7f9294077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:50.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.969+0000 7f92b8e00640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9294074c70 0x7f9294077130 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f92a0000f70 tx=0x7f92a00079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:50.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:50.973+0000 7f92b1600640 1 -- 172.21.15.122:0/3769789099 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f92a805fd60 con 0x7f92b40b2590 2024-09-17T05:37:51.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.165+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f92b40b29e0 con 0x7f9294074c70 2024-09-17T05:37:51.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.165+0000 7f92b1600640 1 -- 172.21.15.122:0/3769789099 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f92b40b29e0 con 0x7f9294074c70 2024-09-17T05:37:51.162 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:37:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:37:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:37:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:37:51.162 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:37:51.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9294074c70 msgr2=0x7f9294077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9294074c70 0x7f9294077130 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f92a0000f70 tx=0x7f92a00079c0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 msgr2=0x7f92b406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 0x7f92b406ed80 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f92a80058d0 tx=0x7f92a800dde0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 shutdown_connections 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9294074c70 0x7f9294077130 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b415f000 0x7f92b4071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40cefb0 0x7f92b406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 --2- 172.21.15.122:0/3769789099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92b40b2590 0x7f92b406ed80 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 >> 172.21.15.122:0/3769789099 conn(0x7f92b409aa10 msgr2=0x7f92b40cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 shutdown_connections 2024-09-17T05:37:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.169+0000 7f92bac00640 1 -- 172.21.15.122:0/3769789099 wait complete. 2024-09-17T05:37:51.183 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:37:51.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- 172.21.15.122:0/1799734878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015bb90 msgr2=0x7fbde0160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 --2- 172.21.15.122:0/1799734878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015bb90 0x7fbde0160060 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009900 tx=0x7fbdd001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- 172.21.15.122:0/1799734878 shutdown_connections 2024-09-17T05:37:51.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 --2- 172.21.15.122:0/1799734878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde015cf80 0x7fbde006e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 --2- 172.21.15.122:0/1799734878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015bb90 0x7fbde0160060 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- 172.21.15.122:0/1799734878 >> 172.21.15.122:0/1799734878 conn(0x7fbde00cfa00 msgr2=0x7fbde00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:51.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- 172.21.15.122:0/1799734878 shutdown_connections 2024-09-17T05:37:51.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- 172.21.15.122:0/1799734878 wait complete. 2024-09-17T05:37:51.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 Processor -- start 2024-09-17T05:37:51.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- start start 2024-09-17T05:37:51.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde015bb90 0x7fbde0156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:51.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015cf80 0x7fbde00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:51.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde01575f0 0x7fbde00b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:51.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbde00729a0 con 0x7fbde015cf80 2024-09-17T05:37:51.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0072820 con 0x7fbde01575f0 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddfe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbde00aece0 con 0x7fbde015bb90 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015cf80 0x7fbde00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015cf80 0x7fbde00b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50174/0 (socket says 172.21.15.122:50174) 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 -- 172.21.15.122:0/3867136155 learned_addr learned my addr 172.21.15.122:0/3867136155 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddea00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde015bb90 0x7fbde0156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddf400640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde01575f0 0x7fbde00b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 -- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde015bb90 msgr2=0x7fbde0156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde015bb90 0x7fbde0156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 -- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde01575f0 msgr2=0x7fbde00b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde01575f0 0x7fbde00b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 -- 172.21.15.122:0/3867136155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde015a5c0 con 0x7fbde015cf80 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbddf400640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde01575f0 0x7fbde00b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:37:51.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.333+0000 7fbdde000640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015cf80 0x7fbde00b4760 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc001ec0 tx=0x7fbdcc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:51.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbdd6a00640 1 -- 172.21.15.122:0/3867136155 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc017070 con 0x7fbde015cf80 2024-09-17T05:37:51.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbdd6a00640 1 -- 172.21.15.122:0/3867136155 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdcc003600 con 0x7fbde015cf80 2024-09-17T05:37:51.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde015a860 con 0x7fbde015cf80 2024-09-17T05:37:51.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbdd6a00640 1 -- 172.21.15.122:0/3867136155 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc003a50 con 0x7fbde015cf80 2024-09-17T05:37:51.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde0157ab0 con 0x7fbde015cf80 2024-09-17T05:37:51.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde0160040 con 0x7fbde015cf80 2024-09-17T05:37:51.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbdd6a00640 1 -- 172.21.15.122:0/3867136155 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fbdcc01a070 con 0x7fbde015cf80 2024-09-17T05:37:51.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbdd6a00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0074c70 0x7fbdc0077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:51.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbdd6a00640 1 -- 172.21.15.122:0/3867136155 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fbdcc092320 con 0x7fbde015cf80 2024-09-17T05:37:51.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.337+0000 7fbddea00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0074c70 0x7fbdc0077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:51.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.341+0000 7fbddea00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0074c70 0x7fbdc0077130 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00af000 tx=0x7fbdd001b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:51.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.341+0000 7fbdd6a00640 1 -- 172.21.15.122:0/3867136155 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbdcc05e7e0 con 0x7fbde015cf80 2024-09-17T05:37:51.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.509+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbde00c9e30 con 0x7fbde015cf80 2024-09-17T05:37:51.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.509+0000 7fbdd6a00640 1 -- 172.21.15.122:0/3867136155 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7fbdcc063210 con 0x7fbde015cf80 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0074c70 msgr2=0x7fbdc0077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0074c70 0x7fbdc0077130 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00af000 tx=0x7fbdd001b120 comp rx=0 tx=0).stop 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015cf80 msgr2=0x7fbde00b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015cf80 0x7fbde00b4760 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc001ec0 tx=0x7fbdcc00c890 comp rx=0 tx=0).stop 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 shutdown_connections 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbdc0074c70 0x7fbdc0077130 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbde01575f0 0x7fbde00b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbde015cf80 0x7fbde00b4760 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 --2- 172.21.15.122:0/3867136155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbde015bb90 0x7fbde0156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 >> 172.21.15.122:0/3867136155 conn(0x7fbde00cfa00 msgr2=0x7fbde006f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:51.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 shutdown_connections 2024-09-17T05:37:51.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.513+0000 7fbddfe00640 1 -- 172.21.15.122:0/3867136155 wait complete. 2024-09-17T05:37:51.526 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:37:51.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.673+0000 7f4582600640 1 -- 172.21.15.122:0/894848127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4560003c20 msgr2=0x7f4560004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.673+0000 7f4582600640 1 --2- 172.21.15.122:0/894848127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4560003c20 0x7f4560004020 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f456c001f50 tx=0x7f456c01cc10 comp rx=0 tx=0).stop 2024-09-17T05:37:51.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.673+0000 7f457be00640 1 -- 172.21.15.122:0/894848127 reap_dead start 2024-09-17T05:37:51.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.673+0000 7f4582600640 1 -- 172.21.15.122:0/894848127 shutdown_connections 2024-09-17T05:37:51.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.673+0000 7f4582600640 1 -- 172.21.15.122:0/894848127 >> 172.21.15.122:0/894848127 conn(0x7f457c0d1df0 msgr2=0x7f457c0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:51.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.673+0000 7f4582600640 1 -- 172.21.15.122:0/894848127 shutdown_connections 2024-09-17T05:37:51.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.673+0000 7f4582600640 1 -- 172.21.15.122:0/894848127 wait complete. 2024-09-17T05:37:51.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 Processor -- start 2024-09-17T05:37:51.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 -- start start 2024-09-17T05:37:51.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4560003c20 0x7f457c0b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457c15d3a0 0x7f457c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c166f00 0x7f457c167380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0b3410 con 0x7f457c166f00 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0b3290 con 0x7f4560003c20 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f457c1684f0 con 0x7f457c15d3a0 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c166f00 0x7f457c167380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c166f00 0x7f457c167380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50190/0 (socket says 172.21.15.122:50190) 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 -- 172.21.15.122:0/108750818 learned_addr learned my addr 172.21.15.122:0/108750818 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457b400640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4560003c20 0x7f457c0b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457aa00640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457c15d3a0 0x7f457c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 -- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457c15d3a0 msgr2=0x7f457c0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457c15d3a0 0x7f457c0b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 -- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4560003c20 msgr2=0x7f457c0b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4560003c20 0x7f457c0b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 -- 172.21.15.122:0/108750818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c15dae0 con 0x7f457c166f00 2024-09-17T05:37:51.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f457be00640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c166f00 0x7f457c167380 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f457c0d0fe0 tx=0x7f457400e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:51.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4573400640 1 -- 172.21.15.122:0/108750818 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574011870 con 0x7f457c166f00 2024-09-17T05:37:51.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4573400640 1 -- 172.21.15.122:0/108750818 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4574017070 con 0x7f457c166f00 2024-09-17T05:37:51.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c15eb10 con 0x7f457c166f00 2024-09-17T05:37:51.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.677+0000 7f4573400640 1 -- 172.21.15.122:0/108750818 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574011da0 con 0x7f457c166f00 2024-09-17T05:37:51.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.681+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c15dd50 con 0x7f457c166f00 2024-09-17T05:37:51.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.681+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457c0cf9c0 con 0x7f457c166f00 2024-09-17T05:37:51.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.681+0000 7f4573400640 1 -- 172.21.15.122:0/108750818 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f457401c460 con 0x7f457c166f00 2024-09-17T05:37:51.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.681+0000 7f4573400640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f455c074ca0 0x7f455c077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:51.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.681+0000 7f4573400640 1 -- 172.21.15.122:0/108750818 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f45740932a0 con 0x7f457c166f00 2024-09-17T05:37:51.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.681+0000 7f457b400640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f455c074ca0 0x7f455c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:51.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.685+0000 7f457b400640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f455c074ca0 0x7f455c077160 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f456c0a9000 tx=0x7f456c019a50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:51.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.685+0000 7f4573400640 1 -- 172.21.15.122:0/108750818 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f457405f760 con 0x7f457c166f00 2024-09-17T05:37:51.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.853+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f457c1625c0 con 0x7f457c166f00 2024-09-17T05:37:51.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.853+0000 7f4573400640 1 -- 172.21.15.122:0/108750818 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v48) ==== 116+0+2 (secure 0 0 0) 0x7f4574064190 con 0x7f457c166f00 2024-09-17T05:37:51.850 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f455c074ca0 msgr2=0x7f455c077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f455c074ca0 0x7f455c077160 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f456c0a9000 tx=0x7f456c019a50 comp rx=0 tx=0).stop 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c166f00 msgr2=0x7f457c167380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c166f00 0x7f457c167380 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f457c0d0fe0 tx=0x7f457400e770 comp rx=0 tx=0).stop 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 shutdown_connections 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f455c074ca0 0x7f455c077160 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f457c166f00 0x7f457c167380 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457c15d3a0 0x7f457c0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 --2- 172.21.15.122:0/108750818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4560003c20 0x7f457c0b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:51.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 >> 172.21.15.122:0/108750818 conn(0x7f457c0d1df0 msgr2=0x7f457c15f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:51.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 shutdown_connections 2024-09-17T05:37:51.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:51.857+0000 7f4582600640 1 -- 172.21.15.122:0/108750818 wait complete. 2024-09-17T05:37:51.872 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:37:52.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 -- 172.21.15.122:0/3665714132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9800052d0 msgr2=0x7fe98000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 --2- 172.21.15.122:0/3665714132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9800052d0 0x7fe98000b9d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe99c15c540 tx=0x7fe99401d250 comp rx=0 tx=0).stop 2024-09-17T05:37:52.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 -- 172.21.15.122:0/3665714132 shutdown_connections 2024-09-17T05:37:52.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 --2- 172.21.15.122:0/3665714132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9800052d0 0x7fe98000b9d0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 -- 172.21.15.122:0/3665714132 >> 172.21.15.122:0/3665714132 conn(0x7fe99c0d1ea0 msgr2=0x7fe99c0d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:52.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 -- 172.21.15.122:0/3665714132 shutdown_connections 2024-09-17T05:37:52.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 -- 172.21.15.122:0/3665714132 wait complete. 2024-09-17T05:37:52.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 Processor -- start 2024-09-17T05:37:52.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 -- start start 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.021+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9800052d0 0x7fe99c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c0b4cd0 0x7fe99c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe9a1400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe99c0c4910 0x7fe99c0c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe9a1400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c0ce970 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe9a1400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c0ce7f0 con 0x7fe9800052d0 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe9a1400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe99c0ceaf0 con 0x7fe99c0c4910 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c0b4cd0 0x7fe99c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c0b4cd0 0x7fe99c0b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50202/0 (socket says 172.21.15.122:50202) 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 -- 172.21.15.122:0/442965856 learned_addr learned my addr 172.21.15.122:0/442965856 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:52.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe99a800640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9800052d0 0x7fe99c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe99b200640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe99c0c4910 0x7fe99c0c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 -- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe99c0c4910 msgr2=0x7fe99c0c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe99c0c4910 0x7fe99c0c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 -- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9800052d0 msgr2=0x7fe99c06eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9800052d0 0x7fe99c06eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 -- 172.21.15.122:0/442965856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c0c5290 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe991e00640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c0b4cd0 0x7fe99c0b6390 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fe984010fc0 tx=0x7fe98400e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe993400640 1 -- 172.21.15.122:0/442965856 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984012850 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe993400640 1 -- 172.21.15.122:0/442965856 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe984012ae0 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe993400640 1 -- 172.21.15.122:0/442965856 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98401a400 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c06f0c0 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c06f640 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99c0c5fe0 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.025+0000 7fe993400640 1 -- 172.21.15.122:0/442965856 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fe98400d030 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.029+0000 7fe993400640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe978074c70 0x7fe978077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.029+0000 7fe993400640 1 -- 172.21.15.122:0/442965856 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fe9840961f0 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.029+0000 7fe99a800640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe978074c70 0x7fe978077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.029+0000 7fe99a800640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe978074c70 0x7fe978077130 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe98c004fd0 tx=0x7fe98c004990 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:52.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.033+0000 7fe993400640 1 -- 172.21.15.122:0/442965856 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe99c0c5290 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.197+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fe99c0cdbb0 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.201+0000 7fe993400640 1 -- 172.21.15.122:0/442965856 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7fe984016030 con 0x7fe99c0b4cd0 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe978074c70 msgr2=0x7fe978077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe978074c70 0x7fe978077130 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe98c004fd0 tx=0x7fe98c004990 comp rx=0 tx=0).stop 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c0b4cd0 msgr2=0x7fe99c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c0b4cd0 0x7fe99c0b6390 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fe984010fc0 tx=0x7fe98400e770 comp rx=0 tx=0).stop 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 shutdown_connections 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe978074c70 0x7fe978077130 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe99c0c4910 0x7fe99c0c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe99c0b4cd0 0x7fe99c0b6390 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 --2- 172.21.15.122:0/442965856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9800052d0 0x7fe99c06eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 >> 172.21.15.122:0/442965856 conn(0x7fe99c0d1ea0 msgr2=0x7fe99c0c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:52.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 shutdown_connections 2024-09-17T05:37:52.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.205+0000 7fe9a1400640 1 -- 172.21.15.122:0/442965856 wait complete. 2024-09-17T05:37:52.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T05:37:52.223 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.227+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.223 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.227+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.223 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.227+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.223 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.227+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.223 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.227+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- 172.21.15.122:0/3965244290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580af330 msgr2=0x7f48580cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 --2- 172.21.15.122:0/3965244290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580af330 0x7f48580cdac0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f485806e2f0 tx=0x7f485001cd80 comp rx=0 tx=0).stop 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- 172.21.15.122:0/3965244290 shutdown_connections 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 --2- 172.21.15.122:0/3965244290 >> v2:172.21.15.173:3300/0 conn(0x7f48580b5170 0x7f48580aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 --2- 172.21.15.122:0/3965244290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580af330 0x7f48580cdac0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 --2- 172.21.15.122:0/3965244290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f485815bf60 0x7f48580733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- 172.21.15.122:0/3965244290 >> 172.21.15.122:0/3965244290 conn(0x7f48580cfbc0 msgr2=0x7f48580cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- 172.21.15.122:0/3965244290 shutdown_connections 2024-09-17T05:37:52.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- 172.21.15.122:0/3965244290 wait complete. 2024-09-17T05:37:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 Processor -- start 2024-09-17T05:37:52.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- start start 2024-09-17T05:37:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48580af330 0x7f4858070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580b5170 0x7f485815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485815bf60 0x7f48581474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4858160100 con 0x7f48580b5170 2024-09-17T05:37:52.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f485815ff80 con 0x7f485815bf60 2024-09-17T05:37:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485fc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4858160280 con 0x7f48580af330 2024-09-17T05:37:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f484fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580b5170 0x7f485815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f484fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580b5170 0x7f485815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50220/0 (socket says 172.21.15.122:50220) 2024-09-17T05:37:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f484fe00640 1 -- 172.21.15.122:0/2229578761 learned_addr learned my addr 172.21.15.122:0/2229578761 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485dc00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485815bf60 0x7f48581474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.373+0000 7f485d200640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48580af330 0x7f4858070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f484fe00640 1 -- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48580af330 msgr2=0x7f4858070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f484fe00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48580af330 0x7f4858070280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f484fe00640 1 -- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485815bf60 msgr2=0x7f48581474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f484fe00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485815bf60 0x7f48581474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f484fe00640 1 -- 172.21.15.122:0/2229578761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485815ee70 con 0x7f48580b5170 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f484fe00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580b5170 0x7f485815e290 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f484000e970 tx=0x7f484000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f4855600640 1 -- 172.21.15.122:0/2229578761 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484000cd80 con 0x7f48580b5170 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f4855600640 1 -- 172.21.15.122:0/2229578761 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4840010430 con 0x7f48580b5170 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f4855600640 1 -- 172.21.15.122:0/2229578761 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4840010880 con 0x7f48580b5170 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485806f7a0 con 0x7f48580b5170 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485806fd60 con 0x7f48580b5170 2024-09-17T05:37:52.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4858069410 con 0x7f48580b5170 2024-09-17T05:37:52.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.377+0000 7f4855600640 1 -- 172.21.15.122:0/2229578761 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7f4840010a20 con 0x7f48580b5170 2024-09-17T05:37:52.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.381+0000 7f4855600640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4834074c70 0x7f4834077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.381+0000 7f4855600640 1 -- 172.21.15.122:0/2229578761 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f4840093ac0 con 0x7f48580b5170 2024-09-17T05:37:52.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.381+0000 7f485d200640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4834074c70 0x7f4834077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.381+0000 7f485d200640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4834074c70 0x7f4834077130 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4848009b40 tx=0x7f48480091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:52.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.385+0000 7f4855600640 1 -- 172.21.15.122:0/2229578761 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f484005ff80 con 0x7f48580b5170 2024-09-17T05:37:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.577+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f48580aedf0 con 0x7f4834074c70 2024-09-17T05:37:52.582 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.587+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.582 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.587+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.582 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.587+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.582 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.587+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.582 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:37:52.587+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:37:52.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.621+0000 7f4855600640 1 -- 172.21.15.122:0/2229578761 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f48580aedf0 con 0x7f4834074c70 2024-09-17T05:37:52.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4834074c70 msgr2=0x7f4834077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4834074c70 0x7f4834077130 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4848009b40 tx=0x7f48480091e0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580b5170 msgr2=0x7f485815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580b5170 0x7f485815e290 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f484000e970 tx=0x7f484000ee30 comp rx=0 tx=0).stop 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 shutdown_connections 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4834074c70 0x7f4834077130 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485815bf60 0x7f48581474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48580b5170 0x7f485815e290 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 --2- 172.21.15.122:0/2229578761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48580af330 0x7f4858070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 >> 172.21.15.122:0/2229578761 conn(0x7f48580cfbc0 msgr2=0x7f48581573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 shutdown_connections 2024-09-17T05:37:52.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.625+0000 7f485fc00640 1 -- 172.21.15.122:0/2229578761 wait complete. 2024-09-17T05:37:52.639 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:37:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.785+0000 7fa4c9800640 1 -- 172.21.15.122:0/1252329096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40afb10 msgr2=0x7fa4c40affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.785+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1252329096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40afb10 0x7fa4c40affc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009c80 tx=0x7fa4bc01cda0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 -- 172.21.15.122:0/1252329096 shutdown_connections 2024-09-17T05:37:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1252329096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40afb10 0x7fa4c40affc0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1252329096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4c41605d0 0x7fa4c40d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 -- 172.21.15.122:0/1252329096 >> 172.21.15.122:0/1252329096 conn(0x7fa4c40d1dd0 msgr2=0x7fa4c40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 -- 172.21.15.122:0/1252329096 shutdown_connections 2024-09-17T05:37:52.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 -- 172.21.15.122:0/1252329096 wait complete. 2024-09-17T05:37:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 Processor -- start 2024-09-17T05:37:52.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 -- start start 2024-09-17T05:37:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4c40afb10 0x7fa4c41fb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40b1650 0x7fa4c4156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4c41605d0 0x7fa4c415d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4169540 con 0x7fa4c40b1650 2024-09-17T05:37:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c40b1430 con 0x7fa4c40afb10 2024-09-17T05:37:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c9800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c41696c0 con 0x7fa4c41605d0 2024-09-17T05:37:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40b1650 0x7fa4c4156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40b1650 0x7fa4c4156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50236/0 (socket says 172.21.15.122:50236) 2024-09-17T05:37:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 -- 172.21.15.122:0/1591854012 learned_addr learned my addr 172.21.15.122:0/1591854012 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2a00640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4c40afb10 0x7fa4c41fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c3400640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4c41605d0 0x7fa4c415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 -- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4c41605d0 msgr2=0x7fa4c415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4c41605d0 0x7fa4c415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 -- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4c40afb10 msgr2=0x7fa4c41fb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4c40afb10 0x7fa4c41fb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 -- 172.21.15.122:0/1591854012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c415f600 con 0x7fa4c40b1650 2024-09-17T05:37:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4c2000640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40b1650 0x7fa4c4156100 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c500 tx=0x7fa4b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4baa00640 1 -- 172.21.15.122:0/1591854012 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0017070 con 0x7fa4c40b1650 2024-09-17T05:37:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4baa00640 1 -- 172.21.15.122:0/1591854012 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4b0003600 con 0x7fa4c40b1650 2024-09-17T05:37:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.789+0000 7fa4baa00640 1 -- 172.21.15.122:0/1591854012 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0003a90 con 0x7fa4c40b1650 2024-09-17T05:37:52.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.793+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c415f8e0 con 0x7fa4c40b1650 2024-09-17T05:37:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.793+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c415cf90 con 0x7fa4c40b1650 2024-09-17T05:37:52.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.793+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c4157030 con 0x7fa4c40b1650 2024-09-17T05:37:52.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.793+0000 7fa4baa00640 1 -- 172.21.15.122:0/1591854012 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fa4b001a070 con 0x7fa4c40b1650 2024-09-17T05:37:52.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.793+0000 7fa4baa00640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4a4074c70 0x7fa4a4077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:52.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.793+0000 7fa4baa00640 1 -- 172.21.15.122:0/1591854012 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fa4b0092500 con 0x7fa4c40b1650 2024-09-17T05:37:52.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.793+0000 7fa4c2a00640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4a4074c70 0x7fa4a4077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:52.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.797+0000 7fa4c2a00640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4a4074c70 0x7fa4a4077130 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400bb50 tx=0x7fa4b400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:52.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.797+0000 7fa4baa00640 1 -- 172.21.15.122:0/1591854012 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa4b005ea80 con 0x7fa4c40b1650 2024-09-17T05:37:52.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.993+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa4c4156640 con 0x7fa4a4074c70 2024-09-17T05:37:52.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:52.997+0000 7fa4baa00640 1 -- 172.21.15.122:0/1591854012 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa4b4010bd0 con 0x7fa4a4074c70 2024-09-17T05:37:52.994 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:37:52.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4a4074c70 msgr2=0x7fa4a4077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4a4074c70 0x7fa4a4077130 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400bb50 tx=0x7fa4b400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40b1650 msgr2=0x7fa4c4156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40b1650 0x7fa4c4156100 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c500 tx=0x7fa4b000c9c0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 shutdown_connections 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4a4074c70 0x7fa4a4077130 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4c41605d0 0x7fa4c415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4c40b1650 0x7fa4c4156100 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 --2- 172.21.15.122:0/1591854012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4c40afb10 0x7fa4c41fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 >> 172.21.15.122:0/1591854012 conn(0x7fa4c40d1dd0 msgr2=0x7fa4c40b6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 shutdown_connections 2024-09-17T05:37:52.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.001+0000 7fa4c9800640 1 -- 172.21.15.122:0/1591854012 wait complete. 2024-09-17T05:37:53.016 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:37:53.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 -- 172.21.15.122:0/1502838495 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4080cdf20 msgr2=0x7fc4080b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:53.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 --2- 172.21.15.122:0/1502838495 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4080cdf20 0x7fc4080b1860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc400009c80 tx=0x7fc40001cd70 comp rx=0 tx=0).stop 2024-09-17T05:37:53.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 -- 172.21.15.122:0/1502838495 shutdown_connections 2024-09-17T05:37:53.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 --2- 172.21.15.122:0/1502838495 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4080cdf20 0x7fc4080b1860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:53.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 --2- 172.21.15.122:0/1502838495 >> v2:172.21.15.173:3300/0 conn(0x7fc4080b3bb0 0x7fc4080b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:53.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 -- 172.21.15.122:0/1502838495 >> 172.21.15.122:0/1502838495 conn(0x7fc4080d1e00 msgr2=0x7fc4080d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:53.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 -- 172.21.15.122:0/1502838495 shutdown_connections 2024-09-17T05:37:53.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 -- 172.21.15.122:0/1502838495 wait complete. 2024-09-17T05:37:53.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.165+0000 7fc40e200640 1 Processor -- start 2024-09-17T05:37:53.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 -- start start 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4080b3bb0 0x7fc408165f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4080cdf20 0x7fc4080b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40815f340 0x7fc4081faf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc408168ee0 con 0x7fc40815f340 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc408168d60 con 0x7fc4080cdf20 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc408169060 con 0x7fc4080b3bb0 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40815f340 0x7fc4081faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40815f340 0x7fc4081faf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50250/0 (socket says 172.21.15.122:50250) 2024-09-17T05:37:53.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 -- 172.21.15.122:0/3624827636 learned_addr learned my addr 172.21.15.122:0/3624827636 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:37:53.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc3fea00640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4080cdf20 0x7fc4080b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:53.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407400640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4080b3bb0 0x7fc408165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:53.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 -- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4080b3bb0 msgr2=0x7fc408165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:53.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4080b3bb0 0x7fc408165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:53.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 -- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4080cdf20 msgr2=0x7fc4080b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:53.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4080cdf20 0x7fc4080b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:53.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 -- 172.21.15.122:0/3624827636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4080b5a00 con 0x7fc40815f340 2024-09-17T05:37:53.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc407e00640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40815f340 0x7fc4081faf40 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc40001c7e0 tx=0x7fc400005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:53.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc3ffe00640 1 -- 172.21.15.122:0/3624827636 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4000b2040 con 0x7fc40815f340 2024-09-17T05:37:53.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc3ffe00640 1 -- 172.21.15.122:0/3624827636 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4000a5c60 con 0x7fc40815f340 2024-09-17T05:37:53.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc3ffe00640 1 -- 172.21.15.122:0/3624827636 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400004240 con 0x7fc40815f340 2024-09-17T05:37:53.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4080b5ce0 con 0x7fc40815f340 2024-09-17T05:37:53.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4080b4fd0 con 0x7fc40815f340 2024-09-17T05:37:53.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.169+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4080b34a0 con 0x7fc40815f340 2024-09-17T05:37:53.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.173+0000 7fc3ffe00640 1 -- 172.21.15.122:0/3624827636 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 43) ==== 99631+0+0 (secure 0 0 0) 0x7fc4000bd030 con 0x7fc40815f340 2024-09-17T05:37:53.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.173+0000 7fc3ffe00640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc3e4074c70 0x7fc3e4077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:37:53.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.173+0000 7fc3ffe00640 1 -- 172.21.15.122:0/3624827636 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fc40012d8f0 con 0x7fc40815f340 2024-09-17T05:37:53.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.173+0000 7fc407400640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc3e4074c70 0x7fc3e4077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:37:53.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.173+0000 7fc407400640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc3e4074c70 0x7fc3e4077130 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8007fd0 tx=0x7fc3f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:37:53.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.177+0000 7fc3ffe00640 1 -- 172.21.15.122:0/3624827636 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc4000c4060 con 0x7fc40815f340 2024-09-17T05:37:53.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.368+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc4080b0e70 con 0x7fc3e4074c70 2024-09-17T05:37:53.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc3ffe00640 1 -- 172.21.15.122:0/3624827636 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fc4080b0e70 con 0x7fc3e4074c70 2024-09-17T05:37:53.371 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc3e4074c70 msgr2=0x7fc3e4077130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc3e4074c70 0x7fc3e4077130 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8007fd0 tx=0x7fc3f8009040 comp rx=0 tx=0).stop 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40815f340 msgr2=0x7fc4081faf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40815f340 0x7fc4081faf40 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc40001c7e0 tx=0x7fc400005c40 comp rx=0 tx=0).stop 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 shutdown_connections 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc3e4074c70 0x7fc3e4077130 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc40815f340 0x7fc4081faf40 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4080cdf20 0x7fc4080b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 --2- 172.21.15.122:0/3624827636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4080b3bb0 0x7fc408165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:37:53.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 >> 172.21.15.122:0/3624827636 conn(0x7fc4080d1e00 msgr2=0x7fc40815b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:37:53.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 shutdown_connections 2024-09-17T05:37:53.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:37:53.376+0000 7fc40e200640 1 -- 172.21.15.122:0/3624827636 wait complete. 2024-09-17T05:37:53.393 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:37:53.393 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:37:53.393 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:37:53.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:37:53.403 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:38:02.208 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:38:02.208 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:38:02.208 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.75025 s, 123 MB/s 2024-09-17T05:38:02.209 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:38:02.209 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:38:02.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:38:02.218 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:38:03.977 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:38:03.981 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:38:11.268 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:38:11.268 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:38:11.269 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.99902 s, 119 MB/s 2024-09-17T05:38:11.270 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:38:11.270 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:38:11.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:38:11.279 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:38:20.051 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:38:20.051 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:38:20.051 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.7232 s, 123 MB/s 2024-09-17T05:38:20.053 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T05:38:20.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.200+0000 7f4fc6400640 1 -- 172.21.15.122:0/2790003382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc015eff0 msgr2=0x7f4fc015f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.200+0000 7f4fc6400640 1 --2- 172.21.15.122:0/2790003382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc015eff0 0x7f4fc015f480 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009bc0 tx=0x7f4fb801cba0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.200+0000 7f4fc6400640 1 -- 172.21.15.122:0/2790003382 shutdown_connections 2024-09-17T05:38:20.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.200+0000 7f4fc6400640 1 --2- 172.21.15.122:0/2790003382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc015eff0 0x7f4fc015f480 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.200+0000 7f4fc6400640 1 --2- 172.21.15.122:0/2790003382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4fc00d09f0 0x7f4fc00d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.200+0000 7f4fc6400640 1 -- 172.21.15.122:0/2790003382 >> 172.21.15.122:0/2790003382 conn(0x7f4fc00d1da0 msgr2=0x7f4fc00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:20.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.200+0000 7f4fc6400640 1 -- 172.21.15.122:0/2790003382 shutdown_connections 2024-09-17T05:38:20.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.200+0000 7f4fc6400640 1 -- 172.21.15.122:0/2790003382 wait complete. 2024-09-17T05:38:20.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 Processor -- start 2024-09-17T05:38:20.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 -- start start 2024-09-17T05:38:20.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc00cff00 0x7f4fc0151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:20.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4fc015eff0 0x7f4fc00cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:20.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc00b67e0 0x7f4fc00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:20.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc00b0d10 con 0x7f4fc00b67e0 2024-09-17T05:38:20.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc00b0b90 con 0x7f4fc00cff00 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc0168bf0 con 0x7f4fc015eff0 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc00b67e0 0x7f4fc00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc00b67e0 0x7f4fc00cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50004/0 (socket says 172.21.15.122:50004) 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 -- 172.21.15.122:0/726514003 learned_addr learned my addr 172.21.15.122:0/726514003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fbfe00640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4fc015eff0 0x7f4fc00cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5000640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc00cff00 0x7f4fc0151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 -- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4fc015eff0 msgr2=0x7f4fc00cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4fc015eff0 0x7f4fc00cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 -- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc00cff00 msgr2=0x7f4fc0151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc00cff00 0x7f4fc0151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 -- 172.21.15.122:0/726514003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc00b6fc0 con 0x7f4fc00b67e0 2024-09-17T05:38:20.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5000640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc00cff00 0x7f4fc0151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:38:20.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc5a00640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc00b67e0 0x7f4fc00cdca0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4fb801c610 tx=0x7f4fb8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:20.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fbcc00640 1 -- 172.21.15.122:0/726514003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80b2040 con 0x7f4fc00b67e0 2024-09-17T05:38:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fbcc00640 1 -- 172.21.15.122:0/726514003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fb80bb070 con 0x7f4fc00b67e0 2024-09-17T05:38:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fbcc00640 1 -- 172.21.15.122:0/726514003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8004280 con 0x7f4fc00b67e0 2024-09-17T05:38:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc00b6ce0 con 0x7f4fc00b67e0 2024-09-17T05:38:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.204+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc00b77b0 con 0x7f4fc00b67e0 2024-09-17T05:38:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.208+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc00d28c0 con 0x7f4fc00b67e0 2024-09-17T05:38:20.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.208+0000 7f4fbcc00640 1 -- 172.21.15.122:0/726514003 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 44) ==== 99733+0+0 (secure 0 0 0) 0x7f4fb80a5a50 con 0x7f4fc00b67e0 2024-09-17T05:38:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.208+0000 7f4fbcc00640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4fa0074dc0 0x7f4fa0077280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.208+0000 7f4fbcc00640 1 -- 172.21.15.122:0/726514003 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f4fb812d6c0 con 0x7f4fc00b67e0 2024-09-17T05:38:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.208+0000 7f4fc5000640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4fa0074dc0 0x7f4fa0077280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:20.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.208+0000 7f4fc5000640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4fa0074dc0 0x7f4fa0077280 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0156c40 tx=0x7f4fb4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:20.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.212+0000 7f4fbcc00640 1 -- 172.21.15.122:0/726514003 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4fb8016790 con 0x7f4fc00b67e0 2024-09-17T05:38:20.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.412+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f4fc0069e40 con 0x7f4fa0074dc0 2024-09-17T05:38:20.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.424+0000 7f4fbcc00640 1 -- 172.21.15.122:0/726514003 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4fc0069e40 con 0x7f4fa0074dc0 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4fa0074dc0 msgr2=0x7f4fa0077280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4fa0074dc0 0x7f4fa0077280 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0156c40 tx=0x7f4fb4009040 comp rx=0 tx=0).stop 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc00b67e0 msgr2=0x7f4fc00cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc00b67e0 0x7f4fc00cdca0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4fb801c610 tx=0x7f4fb8003f40 comp rx=0 tx=0).stop 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 shutdown_connections 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4fa0074dc0 0x7f4fa0077280 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fc00b67e0 0x7f4fc00cdca0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4fc015eff0 0x7f4fc00cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 --2- 172.21.15.122:0/726514003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fc00cff00 0x7f4fc0151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 >> 172.21.15.122:0/726514003 conn(0x7f4fc00d1da0 msgr2=0x7f4fc0071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:20.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 shutdown_connections 2024-09-17T05:38:20.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.428+0000 7f4fc6400640 1 -- 172.21.15.122:0/726514003 wait complete. 2024-09-17T05:38:20.445 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:38:20.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- 172.21.15.122:0/1901528758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd224004d50 msgr2=0x7fd2240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 --2- 172.21.15.122:0/1901528758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd224004d50 0x7fd2240051d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd238009ec0 tx=0x7fd23801d6a0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd245c00640 1 -- 172.21.15.122:0/1901528758 reap_dead start 2024-09-17T05:38:20.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- 172.21.15.122:0/1901528758 shutdown_connections 2024-09-17T05:38:20.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- 172.21.15.122:0/1901528758 >> 172.21.15.122:0/1901528758 conn(0x7fd2400cfa00 msgr2=0x7fd2400cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:20.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- 172.21.15.122:0/1901528758 shutdown_connections 2024-09-17T05:38:20.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- 172.21.15.122:0/1901528758 wait complete. 2024-09-17T05:38:20.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 Processor -- start 2024-09-17T05:38:20.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- start start 2024-09-17T05:38:20.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd224004d50 0x7fd2401d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:20.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2400ad890 0x7fd2400add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:20.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2400b3b10 0x7fd2400b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd2400738b0 con 0x7fd2400ad890 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd2400aec00 con 0x7fd2400b3b10 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd246600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd240073a30 con 0x7fd224004d50 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd237e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2400ad890 0x7fd2400add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd237e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2400ad890 0x7fd2400add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50034/0 (socket says 172.21.15.122:50034) 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd237e00640 1 -- 172.21.15.122:0/3004358962 learned_addr learned my addr 172.21.15.122:0/3004358962 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.596+0000 7fd245200640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd224004d50 0x7fd2401d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd245c00640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2400b3b10 0x7fd2400b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd237e00640 1 -- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd224004d50 msgr2=0x7fd2401d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd237e00640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd224004d50 0x7fd2401d3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd237e00640 1 -- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2400b3b10 msgr2=0x7fd2400b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd237e00640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2400b3b10 0x7fd2400b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd237e00640 1 -- 172.21.15.122:0/3004358962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2401f7e10 con 0x7fd2400ad890 2024-09-17T05:38:20.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd237e00640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2400ad890 0x7fd2400add10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd228006630 tx=0x7fd2280039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:20.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd23d600640 1 -- 172.21.15.122:0/3004358962 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228015020 con 0x7fd2400ad890 2024-09-17T05:38:20.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd23d600640 1 -- 172.21.15.122:0/3004358962 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd228011900 con 0x7fd2400ad890 2024-09-17T05:38:20.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd23d600640 1 -- 172.21.15.122:0/3004358962 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228011cd0 con 0x7fd2400ad890 2024-09-17T05:38:20.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2401f80f0 con 0x7fd2400ad890 2024-09-17T05:38:20.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2400b44d0 con 0x7fd2400ad890 2024-09-17T05:38:20.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24006e650 con 0x7fd2400ad890 2024-09-17T05:38:20.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.600+0000 7fd23d600640 1 -- 172.21.15.122:0/3004358962 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 44) ==== 99733+0+0 (secure 0 0 0) 0x7fd228018020 con 0x7fd2400ad890 2024-09-17T05:38:20.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.604+0000 7fd23d600640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd21c074d90 0x7fd21c077250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:20.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.604+0000 7fd23d600640 1 -- 172.21.15.122:0/3004358962 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fd228093c00 con 0x7fd2400ad890 2024-09-17T05:38:20.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.604+0000 7fd245200640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd21c074d90 0x7fd21c077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:20.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.604+0000 7fd245200640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd21c074d90 0x7fd21c077250 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd23000acd0 tx=0x7fd23000a2f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:20.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.608+0000 7fd23d600640 1 -- 172.21.15.122:0/3004358962 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd22801f020 con 0x7fd2400ad890 2024-09-17T05:38:20.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.800+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd2400ae250 con 0x7fd21c074d90 2024-09-17T05:38:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.804+0000 7fd23d600640 1 -- 172.21.15.122:0/3004358962 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd23000ad00 con 0x7fd21c074d90 2024-09-17T05:38:20.803 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:38:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd21c074d90 msgr2=0x7fd21c077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd21c074d90 0x7fd21c077250 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd23000acd0 tx=0x7fd23000a2f0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2400ad890 msgr2=0x7fd2400add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2400ad890 0x7fd2400add10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd228006630 tx=0x7fd2280039d0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 shutdown_connections 2024-09-17T05:38:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd21c074d90 0x7fd21c077250 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2400b3b10 0x7fd2400b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2400ad890 0x7fd2400add10 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 --2- 172.21.15.122:0/3004358962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd224004d50 0x7fd2401d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 >> 172.21.15.122:0/3004358962 conn(0x7fd2400cfa00 msgr2=0x7fd2400cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 shutdown_connections 2024-09-17T05:38:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:20.808+0000 7fd246600640 1 -- 172.21.15.122:0/3004358962 wait complete. 2024-09-17T05:38:20.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:38:20.825 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:38:20.825 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:38:20.834 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:38:20.834 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T05:38:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:20.835 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T05:38:22.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:38:22.836 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:38:22.836 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:38:22.846 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:38:22.847 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T05:38:22.847 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:22.847 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T05:38:24.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:38:24.849 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:38:24.849 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:38:24.859 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:38:24.860 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T05:38:24.860 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:24.861 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T05:38:24.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone1010 2024-09-17T05:38:25.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 -- 172.21.15.122:0/2047184252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc40d28c0 msgr2=0x7f9dc40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:25.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 --2- 172.21.15.122:0/2047184252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc40d28c0 0x7f9dc40cea20 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4074290 tx=0x7f9dac01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 -- 172.21.15.122:0/2047184252 shutdown_connections 2024-09-17T05:38:25.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 --2- 172.21.15.122:0/2047184252 >> v2:172.21.15.173:3300/0 conn(0x7f9dc40b53f0 0x7f9dc40b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 --2- 172.21.15.122:0/2047184252 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc40d28c0 0x7f9dc40cea20 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 --2- 172.21.15.122:0/2047184252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc415be80 0x7f9dc40ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 -- 172.21.15.122:0/2047184252 >> 172.21.15.122:0/2047184252 conn(0x7f9dc40d1da0 msgr2=0x7f9dc40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:25.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 -- 172.21.15.122:0/2047184252 shutdown_connections 2024-09-17T05:38:25.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.060+0000 7f9dc9800640 1 -- 172.21.15.122:0/2047184252 wait complete. 2024-09-17T05:38:25.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 Processor -- start 2024-09-17T05:38:25.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 -- start start 2024-09-17T05:38:25.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc40b53f0 0x7f9dc4151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:25.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc40d28c0 0x7f9dc4157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:25.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc415be80 0x7f9dc4151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:25.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc40ccfa0 con 0x7f9dc415be80 2024-09-17T05:38:25.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc40cce20 con 0x7f9dc40d28c0 2024-09-17T05:38:25.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc40cd120 con 0x7f9dc40b53f0 2024-09-17T05:38:25.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc415be80 0x7f9dc4151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:25.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc415be80 0x7f9dc4151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46604/0 (socket says 172.21.15.122:46604) 2024-09-17T05:38:25.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 -- 172.21.15.122:0/1812690611 learned_addr learned my addr 172.21.15.122:0/1812690611 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:25.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dba000640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc40d28c0 0x7f9dc4157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:25.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc2a00640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc40b53f0 0x7f9dc4151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:25.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 -- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc40b53f0 msgr2=0x7f9dc4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:25.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc40b53f0 0x7f9dc4151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 -- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc40d28c0 msgr2=0x7f9dc4157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:25.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc40d28c0 0x7f9dc4157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 -- 172.21.15.122:0/1812690611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc40c8ff0 con 0x7f9dc415be80 2024-09-17T05:38:25.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc3400640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc415be80 0x7f9dc4151150 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00d750 tx=0x7f9dbc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:25.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dbb400640 1 -- 172.21.15.122:0/1812690611 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc017070 con 0x7f9dc415be80 2024-09-17T05:38:25.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dbb400640 1 -- 172.21.15.122:0/1812690611 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dbc009d20 con 0x7f9dc415be80 2024-09-17T05:38:25.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dbb400640 1 -- 172.21.15.122:0/1812690611 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00de60 con 0x7f9dc415be80 2024-09-17T05:38:25.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc4155790 con 0x7f9dc415be80 2024-09-17T05:38:25.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.064+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc40c9590 con 0x7f9dc415be80 2024-09-17T05:38:25.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.068+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc40b5880 con 0x7f9dc415be80 2024-09-17T05:38:25.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.068+0000 7f9dbb400640 1 -- 172.21.15.122:0/1812690611 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 44) ==== 99733+0+0 (secure 0 0 0) 0x7f9dbc005280 con 0x7f9dc415be80 2024-09-17T05:38:25.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.068+0000 7f9dbb400640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9da0074dc0 0x7f9da0077280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:25.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.068+0000 7f9dbb400640 1 -- 172.21.15.122:0/1812690611 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f9dbc0931c0 con 0x7f9dc415be80 2024-09-17T05:38:25.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.068+0000 7f9dc2a00640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9da0074dc0 0x7f9da0077280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:25.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.068+0000 7f9dc2a00640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9da0074dc0 0x7f9da0077280 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009dd0 tx=0x7f9db40095e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:25.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.072+0000 7f9dbb400640 1 -- 172.21.15.122:0/1812690611 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9dbc05f680 con 0x7f9dc415be80 2024-09-17T05:38:25.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.272+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone1010", "target": ["mon-mgr", ""]}) -- 0x7f9dc40b3300 con 0x7f9da0074dc0 2024-09-17T05:38:25.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.340+0000 7f9dbb400640 1 -- 172.21.15.122:0/1812690611 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9dc40b3300 con 0x7f9da0074dc0 2024-09-17T05:38:25.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9da0074dc0 msgr2=0x7f9da0077280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:25.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9da0074dc0 0x7f9da0077280 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009dd0 tx=0x7f9db40095e0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc415be80 msgr2=0x7f9dc4151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc415be80 0x7f9dc4151150 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00d750 tx=0x7f9dbc00dc10 comp rx=0 tx=0).stop 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 shutdown_connections 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9da0074dc0 0x7f9da0077280 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9dc415be80 0x7f9dc4151150 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc40d28c0 0x7f9dc4157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 --2- 172.21.15.122:0/1812690611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc40b53f0 0x7f9dc4151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 >> 172.21.15.122:0/1812690611 conn(0x7f9dc40d1da0 msgr2=0x7f9dc4071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 shutdown_connections 2024-09-17T05:38:25.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.344+0000 7f9dc9800640 1 -- 172.21.15.122:0/1812690611 wait complete. 2024-09-17T05:38:25.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.346+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.346+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.346+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.346+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.346+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.358 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.362+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.360 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.362+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.360 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.362+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.360 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.362+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.360 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:25.362+0000 7fc09fa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:25.362 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone291 2024-09-17T05:38:25.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 -- 172.21.15.122:0/3464680072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 msgr2=0x7f1f280aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:25.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3464680072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 0x7f1f280aefb0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1f28073810 tx=0x7f1f2001c8a0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 -- 172.21.15.122:0/3464680072 shutdown_connections 2024-09-17T05:38:25.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3464680072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 0x7f1f280aefb0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3464680072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f280b1650 0x7f1f280b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3464680072 >> v2:172.21.15.173:3300/0 conn(0x7f1f281605d0 0x7f1f280d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 -- 172.21.15.122:0/3464680072 >> 172.21.15.122:0/3464680072 conn(0x7f1f280d1dd0 msgr2=0x7f1f280d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:25.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 -- 172.21.15.122:0/3464680072 shutdown_connections 2024-09-17T05:38:25.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 -- 172.21.15.122:0/3464680072 wait complete. 2024-09-17T05:38:25.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 Processor -- start 2024-09-17T05:38:25.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.516+0000 7f1f2d400640 1 -- start start 2024-09-17T05:38:25.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 0x7f1f2814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:25.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f280b1650 0x7f1f28150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:25.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f281605d0 0x7f1f281512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f280b13a0 con 0x7f1f280aeb00 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1f280b1220 con 0x7f1f280b1650 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1f280afd00 con 0x7f1f281605d0 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 0x7f1f2814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 0x7f1f2814ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46624/0 (socket says 172.21.15.122:46624) 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 -- 172.21.15.122:0/3329067170 learned_addr learned my addr 172.21.15.122:0/3329067170 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f1de00640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f280b1650 0x7f1f28150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f27200640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f281605d0 0x7f1f281512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 -- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f281605d0 msgr2=0x7f1f281512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f281605d0 0x7f1f281512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 -- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f280b1650 msgr2=0x7f1f28150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:25.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f280b1650 0x7f1f28150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:25.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 -- 172.21.15.122:0/3329067170 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2814f320 con 0x7f1f280aeb00 2024-09-17T05:38:25.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f26800640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 0x7f1f2814ede0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800ead0 tx=0x7f1f1800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:25.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f1f400640 1 -- 172.21.15.122:0/3329067170 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1800ce40 con 0x7f1f280aeb00 2024-09-17T05:38:25.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f1f400640 1 -- 172.21.15.122:0/3329067170 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f18011470 con 0x7f1f280aeb00 2024-09-17T05:38:25.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f1f400640 1 -- 172.21.15.122:0/3329067170 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f180118a0 con 0x7f1f280aeb00 2024-09-17T05:38:25.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f280b5440 con 0x7f1f280aeb00 2024-09-17T05:38:25.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f280b7150 con 0x7f1f280aeb00 2024-09-17T05:38:25.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f280b1a30 con 0x7f1f280aeb00 2024-09-17T05:38:25.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.520+0000 7f1f1f400640 1 -- 172.21.15.122:0/3329067170 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 44) ==== 99733+0+0 (secure 0 0 0) 0x7f1f18011a40 con 0x7f1f280aeb00 2024-09-17T05:38:25.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.524+0000 7f1f1f400640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f04074dc0 0x7f1f04077280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:25.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.524+0000 7f1f1f400640 1 -- 172.21.15.122:0/3329067170 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f1f18093c60 con 0x7f1f280aeb00 2024-09-17T05:38:25.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.524+0000 7f1f1de00640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f04074dc0 0x7f1f04077280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:25.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.524+0000 7f1f1de00640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f04074dc0 0x7f1f04077280 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1f28152120 tx=0x7f1f1000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:25.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.528+0000 7f1f1f400640 1 -- 172.21.15.122:0/3329067170 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1f18016030 con 0x7f1f280aeb00 2024-09-17T05:38:25.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:25.724+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone291", "target": ["mon-mgr", ""]}) -- 0x7f1f280d1080 con 0x7f1f04074dc0 2024-09-17T05:38:26.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:26.346+0000 7fc027a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:26.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:26.346+0000 7fc027a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:26.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:26.346+0000 7fc027a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:26.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:26.346+0000 7fc027a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:26.342 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:26.346+0000 7fc027a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:26.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.416+0000 7f1f1f400640 1 -- 172.21.15.122:0/3329067170 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f280d1080 con 0x7f1f04074dc0 2024-09-17T05:38:26.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f04074dc0 msgr2=0x7f1f04077280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:26.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f04074dc0 0x7f1f04077280 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1f28152120 tx=0x7f1f1000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 msgr2=0x7f1f2814ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:26.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 0x7f1f2814ede0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800ead0 tx=0x7f1f1800ef90 comp rx=0 tx=0).stop 2024-09-17T05:38:26.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 shutdown_connections 2024-09-17T05:38:26.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f04074dc0 0x7f1f04077280 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f281605d0 0x7f1f281512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f280b1650 0x7f1f28150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 --2- 172.21.15.122:0/3329067170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f280aeb00 0x7f1f2814ede0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 >> 172.21.15.122:0/3329067170 conn(0x7f1f280d1dd0 msgr2=0x7f1f280ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:26.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 shutdown_connections 2024-09-17T05:38:26.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.420+0000 7f1f2d400640 1 -- 172.21.15.122:0/3329067170 wait complete. 2024-09-17T05:38:26.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone78512 2024-09-17T05:38:26.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.668+0000 7fd04b400640 1 -- 172.21.15.122:0/3684868879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e0e0 msgr2=0x7fd0440afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:26.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.668+0000 7fd04b400640 1 --2- 172.21.15.122:0/3684868879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e0e0 0x7fd0440afa10 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd0440c75b0 tx=0x7fd03801c800 comp rx=0 tx=0).stop 2024-09-17T05:38:26.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.668+0000 7fd04b400640 1 -- 172.21.15.122:0/3684868879 shutdown_connections 2024-09-17T05:38:26.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.668+0000 7fd04b400640 1 --2- 172.21.15.122:0/3684868879 >> v2:172.21.15.173:3300/0 conn(0x7fd044073780 0x7fd044070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.668+0000 7fd04b400640 1 --2- 172.21.15.122:0/3684868879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e0e0 0x7fd0440afa10 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.668+0000 7fd04b400640 1 -- 172.21.15.122:0/3684868879 >> 172.21.15.122:0/3684868879 conn(0x7fd0440b2060 msgr2=0x7fd0440b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:26.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.668+0000 7fd04b400640 1 -- 172.21.15.122:0/3684868879 shutdown_connections 2024-09-17T05:38:26.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.668+0000 7fd04b400640 1 -- 172.21.15.122:0/3684868879 wait complete. 2024-09-17T05:38:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.672+0000 7fd04b400640 1 Processor -- start 2024-09-17T05:38:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.672+0000 7fd04b400640 1 -- start start 2024-09-17T05:38:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.672+0000 7fd04b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd044073780 0x7fd0441568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.672+0000 7fd04b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e7f0 0x7fd0440c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd04b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0440b4d90 0x7fd0440c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd04b400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd044167f20 con 0x7fd04415e7f0 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd04b400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd044167da0 con 0x7fd0440b4d90 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd04b400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd0441680a0 con 0x7fd044073780 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e7f0 0x7fd0440c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e7f0 0x7fd0440c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46642/0 (socket says 172.21.15.122:46642) 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 -- 172.21.15.122:0/1065113332 learned_addr learned my addr 172.21.15.122:0/1065113332 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd048a00640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd044073780 0x7fd0441568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd049400640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0440b4d90 0x7fd0440c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 -- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd044073780 msgr2=0x7fd0441568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd044073780 0x7fd0441568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 -- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0440b4d90 msgr2=0x7fd0440c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0440b4d90 0x7fd0440c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:26.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 -- 172.21.15.122:0/1065113332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0441fbc70 con 0x7fd04415e7f0 2024-09-17T05:38:26.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd048a00640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd044073780 0x7fd0441568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:38:26.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd049400640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0440b4d90 0x7fd0440c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:38:26.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd043e00640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e7f0 0x7fd0440c9090 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c3c0 tx=0x7fd03000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:26.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd040c00640 1 -- 172.21.15.122:0/1065113332 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030011790 con 0x7fd04415e7f0 2024-09-17T05:38:26.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd040c00640 1 -- 172.21.15.122:0/1065113332 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd030017070 con 0x7fd04415e7f0 2024-09-17T05:38:26.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0441fbf50 con 0x7fd04415e7f0 2024-09-17T05:38:26.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd040c00640 1 -- 172.21.15.122:0/1065113332 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030011d40 con 0x7fd04415e7f0 2024-09-17T05:38:26.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0441fc510 con 0x7fd04415e7f0 2024-09-17T05:38:26.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.676+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0440afa10 con 0x7fd04415e7f0 2024-09-17T05:38:26.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.680+0000 7fd040c00640 1 -- 172.21.15.122:0/1065113332 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 44) ==== 99733+0+0 (secure 0 0 0) 0x7fd03001c460 con 0x7fd04415e7f0 2024-09-17T05:38:26.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.680+0000 7fd040c00640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd024074d90 0x7fd024077250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:26.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.680+0000 7fd040c00640 1 -- 172.21.15.122:0/1065113332 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fd030092e60 con 0x7fd04415e7f0 2024-09-17T05:38:26.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.684+0000 7fd048a00640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd024074d90 0x7fd024077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:26.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.684+0000 7fd040c00640 1 -- 172.21.15.122:0/1065113332 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd03005fa10 con 0x7fd04415e7f0 2024-09-17T05:38:26.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:26.688+0000 7fd048a00640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd024074d90 0x7fd024077250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd0380af000 tx=0x7fd03801b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:27.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.072+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone78512", "target": ["mon-mgr", ""]}) -- 0x7fd0441580f0 con 0x7fd024074d90 2024-09-17T05:38:27.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.400+0000 7fd040c00640 1 -- 172.21.15.122:0/1065113332 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 45) ==== 99886+0+0 (secure 0 0 0) 0x7fd03005b580 con 0x7fd04415e7f0 2024-09-17T05:38:27.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.660+0000 7fd040c00640 1 -- 172.21.15.122:0/1065113332 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd0441580f0 con 0x7fd024074d90 2024-09-17T05:38:27.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd024074d90 msgr2=0x7fd024077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:27.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd024074d90 0x7fd024077250 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd0380af000 tx=0x7fd03801b040 comp rx=0 tx=0).stop 2024-09-17T05:38:27.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e7f0 msgr2=0x7fd0440c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:27.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e7f0 0x7fd0440c9090 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c3c0 tx=0x7fd03000c880 comp rx=0 tx=0).stop 2024-09-17T05:38:27.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 shutdown_connections 2024-09-17T05:38:27.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd024074d90 0x7fd024077250 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:27.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0440b4d90 0x7fd0440c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:27.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd04415e7f0 0x7fd0440c9090 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:27.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 --2- 172.21.15.122:0/1065113332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd044073780 0x7fd0441568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:27.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 >> 172.21.15.122:0/1065113332 conn(0x7fd0440b2060 msgr2=0x7fd0441571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:27.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 shutdown_connections 2024-09-17T05:38:27.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:27.664+0000 7fd04b400640 1 -- 172.21.15.122:0/1065113332 wait complete. 2024-09-17T05:38:27.712 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.718+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.712 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.718+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.712 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.718+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.713 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.718+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.713 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.718+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.800 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.802+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.800 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.802+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.800 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.802+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.800 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.802+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:27.800 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:27.802+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:28.706 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone1010 2024-09-17T05:38:28.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.959+0000 7f8de8800640 1 -- 172.21.15.122:0/3483954645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 msgr2=0x7f8de006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:28.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.959+0000 7f8de8800640 1 --2- 172.21.15.122:0/3483954645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 0x7f8de006e7a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009840 tx=0x7f8dd401c790 comp rx=0 tx=0).stop 2024-09-17T05:38:28.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.959+0000 7f8de8800640 1 -- 172.21.15.122:0/3483954645 shutdown_connections 2024-09-17T05:38:28.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.959+0000 7f8de8800640 1 --2- 172.21.15.122:0/3483954645 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8de00b40a0 0x7f8de00ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:28.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.959+0000 7f8de8800640 1 --2- 172.21.15.122:0/3483954645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 0x7f8de006e7a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:28.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.959+0000 7f8de8800640 1 -- 172.21.15.122:0/3483954645 >> 172.21.15.122:0/3483954645 conn(0x7f8de00b18f0 msgr2=0x7f8de00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:28.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.959+0000 7f8de8800640 1 -- 172.21.15.122:0/3483954645 shutdown_connections 2024-09-17T05:38:28.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.959+0000 7f8de8800640 1 -- 172.21.15.122:0/3483954645 wait complete. 2024-09-17T05:38:28.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de8800640 1 Processor -- start 2024-09-17T05:38:28.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de8800640 1 -- start start 2024-09-17T05:38:28.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de8800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de006ece0 0x7f8de01ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:28.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de8800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8de00b40a0 0x7f8de009cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:28.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de8800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 0x7f8de009d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:28.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de8800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8de016aec0 con 0x7f8de0161520 2024-09-17T05:38:28.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de8800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8de016b040 con 0x7f8de006ece0 2024-09-17T05:38:28.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de8800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8de016b1c0 con 0x7f8de00b40a0 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 0x7f8de009d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 0x7f8de009d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46646/0 (socket says 172.21.15.122:46646) 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 -- 172.21.15.122:0/7011605 learned_addr learned my addr 172.21.15.122:0/7011605 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 -- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8de00b40a0 msgr2=0x7f8de009cff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8de00b40a0 0x7f8de009cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 -- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de006ece0 msgr2=0x7f8de01ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de006ece0 0x7f8de01ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 -- 172.21.15.122:0/7011605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de01fc290 con 0x7f8de0161520 2024-09-17T05:38:28.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.963+0000 7f8de6800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 0x7f8de009d530 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00eb20 tx=0x7f8ddc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:28.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.967+0000 7f8dda000640 1 -- 172.21.15.122:0/7011605 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc004b30 con 0x7f8de0161520 2024-09-17T05:38:28.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.967+0000 7f8dda000640 1 -- 172.21.15.122:0/7011605 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ddc004cf0 con 0x7f8de0161520 2024-09-17T05:38:28.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.967+0000 7f8dda000640 1 -- 172.21.15.122:0/7011605 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc005280 con 0x7f8de0161520 2024-09-17T05:38:28.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.967+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de01fc570 con 0x7f8de0161520 2024-09-17T05:38:28.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.967+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de01fcb30 con 0x7f8de0161520 2024-09-17T05:38:28.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.967+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de00ad300 con 0x7f8de0161520 2024-09-17T05:38:28.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.971+0000 7f8dda000640 1 -- 172.21.15.122:0/7011605 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 45) ==== 99886+0+0 (secure 0 0 0) 0x7f8ddc005420 con 0x7f8de0161520 2024-09-17T05:38:28.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.971+0000 7f8dda000640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dc4074f40 0x7f8dc4077400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:28.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.971+0000 7f8dda000640 1 -- 172.21.15.122:0/7011605 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f8ddc093e40 con 0x7f8de0161520 2024-09-17T05:38:28.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.979+0000 7f8dda000640 1 -- 172.21.15.122:0/7011605 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8ddc0609f0 con 0x7f8de0161520 2024-09-17T05:38:28.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.983+0000 7f8de5e00640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dc4074f40 0x7f8dc4077400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:28.991+0000 7f8de5e00640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dc4074f40 0x7f8dc4077400 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40af000 tx=0x7f8dd4019590 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:29.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.323+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone1010", "target": ["mon-mgr", ""]}) -- 0x7f8de00b2750 con 0x7f8dc4074f40 2024-09-17T05:38:29.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.455+0000 7f8dda000640 1 -- 172.21.15.122:0/7011605 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7f8ddc09f080 con 0x7f8de0161520 2024-09-17T05:38:29.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.871+0000 7f8dda000640 1 -- 172.21.15.122:0/7011605 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8de00b2750 con 0x7f8dc4074f40 2024-09-17T05:38:29.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dc4074f40 msgr2=0x7f8dc4077400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dc4074f40 0x7f8dc4077400 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40af000 tx=0x7f8dd4019590 comp rx=0 tx=0).stop 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 msgr2=0x7f8de009d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 0x7f8de009d530 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00eb20 tx=0x7f8ddc00c2d0 comp rx=0 tx=0).stop 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 shutdown_connections 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dc4074f40 0x7f8dc4077400 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8de0161520 0x7f8de009d530 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8de00b40a0 0x7f8de009cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 --2- 172.21.15.122:0/7011605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de006ece0 0x7f8de01ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 >> 172.21.15.122:0/7011605 conn(0x7f8de00b18f0 msgr2=0x7f8de00af810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 shutdown_connections 2024-09-17T05:38:29.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:29.875+0000 7f8de8800640 1 -- 172.21.15.122:0/7011605 wait complete. 2024-09-17T05:38:29.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone291 2024-09-17T05:38:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.167+0000 7fbbd9a00640 1 -- 172.21.15.122:0/3795318530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 msgr2=0x7fbbb8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:30.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.167+0000 7fbbd9a00640 1 --2- 172.21.15.122:0/3795318530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 0x7fbbb8003e10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c610 tx=0x7fbbcc01d300 comp rx=0 tx=0).stop 2024-09-17T05:38:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.167+0000 7fbbd9a00640 1 -- 172.21.15.122:0/3795318530 shutdown_connections 2024-09-17T05:38:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.167+0000 7fbbd9a00640 1 --2- 172.21.15.122:0/3795318530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 0x7fbbb8003e10 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.167+0000 7fbbd9a00640 1 -- 172.21.15.122:0/3795318530 >> 172.21.15.122:0/3795318530 conn(0x7fbbd40b18f0 msgr2=0x7fbbd40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:30.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.167+0000 7fbbd9a00640 1 -- 172.21.15.122:0/3795318530 shutdown_connections 2024-09-17T05:38:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.167+0000 7fbbd9a00640 1 -- 172.21.15.122:0/3795318530 wait complete. 2024-09-17T05:38:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd9a00640 1 Processor -- start 2024-09-17T05:38:30.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd9a00640 1 -- start start 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 0x7fbbd413e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd9a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbd413ee00 0x7fbbd4140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd9a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbd4140950 0x7fbbd41422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd9a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd415eec0 con 0x7fbbb8003a10 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd9a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd415ed40 con 0x7fbbd4140950 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd9a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd415f040 con 0x7fbbd413ee00 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 0x7fbbd413e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 0x7fbbd413e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46666/0 (socket says 172.21.15.122:46666) 2024-09-17T05:38:30.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.171+0000 7fbbd2a00640 1 -- 172.21.15.122:0/867997068 learned_addr learned my addr 172.21.15.122:0/867997068 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd2000640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbd413ee00 0x7fbbd4140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd2a00640 1 -- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbd413ee00 msgr2=0x7fbbd4140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd2a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbd413ee00 0x7fbbd4140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd2a00640 1 -- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbd4140950 msgr2=0x7fbbd41422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd2a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbd4140950 0x7fbbd41422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd2a00640 1 -- 172.21.15.122:0/867997068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd413f370 con 0x7fbbb8003a10 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd2a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 0x7fbbd413e540 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c610 tx=0x7fbbcc0ada90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbcaa00640 1 -- 172.21.15.122:0/867997068 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc004e60 con 0x7fbbb8003a10 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd413f650 con 0x7fbbb8003a10 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbcaa00640 1 -- 172.21.15.122:0/867997068 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbcc0adf70 con 0x7fbbb8003a10 2024-09-17T05:38:30.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbcaa00640 1 -- 172.21.15.122:0/867997068 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc002060 con 0x7fbbb8003a10 2024-09-17T05:38:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd413fc10 con 0x7fbbb8003a10 2024-09-17T05:38:30.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.175+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd415c170 con 0x7fbbb8003a10 2024-09-17T05:38:30.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.179+0000 7fbbcaa00640 1 -- 172.21.15.122:0/867997068 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7fbbcc0b8020 con 0x7fbbb8003a10 2024-09-17T05:38:30.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.179+0000 7fbbcaa00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbb0075090 0x7fbbb0077550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:30.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.179+0000 7fbbcaa00640 1 -- 172.21.15.122:0/867997068 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fbbcc0c1060 con 0x7fbbb8003a10 2024-09-17T05:38:30.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.187+0000 7fbbcaa00640 1 -- 172.21.15.122:0/867997068 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbbcc0170b0 con 0x7fbbb8003a10 2024-09-17T05:38:30.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.191+0000 7fbbd2000640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbb0075090 0x7fbbb0077550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:30.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.195+0000 7fbbd2000640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbb0075090 0x7fbbb0077550 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40729b0 tx=0x7fbbc4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:30.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.511+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone291", "target": ["mon-mgr", ""]}) -- 0x7fbbd413ea80 con 0x7fbbb0075090 2024-09-17T05:38:30.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.967+0000 7fbbcaa00640 1 -- 172.21.15.122:0/867997068 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbbd413ea80 con 0x7fbbb0075090 2024-09-17T05:38:30.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbb0075090 msgr2=0x7fbbb0077550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:30.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbb0075090 0x7fbbb0077550 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40729b0 tx=0x7fbbc4008040 comp rx=0 tx=0).stop 2024-09-17T05:38:30.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 msgr2=0x7fbbd413e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:30.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 0x7fbbd413e540 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c610 tx=0x7fbbcc0ada90 comp rx=0 tx=0).stop 2024-09-17T05:38:30.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 shutdown_connections 2024-09-17T05:38:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbbb0075090 0x7fbbb0077550 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbbd4140950 0x7fbbd41422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbbd413ee00 0x7fbbd4140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 --2- 172.21.15.122:0/867997068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbb8003a10 0x7fbbd413e540 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 >> 172.21.15.122:0/867997068 conn(0x7fbbd40b18f0 msgr2=0x7fbbd40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 shutdown_connections 2024-09-17T05:38:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:30.971+0000 7fbbd9a00640 1 -- 172.21.15.122:0/867997068 wait complete. 2024-09-17T05:38:31.001 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone78512 2024-09-17T05:38:31.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 -- 172.21.15.122:0/3681933023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 msgr2=0x7f2b5806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:31.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 --2- 172.21.15.122:0/3681933023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 0x7f2b5806d420 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009840 tx=0x7f2b4401c790 comp rx=0 tx=0).stop 2024-09-17T05:38:31.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 -- 172.21.15.122:0/3681933023 shutdown_connections 2024-09-17T05:38:31.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 --2- 172.21.15.122:0/3681933023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 0x7f2b5806d420 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 --2- 172.21.15.122:0/3681933023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b5815be60 0x7f2b580d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 -- 172.21.15.122:0/3681933023 >> 172.21.15.122:0/3681933023 conn(0x7f2b580d0c70 msgr2=0x7f2b580d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:31.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 -- 172.21.15.122:0/3681933023 shutdown_connections 2024-09-17T05:38:31.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 -- 172.21.15.122:0/3681933023 wait complete. 2024-09-17T05:38:31.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 Processor -- start 2024-09-17T05:38:31.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.231+0000 7f2b60000640 1 -- start start 2024-09-17T05:38:31.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b60000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 0x7f2b580c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:31.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b60000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b5815be60 0x7f2b58158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:31.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b60000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5815f930 0x7f2b5815e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:31.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b60000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58168cd0 con 0x7f2b58074210 2024-09-17T05:38:31.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b60000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b580cd4d0 con 0x7f2b5815f930 2024-09-17T05:38:31.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b60000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2b58168e50 con 0x7f2b5815be60 2024-09-17T05:38:31.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 0x7f2b580c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:31.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 0x7f2b580c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46686/0 (socket says 172.21.15.122:46686) 2024-09-17T05:38:31.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 -- 172.21.15.122:0/2265085297 learned_addr learned my addr 172.21.15.122:0/2265085297 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:31.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 -- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b5815be60 msgr2=0x7f2b58158330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:38:31.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5e000640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5815f930 0x7f2b5815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:31.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b5815be60 0x7f2b58158330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 -- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5815f930 msgr2=0x7f2b5815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5815f930 0x7f2b5815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 -- 172.21.15.122:0/2265085297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b581fae60 con 0x7f2b58074210 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5e000640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5815f930 0x7f2b5815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b5d600640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 0x7f2b580c3ae0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00b550 tx=0x7f2b4c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b55600640 1 -- 172.21.15.122:0/2265085297 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c014070 con 0x7f2b58074210 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b55600640 1 -- 172.21.15.122:0/2265085297 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b4c010960 con 0x7f2b58074210 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b581fb140 con 0x7f2b58074210 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b55600640 1 -- 172.21.15.122:0/2265085297 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c010db0 con 0x7f2b58074210 2024-09-17T05:38:31.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.235+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b581fb700 con 0x7f2b58074210 2024-09-17T05:38:31.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.239+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b580d0a50 con 0x7f2b58074210 2024-09-17T05:38:31.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.239+0000 7f2b55600640 1 -- 172.21.15.122:0/2265085297 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7f2b4c017070 con 0x7f2b58074210 2024-09-17T05:38:31.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.243+0000 7f2b55600640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b38075060 0x7f2b38077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:31.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.243+0000 7f2b55600640 1 -- 172.21.15.122:0/2265085297 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f2b4c093370 con 0x7f2b58074210 2024-09-17T05:38:31.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.243+0000 7f2b5cc00640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b38075060 0x7f2b38077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:31.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.247+0000 7f2b55600640 1 -- 172.21.15.122:0/2265085297 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2b4c060000 con 0x7f2b58074210 2024-09-17T05:38:31.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.251+0000 7f2b5cc00640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b38075060 0x7f2b38077520 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2b440af000 tx=0x7f2b4401afd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:31.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.567+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone78512", "target": ["mon-mgr", ""]}) -- 0x7f2b5806d400 con 0x7f2b38075060 2024-09-17T05:38:31.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.763+0000 7f2b55600640 1 -- 172.21.15.122:0/2265085297 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2b5806d400 con 0x7f2b38075060 2024-09-17T05:38:31.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b38075060 msgr2=0x7f2b38077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:31.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b38075060 0x7f2b38077520 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2b440af000 tx=0x7f2b4401afd0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 msgr2=0x7f2b580c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:31.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 0x7f2b580c3ae0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00b550 tx=0x7f2b4c00ba10 comp rx=0 tx=0).stop 2024-09-17T05:38:31.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 shutdown_connections 2024-09-17T05:38:31.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b38075060 0x7f2b38077520 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b5815f930 0x7f2b5815e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b5815be60 0x7f2b58158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 --2- 172.21.15.122:0/2265085297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b58074210 0x7f2b580c3ae0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 >> 172.21.15.122:0/2265085297 conn(0x7f2b580d0c70 msgr2=0x7f2b580c5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:31.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 shutdown_connections 2024-09-17T05:38:31.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.767+0000 7f2b60000640 1 -- 172.21.15.122:0/2265085297 wait complete. 2024-09-17T05:38:31.781 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone1010 2024-09-17T05:38:31.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.931+0000 7f67fc200640 1 -- 172.21.15.122:0/1327877675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 msgr2=0x7f67f40cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:31.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.931+0000 7f67fc200640 1 --2- 172.21.15.122:0/1327877675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 0x7f67f40cd050 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f67f40d0780 tx=0x7f67ec01c7c0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.931+0000 7f67fc200640 1 -- 172.21.15.122:0/1327877675 shutdown_connections 2024-09-17T05:38:31.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.931+0000 7f67fc200640 1 --2- 172.21.15.122:0/1327877675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 0x7f67f40cd050 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.931+0000 7f67fc200640 1 -- 172.21.15.122:0/1327877675 >> 172.21.15.122:0/1327877675 conn(0x7f67f40d1ea0 msgr2=0x7f67f40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:31.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.931+0000 7f67fc200640 1 -- 172.21.15.122:0/1327877675 shutdown_connections 2024-09-17T05:38:31.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.931+0000 7f67fc200640 1 -- 172.21.15.122:0/1327877675 wait complete. 2024-09-17T05:38:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 Processor -- start 2024-09-17T05:38:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 -- start start 2024-09-17T05:38:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f40cecf0 0x7f67f414f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67f415bd50 0x7f67f406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 0x7f67f406e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67f416a580 con 0x7f67f415c130 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f67f416a400 con 0x7f67f40cecf0 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f67f416a700 con 0x7f67f415bd50 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 0x7f67f406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 0x7f67f406e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50100/0 (socket says 172.21.15.122:50100) 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 -- 172.21.15.122:0/4027588637 learned_addr learned my addr 172.21.15.122:0/4027588637 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67f9800640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f40cecf0 0x7f67f414f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67f8e00640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67f415bd50 0x7f67f406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 -- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67f415bd50 msgr2=0x7f67f406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67f415bd50 0x7f67f406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 -- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f40cecf0 msgr2=0x7f67f414f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f40cecf0 0x7f67f414f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 -- 172.21.15.122:0/4027588637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f414f710 con 0x7f67f415c130 2024-09-17T05:38:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67f9800640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f40cecf0 0x7f67f414f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:38:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67f8e00640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67f415bd50 0x7f67f406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:38:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fa200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 0x7f67f406e270 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009850 tx=0x7f67ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67f1600640 1 -- 172.21.15.122:0/4027588637 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec0bd070 con 0x7f67f415c130 2024-09-17T05:38:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67f1600640 1 -- 172.21.15.122:0/4027588637 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67ec003600 con 0x7f67f415c130 2024-09-17T05:38:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67f1600640 1 -- 172.21.15.122:0/4027588637 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec0b2040 con 0x7f67f415c130 2024-09-17T05:38:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f4154500 con 0x7f67f415c130 2024-09-17T05:38:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.935+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f40b6390 con 0x7f67f415c130 2024-09-17T05:38:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.939+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f40ce5e0 con 0x7f67f415c130 2024-09-17T05:38:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.939+0000 7f67f1600640 1 -- 172.21.15.122:0/4027588637 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7f67ec003a50 con 0x7f67f415c130 2024-09-17T05:38:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.939+0000 7f67f1600640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67d4075060 0x7f67d4077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.939+0000 7f67f1600640 1 -- 172.21.15.122:0/4027588637 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f67ec12d490 con 0x7f67f415c130 2024-09-17T05:38:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.939+0000 7f67f9800640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67d4075060 0x7f67d4077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.939+0000 7f67f9800640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67d4075060 0x7f67d4077520 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f67e8007fd0 tx=0x7f67e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:31.943+0000 7f67f1600640 1 -- 172.21.15.122:0/4027588637 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f67ec0c6080 con 0x7f67f415c130 2024-09-17T05:38:32.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:32.147+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone1010", "target": ["mon-mgr", ""]}) -- 0x7f67f4076360 con 0x7f67d4075060 2024-09-17T05:38:33.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67f1600640 1 -- 172.21.15.122:0/4027588637 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f67f4076360 con 0x7f67d4075060 2024-09-17T05:38:33.264 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:38:33.264 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:38:33.264 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:38:33.264 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:38:33.264 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:33.265 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:38:33.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67d4075060 msgr2=0x7f67d4077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67d4075060 0x7f67d4077520 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f67e8007fd0 tx=0x7f67e8009040 comp rx=0 tx=0).stop 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 msgr2=0x7f67f406e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 0x7f67f406e270 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009850 tx=0x7f67ec004040 comp rx=0 tx=0).stop 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 shutdown_connections 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67d4075060 0x7f67d4077520 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f415c130 0x7f67f406e270 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67f415bd50 0x7f67f406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 --2- 172.21.15.122:0/4027588637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f40cecf0 0x7f67f414f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 >> 172.21.15.122:0/4027588637 conn(0x7f67f40d1ea0 msgr2=0x7f67f4157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 shutdown_connections 2024-09-17T05:38:33.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.271+0000 7f67fc200640 1 -- 172.21.15.122:0/4027588637 wait complete. 2024-09-17T05:38:33.289 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:38:33.289 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone291 2024-09-17T05:38:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.443+0000 7f43b8000640 1 -- 172.21.15.122:0/3648913802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43b006d330 msgr2=0x7f43b00caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.443+0000 7f43b8000640 1 --2- 172.21.15.122:0/3648913802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43b006d330 0x7f43b00caef0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009c80 tx=0x7f43a801cd70 comp rx=0 tx=0).stop 2024-09-17T05:38:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.443+0000 7f43b8000640 1 -- 172.21.15.122:0/3648913802 shutdown_connections 2024-09-17T05:38:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.443+0000 7f43b8000640 1 --2- 172.21.15.122:0/3648913802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43b006d330 0x7f43b00caef0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.443+0000 7f43b8000640 1 --2- 172.21.15.122:0/3648913802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43b015ddd0 0x7f43b00c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.443+0000 7f43b8000640 1 -- 172.21.15.122:0/3648913802 >> 172.21.15.122:0/3648913802 conn(0x7f43b00cce90 msgr2=0x7f43b00cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.443+0000 7f43b8000640 1 -- 172.21.15.122:0/3648913802 shutdown_connections 2024-09-17T05:38:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.443+0000 7f43b8000640 1 -- 172.21.15.122:0/3648913802 wait complete. 2024-09-17T05:38:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 Processor -- start 2024-09-17T05:38:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 -- start start 2024-09-17T05:38:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43b006d330 0x7f43b00b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43b0073980 0x7f43b00b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b015ddd0 0x7f43b00c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43b016ab40 con 0x7f43b015ddd0 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f43b016a9c0 con 0x7f43b006d330 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f43b016acc0 con 0x7f43b0073980 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b015ddd0 0x7f43b00c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b015ddd0 0x7f43b00c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50104/0 (socket says 172.21.15.122:50104) 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 -- 172.21.15.122:0/3973480459 learned_addr learned my addr 172.21.15.122:0/3973480459 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b5600640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43b006d330 0x7f43b00b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b4c00640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43b0073980 0x7f43b00b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 -- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43b0073980 msgr2=0x7f43b00b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43b0073980 0x7f43b00b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 -- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43b006d330 msgr2=0x7f43b00b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43b006d330 0x7f43b00b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 -- 172.21.15.122:0/3973480459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b00ca0b0 con 0x7f43b015ddd0 2024-09-17T05:38:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b6000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b015ddd0 0x7f43b00c9b70 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009bd0 tx=0x7f43a80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43ad600640 1 -- 172.21.15.122:0/3973480459 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80bb070 con 0x7f43b015ddd0 2024-09-17T05:38:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43ad600640 1 -- 172.21.15.122:0/3973480459 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a8004d40 con 0x7f43b015ddd0 2024-09-17T05:38:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b00b5dc0 con 0x7f43b015ddd0 2024-09-17T05:38:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43ad600640 1 -- 172.21.15.122:0/3973480459 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80b2040 con 0x7f43b015ddd0 2024-09-17T05:38:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b00b6380 con 0x7f43b015ddd0 2024-09-17T05:38:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.447+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b00caef0 con 0x7f43b015ddd0 2024-09-17T05:38:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.451+0000 7f43ad600640 1 -- 172.21.15.122:0/3973480459 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7f43a80ad850 con 0x7f43b015ddd0 2024-09-17T05:38:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.451+0000 7f43ad600640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4390075060 0x7f4390077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.451+0000 7f43ad600640 1 -- 172.21.15.122:0/3973480459 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f43a812d200 con 0x7f43b015ddd0 2024-09-17T05:38:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.451+0000 7f43b5600640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4390075060 0x7f4390077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.451+0000 7f43b5600640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4390075060 0x7f4390077520 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009b40 tx=0x7f43a40091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:33.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.455+0000 7f43ad600640 1 -- 172.21.15.122:0/3973480459 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f43a8016990 con 0x7f43b015ddd0 2024-09-17T05:38:33.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:33.655+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone291", "target": ["mon-mgr", ""]}) -- 0x7f43b0068fd0 con 0x7f4390075060 2024-09-17T05:38:34.029 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:38:34.033 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:38:35.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.347+0000 7f43ad600640 1 -- 172.21.15.122:0/3973480459 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f43b0068fd0 con 0x7f4390075060 2024-09-17T05:38:35.343 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:38:35.343 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:38:35.343 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:35.344 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4390075060 msgr2=0x7f4390077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4390075060 0x7f4390077520 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009b40 tx=0x7f43a40091e0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b015ddd0 msgr2=0x7f43b00c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b015ddd0 0x7f43b00c9b70 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009bd0 tx=0x7f43a80092e0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 shutdown_connections 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4390075060 0x7f4390077520 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b015ddd0 0x7f43b00c9b70 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43b0073980 0x7f43b00b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 --2- 172.21.15.122:0/3973480459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43b006d330 0x7f43b00b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 >> 172.21.15.122:0/3973480459 conn(0x7f43b00cce90 msgr2=0x7f43b015ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 shutdown_connections 2024-09-17T05:38:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.351+0000 7f43b8000640 1 -- 172.21.15.122:0/3973480459 wait complete. 2024-09-17T05:38:35.370 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:38:35.370 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone78512 2024-09-17T05:38:35.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- 172.21.15.122:0/1942276855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 msgr2=0x7fdf70003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:35.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 --2- 172.21.15.122:0/1942276855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 0x7fdf70003e80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c640 tx=0x7fdf7c01d2a0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf8a800640 1 -- 172.21.15.122:0/1942276855 reap_dead start 2024-09-17T05:38:35.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- 172.21.15.122:0/1942276855 shutdown_connections 2024-09-17T05:38:35.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- 172.21.15.122:0/1942276855 >> 172.21.15.122:0/1942276855 conn(0x7fdf8c0d1dd0 msgr2=0x7fdf8c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:35.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- 172.21.15.122:0/1942276855 shutdown_connections 2024-09-17T05:38:35.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- 172.21.15.122:0/1942276855 wait complete. 2024-09-17T05:38:35.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 Processor -- start 2024-09-17T05:38:35.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- start start 2024-09-17T05:38:35.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 0x7fdf8c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:35.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c1f88d0 0x7fdf8c14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:35.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c15f4c0 0x7fdf8c071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:35.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c166d20 con 0x7fdf70003a60 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c0b1430 con 0x7fdf8c1f88d0 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf90a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8c166ea0 con 0x7fdf8c15f4c0 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf89e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 0x7fdf8c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf89e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 0x7fdf8c0cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50132/0 (socket says 172.21.15.122:50132) 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf89e00640 1 -- 172.21.15.122:0/3323048131 learned_addr learned my addr 172.21.15.122:0/3323048131 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf8a800640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c15f4c0 0x7fdf8c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf81400640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c1f88d0 0x7fdf8c14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf89e00640 1 -- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c15f4c0 msgr2=0x7fdf8c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.527+0000 7fdf89e00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c15f4c0 0x7fdf8c071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf89e00640 1 -- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c1f88d0 msgr2=0x7fdf8c14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf89e00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c1f88d0 0x7fdf8c14aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:35.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf89e00640 1 -- 172.21.15.122:0/3323048131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c14b7f0 con 0x7fdf70003a60 2024-09-17T05:38:35.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf89e00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 0x7fdf8c0cbb80 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c01cd60 tx=0x7fdf7c001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:35.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf82a00640 1 -- 172.21.15.122:0/3323048131 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c003b80 con 0x7fdf70003a60 2024-09-17T05:38:35.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf82a00640 1 -- 172.21.15.122:0/3323048131 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf7c003e10 con 0x7fdf70003a60 2024-09-17T05:38:35.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf82a00640 1 -- 172.21.15.122:0/3323048131 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c006330 con 0x7fdf70003a60 2024-09-17T05:38:35.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8c14bad0 con 0x7fdf70003a60 2024-09-17T05:38:35.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8c15ecc0 con 0x7fdf70003a60 2024-09-17T05:38:35.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8c0b1a30 con 0x7fdf70003a60 2024-09-17T05:38:35.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.531+0000 7fdf82a00640 1 -- 172.21.15.122:0/3323048131 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7fdf7c0064d0 con 0x7fdf70003a60 2024-09-17T05:38:35.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.535+0000 7fdf82a00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf68075090 0x7fdf68077550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:35.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.535+0000 7fdf82a00640 1 -- 172.21.15.122:0/3323048131 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fdf7c12da10 con 0x7fdf70003a60 2024-09-17T05:38:35.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.535+0000 7fdf81400640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf68075090 0x7fdf68077550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:35.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.535+0000 7fdf81400640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf68075090 0x7fdf68077550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c1601d0 tx=0x7fdf740091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:35.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.539+0000 7fdf82a00640 1 -- 172.21.15.122:0/3323048131 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdf7c0c0060 con 0x7fdf70003a60 2024-09-17T05:38:35.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:35.743+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone78512", "target": ["mon-mgr", ""]}) -- 0x7fdf8c0d1080 con 0x7fdf68075090 2024-09-17T05:38:38.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.263+0000 7fdf82a00640 1 -- 172.21.15.122:0/3323048131 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fdf8c0d1080 con 0x7fdf68075090 2024-09-17T05:38:38.259 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:38:38.259 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:38.260 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:38:38.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf68075090 msgr2=0x7fdf68077550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf68075090 0x7fdf68077550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c1601d0 tx=0x7fdf740091e0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 msgr2=0x7fdf8c0cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 0x7fdf8c0cbb80 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c01cd60 tx=0x7fdf7c001fe0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 shutdown_connections 2024-09-17T05:38:38.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdf68075090 0x7fdf68077550 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c15f4c0 0x7fdf8c071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c1f88d0 0x7fdf8c14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 --2- 172.21.15.122:0/3323048131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf70003a60 0x7fdf8c0cbb80 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 >> 172.21.15.122:0/3323048131 conn(0x7fdf8c0d1dd0 msgr2=0x7fdf8c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:38.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 shutdown_connections 2024-09-17T05:38:38.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.267+0000 7fdf90a00640 1 -- 172.21.15.122:0/3323048131 wait complete. 2024-09-17T05:38:38.286 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:38:38.286 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:38:38.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.435+0000 7f2319e00640 1 -- 172.21.15.122:0/3266267425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 msgr2=0x7f23140d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.435+0000 7f2319e00640 1 --2- 172.21.15.122:0/3266267425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 0x7f23140d0260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2304009840 tx=0x7f230401c790 comp rx=0 tx=0).stop 2024-09-17T05:38:38.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.435+0000 7f2319e00640 1 -- 172.21.15.122:0/3266267425 shutdown_connections 2024-09-17T05:38:38.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.435+0000 7f2319e00640 1 --2- 172.21.15.122:0/3266267425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23140b2d00 0x7f231415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.435+0000 7f2319e00640 1 --2- 172.21.15.122:0/3266267425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 0x7f23140d0260 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.435+0000 7f2319e00640 1 -- 172.21.15.122:0/3266267425 >> 172.21.15.122:0/3266267425 conn(0x7f23140d1e40 msgr2=0x7f23140d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:38.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.435+0000 7f2319e00640 1 -- 172.21.15.122:0/3266267425 shutdown_connections 2024-09-17T05:38:38.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.435+0000 7f2319e00640 1 -- 172.21.15.122:0/3266267425 wait complete. 2024-09-17T05:38:38.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 Processor -- start 2024-09-17T05:38:38.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 -- start start 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23140b2d00 0x7f231415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 0x7f2314151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314160400 0x7f231415cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2314169520 con 0x7f23140cfe80 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23141693a0 con 0x7f23140b2d00 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f23141696a0 con 0x7f2314160400 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 0x7f2314151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 0x7f2314151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50148/0 (socket says 172.21.15.122:50148) 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 -- 172.21.15.122:0/4193136580 learned_addr learned my addr 172.21.15.122:0/4193136580 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2312a00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23140b2d00 0x7f231415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:38.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2313400640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314160400 0x7f231415cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 -- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314160400 msgr2=0x7f231415cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314160400 0x7f231415cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 -- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23140b2d00 msgr2=0x7f231415ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23140b2d00 0x7f231415ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 -- 172.21.15.122:0/4193136580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2314159a70 con 0x7f23140cfe80 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2312a00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23140b2d00 0x7f231415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230a000640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 0x7f2314151fa0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9f0 tx=0x7f22fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230be00640 1 -- 172.21.15.122:0/4193136580 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc00cd80 con 0x7f23140cfe80 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230be00640 1 -- 172.21.15.122:0/4193136580 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22fc010430 con 0x7f23140cfe80 2024-09-17T05:38:38.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f230be00640 1 -- 172.21.15.122:0/4193136580 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc010860 con 0x7f23140cfe80 2024-09-17T05:38:38.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2314159d50 con 0x7f23140cfe80 2024-09-17T05:38:38.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231415a310 con 0x7f23140cfe80 2024-09-17T05:38:38.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.439+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231415fec0 con 0x7f23140cfe80 2024-09-17T05:38:38.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.443+0000 7f230be00640 1 -- 172.21.15.122:0/4193136580 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7f22fc010a00 con 0x7f23140cfe80 2024-09-17T05:38:38.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.443+0000 7f230be00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f0075090 0x7f22f0077550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:38.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.443+0000 7f230be00640 1 -- 172.21.15.122:0/4193136580 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f22fc093d50 con 0x7f23140cfe80 2024-09-17T05:38:38.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.443+0000 7f2312a00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f0075090 0x7f22f0077550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:38.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.443+0000 7f2312a00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f0075090 0x7f22f0077550 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f23040ae000 tx=0x7f230401afd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:38.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.447+0000 7f230be00640 1 -- 172.21.15.122:0/4193136580 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f22fc060920 con 0x7f23140cfe80 2024-09-17T05:38:38.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2314153960 con 0x7f23140cfe80 2024-09-17T05:38:38.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f230be00640 1 -- 172.21.15.122:0/4193136580 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1921 (secure 0 0 0) 0x7f22fc060220 con 0x7f23140cfe80 2024-09-17T05:38:38.669 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:38.669 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:38:38.670 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 839, 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:38:38.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 119, 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:38:38.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 1622, 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 6670219701, 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 14436032512, 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 758658080768, 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:38:38.673 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 292102550, 2024-09-17T05:38:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 288303663, 2024-09-17T05:38:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 72, 2024-09-17T05:38:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 83 2024-09-17T05:38:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:38:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 77, 2024-09-17T05:38:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:37:50:792793+0000", 2024-09-17T05:38:38.674 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 8, 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 8, 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "a", 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 8364 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:38:38.675 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:38:38.676 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 17, 2024-09-17T05:38:38.677 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:37:30.710278+0000", 2024-09-17T05:38:38.677 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:38:38.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:38.677 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": {} 2024-09-17T05:38:38.677 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f0075090 msgr2=0x7f22f0077550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f0075090 0x7f22f0077550 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f23040ae000 tx=0x7f230401afd0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 msgr2=0x7f2314151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 0x7f2314151fa0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9f0 tx=0x7f22fc00eeb0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 shutdown_connections 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22f0075090 0x7f22f0077550 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314160400 0x7f231415cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23140cfe80 0x7f2314151fa0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 --2- 172.21.15.122:0/4193136580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23140b2d00 0x7f231415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 >> 172.21.15.122:0/4193136580 conn(0x7f23140d1e40 msgr2=0x7f231415bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:38.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 shutdown_connections 2024-09-17T05:38:38.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.675+0000 7f2319e00640 1 -- 172.21.15.122:0/4193136580 wait complete. 2024-09-17T05:38:38.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:38:38.691 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone1010/f62baf42-9540-4e09-96e0-039b5003c455 2024-09-17T05:38:38.691 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:38:38.701 INFO:teuthology.orchestra.run.smithi122.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone1010/f62baf42-9540-4e09-96e0-039b5003c455': No such file or directory 2024-09-17T05:38:38.702 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T05:38:38.702 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:38:38.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- 172.21.15.122:0/385076351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 msgr2=0x7fe51015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 --2- 172.21.15.122:0/385076351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 0x7fe51015c7c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe500009870 tx=0x7fe50001f010 comp rx=0 tx=0).stop 2024-09-17T05:38:38.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- 172.21.15.122:0/385076351 shutdown_connections 2024-09-17T05:38:38.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 --2- 172.21.15.122:0/385076351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 0x7fe51015c7c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- 172.21.15.122:0/385076351 >> 172.21.15.122:0/385076351 conn(0x7fe5100b18f0 msgr2=0x7fe5100b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:38.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- 172.21.15.122:0/385076351 shutdown_connections 2024-09-17T05:38:38.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- 172.21.15.122:0/385076351 wait complete. 2024-09-17T05:38:38.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 Processor -- start 2024-09-17T05:38:38.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- start start 2024-09-17T05:38:38.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe51006e7a0 0x7fe510157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:38.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe51015bfa0 0x7fe510153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 0x7fe5101f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe51006f340 con 0x7fe51015c3e0 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe510074980 con 0x7fe51015bfa0 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe515000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe51006f4c0 con 0x7fe51006e7a0 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe50e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 0x7fe5101f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe50e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 0x7fe5101f5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50168/0 (socket says 172.21.15.122:50168) 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe50e800640 1 -- 172.21.15.122:0/1623137000 learned_addr learned my addr 172.21.15.122:0/1623137000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe50de00640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe51006e7a0 0x7fe510157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.899+0000 7fe507e00640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe51015bfa0 0x7fe510153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe50e800640 1 -- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe51006e7a0 msgr2=0x7fe510157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe50e800640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe51006e7a0 0x7fe510157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe50e800640 1 -- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe51015bfa0 msgr2=0x7fe510153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:38.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe50e800640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe51015bfa0 0x7fe510153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:38.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe50e800640 1 -- 172.21.15.122:0/1623137000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5101578a0 con 0x7fe51015c3e0 2024-09-17T05:38:38.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe50e800640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 0x7fe5101f5dd0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c730 tx=0x7fe50800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:38.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe506000640 1 -- 172.21.15.122:0/1623137000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508003db0 con 0x7fe51015c3e0 2024-09-17T05:38:38.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe506000640 1 -- 172.21.15.122:0/1623137000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe50800ce40 con 0x7fe51015c3e0 2024-09-17T05:38:38.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5101f36a0 con 0x7fe51015c3e0 2024-09-17T05:38:38.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe506000640 1 -- 172.21.15.122:0/1623137000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5080052d0 con 0x7fe51015c3e0 2024-09-17T05:38:38.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe510153470 con 0x7fe51015c3e0 2024-09-17T05:38:38.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51006ec30 con 0x7fe51015c3e0 2024-09-17T05:38:38.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.903+0000 7fe506000640 1 -- 172.21.15.122:0/1623137000 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7fe508005470 con 0x7fe51015c3e0 2024-09-17T05:38:38.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.907+0000 7fe506000640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4ec075060 0x7fe4ec077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:38.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.907+0000 7fe506000640 1 -- 172.21.15.122:0/1623137000 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fe5080939f0 con 0x7fe51015c3e0 2024-09-17T05:38:38.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.907+0000 7fe50de00640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4ec075060 0x7fe4ec077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.907+0000 7fe50de00640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4ec075060 0x7fe4ec077520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe5000b2000 tx=0x7fe50001d850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:38.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:38.907+0000 7fe506000640 1 -- 172.21.15.122:0/1623137000 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe508093c70 con 0x7fe51015c3e0 2024-09-17T05:38:39.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe51015cb40 con 0x7fe4ec075060 2024-09-17T05:38:39.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe506000640 1 -- 172.21.15.122:0/1623137000 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe51015cb40 con 0x7fe4ec075060 2024-09-17T05:38:39.093 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:38:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:38:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:38:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:39.094 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4ec075060 msgr2=0x7fe4ec077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4ec075060 0x7fe4ec077520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe5000b2000 tx=0x7fe50001d850 comp rx=0 tx=0).stop 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 msgr2=0x7fe5101f5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 0x7fe5101f5dd0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c730 tx=0x7fe50800cbf0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 shutdown_connections 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe4ec075060 0x7fe4ec077520 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe51015c3e0 0x7fe5101f5dd0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe51015bfa0 0x7fe510153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 --2- 172.21.15.122:0/1623137000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe51006e7a0 0x7fe510157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.099+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 >> 172.21.15.122:0/1623137000 conn(0x7fe5100b18f0 msgr2=0x7fe5100b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.103+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 shutdown_connections 2024-09-17T05:38:39.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.103+0000 7fe515000640 1 -- 172.21.15.122:0/1623137000 wait complete. 2024-09-17T05:38:39.115 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:38:39.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.263+0000 7ff0e3200640 1 -- 172.21.15.122:0/3531348689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 msgr2=0x7ff0dc06ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.263+0000 7ff0e3200640 1 --2- 172.21.15.122:0/3531348689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 0x7ff0dc06ec50 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009900 tx=0x7ff0cc01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.263+0000 7ff0e3200640 1 -- 172.21.15.122:0/3531348689 shutdown_connections 2024-09-17T05:38:39.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.263+0000 7ff0e3200640 1 --2- 172.21.15.122:0/3531348689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 0x7ff0dc06ec50 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.263+0000 7ff0e3200640 1 -- 172.21.15.122:0/3531348689 >> 172.21.15.122:0/3531348689 conn(0x7ff0dc0d1e00 msgr2=0x7ff0dc0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:39.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.263+0000 7ff0e3200640 1 -- 172.21.15.122:0/3531348689 shutdown_connections 2024-09-17T05:38:39.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.263+0000 7ff0e3200640 1 -- 172.21.15.122:0/3531348689 wait complete. 2024-09-17T05:38:39.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 Processor -- start 2024-09-17T05:38:39.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 -- start start 2024-09-17T05:38:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0dc06dd30 0x7ff0dc15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 0x7ff0dc0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0dc0d0c30 0x7ff0dc15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc16c070 con 0x7ff0dc0d0140 2024-09-17T05:38:39.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc16bef0 con 0x7ff0dc0d0c30 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff0dc16c1f0 con 0x7ff0dc06dd30 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 0x7ff0dc0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 0x7ff0dc0c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50184/0 (socket says 172.21.15.122:50184) 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 -- 172.21.15.122:0/3962468183 learned_addr learned my addr 172.21.15.122:0/3962468183 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e1200640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0dc0d0c30 0x7ff0dc15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0dbe00640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0dc06dd30 0x7ff0dc15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 -- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0dc06dd30 msgr2=0x7ff0dc15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0dc06dd30 0x7ff0dc15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 -- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0dc0d0c30 msgr2=0x7ff0dc15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0dc0d0c30 0x7ff0dc15d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 -- 172.21.15.122:0/3962468183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc0c8590 con 0x7ff0dc0d0140 2024-09-17T05:38:39.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d3400640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 0x7ff0dc0c7e40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c490 tx=0x7ff0c4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:39.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d8c00640 1 -- 172.21.15.122:0/3962468183 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4017070 con 0x7ff0dc0d0140 2024-09-17T05:38:39.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d8c00640 1 -- 172.21.15.122:0/3962468183 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0c4003600 con 0x7ff0dc0d0140 2024-09-17T05:38:39.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0d8c00640 1 -- 172.21.15.122:0/3962468183 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4003a90 con 0x7ff0dc0d0140 2024-09-17T05:38:39.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc0cc680 con 0x7ff0dc0d0140 2024-09-17T05:38:39.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.267+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc0ccc40 con 0x7ff0dc0d0140 2024-09-17T05:38:39.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.271+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc06ec50 con 0x7ff0dc0d0140 2024-09-17T05:38:39.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.271+0000 7ff0d8c00640 1 -- 172.21.15.122:0/3962468183 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7ff0c401a070 con 0x7ff0dc0d0140 2024-09-17T05:38:39.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.271+0000 7ff0d8c00640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0b8075060 0x7ff0b8077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:39.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.271+0000 7ff0d8c00640 1 -- 172.21.15.122:0/3962468183 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7ff0c4092580 con 0x7ff0dc0d0140 2024-09-17T05:38:39.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.271+0000 7ff0dbe00640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0b8075060 0x7ff0b8077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:39.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.271+0000 7ff0dbe00640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0b8075060 0x7ff0b8077520 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0af000 tx=0x7ff0cc01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:39.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.275+0000 7ff0d8c00640 1 -- 172.21.15.122:0/3962468183 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff0c405f110 con 0x7ff0dc0d0140 2024-09-17T05:38:39.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.467+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0dc1570d0 con 0x7ff0b8075060 2024-09-17T05:38:39.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.475+0000 7ff0d8c00640 1 -- 172.21.15.122:0/3962468183 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+182 (secure 0 0 0) 0x7ff0dc1570d0 con 0x7ff0b8075060 2024-09-17T05:38:39.470 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:39.470 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:38:39.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:38:39.470 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sv1" 2024-09-17T05:38:39.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:39.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone78512" 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone291" 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone1010" 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:39.471 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:38:39.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0b8075060 msgr2=0x7ff0b8077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0b8075060 0x7ff0b8077520 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0af000 tx=0x7ff0cc01b120 comp rx=0 tx=0).stop 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 msgr2=0x7ff0dc0c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 0x7ff0dc0c7e40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c490 tx=0x7ff0c4001e80 comp rx=0 tx=0).stop 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 shutdown_connections 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0b8075060 0x7ff0b8077520 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0dc0d0c30 0x7ff0dc15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0dc0d0140 0x7ff0dc0c7e40 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 --2- 172.21.15.122:0/3962468183 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0dc06dd30 0x7ff0dc15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 >> 172.21.15.122:0/3962468183 conn(0x7ff0dc0d1e00 msgr2=0x7ff0dc0b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 shutdown_connections 2024-09-17T05:38:39.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.479+0000 7ff0e3200640 1 -- 172.21.15.122:0/3962468183 wait complete. 2024-09-17T05:38:39.492 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T05:38:39.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 -- 172.21.15.122:0/2185398282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c0cff00 msgr2=0x7f8c1c0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 --2- 172.21.15.122:0/2185398282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c0cff00 0x7f8c1c0d28c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c069e10 tx=0x7f8c0c01c890 comp rx=0 tx=0).stop 2024-09-17T05:38:39.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 -- 172.21.15.122:0/2185398282 shutdown_connections 2024-09-17T05:38:39.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 --2- 172.21.15.122:0/2185398282 >> v2:172.21.15.173:3300/0 conn(0x7f8c1c15f700 0x7f8c1c0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 --2- 172.21.15.122:0/2185398282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c0cff00 0x7f8c1c0d28c0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 -- 172.21.15.122:0/2185398282 >> 172.21.15.122:0/2185398282 conn(0x7f8c1c0d1da0 msgr2=0x7f8c1c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:39.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 -- 172.21.15.122:0/2185398282 shutdown_connections 2024-09-17T05:38:39.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 -- 172.21.15.122:0/2185398282 wait complete. 2024-09-17T05:38:39.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 Processor -- start 2024-09-17T05:38:39.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.643+0000 7f8c20c00640 1 -- start start 2024-09-17T05:38:39.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c0cff00 0x7f8c1c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c0d02e0 0x7f8c1c0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c15f700 0x7f8c1c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c0b0b80 con 0x7f8c1c15f700 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c0b0a00 con 0x7f8c1c0cff00 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1c0b0d00 con 0x7f8c1c0d02e0 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c15f700 0x7f8c1c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c15f700 0x7f8c1c0ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50196/0 (socket says 172.21.15.122:50196) 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 -- 172.21.15.122:0/1039106009 learned_addr learned my addr 172.21.15.122:0/1039106009 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1b400640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c0cff00 0x7f8c1c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1aa00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c0d02e0 0x7f8c1c0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 -- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c0d02e0 msgr2=0x7f8c1c0c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c0d02e0 0x7f8c1c0c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 -- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c0cff00 msgr2=0x7f8c1c152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c0cff00 0x7f8c1c152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 -- 172.21.15.122:0/1039106009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c147290 con 0x7f8c1c15f700 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c1be00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c15f700 0x7f8c1c0ca070 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400e9a0 tx=0x7f8c1400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c13400640 1 -- 172.21.15.122:0/1039106009 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14004d30 con 0x7f8c1c15f700 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c13400640 1 -- 172.21.15.122:0/1039106009 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c1400cb20 con 0x7f8c1c15f700 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c13400640 1 -- 172.21.15.122:0/1039106009 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14005020 con 0x7f8c1c15f700 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c0c9010 con 0x7f8c1c15f700 2024-09-17T05:38:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c1501a0 con 0x7f8c1c15f700 2024-09-17T05:38:39.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.647+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c0d28c0 con 0x7f8c1c15f700 2024-09-17T05:38:39.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.651+0000 7f8c13400640 1 -- 172.21.15.122:0/1039106009 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7f8c140051c0 con 0x7f8c1c15f700 2024-09-17T05:38:39.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.651+0000 7f8c13400640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bfc075060 0x7f8bfc077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.651+0000 7f8c13400640 1 -- 172.21.15.122:0/1039106009 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f8c14093c10 con 0x7f8c1c15f700 2024-09-17T05:38:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.651+0000 7f8c1b400640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bfc075060 0x7f8bfc077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.651+0000 7f8c1b400640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bfc075060 0x7f8bfc077520 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0af000 tx=0x7f8c0c01b0d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:39.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.655+0000 7f8c13400640 1 -- 172.21.15.122:0/1039106009 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8c14016030 con 0x7f8c1c15f700 2024-09-17T05:38:39.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.851+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c1c14f8c0 con 0x7f8bfc075060 2024-09-17T05:38:39.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c13400640 1 -- 172.21.15.122:0/1039106009 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8c1c14f8c0 con 0x7f8bfc075060 2024-09-17T05:38:39.864 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:39.864 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:38:39.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:38:39.864 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1" 2024-09-17T05:38:39.864 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:38:39.864 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:38:39.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bfc075060 msgr2=0x7f8bfc077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bfc075060 0x7f8bfc077520 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0af000 tx=0x7f8c0c01b0d0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c15f700 msgr2=0x7f8c1c0ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:39.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c15f700 0x7f8c1c0ca070 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400e9a0 tx=0x7f8c1400ee60 comp rx=0 tx=0).stop 2024-09-17T05:38:39.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 shutdown_connections 2024-09-17T05:38:39.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8bfc075060 0x7f8bfc077520 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1c15f700 0x7f8c1c0ca070 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c0d02e0 0x7f8c1c0c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 --2- 172.21.15.122:0/1039106009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c0cff00 0x7f8c1c152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:39.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 >> 172.21.15.122:0/1039106009 conn(0x7f8c1c0d1da0 msgr2=0x7f8c1c146a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:39.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 shutdown_connections 2024-09-17T05:38:39.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:39.871+0000 7f8c20c00640 1 -- 172.21.15.122:0/1039106009 wait complete. 2024-09-17T05:38:39.886 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T05:38:40.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- 172.21.15.122:0/1515865118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 msgr2=0x7fb9080acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 --2- 172.21.15.122:0/1515865118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 0x7fb9080acd40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb900009bc0 tx=0x7fb90001cb10 comp rx=0 tx=0).stop 2024-09-17T05:38:40.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- 172.21.15.122:0/1515865118 shutdown_connections 2024-09-17T05:38:40.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 --2- 172.21.15.122:0/1515865118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 0x7fb9080acd40 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 --2- 172.21.15.122:0/1515865118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9080cd100 0x7fb90806f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- 172.21.15.122:0/1515865118 >> 172.21.15.122:0/1515865118 conn(0x7fb9080b18f0 msgr2=0x7fb9080b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:40.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- 172.21.15.122:0/1515865118 shutdown_connections 2024-09-17T05:38:40.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- 172.21.15.122:0/1515865118 wait complete. 2024-09-17T05:38:40.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 Processor -- start 2024-09-17T05:38:40.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- start start 2024-09-17T05:38:40.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 0x7fb908156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9080cd100 0x7fb90806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90815dae0 0x7fb90814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9080cf100 con 0x7fb9080b49e0 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb9080cef80 con 0x7fb90815dae0 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.039+0000 7fb910200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb9080cf280 con 0x7fb9080cd100 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 0x7fb908156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 0x7fb908156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50218/0 (socket says 172.21.15.122:50218) 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 -- 172.21.15.122:0/718438177 learned_addr learned my addr 172.21.15.122:0/718438177 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90ce00640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9080cd100 0x7fb90806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90e200640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90815dae0 0x7fb90814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 -- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9080cd100 msgr2=0x7fb90806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9080cd100 0x7fb90806fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 -- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90815dae0 msgr2=0x7fb90814f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90815dae0 0x7fb90814f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 -- 172.21.15.122:0/718438177 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb908073030 con 0x7fb9080b49e0 2024-09-17T05:38:40.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb90d800640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 0x7fb908156280 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00c530 tx=0x7fb8fc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:40.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb905600640 1 -- 172.21.15.122:0/718438177 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc017070 con 0x7fb9080b49e0 2024-09-17T05:38:40.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb905600640 1 -- 172.21.15.122:0/718438177 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc003600 con 0x7fb9080b49e0 2024-09-17T05:38:40.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9080739c0 con 0x7fb9080b49e0 2024-09-17T05:38:40.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb905600640 1 -- 172.21.15.122:0/718438177 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc003a50 con 0x7fb9080b49e0 2024-09-17T05:38:40.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9081567c0 con 0x7fb9080b49e0 2024-09-17T05:38:40.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb908069410 con 0x7fb9080b49e0 2024-09-17T05:38:40.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.043+0000 7fb905600640 1 -- 172.21.15.122:0/718438177 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7fb8fc01a070 con 0x7fb9080b49e0 2024-09-17T05:38:40.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.047+0000 7fb905600640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8e8075060 0x7fb8e8077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.047+0000 7fb905600640 1 -- 172.21.15.122:0/718438177 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fb8fc0925e0 con 0x7fb9080b49e0 2024-09-17T05:38:40.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.047+0000 7fb90ce00640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8e8075060 0x7fb8e8077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.047+0000 7fb90ce00640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8e8075060 0x7fb8e8077520 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb9081543f0 tx=0x7fb8f400b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:40.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.051+0000 7fb905600640 1 -- 172.21.15.122:0/718438177 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb8fc05eb60 con 0x7fb9080b49e0 2024-09-17T05:38:40.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.247+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9081573a0 con 0x7fb8e8075060 2024-09-17T05:38:40.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.279+0000 7fb905600640 1 -- 172.21.15.122:0/718438177 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb9081573a0 con 0x7fb8e8075060 2024-09-17T05:38:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8e8075060 msgr2=0x7fb8e8077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8e8075060 0x7fb8e8077520 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb9081543f0 tx=0x7fb8f400b390 comp rx=0 tx=0).stop 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 msgr2=0x7fb908156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 0x7fb908156280 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00c530 tx=0x7fb8fc00c9f0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 shutdown_connections 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8e8075060 0x7fb8e8077520 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb90815dae0 0x7fb90814f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9080cd100 0x7fb90806fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 --2- 172.21.15.122:0/718438177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9080b49e0 0x7fb908156280 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 >> 172.21.15.122:0/718438177 conn(0x7fb9080b18f0 msgr2=0x7fb9080aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:40.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 shutdown_connections 2024-09-17T05:38:40.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.283+0000 7fb910200640 1 -- 172.21.15.122:0/718438177 wait complete. 2024-09-17T05:38:40.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T05:38:40.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 -- 172.21.15.122:0/3273362151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c15bd40 msgr2=0x7f670c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 --2- 172.21.15.122:0/3273362151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c15bd40 0x7f670c0d0240 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f670c0cfbf0 tx=0x7f670001c5a0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 -- 172.21.15.122:0/3273362151 shutdown_connections 2024-09-17T05:38:40.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 --2- 172.21.15.122:0/3273362151 >> v2:172.21.15.173:3300/0 conn(0x7f670c0d20f0 0x7f670c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 --2- 172.21.15.122:0/3273362151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f670c0b0d10 0x7f670c0d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 --2- 172.21.15.122:0/3273362151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c15bd40 0x7f670c0d0240 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 -- 172.21.15.122:0/3273362151 >> 172.21.15.122:0/3273362151 conn(0x7f670c09aa10 msgr2=0x7f670c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:40.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 -- 172.21.15.122:0/3273362151 shutdown_connections 2024-09-17T05:38:40.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 -- 172.21.15.122:0/3273362151 wait complete. 2024-09-17T05:38:40.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 Processor -- start 2024-09-17T05:38:40.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.447+0000 7f6712800640 1 -- start start 2024-09-17T05:38:40.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f670c0b0d10 0x7f670c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c0d20f0 0x7f670c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670c15bd40 0x7f670c0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f670c0ccdc0 con 0x7f670c0d20f0 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f670c0ccc40 con 0x7f670c15bd40 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f670c0ccf40 con 0x7f670c0b0d10 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c0d20f0 0x7f670c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c0d20f0 0x7f670c15ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50234/0 (socket says 172.21.15.122:50234) 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 -- 172.21.15.122:0/930509332 learned_addr learned my addr 172.21.15.122:0/930509332 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6711e00640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670c15bd40 0x7f670c0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6711400640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f670c0b0d10 0x7f670c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 -- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f670c0b0d10 msgr2=0x7f670c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f670c0b0d10 0x7f670c1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 -- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670c15bd40 msgr2=0x7f670c0b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670c15bd40 0x7f670c0b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 -- 172.21.15.122:0/930509332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c0c9f90 con 0x7f670c0d20f0 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6710a00640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c0d20f0 0x7f670c15ccb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f66f800d750 tx=0x7f66f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6709600640 1 -- 172.21.15.122:0/930509332 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8017070 con 0x7f670c0d20f0 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6709600640 1 -- 172.21.15.122:0/930509332 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66f8009d20 con 0x7f670c0d20f0 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6709600640 1 -- 172.21.15.122:0/930509332 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f800de60 con 0x7f670c0d20f0 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c0ca270 con 0x7f670c0d20f0 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c1f7870 con 0x7f670c0d20f0 2024-09-17T05:38:40.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.451+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c071d60 con 0x7f670c0d20f0 2024-09-17T05:38:40.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.455+0000 7f6709600640 1 -- 172.21.15.122:0/930509332 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7f66f8005280 con 0x7f670c0d20f0 2024-09-17T05:38:40.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.455+0000 7f6709600640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66ec075060 0x7f66ec077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.455+0000 7f6709600640 1 -- 172.21.15.122:0/930509332 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f66f8093310 con 0x7f670c0d20f0 2024-09-17T05:38:40.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.455+0000 7f6711400640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66ec075060 0x7f66ec077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.455+0000 7f6711400640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66ec075060 0x7f66ec077520 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f67000ae000 tx=0x7f670001ae40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:40.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.459+0000 7f6709600640 1 -- 172.21.15.122:0/930509332 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f66f805f7d0 con 0x7f670c0d20f0 2024-09-17T05:38:40.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.651+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f670c069e30 con 0x7f66ec075060 2024-09-17T05:38:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6709600640 1 -- 172.21.15.122:0/930509332 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f67000af040 con 0x7f66ec075060 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66ec075060 msgr2=0x7f66ec077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66ec075060 0x7f66ec077520 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f67000ae000 tx=0x7f670001ae40 comp rx=0 tx=0).stop 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c0d20f0 msgr2=0x7f670c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c0d20f0 0x7f670c15ccb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f66f800d750 tx=0x7f66f800dc10 comp rx=0 tx=0).stop 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 shutdown_connections 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66ec075060 0x7f66ec077520 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f670c15bd40 0x7f670c0b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f670c0d20f0 0x7f670c15ccb0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 --2- 172.21.15.122:0/930509332 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f670c0b0d10 0x7f670c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 >> 172.21.15.122:0/930509332 conn(0x7f670c09aa10 msgr2=0x7f670c0aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 shutdown_connections 2024-09-17T05:38:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.675+0000 7f6712800640 1 -- 172.21.15.122:0/930509332 wait complete. 2024-09-17T05:38:40.682 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:40.686+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:40.682 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:40.686+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:40.682 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:40.686+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:40.683 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:40.686+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:40.683 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:40.686+0000 7fc09b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:40.693 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone78512 2024-09-17T05:38:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.847+0000 7fbe59000640 1 -- 172.21.15.122:0/3887456347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe5415eff0 msgr2=0x7fbe5415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.847+0000 7fbe59000640 1 --2- 172.21.15.122:0/3887456347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe5415eff0 0x7fbe5415f4a0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fbe54073b80 tx=0x7fbe4c01cd80 comp rx=0 tx=0).stop 2024-09-17T05:38:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- 172.21.15.122:0/3887456347 shutdown_connections 2024-09-17T05:38:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 --2- 172.21.15.122:0/3887456347 >> v2:172.21.15.173:3300/0 conn(0x7fbe540d02e0 0x7fbe540d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 --2- 172.21.15.122:0/3887456347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe5415eff0 0x7fbe5415f4a0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 --2- 172.21.15.122:0/3887456347 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe540cff00 0x7fbe540d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- 172.21.15.122:0/3887456347 >> 172.21.15.122:0/3887456347 conn(0x7fbe540d1da0 msgr2=0x7fbe540d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- 172.21.15.122:0/3887456347 shutdown_connections 2024-09-17T05:38:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- 172.21.15.122:0/3887456347 wait complete. 2024-09-17T05:38:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 Processor -- start 2024-09-17T05:38:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- start start 2024-09-17T05:38:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe540cff00 0x7fbe540ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe540d02e0 0x7fbe541fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe5415eff0 0x7fbe541fad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe540afb90 con 0x7fbe540d02e0 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbe540afa10 con 0x7fbe540cff00 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbe540afd10 con 0x7fbe5415eff0 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe540d02e0 0x7fbe541fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe540d02e0 0x7fbe541fa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50254/0 (socket says 172.21.15.122:50254) 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 -- 172.21.15.122:0/2229106649 learned_addr learned my addr 172.21.15.122:0/2229106649 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe53400640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe540cff00 0x7fbe540ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe53e00640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe5415eff0 0x7fbe541fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 -- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe5415eff0 msgr2=0x7fbe541fad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe5415eff0 0x7fbe541fad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 -- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe540cff00 msgr2=0x7fbe540ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe540cff00 0x7fbe540ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 -- 172.21.15.122:0/2229106649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe540c84b0 con 0x7fbe540d02e0 2024-09-17T05:38:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe52a00640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe540d02e0 0x7fbe541fa820 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000e9c0 tx=0x7fbe4000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe4b400640 1 -- 172.21.15.122:0/2229106649 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4000cd80 con 0x7fbe540d02e0 2024-09-17T05:38:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe4b400640 1 -- 172.21.15.122:0/2229106649 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe40004020 con 0x7fbe540d02e0 2024-09-17T05:38:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe4b400640 1 -- 172.21.15.122:0/2229106649 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40004450 con 0x7fbe540d02e0 2024-09-17T05:38:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.851+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe540ced10 con 0x7fbe540d02e0 2024-09-17T05:38:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.855+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe540c8070 con 0x7fbe540d02e0 2024-09-17T05:38:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.855+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe540d0720 con 0x7fbe540d02e0 2024-09-17T05:38:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.855+0000 7fbe4b400640 1 -- 172.21.15.122:0/2229106649 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7fbe40003600 con 0x7fbe540d02e0 2024-09-17T05:38:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.855+0000 7fbe4b400640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe34075060 0x7fbe34077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:40.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.855+0000 7fbe4b400640 1 -- 172.21.15.122:0/2229106649 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7fbe40093bb0 con 0x7fbe540d02e0 2024-09-17T05:38:40.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.855+0000 7fbe53400640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe34075060 0x7fbe34077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.859+0000 7fbe53400640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe34075060 0x7fbe34077520 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbe541fb8e0 tx=0x7fbe440091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:40.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:40.863+0000 7fbe4b400640 1 -- 172.21.15.122:0/2229106649 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbe40060070 con 0x7fbe540d02e0 2024-09-17T05:38:41.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.059+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone78512", "target": ["mon-mgr", ""]}) -- 0x7fbe54069e60 con 0x7fbe34075060 2024-09-17T05:38:41.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.359+0000 7fbe4b400640 1 -- 172.21.15.122:0/2229106649 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe44011be0 con 0x7fbe34075060 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe34075060 msgr2=0x7fbe34077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe34075060 0x7fbe34077520 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbe541fb8e0 tx=0x7fbe440091e0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe540d02e0 msgr2=0x7fbe541fa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe540d02e0 0x7fbe541fa820 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000e9c0 tx=0x7fbe4000ee80 comp rx=0 tx=0).stop 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 shutdown_connections 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbe34075060 0x7fbe34077520 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe5415eff0 0x7fbe541fad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe540d02e0 0x7fbe541fa820 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 --2- 172.21.15.122:0/2229106649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe540cff00 0x7fbe540ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 >> 172.21.15.122:0/2229106649 conn(0x7fbe540d1da0 msgr2=0x7fbe54071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 shutdown_connections 2024-09-17T05:38:41.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.363+0000 7fbe59000640 1 -- 172.21.15.122:0/2229106649 wait complete. 2024-09-17T05:38:41.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone291 2024-09-17T05:38:41.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.527+0000 7f01e4800640 1 -- 172.21.15.122:0/3158284837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 msgr2=0x7f01dc15ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:41.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.527+0000 7f01e4800640 1 --2- 172.21.15.122:0/3158284837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 0x7f01dc15ebd0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f01dc06e170 tx=0x7f01d801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.527+0000 7f01e4800640 1 -- 172.21.15.122:0/3158284837 shutdown_connections 2024-09-17T05:38:41.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.527+0000 7f01e4800640 1 --2- 172.21.15.122:0/3158284837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 0x7f01dc15ebd0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.527+0000 7f01e4800640 1 --2- 172.21.15.122:0/3158284837 >> v2:172.21.15.173:3300/0 conn(0x7f01dc0aef60 0x7f01dc0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.527+0000 7f01e4800640 1 -- 172.21.15.122:0/3158284837 >> 172.21.15.122:0/3158284837 conn(0x7f01dc0cfbc0 msgr2=0x7f01dc0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:41.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.527+0000 7f01e4800640 1 -- 172.21.15.122:0/3158284837 shutdown_connections 2024-09-17T05:38:41.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.527+0000 7f01e4800640 1 -- 172.21.15.122:0/3158284837 wait complete. 2024-09-17T05:38:41.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 Processor -- start 2024-09-17T05:38:41.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 -- start start 2024-09-17T05:38:41.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0aef60 0x7f01dc0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:41.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc15e360 0x7f01dc154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 0x7f01dc154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc1680b0 con 0x7f01dc15e740 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc167f30 con 0x7f01dc0aef60 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc168230 con 0x7f01dc15e360 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 0x7f01dc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 0x7f01dc154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33758/0 (socket says 172.21.15.122:33758) 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 -- 172.21.15.122:0/1145498822 learned_addr learned my addr 172.21.15.122:0/1145498822 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e1400640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc15e360 0x7f01dc154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e1e00640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0aef60 0x7f01dc0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 -- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc15e360 msgr2=0x7f01dc154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:41.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc15e360 0x7f01dc154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 -- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0aef60 msgr2=0x7f01dc0b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0aef60 0x7f01dc0b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 -- 172.21.15.122:0/1145498822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc06ec30 con 0x7f01dc15e740 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e1400640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc15e360 0x7f01dc154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e2800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 0x7f01dc154890 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009bd0 tx=0x7f01d8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d80bb070 con 0x7f01dc15e740 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01d8004b40 con 0x7f01dc15e740 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d80b2040 con 0x7f01dc15e740 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.531+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc1f6470 con 0x7f01dc15e740 2024-09-17T05:38:41.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.535+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc1f6750 con 0x7f01dc15e740 2024-09-17T05:38:41.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.535+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc15ebd0 con 0x7f01dc15e740 2024-09-17T05:38:41.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.535+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 46) ==== 99988+0+0 (secure 0 0 0) 0x7f01d8004000 con 0x7f01dc15e740 2024-09-17T05:38:41.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.535+0000 7f01d6000640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01c0075060 0x7f01c0077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:41.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.535+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f01d812cec0 con 0x7f01dc15e740 2024-09-17T05:38:41.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.539+0000 7f01e1e00640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01c0075060 0x7f01c0077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:41.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.539+0000 7f01e1e00640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01c0075060 0x7f01c0077520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f01dc14ed50 tx=0x7f01d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:41.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.539+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f01d8016970 con 0x7f01dc15e740 2024-09-17T05:38:41.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.731+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone291", "target": ["mon-mgr", ""]}) -- 0x7f01dc15a240 con 0x7f01c0075060 2024-09-17T05:38:41.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:41.747+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 47) ==== 100039+0+0 (secure 0 0 0) 0x7f01d80f5540 con 0x7f01dc15e740 2024-09-17T05:38:48.268 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:48.270+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:48.268 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:48.270+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:48.268 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:48.274+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:48.268 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:48.274+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:48.268 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:38:48.274+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:38:49.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:49.791+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f01d80f9530 con 0x7f01dc15e740 2024-09-17T05:38:53.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.259+0000 7f01d6000640 1 -- 172.21.15.122:0/1145498822 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f01dc15a240 con 0x7f01c0075060 2024-09-17T05:38:53.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01c0075060 msgr2=0x7f01c0077520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01c0075060 0x7f01c0077520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f01dc14ed50 tx=0x7f01d0009040 comp rx=0 tx=0).stop 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 msgr2=0x7f01dc154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 0x7f01dc154890 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009bd0 tx=0x7f01d8005b00 comp rx=0 tx=0).stop 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 shutdown_connections 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f01c0075060 0x7f01c0077520 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01dc15e740 0x7f01dc154890 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc15e360 0x7f01dc154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 --2- 172.21.15.122:0/1145498822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc0aef60 0x7f01dc0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 >> 172.21.15.122:0/1145498822 conn(0x7f01dc0cfbc0 msgr2=0x7f01dc0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 shutdown_connections 2024-09-17T05:38:53.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.263+0000 7f01e4800640 1 -- 172.21.15.122:0/1145498822 wait complete. 2024-09-17T05:38:53.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1010 2024-09-17T05:38:53.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.435+0000 7fd5b6a00640 1 -- 172.21.15.122:0/2419006094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00cf440 msgr2=0x7fd5b00ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.435+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/2419006094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00cf440 0x7fd5b00ce350 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0051480 tx=0x7fd59c01c890 comp rx=0 tx=0).stop 2024-09-17T05:38:53.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.435+0000 7fd5b6a00640 1 -- 172.21.15.122:0/2419006094 shutdown_connections 2024-09-17T05:38:53.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.435+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/2419006094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00cf440 0x7fd5b00ce350 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.435+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/2419006094 >> v2:172.21.15.173:3300/0 conn(0x7fd5b00734f0 0x7fd5b015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.435+0000 7fd5b6a00640 1 -- 172.21.15.122:0/2419006094 >> 172.21.15.122:0/2419006094 conn(0x7fd5b00cfbc0 msgr2=0x7fd5b00cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:53.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.435+0000 7fd5b6a00640 1 -- 172.21.15.122:0/2419006094 shutdown_connections 2024-09-17T05:38:53.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.435+0000 7fd5b6a00640 1 -- 172.21.15.122:0/2419006094 wait complete. 2024-09-17T05:38:53.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 Processor -- start 2024-09-17T05:38:53.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 -- start start 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00734f0 0x7fd5b01512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b00ce890 0x7fd5b0144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5b00cf440 0x7fd5b0153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0167a30 con 0x7fd5b00734f0 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b01678b0 con 0x7fd5b00ce890 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b0167bb0 con 0x7fd5b00cf440 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00734f0 0x7fd5b01512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00734f0 0x7fd5b01512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56974/0 (socket says 172.21.15.122:56974) 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 -- 172.21.15.122:0/1105925951 learned_addr learned my addr 172.21.15.122:0/1105925951 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5af400640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b00ce890 0x7fd5b0144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b4a00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5b00cf440 0x7fd5b0153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 -- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5b00cf440 msgr2=0x7fd5b0153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5b00cf440 0x7fd5b0153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 -- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b00ce890 msgr2=0x7fd5b0144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b00ce890 0x7fd5b0144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 -- 172.21.15.122:0/1105925951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b01fbb90 con 0x7fd5b00734f0 2024-09-17T05:38:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5afe00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00734f0 0x7fd5b01512c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000b550 tx=0x7fd5a000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5a7e00640 1 -- 172.21.15.122:0/1105925951 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0014070 con 0x7fd5b00734f0 2024-09-17T05:38:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5a7e00640 1 -- 172.21.15.122:0/1105925951 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5a0010980 con 0x7fd5b00734f0 2024-09-17T05:38:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b01fbe70 con 0x7fd5b00734f0 2024-09-17T05:38:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5a7e00640 1 -- 172.21.15.122:0/1105925951 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0010dd0 con 0x7fd5b00734f0 2024-09-17T05:38:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.439+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b01fc430 con 0x7fd5b00734f0 2024-09-17T05:38:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.443+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b0069410 con 0x7fd5b00734f0 2024-09-17T05:38:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.443+0000 7fd5a7e00640 1 -- 172.21.15.122:0/1105925951 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fd5a0017070 con 0x7fd5b00734f0 2024-09-17T05:38:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.443+0000 7fd5a7e00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd590075180 0x7fd590077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:53.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.443+0000 7fd5a7e00640 1 -- 172.21.15.122:0/1105925951 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5907+0+0 (secure 0 0 0) 0x7fd5a0092fa0 con 0x7fd5b00734f0 2024-09-17T05:38:53.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.443+0000 7fd5af400640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd590075180 0x7fd590077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:53.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.447+0000 7fd5af400640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd590075180 0x7fd590077640 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0af000 tx=0x7fd59c01b0d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:53.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.447+0000 7fd5a7e00640 1 -- 172.21.15.122:0/1105925951 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd5a005f500 con 0x7fd5b00734f0 2024-09-17T05:38:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.639+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1010", "target": ["mon-mgr", ""]}) -- 0x7fd5b01fca80 con 0x7fd590075180 2024-09-17T05:38:53.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.659+0000 7fd5a7e00640 1 -- 172.21.15.122:0/1105925951 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5b01fca80 con 0x7fd590075180 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.659+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd590075180 msgr2=0x7fd590077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.659+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd590075180 0x7fd590077640 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0af000 tx=0x7fd59c01b0d0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.659+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00734f0 msgr2=0x7fd5b01512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.659+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00734f0 0x7fd5b01512c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000b550 tx=0x7fd5a000ba10 comp rx=0 tx=0).stop 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.663+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 shutdown_connections 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.663+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd590075180 0x7fd590077640 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.663+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5b00cf440 0x7fd5b0153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.663+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b00ce890 0x7fd5b0144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.663+0000 7fd5b6a00640 1 --2- 172.21.15.122:0/1105925951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5b00734f0 0x7fd5b01512c0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.663+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 >> 172.21.15.122:0/1105925951 conn(0x7fd5b00cfbc0 msgr2=0x7fd5b0150190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:53.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.663+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 shutdown_connections 2024-09-17T05:38:53.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.663+0000 7fd5b6a00640 1 -- 172.21.15.122:0/1105925951 wait complete. 2024-09-17T05:38:53.681 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:38:53.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:38:53.734 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:38:53.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:38:53.743 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:38:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 -- 172.21.15.122:0/3298195074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d2815a4d0 msgr2=0x7f2d2815ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/3298195074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d2815a4d0 0x7f2d2815ea30 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2d280b2950 tx=0x7f2d1801c570 comp rx=0 tx=0).stop 2024-09-17T05:38:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 -- 172.21.15.122:0/3298195074 shutdown_connections 2024-09-17T05:38:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/3298195074 >> v2:172.21.15.173:3300/0 conn(0x7f2d280b47e0 0x7f2d2806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/3298195074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d280703d0 0x7f2d280b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/3298195074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d2815a4d0 0x7f2d2815ea30 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 -- 172.21.15.122:0/3298195074 >> 172.21.15.122:0/3298195074 conn(0x7f2d280b18f0 msgr2=0x7f2d280b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 -- 172.21.15.122:0/3298195074 shutdown_connections 2024-09-17T05:38:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.931+0000 7f2d2dc00640 1 -- 172.21.15.122:0/3298195074 wait complete. 2024-09-17T05:38:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 Processor -- start 2024-09-17T05:38:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 -- start start 2024-09-17T05:38:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d280703d0 0x7f2d2813e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d280b47e0 0x7f2d281f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d2815a4d0 0x7f2d281f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2d280cccf0 con 0x7f2d280b47e0 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d280ccb70 con 0x7f2d2815a4d0 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2d280cce70 con 0x7f2d280703d0 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d280b47e0 0x7f2d281f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d280b47e0 0x7f2d281f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56994/0 (socket says 172.21.15.122:56994) 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 -- 172.21.15.122:0/2711421886 learned_addr learned my addr 172.21.15.122:0/2711421886 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26a00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d280703d0 0x7f2d2813e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 -- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d280703d0 msgr2=0x7f2d2813e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d280703d0 0x7f2d2813e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d27400640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d2815a4d0 0x7f2d281f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:53.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 -- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d2815a4d0 msgr2=0x7f2d281f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d2815a4d0 0x7f2d281f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 -- 172.21.15.122:0/2711421886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2814c7f0 con 0x7f2d280b47e0 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26a00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d280703d0 0x7f2d2813e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d27400640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d2815a4d0 0x7f2d281f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d26000640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d280b47e0 0x7f2d281f5320 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c940 tx=0x7f2d1400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d1ea00640 1 -- 172.21.15.122:0/2711421886 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007bb0 con 0x7f2d280b47e0 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d1ea00640 1 -- 172.21.15.122:0/2711421886 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d1400f450 con 0x7f2d280b47e0 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d1ea00640 1 -- 172.21.15.122:0/2711421886 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007e60 con 0x7f2d280b47e0 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281550e0 con 0x7f2d280b47e0 2024-09-17T05:38:53.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.935+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d28153c00 con 0x7f2d280b47e0 2024-09-17T05:38:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.939+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2806e7a0 con 0x7f2d280b47e0 2024-09-17T05:38:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.939+0000 7f2d1ea00640 1 -- 172.21.15.122:0/2711421886 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f2d1400f8a0 con 0x7f2d280b47e0 2024-09-17T05:38:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.939+0000 7f2d1ea00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d08075180 0x7f2d08077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.939+0000 7f2d1ea00640 1 -- 172.21.15.122:0/2711421886 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5907+0+0 (secure 0 0 0) 0x7f2d14093b10 con 0x7f2d280b47e0 2024-09-17T05:38:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.939+0000 7f2d26a00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d08075180 0x7f2d08077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.943+0000 7f2d26a00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d08075180 0x7f2d08077640 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2d180af000 tx=0x7f2d1801adb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:53.943+0000 7f2d1ea00640 1 -- 172.21.15.122:0/2711421886 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2d14014030 con 0x7f2d280b47e0 2024-09-17T05:38:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.115+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f2d28069b00 con 0x7f2d280b47e0 2024-09-17T05:38:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.115+0000 7f2d1ea00640 1 -- 172.21.15.122:0/2711421886 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7f2d140606a0 con 0x7f2d280b47e0 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d08075180 msgr2=0x7f2d08077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d08075180 0x7f2d08077640 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2d180af000 tx=0x7f2d1801adb0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d280b47e0 msgr2=0x7f2d281f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d280b47e0 0x7f2d281f5320 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c940 tx=0x7f2d1400ce00 comp rx=0 tx=0).stop 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 shutdown_connections 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d08075180 0x7f2d08077640 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d2815a4d0 0x7f2d281f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d280b47e0 0x7f2d281f5320 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 --2- 172.21.15.122:0/2711421886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d280703d0 0x7f2d2813e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 >> 172.21.15.122:0/2711421886 conn(0x7f2d280b18f0 msgr2=0x7f2d281596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:54.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 shutdown_connections 2024-09-17T05:38:54.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.119+0000 7f2d2dc00640 1 -- 172.21.15.122:0/2711421886 wait complete. 2024-09-17T05:38:54.132 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:38:54.132 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:38:54.222 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:38:54.318 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:38:54.319 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:38:54.319 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:38:54.319 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:38:54.328 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:38:54.328 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:38:54.379 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:38:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- 172.21.15.122:0/971760005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 msgr2=0x7f9acc0b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/971760005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 0x7f9acc0b4340 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9acc069950 tx=0x7f9ab401db60 comp rx=0 tx=0).stop 2024-09-17T05:38:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- 172.21.15.122:0/971760005 shutdown_connections 2024-09-17T05:38:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/971760005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 0x7f9acc0b4340 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/971760005 >> v2:172.21.15.173:3300/0 conn(0x7f9acc15db50 0x7f9acc0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- 172.21.15.122:0/971760005 >> 172.21.15.122:0/971760005 conn(0x7f9acc0b18f0 msgr2=0x7f9acc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- 172.21.15.122:0/971760005 shutdown_connections 2024-09-17T05:38:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- 172.21.15.122:0/971760005 wait complete. 2024-09-17T05:38:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 Processor -- start 2024-09-17T05:38:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- start start 2024-09-17T05:38:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9acc0b4880 0x7f9acc15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 0x7f9acc073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc15db50 0x7f9acc14d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc0ccd50 con 0x7f9acc0cd100 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc0ccbd0 con 0x7f9acc15db50 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc167120 con 0x7f9acc0b4880 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 0x7f9acc073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 0x7f9acc073650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57014/0 (socket says 172.21.15.122:57014) 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac1400640 1 -- 172.21.15.122:0/4024006835 learned_addr learned my addr 172.21.15.122:0/4024006835 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9aca800640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc15db50 0x7f9acc14d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac9e00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9acc0b4880 0x7f9acc15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac1400640 1 -- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9acc0b4880 msgr2=0x7f9acc15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac1400640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9acc0b4880 0x7f9acc15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac1400640 1 -- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc15db50 msgr2=0x7f9acc14d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac1400640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc15db50 0x7f9acc14d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.567+0000 7f9ac1400640 1 -- 172.21.15.122:0/4024006835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc074110 con 0x7f9acc0cd100 2024-09-17T05:38:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ac1400640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 0x7f9acc073650 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40b1040 tx=0x7f9ab40a68a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ac2a00640 1 -- 172.21.15.122:0/4024006835 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab40b3070 con 0x7f9acc0cd100 2024-09-17T05:38:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ac2a00640 1 -- 172.21.15.122:0/4024006835 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ab40055f0 con 0x7f9acc0cd100 2024-09-17T05:38:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc1635d0 con 0x7f9acc0cd100 2024-09-17T05:38:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ac2a00640 1 -- 172.21.15.122:0/4024006835 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4004930 con 0x7f9acc0cd100 2024-09-17T05:38:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc163b90 con 0x7f9acc0cd100 2024-09-17T05:38:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc069410 con 0x7f9acc0cd100 2024-09-17T05:38:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ac2a00640 1 -- 172.21.15.122:0/4024006835 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f9ab40be060 con 0x7f9acc0cd100 2024-09-17T05:38:54.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ac2a00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa8075180 0x7f9aa8077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:54.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.571+0000 7f9ac2a00640 1 -- 172.21.15.122:0/4024006835 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5879+0+0 (secure 0 0 0) 0x7f9ab412d490 con 0x7f9acc0cd100 2024-09-17T05:38:54.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.575+0000 7f9ac9e00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa8075180 0x7f9aa8077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:54.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.575+0000 7f9ac9e00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa8075180 0x7f9aa8077640 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0ad170 tx=0x7f9abc00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:54.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.575+0000 7f9ac2a00640 1 -- 172.21.15.122:0/4024006835 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9ab40177e0 con 0x7f9acc0cd100 2024-09-17T05:38:54.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.755+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9acc072a10 con 0x7f9acc0cd100 2024-09-17T05:38:54.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.755+0000 7f9ac2a00640 1 -- 172.21.15.122:0/4024006835 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) ==== 80+0+0 (secure 0 0 0) 0x7f9ab40f9950 con 0x7f9acc0cd100 2024-09-17T05:38:54.751 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:38:54.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa8075180 msgr2=0x7f9aa8077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:54.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa8075180 0x7f9aa8077640 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0ad170 tx=0x7f9abc00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 msgr2=0x7f9acc073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 0x7f9acc073650 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40b1040 tx=0x7f9ab40a68a0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 shutdown_connections 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9aa8075180 0x7f9aa8077640 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc15db50 0x7f9acc14d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9acc0cd100 0x7f9acc073650 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 --2- 172.21.15.122:0/4024006835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9acc0b4880 0x7f9acc15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 >> 172.21.15.122:0/4024006835 conn(0x7f9acc0b18f0 msgr2=0x7f9acc0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 shutdown_connections 2024-09-17T05:38:54.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:54.759+0000 7f9ad0c00640 1 -- 172.21.15.122:0/4024006835 wait complete. 2024-09-17T05:38:54.776 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:38:54.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:38:54.776 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:38:54.821 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:38:54.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:38:54.822 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:38:54.876 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:38:54.876 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:38:54.876 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:38:54.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:38:54.885 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:38:54.886 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:38:54.886 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:38:54.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:38:54.928 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:38:54.928 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:38:54.928 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:38:54.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:38:54.939 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:38:55.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 -- 172.21.15.122:0/1321662687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45900ced80 msgr2=0x7f45900ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 --2- 172.21.15.122:0/1321662687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45900ced80 0x7f45900ae9b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f457c009840 tx=0x7f457c01c820 comp rx=0 tx=0).stop 2024-09-17T05:38:55.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 -- 172.21.15.122:0/1321662687 shutdown_connections 2024-09-17T05:38:55.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 --2- 172.21.15.122:0/1321662687 >> v2:172.21.15.173:3300/0 conn(0x7f45900cdb30 0x7f45900aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 --2- 172.21.15.122:0/1321662687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45900ced80 0x7f45900ae9b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 -- 172.21.15.122:0/1321662687 >> 172.21.15.122:0/1321662687 conn(0x7f45900d1dc0 msgr2=0x7f45900d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:55.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 -- 172.21.15.122:0/1321662687 shutdown_connections 2024-09-17T05:38:55.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 -- 172.21.15.122:0/1321662687 wait complete. 2024-09-17T05:38:55.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 Processor -- start 2024-09-17T05:38:55.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.119+0000 7f4597600640 1 -- start start 2024-09-17T05:38:55.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45900cdb30 0x7f45900c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45900ced80 0x7f45900c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f459015bea0 0x7f459014d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45901692f0 con 0x7f45900ced80 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4590169170 con 0x7f45900cdb30 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4590169470 con 0x7f459015bea0 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45900ced80 0x7f45900c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45900ced80 0x7f45900c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57022/0 (socket says 172.21.15.122:57022) 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 -- 172.21.15.122:0/3577196116 learned_addr learned my addr 172.21.15.122:0/3577196116 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4595600640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f459015bea0 0x7f459014d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4594c00640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45900cdb30 0x7f45900c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 -- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f459015bea0 msgr2=0x7f459014d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f459015bea0 0x7f459014d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 -- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45900cdb30 msgr2=0x7f45900c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45900cdb30 0x7f45900c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 -- 172.21.15.122:0/3577196116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45900c8730 con 0x7f45900ced80 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4594c00640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45900cdb30 0x7f45900c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458fe00640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45900ced80 0x7f45900c5570 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f457c01aed0 tx=0x7f457c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458cc00640 1 -- 172.21.15.122:0/3577196116 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c0b2040 con 0x7f45900ced80 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458cc00640 1 -- 172.21.15.122:0/3577196116 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f457c004320 con 0x7f45900ced80 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45900c8a10 con 0x7f45900ced80 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f458cc00640 1 -- 172.21.15.122:0/3577196116 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c005280 con 0x7f45900ced80 2024-09-17T05:38:55.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45900c93c0 con 0x7f45900ced80 2024-09-17T05:38:55.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.123+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45900ce670 con 0x7f45900ced80 2024-09-17T05:38:55.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.127+0000 7f458cc00640 1 -- 172.21.15.122:0/3577196116 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f457c0bd060 con 0x7f45900ced80 2024-09-17T05:38:55.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.127+0000 7f458cc00640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4570075180 0x7f4570077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.127+0000 7f458cc00640 1 -- 172.21.15.122:0/3577196116 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5879+0+0 (secure 0 0 0) 0x7f457c12c8a0 con 0x7f45900ced80 2024-09-17T05:38:55.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.127+0000 7f4594c00640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4570075180 0x7f4570077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.127+0000 7f4594c00640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4570075180 0x7f4570077640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f45800017b0 tx=0x7f4580009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:55.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.131+0000 7f458cc00640 1 -- 172.21.15.122:0/3577196116 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f457c016410 con 0x7f45900ced80 2024-09-17T05:38:55.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45900afda0 con 0x7f45900ced80 2024-09-17T05:38:55.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f458cc00640 1 -- 172.21.15.122:0/3577196116 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5234 (secure 0 0 0) 0x7f457c0f9430 con 0x7f45900ced80 2024-09-17T05:38:55.329 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:55.329 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":77,"btime":"2024-09-17T05:37:50:792793+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/4034540416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":4034540416},{"type":"v1","addr":"172.21.15.173:6836","nonce":4034540416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7651,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4130637584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4130637584},{"type":"v1","addr":"172.21.15.173:6837","nonce":4130637584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/543665156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":543665156},{"type":"v1","addr":"172.21.15.122:6833","nonce":543665156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:37:42.117978+0000","modified":"2024-09-17T05:37:45.159422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2875934571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2875934571},{"type":"v1","addr":"172.21.15.122:6835","nonce":2875934571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8364,"qdb_cluster":[8364]},"id":8}]} 2024-09-17T05:38:55.329 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 77 2024-09-17T05:38:55.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4570075180 msgr2=0x7f4570077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4570075180 0x7f4570077640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f45800017b0 tx=0x7f4580009040 comp rx=0 tx=0).stop 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45900ced80 msgr2=0x7f45900c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45900ced80 0x7f45900c5570 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f457c01aed0 tx=0x7f457c005dc0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 shutdown_connections 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4570075180 0x7f4570077640 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f459015bea0 0x7f459014d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45900ced80 0x7f45900c5570 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 --2- 172.21.15.122:0/3577196116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45900cdb30 0x7f45900c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 >> 172.21.15.122:0/3577196116 conn(0x7f45900d1dc0 msgr2=0x7f459006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 shutdown_connections 2024-09-17T05:38:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.335+0000 7f4597600640 1 -- 172.21.15.122:0/3577196116 wait complete. 2024-09-17T05:38:55.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:38:55.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- 172.21.15.122:0/633287802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40cdb30 msgr2=0x7f88f40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 --2- 172.21.15.122:0/633287802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40cdb30 0x7f88f40cebb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009900 tx=0x7f88e001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- 172.21.15.122:0/633287802 shutdown_connections 2024-09-17T05:38:55.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 --2- 172.21.15.122:0/633287802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40cdb30 0x7f88f40cebb0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 --2- 172.21.15.122:0/633287802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415bea0 0x7f88f40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- 172.21.15.122:0/633287802 >> 172.21.15.122:0/633287802 conn(0x7f88f40d1dc0 msgr2=0x7f88f40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:55.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- 172.21.15.122:0/633287802 shutdown_connections 2024-09-17T05:38:55.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- 172.21.15.122:0/633287802 wait complete. 2024-09-17T05:38:55.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 Processor -- start 2024-09-17T05:38:55.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- start start 2024-09-17T05:38:55.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40cdb30 0x7f88f40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40d28e0 0x7f88f414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415bea0 0x7f88f40cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4168fc0 con 0x7f88f40d28e0 2024-09-17T05:38:55.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4168e40 con 0x7f88f40cdb30 2024-09-17T05:38:55.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f9e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f88f4169140 con 0x7f88f415bea0 2024-09-17T05:38:55.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40d28e0 0x7f88f414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40d28e0 0x7f88f414d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57038/0 (socket says 172.21.15.122:57038) 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2000640 1 -- 172.21.15.122:0/2927411922 learned_addr learned my addr 172.21.15.122:0/2927411922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2a00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40cdb30 0x7f88f40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f3400640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415bea0 0x7f88f40cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2000640 1 -- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415bea0 msgr2=0x7f88f40cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2000640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415bea0 0x7f88f40cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2000640 1 -- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40cdb30 msgr2=0x7f88f40b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2000640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40cdb30 0x7f88f40b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.499+0000 7f88f2000640 1 -- 172.21.15.122:0/2927411922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f40b1b20 con 0x7f88f40d28e0 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88f2000640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40d28e0 0x7f88f414d170 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f88e00b0040 tx=0x7f88e00a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88eb400640 1 -- 172.21.15.122:0/2927411922 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e00b2040 con 0x7f88f40d28e0 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88eb400640 1 -- 172.21.15.122:0/2927411922 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88e0005030 con 0x7f88f40d28e0 2024-09-17T05:38:55.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f40b1e00 con 0x7f88f40d28e0 2024-09-17T05:38:55.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88eb400640 1 -- 172.21.15.122:0/2927411922 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e00044f0 con 0x7f88f40d28e0 2024-09-17T05:38:55.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f409e700 con 0x7f88f40d28e0 2024-09-17T05:38:55.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f40cc9d0 con 0x7f88f40d28e0 2024-09-17T05:38:55.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88eb400640 1 -- 172.21.15.122:0/2927411922 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f88e00be060 con 0x7f88f40d28e0 2024-09-17T05:38:55.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.503+0000 7f88eb400640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4075180 0x7f88d4077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.507+0000 7f88eb400640 1 -- 172.21.15.122:0/2927411922 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5879+0+0 (secure 0 0 0) 0x7f88e012c6c0 con 0x7f88f40d28e0 2024-09-17T05:38:55.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.507+0000 7f88f2a00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4075180 0x7f88d4077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.507+0000 7f88f2a00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4075180 0x7f88d4077640 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009ec0 tx=0x7f88e4009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:55.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.507+0000 7f88eb400640 1 -- 172.21.15.122:0/2927411922 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f88e00164d0 con 0x7f88f40d28e0 2024-09-17T05:38:55.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88f415b8c0 con 0x7f88f40d28e0 2024-09-17T05:38:55.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88eb400640 1 -- 172.21.15.122:0/2927411922 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5234 (secure 0 0 0) 0x7f88e00f91d0 con 0x7f88f40d28e0 2024-09-17T05:38:55.705 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:55.706 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":77,"btime":"2024-09-17T05:37:50:792793+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/4034540416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":4034540416},{"type":"v1","addr":"172.21.15.173:6836","nonce":4034540416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7651,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4130637584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4130637584},{"type":"v1","addr":"172.21.15.173:6837","nonce":4130637584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/543665156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":543665156},{"type":"v1","addr":"172.21.15.122:6833","nonce":543665156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:37:42.117978+0000","modified":"2024-09-17T05:37:45.159422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2875934571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2875934571},{"type":"v1","addr":"172.21.15.122:6835","nonce":2875934571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8364,"qdb_cluster":[8364]},"id":8}]} 2024-09-17T05:38:55.706 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 77 2024-09-17T05:38:55.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4075180 msgr2=0x7f88d4077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4075180 0x7f88d4077640 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009ec0 tx=0x7f88e4009500 comp rx=0 tx=0).stop 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40d28e0 msgr2=0x7f88f414d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40d28e0 0x7f88f414d170 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f88e00b0040 tx=0x7f88e00a57c0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 shutdown_connections 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f88d4075180 0x7f88d4077640 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88f415bea0 0x7f88f40cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88f40d28e0 0x7f88f414d170 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 --2- 172.21.15.122:0/2927411922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88f40cdb30 0x7f88f40b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 >> 172.21.15.122:0/2927411922 conn(0x7f88f40d1dc0 msgr2=0x7f88f406e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 shutdown_connections 2024-09-17T05:38:55.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.711+0000 7f88f9e00640 1 -- 172.21.15.122:0/2927411922 wait complete. 2024-09-17T05:38:55.727 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:38:55.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.895+0000 7febe7e00640 1 -- 172.21.15.122:0/1150348942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe8160310 msgr2=0x7febe80cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.895+0000 7febe7e00640 1 --2- 172.21.15.122:0/1150348942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe8160310 0x7febe80cfa60 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7febe8074d60 tx=0x7febd801c830 comp rx=0 tx=0).stop 2024-09-17T05:38:55.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.895+0000 7febe7e00640 1 -- 172.21.15.122:0/1150348942 shutdown_connections 2024-09-17T05:38:55.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.895+0000 7febe7e00640 1 --2- 172.21.15.122:0/1150348942 >> v2:172.21.15.173:3300/0 conn(0x7febe80b42d0 0x7febe80cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.895+0000 7febe7e00640 1 --2- 172.21.15.122:0/1150348942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe8160310 0x7febe80cfa60 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.895+0000 7febe7e00640 1 -- 172.21.15.122:0/1150348942 >> 172.21.15.122:0/1150348942 conn(0x7febe80b37b0 msgr2=0x7febe80b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:55.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.899+0000 7febe7e00640 1 -- 172.21.15.122:0/1150348942 shutdown_connections 2024-09-17T05:38:55.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.899+0000 7febe7e00640 1 -- 172.21.15.122:0/1150348942 wait complete. 2024-09-17T05:38:55.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe7e00640 1 Processor -- start 2024-09-17T05:38:55.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe7e00640 1 -- start start 2024-09-17T05:38:55.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8070930 0x7febe80c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe80b42d0 0x7febe80b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe8160310 0x7febe81fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe7e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7febe8169180 con 0x7febe80b42d0 2024-09-17T05:38:55.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe7e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7febe8169000 con 0x7febe8070930 2024-09-17T05:38:55.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe7e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7febe8169300 con 0x7febe8160310 2024-09-17T05:38:55.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe80b42d0 0x7febe80b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe80b42d0 0x7febe80b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57062/0 (socket says 172.21.15.122:57062) 2024-09-17T05:38:55.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 -- 172.21.15.122:0/1783497054 learned_addr learned my addr 172.21.15.122:0/1783497054 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:55.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 -- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe8160310 msgr2=0x7febe81fb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe8160310 0x7febe81fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 -- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8070930 msgr2=0x7febe80c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8070930 0x7febe80c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 -- 172.21.15.122:0/1783497054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe81fc860 con 0x7febe80b42d0 2024-09-17T05:38:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.903+0000 7febe6000640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe80b42d0 0x7febe80b4ad0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7febd400e9f0 tx=0x7febd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.907+0000 7febdf400640 1 -- 172.21.15.122:0/1783497054 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd400cd80 con 0x7febe80b42d0 2024-09-17T05:38:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.907+0000 7febdf400640 1 -- 172.21.15.122:0/1783497054 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febd4010430 con 0x7febe80b42d0 2024-09-17T05:38:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.907+0000 7febdf400640 1 -- 172.21.15.122:0/1783497054 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4010860 con 0x7febe80b42d0 2024-09-17T05:38:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.907+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe80b7790 con 0x7febe80b42d0 2024-09-17T05:38:55.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.907+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe80c7e60 con 0x7febe80b42d0 2024-09-17T05:38:55.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.907+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe80c8190 con 0x7febe80b42d0 2024-09-17T05:38:55.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.911+0000 7febdf400640 1 -- 172.21.15.122:0/1783497054 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7febd4010a00 con 0x7febe80b42d0 2024-09-17T05:38:55.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.911+0000 7febdf400640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc80751b0 0x7febc8077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:55.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.911+0000 7febdf400640 1 -- 172.21.15.122:0/1783497054 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5879+0+0 (secure 0 0 0) 0x7febd4093df0 con 0x7febe80b42d0 2024-09-17T05:38:55.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.911+0000 7febe6a00640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc80751b0 0x7febc8077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:55.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.911+0000 7febe6a00640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc80751b0 0x7febc8077670 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7febd80af000 tx=0x7febd80195d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:55.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:55.919+0000 7febdf400640 1 -- 172.21.15.122:0/1783497054 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7febd40609a0 con 0x7febe80b42d0 2024-09-17T05:38:56.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.115+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febe8070d80 con 0x7febe80b42d0 2024-09-17T05:38:56.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.115+0000 7febdf400640 1 -- 172.21.15.122:0/1783497054 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17087 (secure 0 0 0) 0x7febd40602a0 con 0x7febe80b42d0 2024-09-17T05:38:56.110 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:56.110 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":121,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:38:54.073999+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:37:39.824175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T05:37:41.091437+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.5","mappings":[{"from":4,"to":3}]},{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:38:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc80751b0 msgr2=0x7febc8077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc80751b0 0x7febc8077670 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7febd80af000 tx=0x7febd80195d0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe80b42d0 msgr2=0x7febe80b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe80b42d0 0x7febe80b4ad0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7febd400e9f0 tx=0x7febd400eeb0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 shutdown_connections 2024-09-17T05:38:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc80751b0 0x7febc8077670 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe8160310 0x7febe81fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe80b42d0 0x7febe80b4ad0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 --2- 172.21.15.122:0/1783497054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe8070930 0x7febe80c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 >> 172.21.15.122:0/1783497054 conn(0x7febe80b37b0 msgr2=0x7febe80ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:56.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 shutdown_connections 2024-09-17T05:38:56.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.119+0000 7febe7e00640 1 -- 172.21.15.122:0/1783497054 wait complete. 2024-09-17T05:38:56.132 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:38:56.132 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:38:56.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 -- 172.21.15.122:0/3220821827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b601613d0 msgr2=0x7f3b600d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 --2- 172.21.15.122:0/3220821827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b601613d0 0x7f3b600d0540 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009870 tx=0x7f3b5001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 -- 172.21.15.122:0/3220821827 shutdown_connections 2024-09-17T05:38:56.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 --2- 172.21.15.122:0/3220821827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b601613d0 0x7f3b600d0540 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 -- 172.21.15.122:0/3220821827 >> 172.21.15.122:0/3220821827 conn(0x7f3b600d3e00 msgr2=0x7f3b600d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:56.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 -- 172.21.15.122:0/3220821827 shutdown_connections 2024-09-17T05:38:56.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 -- 172.21.15.122:0/3220821827 wait complete. 2024-09-17T05:38:56.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 Processor -- start 2024-09-17T05:38:56.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.279+0000 7f3b66400640 1 -- start start 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b600d29e0 0x7f3b601ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b601613d0 0x7f3b601585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b601617b0 0x7f3b6015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6016bee0 con 0x7f3b600d29e0 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600b8280 con 0x7f3b601613d0 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6016c060 con 0x7f3b601617b0 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b600d29e0 0x7f3b601ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b600d29e0 0x7f3b601ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57068/0 (socket says 172.21.15.122:57068) 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 -- 172.21.15.122:0/24635999 learned_addr learned my addr 172.21.15.122:0/24635999 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65a00640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b601617b0 0x7f3b6015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:56.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b57e00640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b601613d0 0x7f3b601585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 -- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b601617b0 msgr2=0x7f3b6015ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b601617b0 0x7f3b6015ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 -- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b601613d0 msgr2=0x7f3b601585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b601613d0 0x7f3b601585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 -- 172.21.15.122:0/24635999 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b600c8530 con 0x7f3b600d29e0 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b65000640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b600d29e0 0x7f3b601ffa10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009790 tx=0x7f3b50005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b5d600640 1 -- 172.21.15.122:0/24635999 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b500b2040 con 0x7f3b600d29e0 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b5d600640 1 -- 172.21.15.122:0/24635999 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b500043f0 con 0x7f3b600d29e0 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b5d600640 1 -- 172.21.15.122:0/24635999 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50005220 con 0x7f3b600d29e0 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b600c8810 con 0x7f3b600d29e0 2024-09-17T05:38:56.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b600c8dd0 con 0x7f3b600d29e0 2024-09-17T05:38:56.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.283+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b600b6f80 con 0x7f3b600d29e0 2024-09-17T05:38:56.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.287+0000 7f3b5d600640 1 -- 172.21.15.122:0/24635999 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f3b500bd060 con 0x7f3b600d29e0 2024-09-17T05:38:56.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.287+0000 7f3b5d600640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3b3c075180 0x7f3b3c077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:56.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.287+0000 7f3b5d600640 1 -- 172.21.15.122:0/24635999 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5879+0+0 (secure 0 0 0) 0x7f3b5012ca60 con 0x7f3b600d29e0 2024-09-17T05:38:56.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.287+0000 7f3b57e00640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3b3c075180 0x7f3b3c077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:56.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.287+0000 7f3b57e00640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3b3c075180 0x7f3b3c077640 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3b60074320 tx=0x7f3b48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:56.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.291+0000 7f3b5d600640 1 -- 172.21.15.122:0/24635999 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3b50016410 con 0x7f3b600d29e0 2024-09-17T05:38:56.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.491+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b6006dc80 con 0x7f3b600d29e0 2024-09-17T05:38:56.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b5d600640 1 -- 172.21.15.122:0/24635999 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5234 (secure 0 0 0) 0x7f3b500f94f0 con 0x7f3b600d29e0 2024-09-17T05:38:56.488 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:56.488 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":77,"btime":"2024-09-17T05:37:50:792793+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/4034540416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":4034540416},{"type":"v1","addr":"172.21.15.173:6836","nonce":4034540416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7651,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4130637584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4130637584},{"type":"v1","addr":"172.21.15.173:6837","nonce":4130637584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/543665156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":543665156},{"type":"v1","addr":"172.21.15.122:6833","nonce":543665156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:37:42.117978+0000","modified":"2024-09-17T05:37:45.159422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2875934571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2875934571},{"type":"v1","addr":"172.21.15.122:6835","nonce":2875934571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8364,"qdb_cluster":[8364]},"id":8}]} 2024-09-17T05:38:56.488 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 77 2024-09-17T05:38:56.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3b3c075180 msgr2=0x7f3b3c077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3b3c075180 0x7f3b3c077640 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3b60074320 tx=0x7f3b48009040 comp rx=0 tx=0).stop 2024-09-17T05:38:56.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b600d29e0 msgr2=0x7f3b601ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b600d29e0 0x7f3b601ffa10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009790 tx=0x7f3b50005db0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 shutdown_connections 2024-09-17T05:38:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3b3c075180 0x7f3b3c077640 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b601617b0 0x7f3b6015ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b601613d0 0x7f3b601585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 --2- 172.21.15.122:0/24635999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b600d29e0 0x7f3b601ffa10 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 >> 172.21.15.122:0/24635999 conn(0x7f3b600d3e00 msgr2=0x7f3b6006df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 shutdown_connections 2024-09-17T05:38:56.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.495+0000 7f3b66400640 1 -- 172.21.15.122:0/24635999 wait complete. 2024-09-17T05:38:56.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:38:56.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 -- 172.21.15.122:0/3867286824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe4071cb0 msgr2=0x7effe4071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 --2- 172.21.15.122:0/3867286824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe4071cb0 0x7effe4071090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7effcc009870 tx=0x7effcc01c860 comp rx=0 tx=0).stop 2024-09-17T05:38:56.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 -- 172.21.15.122:0/3867286824 shutdown_connections 2024-09-17T05:38:56.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 --2- 172.21.15.122:0/3867286824 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe4071cb0 0x7effe4071090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 -- 172.21.15.122:0/3867286824 >> 172.21.15.122:0/3867286824 conn(0x7effe40d1be0 msgr2=0x7effe40d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:56.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 -- 172.21.15.122:0/3867286824 shutdown_connections 2024-09-17T05:38:56.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 -- 172.21.15.122:0/3867286824 wait complete. 2024-09-17T05:38:56.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 Processor -- start 2024-09-17T05:38:56.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 -- start start 2024-09-17T05:38:56.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe4071cb0 0x7effe41f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:56.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.659+0000 7effe9800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe40ce5e0 0x7effe41539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:56.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe9800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe40ce9c0 0x7effe415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:56.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe9800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7effe416a240 con 0x7effe40ce9c0 2024-09-17T05:38:56.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe9800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7effe416a0c0 con 0x7effe4071cb0 2024-09-17T05:38:56.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe9800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7effe416a3c0 con 0x7effe40ce5e0 2024-09-17T05:38:56.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe40ce9c0 0x7effe415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:56.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe4071cb0 0x7effe41f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:56.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effdbe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe40ce5e0 0x7effe41539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:56.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe4071cb0 0x7effe41f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:39436/0 (socket says 172.21.15.122:39436) 2024-09-17T05:38:56.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe40ce9c0 0x7effe415dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57074/0 (socket says 172.21.15.122:57074) 2024-09-17T05:38:56.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effdbe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe40ce5e0 0x7effe41539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:36236/0 (socket says 172.21.15.122:36236) 2024-09-17T05:38:56.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe3e00640 1 -- 172.21.15.122:0/2383180805 learned_addr learned my addr 172.21.15.122:0/2383180805 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effdbe00640 1 -- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe4071cb0 msgr2=0x7effe41f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effdbe00640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe4071cb0 0x7effe41f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effdbe00640 1 -- 172.21.15.122:0/2383180805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe40ce9c0 msgr2=0x7effe415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effdbe00640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe40ce9c0 0x7effe415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effdbe00640 1 -- 172.21.15.122:0/2383180805 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe40cca00 con 0x7effe40ce5e0 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe8e00640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe40ce9c0 0x7effe415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effdbe00640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe40ce5e0 0x7effe41539e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7effe4069b30 tx=0x7effcc005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe0c00640 1 -- 172.21.15.122:0/2383180805 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b2040 con 0x7effe40ce5e0 2024-09-17T05:38:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe0c00640 1 -- 172.21.15.122:0/2383180805 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effcc004040 con 0x7effe40ce5e0 2024-09-17T05:38:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe40b55c0 con 0x7effe40ce5e0 2024-09-17T05:38:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe0c00640 1 -- 172.21.15.122:0/2383180805 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc004410 con 0x7effe40ce5e0 2024-09-17T05:38:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7effe40b5b80 con 0x7effe40ce5e0 2024-09-17T05:38:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe40cc4f0 con 0x7effe40ce5e0 2024-09-17T05:38:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.663+0000 7effe0c00640 1 -- 172.21.15.122:0/2383180805 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7effcc0bd030 con 0x7effe40ce5e0 2024-09-17T05:38:56.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.667+0000 7effe0c00640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effc0075180 0x7effc0077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:56.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.667+0000 7effe0c00640 1 -- 172.21.15.122:0/2383180805 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(121..121 src has 1..121) ==== 5879+0+0 (secure 0 0 0) 0x7effcc12c990 con 0x7effe40ce5e0 2024-09-17T05:38:56.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.667+0000 7effe3e00640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effc0075180 0x7effc0077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:56.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.667+0000 7effe3e00640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effc0075180 0x7effc0077640 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7effd4007b40 tx=0x7effd4007a90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:56.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.671+0000 7effe0c00640 1 -- 172.21.15.122:0/2383180805 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7effcc0c4060 con 0x7effe40ce5e0 2024-09-17T05:38:56.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.875+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effe40aff10 con 0x7effe40ce5e0 2024-09-17T05:38:56.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.875+0000 7effe0c00640 1 -- 172.21.15.122:0/2383180805 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5234 (secure 0 0 0) 0x7effcc016450 con 0x7effe40ce5e0 2024-09-17T05:38:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:56.872 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":77,"btime":"2024-09-17T05:37:50:792793+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/4034540416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":4034540416},{"type":"v1","addr":"172.21.15.173:6836","nonce":4034540416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7651,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4130637584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4130637584},{"type":"v1","addr":"172.21.15.173:6837","nonce":4130637584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/543665156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":543665156},{"type":"v1","addr":"172.21.15.122:6833","nonce":543665156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:37:42.117978+0000","modified":"2024-09-17T05:37:45.159422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8364":{"gid":8364,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2875934571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2875934571},{"type":"v1","addr":"172.21.15.122:6835","nonce":2875934571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8364,"qdb_cluster":[8364]},"id":8}]} 2024-09-17T05:38:56.872 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 77 2024-09-17T05:38:56.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effc0075180 msgr2=0x7effc0077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effc0075180 0x7effc0077640 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7effd4007b40 tx=0x7effd4007a90 comp rx=0 tx=0).stop 2024-09-17T05:38:56.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe40ce5e0 msgr2=0x7effe41539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:56.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe40ce5e0 0x7effe41539e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7effe4069b30 tx=0x7effcc005760 comp rx=0 tx=0).stop 2024-09-17T05:38:56.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 shutdown_connections 2024-09-17T05:38:56.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7effc0075180 0x7effc0077640 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe40ce9c0 0x7effe415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effe40ce5e0 0x7effe41539e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 --2- 172.21.15.122:0/2383180805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effe4071cb0 0x7effe41f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:56.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 >> 172.21.15.122:0/2383180805 conn(0x7effe40d1be0 msgr2=0x7effe406dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:56.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 shutdown_connections 2024-09-17T05:38:56.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:56.879+0000 7effe9800640 1 -- 172.21.15.122:0/2383180805 wait complete. 2024-09-17T05:38:56.893 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:38:57.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 -- 172.21.15.122:0/332009692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc280b2d60 msgr2=0x7fcc280af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 --2- 172.21.15.122:0/332009692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc280b2d60 0x7fcc280af710 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009c80 tx=0x7fcc2001f5a0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 -- 172.21.15.122:0/332009692 shutdown_connections 2024-09-17T05:38:57.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 --2- 172.21.15.122:0/332009692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc280b2d60 0x7fcc280af710 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 --2- 172.21.15.122:0/332009692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc2815f300 0x7fcc280cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 -- 172.21.15.122:0/332009692 >> 172.21.15.122:0/332009692 conn(0x7fcc280d1e30 msgr2=0x7fcc280d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:57.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 -- 172.21.15.122:0/332009692 shutdown_connections 2024-09-17T05:38:57.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 -- 172.21.15.122:0/332009692 wait complete. 2024-09-17T05:38:57.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 Processor -- start 2024-09-17T05:38:57.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.039+0000 7fcc30600640 1 -- start start 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc280b2d60 0x7fcc280cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc2815f300 0x7fcc2815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc2815f6e0 0x7fcc281f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc280affe0 con 0x7fcc2815f300 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc280afe60 con 0x7fcc280b2d60 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcc280b0160 con 0x7fcc2815f6e0 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc2815f300 0x7fcc2815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc2815f300 0x7fcc2815b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57084/0 (socket says 172.21.15.122:57084) 2024-09-17T05:38:57.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 -- 172.21.15.122:0/3735505982 learned_addr learned my addr 172.21.15.122:0/3735505982 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:57.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2e600640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc2815f6e0 0x7fcc281f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2dc00640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc280b2d60 0x7fcc280cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 -- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc2815f6e0 msgr2=0x7fcc281f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc2815f6e0 0x7fcc281f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 -- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc280b2d60 msgr2=0x7fcc280cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc280b2d60 0x7fcc280cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 -- 172.21.15.122:0/3735505982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc281f5490 con 0x7fcc2815f300 2024-09-17T05:38:57.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc2d200640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc2815f300 0x7fcc2815b640 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c420 tx=0x7fcc1400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:57.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc25600640 1 -- 172.21.15.122:0/3735505982 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14017070 con 0x7fcc2815f300 2024-09-17T05:38:57.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc25600640 1 -- 172.21.15.122:0/3735505982 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc140053c0 con 0x7fcc2815f300 2024-09-17T05:38:57.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc25600640 1 -- 172.21.15.122:0/3735505982 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14003860 con 0x7fcc2815f300 2024-09-17T05:38:57.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc281f5770 con 0x7fcc2815f300 2024-09-17T05:38:57.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc280cd6f0 con 0x7fcc2815f300 2024-09-17T05:38:57.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.043+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc280af710 con 0x7fcc2815f300 2024-09-17T05:38:57.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.047+0000 7fcc25600640 1 -- 172.21.15.122:0/3735505982 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fcc1401a070 con 0x7fcc2815f300 2024-09-17T05:38:57.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.047+0000 7fcc25600640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcc08075180 0x7fcc08077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.047+0000 7fcc25600640 1 -- 172.21.15.122:0/3735505982 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5879+0+0 (secure 0 0 0) 0x7fcc14092580 con 0x7fcc2815f300 2024-09-17T05:38:57.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.047+0000 7fcc2dc00640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcc08075180 0x7fcc08077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.047+0000 7fcc2dc00640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcc08075180 0x7fcc08077640 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00bc00 tx=0x7fcc1c00b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:57.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.051+0000 7fcc25600640 1 -- 172.21.15.122:0/3735505982 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcc1405ea40 con 0x7fcc2815f300 2024-09-17T05:38:57.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.219+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc28072480 con 0x7fcc2815f300 2024-09-17T05:38:57.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc25600640 1 -- 172.21.15.122:0/3735505982 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17087 (secure 0 0 0) 0x7fcc14063470 con 0x7fcc2815f300 2024-09-17T05:38:57.216 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:38:57.216 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":121,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:38:54.073999+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:37:39.824175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-17T05:37:41.091437+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.5","mappings":[{"from":4,"to":3}]},{"pgid":"17.37","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:38:57.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcc08075180 msgr2=0x7fcc08077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcc08075180 0x7fcc08077640 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00bc00 tx=0x7fcc1c00b390 comp rx=0 tx=0).stop 2024-09-17T05:38:57.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc2815f300 msgr2=0x7fcc2815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc2815f300 0x7fcc2815b640 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c420 tx=0x7fcc1400c8e0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 shutdown_connections 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcc08075180 0x7fcc08077640 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc2815f6e0 0x7fcc281f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc2815f300 0x7fcc2815b640 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 --2- 172.21.15.122:0/3735505982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc280b2d60 0x7fcc280cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 >> 172.21.15.122:0/3735505982 conn(0x7fcc280d1e30 msgr2=0x7fcc2815d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 shutdown_connections 2024-09-17T05:38:57.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.223+0000 7fcc30600640 1 -- 172.21.15.122:0/3735505982 wait complete. 2024-09-17T05:38:57.238 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:38:57.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- 172.21.15.122:0/4124963702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a0004d50 msgr2=0x7f68a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 --2- 172.21.15.122:0/4124963702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a0004d50 0x7f68a00051d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c640 tx=0x7f68b401d320 comp rx=0 tx=0).stop 2024-09-17T05:38:57.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c1a00640 1 -- 172.21.15.122:0/4124963702 reap_dead start 2024-09-17T05:38:57.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- 172.21.15.122:0/4124963702 shutdown_connections 2024-09-17T05:38:57.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- 172.21.15.122:0/4124963702 >> 172.21.15.122:0/4124963702 conn(0x7f68bc0d1dc0 msgr2=0x7f68bc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:57.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- 172.21.15.122:0/4124963702 shutdown_connections 2024-09-17T05:38:57.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- 172.21.15.122:0/4124963702 wait complete. 2024-09-17T05:38:57.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 Processor -- start 2024-09-17T05:38:57.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- start start 2024-09-17T05:38:57.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a0004d50 0x7f68bc0b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68bc0c90a0 0x7f68bc0b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68bc154ec0 0x7f68bc155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc15c490 con 0x7f68bc154ec0 2024-09-17T05:38:57.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc0cfb50 con 0x7f68bc0c90a0 2024-09-17T05:38:57.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.387+0000 7f68c3a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f68bc0cebb0 con 0x7f68a0004d50 2024-09-17T05:38:57.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68bc154ec0 0x7f68bc155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68bc154ec0 0x7f68bc155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57092/0 (socket says 172.21.15.122:57092) 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 -- 172.21.15.122:0/1676902307 learned_addr learned my addr 172.21.15.122:0/1676902307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1000640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a0004d50 0x7f68bc0b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68b3e00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68bc0c90a0 0x7f68bc0b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 -- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a0004d50 msgr2=0x7f68bc0b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a0004d50 0x7f68bc0b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 -- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68bc0c90a0 msgr2=0x7f68bc0b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68bc0c90a0 0x7f68bc0b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 -- 172.21.15.122:0/1676902307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc06ebd0 con 0x7f68bc154ec0 2024-09-17T05:38:57.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c1a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68bc154ec0 0x7f68bc155340 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f68b40014a0 tx=0x7f68b40a5fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:57.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68b9600640 1 -- 172.21.15.122:0/1676902307 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b40bc070 con 0x7f68bc154ec0 2024-09-17T05:38:57.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68b9600640 1 -- 172.21.15.122:0/1676902307 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68b4003b80 con 0x7f68bc154ec0 2024-09-17T05:38:57.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68b9600640 1 -- 172.21.15.122:0/1676902307 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b40b2040 con 0x7f68bc154ec0 2024-09-17T05:38:57.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc0c9730 con 0x7f68bc154ec0 2024-09-17T05:38:57.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc0c9cf0 con 0x7f68bc154ec0 2024-09-17T05:38:57.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc0cfcd0 con 0x7f68bc154ec0 2024-09-17T05:38:57.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.391+0000 7f68b9600640 1 -- 172.21.15.122:0/1676902307 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f68b40ab070 con 0x7f68bc154ec0 2024-09-17T05:38:57.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.395+0000 7f68b9600640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6898075180 0x7f6898077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.395+0000 7f68b9600640 1 -- 172.21.15.122:0/1676902307 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5879+0+0 (secure 0 0 0) 0x7f68b412d010 con 0x7f68bc154ec0 2024-09-17T05:38:57.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.395+0000 7f68c1000640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6898075180 0x7f6898077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.395+0000 7f68c1000640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6898075180 0x7f6898077640 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009ce0 tx=0x7f68ac0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:57.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.399+0000 7f68b9600640 1 -- 172.21.15.122:0/1676902307 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f68b40170d0 con 0x7f68bc154ec0 2024-09-17T05:38:57.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.599+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f68bc0c9480 con 0x7f68bc154ec0 2024-09-17T05:38:57.594 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:38:57.599+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:38:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.603+0000 7f68b9600640 1 -- 172.21.15.122:0/1676902307 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7f68b40f9be0 con 0x7f68bc154ec0 2024-09-17T05:38:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:38:57.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.603+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6898075180 msgr2=0x7f6898077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.603+0000 7f68c3a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6898075180 0x7f6898077640 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009ce0 tx=0x7f68ac0091e0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.603+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68bc154ec0 msgr2=0x7f68bc155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.603+0000 7f68c3a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68bc154ec0 0x7f68bc155340 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f68b40014a0 tx=0x7f68b40a5fd0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.607+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 shutdown_connections 2024-09-17T05:38:57.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.607+0000 7f68c3a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6898075180 0x7f6898077640 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.607+0000 7f68c3a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68bc154ec0 0x7f68bc155340 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.607+0000 7f68c3a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68bc0c90a0 0x7f68bc0b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.607+0000 7f68c3a00640 1 --2- 172.21.15.122:0/1676902307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a0004d50 0x7f68bc0b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.607+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 >> 172.21.15.122:0/1676902307 conn(0x7f68bc0d1dc0 msgr2=0x7f68bc06e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:57.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.607+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 shutdown_connections 2024-09-17T05:38:57.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.607+0000 7f68c3a00640 1 -- 172.21.15.122:0/1676902307 wait complete. 2024-09-17T05:38:57.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:38:57.666 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:38:57.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- 172.21.15.122:0/4109892814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c06fea0 msgr2=0x7f267c15e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 --2- 172.21.15.122:0/4109892814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c06fea0 0x7f267c15e7f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f266c009900 tx=0x7f266c01c850 comp rx=0 tx=0).stop 2024-09-17T05:38:57.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- 172.21.15.122:0/4109892814 shutdown_connections 2024-09-17T05:38:57.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 --2- 172.21.15.122:0/4109892814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c06fea0 0x7f267c15e7f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- 172.21.15.122:0/4109892814 >> 172.21.15.122:0/4109892814 conn(0x7f267c0cfc20 msgr2=0x7f267c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:57.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- 172.21.15.122:0/4109892814 shutdown_connections 2024-09-17T05:38:57.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- 172.21.15.122:0/4109892814 wait complete. 2024-09-17T05:38:57.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 Processor -- start 2024-09-17T05:38:57.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- start start 2024-09-17T05:38:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f267c06fea0 0x7f267c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c0af220 0x7f267c0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f267c0cddd0 0x7f267c1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f267c1693d0 con 0x7f267c0af220 2024-09-17T05:38:57.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f267c169250 con 0x7f267c0cddd0 2024-09-17T05:38:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2683600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f267c169550 con 0x7f267c06fea0 2024-09-17T05:38:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2673e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c0af220 0x7f267c0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2673e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c0af220 0x7f267c0b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57116/0 (socket says 172.21.15.122:57116) 2024-09-17T05:38:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2673e00640 1 -- 172.21.15.122:0/2906551578 learned_addr learned my addr 172.21.15.122:0/2906551578 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2680c00640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f267c06fea0 0x7f267c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2681600640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f267c0cddd0 0x7f267c1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2673e00640 1 -- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f267c06fea0 msgr2=0x7f267c0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2673e00640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f267c06fea0 0x7f267c0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2673e00640 1 -- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f267c0cddd0 msgr2=0x7f267c1591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2673e00640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f267c0cddd0 0x7f267c1591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2673e00640 1 -- 172.21.15.122:0/2906551578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0b3ae0 con 0x7f267c0af220 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.775+0000 7f2680c00640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f267c06fea0 0x7f267c0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.779+0000 7f2673e00640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c0af220 0x7f267c0b55f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f266400c500 tx=0x7f266400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.779+0000 7f2679600640 1 -- 172.21.15.122:0/2906551578 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664017070 con 0x7f267c0af220 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.779+0000 7f2679600640 1 -- 172.21.15.122:0/2906551578 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26640053c0 con 0x7f267c0af220 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.779+0000 7f2679600640 1 -- 172.21.15.122:0/2906551578 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664012890 con 0x7f267c0af220 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.779+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c1f9590 con 0x7f267c0af220 2024-09-17T05:38:57.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.779+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c1f9b50 con 0x7f267c0af220 2024-09-17T05:38:57.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.779+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c0ca270 con 0x7f267c0af220 2024-09-17T05:38:57.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.779+0000 7f2679600640 1 -- 172.21.15.122:0/2906551578 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f266401a070 con 0x7f267c0af220 2024-09-17T05:38:57.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.783+0000 7f2679600640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26580751b0 0x7f2658077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:57.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.783+0000 7f2679600640 1 -- 172.21.15.122:0/2906551578 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5937+0+0 (secure 0 0 0) 0x7f2664092560 con 0x7f267c0af220 2024-09-17T05:38:57.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.783+0000 7f2680c00640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26580751b0 0x7f2658077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:57.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.783+0000 7f2680c00640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26580751b0 0x7f2658077670 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f266c0af000 tx=0x7f266c01b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:57.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.787+0000 7f2679600640 1 -- 172.21.15.122:0/2906551578 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f266405f070 con 0x7f267c0af220 2024-09-17T05:38:57.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:57.991+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f267c0d0810 con 0x7f267c0af220 2024-09-17T05:38:58.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.603+0000 7f2679600640 1 -- 172.21.15.122:0/2906551578 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7f266405e970 con 0x7f267c0af220 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26580751b0 msgr2=0x7f2658077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26580751b0 0x7f2658077670 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f266c0af000 tx=0x7f266c01b090 comp rx=0 tx=0).stop 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c0af220 msgr2=0x7f267c0b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c0af220 0x7f267c0b55f0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f266400c500 tx=0x7f266400c9c0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 shutdown_connections 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f26580751b0 0x7f2658077670 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f267c0cddd0 0x7f267c1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f267c0af220 0x7f267c0b55f0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 --2- 172.21.15.122:0/2906551578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f267c06fea0 0x7f267c0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 >> 172.21.15.122:0/2906551578 conn(0x7f267c0cfc20 msgr2=0x7f267c0acdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:58.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 shutdown_connections 2024-09-17T05:38:58.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.607+0000 7f2683600640 1 -- 172.21.15.122:0/2906551578 wait complete. 2024-09-17T05:38:58.624 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:38:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.823+0000 7f71abc00640 1 -- 172.21.15.122:0/1636365602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a415f390 msgr2=0x7f71a40cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.823+0000 7f71abc00640 1 --2- 172.21.15.122:0/1636365602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a415f390 0x7f71a40cfc30 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f71a40b3850 tx=0x7f719401c890 comp rx=0 tx=0).stop 2024-09-17T05:38:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.823+0000 7f71abc00640 1 -- 172.21.15.122:0/1636365602 shutdown_connections 2024-09-17T05:38:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.823+0000 7f71abc00640 1 --2- 172.21.15.122:0/1636365602 >> v2:172.21.15.173:3300/0 conn(0x7f71a40af850 0x7f71a40afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.823+0000 7f71abc00640 1 --2- 172.21.15.122:0/1636365602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a415f390 0x7f71a40cfc30 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.823+0000 7f71abc00640 1 -- 172.21.15.122:0/1636365602 >> 172.21.15.122:0/1636365602 conn(0x7f71a40d1dc0 msgr2=0x7f71a40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.823+0000 7f71abc00640 1 -- 172.21.15.122:0/1636365602 shutdown_connections 2024-09-17T05:38:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.823+0000 7f71abc00640 1 -- 172.21.15.122:0/1636365602 wait complete. 2024-09-17T05:38:58.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 Processor -- start 2024-09-17T05:38:58.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 -- start start 2024-09-17T05:38:58.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a40af850 0x7f71a406e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a40d28e0 0x7f71a40c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a415f390 0x7f71a4152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4168ea0 con 0x7f71a40d28e0 2024-09-17T05:38:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4168d20 con 0x7f71a415f390 2024-09-17T05:38:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f71a4169020 con 0x7f71a40af850 2024-09-17T05:38:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a40d28e0 0x7f71a40c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a40d28e0 0x7f71a40c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57138/0 (socket says 172.21.15.122:57138) 2024-09-17T05:38:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71a9c00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a415f390 0x7f71a4152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 -- 172.21.15.122:0/2994553371 learned_addr learned my addr 172.21.15.122:0/2994553371 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71a9200640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a40af850 0x7f71a406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 -- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a40af850 msgr2=0x7f71a406e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a40af850 0x7f71a406e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 -- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a415f390 msgr2=0x7f71a4152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a415f390 0x7f71a4152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 -- 172.21.15.122:0/2994553371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a40b6e20 con 0x7f71a40d28e0 2024-09-17T05:38:58.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f719be00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a40d28e0 0x7f71a40c76f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c4e0 tx=0x7f718c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:58.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71a1600640 1 -- 172.21.15.122:0/2994553371 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c0052d0 con 0x7f71a40d28e0 2024-09-17T05:38:58.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71a1600640 1 -- 172.21.15.122:0/2994553371 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f718c005470 con 0x7f71a40d28e0 2024-09-17T05:38:58.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71a1600640 1 -- 172.21.15.122:0/2994553371 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c012890 con 0x7f71a40d28e0 2024-09-17T05:38:58.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a40b7100 con 0x7f71a40d28e0 2024-09-17T05:38:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.827+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a40b76c0 con 0x7f71a40d28e0 2024-09-17T05:38:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.831+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a40b5520 con 0x7f71a40d28e0 2024-09-17T05:38:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.831+0000 7f71a1600640 1 -- 172.21.15.122:0/2994553371 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f718c003600 con 0x7f71a40d28e0 2024-09-17T05:38:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.831+0000 7f71a1600640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7180075180 0x7f7180077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.831+0000 7f71a1600640 1 -- 172.21.15.122:0/2994553371 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5937+0+0 (secure 0 0 0) 0x7f718c092c70 con 0x7f71a40d28e0 2024-09-17T05:38:58.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.831+0000 7f71a9200640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7180075180 0x7f7180077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:58.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.831+0000 7f71a9200640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7180075180 0x7f7180077640 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f71940af000 tx=0x7f719401b0d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:58.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:58.835+0000 7f71a1600640 1 -- 172.21.15.122:0/2994553371 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f71a40b5520 con 0x7f71a40d28e0 2024-09-17T05:38:59.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.007+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f71a40afc30 con 0x7f71a40d28e0 2024-09-17T05:38:59.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71a1600640 1 -- 172.21.15.122:0/2994553371 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) ==== 181+0+0 (secure 0 0 0) 0x7f718c05f130 con 0x7f71a40d28e0 2024-09-17T05:38:59.605 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:38:59.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7180075180 msgr2=0x7f7180077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7180075180 0x7f7180077640 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f71940af000 tx=0x7f719401b0d0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a40d28e0 msgr2=0x7f71a40c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a40d28e0 0x7f71a40c76f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f718c00c4e0 tx=0x7f718c00c9a0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 shutdown_connections 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7180075180 0x7f7180077640 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a415f390 0x7f71a4152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71a40d28e0 0x7f71a40c76f0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 --2- 172.21.15.122:0/2994553371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a40af850 0x7f71a406e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 >> 172.21.15.122:0/2994553371 conn(0x7f71a40d1dc0 msgr2=0x7f71a415be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 shutdown_connections 2024-09-17T05:38:59.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.611+0000 7f71abc00640 1 -- 172.21.15.122:0/2994553371 wait complete. 2024-09-17T05:38:59.629 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:38:59.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.779+0000 7fc1c4000640 1 -- 172.21.15.122:0/1961043854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 msgr2=0x7fc1a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:59.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.779+0000 7fc1c4000640 1 --2- 172.21.15.122:0/1961043854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 0x7fc1a400b9d0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400c640 tx=0x7fc1b401d2c0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.779+0000 7fc1c4000640 1 -- 172.21.15.122:0/1961043854 shutdown_connections 2024-09-17T05:38:59.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.779+0000 7fc1c4000640 1 --2- 172.21.15.122:0/1961043854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 0x7fc1a400b9d0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.779+0000 7fc1c4000640 1 -- 172.21.15.122:0/1961043854 >> 172.21.15.122:0/1961043854 conn(0x7fc1bc0d1da0 msgr2=0x7fc1bc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:38:59.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.779+0000 7fc1c4000640 1 -- 172.21.15.122:0/1961043854 shutdown_connections 2024-09-17T05:38:59.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.779+0000 7fc1c4000640 1 -- 172.21.15.122:0/1961043854 wait complete. 2024-09-17T05:38:59.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 Processor -- start 2024-09-17T05:38:59.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 -- start start 2024-09-17T05:38:59.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 0x7fc1bc06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:59.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1bc06f6b0 0x7fc1bc151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:59.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1bc1f5500 0x7fc1bc0c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:59.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc0ceba0 con 0x7fc1a40051d0 2024-09-17T05:38:59.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc0cea20 con 0x7fc1bc1f5500 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc1bc0ced20 con 0x7fc1bc06f6b0 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 0x7fc1bc06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 0x7fc1bc06df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57160/0 (socket says 172.21.15.122:57160) 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 -- 172.21.15.122:0/3878741742 learned_addr learned my addr 172.21.15.122:0/3878741742 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c2000640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1bc1f5500 0x7fc1bc0c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c0c00640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1bc06f6b0 0x7fc1bc151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 -- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1bc06f6b0 msgr2=0x7fc1bc151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1bc06f6b0 0x7fc1bc151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 -- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1bc1f5500 msgr2=0x7fc1bc0c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1bc1f5500 0x7fc1bc0c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 -- 172.21.15.122:0/3878741742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc1f5960 con 0x7fc1a40051d0 2024-09-17T05:38:59.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c1600640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 0x7fc1bc06df20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0003ee0 tx=0x7fc1b0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1b9600640 1 -- 172.21.15.122:0/3878741742 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0015070 con 0x7fc1a40051d0 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1b9600640 1 -- 172.21.15.122:0/3878741742 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1b0011840 con 0x7fc1a40051d0 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc06e5b0 con 0x7fc1a40051d0 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1b9600640 1 -- 172.21.15.122:0/3878741742 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0011c70 con 0x7fc1a40051d0 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc06ebd0 con 0x7fc1a40051d0 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.783+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1bc0cf280 con 0x7fc1a40051d0 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.787+0000 7fc1b9600640 1 -- 172.21.15.122:0/3878741742 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fc1b0018070 con 0x7fc1a40051d0 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.787+0000 7fc1b9600640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc19c0751b0 0x7fc19c077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:38:59.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.787+0000 7fc1b9600640 1 -- 172.21.15.122:0/3878741742 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5470+0+0 (secure 0 0 0) 0x7fc1b0093e40 con 0x7fc1a40051d0 2024-09-17T05:38:59.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.787+0000 7fc1c0c00640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc19c0751b0 0x7fc19c077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:38:59.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.787+0000 7fc1c0c00640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc19c0751b0 0x7fc19c077670 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc06deb0 tx=0x7fc1a80091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:38:59.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.791+0000 7fc1b9600640 1 -- 172.21.15.122:0/3878741742 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc1b0060950 con 0x7fc1a40051d0 2024-09-17T05:38:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:38:59.963+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1bc070d80 con 0x7fc1a40051d0 2024-09-17T05:39:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.611+0000 7fc1b9600640 1 -- 172.21.15.122:0/3878741742 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) ==== 169+0+0 (secure 0 0 0) 0x7fc1b0060250 con 0x7fc1a40051d0 2024-09-17T05:39:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:39:00.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc19c0751b0 msgr2=0x7fc19c077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:00.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc19c0751b0 0x7fc19c077670 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc06deb0 tx=0x7fc1a80091e0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 msgr2=0x7fc1bc06df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:00.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 0x7fc1bc06df20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0003ee0 tx=0x7fc1b0001640 comp rx=0 tx=0).stop 2024-09-17T05:39:00.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 shutdown_connections 2024-09-17T05:39:00.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc19c0751b0 0x7fc19c077670 secure :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc06deb0 tx=0x7fc1a80091e0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1bc1f5500 0x7fc1bc0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1bc06f6b0 0x7fc1bc151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 --2- 172.21.15.122:0/3878741742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1a40051d0 0x7fc1bc06df20 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.615+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 >> 172.21.15.122:0/3878741742 conn(0x7fc1bc0d1da0 msgr2=0x7fc1bc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:00.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.619+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 shutdown_connections 2024-09-17T05:39:00.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.619+0000 7fc1c4000640 1 -- 172.21.15.122:0/3878741742 wait complete. 2024-09-17T05:39:00.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:39:00.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.787+0000 7fefedc00640 1 -- 172.21.15.122:0/132394379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 msgr2=0x7fefe80d1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:00.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.787+0000 7fefedc00640 1 --2- 172.21.15.122:0/132394379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 0x7fefe80d1ba0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009840 tx=0x7fefd801c820 comp rx=0 tx=0).stop 2024-09-17T05:39:00.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.787+0000 7fefedc00640 1 -- 172.21.15.122:0/132394379 shutdown_connections 2024-09-17T05:39:00.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.787+0000 7fefedc00640 1 --2- 172.21.15.122:0/132394379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe80cf5a0 0x7fefe80d20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.787+0000 7fefedc00640 1 --2- 172.21.15.122:0/132394379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 0x7fefe80d1ba0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.787+0000 7fefedc00640 1 -- 172.21.15.122:0/132394379 >> 172.21.15.122:0/132394379 conn(0x7fefe80d3fd0 msgr2=0x7fefe80d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:00.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 -- 172.21.15.122:0/132394379 shutdown_connections 2024-09-17T05:39:00.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 -- 172.21.15.122:0/132394379 wait complete. 2024-09-17T05:39:00.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 Processor -- start 2024-09-17T05:39:00.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 -- start start 2024-09-17T05:39:00.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe80cf5a0 0x7fefe814f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe80d2620 0x7fefe815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 0x7fefe81604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fefe816b660 con 0x7fefe815e240 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefe816b4e0 con 0x7fefe80cf5a0 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefedc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fefe816b7e0 con 0x7fefe80d2620 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 0x7fefe81604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 0x7fefe81604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57186/0 (socket says 172.21.15.122:57186) 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 -- 172.21.15.122:0/2622200670 learned_addr learned my addr 172.21.15.122:0/2622200670 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe6a00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe80cf5a0 0x7fefe814f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe6000640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe80d2620 0x7fefe815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:00.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 -- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe80d2620 msgr2=0x7fefe815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe80d2620 0x7fefe815e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 -- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe80cf5a0 msgr2=0x7fefe814f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe80cf5a0 0x7fefe814f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 -- 172.21.15.122:0/2622200670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe8155fb0 con 0x7fefe815e240 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe6a00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe80cf5a0 0x7fefe814f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefe7400640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 0x7fefe81604b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c770 tx=0x7fefe000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefdea00640 1 -- 172.21.15.122:0/2622200670 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0017070 con 0x7fefe815e240 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.791+0000 7fefdea00640 1 -- 172.21.15.122:0/2622200670 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefe0005070 con 0x7fefe815e240 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.795+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe8156290 con 0x7fefe815e240 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.795+0000 7fefdea00640 1 -- 172.21.15.122:0/2622200670 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe00054c0 con 0x7fefe815e240 2024-09-17T05:39:00.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.795+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe8150050 con 0x7fefe815e240 2024-09-17T05:39:00.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.795+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe8150250 con 0x7fefe815e240 2024-09-17T05:39:00.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.795+0000 7fefdea00640 1 -- 172.21.15.122:0/2622200670 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fefe001a070 con 0x7fefe815e240 2024-09-17T05:39:00.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.795+0000 7fefdea00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefc80751b0 0x7fefc8077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.795+0000 7fefdea00640 1 -- 172.21.15.122:0/2622200670 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7fefe00928f0 con 0x7fefe815e240 2024-09-17T05:39:00.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.795+0000 7fefe6a00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefc80751b0 0x7fefc8077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:00.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.799+0000 7fefe6a00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefc80751b0 0x7fefc8077670 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fefd80af000 tx=0x7fefd801b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:00.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.799+0000 7fefdea00640 1 -- 172.21.15.122:0/2622200670 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fefe8150250 con 0x7fefe815e240 2024-09-17T05:39:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.967+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fefe80d08a0 con 0x7fefe815e240 2024-09-17T05:39:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.971+0000 7fefdea00640 1 -- 172.21.15.122:0/2622200670 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7fefe005f400 con 0x7fefe815e240 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefc80751b0 msgr2=0x7fefc8077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefc80751b0 0x7fefc8077670 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fefd80af000 tx=0x7fefd801b060 comp rx=0 tx=0).stop 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 msgr2=0x7fefe81604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 0x7fefe81604b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fefe000c770 tx=0x7fefe000cc30 comp rx=0 tx=0).stop 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 shutdown_connections 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefc80751b0 0x7fefc8077670 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe815e240 0x7fefe81604b0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe80d2620 0x7fefe815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 --2- 172.21.15.122:0/2622200670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe80cf5a0 0x7fefe814f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 >> 172.21.15.122:0/2622200670 conn(0x7fefe80d3fd0 msgr2=0x7fefe81566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:00.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 shutdown_connections 2024-09-17T05:39:00.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:00.975+0000 7fefedc00640 1 -- 172.21.15.122:0/2622200670 wait complete. 2024-09-17T05:39:00.987 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-17T05:39:01.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.135+0000 7f6629000640 1 -- 172.21.15.122:0/4158948565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6608004d30 msgr2=0x7f6608005130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.135+0000 7f6629000640 1 --2- 172.21.15.122:0/4158948565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6608004d30 0x7f6608005130 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6614001860 tx=0x7f661401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.135+0000 7f6622800640 1 -- 172.21.15.122:0/4158948565 reap_dead start 2024-09-17T05:39:01.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.135+0000 7f6629000640 1 -- 172.21.15.122:0/4158948565 shutdown_connections 2024-09-17T05:39:01.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.135+0000 7f6629000640 1 -- 172.21.15.122:0/4158948565 >> 172.21.15.122:0/4158948565 conn(0x7f66240b18f0 msgr2=0x7f66240b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:01.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.135+0000 7f6629000640 1 -- 172.21.15.122:0/4158948565 shutdown_connections 2024-09-17T05:39:01.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.135+0000 7f6629000640 1 -- 172.21.15.122:0/4158948565 wait complete. 2024-09-17T05:39:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 Processor -- start 2024-09-17T05:39:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 -- start start 2024-09-17T05:39:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6608004d30 0x7f662409d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6624157f40 0x7f66240aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66241583c0 0x7f66240af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66240cccf0 con 0x7f6624157f40 2024-09-17T05:39:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66240ccb70 con 0x7f6608004d30 2024-09-17T05:39:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6624166120 con 0x7f66241583c0 2024-09-17T05:39:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6624157f40 0x7f66240aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6624157f40 0x7f66240aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57206/0 (socket says 172.21.15.122:57206) 2024-09-17T05:39:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 -- 172.21.15.122:0/4273663608 learned_addr learned my addr 172.21.15.122:0/4273663608 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6622800640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66241583c0 0x7f66240af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6621e00640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6608004d30 0x7f662409d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:01.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 -- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66241583c0 msgr2=0x7f66240af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66241583c0 0x7f66240af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 -- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6608004d30 msgr2=0x7f662409d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6608004d30 0x7f662409d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 -- 172.21.15.122:0/4273663608 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662409aa10 con 0x7f6624157f40 2024-09-17T05:39:01.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6619400640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6624157f40 0x7f66240aedb0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f660c006fd0 tx=0x7f660c0066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:01.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f661aa00640 1 -- 172.21.15.122:0/4273663608 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c015070 con 0x7f6624157f40 2024-09-17T05:39:01.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f661aa00640 1 -- 172.21.15.122:0/4273663608 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f660c011840 con 0x7f6624157f40 2024-09-17T05:39:01.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662409acb0 con 0x7f6624157f40 2024-09-17T05:39:01.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f661aa00640 1 -- 172.21.15.122:0/4273663608 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c011c30 con 0x7f6624157f40 2024-09-17T05:39:01.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662409c9f0 con 0x7f6624157f40 2024-09-17T05:39:01.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.139+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6624069410 con 0x7f6624157f40 2024-09-17T05:39:01.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.143+0000 7f661aa00640 1 -- 172.21.15.122:0/4273663608 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f660c018070 con 0x7f6624157f40 2024-09-17T05:39:01.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.143+0000 7f661aa00640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66000751b0 0x7f6600077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:01.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.143+0000 7f661aa00640 1 -- 172.21.15.122:0/4273663608 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7f660c093fd0 con 0x7f6624157f40 2024-09-17T05:39:01.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.143+0000 7f6621e00640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66000751b0 0x7f6600077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:01.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.143+0000 7f6621e00640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66000751b0 0x7f6600077670 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6614001830 tx=0x7f6614019720 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:01.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.147+0000 7f661aa00640 1 -- 172.21.15.122:0/4273663608 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f660c060490 con 0x7f6624157f40 2024-09-17T05:39:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.371+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7f6624074530 con 0x7f6624157f40 2024-09-17T05:39:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f661aa00640 1 -- 172.21.15.122:0/4273663608 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v636) ==== 185+0+0 (secure 0 0 0) 0x7f660c064ec0 con 0x7f6624157f40 2024-09-17T05:39:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66000751b0 msgr2=0x7f6600077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66000751b0 0x7f6600077670 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6614001830 tx=0x7f6614019720 comp rx=0 tx=0).stop 2024-09-17T05:39:01.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6624157f40 msgr2=0x7f66240aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6624157f40 0x7f66240aedb0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f660c006fd0 tx=0x7f660c0066c0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 shutdown_connections 2024-09-17T05:39:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f66000751b0 0x7f6600077670 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66241583c0 0x7f66240af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6624157f40 0x7f66240aedb0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 --2- 172.21.15.122:0/4273663608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6608004d30 0x7f662409d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.615+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 >> 172.21.15.122:0/4273663608 conn(0x7f66240b18f0 msgr2=0x7f66241637c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.619+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 shutdown_connections 2024-09-17T05:39:01.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.619+0000 7f6629000640 1 -- 172.21.15.122:0/4273663608 wait complete. 2024-09-17T05:39:01.634 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-09-17T05:39:01.634 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar that is printed for 1 ongoing clone job is 2024-09-17T05:39:01.634 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:39:01.635 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:39:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 -- 172.21.15.122:0/1303407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7980d28e0 msgr2=0x7fc7980cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 --2- 172.21.15.122:0/1303407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7980d28e0 0x7fc7980cc9d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc7980745d0 tx=0x7fc79001cd80 comp rx=0 tx=0).stop 2024-09-17T05:39:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 -- 172.21.15.122:0/1303407 shutdown_connections 2024-09-17T05:39:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 --2- 172.21.15.122:0/1303407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7980d28e0 0x7fc7980cc9d0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 --2- 172.21.15.122:0/1303407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7980cdb30 0x7fc7980cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 --2- 172.21.15.122:0/1303407 >> v2:172.21.15.173:3300/0 conn(0x7fc79815bea0 0x7fc7980ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 -- 172.21.15.122:0/1303407 >> 172.21.15.122:0/1303407 conn(0x7fc7980d1dc0 msgr2=0x7fc7980d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 -- 172.21.15.122:0/1303407 shutdown_connections 2024-09-17T05:39:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.783+0000 7fc79d200640 1 -- 172.21.15.122:0/1303407 wait complete. 2024-09-17T05:39:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 Processor -- start 2024-09-17T05:39:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 -- start start 2024-09-17T05:39:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7980cdb30 0x7fc79806f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7980d28e0 0x7fc798153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79815bea0 0x7fc7980b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980af120 con 0x7fc79815bea0 2024-09-17T05:39:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980aefa0 con 0x7fc7980d28e0 2024-09-17T05:39:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc7980af2a0 con 0x7fc7980cdb30 2024-09-17T05:39:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79815bea0 0x7fc7980b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79815bea0 0x7fc7980b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44460/0 (socket says 172.21.15.122:44460) 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 -- 172.21.15.122:0/3099499828 learned_addr learned my addr 172.21.15.122:0/3099499828 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc795400640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7980d28e0 0x7fc798153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc795e00640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7980cdb30 0x7fc79806f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 -- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7980cdb30 msgr2=0x7fc79806f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7980cdb30 0x7fc79806f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 -- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7980d28e0 msgr2=0x7fc798153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7980d28e0 0x7fc798153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 -- 172.21.15.122:0/3099499828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79806fd70 con 0x7fc79815bea0 2024-09-17T05:39:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc795400640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7980d28e0 0x7fc798153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc796800640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79815bea0 0x7fc7980b5c40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc790009bd0 tx=0x7fc790003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc78e000640 1 -- 172.21.15.122:0/3099499828 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7900bb070 con 0x7fc79815bea0 2024-09-17T05:39:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc78e000640 1 -- 172.21.15.122:0/3099499828 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc790004e00 con 0x7fc79815bea0 2024-09-17T05:39:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79806faa0 con 0x7fc79815bea0 2024-09-17T05:39:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc78e000640 1 -- 172.21.15.122:0/3099499828 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7900b2040 con 0x7fc79815bea0 2024-09-17T05:39:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.787+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7980b1960 con 0x7fc79815bea0 2024-09-17T05:39:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.791+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7980af830 con 0x7fc79815bea0 2024-09-17T05:39:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.791+0000 7fc78e000640 1 -- 172.21.15.122:0/3099499828 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fc7900be070 con 0x7fc79815bea0 2024-09-17T05:39:01.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.791+0000 7fc78e000640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7780751b0 0x7fc778077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:01.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.791+0000 7fc78e000640 1 -- 172.21.15.122:0/3099499828 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7fc79012ce80 con 0x7fc79815bea0 2024-09-17T05:39:01.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.791+0000 7fc795e00640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7780751b0 0x7fc778077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.791+0000 7fc795e00640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7780751b0 0x7fc778077670 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc7980b0f80 tx=0x7fc78800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:01.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.795+0000 7fc78e000640 1 -- 172.21.15.122:0/3099499828 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc790016970 con 0x7fc79815bea0 2024-09-17T05:39:01.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.963+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc79815b6a0 con 0x7fc79815bea0 2024-09-17T05:39:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc78e000640 1 -- 172.21.15.122:0/3099499828 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+13236 (secure 0 0 0) 0x7fc79012e140 con 0x7fc79815bea0 2024-09-17T05:39:01.960 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:01.960 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":124,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:39:00.613044+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6835/2875934571":"2024-09-18T05:38:57.602233+0000","172.21.15.122:6834/2875934571":"2024-09-18T05:38:57.602233+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7780751b0 msgr2=0x7fc778077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7780751b0 0x7fc778077670 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc7980b0f80 tx=0x7fc78800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79815bea0 msgr2=0x7fc7980b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79815bea0 0x7fc7980b5c40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc790009bd0 tx=0x7fc790003f60 comp rx=0 tx=0).stop 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 shutdown_connections 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc7780751b0 0x7fc778077670 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc79815bea0 0x7fc7980b5c40 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7980d28e0 0x7fc798153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 --2- 172.21.15.122:0/3099499828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7980cdb30 0x7fc79806f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:01.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 >> 172.21.15.122:0/3099499828 conn(0x7fc7980d1dc0 msgr2=0x7fc798159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:01.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 shutdown_connections 2024-09-17T05:39:01.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:01.967+0000 7fc79d200640 1 -- 172.21.15.122:0/3099499828 wait complete. 2024-09-17T05:39:01.981 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:39:01.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:39:02.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 -- 172.21.15.122:0/3008528878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f978c0d20f0 msgr2=0x7f978c0cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:02.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 --2- 172.21.15.122:0/3008528878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f978c0d20f0 0x7f978c0cc620 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9784009bc0 tx=0x7f978401cba0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 -- 172.21.15.122:0/3008528878 shutdown_connections 2024-09-17T05:39:02.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 --2- 172.21.15.122:0/3008528878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f978c0d20f0 0x7f978c0cc620 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 --2- 172.21.15.122:0/3008528878 >> v2:172.21.15.173:3300/0 conn(0x7f978c0b0d10 0x7f978c0d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 -- 172.21.15.122:0/3008528878 >> 172.21.15.122:0/3008528878 conn(0x7f978c09aa10 msgr2=0x7f978c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:02.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 -- 172.21.15.122:0/3008528878 shutdown_connections 2024-09-17T05:39:02.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 -- 172.21.15.122:0/3008528878 wait complete. 2024-09-17T05:39:02.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 Processor -- start 2024-09-17T05:39:02.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.131+0000 7f978be00640 1 -- start start 2024-09-17T05:39:02.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f978c0b0d10 0x7f978c1512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:02.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f978c0d20f0 0x7f978c150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:02.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f978c15bd40 0x7f978c1fb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:02.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f978c168d10 con 0x7f978c0b0d10 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f978c168b90 con 0x7f978c0d20f0 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f978c168e90 con 0x7f978c15bd40 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f978c0b0d10 0x7f978c1512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f978c0b0d10 0x7f978c1512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44470/0 (socket says 172.21.15.122:44470) 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 -- 172.21.15.122:0/224555195 learned_addr learned my addr 172.21.15.122:0/224555195 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f9782000640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f978c0d20f0 0x7f978c150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978b400640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f978c15bd40 0x7f978c1fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 -- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f978c15bd40 msgr2=0x7f978c1fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f978c15bd40 0x7f978c1fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 -- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f978c0d20f0 msgr2=0x7f978c150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f978c0d20f0 0x7f978c150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 -- 172.21.15.122:0/224555195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c153cb0 con 0x7f978c0b0d10 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978aa00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f978c0b0d10 0x7f978c1512a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c3e0 tx=0x7f977c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:02.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f9783400640 1 -- 172.21.15.122:0/224555195 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c011790 con 0x7f978c0b0d10 2024-09-17T05:39:02.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c153f70 con 0x7f978c0b0d10 2024-09-17T05:39:02.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f9783400640 1 -- 172.21.15.122:0/224555195 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f977c017070 con 0x7f978c0b0d10 2024-09-17T05:39:02.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f9783400640 1 -- 172.21.15.122:0/224555195 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c011ce0 con 0x7f978c0b0d10 2024-09-17T05:39:02.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c1fad70 con 0x7f978c0b0d10 2024-09-17T05:39:02.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c0d1bb0 con 0x7f978c0b0d10 2024-09-17T05:39:02.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.135+0000 7f9783400640 1 -- 172.21.15.122:0/224555195 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f977c01c460 con 0x7f978c0b0d10 2024-09-17T05:39:02.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.139+0000 7f9783400640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9768075180 0x7f9768077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:02.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.139+0000 7f9783400640 1 -- 172.21.15.122:0/224555195 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7f977c092f20 con 0x7f978c0b0d10 2024-09-17T05:39:02.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.139+0000 7f9782000640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9768075180 0x7f9768077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:02.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.139+0000 7f9782000640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9768075180 0x7f9768077640 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f978c1538a0 tx=0x7f97740091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:02.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.143+0000 7f9783400640 1 -- 172.21.15.122:0/224555195 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f977c05f3e0 con 0x7f978c0b0d10 2024-09-17T05:39:02.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.311+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f978c0724a0 con 0x7f978c0b0d10 2024-09-17T05:39:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f9783400640 1 -- 172.21.15.122:0/224555195 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) ==== 93+0+10 (secure 0 0 0) 0x7f977c063e10 con 0x7f978c0b0d10 2024-09-17T05:39:02.308 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:39:02.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9768075180 msgr2=0x7f9768077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:02.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9768075180 0x7f9768077640 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f978c1538a0 tx=0x7f97740091e0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f978c0b0d10 msgr2=0x7f978c1512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f978c0b0d10 0x7f978c1512a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c3e0 tx=0x7f977c00c8a0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 shutdown_connections 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9768075180 0x7f9768077640 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f978c15bd40 0x7f978c1fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f978c0d20f0 0x7f978c150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 --2- 172.21.15.122:0/224555195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f978c0b0d10 0x7f978c1512a0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 >> 172.21.15.122:0/224555195 conn(0x7f978c09aa10 msgr2=0x7f978c0ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 shutdown_connections 2024-09-17T05:39:02.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.315+0000 7f978be00640 1 -- 172.21.15.122:0/224555195 wait complete. 2024-09-17T05:39:02.330 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-17T05:39:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- 172.21.15.122:0/504189225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f740b49e0 msgr2=0x7f7f7406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 --2- 172.21.15.122:0/504189225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f740b49e0 0x7f7f7406f7b0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009c80 tx=0x7f7f6c01cd10 comp rx=0 tx=0).stop 2024-09-17T05:39:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- 172.21.15.122:0/504189225 shutdown_connections 2024-09-17T05:39:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 --2- 172.21.15.122:0/504189225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f740b49e0 0x7f7f7406f7b0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 --2- 172.21.15.122:0/504189225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f7415dae0 0x7f7f740acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- 172.21.15.122:0/504189225 >> 172.21.15.122:0/504189225 conn(0x7f7f740b18f0 msgr2=0x7f7f740b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- 172.21.15.122:0/504189225 shutdown_connections 2024-09-17T05:39:02.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- 172.21.15.122:0/504189225 wait complete. 2024-09-17T05:39:02.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 Processor -- start 2024-09-17T05:39:02.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- start start 2024-09-17T05:39:02.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f740b49e0 0x7f7f74163be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:02.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f740cd100 0x7f7f74144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:02.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f7415dae0 0x7f7f740af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:02.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f740cc9f0 con 0x7f7f7415dae0 2024-09-17T05:39:02.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f740cc870 con 0x7f7f740b49e0 2024-09-17T05:39:02.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7c600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7f740ccb70 con 0x7f7f740cd100 2024-09-17T05:39:02.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7a600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f7415dae0 0x7f7f740af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:02.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7a600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f7415dae0 0x7f7f740af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44488/0 (socket says 172.21.15.122:44488) 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f7a600640 1 -- 172.21.15.122:0/1732722838 learned_addr learned my addr 172.21.15.122:0/1732722838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.479+0000 7f7f79c00640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f740b49e0 0x7f7f74163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f79200640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f740cd100 0x7f7f74144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7a600640 1 -- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f740cd100 msgr2=0x7f7f74144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7a600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f740cd100 0x7f7f74144ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7a600640 1 -- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f740b49e0 msgr2=0x7f7f74163be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7a600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f740b49e0 0x7f7f74163be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7a600640 1 -- 172.21.15.122:0/1732722838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7409aab0 con 0x7f7f7415dae0 2024-09-17T05:39:02.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7a600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f7415dae0 0x7f7f740af860 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c01cb40 tx=0x7f7f6c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:02.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f71600640 1 -- 172.21.15.122:0/1732722838 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0bc070 con 0x7f7f7415dae0 2024-09-17T05:39:02.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f71600640 1 -- 172.21.15.122:0/1732722838 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f6c0044b0 con 0x7f7f7415dae0 2024-09-17T05:39:02.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7409ad20 con 0x7f7f7415dae0 2024-09-17T05:39:02.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f71600640 1 -- 172.21.15.122:0/1732722838 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c0b2040 con 0x7f7f7415dae0 2024-09-17T05:39:02.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f74158970 con 0x7f7f7415dae0 2024-09-17T05:39:02.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f74069410 con 0x7f7f7415dae0 2024-09-17T05:39:02.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.483+0000 7f7f71600640 1 -- 172.21.15.122:0/1732722838 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f7f6c0bf070 con 0x7f7f7415dae0 2024-09-17T05:39:02.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.487+0000 7f7f71600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f54075180 0x7f7f54077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:02.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.487+0000 7f7f71600640 1 -- 172.21.15.122:0/1732722838 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7f7f6c12d0b0 con 0x7f7f7415dae0 2024-09-17T05:39:02.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.487+0000 7f7f79c00640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f54075180 0x7f7f54077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:02.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.487+0000 7f7f79c00640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f54075180 0x7f7f54077640 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006fd0 tx=0x7f7f68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.491+0000 7f7f71600640 1 -- 172.21.15.122:0/1732722838 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7f6c016990 con 0x7f7f7415dae0 2024-09-17T05:39:02.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:02.715+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7f7f740acd40 con 0x7f7f7415dae0 2024-09-17T05:39:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.623+0000 7f7f71600640 1 -- 172.21.15.122:0/1732722838 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v638) ==== 189+0+0 (secure 0 0 0) 0x7f7f6c0f9c40 con 0x7f7f7415dae0 2024-09-17T05:39:03.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f54075180 msgr2=0x7f7f54077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:03.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f54075180 0x7f7f54077640 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006fd0 tx=0x7f7f68008040 comp rx=0 tx=0).stop 2024-09-17T05:39:03.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f7415dae0 msgr2=0x7f7f740af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:03.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f7415dae0 0x7f7f740af860 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c01cb40 tx=0x7f7f6c003f60 comp rx=0 tx=0).stop 2024-09-17T05:39:03.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 shutdown_connections 2024-09-17T05:39:03.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7f54075180 0x7f7f54077640 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f7415dae0 0x7f7f740af860 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f740cd100 0x7f7f74144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 --2- 172.21.15.122:0/1732722838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f740b49e0 0x7f7f74163be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 >> 172.21.15.122:0/1732722838 conn(0x7f7f740b18f0 msgr2=0x7f7f7415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:03.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 shutdown_connections 2024-09-17T05:39:03.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.627+0000 7f7f7c600640 1 -- 172.21.15.122:0/1732722838 wait complete. 2024-09-17T05:39:03.646 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:39:03.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c3a00640 1 -- 172.21.15.122:0/1923789811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a0004d50 msgr2=0x7f40a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:03.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c3a00640 1 --2- 172.21.15.122:0/1923789811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a0004d50 0x7f40a00051d0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f40b400c470 tx=0x7f40b401f6a0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c1a00640 1 -- 172.21.15.122:0/1923789811 reap_dead start 2024-09-17T05:39:03.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c3a00640 1 -- 172.21.15.122:0/1923789811 shutdown_connections 2024-09-17T05:39:03.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c3a00640 1 -- 172.21.15.122:0/1923789811 >> 172.21.15.122:0/1923789811 conn(0x7f40bc0b2890 msgr2=0x7f40bc0b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:03.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c3a00640 1 -- 172.21.15.122:0/1923789811 shutdown_connections 2024-09-17T05:39:03.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c3a00640 1 -- 172.21.15.122:0/1923789811 wait complete. 2024-09-17T05:39:03.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c3a00640 1 Processor -- start 2024-09-17T05:39:03.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.799+0000 7f40c3a00640 1 -- start start 2024-09-17T05:39:03.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a0004d50 0x7f40bc0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:03.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40bc0ccda0 0x7f40bc15e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:03.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40bc15e810 0x7f40bc148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:03.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc0ae0e0 con 0x7f40bc15e810 2024-09-17T05:39:03.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc0adf60 con 0x7f40a0004d50 2024-09-17T05:39:03.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc0b42a0 con 0x7f40bc0ccda0 2024-09-17T05:39:03.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40bc15e810 0x7f40bc148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:03.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40bc15e810 0x7f40bc148690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44506/0 (socket says 172.21.15.122:44506) 2024-09-17T05:39:03.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 -- 172.21.15.122:0/3670825040 learned_addr learned my addr 172.21.15.122:0/3670825040 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:03.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40bbe00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40bc0ccda0 0x7f40bc15e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:03.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1000640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a0004d50 0x7f40bc0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:03.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 -- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40bc0ccda0 msgr2=0x7f40bc15e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:03.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40bc0ccda0 0x7f40bc15e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 -- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a0004d50 msgr2=0x7f40bc0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:03.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a0004d50 0x7f40bc0ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 -- 172.21.15.122:0/3670825040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc1466a0 con 0x7f40bc15e810 2024-09-17T05:39:03.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40bbe00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40bc0ccda0 0x7f40bc15e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:03.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1000640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a0004d50 0x7f40bc0ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:03.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c1a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40bc15e810 0x7f40bc148690 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f40b40263b0 tx=0x7f40b4006620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:03.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40b8c00640 1 -- 172.21.15.122:0/3670825040 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b40be070 con 0x7f40bc15e810 2024-09-17T05:39:03.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40b8c00640 1 -- 172.21.15.122:0/3670825040 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40b40b4040 con 0x7f40bc15e810 2024-09-17T05:39:03.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40b8c00640 1 -- 172.21.15.122:0/3670825040 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b40b3070 con 0x7f40bc15e810 2024-09-17T05:39:03.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc0ce590 con 0x7f40bc15e810 2024-09-17T05:39:03.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc146910 con 0x7f40bc15e810 2024-09-17T05:39:03.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40bc06d0d0 con 0x7f40bc15e810 2024-09-17T05:39:03.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.803+0000 7f40b8c00640 1 -- 172.21.15.122:0/3670825040 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f40b40c1070 con 0x7f40bc15e810 2024-09-17T05:39:03.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.807+0000 7f40b8c00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f409c0751b0 0x7f409c077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:03.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.807+0000 7f40b8c00640 1 -- 172.21.15.122:0/3670825040 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7f40b412faf0 con 0x7f40bc15e810 2024-09-17T05:39:03.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.807+0000 7f40c1000640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f409c0751b0 0x7f409c077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:03.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.807+0000 7f40c1000640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f409c0751b0 0x7f409c077670 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f40ac012450 tx=0x7f40ac009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:03.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.811+0000 7f40b8c00640 1 -- 172.21.15.122:0/3670825040 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f40bc1565a0 con 0x7f40bc15e810 2024-09-17T05:39:03.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f40bc0b07f0 con 0x7f40bc15e810 2024-09-17T05:39:03.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40b8c00640 1 -- 172.21.15.122:0/3670825040 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v50) ==== 59+0+1998 (secure 0 0 0) 0x7f40b40fbfc0 con 0x7f40bc15e810 2024-09-17T05:39:03.973 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:39:03.973 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:39:03.973 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:39:03.974 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:39:03.975 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:39:03.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f409c0751b0 msgr2=0x7f409c077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:03.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f409c0751b0 0x7f409c077670 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f40ac012450 tx=0x7f40ac009500 comp rx=0 tx=0).stop 2024-09-17T05:39:03.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40bc15e810 msgr2=0x7f40bc148690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40bc15e810 0x7f40bc148690 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f40b40263b0 tx=0x7f40b4006620 comp rx=0 tx=0).stop 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 shutdown_connections 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f409c0751b0 0x7f409c077670 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40bc15e810 0x7f40bc148690 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f40bc0ccda0 0x7f40bc15e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 --2- 172.21.15.122:0/3670825040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40a0004d50 0x7f40bc0ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 >> 172.21.15.122:0/3670825040 conn(0x7f40bc0b2890 msgr2=0x7f40bc0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 shutdown_connections 2024-09-17T05:39:03.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:03.979+0000 7f40c3a00640 1 -- 172.21.15.122:0/3670825040 wait complete. 2024-09-17T05:39:03.995 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:39:04.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:39:04.114 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:39:04.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 -- 172.21.15.122:0/1574123541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0b3f70 msgr2=0x7f70ac0cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:04.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 --2- 172.21.15.122:0/1574123541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0b3f70 0x7f70ac0cc850 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009bc0 tx=0x7f70a401cb10 comp rx=0 tx=0).stop 2024-09-17T05:39:04.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 -- 172.21.15.122:0/1574123541 shutdown_connections 2024-09-17T05:39:04.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 --2- 172.21.15.122:0/1574123541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0b3f70 0x7f70ac0cc850 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 --2- 172.21.15.122:0/1574123541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b2990 0x7f70ac0d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 -- 172.21.15.122:0/1574123541 >> 172.21.15.122:0/1574123541 conn(0x7f70ac0d1e00 msgr2=0x7f70ac0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:04.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 -- 172.21.15.122:0/1574123541 shutdown_connections 2024-09-17T05:39:04.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 -- 172.21.15.122:0/1574123541 wait complete. 2024-09-17T05:39:04.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 Processor -- start 2024-09-17T05:39:04.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.147+0000 7f70b0a00640 1 -- start start 2024-09-17T05:39:04.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac0b2990 0x7f70ac0af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:04.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3f70 0x7f70ac0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:04.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0d05b0 0x7f70ac156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:04.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac169150 con 0x7f70ac0d05b0 2024-09-17T05:39:04.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac0cdaf0 con 0x7f70ac0b2990 2024-09-17T05:39:04.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f70ac1692d0 con 0x7f70ac0b3f70 2024-09-17T05:39:04.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0d05b0 0x7f70ac156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:04.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0d05b0 0x7f70ac156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44518/0 (socket says 172.21.15.122:44518) 2024-09-17T05:39:04.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 -- 172.21.15.122:0/3375079153 learned_addr learned my addr 172.21.15.122:0/3375079153 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:04.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70a9e00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac0b2990 0x7f70ac0af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:04.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70a3e00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3f70 0x7f70ac0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:04.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 -- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3f70 msgr2=0x7f70ac0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:04.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3f70 0x7f70ac0afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 -- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac0b2990 msgr2=0x7f70ac0af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:04.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac0b2990 0x7f70ac0af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 -- 172.21.15.122:0/3375079153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac09ee90 con 0x7f70ac0d05b0 2024-09-17T05:39:04.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70a3e00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3f70 0x7f70ac0afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:04.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70a9e00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac0b2990 0x7f70ac0af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:04.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70aa800640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0d05b0 0x7f70ac156f90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f70a401c580 tx=0x7f70a4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:04.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70a2000640 1 -- 172.21.15.122:0/3375079153 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a40b2040 con 0x7f70ac0d05b0 2024-09-17T05:39:04.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70a2000640 1 -- 172.21.15.122:0/3375079153 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70a40bb070 con 0x7f70ac0d05b0 2024-09-17T05:39:04.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70a2000640 1 -- 172.21.15.122:0/3375079153 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4004370 con 0x7f70ac0d05b0 2024-09-17T05:39:04.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac09f170 con 0x7f70ac0d05b0 2024-09-17T05:39:04.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac0c8310 con 0x7f70ac0d05b0 2024-09-17T05:39:04.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.151+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ac09f3b0 con 0x7f70ac0d05b0 2024-09-17T05:39:04.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.155+0000 7f70a2000640 1 -- 172.21.15.122:0/3375079153 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f70a4004510 con 0x7f70ac0d05b0 2024-09-17T05:39:04.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.155+0000 7f70a2000640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880751b0 0x7f7088077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:04.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.155+0000 7f70a2000640 1 -- 172.21.15.122:0/3375079153 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7f70a412dae0 con 0x7f70ac0d05b0 2024-09-17T05:39:04.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.155+0000 7f70a9e00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880751b0 0x7f7088077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:04.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.155+0000 7f70a9e00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880751b0 0x7f7088077670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f709c007a60 tx=0x7f709c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:04.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.159+0000 7f70a2000640 1 -- 172.21.15.122:0/3375079153 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f70ac09f3b0 con 0x7f70ac0d05b0 2024-09-17T05:39:04.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.327+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f70ac0d0990 con 0x7f70ac0d05b0 2024-09-17T05:39:04.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.331+0000 7f70a2000640 1 -- 172.21.15.122:0/3375079153 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f70a4016790 con 0x7f70ac0d05b0 2024-09-17T05:39:04.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880751b0 msgr2=0x7f7088077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:04.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880751b0 0x7f7088077670 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f709c007a60 tx=0x7f709c009040 comp rx=0 tx=0).stop 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0d05b0 msgr2=0x7f70ac156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0d05b0 0x7f70ac156f90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f70a401c580 tx=0x7f70a4005b00 comp rx=0 tx=0).stop 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 shutdown_connections 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880751b0 0x7f7088077670 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0d05b0 0x7f70ac156f90 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3f70 0x7f70ac0afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 --2- 172.21.15.122:0/3375079153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac0b2990 0x7f70ac0af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 >> 172.21.15.122:0/3375079153 conn(0x7f70ac0d1e00 msgr2=0x7f70ac0b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 shutdown_connections 2024-09-17T05:39:04.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.335+0000 7f70b0a00640 1 -- 172.21.15.122:0/3375079153 wait complete. 2024-09-17T05:39:04.347 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:39:04.348 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:39:04.442 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:39:04.538 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:39:04.539 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:39:04.539 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:39:04.547 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:39:04.548 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:39:04.548 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:39:04.558 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:39:04.558 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:39:04.558 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:39:04.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:39:04.599 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:39:04.599 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:39:04.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:39:04.612 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:39:04.612 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:39:04.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 -- 172.21.15.122:0/1017923819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb00d28e0 msgr2=0x7fdbb00cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:04.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 --2- 172.21.15.122:0/1017923819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb00d28e0 0x7fdbb00cc9d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00745d0 tx=0x7fdba801cd80 comp rx=0 tx=0).stop 2024-09-17T05:39:04.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 -- 172.21.15.122:0/1017923819 shutdown_connections 2024-09-17T05:39:04.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 --2- 172.21.15.122:0/1017923819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb00d28e0 0x7fdbb00cc9d0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 --2- 172.21.15.122:0/1017923819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbb00cdb30 0x7fdbb00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 --2- 172.21.15.122:0/1017923819 >> v2:172.21.15.173:3300/0 conn(0x7fdbb015bea0 0x7fdbb00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 -- 172.21.15.122:0/1017923819 >> 172.21.15.122:0/1017923819 conn(0x7fdbb00d1dc0 msgr2=0x7fdbb00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:04.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 -- 172.21.15.122:0/1017923819 shutdown_connections 2024-09-17T05:39:04.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.791+0000 7fdbb8200640 1 -- 172.21.15.122:0/1017923819 wait complete. 2024-09-17T05:39:04.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 Processor -- start 2024-09-17T05:39:04.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 -- start start 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbb00cdb30 0x7fdbb01524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbb00d28e0 0x7fdbb00ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb015bea0 0x7fdbb0157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb00af120 con 0x7fdbb015bea0 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb00aefa0 con 0x7fdbb00cdb30 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb00af2a0 con 0x7fdbb00d28e0 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb015bea0 0x7fdbb0157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb015bea0 0x7fdbb0157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44534/0 (socket says 172.21.15.122:44534) 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 -- 172.21.15.122:0/3108411879 learned_addr learned my addr 172.21.15.122:0/3108411879 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb4e00640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbb00d28e0 0x7fdbb00ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:04.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb5800640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbb00cdb30 0x7fdbb01524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:04.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 -- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbb00d28e0 msgr2=0x7fdbb00ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:04.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbb00d28e0 0x7fdbb00ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 -- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbb00cdb30 msgr2=0x7fdbb01524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:04.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbb00cdb30 0x7fdbb01524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:04.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 -- 172.21.15.122:0/3108411879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb00cac20 con 0x7fdbb015bea0 2024-09-17T05:39:04.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb6200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb015bea0 0x7fdbb0157f80 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fdba801b430 tx=0x7fdba8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:04.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbad600640 1 -- 172.21.15.122:0/3108411879 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80bb070 con 0x7fdbb015bea0 2024-09-17T05:39:04.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbad600640 1 -- 172.21.15.122:0/3108411879 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdba8004e00 con 0x7fdbb015bea0 2024-09-17T05:39:04.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbad600640 1 -- 172.21.15.122:0/3108411879 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80b2040 con 0x7fdbb015bea0 2024-09-17T05:39:04.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb01558c0 con 0x7fdbb015bea0 2024-09-17T05:39:04.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb0155020 con 0x7fdbb015bea0 2024-09-17T05:39:04.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.795+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb00af830 con 0x7fdbb015bea0 2024-09-17T05:39:04.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.799+0000 7fdbad600640 1 -- 172.21.15.122:0/3108411879 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fdba80be070 con 0x7fdbb015bea0 2024-09-17T05:39:04.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.799+0000 7fdbad600640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb90075180 0x7fdb90077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:04.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.799+0000 7fdbad600640 1 -- 172.21.15.122:0/3108411879 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7fdba812d310 con 0x7fdbb015bea0 2024-09-17T05:39:04.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.799+0000 7fdbb5800640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb90075180 0x7fdb90077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:04.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.799+0000 7fdbb5800640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb90075180 0x7fdb90077640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdbb01f8de0 tx=0x7fdba400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:04.803+0000 7fdbad600640 1 -- 172.21.15.122:0/3108411879 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdba8016970 con 0x7fdbb015bea0 2024-09-17T05:39:04.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.003+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb0156980 con 0x7fdbb015bea0 2024-09-17T05:39:05.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbad600640 1 -- 172.21.15.122:0/3108411879 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+3920 (secure 0 0 0) 0x7fdba80f9850 con 0x7fdbb015bea0 2024-09-17T05:39:05.000 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:05.000 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":79,"btime":"2024-09-17T05:38:58:604470+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6968,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/4034540416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":4034540416},{"type":"v1","addr":"172.21.15.173:6836","nonce":4034540416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7651,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4130637584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4130637584},{"type":"v1","addr":"172.21.15.173:6837","nonce":4130637584}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/543665156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":543665156},{"type":"v1","addr":"172.21.15.122:6833","nonce":543665156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2035476779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2035476779},{"type":"v1","addr":"172.21.15.122:6835","nonce":2035476779}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[]} 2024-09-17T05:39:05.001 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 79 2024-09-17T05:39:05.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb90075180 msgr2=0x7fdb90077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb90075180 0x7fdb90077640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdbb01f8de0 tx=0x7fdba400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb015bea0 msgr2=0x7fdbb0157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb015bea0 0x7fdbb0157f80 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fdba801b430 tx=0x7fdba8003f60 comp rx=0 tx=0).stop 2024-09-17T05:39:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 shutdown_connections 2024-09-17T05:39:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdb90075180 0x7fdb90077640 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbb015bea0 0x7fdbb0157f80 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbb00d28e0 0x7fdbb00ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 --2- 172.21.15.122:0/3108411879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbb00cdb30 0x7fdbb01524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 >> 172.21.15.122:0/3108411879 conn(0x7fdbb00d1dc0 msgr2=0x7fdbb0159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 shutdown_connections 2024-09-17T05:39:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.007+0000 7fdbb8200640 1 -- 172.21.15.122:0/3108411879 wait complete. 2024-09-17T05:39:05.021 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:39:05.021 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:39:05.022 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:39:05.022 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:39:05.022 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:39:05.022 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:39:05.022 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:39:05.022 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:39:05.022 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:39:05.023 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:39:05.023 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:39:05.023 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:39:05.023 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:39:05.023 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:39:05.023 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:39:05.023 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:39:05.033 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:39:05.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:39:05.034 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:39:05.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:39:05.038 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:39:05.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:39:05.039 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:39:05.039 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:39:05.078 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:39:05.078 DEBUG:teuthology.parallel:result is None 2024-09-17T05:39:05.080 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:39:05.080 DEBUG:teuthology.parallel:result is None 2024-09-17T05:39:05.083 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:39:05.083 DEBUG:teuthology.parallel:result is None 2024-09-17T05:39:05.084 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:39:05.084 DEBUG:teuthology.parallel:result is None 2024-09-17T05:39:05.085 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:39:05.164 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:39:05.165 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:39:05.166 INFO:tasks.ceph.mds.c.smithi122.stderr:2024-09-17T05:39:05.171+0000 7f4798400640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-17T05:39:05.173 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:39:05.174 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:39:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.239+0000 7fbee0800640 1 -- 172.21.15.122:0/1396692019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed80cfe80 msgr2=0x7fbed80d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.239+0000 7fbee0800640 1 --2- 172.21.15.122:0/1396692019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed80cfe80 0x7fbed80d0260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009840 tx=0x7fbecc01c6b0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.239+0000 7fbee0800640 1 -- 172.21.15.122:0/1396692019 shutdown_connections 2024-09-17T05:39:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.239+0000 7fbee0800640 1 --2- 172.21.15.122:0/1396692019 >> v2:172.21.15.173:3300/0 conn(0x7fbed80b2760 0x7fbed81605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.239+0000 7fbee0800640 1 --2- 172.21.15.122:0/1396692019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed80cfe80 0x7fbed80d0260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.239+0000 7fbee0800640 1 -- 172.21.15.122:0/1396692019 >> 172.21.15.122:0/1396692019 conn(0x7fbed80d1e40 msgr2=0x7fbed80d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.239+0000 7fbee0800640 1 -- 172.21.15.122:0/1396692019 shutdown_connections 2024-09-17T05:39:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.239+0000 7fbee0800640 1 -- 172.21.15.122:0/1396692019 wait complete. 2024-09-17T05:39:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 Processor -- start 2024-09-17T05:39:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 -- start start 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed80b2760 0x7fbed80b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed80b2d00 0x7fbed815ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbed80cfe80 0x7fbed81583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbed80b0940 con 0x7fbed80cfe80 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbed80b07c0 con 0x7fbed80b2760 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbed80b0ac0 con 0x7fbed80b2d00 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbed80cfe80 0x7fbed81583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbed80cfe80 0x7fbed81583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44568/0 (socket says 172.21.15.122:44568) 2024-09-17T05:39:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 -- 172.21.15.122:0/956171307 learned_addr learned my addr 172.21.15.122:0/956171307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbedd400640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed80b2d00 0x7fbed815ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbedde00640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed80b2760 0x7fbed80b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 -- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed80b2d00 msgr2=0x7fbed815ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed80b2d00 0x7fbed815ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 -- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed80b2760 msgr2=0x7fbed80b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed80b2760 0x7fbed80b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 -- 172.21.15.122:0/956171307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed8160760 con 0x7fbed80cfe80 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbedde00640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed80b2760 0x7fbed80b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbede800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbed80cfe80 0x7fbed81583a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009ca0 tx=0x7fbed400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbed2000640 1 -- 172.21.15.122:0/956171307 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4010630 con 0x7fbed80cfe80 2024-09-17T05:39:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbed2000640 1 -- 172.21.15.122:0/956171307 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbed40108e0 con 0x7fbed80cfe80 2024-09-17T05:39:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed8153850 con 0x7fbed80cfe80 2024-09-17T05:39:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbed2000640 1 -- 172.21.15.122:0/956171307 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4010d30 con 0x7fbed80cfe80 2024-09-17T05:39:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.243+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed806e0b0 con 0x7fbed80cfe80 2024-09-17T05:39:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.247+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed80d0260 con 0x7fbed80cfe80 2024-09-17T05:39:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.247+0000 7fbed2000640 1 -- 172.21.15.122:0/956171307 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fbed401f020 con 0x7fbed80cfe80 2024-09-17T05:39:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.247+0000 7fbed2000640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbebc0751b0 0x7fbebc077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.247+0000 7fbed2000640 1 -- 172.21.15.122:0/956171307 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5027+0+0 (secure 0 0 0) 0x7fbed4093dd0 con 0x7fbed80cfe80 2024-09-17T05:39:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.247+0000 7fbedde00640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbebc0751b0 0x7fbebc077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.247+0000 7fbedde00640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbebc0751b0 0x7fbebc077670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0ae000 tx=0x7fbecc01aef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:05.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.251+0000 7fbed2000640 1 -- 172.21.15.122:0/956171307 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbed4060860 con 0x7fbed80cfe80 2024-09-17T05:39:05.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.431+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbed8070fb0 con 0x7fbed80cfe80 2024-09-17T05:39:05.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbed2000640 1 -- 172.21.15.122:0/956171307 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) ==== 97+0+0 (secure 0 0 0) 0x7fbed4060160 con 0x7fbed80cfe80 2024-09-17T05:39:05.641 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:39:05.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbebc0751b0 msgr2=0x7fbebc077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbebc0751b0 0x7fbebc077670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0ae000 tx=0x7fbecc01aef0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbed80cfe80 msgr2=0x7fbed81583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbed80cfe80 0x7fbed81583a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009ca0 tx=0x7fbed400c6a0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 shutdown_connections 2024-09-17T05:39:05.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbebc0751b0 0x7fbebc077670 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbed80cfe80 0x7fbed81583a0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed80b2d00 0x7fbed815ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 --2- 172.21.15.122:0/956171307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed80b2760 0x7fbed80b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 >> 172.21.15.122:0/956171307 conn(0x7fbed80d1e40 msgr2=0x7fbed8159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:05.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 shutdown_connections 2024-09-17T05:39:05.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.647+0000 7fbee0800640 1 -- 172.21.15.122:0/956171307 wait complete. 2024-09-17T05:39:05.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 -- 172.21.15.122:0/716049348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 msgr2=0x7fc5640b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 --2- 172.21.15.122:0/716049348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 0x7fc5640b1bd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc550009870 tx=0x7fc55001c710 comp rx=0 tx=0).stop 2024-09-17T05:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 -- 172.21.15.122:0/716049348 shutdown_connections 2024-09-17T05:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 --2- 172.21.15.122:0/716049348 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 0x7fc5640b1bd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 -- 172.21.15.122:0/716049348 >> 172.21.15.122:0/716049348 conn(0x7fc5640d1e30 msgr2=0x7fc5640d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 -- 172.21.15.122:0/716049348 shutdown_connections 2024-09-17T05:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 -- 172.21.15.122:0/716049348 wait complete. 2024-09-17T05:39:05.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 Processor -- start 2024-09-17T05:39:05.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.811+0000 7fc56c000640 1 -- start start 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5640afca0 0x7fc5641506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 0x7fc5640b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56415f490 0x7fc564150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5640b2290 con 0x7fc56415f490 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc5640b2110 con 0x7fc5640afca0 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc56416a4a0 con 0x7fc5640b2e80 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56415f490 0x7fc564150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56415f490 0x7fc564150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44584/0 (socket says 172.21.15.122:44584) 2024-09-17T05:39:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 -- 172.21.15.122:0/77883879 learned_addr learned my addr 172.21.15.122:0/77883879 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc568c00640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 0x7fc5640b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc569600640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5640afca0 0x7fc5641506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 -- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 msgr2=0x7fc5640b6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 0x7fc5640b6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 -- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5640afca0 msgr2=0x7fc5641506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5640afca0 0x7fc5641506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 -- 172.21.15.122:0/77883879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5640b5de0 con 0x7fc56415f490 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc568c00640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 0x7fc5640b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56a000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56415f490 0x7fc564150d40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00c410 tx=0x7fc55c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc561600640 1 -- 172.21.15.122:0/77883879 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c017070 con 0x7fc56415f490 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc561600640 1 -- 172.21.15.122:0/77883879 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc55c0053c0 con 0x7fc56415f490 2024-09-17T05:39:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc561600640 1 -- 172.21.15.122:0/77883879 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c003860 con 0x7fc56415f490 2024-09-17T05:39:05.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5640b6080 con 0x7fc56415f490 2024-09-17T05:39:05.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5640b6640 con 0x7fc56415f490 2024-09-17T05:39:05.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.815+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5640b0130 con 0x7fc56415f490 2024-09-17T05:39:05.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.819+0000 7fc561600640 1 -- 172.21.15.122:0/77883879 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fc55c01a070 con 0x7fc56415f490 2024-09-17T05:39:05.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.819+0000 7fc561600640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc544075180 0x7fc544077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:05.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.819+0000 7fc561600640 1 -- 172.21.15.122:0/77883879 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4941+0+0 (secure 0 0 0) 0x7fc55c092610 con 0x7fc56415f490 2024-09-17T05:39:05.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.819+0000 7fc569600640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc544075180 0x7fc544077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:05.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.819+0000 7fc569600640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc544075180 0x7fc544077640 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc5640cc2a0 tx=0x7fc558001e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:05.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:05.823+0000 7fc561600640 1 -- 172.21.15.122:0/77883879 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc55c05ead0 con 0x7fc56415f490 2024-09-17T05:39:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.047+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc564070ee0 con 0x7fc56415f490 2024-09-17T05:39:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.047+0000 7fc561600640 1 -- 172.21.15.122:0/77883879 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7fc55c063500 con 0x7fc56415f490 2024-09-17T05:39:06.043 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:06.043 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:39:06.043 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:39:06.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.043 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:39:06.043 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:39:06.043 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:39:06.044 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.045 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:39:06.047 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.049 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.050 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:39:06.052 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow r", 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:39:06.053 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:39:06.055 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:39:06.056 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:39:06.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.058 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.058 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:39:06.058 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc544075180 msgr2=0x7fc544077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc544075180 0x7fc544077640 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc5640cc2a0 tx=0x7fc558001e40 comp rx=0 tx=0).stop 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56415f490 msgr2=0x7fc564150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56415f490 0x7fc564150d40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00c410 tx=0x7fc55c00c8d0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 shutdown_connections 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc544075180 0x7fc544077640 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc56415f490 0x7fc564150d40 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5640b2e80 0x7fc5640b6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 --2- 172.21.15.122:0/77883879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5640afca0 0x7fc5641506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 >> 172.21.15.122:0/77883879 conn(0x7fc5640d1e30 msgr2=0x7fc5640ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 shutdown_connections 2024-09-17T05:39:06.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.051+0000 7fc56c000640 1 -- 172.21.15.122:0/77883879 wait complete. 2024-09-17T05:39:06.065 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:39:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- 172.21.15.122:0/1911124290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c80cdb30 msgr2=0x7fb7c80aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 --2- 172.21.15.122:0/1911124290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c80cdb30 0x7fb7c80aeef0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009bc0 tx=0x7fb7c001cba0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- 172.21.15.122:0/1911124290 shutdown_connections 2024-09-17T05:39:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 --2- 172.21.15.122:0/1911124290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c80cdb30 0x7fb7c80aeef0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 --2- 172.21.15.122:0/1911124290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7c80ced80 0x7fb7c80ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- 172.21.15.122:0/1911124290 >> 172.21.15.122:0/1911124290 conn(0x7fb7c80d1dc0 msgr2=0x7fb7c80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- 172.21.15.122:0/1911124290 shutdown_connections 2024-09-17T05:39:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- 172.21.15.122:0/1911124290 wait complete. 2024-09-17T05:39:06.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 Processor -- start 2024-09-17T05:39:06.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- start start 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7c80cdb30 0x7fb7c81f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c80ced80 0x7fb7c8156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c815bea0 0x7fb7c8158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8169290 con 0x7fb7c80ced80 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c8169110 con 0x7fb7c815bea0 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7ce200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c8169410 con 0x7fb7c80cdb30 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7bea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c80ced80 0x7fb7c8156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7bea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c80ced80 0x7fb7c8156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44588/0 (socket says 172.21.15.122:44588) 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7bea00640 1 -- 172.21.15.122:0/690909076 learned_addr learned my addr 172.21.15.122:0/690909076 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:06.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7c7400640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7c80cdb30 0x7fb7c81f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7bea00640 1 -- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7c80cdb30 msgr2=0x7fb7c81f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7bea00640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7c80cdb30 0x7fb7c81f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7bea00640 1 -- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c815bea0 msgr2=0x7fb7c8158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.215+0000 7fb7c7e00640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c815bea0 0x7fb7c8158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bea00640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c815bea0 0x7fb7c8158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bea00640 1 -- 172.21.15.122:0/690909076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c81fb120 con 0x7fb7c80ced80 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7c7400640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7c80cdb30 0x7fb7c81f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7c7e00640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c815bea0 0x7fb7c8158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bea00640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c80ced80 0x7fb7c8156c20 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000ea20 tx=0x7fb7b000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bfe00640 1 -- 172.21.15.122:0/690909076 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000cd80 con 0x7fb7c80ced80 2024-09-17T05:39:06.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bfe00640 1 -- 172.21.15.122:0/690909076 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7b0010430 con 0x7fb7c80ced80 2024-09-17T05:39:06.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bfe00640 1 -- 172.21.15.122:0/690909076 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0010860 con 0x7fb7c80ced80 2024-09-17T05:39:06.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c81fb400 con 0x7fb7c80ced80 2024-09-17T05:39:06.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c81fb9c0 con 0x7fb7c80ced80 2024-09-17T05:39:06.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c80ce670 con 0x7fb7c80ced80 2024-09-17T05:39:06.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bfe00640 1 -- 172.21.15.122:0/690909076 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fb7b0010a00 con 0x7fb7c80ced80 2024-09-17T05:39:06.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bfe00640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7a40751b0 0x7fb7a4077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:06.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.219+0000 7fb7bfe00640 1 -- 172.21.15.122:0/690909076 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4941+0+0 (secure 0 0 0) 0x7fb7b0094610 con 0x7fb7c80ced80 2024-09-17T05:39:06.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.223+0000 7fb7c7400640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7a40751b0 0x7fb7a4077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:06.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.223+0000 7fb7c7400640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7a40751b0 0x7fb7a4077670 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8001230 tx=0x7fb7b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:06.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.223+0000 7fb7bfe00640 1 -- 172.21.15.122:0/690909076 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb7b00611a0 con 0x7fb7c80ced80 2024-09-17T05:39:06.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb7c80d23f0 con 0x7fb7c80ced80 2024-09-17T05:39:06.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7bfe00640 1 -- 172.21.15.122:0/690909076 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) ==== 84+0+65 (secure 0 0 0) 0x7fb7b0060aa0 con 0x7fb7c80ced80 2024-09-17T05:39:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:06.389 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:39:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:39:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:39:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:06.389 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:39:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7a40751b0 msgr2=0x7fb7a4077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7a40751b0 0x7fb7a4077670 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8001230 tx=0x7fb7b8009040 comp rx=0 tx=0).stop 2024-09-17T05:39:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c80ced80 msgr2=0x7fb7c8156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c80ced80 0x7fb7c8156c20 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000ea20 tx=0x7fb7b000eee0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 shutdown_connections 2024-09-17T05:39:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb7a40751b0 0x7fb7a4077670 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7c815bea0 0x7fb7c8158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7c80ced80 0x7fb7c8156c20 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 --2- 172.21.15.122:0/690909076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7c80cdb30 0x7fb7c81f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 >> 172.21.15.122:0/690909076 conn(0x7fb7c80d1dc0 msgr2=0x7fb7c8071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:06.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 shutdown_connections 2024-09-17T05:39:06.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.395+0000 7fb7ce200640 1 -- 172.21.15.122:0/690909076 wait complete. 2024-09-17T05:39:06.411 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:39:06.412 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:39:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- 172.21.15.122:0/2100521813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c0b0b60 msgr2=0x7f8e3c0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 --2- 172.21.15.122:0/2100521813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c0b0b60 0x7f8e3c0ccbd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009c80 tx=0x7f8e3401f5a0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- 172.21.15.122:0/2100521813 shutdown_connections 2024-09-17T05:39:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 --2- 172.21.15.122:0/2100521813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c0b0b60 0x7f8e3c0ccbd0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 --2- 172.21.15.122:0/2100521813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3c15be10 0x7f8e3c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- 172.21.15.122:0/2100521813 >> 172.21.15.122:0/2100521813 conn(0x7f8e3c0d1da0 msgr2=0x7f8e3c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- 172.21.15.122:0/2100521813 shutdown_connections 2024-09-17T05:39:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- 172.21.15.122:0/2100521813 wait complete. 2024-09-17T05:39:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 Processor -- start 2024-09-17T05:39:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- start start 2024-09-17T05:39:06.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3c0b0b60 0x7f8e3c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:06.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3c15be10 0x7f8e3c0b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:06.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c15c1f0 0x7f8e3c0b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c0cd4a0 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c0cd320 con 0x7f8e3c0b0b60 2024-09-17T05:39:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e44600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c0cd620 con 0x7f8e3c15be10 2024-09-17T05:39:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e42600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c15c1f0 0x7f8e3c0b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e42600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c15c1f0 0x7f8e3c0b7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44594/0 (socket says 172.21.15.122:44594) 2024-09-17T05:39:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.562+0000 7f8e42600640 1 -- 172.21.15.122:0/1517498833 learned_addr learned my addr 172.21.15.122:0/1517498833 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e42600640 1 -- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3c15be10 msgr2=0x7f8e3c0b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e41c00640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3c0b0b60 0x7f8e3c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e41200640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3c15be10 0x7f8e3c0b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e42600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3c15be10 0x7f8e3c0b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e42600640 1 -- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3c0b0b60 msgr2=0x7f8e3c156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e42600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3c0b0b60 0x7f8e3c156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e42600640 1 -- 172.21.15.122:0/1517498833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c157360 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e41c00640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3c0b0b60 0x7f8e3c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e41200640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3c15be10 0x7f8e3c0b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:39:06.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e42600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c15c1f0 0x7f8e3c0b7520 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009bd0 tx=0x7f8e34009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e39600640 1 -- 172.21.15.122:0/1517498833 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34004e60 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e39600640 1 -- 172.21.15.122:0/1517498833 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e340be030 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e39600640 1 -- 172.21.15.122:0/1517498833 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340afb00 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c159ed0 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c156e90 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3c0c6c60 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.566+0000 7f8e39600640 1 -- 172.21.15.122:0/1517498833 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f8e340afca0 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.570+0000 7f8e39600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e1c0751b0 0x7f8e1c077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.570+0000 7f8e39600640 1 -- 172.21.15.122:0/1517498833 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4941+0+0 (secure 0 0 0) 0x7f8e34131db0 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.570+0000 7f8e41c00640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e1c0751b0 0x7f8e1c077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:06.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.570+0000 7f8e41c00640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e1c0751b0 0x7f8e1c077670 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c157f90 tx=0x7f8e300091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:06.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.574+0000 7f8e39600640 1 -- 172.21.15.122:0/1517498833 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8e340ac050 con 0x7f8e3c15c1f0 2024-09-17T05:39:06.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:06.746+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8e3c0cdb10 con 0x7f8e3c15c1f0 2024-09-17T05:39:07.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.650+0000 7f8e39600640 1 -- 172.21.15.122:0/1517498833 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) ==== 138+0+0 (secure 0 0 0) 0x7f8e34019100 con 0x7f8e3c15c1f0 2024-09-17T05:39:07.648 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:39:07.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e1c0751b0 msgr2=0x7f8e1c077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:07.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e1c0751b0 0x7f8e1c077670 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c157f90 tx=0x7f8e300091e0 comp rx=0 tx=0).stop 2024-09-17T05:39:07.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c15c1f0 msgr2=0x7f8e3c0b7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:07.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c15c1f0 0x7f8e3c0b7520 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009bd0 tx=0x7f8e34009390 comp rx=0 tx=0).stop 2024-09-17T05:39:07.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 shutdown_connections 2024-09-17T05:39:07.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8e1c0751b0 0x7f8e1c077670 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:07.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e3c15c1f0 0x7f8e3c0b7520 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:07.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e3c15be10 0x7f8e3c0b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:07.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 --2- 172.21.15.122:0/1517498833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e3c0b0b60 0x7f8e3c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:07.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.654+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 >> 172.21.15.122:0/1517498833 conn(0x7f8e3c0d1da0 msgr2=0x7f8e3c0b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:07.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.658+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 shutdown_connections 2024-09-17T05:39:07.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.658+0000 7f8e44600640 1 -- 172.21.15.122:0/1517498833 wait complete. 2024-09-17T05:39:07.676 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:39:07.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.830+0000 7f2b8e600640 1 -- 172.21.15.122:0/2063452457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c004030 msgr2=0x7f2b6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:07.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.830+0000 7f2b8e600640 1 --2- 172.21.15.122:0/2063452457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b6c004030 0x7f2b6c004810 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006f90 tx=0x7f2b7401cbc0 comp rx=0 tx=0).stop 2024-09-17T05:39:07.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.830+0000 7f2b87e00640 1 -- 172.21.15.122:0/2063452457 reap_dead start 2024-09-17T05:39:07.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.830+0000 7f2b8e600640 1 -- 172.21.15.122:0/2063452457 shutdown_connections 2024-09-17T05:39:07.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.830+0000 7f2b8e600640 1 -- 172.21.15.122:0/2063452457 >> 172.21.15.122:0/2063452457 conn(0x7f2b880d1ea0 msgr2=0x7f2b880d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:07.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.830+0000 7f2b8e600640 1 -- 172.21.15.122:0/2063452457 shutdown_connections 2024-09-17T05:39:07.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.830+0000 7f2b8e600640 1 -- 172.21.15.122:0/2063452457 wait complete. 2024-09-17T05:39:07.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 Processor -- start 2024-09-17T05:39:07.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 -- start start 2024-09-17T05:39:07.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6c004030 0x7f2b880b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:07.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8814e3b0 0x7f2b8814da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:07.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b8814df50 0x7f2b88158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:07.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b881672c0 con 0x7f2b8814e3b0 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88167140 con 0x7f2b6c004030 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2b88167440 con 0x7f2b8814df50 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8814e3b0 0x7f2b8814da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8814e3b0 0x7f2b8814da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44612/0 (socket says 172.21.15.122:44612) 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 -- 172.21.15.122:0/2350221790 learned_addr learned my addr 172.21.15.122:0/2350221790 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 -- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b8814df50 msgr2=0x7f2b88158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b8814df50 0x7f2b88158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 -- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6c004030 msgr2=0x7f2b880b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6c004030 0x7f2b880b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 -- 172.21.15.122:0/2350221790 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b880b6c70 con 0x7f2b8814e3b0 2024-09-17T05:39:07.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b86a00640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8814e3b0 0x7f2b8814da10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f2b7401b2c0 tx=0x7f2b74006bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:07.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b7f400640 1 -- 172.21.15.122:0/2350221790 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740bc070 con 0x7f2b8814e3b0 2024-09-17T05:39:07.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b7f400640 1 -- 172.21.15.122:0/2350221790 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b74003600 con 0x7f2b8814e3b0 2024-09-17T05:39:07.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b7f400640 1 -- 172.21.15.122:0/2350221790 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740b1040 con 0x7f2b8814e3b0 2024-09-17T05:39:07.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b880b4e80 con 0x7f2b8814e3b0 2024-09-17T05:39:07.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.834+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b880b5440 con 0x7f2b8814e3b0 2024-09-17T05:39:07.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.838+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b880ce5e0 con 0x7f2b8814e3b0 2024-09-17T05:39:07.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.838+0000 7f2b7f400640 1 -- 172.21.15.122:0/2350221790 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f2b74006150 con 0x7f2b8814e3b0 2024-09-17T05:39:07.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.838+0000 7f2b7f400640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b68075180 0x7f2b68077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:07.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.838+0000 7f2b7f400640 1 -- 172.21.15.122:0/2350221790 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5311+0+0 (secure 0 0 0) 0x7f2b7412c6f0 con 0x7f2b8814e3b0 2024-09-17T05:39:07.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.838+0000 7f2b87400640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b68075180 0x7f2b68077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:07.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.838+0000 7f2b87400640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b68075180 0x7f2b68077640 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009bd0 tx=0x7f2b78009690 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:07.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:07.842+0000 7f2b7f400640 1 -- 172.21.15.122:0/2350221790 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2b740a8070 con 0x7f2b8814e3b0 2024-09-17T05:39:08.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.014+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2b880d2430 con 0x7f2b8814e3b0 2024-09-17T05:39:08.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.654+0000 7f2b7f400640 1 -- 172.21.15.122:0/2350221790 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) ==== 170+0+0 (secure 0 0 0) 0x7f2b74016c90 con 0x7f2b8814e3b0 2024-09-17T05:39:08.651 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:39:08.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b68075180 msgr2=0x7f2b68077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:08.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b68075180 0x7f2b68077640 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009bd0 tx=0x7f2b78009690 comp rx=0 tx=0).stop 2024-09-17T05:39:08.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8814e3b0 msgr2=0x7f2b8814da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8814e3b0 0x7f2b8814da10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f2b7401b2c0 tx=0x7f2b74006bc0 comp rx=0 tx=0).stop 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 shutdown_connections 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2b68075180 0x7f2b68077640 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b8814df50 0x7f2b88158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8814e3b0 0x7f2b8814da10 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 --2- 172.21.15.122:0/2350221790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6c004030 0x7f2b880b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 >> 172.21.15.122:0/2350221790 conn(0x7f2b880d1ea0 msgr2=0x7f2b8815a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 shutdown_connections 2024-09-17T05:39:08.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.658+0000 7f2b8e600640 1 -- 172.21.15.122:0/2350221790 wait complete. 2024-09-17T05:39:08.687 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:39:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- 172.21.15.122:0/3382847712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0b4a50 msgr2=0x7f193c0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 --2- 172.21.15.122:0/3382847712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0b4a50 0x7f193c0acd40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1934009bc0 tx=0x7f193401ca30 comp rx=0 tx=0).stop 2024-09-17T05:39:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- 172.21.15.122:0/3382847712 shutdown_connections 2024-09-17T05:39:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 --2- 172.21.15.122:0/3382847712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0b4a50 0x7f193c0acd40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- 172.21.15.122:0/3382847712 >> 172.21.15.122:0/3382847712 conn(0x7f193c0b18f0 msgr2=0x7f193c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- 172.21.15.122:0/3382847712 shutdown_connections 2024-09-17T05:39:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- 172.21.15.122:0/3382847712 wait complete. 2024-09-17T05:39:08.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 Processor -- start 2024-09-17T05:39:08.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- start start 2024-09-17T05:39:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193c06f8b0 0x7f193c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f193c0b4a50 0x7f193c15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0cd100 0x7f193c153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1692c0 con 0x7f193c0b4a50 2024-09-17T05:39:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f193c169140 con 0x7f193c0cd100 2024-09-17T05:39:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1940a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f193c169440 con 0x7f193c06f8b0 2024-09-17T05:39:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1931400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f193c0b4a50 0x7f193c15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1931400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f193c0b4a50 0x7f193c15c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44624/0 (socket says 172.21.15.122:44624) 2024-09-17T05:39:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.838+0000 7f1931400640 1 -- 172.21.15.122:0/2208496111 learned_addr learned my addr 172.21.15.122:0/2208496111 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:08.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1931400640 1 -- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193c06f8b0 msgr2=0x7f193c15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:08.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1939e00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193c06f8b0 0x7f193c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:08.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f193a800640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0cd100 0x7f193c153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:08.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1931400640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193c06f8b0 0x7f193c15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1931400640 1 -- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0cd100 msgr2=0x7f193c153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1931400640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0cd100 0x7f193c153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1931400640 1 -- 172.21.15.122:0/2208496111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c0afe70 con 0x7f193c0b4a50 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f193a800640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0cd100 0x7f193c153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1939e00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193c06f8b0 0x7f193c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1931400640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f193c0b4a50 0x7f193c15c600 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f192400c430 tx=0x7f192400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1932a00640 1 -- 172.21.15.122:0/2208496111 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924017070 con 0x7f193c0b4a50 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1932a00640 1 -- 172.21.15.122:0/2208496111 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19240053c0 con 0x7f193c0b4a50 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1932a00640 1 -- 172.21.15.122:0/2208496111 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924003860 con 0x7f193c0b4a50 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c0b0150 con 0x7f193c0b4a50 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c0b0710 con 0x7f193c0b4a50 2024-09-17T05:39:08.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193c069410 con 0x7f193c0b4a50 2024-09-17T05:39:08.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.842+0000 7f1932a00640 1 -- 172.21.15.122:0/2208496111 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f192401a070 con 0x7f193c0b4a50 2024-09-17T05:39:08.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.846+0000 7f1932a00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1918075180 0x7f1918077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:08.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.846+0000 7f1932a00640 1 -- 172.21.15.122:0/2208496111 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5693+0+0 (secure 0 0 0) 0x7f1924092a00 con 0x7f193c0b4a50 2024-09-17T05:39:08.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.846+0000 7f1939e00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1918075180 0x7f1918077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:08.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.846+0000 7f1939e00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1918075180 0x7f1918077640 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f192c007b10 tx=0x7f192c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:08.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:08.850+0000 7f1932a00640 1 -- 172.21.15.122:0/2208496111 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1924021020 con 0x7f193c0b4a50 2024-09-17T05:39:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.050+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f193c157280 con 0x7f193c0b4a50 2024-09-17T05:39:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.690+0000 7f1932a00640 1 -- 172.21.15.122:0/2208496111 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v82) ==== 174+0+0 (secure 0 0 0) 0x7f192405f590 con 0x7f193c0b4a50 2024-09-17T05:39:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 18 and data pool 19 2024-09-17T05:39:09.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1918075180 msgr2=0x7f1918077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:09.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1918075180 0x7f1918077640 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f192c007b10 tx=0x7f192c009040 comp rx=0 tx=0).stop 2024-09-17T05:39:09.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f193c0b4a50 msgr2=0x7f193c15c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:09.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f193c0b4a50 0x7f193c15c600 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f192400c430 tx=0x7f192400c8f0 comp rx=0 tx=0).stop 2024-09-17T05:39:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 shutdown_connections 2024-09-17T05:39:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1918075180 0x7f1918077640 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f193c0cd100 0x7f193c153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f193c0b4a50 0x7f193c15c600 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 --2- 172.21.15.122:0/2208496111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193c06f8b0 0x7f193c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 >> 172.21.15.122:0/2208496111 conn(0x7f193c0b18f0 msgr2=0x7f193c15a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 shutdown_connections 2024-09-17T05:39:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.698+0000 7f1940a00640 1 -- 172.21.15.122:0/2208496111 wait complete. 2024-09-17T05:39:09.725 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:39:09.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- 172.21.15.122:0/3553442884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00005270 msgr2=0x7fcf0000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:09.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 --2- 172.21.15.122:0/3553442884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00005270 0x7fcf0000b9d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400c630 tx=0x7fcf1401d1e0 comp rx=0 tx=0).stop 2024-09-17T05:39:09.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf1b400640 1 -- 172.21.15.122:0/3553442884 reap_dead start 2024-09-17T05:39:09.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- 172.21.15.122:0/3553442884 shutdown_connections 2024-09-17T05:39:09.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- 172.21.15.122:0/3553442884 >> 172.21.15.122:0/3553442884 conn(0x7fcf1c0b18f0 msgr2=0x7fcf1c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:09.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- 172.21.15.122:0/3553442884 shutdown_connections 2024-09-17T05:39:09.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- 172.21.15.122:0/3553442884 wait complete. 2024-09-17T05:39:09.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 Processor -- start 2024-09-17T05:39:09.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- start start 2024-09-17T05:39:09.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf00005270 0x7fcf1c1f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:09.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf1c1fc600 0x7fcf1c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:09.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf1c15c140 0x7fcf1c15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:09.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0acd40 con 0x7fcf1c15c140 2024-09-17T05:39:09.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0acec0 con 0x7fcf00005270 2024-09-17T05:39:09.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf21600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c0ad040 con 0x7fcf1c1fc600 2024-09-17T05:39:09.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf1c15c140 0x7fcf1c15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:09.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf1c15c140 0x7fcf1c15c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44634/0 (socket says 172.21.15.122:44634) 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf1b400640 1 -- 172.21.15.122:0/185404066 learned_addr learned my addr 172.21.15.122:0/185404066 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.874+0000 7fcf12000640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf1c1fc600 0x7fcf1c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf1aa00640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf00005270 0x7fcf1c1f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf1b400640 1 -- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf1c1fc600 msgr2=0x7fcf1c158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf1b400640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf1c1fc600 0x7fcf1c158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf1b400640 1 -- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf00005270 msgr2=0x7fcf1c1f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf1b400640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf00005270 0x7fcf1c1f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf1b400640 1 -- 172.21.15.122:0/185404066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c1fcba0 con 0x7fcf1c15c140 2024-09-17T05:39:09.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf1b400640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf1c15c140 0x7fcf1c15c5c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fcf14098d70 tx=0x7fcf14006600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf13400640 1 -- 172.21.15.122:0/185404066 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14004e60 con 0x7fcf1c15c140 2024-09-17T05:39:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf13400640 1 -- 172.21.15.122:0/185404066 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf14006770 con 0x7fcf1c15c140 2024-09-17T05:39:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c1fce80 con 0x7fcf1c15c140 2024-09-17T05:39:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf13400640 1 -- 172.21.15.122:0/185404066 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14006d30 con 0x7fcf1c15c140 2024-09-17T05:39:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c1fd440 con 0x7fcf1c15c140 2024-09-17T05:39:09.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c069410 con 0x7fcf1c15c140 2024-09-17T05:39:09.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.878+0000 7fcf13400640 1 -- 172.21.15.122:0/185404066 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fcf14003980 con 0x7fcf1c15c140 2024-09-17T05:39:09.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.882+0000 7fcf13400640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcef80751b0 0x7fcef8077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:09.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.882+0000 7fcf13400640 1 -- 172.21.15.122:0/185404066 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5793+0+0 (secure 0 0 0) 0x7fcf1412d780 con 0x7fcf1c15c140 2024-09-17T05:39:09.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.882+0000 7fcf1aa00640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcef80751b0 0x7fcef8077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:09.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.882+0000 7fcf1aa00640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcef80751b0 0x7fcef8077670 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c006fd0 tx=0x7fcf0c006b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:09.886+0000 7fcf13400640 1 -- 172.21.15.122:0/185404066 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcf14016f90 con 0x7fcf1c15c140 2024-09-17T05:39:10.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.054+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf1c0698f0 con 0x7fcf1c15c140 2024-09-17T05:39:10.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf13400640 1 -- 172.21.15.122:0/185404066 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16892 (secure 0 0 0) 0x7fcf140f9bf0 con 0x7fcf1c15c140 2024-09-17T05:39:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:10.052 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":131,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:39:09.055718+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:39:06.760215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T05:39:08.026267+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:39:10.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcef80751b0 msgr2=0x7fcef8077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcef80751b0 0x7fcef8077670 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c006fd0 tx=0x7fcf0c006b20 comp rx=0 tx=0).stop 2024-09-17T05:39:10.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf1c15c140 msgr2=0x7fcf1c15c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf1c15c140 0x7fcf1c15c5c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fcf14098d70 tx=0x7fcf14006600 comp rx=0 tx=0).stop 2024-09-17T05:39:10.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 shutdown_connections 2024-09-17T05:39:10.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcef80751b0 0x7fcef8077670 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf1c15c140 0x7fcf1c15c5c0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf1c1fc600 0x7fcf1c158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 --2- 172.21.15.122:0/185404066 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf00005270 0x7fcf1c1f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 >> 172.21.15.122:0/185404066 conn(0x7fcf1c0b18f0 msgr2=0x7fcf1c0ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:10.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 shutdown_connections 2024-09-17T05:39:10.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.058+0000 7fcf21600640 1 -- 172.21.15.122:0/185404066 wait complete. 2024-09-17T05:39:10.074 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:39:10.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.218+0000 7fa281600640 1 -- 172.21.15.122:0/1113793483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27c15bea0 msgr2=0x7fa27c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.218+0000 7fa281600640 1 --2- 172.21.15.122:0/1113793483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27c15bea0 0x7fa27c0ce670 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009900 tx=0x7fa26c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.218+0000 7fa281600640 1 -- 172.21.15.122:0/1113793483 shutdown_connections 2024-09-17T05:39:10.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.218+0000 7fa281600640 1 --2- 172.21.15.122:0/1113793483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa27c0cdb30 0x7fa27c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.218+0000 7fa281600640 1 --2- 172.21.15.122:0/1113793483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27c15bea0 0x7fa27c0ce670 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.218+0000 7fa281600640 1 -- 172.21.15.122:0/1113793483 >> 172.21.15.122:0/1113793483 conn(0x7fa27c0d1dc0 msgr2=0x7fa27c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:10.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 -- 172.21.15.122:0/1113793483 shutdown_connections 2024-09-17T05:39:10.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 -- 172.21.15.122:0/1113793483 wait complete. 2024-09-17T05:39:10.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 Processor -- start 2024-09-17T05:39:10.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 -- start start 2024-09-17T05:39:10.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27c0cdb30 0x7fa27c153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:10.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa27c0d28e0 0x7fa27c155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:10.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa27c15bea0 0x7fa27c0cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:10.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c168fc0 con 0x7fa27c15bea0 2024-09-17T05:39:10.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c168e40 con 0x7fa27c0cdb30 2024-09-17T05:39:10.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa281600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa27c169140 con 0x7fa27c0d28e0 2024-09-17T05:39:10.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa27c15bea0 0x7fa27c0cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:10.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa27c15bea0 0x7fa27c0cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44642/0 (socket says 172.21.15.122:44642) 2024-09-17T05:39:10.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 -- 172.21.15.122:0/2725768194 learned_addr learned my addr 172.21.15.122:0/2725768194 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:10.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27aa00640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27c0cdb30 0x7fa27c153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:10.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa272000640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa27c0d28e0 0x7fa27c155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:10.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 -- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa27c0d28e0 msgr2=0x7fa27c155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa27c0d28e0 0x7fa27c155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 -- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27c0cdb30 msgr2=0x7fa27c153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27c0cdb30 0x7fa27c153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 -- 172.21.15.122:0/2725768194 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c0b6a80 con 0x7fa27c15bea0 2024-09-17T05:39:10.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa27b400640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa27c15bea0 0x7fa27c0cdfa0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c500 tx=0x7fa27400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:10.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.222+0000 7fa273400640 1 -- 172.21.15.122:0/2725768194 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2740052d0 con 0x7fa27c15bea0 2024-09-17T05:39:10.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.226+0000 7fa273400640 1 -- 172.21.15.122:0/2725768194 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa274005470 con 0x7fa27c15bea0 2024-09-17T05:39:10.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.226+0000 7fa273400640 1 -- 172.21.15.122:0/2725768194 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274012870 con 0x7fa27c15bea0 2024-09-17T05:39:10.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.226+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c0b3110 con 0x7fa27c15bea0 2024-09-17T05:39:10.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.226+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c0b36d0 con 0x7fa27c15bea0 2024-09-17T05:39:10.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.226+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27c0cc9d0 con 0x7fa27c15bea0 2024-09-17T05:39:10.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.226+0000 7fa273400640 1 -- 172.21.15.122:0/2725768194 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fa274003600 con 0x7fa27c15bea0 2024-09-17T05:39:10.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.226+0000 7fa273400640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa258075180 0x7fa258077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:10.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.226+0000 7fa273400640 1 -- 172.21.15.122:0/2725768194 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5793+0+0 (secure 0 0 0) 0x7fa274093870 con 0x7fa27c15bea0 2024-09-17T05:39:10.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.230+0000 7fa27aa00640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa258075180 0x7fa258077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:10.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.230+0000 7fa27aa00640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa258075180 0x7fa258077640 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0af000 tx=0x7fa26c01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.230+0000 7fa273400640 1 -- 172.21.15.122:0/2725768194 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa274060380 con 0x7fa27c15bea0 2024-09-17T05:39:10.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.398+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa27c0b5790 con 0x7fa27c15bea0 2024-09-17T05:39:10.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa273400640 1 -- 172.21.15.122:0/2725768194 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16892 (secure 0 0 0) 0x7fa27405fc80 con 0x7fa27c15bea0 2024-09-17T05:39:10.396 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:10.396 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":131,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:39:09.055718+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:39:06.760215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T05:39:08.026267+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:39:10.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa258075180 msgr2=0x7fa258077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa258075180 0x7fa258077640 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0af000 tx=0x7fa26c01b120 comp rx=0 tx=0).stop 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa27c15bea0 msgr2=0x7fa27c0cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa27c15bea0 0x7fa27c0cdfa0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c500 tx=0x7fa27400c9c0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 shutdown_connections 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa258075180 0x7fa258077640 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa27c15bea0 0x7fa27c0cdfa0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa27c0d28e0 0x7fa27c155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 --2- 172.21.15.122:0/2725768194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27c0cdb30 0x7fa27c153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 >> 172.21.15.122:0/2725768194 conn(0x7fa27c0d1dc0 msgr2=0x7fa27c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 shutdown_connections 2024-09-17T05:39:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.402+0000 7fa281600640 1 -- 172.21.15.122:0/2725768194 wait complete. 2024-09-17T05:39:10.418 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:39:10.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 -- 172.21.15.122:0/2909798400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0b2d80 msgr2=0x7f99ec0d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 --2- 172.21.15.122:0/2909798400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0b2d80 0x7f99ec0d0840 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0d0d80 tx=0x7f99d801f0b0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 -- 172.21.15.122:0/2909798400 shutdown_connections 2024-09-17T05:39:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 --2- 172.21.15.122:0/2909798400 >> v2:172.21.15.173:3300/0 conn(0x7f99ec0b14c0 0x7f99ec0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 --2- 172.21.15.122:0/2909798400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0b2d80 0x7f99ec0d0840 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 --2- 172.21.15.122:0/2909798400 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99ec0cff20 0x7f99ec0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 -- 172.21.15.122:0/2909798400 >> 172.21.15.122:0/2909798400 conn(0x7f99ec0d1dc0 msgr2=0x7f99ec0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 -- 172.21.15.122:0/2909798400 shutdown_connections 2024-09-17T05:39:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.566+0000 7f99f4600640 1 -- 172.21.15.122:0/2909798400 wait complete. 2024-09-17T05:39:10.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 Processor -- start 2024-09-17T05:39:10.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 -- start start 2024-09-17T05:39:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec0b14c0 0x7f99ec149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99ec0b2d80 0x7f99ec0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0cff20 0x7f99ec0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec160110 con 0x7f99ec0cff20 2024-09-17T05:39:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec15ff90 con 0x7f99ec0b14c0 2024-09-17T05:39:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f99ec160290 con 0x7f99ec0b2d80 2024-09-17T05:39:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0cff20 0x7f99ec0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0cff20 0x7f99ec0cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44668/0 (socket says 172.21.15.122:44668) 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 -- 172.21.15.122:0/2153585891 learned_addr learned my addr 172.21.15.122:0/2153585891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f1c00640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec0b14c0 0x7f99ec149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f1200640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99ec0b2d80 0x7f99ec0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 -- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99ec0b2d80 msgr2=0x7f99ec0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99ec0b2d80 0x7f99ec0cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 -- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec0b14c0 msgr2=0x7f99ec149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec0b14c0 0x7f99ec149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 -- 172.21.15.122:0/2153585891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec149e60 con 0x7f99ec0cff20 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f2600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0cff20 0x7f99ec0cb6c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f99e400e990 tx=0x7f99e400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99e9600640 1 -- 172.21.15.122:0/2153585891 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4004d30 con 0x7f99ec0cff20 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99e9600640 1 -- 172.21.15.122:0/2153585891 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99e400cb00 con 0x7f99ec0cff20 2024-09-17T05:39:10.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99e9600640 1 -- 172.21.15.122:0/2153585891 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4005020 con 0x7f99ec0cff20 2024-09-17T05:39:10.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ec0cf6a0 con 0x7f99ec0cff20 2024-09-17T05:39:10.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ec0c9510 con 0x7f99ec0cff20 2024-09-17T05:39:10.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.570+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ec0b1940 con 0x7f99ec0cff20 2024-09-17T05:39:10.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.574+0000 7f99e9600640 1 -- 172.21.15.122:0/2153585891 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f99e40051c0 con 0x7f99ec0cff20 2024-09-17T05:39:10.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.574+0000 7f99e9600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99cc075180 0x7f99cc077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:10.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.574+0000 7f99e9600640 1 -- 172.21.15.122:0/2153585891 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5793+0+0 (secure 0 0 0) 0x7f99e4093f50 con 0x7f99ec0cff20 2024-09-17T05:39:10.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.574+0000 7f99f1c00640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99cc075180 0x7f99cc077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:10.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.574+0000 7f99f1c00640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99cc075180 0x7f99cc077640 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f99e000ab40 tx=0x7f99e000a250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:10.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.578+0000 7f99e9600640 1 -- 172.21.15.122:0/2153585891 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f99ec149e60 con 0x7f99ec0cff20 2024-09-17T05:39:10.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:10.786+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f99ec06aba0 con 0x7f99ec0cff20 2024-09-17T05:39:11.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.694+0000 7f99e9600640 1 -- 172.21.15.122:0/2153585891 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) ==== 118+0+0 (secure 0 0 0) 0x7f99e4069020 con 0x7f99ec0cff20 2024-09-17T05:39:11.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99cc075180 msgr2=0x7f99cc077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99cc075180 0x7f99cc077640 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f99e000ab40 tx=0x7f99e000a250 comp rx=0 tx=0).stop 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0cff20 msgr2=0x7f99ec0cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0cff20 0x7f99ec0cb6c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f99e400e990 tx=0x7f99e400ee50 comp rx=0 tx=0).stop 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 shutdown_connections 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f99cc075180 0x7f99cc077640 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99ec0cff20 0x7f99ec0cb6c0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99ec0b2d80 0x7f99ec0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 --2- 172.21.15.122:0/2153585891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99ec0b14c0 0x7f99ec149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 >> 172.21.15.122:0/2153585891 conn(0x7f99ec0d1dc0 msgr2=0x7f99ec0aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:11.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 shutdown_connections 2024-09-17T05:39:11.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.698+0000 7f99f4600640 1 -- 172.21.15.122:0/2153585891 wait complete. 2024-09-17T05:39:11.714 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:39:11.715 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:39:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.866+0000 7f1533c00640 1 -- 172.21.15.122:0/3739584569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1510003a10 msgr2=0x7f1510003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.866+0000 7f1533c00640 1 --2- 172.21.15.122:0/3739584569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1510003a10 0x7f1510003e10 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f151c00c700 tx=0x7f151c01c680 comp rx=0 tx=0).stop 2024-09-17T05:39:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.866+0000 7f1533c00640 1 -- 172.21.15.122:0/3739584569 shutdown_connections 2024-09-17T05:39:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.866+0000 7f1533c00640 1 --2- 172.21.15.122:0/3739584569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1510003a10 0x7f1510003e10 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.866+0000 7f1533c00640 1 -- 172.21.15.122:0/3739584569 >> 172.21.15.122:0/3739584569 conn(0x7f152c0cce90 msgr2=0x7f152c0cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.866+0000 7f1533c00640 1 -- 172.21.15.122:0/3739584569 shutdown_connections 2024-09-17T05:39:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.866+0000 7f1533c00640 1 -- 172.21.15.122:0/3739584569 wait complete. 2024-09-17T05:39:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.866+0000 7f1533c00640 1 Processor -- start 2024-09-17T05:39:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 -- start start 2024-09-17T05:39:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1510003a10 0x7f152c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f152c1faf20 0x7f152c0b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f152c0b3360 0x7f152c0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f152c073da0 con 0x7f152c0b3360 2024-09-17T05:39:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f152c073c20 con 0x7f1510003a10 2024-09-17T05:39:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f152c073f20 con 0x7f152c1faf20 2024-09-17T05:39:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f152c0b3360 0x7f152c0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f152c0b3360 0x7f152c0b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38754/0 (socket says 172.21.15.122:38754) 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 -- 172.21.15.122:0/3867058916 learned_addr learned my addr 172.21.15.122:0/3867058916 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531200640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1510003a10 0x7f152c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f152be00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f152c1faf20 0x7f152c0b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 -- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f152c1faf20 msgr2=0x7f152c0b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f152c1faf20 0x7f152c0b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 -- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1510003a10 msgr2=0x7f152c06fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1510003a10 0x7f152c06fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 -- 172.21.15.122:0/3867058916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c1fb600 con 0x7f152c0b3360 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1531c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f152c0b3360 0x7f152c0b3810 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f152c0b6620 tx=0x7f1524010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1528c00640 1 -- 172.21.15.122:0/3867058916 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524003b90 con 0x7f152c0b3360 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1528c00640 1 -- 172.21.15.122:0/3867058916 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1524003e20 con 0x7f152c0b3360 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1528c00640 1 -- 172.21.15.122:0/3867058916 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524004d70 con 0x7f152c0b3360 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152c1fb8e0 con 0x7f152c0b3360 2024-09-17T05:39:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152c1f7ad0 con 0x7f152c0b3360 2024-09-17T05:39:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.870+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152c0cf1a0 con 0x7f152c0b3360 2024-09-17T05:39:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.874+0000 7f1528c00640 1 -- 172.21.15.122:0/3867058916 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f152401c020 con 0x7f152c0b3360 2024-09-17T05:39:11.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.874+0000 7f1528c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f150c0751b0 0x7f150c077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:11.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.874+0000 7f1528c00640 1 -- 172.21.15.122:0/3867058916 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f1524093320 con 0x7f152c0b3360 2024-09-17T05:39:11.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.874+0000 7f1531200640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f150c0751b0 0x7f150c077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:11.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.874+0000 7f1531200640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f150c0751b0 0x7f150c077670 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f15100057f0 tx=0x7f151c019420 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:11.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:11.878+0000 7f1528c00640 1 -- 172.21.15.122:0/3867058916 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f152405fdb0 con 0x7f152c0b3360 2024-09-17T05:39:12.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.082+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f152c068e70 con 0x7f152c0b3360 2024-09-17T05:39:12.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1528c00640 1 -- 172.21.15.122:0/3867058916 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) ==== 122+0+0 (secure 0 0 0) 0x7f152405f6b0 con 0x7f152c0b3360 2024-09-17T05:39:12.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f150c0751b0 msgr2=0x7f150c077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:12.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f150c0751b0 0x7f150c077670 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f15100057f0 tx=0x7f151c019420 comp rx=0 tx=0).stop 2024-09-17T05:39:12.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f152c0b3360 msgr2=0x7f152c0b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f152c0b3360 0x7f152c0b3810 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f152c0b6620 tx=0x7f1524010da0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 shutdown_connections 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f150c0751b0 0x7f150c077670 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f152c0b3360 0x7f152c0b3810 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f152c1faf20 0x7f152c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 --2- 172.21.15.122:0/3867058916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1510003a10 0x7f152c06fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 >> 172.21.15.122:0/3867058916 conn(0x7f152c0cce90 msgr2=0x7f152c1e9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 shutdown_connections 2024-09-17T05:39:12.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.702+0000 7f1533c00640 1 -- 172.21.15.122:0/3867058916 wait complete. 2024-09-17T05:39:12.718 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:39:12.718 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:39:12.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- 172.21.15.122:0/2026916145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa80415be80 msgr2=0x7fa8040ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:12.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 --2- 172.21.15.122:0/2026916145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa80415be80 0x7fa8040ce4e0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009840 tx=0x7fa7f401c820 comp rx=0 tx=0).stop 2024-09-17T05:39:12.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- 172.21.15.122:0/2026916145 shutdown_connections 2024-09-17T05:39:12.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 --2- 172.21.15.122:0/2026916145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8040cd8c0 0x7fa8040cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 --2- 172.21.15.122:0/2026916145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa80415be80 0x7fa8040ce4e0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- 172.21.15.122:0/2026916145 >> 172.21.15.122:0/2026916145 conn(0x7fa8040d1da0 msgr2=0x7fa8040d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:12.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- 172.21.15.122:0/2026916145 shutdown_connections 2024-09-17T05:39:12.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- 172.21.15.122:0/2026916145 wait complete. 2024-09-17T05:39:12.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 Processor -- start 2024-09-17T05:39:12.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- start start 2024-09-17T05:39:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8040cd8c0 0x7fa8041f9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8040d28c0 0x7fa80415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa80415be80 0x7fa80415b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa804168c60 con 0x7fa8040cd8c0 2024-09-17T05:39:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa8040b66f0 con 0x7fa8040d28c0 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa808c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa804168de0 con 0x7fa80415be80 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8040cd8c0 0x7fa8041f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8040cd8c0 0x7fa8041f9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38774/0 (socket says 172.21.15.122:38774) 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801e00640 1 -- 172.21.15.122:0/3158862243 learned_addr learned my addr 172.21.15.122:0/3158862243 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801400640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8040d28c0 0x7fa80415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa802800640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa80415be80 0x7fa80415b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801e00640 1 -- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa80415be80 msgr2=0x7fa80415b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801e00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa80415be80 0x7fa80415b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801e00640 1 -- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8040d28c0 msgr2=0x7fa80415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801e00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8040d28c0 0x7fa80415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.866+0000 7fa801e00640 1 -- 172.21.15.122:0/3158862243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8040cacb0 con 0x7fa8040cd8c0 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa801e00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8040cd8c0 0x7fa8041f9260 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009790 tx=0x7fa7f40a67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa7fa000640 1 -- 172.21.15.122:0/3158862243 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40041c0 con 0x7fa8040cd8c0 2024-09-17T05:39:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa7fa000640 1 -- 172.21.15.122:0/3158862243 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7f4004450 con 0x7fa8040cd8c0 2024-09-17T05:39:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa804151f90 con 0x7fa8040cd8c0 2024-09-17T05:39:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa7fa000640 1 -- 172.21.15.122:0/3158862243 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4003860 con 0x7fa8040cd8c0 2024-09-17T05:39:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8040c9260 con 0x7fa8040cd8c0 2024-09-17T05:39:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8040b53f0 con 0x7fa8040cd8c0 2024-09-17T05:39:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa7fa000640 1 -- 172.21.15.122:0/3158862243 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fa7f40c0020 con 0x7fa8040cd8c0 2024-09-17T05:39:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa7fa000640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7e4075180 0x7fa7e4077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.870+0000 7fa7fa000640 1 -- 172.21.15.122:0/3158862243 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7fa7f412e050 con 0x7fa8040cd8c0 2024-09-17T05:39:12.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.874+0000 7fa801400640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7e4075180 0x7fa7e4077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:12.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.874+0000 7fa801400640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7e4075180 0x7fa7e4077640 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa8040b2b10 tx=0x7fa7f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:12.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:12.878+0000 7fa7fa000640 1 -- 172.21.15.122:0/3158862243 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa7f4016410 con 0x7fa8040cd8c0 2024-09-17T05:39:13.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.078+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8040b7a80 con 0x7fa8040cd8c0 2024-09-17T05:39:13.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.078+0000 7fa7fa000640 1 -- 172.21.15.122:0/3158862243 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5234 (secure 0 0 0) 0x7fa7f40fab60 con 0x7fa8040cd8c0 2024-09-17T05:39:13.075 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:13.075 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":86,"btime":"2024-09-17T05:39:12:702401+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/609693749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":609693749},{"type":"v1","addr":"172.21.15.122:6834","nonce":609693749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2826125653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2826125653},{"type":"v1","addr":"172.21.15.173:6836","nonce":2826125653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4138151052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4138151052},{"type":"v1","addr":"172.21.15.173:6837","nonce":4138151052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:39:09.055675+0000","modified":"2024-09-17T05:39:12.084809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2463174917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2463174917},{"type":"v1","addr":"172.21.15.122:6835","nonce":2463174917}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8916,"qdb_cluster":[8916]},"id":9}]} 2024-09-17T05:39:13.075 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 86 2024-09-17T05:39:13.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7e4075180 msgr2=0x7fa7e4077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7e4075180 0x7fa7e4077640 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa8040b2b10 tx=0x7fa7f0009040 comp rx=0 tx=0).stop 2024-09-17T05:39:13.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8040cd8c0 msgr2=0x7fa8041f9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8040cd8c0 0x7fa8041f9260 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009790 tx=0x7fa7f40a67b0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 shutdown_connections 2024-09-17T05:39:13.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7e4075180 0x7fa7e4077640 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa80415be80 0x7fa80415b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8040d28c0 0x7fa80415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 --2- 172.21.15.122:0/3158862243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8040cd8c0 0x7fa8041f9260 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 >> 172.21.15.122:0/3158862243 conn(0x7fa8040d1da0 msgr2=0x7fa8040b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:13.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 shutdown_connections 2024-09-17T05:39:13.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.082+0000 7fa808c00640 1 -- 172.21.15.122:0/3158862243 wait complete. 2024-09-17T05:39:13.100 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:39:13.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.246+0000 7f54fa400640 1 -- 172.21.15.122:0/2538464840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f415bea0 msgr2=0x7f54f40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.246+0000 7f54fa400640 1 --2- 172.21.15.122:0/2538464840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f415bea0 0x7f54f40ce670 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009840 tx=0x7f54e401c790 comp rx=0 tx=0).stop 2024-09-17T05:39:13.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.246+0000 7f54fa400640 1 -- 172.21.15.122:0/2538464840 shutdown_connections 2024-09-17T05:39:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.246+0000 7f54fa400640 1 --2- 172.21.15.122:0/2538464840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f415bea0 0x7f54f40ce670 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.246+0000 7f54fa400640 1 -- 172.21.15.122:0/2538464840 >> 172.21.15.122:0/2538464840 conn(0x7f54f40d1dc0 msgr2=0x7f54f40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.246+0000 7f54fa400640 1 -- 172.21.15.122:0/2538464840 shutdown_connections 2024-09-17T05:39:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.246+0000 7f54fa400640 1 -- 172.21.15.122:0/2538464840 wait complete. 2024-09-17T05:39:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.246+0000 7f54fa400640 1 Processor -- start 2024-09-17T05:39:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 -- start start 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f40cdb30 0x7f54f40cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54f40d28e0 0x7f54f4159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54f415bea0 0x7f54f415ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f54f416a930 con 0x7f54f40cdb30 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f54f416a7b0 con 0x7f54f415bea0 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f54f416aab0 con 0x7f54f40d28e0 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f40cdb30 0x7f54f40cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f40cdb30 0x7f54f40cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38790/0 (socket says 172.21.15.122:38790) 2024-09-17T05:39:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 -- 172.21.15.122:0/4033233558 learned_addr learned my addr 172.21.15.122:0/4033233558 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f2a00640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54f40d28e0 0x7f54f4159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3e00640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54f415bea0 0x7f54f415ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 -- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54f40d28e0 msgr2=0x7f54f4159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54f40d28e0 0x7f54f4159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 -- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54f415bea0 msgr2=0x7f54f415ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54f415bea0 0x7f54f415ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 -- 172.21.15.122:0/4033233558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f415a0e0 con 0x7f54f40cdb30 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3e00640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54f415bea0 0x7f54f415ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54f3400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f40cdb30 0x7f54f40cb960 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f54e4000c10 tx=0x7f54e4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54eb400640 1 -- 172.21.15.122:0/4033233558 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e40ac030 con 0x7f54f40cdb30 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54eb400640 1 -- 172.21.15.122:0/4033233558 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e4005520 con 0x7f54f40cdb30 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54eb400640 1 -- 172.21.15.122:0/4033233558 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4004930 con 0x7f54f40cdb30 2024-09-17T05:39:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f40b3d10 con 0x7f54f40cdb30 2024-09-17T05:39:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f40b42d0 con 0x7f54f40cdb30 2024-09-17T05:39:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.250+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f40ce670 con 0x7f54f40cdb30 2024-09-17T05:39:13.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.254+0000 7f54eb400640 1 -- 172.21.15.122:0/4033233558 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f54e40af070 con 0x7f54f40cdb30 2024-09-17T05:39:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.254+0000 7f54eb400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54d4075180 0x7f54d4077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.254+0000 7f54eb400640 1 -- 172.21.15.122:0/4033233558 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f54e412d9e0 con 0x7f54f40cdb30 2024-09-17T05:39:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.254+0000 7f54f2a00640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54d4075180 0x7f54d4077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.254+0000 7f54f2a00640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54d4075180 0x7f54d4077640 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f54f41fb770 tx=0x7f54e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.258+0000 7f54eb400640 1 -- 172.21.15.122:0/4033233558 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f54e4016410 con 0x7f54f40cdb30 2024-09-17T05:39:13.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.434+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54f40d2340 con 0x7f54f40cdb30 2024-09-17T05:39:13.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54eb400640 1 -- 172.21.15.122:0/4033233558 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16879 (secure 0 0 0) 0x7f54e40fa4f0 con 0x7f54f40cdb30 2024-09-17T05:39:13.432 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:13.432 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":132,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:39:10.673898+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:39:06.760215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T05:39:08.026267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:39:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54d4075180 msgr2=0x7f54d4077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54d4075180 0x7f54d4077640 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f54f41fb770 tx=0x7f54e0009040 comp rx=0 tx=0).stop 2024-09-17T05:39:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f40cdb30 msgr2=0x7f54f40cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f40cdb30 0x7f54f40cb960 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f54e4000c10 tx=0x7f54e4005040 comp rx=0 tx=0).stop 2024-09-17T05:39:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 shutdown_connections 2024-09-17T05:39:13.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54d4075180 0x7f54d4077640 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54f415bea0 0x7f54f415ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54f40d28e0 0x7f54f4159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 --2- 172.21.15.122:0/4033233558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f40cdb30 0x7f54f40cb960 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 >> 172.21.15.122:0/4033233558 conn(0x7f54f40d1dc0 msgr2=0x7f54f406ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 shutdown_connections 2024-09-17T05:39:13.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.438+0000 7f54fa400640 1 -- 172.21.15.122:0/4033233558 wait complete. 2024-09-17T05:39:13.455 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:39:13.455 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:39:13.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.598+0000 7f6a25c00640 1 -- 172.21.15.122:0/2260548367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 msgr2=0x7f6a200737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.598+0000 7f6a25c00640 1 --2- 172.21.15.122:0/2260548367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 0x7f6a200737c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009930 tx=0x7f6a1001c8b0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.598+0000 7f6a25c00640 1 -- 172.21.15.122:0/2260548367 shutdown_connections 2024-09-17T05:39:13.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.598+0000 7f6a25c00640 1 --2- 172.21.15.122:0/2260548367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 0x7f6a200737c0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.598+0000 7f6a25c00640 1 -- 172.21.15.122:0/2260548367 >> 172.21.15.122:0/2260548367 conn(0x7f6a200d0c90 msgr2=0x7f6a200d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:13.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 -- 172.21.15.122:0/2260548367 shutdown_connections 2024-09-17T05:39:13.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 -- 172.21.15.122:0/2260548367 wait complete. 2024-09-17T05:39:13.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 Processor -- start 2024-09-17T05:39:13.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 -- start start 2024-09-17T05:39:13.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 0x7f6a20155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a20071030 0x7f6a201fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a2015c1c0 0x7f6a201fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2016a900 con 0x7f6a2006cc20 2024-09-17T05:39:13.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2016a780 con 0x7f6a2015c1c0 2024-09-17T05:39:13.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2016aa80 con 0x7f6a20071030 2024-09-17T05:39:13.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 0x7f6a20155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 0x7f6a20155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38808/0 (socket says 172.21.15.122:38808) 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 -- 172.21.15.122:0/1397709674 learned_addr learned my addr 172.21.15.122:0/1397709674 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1f400640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a2015c1c0 0x7f6a201fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a16000640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a20071030 0x7f6a201fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 -- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a20071030 msgr2=0x7f6a201fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a20071030 0x7f6a201fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 -- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a2015c1c0 msgr2=0x7f6a201fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a2015c1c0 0x7f6a201fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 -- 172.21.15.122:0/1397709674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a200719a0 con 0x7f6a2006cc20 2024-09-17T05:39:13.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a1ea00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 0x7f6a20155bf0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f6a100b0040 tx=0x7f6a10004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:13.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a17400640 1 -- 172.21.15.122:0/1397709674 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a100b2040 con 0x7f6a2006cc20 2024-09-17T05:39:13.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a17400640 1 -- 172.21.15.122:0/1397709674 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a10004430 con 0x7f6a2006cc20 2024-09-17T05:39:13.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a17400640 1 -- 172.21.15.122:0/1397709674 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10003860 con 0x7f6a2006cc20 2024-09-17T05:39:13.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.602+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a201fce20 con 0x7f6a2006cc20 2024-09-17T05:39:13.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.606+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a200c69d0 con 0x7f6a2006cc20 2024-09-17T05:39:13.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.606+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a200737c0 con 0x7f6a2006cc20 2024-09-17T05:39:13.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.606+0000 7f6a17400640 1 -- 172.21.15.122:0/1397709674 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f6a10003a00 con 0x7f6a2006cc20 2024-09-17T05:39:13.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.606+0000 7f6a17400640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69fc075180 0x7f69fc077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.606+0000 7f6a17400640 1 -- 172.21.15.122:0/1397709674 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f6a1012d980 con 0x7f6a2006cc20 2024-09-17T05:39:13.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.606+0000 7f6a16000640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69fc075180 0x7f69fc077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.610+0000 7f6a16000640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69fc075180 0x7f69fc077640 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6a200a1dd0 tx=0x7f6a08009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:13.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.610+0000 7f6a17400640 1 -- 172.21.15.122:0/1397709674 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6a100c6060 con 0x7f6a2006cc20 2024-09-17T05:39:13.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.778+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a200b0020 con 0x7f69fc075180 2024-09-17T05:39:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.782+0000 7f6a17400640 1 -- 172.21.15.122:0/1397709674 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247873 (secure 0 0 0) 0x7f6a200b0020 con 0x7f69fc075180 2024-09-17T05:39:13.782 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:39:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69fc075180 msgr2=0x7f69fc077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69fc075180 0x7f69fc077640 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6a200a1dd0 tx=0x7f6a08009040 comp rx=0 tx=0).stop 2024-09-17T05:39:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 msgr2=0x7f6a20155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 0x7f6a20155bf0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f6a100b0040 tx=0x7f6a10004040 comp rx=0 tx=0).stop 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 shutdown_connections 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f69fc075180 0x7f69fc077640 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a2015c1c0 0x7f6a201fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a20071030 0x7f6a201fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 --2- 172.21.15.122:0/1397709674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a2006cc20 0x7f6a20155bf0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 >> 172.21.15.122:0/1397709674 conn(0x7f6a200d0c90 msgr2=0x7f6a200a1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 shutdown_connections 2024-09-17T05:39:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.786+0000 7f6a25c00640 1 -- 172.21.15.122:0/1397709674 wait complete. 2024-09-17T05:39:13.802 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":555,"stamp":"2024-09-17T05:39:12.736127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076260,"kb_used_data":13732,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898460,"statfs":{"total":773094113280,"available":771992023040,"internally_reserved":0,"allocated":14061568,"data_stored":6836827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1087819318},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.080111"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675684+0000","last_change":"2024-09-17T05:39:08.675684+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675684+0000","last_undegraded":"2024-09-17T05:39:08.675684+0000","last_fullsized":"2024-09-17T05:39:08.675684+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.681063+0000","last_change":"2024-09-17T05:39:08.681063+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.681063+0000","last_undegraded":"2024-09-17T05:39:08.681063+0000","last_fullsized":"2024-09-17T05:39:08.681063+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668951+0000","last_change":"2024-09-17T05:39:08.668951+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668951+0000","last_undegraded":"2024-09-17T05:39:08.668951+0000","last_fullsized":"2024-09-17T05:39:08.668951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666359+0000","last_change":"2024-09-17T05:39:08.666359+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666359+0000","last_undegraded":"2024-09-17T05:39:08.666359+0000","last_fullsized":"2024-09-17T05:39:08.666359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667851+0000","last_change":"2024-09-17T05:39:08.667851+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667851+0000","last_undegraded":"2024-09-17T05:39:08.667851+0000","last_fullsized":"2024-09-17T05:39:08.667851+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674261+0000","last_change":"2024-09-17T05:39:08.674261+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674261+0000","last_undegraded":"2024-09-17T05:39:08.674261+0000","last_fullsized":"2024-09-17T05:39:08.674261+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667467+0000","last_change":"2024-09-17T05:39:08.667467+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667467+0000","last_undegraded":"2024-09-17T05:39:08.667467+0000","last_fullsized":"2024-09-17T05:39:08.667467+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668743+0000","last_change":"2024-09-17T05:39:08.668743+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668743+0000","last_undegraded":"2024-09-17T05:39:08.668743+0000","last_fullsized":"2024-09-17T05:39:08.668743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667026+0000","last_change":"2024-09-17T05:39:08.667026+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667026+0000","last_undegraded":"2024-09-17T05:39:08.667026+0000","last_fullsized":"2024-09-17T05:39:08.667026+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678847+0000","last_change":"2024-09-17T05:39:08.678847+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678847+0000","last_undegraded":"2024-09-17T05:39:08.678847+0000","last_fullsized":"2024-09-17T05:39:08.678847+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666857+0000","last_change":"2024-09-17T05:39:08.666857+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666857+0000","last_undegraded":"2024-09-17T05:39:08.666857+0000","last_fullsized":"2024-09-17T05:39:08.666857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678015+0000","last_change":"2024-09-17T05:39:08.678015+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678015+0000","last_undegraded":"2024-09-17T05:39:08.678015+0000","last_fullsized":"2024-09-17T05:39:08.678015+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668363+0000","last_change":"2024-09-17T05:39:08.668363+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668363+0000","last_undegraded":"2024-09-17T05:39:08.668363+0000","last_fullsized":"2024-09-17T05:39:08.668363+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.694612+0000","last_change":"2024-09-17T05:39:09.692704+0000","last_active":"2024-09-17T05:39:09.694612+0000","last_peered":"2024-09-17T05:39:09.694612+0000","last_clean":"2024-09-17T05:39:09.694612+0000","last_became_active":"2024-09-17T05:39:09.690032+0000","last_became_peered":"2024-09-17T05:39:09.690032+0000","last_unstale":"2024-09-17T05:39:09.694612+0000","last_undegraded":"2024-09-17T05:39:09.694612+0000","last_fullsized":"2024-09-17T05:39:09.694612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:10.630444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662857+0000","last_change":"2024-09-17T05:39:08.662857+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662857+0000","last_undegraded":"2024-09-17T05:39:08.662857+0000","last_fullsized":"2024-09-17T05:39:08.662857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702572+0000","last_change":"2024-09-17T05:39:09.698973+0000","last_active":"2024-09-17T05:39:09.702572+0000","last_peered":"2024-09-17T05:39:09.702572+0000","last_clean":"2024-09-17T05:39:09.702572+0000","last_became_active":"2024-09-17T05:39:09.698375+0000","last_became_peered":"2024-09-17T05:39:09.698375+0000","last_unstale":"2024-09-17T05:39:09.702572+0000","last_undegraded":"2024-09-17T05:39:09.702572+0000","last_fullsized":"2024-09-17T05:39:09.702572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:28.895178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666042+0000","last_change":"2024-09-17T05:39:08.666042+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666042+0000","last_undegraded":"2024-09-17T05:39:08.666042+0000","last_fullsized":"2024-09-17T05:39:08.666042+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662651+0000","last_change":"2024-09-17T05:39:08.662651+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662651+0000","last_undegraded":"2024-09-17T05:39:08.662651+0000","last_fullsized":"2024-09-17T05:39:08.662651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675522+0000","last_change":"2024-09-17T05:39:08.675522+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675522+0000","last_undegraded":"2024-09-17T05:39:08.675522+0000","last_fullsized":"2024-09-17T05:39:08.675522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662332+0000","last_change":"2024-09-17T05:39:08.662332+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662332+0000","last_undegraded":"2024-09-17T05:39:08.662332+0000","last_fullsized":"2024-09-17T05:39:08.662332+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.677152+0000","last_change":"2024-09-17T05:39:08.677152+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.677152+0000","last_undegraded":"2024-09-17T05:39:08.677152+0000","last_fullsized":"2024-09-17T05:39:08.677152+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.677957+0000","last_change":"2024-09-17T05:39:08.677957+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.677957+0000","last_undegraded":"2024-09-17T05:39:08.677957+0000","last_fullsized":"2024-09-17T05:39:08.677957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676749+0000","last_change":"2024-09-17T05:39:08.676749+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676749+0000","last_undegraded":"2024-09-17T05:39:08.676749+0000","last_fullsized":"2024-09-17T05:39:08.676749+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.663391+0000","last_change":"2024-09-17T05:39:08.663391+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.663391+0000","last_undegraded":"2024-09-17T05:39:08.663391+0000","last_fullsized":"2024-09-17T05:39:08.663391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668585+0000","last_change":"2024-09-17T05:39:08.668585+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668585+0000","last_undegraded":"2024-09-17T05:39:08.668585+0000","last_fullsized":"2024-09-17T05:39:08.668585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678343+0000","last_change":"2024-09-17T05:39:08.678343+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678343+0000","last_undegraded":"2024-09-17T05:39:08.678343+0000","last_fullsized":"2024-09-17T05:39:08.678343+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.699887+0000","last_change":"2024-09-17T05:39:09.693510+0000","last_active":"2024-09-17T05:39:09.699887+0000","last_peered":"2024-09-17T05:39:09.699887+0000","last_clean":"2024-09-17T05:39:09.699887+0000","last_became_active":"2024-09-17T05:39:09.693223+0000","last_became_peered":"2024-09-17T05:39:09.693223+0000","last_unstale":"2024-09-17T05:39:09.699887+0000","last_undegraded":"2024-09-17T05:39:09.699887+0000","last_fullsized":"2024-09-17T05:39:09.699887+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:54.958914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674848+0000","last_change":"2024-09-17T05:39:08.674848+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674848+0000","last_undegraded":"2024-09-17T05:39:08.674848+0000","last_fullsized":"2024-09-17T05:39:08.674848+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.670468+0000","last_change":"2024-09-17T05:39:08.670468+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.670468+0000","last_undegraded":"2024-09-17T05:39:08.670468+0000","last_fullsized":"2024-09-17T05:39:08.670468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671200+0000","last_change":"2024-09-17T05:39:08.671200+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671200+0000","last_undegraded":"2024-09-17T05:39:08.671200+0000","last_fullsized":"2024-09-17T05:39:08.671200+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675452+0000","last_change":"2024-09-17T05:39:08.675452+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675452+0000","last_undegraded":"2024-09-17T05:39:08.675452+0000","last_fullsized":"2024-09-17T05:39:08.675452+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669290+0000","last_change":"2024-09-17T05:39:08.669290+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669290+0000","last_undegraded":"2024-09-17T05:39:08.669290+0000","last_fullsized":"2024-09-17T05:39:08.669290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.703188+0000","last_change":"2024-09-17T05:39:09.698329+0000","last_active":"2024-09-17T05:39:09.703188+0000","last_peered":"2024-09-17T05:39:09.703188+0000","last_clean":"2024-09-17T05:39:09.703188+0000","last_became_active":"2024-09-17T05:39:09.698084+0000","last_became_peered":"2024-09-17T05:39:09.698084+0000","last_unstale":"2024-09-17T05:39:09.703188+0000","last_undegraded":"2024-09-17T05:39:09.703188+0000","last_fullsized":"2024-09-17T05:39:09.703188+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:47.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.678152+0000","last_change":"2024-09-17T05:39:08.676570+0000","last_active":"2024-09-17T05:39:08.678152+0000","last_peered":"2024-09-17T05:39:08.678152+0000","last_clean":"2024-09-17T05:39:08.678152+0000","last_became_active":"2024-09-17T05:39:08.676332+0000","last_became_peered":"2024-09-17T05:39:08.676332+0000","last_unstale":"2024-09-17T05:39:08.678152+0000","last_undegraded":"2024-09-17T05:39:08.678152+0000","last_fullsized":"2024-09-17T05:39:08.678152+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:03.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666556+0000","last_change":"2024-09-17T05:39:08.666556+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666556+0000","last_undegraded":"2024-09-17T05:39:08.666556+0000","last_fullsized":"2024-09-17T05:39:08.666556+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684311+0000","last_change":"2024-09-17T05:39:08.681409+0000","last_active":"2024-09-17T05:39:08.684311+0000","last_peered":"2024-09-17T05:39:08.684311+0000","last_clean":"2024-09-17T05:39:08.684311+0000","last_became_active":"2024-09-17T05:39:08.681149+0000","last_became_peered":"2024-09-17T05:39:08.681149+0000","last_unstale":"2024-09-17T05:39:08.684311+0000","last_undegraded":"2024-09-17T05:39:08.684311+0000","last_fullsized":"2024-09-17T05:39:08.684311+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:57.845999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667745+0000","last_change":"2024-09-17T05:39:08.667745+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667745+0000","last_undegraded":"2024-09-17T05:39:08.667745+0000","last_fullsized":"2024-09-17T05:39:08.667745+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.674756+0000","last_change":"2024-09-17T05:39:08.673698+0000","last_active":"2024-09-17T05:39:08.674756+0000","last_peered":"2024-09-17T05:39:08.674756+0000","last_clean":"2024-09-17T05:39:08.674756+0000","last_became_active":"2024-09-17T05:39:08.673385+0000","last_became_peered":"2024-09-17T05:39:08.673385+0000","last_unstale":"2024-09-17T05:39:08.674756+0000","last_undegraded":"2024-09-17T05:39:08.674756+0000","last_fullsized":"2024-09-17T05:39:08.674756+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:08.684706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674269+0000","last_change":"2024-09-17T05:39:08.674269+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674269+0000","last_undegraded":"2024-09-17T05:39:08.674269+0000","last_fullsized":"2024-09-17T05:39:08.674269+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.683316+0000","last_change":"2024-09-17T05:39:08.680638+0000","last_active":"2024-09-17T05:39:08.683316+0000","last_peered":"2024-09-17T05:39:08.683316+0000","last_clean":"2024-09-17T05:39:08.683316+0000","last_became_active":"2024-09-17T05:39:08.680162+0000","last_became_peered":"2024-09-17T05:39:08.680162+0000","last_unstale":"2024-09-17T05:39:08.683316+0000","last_undegraded":"2024-09-17T05:39:08.683316+0000","last_fullsized":"2024-09-17T05:39:08.683316+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:50.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662482+0000","last_change":"2024-09-17T05:39:08.662482+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662482+0000","last_undegraded":"2024-09-17T05:39:08.662482+0000","last_fullsized":"2024-09-17T05:39:08.662482+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.680929+0000","last_change":"2024-09-17T05:39:08.679285+0000","last_active":"2024-09-17T05:39:08.680929+0000","last_peered":"2024-09-17T05:39:08.680929+0000","last_clean":"2024-09-17T05:39:08.680929+0000","last_became_active":"2024-09-17T05:39:08.679008+0000","last_became_peered":"2024-09-17T05:39:08.679008+0000","last_unstale":"2024-09-17T05:39:08.680929+0000","last_undegraded":"2024-09-17T05:39:08.680929+0000","last_fullsized":"2024-09-17T05:39:08.680929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:43.119124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676787+0000","last_change":"2024-09-17T05:39:08.676787+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676787+0000","last_undegraded":"2024-09-17T05:39:08.676787+0000","last_fullsized":"2024-09-17T05:39:08.676787+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685185+0000","last_change":"2024-09-17T05:39:08.682028+0000","last_active":"2024-09-17T05:39:08.685185+0000","last_peered":"2024-09-17T05:39:08.685185+0000","last_clean":"2024-09-17T05:39:08.685185+0000","last_became_active":"2024-09-17T05:39:08.680621+0000","last_became_peered":"2024-09-17T05:39:08.680621+0000","last_unstale":"2024-09-17T05:39:08.685185+0000","last_undegraded":"2024-09-17T05:39:08.685185+0000","last_fullsized":"2024-09-17T05:39:08.685185+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:58.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673730+0000","last_change":"2024-09-17T05:39:08.673730+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673730+0000","last_undegraded":"2024-09-17T05:39:08.673730+0000","last_fullsized":"2024-09-17T05:39:08.673730+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687379+0000","last_change":"2024-09-17T05:39:08.683990+0000","last_active":"2024-09-17T05:39:08.687379+0000","last_peered":"2024-09-17T05:39:08.687379+0000","last_clean":"2024-09-17T05:39:08.687379+0000","last_became_active":"2024-09-17T05:39:08.683688+0000","last_became_peered":"2024-09-17T05:39:08.683688+0000","last_unstale":"2024-09-17T05:39:08.687379+0000","last_undegraded":"2024-09-17T05:39:08.687379+0000","last_fullsized":"2024-09-17T05:39:08.687379+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:42.151787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669585+0000","last_change":"2024-09-17T05:39:08.669585+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669585+0000","last_undegraded":"2024-09-17T05:39:08.669585+0000","last_fullsized":"2024-09-17T05:39:08.669585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.686789+0000","last_change":"2024-09-17T05:39:08.686070+0000","last_active":"2024-09-17T05:39:08.686789+0000","last_peered":"2024-09-17T05:39:08.686789+0000","last_clean":"2024-09-17T05:39:08.686789+0000","last_became_active":"2024-09-17T05:39:08.685786+0000","last_became_peered":"2024-09-17T05:39:08.685786+0000","last_unstale":"2024-09-17T05:39:08.686789+0000","last_undegraded":"2024-09-17T05:39:08.686789+0000","last_fullsized":"2024-09-17T05:39:08.686789+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:13.271583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671412+0000","last_change":"2024-09-17T05:39:08.671412+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671412+0000","last_undegraded":"2024-09-17T05:39:08.671412+0000","last_fullsized":"2024-09-17T05:39:08.671412+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685545+0000","last_change":"2024-09-17T05:39:08.680218+0000","last_active":"2024-09-17T05:39:08.685545+0000","last_peered":"2024-09-17T05:39:08.685545+0000","last_clean":"2024-09-17T05:39:08.685545+0000","last_became_active":"2024-09-17T05:39:08.678856+0000","last_became_peered":"2024-09-17T05:39:08.678856+0000","last_unstale":"2024-09-17T05:39:08.685545+0000","last_undegraded":"2024-09-17T05:39:08.685545+0000","last_fullsized":"2024-09-17T05:39:08.685545+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:25.993270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667957+0000","last_change":"2024-09-17T05:39:08.667957+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667957+0000","last_undegraded":"2024-09-17T05:39:08.667957+0000","last_fullsized":"2024-09-17T05:39:08.667957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.681739+0000","last_change":"2024-09-17T05:39:08.677955+0000","last_active":"2024-09-17T05:39:08.681739+0000","last_peered":"2024-09-17T05:39:08.681739+0000","last_clean":"2024-09-17T05:39:08.681739+0000","last_became_active":"2024-09-17T05:39:08.677694+0000","last_became_peered":"2024-09-17T05:39:08.677694+0000","last_unstale":"2024-09-17T05:39:08.681739+0000","last_undegraded":"2024-09-17T05:39:08.681739+0000","last_fullsized":"2024-09-17T05:39:08.681739+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:41.299619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674657+0000","last_change":"2024-09-17T05:39:08.674657+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674657+0000","last_undegraded":"2024-09-17T05:39:08.674657+0000","last_fullsized":"2024-09-17T05:39:08.674657+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.677043+0000","last_change":"2024-09-17T05:39:08.672074+0000","last_active":"2024-09-17T05:39:08.677043+0000","last_peered":"2024-09-17T05:39:08.677043+0000","last_clean":"2024-09-17T05:39:08.677043+0000","last_became_active":"2024-09-17T05:39:08.671800+0000","last_became_peered":"2024-09-17T05:39:08.671800+0000","last_unstale":"2024-09-17T05:39:08.677043+0000","last_undegraded":"2024-09-17T05:39:08.677043+0000","last_fullsized":"2024-09-17T05:39:08.677043+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:25.993270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673883+0000","last_change":"2024-09-17T05:39:08.673883+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673883+0000","last_undegraded":"2024-09-17T05:39:08.673883+0000","last_fullsized":"2024-09-17T05:39:08.673883+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.689377+0000","last_change":"2024-09-17T05:39:08.686779+0000","last_active":"2024-09-17T05:39:08.689377+0000","last_peered":"2024-09-17T05:39:08.689377+0000","last_clean":"2024-09-17T05:39:08.689377+0000","last_became_active":"2024-09-17T05:39:08.686086+0000","last_became_peered":"2024-09-17T05:39:08.686086+0000","last_unstale":"2024-09-17T05:39:08.689377+0000","last_undegraded":"2024-09-17T05:39:08.689377+0000","last_fullsized":"2024-09-17T05:39:08.689377+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:00.213923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.707208+0000","last_change":"2024-09-17T05:39:09.702938+0000","last_active":"2024-09-17T05:39:09.707208+0000","last_peered":"2024-09-17T05:39:09.707208+0000","last_clean":"2024-09-17T05:39:09.707208+0000","last_became_active":"2024-09-17T05:39:09.702616+0000","last_became_peered":"2024-09-17T05:39:09.702616+0000","last_unstale":"2024-09-17T05:39:09.707208+0000","last_undegraded":"2024-09-17T05:39:09.707208+0000","last_fullsized":"2024-09-17T05:39:09.707208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:30.455771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683424+0000","last_change":"2024-09-17T05:39:08.674569+0000","last_active":"2024-09-17T05:39:09.683424+0000","last_peered":"2024-09-17T05:39:09.683424+0000","last_clean":"2024-09-17T05:39:09.683424+0000","last_became_active":"2024-09-17T05:39:08.674285+0000","last_became_peered":"2024-09-17T05:39:08.674285+0000","last_unstale":"2024-09-17T05:39:09.683424+0000","last_undegraded":"2024-09-17T05:39:09.683424+0000","last_fullsized":"2024-09-17T05:39:09.683424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:13.783924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702018+0000","last_change":"2024-09-17T05:39:09.698643+0000","last_active":"2024-09-17T05:39:09.702018+0000","last_peered":"2024-09-17T05:39:09.702018+0000","last_clean":"2024-09-17T05:39:09.702018+0000","last_became_active":"2024-09-17T05:39:09.698013+0000","last_became_peered":"2024-09-17T05:39:09.698013+0000","last_unstale":"2024-09-17T05:39:09.702018+0000","last_undegraded":"2024-09-17T05:39:09.702018+0000","last_fullsized":"2024-09-17T05:39:09.702018+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:42.265017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685479+0000","last_change":"2024-09-17T05:39:08.684608+0000","last_active":"2024-09-17T05:39:08.685479+0000","last_peered":"2024-09-17T05:39:08.685479+0000","last_clean":"2024-09-17T05:39:08.685479+0000","last_became_active":"2024-09-17T05:39:08.684343+0000","last_became_peered":"2024-09-17T05:39:08.684343+0000","last_unstale":"2024-09-17T05:39:08.685479+0000","last_undegraded":"2024-09-17T05:39:08.685479+0000","last_fullsized":"2024-09-17T05:39:08.685479+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:17.208839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675898+0000","last_change":"2024-09-17T05:39:08.675898+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675898+0000","last_undegraded":"2024-09-17T05:39:08.675898+0000","last_fullsized":"2024-09-17T05:39:08.675898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687856+0000","last_change":"2024-09-17T05:39:08.682503+0000","last_active":"2024-09-17T05:39:08.687856+0000","last_peered":"2024-09-17T05:39:08.687856+0000","last_clean":"2024-09-17T05:39:08.687856+0000","last_became_active":"2024-09-17T05:39:08.682225+0000","last_became_peered":"2024-09-17T05:39:08.682225+0000","last_unstale":"2024-09-17T05:39:08.687856+0000","last_undegraded":"2024-09-17T05:39:08.687856+0000","last_fullsized":"2024-09-17T05:39:08.687856+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:03.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669746+0000","last_change":"2024-09-17T05:39:08.669746+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669746+0000","last_undegraded":"2024-09-17T05:39:08.669746+0000","last_fullsized":"2024-09-17T05:39:08.669746+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.689512+0000","last_change":"2024-09-17T05:39:08.685708+0000","last_active":"2024-09-17T05:39:08.689512+0000","last_peered":"2024-09-17T05:39:08.689512+0000","last_clean":"2024-09-17T05:39:08.689512+0000","last_became_active":"2024-09-17T05:39:08.685380+0000","last_became_peered":"2024-09-17T05:39:08.685380+0000","last_unstale":"2024-09-17T05:39:08.689512+0000","last_undegraded":"2024-09-17T05:39:08.689512+0000","last_fullsized":"2024-09-17T05:39:08.689512+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:54.394077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667465+0000","last_change":"2024-09-17T05:39:08.667465+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667465+0000","last_undegraded":"2024-09-17T05:39:08.667465+0000","last_fullsized":"2024-09-17T05:39:08.667465+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682509+0000","last_change":"2024-09-17T05:39:08.679962+0000","last_active":"2024-09-17T05:39:08.682509+0000","last_peered":"2024-09-17T05:39:08.682509+0000","last_clean":"2024-09-17T05:39:08.682509+0000","last_became_active":"2024-09-17T05:39:08.679323+0000","last_became_peered":"2024-09-17T05:39:08.679323+0000","last_unstale":"2024-09-17T05:39:08.682509+0000","last_undegraded":"2024-09-17T05:39:08.682509+0000","last_fullsized":"2024-09-17T05:39:08.682509+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:22:59.521222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676378+0000","last_change":"2024-09-17T05:39:08.676378+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676378+0000","last_undegraded":"2024-09-17T05:39:08.676378+0000","last_fullsized":"2024-09-17T05:39:08.676378+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.683040+0000","last_change":"2024-09-17T05:39:08.679584+0000","last_active":"2024-09-17T05:39:08.683040+0000","last_peered":"2024-09-17T05:39:08.683040+0000","last_clean":"2024-09-17T05:39:08.683040+0000","last_became_active":"2024-09-17T05:39:08.679238+0000","last_became_peered":"2024-09-17T05:39:08.679238+0000","last_unstale":"2024-09-17T05:39:08.683040+0000","last_undegraded":"2024-09-17T05:39:08.683040+0000","last_fullsized":"2024-09-17T05:39:08.683040+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:09.538974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671695+0000","last_change":"2024-09-17T05:39:08.671695+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671695+0000","last_undegraded":"2024-09-17T05:39:08.671695+0000","last_fullsized":"2024-09-17T05:39:08.671695+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.686594+0000","last_change":"2024-09-17T05:39:08.683294+0000","last_active":"2024-09-17T05:39:08.686594+0000","last_peered":"2024-09-17T05:39:08.686594+0000","last_clean":"2024-09-17T05:39:08.686594+0000","last_became_active":"2024-09-17T05:39:08.683029+0000","last_became_peered":"2024-09-17T05:39:08.683029+0000","last_unstale":"2024-09-17T05:39:08.686594+0000","last_undegraded":"2024-09-17T05:39:08.686594+0000","last_fullsized":"2024-09-17T05:39:08.686594+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:59.049915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.670707+0000","last_change":"2024-09-17T05:39:08.670707+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.670707+0000","last_undegraded":"2024-09-17T05:39:08.670707+0000","last_fullsized":"2024-09-17T05:39:08.670707+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682915+0000","last_change":"2024-09-17T05:39:08.677212+0000","last_active":"2024-09-17T05:39:08.682915+0000","last_peered":"2024-09-17T05:39:08.682915+0000","last_clean":"2024-09-17T05:39:08.682915+0000","last_became_active":"2024-09-17T05:39:08.676885+0000","last_became_peered":"2024-09-17T05:39:08.676885+0000","last_unstale":"2024-09-17T05:39:08.682915+0000","last_undegraded":"2024-09-17T05:39:08.682915+0000","last_fullsized":"2024-09-17T05:39:08.682915+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:39.434467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673322+0000","last_change":"2024-09-17T05:39:08.673322+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673322+0000","last_undegraded":"2024-09-17T05:39:08.673322+0000","last_fullsized":"2024-09-17T05:39:08.673322+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685210+0000","last_change":"2024-09-17T05:39:08.681401+0000","last_active":"2024-09-17T05:39:08.685210+0000","last_peered":"2024-09-17T05:39:08.685210+0000","last_clean":"2024-09-17T05:39:08.685210+0000","last_became_active":"2024-09-17T05:39:08.681071+0000","last_became_peered":"2024-09-17T05:39:08.681071+0000","last_unstale":"2024-09-17T05:39:08.685210+0000","last_undegraded":"2024-09-17T05:39:08.685210+0000","last_fullsized":"2024-09-17T05:39:08.685210+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:07.351912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.690838+0000","last_change":"2024-09-17T05:39:09.688991+0000","last_active":"2024-09-17T05:39:09.690838+0000","last_peered":"2024-09-17T05:39:09.690838+0000","last_clean":"2024-09-17T05:39:09.690838+0000","last_became_active":"2024-09-17T05:39:09.688714+0000","last_became_peered":"2024-09-17T05:39:09.688714+0000","last_unstale":"2024-09-17T05:39:09.690838+0000","last_undegraded":"2024-09-17T05:39:09.690838+0000","last_fullsized":"2024-09-17T05:39:09.690838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:44.618638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683352+0000","last_change":"2024-09-17T05:39:08.670676+0000","last_active":"2024-09-17T05:39:09.683352+0000","last_peered":"2024-09-17T05:39:09.683352+0000","last_clean":"2024-09-17T05:39:09.683352+0000","last_became_active":"2024-09-17T05:39:08.669918+0000","last_became_peered":"2024-09-17T05:39:08.669918+0000","last_unstale":"2024-09-17T05:39:09.683352+0000","last_undegraded":"2024-09-17T05:39:09.683352+0000","last_fullsized":"2024-09-17T05:39:09.683352+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:23.308245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674275+0000","last_change":"2024-09-17T05:39:08.674275+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674275+0000","last_undegraded":"2024-09-17T05:39:08.674275+0000","last_fullsized":"2024-09-17T05:39:08.674275+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.680290+0000","last_change":"2024-09-17T05:39:08.676843+0000","last_active":"2024-09-17T05:39:08.680290+0000","last_peered":"2024-09-17T05:39:08.680290+0000","last_clean":"2024-09-17T05:39:08.680290+0000","last_became_active":"2024-09-17T05:39:08.676520+0000","last_became_peered":"2024-09-17T05:39:08.676520+0000","last_unstale":"2024-09-17T05:39:08.680290+0000","last_undegraded":"2024-09-17T05:39:08.680290+0000","last_fullsized":"2024-09-17T05:39:08.680290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:09.569656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.672842+0000","last_change":"2024-09-17T05:39:08.672842+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.672842+0000","last_undegraded":"2024-09-17T05:39:08.672842+0000","last_fullsized":"2024-09-17T05:39:08.672842+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684571+0000","last_change":"2024-09-17T05:39:08.681472+0000","last_active":"2024-09-17T05:39:08.684571+0000","last_peered":"2024-09-17T05:39:08.684571+0000","last_clean":"2024-09-17T05:39:08.684571+0000","last_became_active":"2024-09-17T05:39:08.680987+0000","last_became_peered":"2024-09-17T05:39:08.680987+0000","last_unstale":"2024-09-17T05:39:08.684571+0000","last_undegraded":"2024-09-17T05:39:08.684571+0000","last_fullsized":"2024-09-17T05:39:08.684571+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:46.955054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671568+0000","last_change":"2024-09-17T05:39:08.671568+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671568+0000","last_undegraded":"2024-09-17T05:39:08.671568+0000","last_fullsized":"2024-09-17T05:39:08.671568+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.678672+0000","last_change":"2024-09-17T05:39:08.674902+0000","last_active":"2024-09-17T05:39:08.678672+0000","last_peered":"2024-09-17T05:39:08.678672+0000","last_clean":"2024-09-17T05:39:08.678672+0000","last_became_active":"2024-09-17T05:39:08.674649+0000","last_became_peered":"2024-09-17T05:39:08.674649+0000","last_unstale":"2024-09-17T05:39:08.678672+0000","last_undegraded":"2024-09-17T05:39:08.678672+0000","last_fullsized":"2024-09-17T05:39:08.678672+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:23.479650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702767+0000","last_change":"2024-09-17T05:39:09.699303+0000","last_active":"2024-09-17T05:39:09.702767+0000","last_peered":"2024-09-17T05:39:09.702767+0000","last_clean":"2024-09-17T05:39:09.702767+0000","last_became_active":"2024-09-17T05:39:09.698962+0000","last_became_peered":"2024-09-17T05:39:09.698962+0000","last_unstale":"2024-09-17T05:39:09.702767+0000","last_undegraded":"2024-09-17T05:39:09.702767+0000","last_fullsized":"2024-09-17T05:39:09.702767+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:00.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685349+0000","last_change":"2024-09-17T05:39:08.682089+0000","last_active":"2024-09-17T05:39:08.685349+0000","last_peered":"2024-09-17T05:39:08.685349+0000","last_clean":"2024-09-17T05:39:08.685349+0000","last_became_active":"2024-09-17T05:39:08.680263+0000","last_became_peered":"2024-09-17T05:39:08.680263+0000","last_unstale":"2024-09-17T05:39:08.685349+0000","last_undegraded":"2024-09-17T05:39:08.685349+0000","last_fullsized":"2024-09-17T05:39:08.685349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:47.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668496+0000","last_change":"2024-09-17T05:39:08.668496+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668496+0000","last_undegraded":"2024-09-17T05:39:08.668496+0000","last_fullsized":"2024-09-17T05:39:08.668496+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.684419+0000","last_change":"2024-09-17T05:39:08.673802+0000","last_active":"2024-09-17T05:39:09.684419+0000","last_peered":"2024-09-17T05:39:09.684419+0000","last_clean":"2024-09-17T05:39:09.684419+0000","last_became_active":"2024-09-17T05:39:08.673520+0000","last_became_peered":"2024-09-17T05:39:08.673520+0000","last_unstale":"2024-09-17T05:39:09.684419+0000","last_undegraded":"2024-09-17T05:39:09.684419+0000","last_fullsized":"2024-09-17T05:39:09.684419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:57.829343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673821+0000","last_change":"2024-09-17T05:39:08.673821+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673821+0000","last_undegraded":"2024-09-17T05:39:08.673821+0000","last_fullsized":"2024-09-17T05:39:08.673821+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684939+0000","last_change":"2024-09-17T05:39:08.683259+0000","last_active":"2024-09-17T05:39:08.684939+0000","last_peered":"2024-09-17T05:39:08.684939+0000","last_clean":"2024-09-17T05:39:08.684939+0000","last_became_active":"2024-09-17T05:39:08.682951+0000","last_became_peered":"2024-09-17T05:39:08.682951+0000","last_unstale":"2024-09-17T05:39:08.684939+0000","last_undegraded":"2024-09-17T05:39:08.684939+0000","last_fullsized":"2024-09-17T05:39:08.684939+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:33.782284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676324+0000","last_change":"2024-09-17T05:39:08.676324+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676324+0000","last_undegraded":"2024-09-17T05:39:08.676324+0000","last_fullsized":"2024-09-17T05:39:08.676324+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687860+0000","last_change":"2024-09-17T05:39:08.683416+0000","last_active":"2024-09-17T05:39:08.687860+0000","last_peered":"2024-09-17T05:39:08.687860+0000","last_clean":"2024-09-17T05:39:08.687860+0000","last_became_active":"2024-09-17T05:39:08.682597+0000","last_became_peered":"2024-09-17T05:39:08.682597+0000","last_unstale":"2024-09-17T05:39:08.687860+0000","last_undegraded":"2024-09-17T05:39:08.687860+0000","last_fullsized":"2024-09-17T05:39:08.687860+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:59.049915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662397+0000","last_change":"2024-09-17T05:39:08.662397+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662397+0000","last_undegraded":"2024-09-17T05:39:08.662397+0000","last_fullsized":"2024-09-17T05:39:08.662397+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685390+0000","last_change":"2024-09-17T05:39:08.683995+0000","last_active":"2024-09-17T05:39:08.685390+0000","last_peered":"2024-09-17T05:39:08.685390+0000","last_clean":"2024-09-17T05:39:08.685390+0000","last_became_active":"2024-09-17T05:39:08.683726+0000","last_became_peered":"2024-09-17T05:39:08.683726+0000","last_unstale":"2024-09-17T05:39:08.685390+0000","last_undegraded":"2024-09-17T05:39:08.685390+0000","last_fullsized":"2024-09-17T05:39:08.685390+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:12.468607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.693873+0000","last_change":"2024-09-17T05:39:09.690892+0000","last_active":"2024-09-17T05:39:09.693873+0000","last_peered":"2024-09-17T05:39:09.693873+0000","last_clean":"2024-09-17T05:39:09.693873+0000","last_became_active":"2024-09-17T05:39:09.690610+0000","last_became_peered":"2024-09-17T05:39:09.690610+0000","last_unstale":"2024-09-17T05:39:09.693873+0000","last_undegraded":"2024-09-17T05:39:09.693873+0000","last_fullsized":"2024-09-17T05:39:09.693873+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:16.440719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683840+0000","last_change":"2024-09-17T05:39:08.673866+0000","last_active":"2024-09-17T05:39:09.683840+0000","last_peered":"2024-09-17T05:39:09.683840+0000","last_clean":"2024-09-17T05:39:09.683840+0000","last_became_active":"2024-09-17T05:39:08.673542+0000","last_became_peered":"2024-09-17T05:39:08.673542+0000","last_unstale":"2024-09-17T05:39:09.683840+0000","last_undegraded":"2024-09-17T05:39:09.683840+0000","last_fullsized":"2024-09-17T05:39:09.683840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:11.138555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666672+0000","last_change":"2024-09-17T05:39:08.666672+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666672+0000","last_undegraded":"2024-09-17T05:39:08.666672+0000","last_fullsized":"2024-09-17T05:39:08.666672+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682043+0000","last_change":"2024-09-17T05:39:08.680453+0000","last_active":"2024-09-17T05:39:08.682043+0000","last_peered":"2024-09-17T05:39:08.682043+0000","last_clean":"2024-09-17T05:39:08.682043+0000","last_became_active":"2024-09-17T05:39:08.680180+0000","last_became_peered":"2024-09-17T05:39:08.680180+0000","last_unstale":"2024-09-17T05:39:08.682043+0000","last_undegraded":"2024-09-17T05:39:08.682043+0000","last_fullsized":"2024-09-17T05:39:08.682043+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:23.479650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":317,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.666710+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:39:08.666710+0000","last_peered":"2024-09-17T05:39:08.666710+0000","last_clean":"2024-09-17T05:39:08.666710+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:39:08.666710+0000","last_undegraded":"2024-09-17T05:39:08.666710+0000","last_fullsized":"2024-09-17T05:39:08.666710+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509614,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127104,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244736,"statfs":{"total":96636764160,"available":96506609664,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136832,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235008,"statfs":{"total":96636764160,"available":96496648192,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131264,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240576,"statfs":{"total":96636764160,"available":96502349824,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509613,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131268,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240572,"statfs":{"total":96636764160,"available":96502345728,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127092,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244748,"statfs":{"total":96636764160,"available":96506621952,"internally_reserved":0,"allocated":1601536,"data_stored":703037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509615,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145716,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226124,"statfs":{"total":96636764160,"available":96487550976,"internally_reserved":0,"allocated":2195456,"data_stored":1293405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509613,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145148,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226692,"statfs":{"total":96636764160,"available":96488132608,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":146995659},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542319,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131836,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240004,"statfs":{"total":96636764160,"available":96501764096,"internally_reserved":0,"allocated":2195456,"data_stored":1293405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:39:13.807 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:39:13.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.990+0000 7f259b600640 1 -- 172.21.15.122:0/2512858148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f259415ade0 msgr2=0x7f25940cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.990+0000 7f259b600640 1 --2- 172.21.15.122:0/2512858148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f259415ade0 0x7f25940cebf0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f2584009900 tx=0x7f258401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 -- 172.21.15.122:0/2512858148 shutdown_connections 2024-09-17T05:39:13.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 --2- 172.21.15.122:0/2512858148 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f259406e810 0x7f25940cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 --2- 172.21.15.122:0/2512858148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f259415ade0 0x7f25940cebf0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 -- 172.21.15.122:0/2512858148 >> 172.21.15.122:0/2512858148 conn(0x7f25940b18f0 msgr2=0x7f25940b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:13.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 -- 172.21.15.122:0/2512858148 shutdown_connections 2024-09-17T05:39:13.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 -- 172.21.15.122:0/2512858148 wait complete. 2024-09-17T05:39:13.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 Processor -- start 2024-09-17T05:39:13.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 -- start start 2024-09-17T05:39:13.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f259406e810 0x7f25940b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25940b4bd0 0x7f2594152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f259415ade0 0x7f25940b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25941680e0 con 0x7f25940b4bd0 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2594167f60 con 0x7f259406e810 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f259b600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2594168260 con 0x7f259415ade0 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25940b4bd0 0x7f2594152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25940b4bd0 0x7f2594152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38824/0 (socket says 172.21.15.122:38824) 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 -- 172.21.15.122:0/608035363 learned_addr learned my addr 172.21.15.122:0/608035363 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f2598c00640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f259406e810 0x7f25940b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f2599600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f259415ade0 0x7f25940b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 -- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f259415ade0 msgr2=0x7f25940b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f259415ade0 0x7f25940b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 -- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f259406e810 msgr2=0x7f25940b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:13.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f259406e810 0x7f25940b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:13.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 -- 172.21.15.122:0/608035363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2594165320 con 0x7f25940b4bd0 2024-09-17T05:39:13.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f2599600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f259415ade0 0x7f25940b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:13.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.994+0000 7f258be00640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25940b4bd0 0x7f2594152df0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f257c00b550 tx=0x7f257c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.998+0000 7f2591600640 1 -- 172.21.15.122:0/608035363 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c014070 con 0x7f25940b4bd0 2024-09-17T05:39:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.998+0000 7f2591600640 1 -- 172.21.15.122:0/608035363 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f257c010980 con 0x7f25940b4bd0 2024-09-17T05:39:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.998+0000 7f2591600640 1 -- 172.21.15.122:0/608035363 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c010dd0 con 0x7f25940b4bd0 2024-09-17T05:39:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.998+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2594165600 con 0x7f25940b4bd0 2024-09-17T05:39:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.998+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25940afc30 con 0x7f25940b4bd0 2024-09-17T05:39:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.998+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25940afe30 con 0x7f25940b4bd0 2024-09-17T05:39:13.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:13.998+0000 7f2591600640 1 -- 172.21.15.122:0/608035363 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f257c017070 con 0x7f25940b4bd0 2024-09-17T05:39:13.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.002+0000 7f2591600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25700751d0 0x7f2570077690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:13.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.002+0000 7f2591600640 1 -- 172.21.15.122:0/608035363 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f257c092fc0 con 0x7f25940b4bd0 2024-09-17T05:39:13.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.002+0000 7f2598c00640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25700751d0 0x7f2570077690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:13.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.002+0000 7f2598c00640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25700751d0 0x7f2570077690 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f25840af000 tx=0x7f258401b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:13.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.006+0000 7f2591600640 1 -- 172.21.15.122:0/608035363 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f25940afe30 con 0x7f25940b4bd0 2024-09-17T05:39:14.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.170+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f259415e8e0 con 0x7f25700751d0 2024-09-17T05:39:14.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.174+0000 7f2591600640 1 -- 172.21.15.122:0/608035363 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247873 (secure 0 0 0) 0x7f259415e8e0 con 0x7f25700751d0 2024-09-17T05:39:14.172 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:14.174 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:39:14.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25700751d0 msgr2=0x7f2570077690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25700751d0 0x7f2570077690 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f25840af000 tx=0x7f258401b120 comp rx=0 tx=0).stop 2024-09-17T05:39:14.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25940b4bd0 msgr2=0x7f2594152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25940b4bd0 0x7f2594152df0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f257c00b550 tx=0x7f257c00ba10 comp rx=0 tx=0).stop 2024-09-17T05:39:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 shutdown_connections 2024-09-17T05:39:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25700751d0 0x7f2570077690 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f259415ade0 0x7f25940b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25940b4bd0 0x7f2594152df0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 --2- 172.21.15.122:0/608035363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f259406e810 0x7f25940b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 >> 172.21.15.122:0/608035363 conn(0x7f25940b18f0 msgr2=0x7f25940aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 shutdown_connections 2024-09-17T05:39:14.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.178+0000 7f259b600640 1 -- 172.21.15.122:0/608035363 wait complete. 2024-09-17T05:39:14.192 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":555,"stamp":"2024-09-17T05:39:12.736127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076260,"kb_used_data":13732,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898460,"statfs":{"total":773094113280,"available":771992023040,"internally_reserved":0,"allocated":14061568,"data_stored":6836827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1087819318},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.080111"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675684+0000","last_change":"2024-09-17T05:39:08.675684+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675684+0000","last_undegraded":"2024-09-17T05:39:08.675684+0000","last_fullsized":"2024-09-17T05:39:08.675684+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.681063+0000","last_change":"2024-09-17T05:39:08.681063+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.681063+0000","last_undegraded":"2024-09-17T05:39:08.681063+0000","last_fullsized":"2024-09-17T05:39:08.681063+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668951+0000","last_change":"2024-09-17T05:39:08.668951+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668951+0000","last_undegraded":"2024-09-17T05:39:08.668951+0000","last_fullsized":"2024-09-17T05:39:08.668951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666359+0000","last_change":"2024-09-17T05:39:08.666359+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666359+0000","last_undegraded":"2024-09-17T05:39:08.666359+0000","last_fullsized":"2024-09-17T05:39:08.666359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667851+0000","last_change":"2024-09-17T05:39:08.667851+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667851+0000","last_undegraded":"2024-09-17T05:39:08.667851+0000","last_fullsized":"2024-09-17T05:39:08.667851+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674261+0000","last_change":"2024-09-17T05:39:08.674261+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674261+0000","last_undegraded":"2024-09-17T05:39:08.674261+0000","last_fullsized":"2024-09-17T05:39:08.674261+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667467+0000","last_change":"2024-09-17T05:39:08.667467+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667467+0000","last_undegraded":"2024-09-17T05:39:08.667467+0000","last_fullsized":"2024-09-17T05:39:08.667467+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668743+0000","last_change":"2024-09-17T05:39:08.668743+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668743+0000","last_undegraded":"2024-09-17T05:39:08.668743+0000","last_fullsized":"2024-09-17T05:39:08.668743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667026+0000","last_change":"2024-09-17T05:39:08.667026+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667026+0000","last_undegraded":"2024-09-17T05:39:08.667026+0000","last_fullsized":"2024-09-17T05:39:08.667026+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678847+0000","last_change":"2024-09-17T05:39:08.678847+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678847+0000","last_undegraded":"2024-09-17T05:39:08.678847+0000","last_fullsized":"2024-09-17T05:39:08.678847+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666857+0000","last_change":"2024-09-17T05:39:08.666857+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666857+0000","last_undegraded":"2024-09-17T05:39:08.666857+0000","last_fullsized":"2024-09-17T05:39:08.666857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678015+0000","last_change":"2024-09-17T05:39:08.678015+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678015+0000","last_undegraded":"2024-09-17T05:39:08.678015+0000","last_fullsized":"2024-09-17T05:39:08.678015+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668363+0000","last_change":"2024-09-17T05:39:08.668363+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668363+0000","last_undegraded":"2024-09-17T05:39:08.668363+0000","last_fullsized":"2024-09-17T05:39:08.668363+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.694612+0000","last_change":"2024-09-17T05:39:09.692704+0000","last_active":"2024-09-17T05:39:09.694612+0000","last_peered":"2024-09-17T05:39:09.694612+0000","last_clean":"2024-09-17T05:39:09.694612+0000","last_became_active":"2024-09-17T05:39:09.690032+0000","last_became_peered":"2024-09-17T05:39:09.690032+0000","last_unstale":"2024-09-17T05:39:09.694612+0000","last_undegraded":"2024-09-17T05:39:09.694612+0000","last_fullsized":"2024-09-17T05:39:09.694612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:10.630444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662857+0000","last_change":"2024-09-17T05:39:08.662857+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662857+0000","last_undegraded":"2024-09-17T05:39:08.662857+0000","last_fullsized":"2024-09-17T05:39:08.662857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702572+0000","last_change":"2024-09-17T05:39:09.698973+0000","last_active":"2024-09-17T05:39:09.702572+0000","last_peered":"2024-09-17T05:39:09.702572+0000","last_clean":"2024-09-17T05:39:09.702572+0000","last_became_active":"2024-09-17T05:39:09.698375+0000","last_became_peered":"2024-09-17T05:39:09.698375+0000","last_unstale":"2024-09-17T05:39:09.702572+0000","last_undegraded":"2024-09-17T05:39:09.702572+0000","last_fullsized":"2024-09-17T05:39:09.702572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:28.895178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666042+0000","last_change":"2024-09-17T05:39:08.666042+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666042+0000","last_undegraded":"2024-09-17T05:39:08.666042+0000","last_fullsized":"2024-09-17T05:39:08.666042+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662651+0000","last_change":"2024-09-17T05:39:08.662651+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662651+0000","last_undegraded":"2024-09-17T05:39:08.662651+0000","last_fullsized":"2024-09-17T05:39:08.662651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675522+0000","last_change":"2024-09-17T05:39:08.675522+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675522+0000","last_undegraded":"2024-09-17T05:39:08.675522+0000","last_fullsized":"2024-09-17T05:39:08.675522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662332+0000","last_change":"2024-09-17T05:39:08.662332+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662332+0000","last_undegraded":"2024-09-17T05:39:08.662332+0000","last_fullsized":"2024-09-17T05:39:08.662332+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.677152+0000","last_change":"2024-09-17T05:39:08.677152+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.677152+0000","last_undegraded":"2024-09-17T05:39:08.677152+0000","last_fullsized":"2024-09-17T05:39:08.677152+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.677957+0000","last_change":"2024-09-17T05:39:08.677957+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.677957+0000","last_undegraded":"2024-09-17T05:39:08.677957+0000","last_fullsized":"2024-09-17T05:39:08.677957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676749+0000","last_change":"2024-09-17T05:39:08.676749+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676749+0000","last_undegraded":"2024-09-17T05:39:08.676749+0000","last_fullsized":"2024-09-17T05:39:08.676749+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.663391+0000","last_change":"2024-09-17T05:39:08.663391+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.663391+0000","last_undegraded":"2024-09-17T05:39:08.663391+0000","last_fullsized":"2024-09-17T05:39:08.663391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668585+0000","last_change":"2024-09-17T05:39:08.668585+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668585+0000","last_undegraded":"2024-09-17T05:39:08.668585+0000","last_fullsized":"2024-09-17T05:39:08.668585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678343+0000","last_change":"2024-09-17T05:39:08.678343+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678343+0000","last_undegraded":"2024-09-17T05:39:08.678343+0000","last_fullsized":"2024-09-17T05:39:08.678343+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.699887+0000","last_change":"2024-09-17T05:39:09.693510+0000","last_active":"2024-09-17T05:39:09.699887+0000","last_peered":"2024-09-17T05:39:09.699887+0000","last_clean":"2024-09-17T05:39:09.699887+0000","last_became_active":"2024-09-17T05:39:09.693223+0000","last_became_peered":"2024-09-17T05:39:09.693223+0000","last_unstale":"2024-09-17T05:39:09.699887+0000","last_undegraded":"2024-09-17T05:39:09.699887+0000","last_fullsized":"2024-09-17T05:39:09.699887+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:54.958914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674848+0000","last_change":"2024-09-17T05:39:08.674848+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674848+0000","last_undegraded":"2024-09-17T05:39:08.674848+0000","last_fullsized":"2024-09-17T05:39:08.674848+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.670468+0000","last_change":"2024-09-17T05:39:08.670468+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.670468+0000","last_undegraded":"2024-09-17T05:39:08.670468+0000","last_fullsized":"2024-09-17T05:39:08.670468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671200+0000","last_change":"2024-09-17T05:39:08.671200+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671200+0000","last_undegraded":"2024-09-17T05:39:08.671200+0000","last_fullsized":"2024-09-17T05:39:08.671200+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675452+0000","last_change":"2024-09-17T05:39:08.675452+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675452+0000","last_undegraded":"2024-09-17T05:39:08.675452+0000","last_fullsized":"2024-09-17T05:39:08.675452+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669290+0000","last_change":"2024-09-17T05:39:08.669290+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669290+0000","last_undegraded":"2024-09-17T05:39:08.669290+0000","last_fullsized":"2024-09-17T05:39:08.669290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.703188+0000","last_change":"2024-09-17T05:39:09.698329+0000","last_active":"2024-09-17T05:39:09.703188+0000","last_peered":"2024-09-17T05:39:09.703188+0000","last_clean":"2024-09-17T05:39:09.703188+0000","last_became_active":"2024-09-17T05:39:09.698084+0000","last_became_peered":"2024-09-17T05:39:09.698084+0000","last_unstale":"2024-09-17T05:39:09.703188+0000","last_undegraded":"2024-09-17T05:39:09.703188+0000","last_fullsized":"2024-09-17T05:39:09.703188+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:47.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.678152+0000","last_change":"2024-09-17T05:39:08.676570+0000","last_active":"2024-09-17T05:39:08.678152+0000","last_peered":"2024-09-17T05:39:08.678152+0000","last_clean":"2024-09-17T05:39:08.678152+0000","last_became_active":"2024-09-17T05:39:08.676332+0000","last_became_peered":"2024-09-17T05:39:08.676332+0000","last_unstale":"2024-09-17T05:39:08.678152+0000","last_undegraded":"2024-09-17T05:39:08.678152+0000","last_fullsized":"2024-09-17T05:39:08.678152+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:03.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666556+0000","last_change":"2024-09-17T05:39:08.666556+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666556+0000","last_undegraded":"2024-09-17T05:39:08.666556+0000","last_fullsized":"2024-09-17T05:39:08.666556+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684311+0000","last_change":"2024-09-17T05:39:08.681409+0000","last_active":"2024-09-17T05:39:08.684311+0000","last_peered":"2024-09-17T05:39:08.684311+0000","last_clean":"2024-09-17T05:39:08.684311+0000","last_became_active":"2024-09-17T05:39:08.681149+0000","last_became_peered":"2024-09-17T05:39:08.681149+0000","last_unstale":"2024-09-17T05:39:08.684311+0000","last_undegraded":"2024-09-17T05:39:08.684311+0000","last_fullsized":"2024-09-17T05:39:08.684311+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:57.845999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667745+0000","last_change":"2024-09-17T05:39:08.667745+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667745+0000","last_undegraded":"2024-09-17T05:39:08.667745+0000","last_fullsized":"2024-09-17T05:39:08.667745+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.674756+0000","last_change":"2024-09-17T05:39:08.673698+0000","last_active":"2024-09-17T05:39:08.674756+0000","last_peered":"2024-09-17T05:39:08.674756+0000","last_clean":"2024-09-17T05:39:08.674756+0000","last_became_active":"2024-09-17T05:39:08.673385+0000","last_became_peered":"2024-09-17T05:39:08.673385+0000","last_unstale":"2024-09-17T05:39:08.674756+0000","last_undegraded":"2024-09-17T05:39:08.674756+0000","last_fullsized":"2024-09-17T05:39:08.674756+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:08.684706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674269+0000","last_change":"2024-09-17T05:39:08.674269+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674269+0000","last_undegraded":"2024-09-17T05:39:08.674269+0000","last_fullsized":"2024-09-17T05:39:08.674269+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.683316+0000","last_change":"2024-09-17T05:39:08.680638+0000","last_active":"2024-09-17T05:39:08.683316+0000","last_peered":"2024-09-17T05:39:08.683316+0000","last_clean":"2024-09-17T05:39:08.683316+0000","last_became_active":"2024-09-17T05:39:08.680162+0000","last_became_peered":"2024-09-17T05:39:08.680162+0000","last_unstale":"2024-09-17T05:39:08.683316+0000","last_undegraded":"2024-09-17T05:39:08.683316+0000","last_fullsized":"2024-09-17T05:39:08.683316+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:50.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662482+0000","last_change":"2024-09-17T05:39:08.662482+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662482+0000","last_undegraded":"2024-09-17T05:39:08.662482+0000","last_fullsized":"2024-09-17T05:39:08.662482+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.680929+0000","last_change":"2024-09-17T05:39:08.679285+0000","last_active":"2024-09-17T05:39:08.680929+0000","last_peered":"2024-09-17T05:39:08.680929+0000","last_clean":"2024-09-17T05:39:08.680929+0000","last_became_active":"2024-09-17T05:39:08.679008+0000","last_became_peered":"2024-09-17T05:39:08.679008+0000","last_unstale":"2024-09-17T05:39:08.680929+0000","last_undegraded":"2024-09-17T05:39:08.680929+0000","last_fullsized":"2024-09-17T05:39:08.680929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:43.119124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676787+0000","last_change":"2024-09-17T05:39:08.676787+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676787+0000","last_undegraded":"2024-09-17T05:39:08.676787+0000","last_fullsized":"2024-09-17T05:39:08.676787+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685185+0000","last_change":"2024-09-17T05:39:08.682028+0000","last_active":"2024-09-17T05:39:08.685185+0000","last_peered":"2024-09-17T05:39:08.685185+0000","last_clean":"2024-09-17T05:39:08.685185+0000","last_became_active":"2024-09-17T05:39:08.680621+0000","last_became_peered":"2024-09-17T05:39:08.680621+0000","last_unstale":"2024-09-17T05:39:08.685185+0000","last_undegraded":"2024-09-17T05:39:08.685185+0000","last_fullsized":"2024-09-17T05:39:08.685185+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:58.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673730+0000","last_change":"2024-09-17T05:39:08.673730+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673730+0000","last_undegraded":"2024-09-17T05:39:08.673730+0000","last_fullsized":"2024-09-17T05:39:08.673730+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687379+0000","last_change":"2024-09-17T05:39:08.683990+0000","last_active":"2024-09-17T05:39:08.687379+0000","last_peered":"2024-09-17T05:39:08.687379+0000","last_clean":"2024-09-17T05:39:08.687379+0000","last_became_active":"2024-09-17T05:39:08.683688+0000","last_became_peered":"2024-09-17T05:39:08.683688+0000","last_unstale":"2024-09-17T05:39:08.687379+0000","last_undegraded":"2024-09-17T05:39:08.687379+0000","last_fullsized":"2024-09-17T05:39:08.687379+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:42.151787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669585+0000","last_change":"2024-09-17T05:39:08.669585+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669585+0000","last_undegraded":"2024-09-17T05:39:08.669585+0000","last_fullsized":"2024-09-17T05:39:08.669585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.686789+0000","last_change":"2024-09-17T05:39:08.686070+0000","last_active":"2024-09-17T05:39:08.686789+0000","last_peered":"2024-09-17T05:39:08.686789+0000","last_clean":"2024-09-17T05:39:08.686789+0000","last_became_active":"2024-09-17T05:39:08.685786+0000","last_became_peered":"2024-09-17T05:39:08.685786+0000","last_unstale":"2024-09-17T05:39:08.686789+0000","last_undegraded":"2024-09-17T05:39:08.686789+0000","last_fullsized":"2024-09-17T05:39:08.686789+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:13.271583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671412+0000","last_change":"2024-09-17T05:39:08.671412+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671412+0000","last_undegraded":"2024-09-17T05:39:08.671412+0000","last_fullsized":"2024-09-17T05:39:08.671412+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685545+0000","last_change":"2024-09-17T05:39:08.680218+0000","last_active":"2024-09-17T05:39:08.685545+0000","last_peered":"2024-09-17T05:39:08.685545+0000","last_clean":"2024-09-17T05:39:08.685545+0000","last_became_active":"2024-09-17T05:39:08.678856+0000","last_became_peered":"2024-09-17T05:39:08.678856+0000","last_unstale":"2024-09-17T05:39:08.685545+0000","last_undegraded":"2024-09-17T05:39:08.685545+0000","last_fullsized":"2024-09-17T05:39:08.685545+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:25.993270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667957+0000","last_change":"2024-09-17T05:39:08.667957+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667957+0000","last_undegraded":"2024-09-17T05:39:08.667957+0000","last_fullsized":"2024-09-17T05:39:08.667957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.681739+0000","last_change":"2024-09-17T05:39:08.677955+0000","last_active":"2024-09-17T05:39:08.681739+0000","last_peered":"2024-09-17T05:39:08.681739+0000","last_clean":"2024-09-17T05:39:08.681739+0000","last_became_active":"2024-09-17T05:39:08.677694+0000","last_became_peered":"2024-09-17T05:39:08.677694+0000","last_unstale":"2024-09-17T05:39:08.681739+0000","last_undegraded":"2024-09-17T05:39:08.681739+0000","last_fullsized":"2024-09-17T05:39:08.681739+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:41.299619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674657+0000","last_change":"2024-09-17T05:39:08.674657+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674657+0000","last_undegraded":"2024-09-17T05:39:08.674657+0000","last_fullsized":"2024-09-17T05:39:08.674657+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.677043+0000","last_change":"2024-09-17T05:39:08.672074+0000","last_active":"2024-09-17T05:39:08.677043+0000","last_peered":"2024-09-17T05:39:08.677043+0000","last_clean":"2024-09-17T05:39:08.677043+0000","last_became_active":"2024-09-17T05:39:08.671800+0000","last_became_peered":"2024-09-17T05:39:08.671800+0000","last_unstale":"2024-09-17T05:39:08.677043+0000","last_undegraded":"2024-09-17T05:39:08.677043+0000","last_fullsized":"2024-09-17T05:39:08.677043+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:25.993270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673883+0000","last_change":"2024-09-17T05:39:08.673883+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673883+0000","last_undegraded":"2024-09-17T05:39:08.673883+0000","last_fullsized":"2024-09-17T05:39:08.673883+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.689377+0000","last_change":"2024-09-17T05:39:08.686779+0000","last_active":"2024-09-17T05:39:08.689377+0000","last_peered":"2024-09-17T05:39:08.689377+0000","last_clean":"2024-09-17T05:39:08.689377+0000","last_became_active":"2024-09-17T05:39:08.686086+0000","last_became_peered":"2024-09-17T05:39:08.686086+0000","last_unstale":"2024-09-17T05:39:08.689377+0000","last_undegraded":"2024-09-17T05:39:08.689377+0000","last_fullsized":"2024-09-17T05:39:08.689377+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:00.213923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.707208+0000","last_change":"2024-09-17T05:39:09.702938+0000","last_active":"2024-09-17T05:39:09.707208+0000","last_peered":"2024-09-17T05:39:09.707208+0000","last_clean":"2024-09-17T05:39:09.707208+0000","last_became_active":"2024-09-17T05:39:09.702616+0000","last_became_peered":"2024-09-17T05:39:09.702616+0000","last_unstale":"2024-09-17T05:39:09.707208+0000","last_undegraded":"2024-09-17T05:39:09.707208+0000","last_fullsized":"2024-09-17T05:39:09.707208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:30.455771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683424+0000","last_change":"2024-09-17T05:39:08.674569+0000","last_active":"2024-09-17T05:39:09.683424+0000","last_peered":"2024-09-17T05:39:09.683424+0000","last_clean":"2024-09-17T05:39:09.683424+0000","last_became_active":"2024-09-17T05:39:08.674285+0000","last_became_peered":"2024-09-17T05:39:08.674285+0000","last_unstale":"2024-09-17T05:39:09.683424+0000","last_undegraded":"2024-09-17T05:39:09.683424+0000","last_fullsized":"2024-09-17T05:39:09.683424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:13.783924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702018+0000","last_change":"2024-09-17T05:39:09.698643+0000","last_active":"2024-09-17T05:39:09.702018+0000","last_peered":"2024-09-17T05:39:09.702018+0000","last_clean":"2024-09-17T05:39:09.702018+0000","last_became_active":"2024-09-17T05:39:09.698013+0000","last_became_peered":"2024-09-17T05:39:09.698013+0000","last_unstale":"2024-09-17T05:39:09.702018+0000","last_undegraded":"2024-09-17T05:39:09.702018+0000","last_fullsized":"2024-09-17T05:39:09.702018+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:42.265017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685479+0000","last_change":"2024-09-17T05:39:08.684608+0000","last_active":"2024-09-17T05:39:08.685479+0000","last_peered":"2024-09-17T05:39:08.685479+0000","last_clean":"2024-09-17T05:39:08.685479+0000","last_became_active":"2024-09-17T05:39:08.684343+0000","last_became_peered":"2024-09-17T05:39:08.684343+0000","last_unstale":"2024-09-17T05:39:08.685479+0000","last_undegraded":"2024-09-17T05:39:08.685479+0000","last_fullsized":"2024-09-17T05:39:08.685479+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:17.208839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675898+0000","last_change":"2024-09-17T05:39:08.675898+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675898+0000","last_undegraded":"2024-09-17T05:39:08.675898+0000","last_fullsized":"2024-09-17T05:39:08.675898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687856+0000","last_change":"2024-09-17T05:39:08.682503+0000","last_active":"2024-09-17T05:39:08.687856+0000","last_peered":"2024-09-17T05:39:08.687856+0000","last_clean":"2024-09-17T05:39:08.687856+0000","last_became_active":"2024-09-17T05:39:08.682225+0000","last_became_peered":"2024-09-17T05:39:08.682225+0000","last_unstale":"2024-09-17T05:39:08.687856+0000","last_undegraded":"2024-09-17T05:39:08.687856+0000","last_fullsized":"2024-09-17T05:39:08.687856+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:03.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669746+0000","last_change":"2024-09-17T05:39:08.669746+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669746+0000","last_undegraded":"2024-09-17T05:39:08.669746+0000","last_fullsized":"2024-09-17T05:39:08.669746+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.689512+0000","last_change":"2024-09-17T05:39:08.685708+0000","last_active":"2024-09-17T05:39:08.689512+0000","last_peered":"2024-09-17T05:39:08.689512+0000","last_clean":"2024-09-17T05:39:08.689512+0000","last_became_active":"2024-09-17T05:39:08.685380+0000","last_became_peered":"2024-09-17T05:39:08.685380+0000","last_unstale":"2024-09-17T05:39:08.689512+0000","last_undegraded":"2024-09-17T05:39:08.689512+0000","last_fullsized":"2024-09-17T05:39:08.689512+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:54.394077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667465+0000","last_change":"2024-09-17T05:39:08.667465+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667465+0000","last_undegraded":"2024-09-17T05:39:08.667465+0000","last_fullsized":"2024-09-17T05:39:08.667465+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682509+0000","last_change":"2024-09-17T05:39:08.679962+0000","last_active":"2024-09-17T05:39:08.682509+0000","last_peered":"2024-09-17T05:39:08.682509+0000","last_clean":"2024-09-17T05:39:08.682509+0000","last_became_active":"2024-09-17T05:39:08.679323+0000","last_became_peered":"2024-09-17T05:39:08.679323+0000","last_unstale":"2024-09-17T05:39:08.682509+0000","last_undegraded":"2024-09-17T05:39:08.682509+0000","last_fullsized":"2024-09-17T05:39:08.682509+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:22:59.521222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676378+0000","last_change":"2024-09-17T05:39:08.676378+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676378+0000","last_undegraded":"2024-09-17T05:39:08.676378+0000","last_fullsized":"2024-09-17T05:39:08.676378+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.683040+0000","last_change":"2024-09-17T05:39:08.679584+0000","last_active":"2024-09-17T05:39:08.683040+0000","last_peered":"2024-09-17T05:39:08.683040+0000","last_clean":"2024-09-17T05:39:08.683040+0000","last_became_active":"2024-09-17T05:39:08.679238+0000","last_became_peered":"2024-09-17T05:39:08.679238+0000","last_unstale":"2024-09-17T05:39:08.683040+0000","last_undegraded":"2024-09-17T05:39:08.683040+0000","last_fullsized":"2024-09-17T05:39:08.683040+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:09.538974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671695+0000","last_change":"2024-09-17T05:39:08.671695+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671695+0000","last_undegraded":"2024-09-17T05:39:08.671695+0000","last_fullsized":"2024-09-17T05:39:08.671695+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.686594+0000","last_change":"2024-09-17T05:39:08.683294+0000","last_active":"2024-09-17T05:39:08.686594+0000","last_peered":"2024-09-17T05:39:08.686594+0000","last_clean":"2024-09-17T05:39:08.686594+0000","last_became_active":"2024-09-17T05:39:08.683029+0000","last_became_peered":"2024-09-17T05:39:08.683029+0000","last_unstale":"2024-09-17T05:39:08.686594+0000","last_undegraded":"2024-09-17T05:39:08.686594+0000","last_fullsized":"2024-09-17T05:39:08.686594+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:59.049915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.670707+0000","last_change":"2024-09-17T05:39:08.670707+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.670707+0000","last_undegraded":"2024-09-17T05:39:08.670707+0000","last_fullsized":"2024-09-17T05:39:08.670707+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682915+0000","last_change":"2024-09-17T05:39:08.677212+0000","last_active":"2024-09-17T05:39:08.682915+0000","last_peered":"2024-09-17T05:39:08.682915+0000","last_clean":"2024-09-17T05:39:08.682915+0000","last_became_active":"2024-09-17T05:39:08.676885+0000","last_became_peered":"2024-09-17T05:39:08.676885+0000","last_unstale":"2024-09-17T05:39:08.682915+0000","last_undegraded":"2024-09-17T05:39:08.682915+0000","last_fullsized":"2024-09-17T05:39:08.682915+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:39.434467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673322+0000","last_change":"2024-09-17T05:39:08.673322+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673322+0000","last_undegraded":"2024-09-17T05:39:08.673322+0000","last_fullsized":"2024-09-17T05:39:08.673322+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685210+0000","last_change":"2024-09-17T05:39:08.681401+0000","last_active":"2024-09-17T05:39:08.685210+0000","last_peered":"2024-09-17T05:39:08.685210+0000","last_clean":"2024-09-17T05:39:08.685210+0000","last_became_active":"2024-09-17T05:39:08.681071+0000","last_became_peered":"2024-09-17T05:39:08.681071+0000","last_unstale":"2024-09-17T05:39:08.685210+0000","last_undegraded":"2024-09-17T05:39:08.685210+0000","last_fullsized":"2024-09-17T05:39:08.685210+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:07.351912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.690838+0000","last_change":"2024-09-17T05:39:09.688991+0000","last_active":"2024-09-17T05:39:09.690838+0000","last_peered":"2024-09-17T05:39:09.690838+0000","last_clean":"2024-09-17T05:39:09.690838+0000","last_became_active":"2024-09-17T05:39:09.688714+0000","last_became_peered":"2024-09-17T05:39:09.688714+0000","last_unstale":"2024-09-17T05:39:09.690838+0000","last_undegraded":"2024-09-17T05:39:09.690838+0000","last_fullsized":"2024-09-17T05:39:09.690838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:44.618638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683352+0000","last_change":"2024-09-17T05:39:08.670676+0000","last_active":"2024-09-17T05:39:09.683352+0000","last_peered":"2024-09-17T05:39:09.683352+0000","last_clean":"2024-09-17T05:39:09.683352+0000","last_became_active":"2024-09-17T05:39:08.669918+0000","last_became_peered":"2024-09-17T05:39:08.669918+0000","last_unstale":"2024-09-17T05:39:09.683352+0000","last_undegraded":"2024-09-17T05:39:09.683352+0000","last_fullsized":"2024-09-17T05:39:09.683352+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:23.308245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674275+0000","last_change":"2024-09-17T05:39:08.674275+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674275+0000","last_undegraded":"2024-09-17T05:39:08.674275+0000","last_fullsized":"2024-09-17T05:39:08.674275+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.680290+0000","last_change":"2024-09-17T05:39:08.676843+0000","last_active":"2024-09-17T05:39:08.680290+0000","last_peered":"2024-09-17T05:39:08.680290+0000","last_clean":"2024-09-17T05:39:08.680290+0000","last_became_active":"2024-09-17T05:39:08.676520+0000","last_became_peered":"2024-09-17T05:39:08.676520+0000","last_unstale":"2024-09-17T05:39:08.680290+0000","last_undegraded":"2024-09-17T05:39:08.680290+0000","last_fullsized":"2024-09-17T05:39:08.680290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:09.569656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.672842+0000","last_change":"2024-09-17T05:39:08.672842+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.672842+0000","last_undegraded":"2024-09-17T05:39:08.672842+0000","last_fullsized":"2024-09-17T05:39:08.672842+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684571+0000","last_change":"2024-09-17T05:39:08.681472+0000","last_active":"2024-09-17T05:39:08.684571+0000","last_peered":"2024-09-17T05:39:08.684571+0000","last_clean":"2024-09-17T05:39:08.684571+0000","last_became_active":"2024-09-17T05:39:08.680987+0000","last_became_peered":"2024-09-17T05:39:08.680987+0000","last_unstale":"2024-09-17T05:39:08.684571+0000","last_undegraded":"2024-09-17T05:39:08.684571+0000","last_fullsized":"2024-09-17T05:39:08.684571+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:46.955054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671568+0000","last_change":"2024-09-17T05:39:08.671568+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671568+0000","last_undegraded":"2024-09-17T05:39:08.671568+0000","last_fullsized":"2024-09-17T05:39:08.671568+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.678672+0000","last_change":"2024-09-17T05:39:08.674902+0000","last_active":"2024-09-17T05:39:08.678672+0000","last_peered":"2024-09-17T05:39:08.678672+0000","last_clean":"2024-09-17T05:39:08.678672+0000","last_became_active":"2024-09-17T05:39:08.674649+0000","last_became_peered":"2024-09-17T05:39:08.674649+0000","last_unstale":"2024-09-17T05:39:08.678672+0000","last_undegraded":"2024-09-17T05:39:08.678672+0000","last_fullsized":"2024-09-17T05:39:08.678672+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:23.479650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702767+0000","last_change":"2024-09-17T05:39:09.699303+0000","last_active":"2024-09-17T05:39:09.702767+0000","last_peered":"2024-09-17T05:39:09.702767+0000","last_clean":"2024-09-17T05:39:09.702767+0000","last_became_active":"2024-09-17T05:39:09.698962+0000","last_became_peered":"2024-09-17T05:39:09.698962+0000","last_unstale":"2024-09-17T05:39:09.702767+0000","last_undegraded":"2024-09-17T05:39:09.702767+0000","last_fullsized":"2024-09-17T05:39:09.702767+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:00.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685349+0000","last_change":"2024-09-17T05:39:08.682089+0000","last_active":"2024-09-17T05:39:08.685349+0000","last_peered":"2024-09-17T05:39:08.685349+0000","last_clean":"2024-09-17T05:39:08.685349+0000","last_became_active":"2024-09-17T05:39:08.680263+0000","last_became_peered":"2024-09-17T05:39:08.680263+0000","last_unstale":"2024-09-17T05:39:08.685349+0000","last_undegraded":"2024-09-17T05:39:08.685349+0000","last_fullsized":"2024-09-17T05:39:08.685349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:47.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668496+0000","last_change":"2024-09-17T05:39:08.668496+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668496+0000","last_undegraded":"2024-09-17T05:39:08.668496+0000","last_fullsized":"2024-09-17T05:39:08.668496+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.684419+0000","last_change":"2024-09-17T05:39:08.673802+0000","last_active":"2024-09-17T05:39:09.684419+0000","last_peered":"2024-09-17T05:39:09.684419+0000","last_clean":"2024-09-17T05:39:09.684419+0000","last_became_active":"2024-09-17T05:39:08.673520+0000","last_became_peered":"2024-09-17T05:39:08.673520+0000","last_unstale":"2024-09-17T05:39:09.684419+0000","last_undegraded":"2024-09-17T05:39:09.684419+0000","last_fullsized":"2024-09-17T05:39:09.684419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:57.829343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673821+0000","last_change":"2024-09-17T05:39:08.673821+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673821+0000","last_undegraded":"2024-09-17T05:39:08.673821+0000","last_fullsized":"2024-09-17T05:39:08.673821+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684939+0000","last_change":"2024-09-17T05:39:08.683259+0000","last_active":"2024-09-17T05:39:08.684939+0000","last_peered":"2024-09-17T05:39:08.684939+0000","last_clean":"2024-09-17T05:39:08.684939+0000","last_became_active":"2024-09-17T05:39:08.682951+0000","last_became_peered":"2024-09-17T05:39:08.682951+0000","last_unstale":"2024-09-17T05:39:08.684939+0000","last_undegraded":"2024-09-17T05:39:08.684939+0000","last_fullsized":"2024-09-17T05:39:08.684939+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:33.782284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676324+0000","last_change":"2024-09-17T05:39:08.676324+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676324+0000","last_undegraded":"2024-09-17T05:39:08.676324+0000","last_fullsized":"2024-09-17T05:39:08.676324+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687860+0000","last_change":"2024-09-17T05:39:08.683416+0000","last_active":"2024-09-17T05:39:08.687860+0000","last_peered":"2024-09-17T05:39:08.687860+0000","last_clean":"2024-09-17T05:39:08.687860+0000","last_became_active":"2024-09-17T05:39:08.682597+0000","last_became_peered":"2024-09-17T05:39:08.682597+0000","last_unstale":"2024-09-17T05:39:08.687860+0000","last_undegraded":"2024-09-17T05:39:08.687860+0000","last_fullsized":"2024-09-17T05:39:08.687860+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:59.049915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662397+0000","last_change":"2024-09-17T05:39:08.662397+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662397+0000","last_undegraded":"2024-09-17T05:39:08.662397+0000","last_fullsized":"2024-09-17T05:39:08.662397+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685390+0000","last_change":"2024-09-17T05:39:08.683995+0000","last_active":"2024-09-17T05:39:08.685390+0000","last_peered":"2024-09-17T05:39:08.685390+0000","last_clean":"2024-09-17T05:39:08.685390+0000","last_became_active":"2024-09-17T05:39:08.683726+0000","last_became_peered":"2024-09-17T05:39:08.683726+0000","last_unstale":"2024-09-17T05:39:08.685390+0000","last_undegraded":"2024-09-17T05:39:08.685390+0000","last_fullsized":"2024-09-17T05:39:08.685390+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:12.468607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.693873+0000","last_change":"2024-09-17T05:39:09.690892+0000","last_active":"2024-09-17T05:39:09.693873+0000","last_peered":"2024-09-17T05:39:09.693873+0000","last_clean":"2024-09-17T05:39:09.693873+0000","last_became_active":"2024-09-17T05:39:09.690610+0000","last_became_peered":"2024-09-17T05:39:09.690610+0000","last_unstale":"2024-09-17T05:39:09.693873+0000","last_undegraded":"2024-09-17T05:39:09.693873+0000","last_fullsized":"2024-09-17T05:39:09.693873+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:16.440719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683840+0000","last_change":"2024-09-17T05:39:08.673866+0000","last_active":"2024-09-17T05:39:09.683840+0000","last_peered":"2024-09-17T05:39:09.683840+0000","last_clean":"2024-09-17T05:39:09.683840+0000","last_became_active":"2024-09-17T05:39:08.673542+0000","last_became_peered":"2024-09-17T05:39:08.673542+0000","last_unstale":"2024-09-17T05:39:09.683840+0000","last_undegraded":"2024-09-17T05:39:09.683840+0000","last_fullsized":"2024-09-17T05:39:09.683840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:11.138555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666672+0000","last_change":"2024-09-17T05:39:08.666672+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666672+0000","last_undegraded":"2024-09-17T05:39:08.666672+0000","last_fullsized":"2024-09-17T05:39:08.666672+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682043+0000","last_change":"2024-09-17T05:39:08.680453+0000","last_active":"2024-09-17T05:39:08.682043+0000","last_peered":"2024-09-17T05:39:08.682043+0000","last_clean":"2024-09-17T05:39:08.682043+0000","last_became_active":"2024-09-17T05:39:08.680180+0000","last_became_peered":"2024-09-17T05:39:08.680180+0000","last_unstale":"2024-09-17T05:39:08.682043+0000","last_undegraded":"2024-09-17T05:39:08.682043+0000","last_fullsized":"2024-09-17T05:39:08.682043+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:23.479650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":317,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.666710+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:39:08.666710+0000","last_peered":"2024-09-17T05:39:08.666710+0000","last_clean":"2024-09-17T05:39:08.666710+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:39:08.666710+0000","last_undegraded":"2024-09-17T05:39:08.666710+0000","last_fullsized":"2024-09-17T05:39:08.666710+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509614,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127104,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244736,"statfs":{"total":96636764160,"available":96506609664,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136832,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235008,"statfs":{"total":96636764160,"available":96496648192,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131264,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240576,"statfs":{"total":96636764160,"available":96502349824,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509613,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131268,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240572,"statfs":{"total":96636764160,"available":96502345728,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127092,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244748,"statfs":{"total":96636764160,"available":96506621952,"internally_reserved":0,"allocated":1601536,"data_stored":703037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509615,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145716,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226124,"statfs":{"total":96636764160,"available":96487550976,"internally_reserved":0,"allocated":2195456,"data_stored":1293405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509613,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145148,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226692,"statfs":{"total":96636764160,"available":96488132608,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":146995659},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542319,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131836,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240004,"statfs":{"total":96636764160,"available":96501764096,"internally_reserved":0,"allocated":2195456,"data_stored":1293405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:39:14.197 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:39:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- 172.21.15.122:0/485733635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f6806e810 msgr2=0x7f1f680cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 --2- 172.21.15.122:0/485733635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f6806e810 0x7f1f680cf130 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1f6806aab0 tx=0x7f1f5401c5d0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- 172.21.15.122:0/485733635 shutdown_connections 2024-09-17T05:39:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 --2- 172.21.15.122:0/485733635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f680b4bd0 0x7f1f6815e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 --2- 172.21.15.122:0/485733635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f6806e810 0x7f1f680cf130 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 --2- 172.21.15.122:0/485733635 >> v2:172.21.15.173:3300/0 conn(0x7f1f6815ade0 0x7f1f680cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- 172.21.15.122:0/485733635 >> 172.21.15.122:0/485733635 conn(0x7f1f680b18f0 msgr2=0x7f1f680b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- 172.21.15.122:0/485733635 shutdown_connections 2024-09-17T05:39:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- 172.21.15.122:0/485733635 wait complete. 2024-09-17T05:39:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 Processor -- start 2024-09-17T05:39:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- start start 2024-09-17T05:39:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f6806e810 0x7f1f68147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f680b4bd0 0x7f1f681449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f6815ade0 0x7f1f680739f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6806f100 con 0x7f1f680b4bd0 2024-09-17T05:39:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6806ef80 con 0x7f1f6806e810 2024-09-17T05:39:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f6e000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6806f280 con 0x7f1f6815ade0 2024-09-17T05:39:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f66a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f680b4bd0 0x7f1f681449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f66a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f680b4bd0 0x7f1f681449e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38842/0 (socket says 172.21.15.122:38842) 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f66a00640 1 -- 172.21.15.122:0/2569818867 learned_addr learned my addr 172.21.15.122:0/2569818867 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f67400640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f6806e810 0x7f1f68147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f67e00640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f6815ade0 0x7f1f680739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f66a00640 1 -- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f6815ade0 msgr2=0x7f1f680739f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f66a00640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f6815ade0 0x7f1f680739f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f66a00640 1 -- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f6806e810 msgr2=0x7f1f68147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f66a00640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f6806e810 0x7f1f68147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f66a00640 1 -- 172.21.15.122:0/2569818867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681482b0 con 0x7f1f680b4bd0 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.382+0000 7f1f67e00640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f6815ade0 0x7f1f680739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f66a00640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f680b4bd0 0x7f1f681449e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1f5401ac80 tx=0x7f1f540a94f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f5f400640 1 -- 172.21.15.122:0/2569818867 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f540ad040 con 0x7f1f680b4bd0 2024-09-17T05:39:14.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f5f400640 1 -- 172.21.15.122:0/2569818867 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f540aad50 con 0x7f1f680b4bd0 2024-09-17T05:39:14.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6809d870 con 0x7f1f680b4bd0 2024-09-17T05:39:14.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f5f400640 1 -- 172.21.15.122:0/2569818867 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54005290 con 0x7f1f680b4bd0 2024-09-17T05:39:14.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6815cdd0 con 0x7f1f680b4bd0 2024-09-17T05:39:14.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f68069410 con 0x7f1f680b4bd0 2024-09-17T05:39:14.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f5f400640 1 -- 172.21.15.122:0/2569818867 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f1f540b0070 con 0x7f1f680b4bd0 2024-09-17T05:39:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f5f400640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f48075180 0x7f1f48077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f5f400640 1 -- 172.21.15.122:0/2569818867 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f1f5412dba0 con 0x7f1f680b4bd0 2024-09-17T05:39:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.386+0000 7f1f67400640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f48075180 0x7f1f48077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:14.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.390+0000 7f1f67400640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f48075180 0x7f1f48077640 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009e50 tx=0x7f1f580095e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:14.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.390+0000 7f1f5f400640 1 -- 172.21.15.122:0/2569818867 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1f681482b0 con 0x7f1f680b4bd0 2024-09-17T05:39:14.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.558+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f680cf720 con 0x7f1f48075180 2024-09-17T05:39:14.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.562+0000 7f1f5f400640 1 -- 172.21.15.122:0/2569818867 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247873 (secure 0 0 0) 0x7f1f680cf720 con 0x7f1f48075180 2024-09-17T05:39:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:14.562 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:39:14.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f48075180 msgr2=0x7f1f48077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f48075180 0x7f1f48077640 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009e50 tx=0x7f1f580095e0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f680b4bd0 msgr2=0x7f1f681449e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f680b4bd0 0x7f1f681449e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1f5401ac80 tx=0x7f1f540a94f0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 shutdown_connections 2024-09-17T05:39:14.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1f48075180 0x7f1f48077640 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f6815ade0 0x7f1f680739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f680b4bd0 0x7f1f681449e0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 --2- 172.21.15.122:0/2569818867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1f6806e810 0x7f1f68147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 >> 172.21.15.122:0/2569818867 conn(0x7f1f680b18f0 msgr2=0x7f1f680b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:14.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 shutdown_connections 2024-09-17T05:39:14.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.566+0000 7f1f6e000640 1 -- 172.21.15.122:0/2569818867 wait complete. 2024-09-17T05:39:14.581 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":555,"stamp":"2024-09-17T05:39:12.736127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076260,"kb_used_data":13732,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898460,"statfs":{"total":773094113280,"available":771992023040,"internally_reserved":0,"allocated":14061568,"data_stored":6836827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1087819318},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.080111"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675684+0000","last_change":"2024-09-17T05:39:08.675684+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675684+0000","last_undegraded":"2024-09-17T05:39:08.675684+0000","last_fullsized":"2024-09-17T05:39:08.675684+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.681063+0000","last_change":"2024-09-17T05:39:08.681063+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.681063+0000","last_undegraded":"2024-09-17T05:39:08.681063+0000","last_fullsized":"2024-09-17T05:39:08.681063+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668951+0000","last_change":"2024-09-17T05:39:08.668951+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668951+0000","last_undegraded":"2024-09-17T05:39:08.668951+0000","last_fullsized":"2024-09-17T05:39:08.668951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666359+0000","last_change":"2024-09-17T05:39:08.666359+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666359+0000","last_undegraded":"2024-09-17T05:39:08.666359+0000","last_fullsized":"2024-09-17T05:39:08.666359+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667851+0000","last_change":"2024-09-17T05:39:08.667851+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667851+0000","last_undegraded":"2024-09-17T05:39:08.667851+0000","last_fullsized":"2024-09-17T05:39:08.667851+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674261+0000","last_change":"2024-09-17T05:39:08.674261+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674261+0000","last_undegraded":"2024-09-17T05:39:08.674261+0000","last_fullsized":"2024-09-17T05:39:08.674261+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667467+0000","last_change":"2024-09-17T05:39:08.667467+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667467+0000","last_undegraded":"2024-09-17T05:39:08.667467+0000","last_fullsized":"2024-09-17T05:39:08.667467+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668743+0000","last_change":"2024-09-17T05:39:08.668743+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668743+0000","last_undegraded":"2024-09-17T05:39:08.668743+0000","last_fullsized":"2024-09-17T05:39:08.668743+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667026+0000","last_change":"2024-09-17T05:39:08.667026+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667026+0000","last_undegraded":"2024-09-17T05:39:08.667026+0000","last_fullsized":"2024-09-17T05:39:08.667026+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678847+0000","last_change":"2024-09-17T05:39:08.678847+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678847+0000","last_undegraded":"2024-09-17T05:39:08.678847+0000","last_fullsized":"2024-09-17T05:39:08.678847+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666857+0000","last_change":"2024-09-17T05:39:08.666857+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666857+0000","last_undegraded":"2024-09-17T05:39:08.666857+0000","last_fullsized":"2024-09-17T05:39:08.666857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678015+0000","last_change":"2024-09-17T05:39:08.678015+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678015+0000","last_undegraded":"2024-09-17T05:39:08.678015+0000","last_fullsized":"2024-09-17T05:39:08.678015+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668363+0000","last_change":"2024-09-17T05:39:08.668363+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668363+0000","last_undegraded":"2024-09-17T05:39:08.668363+0000","last_fullsized":"2024-09-17T05:39:08.668363+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.694612+0000","last_change":"2024-09-17T05:39:09.692704+0000","last_active":"2024-09-17T05:39:09.694612+0000","last_peered":"2024-09-17T05:39:09.694612+0000","last_clean":"2024-09-17T05:39:09.694612+0000","last_became_active":"2024-09-17T05:39:09.690032+0000","last_became_peered":"2024-09-17T05:39:09.690032+0000","last_unstale":"2024-09-17T05:39:09.694612+0000","last_undegraded":"2024-09-17T05:39:09.694612+0000","last_fullsized":"2024-09-17T05:39:09.694612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:10.630444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662857+0000","last_change":"2024-09-17T05:39:08.662857+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662857+0000","last_undegraded":"2024-09-17T05:39:08.662857+0000","last_fullsized":"2024-09-17T05:39:08.662857+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702572+0000","last_change":"2024-09-17T05:39:09.698973+0000","last_active":"2024-09-17T05:39:09.702572+0000","last_peered":"2024-09-17T05:39:09.702572+0000","last_clean":"2024-09-17T05:39:09.702572+0000","last_became_active":"2024-09-17T05:39:09.698375+0000","last_became_peered":"2024-09-17T05:39:09.698375+0000","last_unstale":"2024-09-17T05:39:09.702572+0000","last_undegraded":"2024-09-17T05:39:09.702572+0000","last_fullsized":"2024-09-17T05:39:09.702572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:28.895178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666042+0000","last_change":"2024-09-17T05:39:08.666042+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666042+0000","last_undegraded":"2024-09-17T05:39:08.666042+0000","last_fullsized":"2024-09-17T05:39:08.666042+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662651+0000","last_change":"2024-09-17T05:39:08.662651+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662651+0000","last_undegraded":"2024-09-17T05:39:08.662651+0000","last_fullsized":"2024-09-17T05:39:08.662651+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675522+0000","last_change":"2024-09-17T05:39:08.675522+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675522+0000","last_undegraded":"2024-09-17T05:39:08.675522+0000","last_fullsized":"2024-09-17T05:39:08.675522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662332+0000","last_change":"2024-09-17T05:39:08.662332+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662332+0000","last_undegraded":"2024-09-17T05:39:08.662332+0000","last_fullsized":"2024-09-17T05:39:08.662332+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.677152+0000","last_change":"2024-09-17T05:39:08.677152+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.677152+0000","last_undegraded":"2024-09-17T05:39:08.677152+0000","last_fullsized":"2024-09-17T05:39:08.677152+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.677957+0000","last_change":"2024-09-17T05:39:08.677957+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.677957+0000","last_undegraded":"2024-09-17T05:39:08.677957+0000","last_fullsized":"2024-09-17T05:39:08.677957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676749+0000","last_change":"2024-09-17T05:39:08.676749+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676749+0000","last_undegraded":"2024-09-17T05:39:08.676749+0000","last_fullsized":"2024-09-17T05:39:08.676749+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.663391+0000","last_change":"2024-09-17T05:39:08.663391+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.663391+0000","last_undegraded":"2024-09-17T05:39:08.663391+0000","last_fullsized":"2024-09-17T05:39:08.663391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668585+0000","last_change":"2024-09-17T05:39:08.668585+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668585+0000","last_undegraded":"2024-09-17T05:39:08.668585+0000","last_fullsized":"2024-09-17T05:39:08.668585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.678343+0000","last_change":"2024-09-17T05:39:08.678343+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.678343+0000","last_undegraded":"2024-09-17T05:39:08.678343+0000","last_fullsized":"2024-09-17T05:39:08.678343+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.699887+0000","last_change":"2024-09-17T05:39:09.693510+0000","last_active":"2024-09-17T05:39:09.699887+0000","last_peered":"2024-09-17T05:39:09.699887+0000","last_clean":"2024-09-17T05:39:09.699887+0000","last_became_active":"2024-09-17T05:39:09.693223+0000","last_became_peered":"2024-09-17T05:39:09.693223+0000","last_unstale":"2024-09-17T05:39:09.699887+0000","last_undegraded":"2024-09-17T05:39:09.699887+0000","last_fullsized":"2024-09-17T05:39:09.699887+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:54.958914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674848+0000","last_change":"2024-09-17T05:39:08.674848+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674848+0000","last_undegraded":"2024-09-17T05:39:08.674848+0000","last_fullsized":"2024-09-17T05:39:08.674848+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.670468+0000","last_change":"2024-09-17T05:39:08.670468+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.670468+0000","last_undegraded":"2024-09-17T05:39:08.670468+0000","last_fullsized":"2024-09-17T05:39:08.670468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671200+0000","last_change":"2024-09-17T05:39:08.671200+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671200+0000","last_undegraded":"2024-09-17T05:39:08.671200+0000","last_fullsized":"2024-09-17T05:39:08.671200+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675452+0000","last_change":"2024-09-17T05:39:08.675452+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675452+0000","last_undegraded":"2024-09-17T05:39:08.675452+0000","last_fullsized":"2024-09-17T05:39:08.675452+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669290+0000","last_change":"2024-09-17T05:39:08.669290+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669290+0000","last_undegraded":"2024-09-17T05:39:08.669290+0000","last_fullsized":"2024-09-17T05:39:08.669290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.703188+0000","last_change":"2024-09-17T05:39:09.698329+0000","last_active":"2024-09-17T05:39:09.703188+0000","last_peered":"2024-09-17T05:39:09.703188+0000","last_clean":"2024-09-17T05:39:09.703188+0000","last_became_active":"2024-09-17T05:39:09.698084+0000","last_became_peered":"2024-09-17T05:39:09.698084+0000","last_unstale":"2024-09-17T05:39:09.703188+0000","last_undegraded":"2024-09-17T05:39:09.703188+0000","last_fullsized":"2024-09-17T05:39:09.703188+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:47.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.678152+0000","last_change":"2024-09-17T05:39:08.676570+0000","last_active":"2024-09-17T05:39:08.678152+0000","last_peered":"2024-09-17T05:39:08.678152+0000","last_clean":"2024-09-17T05:39:08.678152+0000","last_became_active":"2024-09-17T05:39:08.676332+0000","last_became_peered":"2024-09-17T05:39:08.676332+0000","last_unstale":"2024-09-17T05:39:08.678152+0000","last_undegraded":"2024-09-17T05:39:08.678152+0000","last_fullsized":"2024-09-17T05:39:08.678152+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:03.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666556+0000","last_change":"2024-09-17T05:39:08.666556+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666556+0000","last_undegraded":"2024-09-17T05:39:08.666556+0000","last_fullsized":"2024-09-17T05:39:08.666556+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684311+0000","last_change":"2024-09-17T05:39:08.681409+0000","last_active":"2024-09-17T05:39:08.684311+0000","last_peered":"2024-09-17T05:39:08.684311+0000","last_clean":"2024-09-17T05:39:08.684311+0000","last_became_active":"2024-09-17T05:39:08.681149+0000","last_became_peered":"2024-09-17T05:39:08.681149+0000","last_unstale":"2024-09-17T05:39:08.684311+0000","last_undegraded":"2024-09-17T05:39:08.684311+0000","last_fullsized":"2024-09-17T05:39:08.684311+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:57.845999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667745+0000","last_change":"2024-09-17T05:39:08.667745+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667745+0000","last_undegraded":"2024-09-17T05:39:08.667745+0000","last_fullsized":"2024-09-17T05:39:08.667745+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.674756+0000","last_change":"2024-09-17T05:39:08.673698+0000","last_active":"2024-09-17T05:39:08.674756+0000","last_peered":"2024-09-17T05:39:08.674756+0000","last_clean":"2024-09-17T05:39:08.674756+0000","last_became_active":"2024-09-17T05:39:08.673385+0000","last_became_peered":"2024-09-17T05:39:08.673385+0000","last_unstale":"2024-09-17T05:39:08.674756+0000","last_undegraded":"2024-09-17T05:39:08.674756+0000","last_fullsized":"2024-09-17T05:39:08.674756+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:08.684706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674269+0000","last_change":"2024-09-17T05:39:08.674269+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674269+0000","last_undegraded":"2024-09-17T05:39:08.674269+0000","last_fullsized":"2024-09-17T05:39:08.674269+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.683316+0000","last_change":"2024-09-17T05:39:08.680638+0000","last_active":"2024-09-17T05:39:08.683316+0000","last_peered":"2024-09-17T05:39:08.683316+0000","last_clean":"2024-09-17T05:39:08.683316+0000","last_became_active":"2024-09-17T05:39:08.680162+0000","last_became_peered":"2024-09-17T05:39:08.680162+0000","last_unstale":"2024-09-17T05:39:08.683316+0000","last_undegraded":"2024-09-17T05:39:08.683316+0000","last_fullsized":"2024-09-17T05:39:08.683316+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:50.183571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662482+0000","last_change":"2024-09-17T05:39:08.662482+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662482+0000","last_undegraded":"2024-09-17T05:39:08.662482+0000","last_fullsized":"2024-09-17T05:39:08.662482+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.680929+0000","last_change":"2024-09-17T05:39:08.679285+0000","last_active":"2024-09-17T05:39:08.680929+0000","last_peered":"2024-09-17T05:39:08.680929+0000","last_clean":"2024-09-17T05:39:08.680929+0000","last_became_active":"2024-09-17T05:39:08.679008+0000","last_became_peered":"2024-09-17T05:39:08.679008+0000","last_unstale":"2024-09-17T05:39:08.680929+0000","last_undegraded":"2024-09-17T05:39:08.680929+0000","last_fullsized":"2024-09-17T05:39:08.680929+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:43.119124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676787+0000","last_change":"2024-09-17T05:39:08.676787+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676787+0000","last_undegraded":"2024-09-17T05:39:08.676787+0000","last_fullsized":"2024-09-17T05:39:08.676787+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685185+0000","last_change":"2024-09-17T05:39:08.682028+0000","last_active":"2024-09-17T05:39:08.685185+0000","last_peered":"2024-09-17T05:39:08.685185+0000","last_clean":"2024-09-17T05:39:08.685185+0000","last_became_active":"2024-09-17T05:39:08.680621+0000","last_became_peered":"2024-09-17T05:39:08.680621+0000","last_unstale":"2024-09-17T05:39:08.685185+0000","last_undegraded":"2024-09-17T05:39:08.685185+0000","last_fullsized":"2024-09-17T05:39:08.685185+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:58.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673730+0000","last_change":"2024-09-17T05:39:08.673730+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673730+0000","last_undegraded":"2024-09-17T05:39:08.673730+0000","last_fullsized":"2024-09-17T05:39:08.673730+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687379+0000","last_change":"2024-09-17T05:39:08.683990+0000","last_active":"2024-09-17T05:39:08.687379+0000","last_peered":"2024-09-17T05:39:08.687379+0000","last_clean":"2024-09-17T05:39:08.687379+0000","last_became_active":"2024-09-17T05:39:08.683688+0000","last_became_peered":"2024-09-17T05:39:08.683688+0000","last_unstale":"2024-09-17T05:39:08.687379+0000","last_undegraded":"2024-09-17T05:39:08.687379+0000","last_fullsized":"2024-09-17T05:39:08.687379+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:42.151787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669585+0000","last_change":"2024-09-17T05:39:08.669585+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669585+0000","last_undegraded":"2024-09-17T05:39:08.669585+0000","last_fullsized":"2024-09-17T05:39:08.669585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.686789+0000","last_change":"2024-09-17T05:39:08.686070+0000","last_active":"2024-09-17T05:39:08.686789+0000","last_peered":"2024-09-17T05:39:08.686789+0000","last_clean":"2024-09-17T05:39:08.686789+0000","last_became_active":"2024-09-17T05:39:08.685786+0000","last_became_peered":"2024-09-17T05:39:08.685786+0000","last_unstale":"2024-09-17T05:39:08.686789+0000","last_undegraded":"2024-09-17T05:39:08.686789+0000","last_fullsized":"2024-09-17T05:39:08.686789+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:13.271583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671412+0000","last_change":"2024-09-17T05:39:08.671412+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671412+0000","last_undegraded":"2024-09-17T05:39:08.671412+0000","last_fullsized":"2024-09-17T05:39:08.671412+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685545+0000","last_change":"2024-09-17T05:39:08.680218+0000","last_active":"2024-09-17T05:39:08.685545+0000","last_peered":"2024-09-17T05:39:08.685545+0000","last_clean":"2024-09-17T05:39:08.685545+0000","last_became_active":"2024-09-17T05:39:08.678856+0000","last_became_peered":"2024-09-17T05:39:08.678856+0000","last_unstale":"2024-09-17T05:39:08.685545+0000","last_undegraded":"2024-09-17T05:39:08.685545+0000","last_fullsized":"2024-09-17T05:39:08.685545+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:25.993270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667957+0000","last_change":"2024-09-17T05:39:08.667957+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667957+0000","last_undegraded":"2024-09-17T05:39:08.667957+0000","last_fullsized":"2024-09-17T05:39:08.667957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.681739+0000","last_change":"2024-09-17T05:39:08.677955+0000","last_active":"2024-09-17T05:39:08.681739+0000","last_peered":"2024-09-17T05:39:08.681739+0000","last_clean":"2024-09-17T05:39:08.681739+0000","last_became_active":"2024-09-17T05:39:08.677694+0000","last_became_peered":"2024-09-17T05:39:08.677694+0000","last_unstale":"2024-09-17T05:39:08.681739+0000","last_undegraded":"2024-09-17T05:39:08.681739+0000","last_fullsized":"2024-09-17T05:39:08.681739+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:41.299619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674657+0000","last_change":"2024-09-17T05:39:08.674657+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674657+0000","last_undegraded":"2024-09-17T05:39:08.674657+0000","last_fullsized":"2024-09-17T05:39:08.674657+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.677043+0000","last_change":"2024-09-17T05:39:08.672074+0000","last_active":"2024-09-17T05:39:08.677043+0000","last_peered":"2024-09-17T05:39:08.677043+0000","last_clean":"2024-09-17T05:39:08.677043+0000","last_became_active":"2024-09-17T05:39:08.671800+0000","last_became_peered":"2024-09-17T05:39:08.671800+0000","last_unstale":"2024-09-17T05:39:08.677043+0000","last_undegraded":"2024-09-17T05:39:08.677043+0000","last_fullsized":"2024-09-17T05:39:08.677043+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:25.993270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673883+0000","last_change":"2024-09-17T05:39:08.673883+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673883+0000","last_undegraded":"2024-09-17T05:39:08.673883+0000","last_fullsized":"2024-09-17T05:39:08.673883+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.689377+0000","last_change":"2024-09-17T05:39:08.686779+0000","last_active":"2024-09-17T05:39:08.689377+0000","last_peered":"2024-09-17T05:39:08.689377+0000","last_clean":"2024-09-17T05:39:08.689377+0000","last_became_active":"2024-09-17T05:39:08.686086+0000","last_became_peered":"2024-09-17T05:39:08.686086+0000","last_unstale":"2024-09-17T05:39:08.689377+0000","last_undegraded":"2024-09-17T05:39:08.689377+0000","last_fullsized":"2024-09-17T05:39:08.689377+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:00.213923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.707208+0000","last_change":"2024-09-17T05:39:09.702938+0000","last_active":"2024-09-17T05:39:09.707208+0000","last_peered":"2024-09-17T05:39:09.707208+0000","last_clean":"2024-09-17T05:39:09.707208+0000","last_became_active":"2024-09-17T05:39:09.702616+0000","last_became_peered":"2024-09-17T05:39:09.702616+0000","last_unstale":"2024-09-17T05:39:09.707208+0000","last_undegraded":"2024-09-17T05:39:09.707208+0000","last_fullsized":"2024-09-17T05:39:09.707208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:30.455771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683424+0000","last_change":"2024-09-17T05:39:08.674569+0000","last_active":"2024-09-17T05:39:09.683424+0000","last_peered":"2024-09-17T05:39:09.683424+0000","last_clean":"2024-09-17T05:39:09.683424+0000","last_became_active":"2024-09-17T05:39:08.674285+0000","last_became_peered":"2024-09-17T05:39:08.674285+0000","last_unstale":"2024-09-17T05:39:09.683424+0000","last_undegraded":"2024-09-17T05:39:09.683424+0000","last_fullsized":"2024-09-17T05:39:09.683424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:13.783924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702018+0000","last_change":"2024-09-17T05:39:09.698643+0000","last_active":"2024-09-17T05:39:09.702018+0000","last_peered":"2024-09-17T05:39:09.702018+0000","last_clean":"2024-09-17T05:39:09.702018+0000","last_became_active":"2024-09-17T05:39:09.698013+0000","last_became_peered":"2024-09-17T05:39:09.698013+0000","last_unstale":"2024-09-17T05:39:09.702018+0000","last_undegraded":"2024-09-17T05:39:09.702018+0000","last_fullsized":"2024-09-17T05:39:09.702018+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:42.265017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685479+0000","last_change":"2024-09-17T05:39:08.684608+0000","last_active":"2024-09-17T05:39:08.685479+0000","last_peered":"2024-09-17T05:39:08.685479+0000","last_clean":"2024-09-17T05:39:08.685479+0000","last_became_active":"2024-09-17T05:39:08.684343+0000","last_became_peered":"2024-09-17T05:39:08.684343+0000","last_unstale":"2024-09-17T05:39:08.685479+0000","last_undegraded":"2024-09-17T05:39:08.685479+0000","last_fullsized":"2024-09-17T05:39:08.685479+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:17.208839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.675898+0000","last_change":"2024-09-17T05:39:08.675898+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.675898+0000","last_undegraded":"2024-09-17T05:39:08.675898+0000","last_fullsized":"2024-09-17T05:39:08.675898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687856+0000","last_change":"2024-09-17T05:39:08.682503+0000","last_active":"2024-09-17T05:39:08.687856+0000","last_peered":"2024-09-17T05:39:08.687856+0000","last_clean":"2024-09-17T05:39:08.687856+0000","last_became_active":"2024-09-17T05:39:08.682225+0000","last_became_peered":"2024-09-17T05:39:08.682225+0000","last_unstale":"2024-09-17T05:39:08.687856+0000","last_undegraded":"2024-09-17T05:39:08.687856+0000","last_fullsized":"2024-09-17T05:39:08.687856+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:03.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.669746+0000","last_change":"2024-09-17T05:39:08.669746+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.669746+0000","last_undegraded":"2024-09-17T05:39:08.669746+0000","last_fullsized":"2024-09-17T05:39:08.669746+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.689512+0000","last_change":"2024-09-17T05:39:08.685708+0000","last_active":"2024-09-17T05:39:08.689512+0000","last_peered":"2024-09-17T05:39:08.689512+0000","last_clean":"2024-09-17T05:39:08.689512+0000","last_became_active":"2024-09-17T05:39:08.685380+0000","last_became_peered":"2024-09-17T05:39:08.685380+0000","last_unstale":"2024-09-17T05:39:08.689512+0000","last_undegraded":"2024-09-17T05:39:08.689512+0000","last_fullsized":"2024-09-17T05:39:08.689512+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:54.394077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.667465+0000","last_change":"2024-09-17T05:39:08.667465+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.667465+0000","last_undegraded":"2024-09-17T05:39:08.667465+0000","last_fullsized":"2024-09-17T05:39:08.667465+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682509+0000","last_change":"2024-09-17T05:39:08.679962+0000","last_active":"2024-09-17T05:39:08.682509+0000","last_peered":"2024-09-17T05:39:08.682509+0000","last_clean":"2024-09-17T05:39:08.682509+0000","last_became_active":"2024-09-17T05:39:08.679323+0000","last_became_peered":"2024-09-17T05:39:08.679323+0000","last_unstale":"2024-09-17T05:39:08.682509+0000","last_undegraded":"2024-09-17T05:39:08.682509+0000","last_fullsized":"2024-09-17T05:39:08.682509+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:22:59.521222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676378+0000","last_change":"2024-09-17T05:39:08.676378+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676378+0000","last_undegraded":"2024-09-17T05:39:08.676378+0000","last_fullsized":"2024-09-17T05:39:08.676378+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.683040+0000","last_change":"2024-09-17T05:39:08.679584+0000","last_active":"2024-09-17T05:39:08.683040+0000","last_peered":"2024-09-17T05:39:08.683040+0000","last_clean":"2024-09-17T05:39:08.683040+0000","last_became_active":"2024-09-17T05:39:08.679238+0000","last_became_peered":"2024-09-17T05:39:08.679238+0000","last_unstale":"2024-09-17T05:39:08.683040+0000","last_undegraded":"2024-09-17T05:39:08.683040+0000","last_fullsized":"2024-09-17T05:39:08.683040+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:09.538974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671695+0000","last_change":"2024-09-17T05:39:08.671695+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671695+0000","last_undegraded":"2024-09-17T05:39:08.671695+0000","last_fullsized":"2024-09-17T05:39:08.671695+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.686594+0000","last_change":"2024-09-17T05:39:08.683294+0000","last_active":"2024-09-17T05:39:08.686594+0000","last_peered":"2024-09-17T05:39:08.686594+0000","last_clean":"2024-09-17T05:39:08.686594+0000","last_became_active":"2024-09-17T05:39:08.683029+0000","last_became_peered":"2024-09-17T05:39:08.683029+0000","last_unstale":"2024-09-17T05:39:08.686594+0000","last_undegraded":"2024-09-17T05:39:08.686594+0000","last_fullsized":"2024-09-17T05:39:08.686594+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:59.049915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.670707+0000","last_change":"2024-09-17T05:39:08.670707+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.670707+0000","last_undegraded":"2024-09-17T05:39:08.670707+0000","last_fullsized":"2024-09-17T05:39:08.670707+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682915+0000","last_change":"2024-09-17T05:39:08.677212+0000","last_active":"2024-09-17T05:39:08.682915+0000","last_peered":"2024-09-17T05:39:08.682915+0000","last_clean":"2024-09-17T05:39:08.682915+0000","last_became_active":"2024-09-17T05:39:08.676885+0000","last_became_peered":"2024-09-17T05:39:08.676885+0000","last_unstale":"2024-09-17T05:39:08.682915+0000","last_undegraded":"2024-09-17T05:39:08.682915+0000","last_fullsized":"2024-09-17T05:39:08.682915+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:39.434467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673322+0000","last_change":"2024-09-17T05:39:08.673322+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673322+0000","last_undegraded":"2024-09-17T05:39:08.673322+0000","last_fullsized":"2024-09-17T05:39:08.673322+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685210+0000","last_change":"2024-09-17T05:39:08.681401+0000","last_active":"2024-09-17T05:39:08.685210+0000","last_peered":"2024-09-17T05:39:08.685210+0000","last_clean":"2024-09-17T05:39:08.685210+0000","last_became_active":"2024-09-17T05:39:08.681071+0000","last_became_peered":"2024-09-17T05:39:08.681071+0000","last_unstale":"2024-09-17T05:39:08.685210+0000","last_undegraded":"2024-09-17T05:39:08.685210+0000","last_fullsized":"2024-09-17T05:39:08.685210+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:07.351912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.690838+0000","last_change":"2024-09-17T05:39:09.688991+0000","last_active":"2024-09-17T05:39:09.690838+0000","last_peered":"2024-09-17T05:39:09.690838+0000","last_clean":"2024-09-17T05:39:09.690838+0000","last_became_active":"2024-09-17T05:39:09.688714+0000","last_became_peered":"2024-09-17T05:39:09.688714+0000","last_unstale":"2024-09-17T05:39:09.690838+0000","last_undegraded":"2024-09-17T05:39:09.690838+0000","last_fullsized":"2024-09-17T05:39:09.690838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:44.618638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683352+0000","last_change":"2024-09-17T05:39:08.670676+0000","last_active":"2024-09-17T05:39:09.683352+0000","last_peered":"2024-09-17T05:39:09.683352+0000","last_clean":"2024-09-17T05:39:09.683352+0000","last_became_active":"2024-09-17T05:39:08.669918+0000","last_became_peered":"2024-09-17T05:39:08.669918+0000","last_unstale":"2024-09-17T05:39:09.683352+0000","last_undegraded":"2024-09-17T05:39:09.683352+0000","last_fullsized":"2024-09-17T05:39:09.683352+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:23.308245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.674275+0000","last_change":"2024-09-17T05:39:08.674275+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.674275+0000","last_undegraded":"2024-09-17T05:39:08.674275+0000","last_fullsized":"2024-09-17T05:39:08.674275+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.680290+0000","last_change":"2024-09-17T05:39:08.676843+0000","last_active":"2024-09-17T05:39:08.680290+0000","last_peered":"2024-09-17T05:39:08.680290+0000","last_clean":"2024-09-17T05:39:08.680290+0000","last_became_active":"2024-09-17T05:39:08.676520+0000","last_became_peered":"2024-09-17T05:39:08.676520+0000","last_unstale":"2024-09-17T05:39:08.680290+0000","last_undegraded":"2024-09-17T05:39:08.680290+0000","last_fullsized":"2024-09-17T05:39:08.680290+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:09.569656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.672842+0000","last_change":"2024-09-17T05:39:08.672842+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.672842+0000","last_undegraded":"2024-09-17T05:39:08.672842+0000","last_fullsized":"2024-09-17T05:39:08.672842+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684571+0000","last_change":"2024-09-17T05:39:08.681472+0000","last_active":"2024-09-17T05:39:08.684571+0000","last_peered":"2024-09-17T05:39:08.684571+0000","last_clean":"2024-09-17T05:39:08.684571+0000","last_became_active":"2024-09-17T05:39:08.680987+0000","last_became_peered":"2024-09-17T05:39:08.680987+0000","last_unstale":"2024-09-17T05:39:08.684571+0000","last_undegraded":"2024-09-17T05:39:08.684571+0000","last_fullsized":"2024-09-17T05:39:08.684571+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:46.955054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.671568+0000","last_change":"2024-09-17T05:39:08.671568+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.671568+0000","last_undegraded":"2024-09-17T05:39:08.671568+0000","last_fullsized":"2024-09-17T05:39:08.671568+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.678672+0000","last_change":"2024-09-17T05:39:08.674902+0000","last_active":"2024-09-17T05:39:08.678672+0000","last_peered":"2024-09-17T05:39:08.678672+0000","last_clean":"2024-09-17T05:39:08.678672+0000","last_became_active":"2024-09-17T05:39:08.674649+0000","last_became_peered":"2024-09-17T05:39:08.674649+0000","last_unstale":"2024-09-17T05:39:08.678672+0000","last_undegraded":"2024-09-17T05:39:08.678672+0000","last_fullsized":"2024-09-17T05:39:08.678672+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:23.479650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702767+0000","last_change":"2024-09-17T05:39:09.699303+0000","last_active":"2024-09-17T05:39:09.702767+0000","last_peered":"2024-09-17T05:39:09.702767+0000","last_clean":"2024-09-17T05:39:09.702767+0000","last_became_active":"2024-09-17T05:39:09.698962+0000","last_became_peered":"2024-09-17T05:39:09.698962+0000","last_unstale":"2024-09-17T05:39:09.702767+0000","last_undegraded":"2024-09-17T05:39:09.702767+0000","last_fullsized":"2024-09-17T05:39:09.702767+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:00.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685349+0000","last_change":"2024-09-17T05:39:08.682089+0000","last_active":"2024-09-17T05:39:08.685349+0000","last_peered":"2024-09-17T05:39:08.685349+0000","last_clean":"2024-09-17T05:39:08.685349+0000","last_became_active":"2024-09-17T05:39:08.680263+0000","last_became_peered":"2024-09-17T05:39:08.680263+0000","last_unstale":"2024-09-17T05:39:08.685349+0000","last_undegraded":"2024-09-17T05:39:08.685349+0000","last_fullsized":"2024-09-17T05:39:08.685349+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:47.036807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.668496+0000","last_change":"2024-09-17T05:39:08.668496+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.668496+0000","last_undegraded":"2024-09-17T05:39:08.668496+0000","last_fullsized":"2024-09-17T05:39:08.668496+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.684419+0000","last_change":"2024-09-17T05:39:08.673802+0000","last_active":"2024-09-17T05:39:09.684419+0000","last_peered":"2024-09-17T05:39:09.684419+0000","last_clean":"2024-09-17T05:39:09.684419+0000","last_became_active":"2024-09-17T05:39:08.673520+0000","last_became_peered":"2024-09-17T05:39:08.673520+0000","last_unstale":"2024-09-17T05:39:09.684419+0000","last_undegraded":"2024-09-17T05:39:09.684419+0000","last_fullsized":"2024-09-17T05:39:09.684419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:57.829343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.673821+0000","last_change":"2024-09-17T05:39:08.673821+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.673821+0000","last_undegraded":"2024-09-17T05:39:08.673821+0000","last_fullsized":"2024-09-17T05:39:08.673821+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.684939+0000","last_change":"2024-09-17T05:39:08.683259+0000","last_active":"2024-09-17T05:39:08.684939+0000","last_peered":"2024-09-17T05:39:08.684939+0000","last_clean":"2024-09-17T05:39:08.684939+0000","last_became_active":"2024-09-17T05:39:08.682951+0000","last_became_peered":"2024-09-17T05:39:08.682951+0000","last_unstale":"2024-09-17T05:39:08.684939+0000","last_undegraded":"2024-09-17T05:39:08.684939+0000","last_fullsized":"2024-09-17T05:39:08.684939+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:33.782284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.676324+0000","last_change":"2024-09-17T05:39:08.676324+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.676324+0000","last_undegraded":"2024-09-17T05:39:08.676324+0000","last_fullsized":"2024-09-17T05:39:08.676324+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.687860+0000","last_change":"2024-09-17T05:39:08.683416+0000","last_active":"2024-09-17T05:39:08.687860+0000","last_peered":"2024-09-17T05:39:08.687860+0000","last_clean":"2024-09-17T05:39:08.687860+0000","last_became_active":"2024-09-17T05:39:08.682597+0000","last_became_peered":"2024-09-17T05:39:08.682597+0000","last_unstale":"2024-09-17T05:39:08.687860+0000","last_undegraded":"2024-09-17T05:39:08.687860+0000","last_fullsized":"2024-09-17T05:39:08.687860+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:59.049915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.662397+0000","last_change":"2024-09-17T05:39:08.662397+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.662397+0000","last_undegraded":"2024-09-17T05:39:08.662397+0000","last_fullsized":"2024-09-17T05:39:08.662397+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.685390+0000","last_change":"2024-09-17T05:39:08.683995+0000","last_active":"2024-09-17T05:39:08.685390+0000","last_peered":"2024-09-17T05:39:08.685390+0000","last_clean":"2024-09-17T05:39:08.685390+0000","last_became_active":"2024-09-17T05:39:08.683726+0000","last_became_peered":"2024-09-17T05:39:08.683726+0000","last_unstale":"2024-09-17T05:39:08.685390+0000","last_undegraded":"2024-09-17T05:39:08.685390+0000","last_fullsized":"2024-09-17T05:39:08.685390+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:12.468607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.693873+0000","last_change":"2024-09-17T05:39:09.690892+0000","last_active":"2024-09-17T05:39:09.693873+0000","last_peered":"2024-09-17T05:39:09.693873+0000","last_clean":"2024-09-17T05:39:09.693873+0000","last_became_active":"2024-09-17T05:39:09.690610+0000","last_became_peered":"2024-09-17T05:39:09.690610+0000","last_unstale":"2024-09-17T05:39:09.693873+0000","last_undegraded":"2024-09-17T05:39:09.693873+0000","last_fullsized":"2024-09-17T05:39:09.693873+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:16.440719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683840+0000","last_change":"2024-09-17T05:39:08.673866+0000","last_active":"2024-09-17T05:39:09.683840+0000","last_peered":"2024-09-17T05:39:09.683840+0000","last_clean":"2024-09-17T05:39:09.683840+0000","last_became_active":"2024-09-17T05:39:08.673542+0000","last_became_peered":"2024-09-17T05:39:08.673542+0000","last_unstale":"2024-09-17T05:39:09.683840+0000","last_undegraded":"2024-09-17T05:39:09.683840+0000","last_fullsized":"2024-09-17T05:39:09.683840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:11.138555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-17T05:39:08.666672+0000","last_change":"2024-09-17T05:39:08.666672+0000","last_active":"2024-09-17T05:39:08.655235+0000","last_peered":"2024-09-17T05:39:08.655235+0000","last_clean":"2024-09-17T05:39:08.655235+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:39:08.666672+0000","last_undegraded":"2024-09-17T05:39:08.666672+0000","last_fullsized":"2024-09-17T05:39:08.666672+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.682043+0000","last_change":"2024-09-17T05:39:08.680453+0000","last_active":"2024-09-17T05:39:08.682043+0000","last_peered":"2024-09-17T05:39:08.682043+0000","last_clean":"2024-09-17T05:39:08.682043+0000","last_became_active":"2024-09-17T05:39:08.680180+0000","last_became_peered":"2024-09-17T05:39:08.680180+0000","last_unstale":"2024-09-17T05:39:08.682043+0000","last_undegraded":"2024-09-17T05:39:08.682043+0000","last_fullsized":"2024-09-17T05:39:08.682043+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:23.479650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":317,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-17T05:39:08.666710+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:39:08.666710+0000","last_peered":"2024-09-17T05:39:08.666710+0000","last_clean":"2024-09-17T05:39:08.666710+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:39:08.666710+0000","last_undegraded":"2024-09-17T05:39:08.666710+0000","last_fullsized":"2024-09-17T05:39:08.666710+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509614,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127104,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244736,"statfs":{"total":96636764160,"available":96506609664,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136832,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235008,"statfs":{"total":96636764160,"available":96496648192,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131264,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240576,"statfs":{"total":96636764160,"available":96502349824,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509613,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131268,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240572,"statfs":{"total":96636764160,"available":96502345728,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127092,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244748,"statfs":{"total":96636764160,"available":96506621952,"internally_reserved":0,"allocated":1601536,"data_stored":703037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509615,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145716,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226124,"statfs":{"total":96636764160,"available":96487550976,"internally_reserved":0,"allocated":2195456,"data_stored":1293405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509613,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145148,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226692,"statfs":{"total":96636764160,"available":96488132608,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":146995659},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542319,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131836,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240004,"statfs":{"total":96636764160,"available":96501764096,"internally_reserved":0,"allocated":2195456,"data_stored":1293405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:39:14.585 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3f is not active+clean 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.675684+0000', 'last_change': '2024-09-17T05:39:08.675684+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.675684+0000', 'last_undegraded': '2024-09-17T05:39:08.675684+0000', 'last_fullsized': '2024-09-17T05:39:08.675684+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3e is not active+clean 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.681063+0000', 'last_change': '2024-09-17T05:39:08.681063+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.681063+0000', 'last_undegraded': '2024-09-17T05:39:08.681063+0000', 'last_fullsized': '2024-09-17T05:39:08.681063+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3d is not active+clean 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.668951+0000', 'last_change': '2024-09-17T05:39:08.668951+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.668951+0000', 'last_undegraded': '2024-09-17T05:39:08.668951+0000', 'last_fullsized': '2024-09-17T05:39:08.668951+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3c is not active+clean 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.666359+0000', 'last_change': '2024-09-17T05:39:08.666359+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.666359+0000', 'last_undegraded': '2024-09-17T05:39:08.666359+0000', 'last_fullsized': '2024-09-17T05:39:08.666359+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3b is not active+clean 2024-09-17T05:39:14.586 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.667851+0000', 'last_change': '2024-09-17T05:39:08.667851+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.667851+0000', 'last_undegraded': '2024-09-17T05:39:08.667851+0000', 'last_fullsized': '2024-09-17T05:39:08.667851+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3a is not active+clean 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.674261+0000', 'last_change': '2024-09-17T05:39:08.674261+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.674261+0000', 'last_undegraded': '2024-09-17T05:39:08.674261+0000', 'last_fullsized': '2024-09-17T05:39:08.674261+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:PG 19.39 is not active+clean 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.667467+0000', 'last_change': '2024-09-17T05:39:08.667467+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.667467+0000', 'last_undegraded': '2024-09-17T05:39:08.667467+0000', 'last_fullsized': '2024-09-17T05:39:08.667467+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:PG 19.38 is not active+clean 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.668743+0000', 'last_change': '2024-09-17T05:39:08.668743+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.668743+0000', 'last_undegraded': '2024-09-17T05:39:08.668743+0000', 'last_fullsized': '2024-09-17T05:39:08.668743+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:PG 19.37 is not active+clean 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.667026+0000', 'last_change': '2024-09-17T05:39:08.667026+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.667026+0000', 'last_undegraded': '2024-09-17T05:39:08.667026+0000', 'last_fullsized': '2024-09-17T05:39:08.667026+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:PG 19.36 is not active+clean 2024-09-17T05:39:14.587 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.678847+0000', 'last_change': '2024-09-17T05:39:08.678847+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.678847+0000', 'last_undegraded': '2024-09-17T05:39:08.678847+0000', 'last_fullsized': '2024-09-17T05:39:08.678847+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:PG 19.35 is not active+clean 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.666857+0000', 'last_change': '2024-09-17T05:39:08.666857+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.666857+0000', 'last_undegraded': '2024-09-17T05:39:08.666857+0000', 'last_fullsized': '2024-09-17T05:39:08.666857+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:PG 19.34 is not active+clean 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.678015+0000', 'last_change': '2024-09-17T05:39:08.678015+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.678015+0000', 'last_undegraded': '2024-09-17T05:39:08.678015+0000', 'last_fullsized': '2024-09-17T05:39:08.678015+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:PG 19.33 is not active+clean 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.668363+0000', 'last_change': '2024-09-17T05:39:08.668363+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.668363+0000', 'last_undegraded': '2024-09-17T05:39:08.668363+0000', 'last_fullsized': '2024-09-17T05:39:08.668363+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:PG 19.31 is not active+clean 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.662857+0000', 'last_change': '2024-09-17T05:39:08.662857+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.662857+0000', 'last_undegraded': '2024-09-17T05:39:08.662857+0000', 'last_fullsized': '2024-09-17T05:39:08.662857+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2f is not active+clean 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.666042+0000', 'last_change': '2024-09-17T05:39:08.666042+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.666042+0000', 'last_undegraded': '2024-09-17T05:39:08.666042+0000', 'last_fullsized': '2024-09-17T05:39:08.666042+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2e is not active+clean 2024-09-17T05:39:14.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.662651+0000', 'last_change': '2024-09-17T05:39:08.662651+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.662651+0000', 'last_undegraded': '2024-09-17T05:39:08.662651+0000', 'last_fullsized': '2024-09-17T05:39:08.662651+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:39:14.589 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2d is not active+clean 2024-09-17T05:39:14.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.675522+0000', 'last_change': '2024-09-17T05:39:08.675522+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.675522+0000', 'last_undegraded': '2024-09-17T05:39:08.675522+0000', 'last_fullsized': '2024-09-17T05:39:08.675522+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.589 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2c is not active+clean 2024-09-17T05:39:14.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.662332+0000', 'last_change': '2024-09-17T05:39:08.662332+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.662332+0000', 'last_undegraded': '2024-09-17T05:39:08.662332+0000', 'last_fullsized': '2024-09-17T05:39:08.662332+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:39:14.589 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2b is not active+clean 2024-09-17T05:39:14.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.677152+0000', 'last_change': '2024-09-17T05:39:08.677152+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.677152+0000', 'last_undegraded': '2024-09-17T05:39:08.677152+0000', 'last_fullsized': '2024-09-17T05:39:08.677152+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.589 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2a is not active+clean 2024-09-17T05:39:14.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.677957+0000', 'last_change': '2024-09-17T05:39:08.677957+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.677957+0000', 'last_undegraded': '2024-09-17T05:39:08.677957+0000', 'last_fullsized': '2024-09-17T05:39:08.677957+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:PG 19.29 is not active+clean 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.676749+0000', 'last_change': '2024-09-17T05:39:08.676749+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.676749+0000', 'last_undegraded': '2024-09-17T05:39:08.676749+0000', 'last_fullsized': '2024-09-17T05:39:08.676749+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:PG 19.28 is not active+clean 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.663391+0000', 'last_change': '2024-09-17T05:39:08.663391+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.663391+0000', 'last_undegraded': '2024-09-17T05:39:08.663391+0000', 'last_fullsized': '2024-09-17T05:39:08.663391+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:PG 19.27 is not active+clean 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.668585+0000', 'last_change': '2024-09-17T05:39:08.668585+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.668585+0000', 'last_undegraded': '2024-09-17T05:39:08.668585+0000', 'last_fullsized': '2024-09-17T05:39:08.668585+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:PG 19.26 is not active+clean 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.678343+0000', 'last_change': '2024-09-17T05:39:08.678343+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.678343+0000', 'last_undegraded': '2024-09-17T05:39:08.678343+0000', 'last_fullsized': '2024-09-17T05:39:08.678343+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:PG 19.24 is not active+clean 2024-09-17T05:39:14.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.674848+0000', 'last_change': '2024-09-17T05:39:08.674848+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.674848+0000', 'last_undegraded': '2024-09-17T05:39:08.674848+0000', 'last_fullsized': '2024-09-17T05:39:08.674848+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:39:14.591 INFO:tasks.ceph.ceph_manager.ceph:PG 19.23 is not active+clean 2024-09-17T05:39:14.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.670468+0000', 'last_change': '2024-09-17T05:39:08.670468+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.670468+0000', 'last_undegraded': '2024-09-17T05:39:08.670468+0000', 'last_fullsized': '2024-09-17T05:39:08.670468+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.591 INFO:tasks.ceph.ceph_manager.ceph:PG 19.22 is not active+clean 2024-09-17T05:39:14.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.671200+0000', 'last_change': '2024-09-17T05:39:08.671200+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.671200+0000', 'last_undegraded': '2024-09-17T05:39:08.671200+0000', 'last_fullsized': '2024-09-17T05:39:08.671200+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.591 INFO:tasks.ceph.ceph_manager.ceph:PG 19.21 is not active+clean 2024-09-17T05:39:14.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.675452+0000', 'last_change': '2024-09-17T05:39:08.675452+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.675452+0000', 'last_undegraded': '2024-09-17T05:39:08.675452+0000', 'last_fullsized': '2024-09-17T05:39:08.675452+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:39:14.591 INFO:tasks.ceph.ceph_manager.ceph:PG 19.20 is not active+clean 2024-09-17T05:39:14.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.669290+0000', 'last_change': '2024-09-17T05:39:08.669290+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.669290+0000', 'last_undegraded': '2024-09-17T05:39:08.669290+0000', 'last_fullsized': '2024-09-17T05:39:08.669290+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1f is not active+clean 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.666556+0000', 'last_change': '2024-09-17T05:39:08.666556+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.666556+0000', 'last_undegraded': '2024-09-17T05:39:08.666556+0000', 'last_fullsized': '2024-09-17T05:39:08.666556+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1c is not active+clean 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.667745+0000', 'last_change': '2024-09-17T05:39:08.667745+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.667745+0000', 'last_undegraded': '2024-09-17T05:39:08.667745+0000', 'last_fullsized': '2024-09-17T05:39:08.667745+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1d is not active+clean 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.674269+0000', 'last_change': '2024-09-17T05:39:08.674269+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.674269+0000', 'last_undegraded': '2024-09-17T05:39:08.674269+0000', 'last_fullsized': '2024-09-17T05:39:08.674269+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1a is not active+clean 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.662482+0000', 'last_change': '2024-09-17T05:39:08.662482+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.662482+0000', 'last_undegraded': '2024-09-17T05:39:08.662482+0000', 'last_fullsized': '2024-09-17T05:39:08.662482+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:39:14.592 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1b is not active+clean 2024-09-17T05:39:14.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.676787+0000', 'last_change': '2024-09-17T05:39:08.676787+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.676787+0000', 'last_undegraded': '2024-09-17T05:39:08.676787+0000', 'last_fullsized': '2024-09-17T05:39:08.676787+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.593 INFO:tasks.ceph.ceph_manager.ceph:PG 19.18 is not active+clean 2024-09-17T05:39:14.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.673730+0000', 'last_change': '2024-09-17T05:39:08.673730+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.673730+0000', 'last_undegraded': '2024-09-17T05:39:08.673730+0000', 'last_fullsized': '2024-09-17T05:39:08.673730+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.593 INFO:tasks.ceph.ceph_manager.ceph:PG 19.19 is not active+clean 2024-09-17T05:39:14.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.669585+0000', 'last_change': '2024-09-17T05:39:08.669585+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.669585+0000', 'last_undegraded': '2024-09-17T05:39:08.669585+0000', 'last_fullsized': '2024-09-17T05:39:08.669585+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.593 INFO:tasks.ceph.ceph_manager.ceph:PG 19.16 is not active+clean 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.671412+0000', 'last_change': '2024-09-17T05:39:08.671412+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.671412+0000', 'last_undegraded': '2024-09-17T05:39:08.671412+0000', 'last_fullsized': '2024-09-17T05:39:08.671412+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:PG 19.17 is not active+clean 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.667957+0000', 'last_change': '2024-09-17T05:39:08.667957+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.667957+0000', 'last_undegraded': '2024-09-17T05:39:08.667957+0000', 'last_fullsized': '2024-09-17T05:39:08.667957+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:PG 19.14 is not active+clean 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.674657+0000', 'last_change': '2024-09-17T05:39:08.674657+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.674657+0000', 'last_undegraded': '2024-09-17T05:39:08.674657+0000', 'last_fullsized': '2024-09-17T05:39:08.674657+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:PG 19.15 is not active+clean 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.673883+0000', 'last_change': '2024-09-17T05:39:08.673883+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.673883+0000', 'last_undegraded': '2024-09-17T05:39:08.673883+0000', 'last_fullsized': '2024-09-17T05:39:08.673883+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:PG 19.11 is not active+clean 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.675898+0000', 'last_change': '2024-09-17T05:39:08.675898+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.675898+0000', 'last_undegraded': '2024-09-17T05:39:08.675898+0000', 'last_fullsized': '2024-09-17T05:39:08.675898+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.594 INFO:tasks.ceph.ceph_manager.ceph:PG 19.e is not active+clean 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.669746+0000', 'last_change': '2024-09-17T05:39:08.669746+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.669746+0000', 'last_undegraded': '2024-09-17T05:39:08.669746+0000', 'last_fullsized': '2024-09-17T05:39:08.669746+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:PG 19.f is not active+clean 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.667465+0000', 'last_change': '2024-09-17T05:39:08.667465+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.667465+0000', 'last_undegraded': '2024-09-17T05:39:08.667465+0000', 'last_fullsized': '2024-09-17T05:39:08.667465+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:PG 19.c is not active+clean 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.676378+0000', 'last_change': '2024-09-17T05:39:08.676378+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.676378+0000', 'last_undegraded': '2024-09-17T05:39:08.676378+0000', 'last_fullsized': '2024-09-17T05:39:08.676378+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:PG 19.d is not active+clean 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.671695+0000', 'last_change': '2024-09-17T05:39:08.671695+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.671695+0000', 'last_undegraded': '2024-09-17T05:39:08.671695+0000', 'last_fullsized': '2024-09-17T05:39:08.671695+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:PG 19.a is not active+clean 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.670707+0000', 'last_change': '2024-09-17T05:39:08.670707+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.670707+0000', 'last_undegraded': '2024-09-17T05:39:08.670707+0000', 'last_fullsized': '2024-09-17T05:39:08.670707+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.595 INFO:tasks.ceph.ceph_manager.ceph:PG 19.b is not active+clean 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.673322+0000', 'last_change': '2024-09-17T05:39:08.673322+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.673322+0000', 'last_undegraded': '2024-09-17T05:39:08.673322+0000', 'last_fullsized': '2024-09-17T05:39:08.673322+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:PG 19.9 is not active+clean 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.674275+0000', 'last_change': '2024-09-17T05:39:08.674275+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.674275+0000', 'last_undegraded': '2024-09-17T05:39:08.674275+0000', 'last_fullsized': '2024-09-17T05:39:08.674275+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:PG 19.6 is not active+clean 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.672842+0000', 'last_change': '2024-09-17T05:39:08.672842+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.672842+0000', 'last_undegraded': '2024-09-17T05:39:08.672842+0000', 'last_fullsized': '2024-09-17T05:39:08.672842+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:PG 19.7 is not active+clean 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.671568+0000', 'last_change': '2024-09-17T05:39:08.671568+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.671568+0000', 'last_undegraded': '2024-09-17T05:39:08.671568+0000', 'last_fullsized': '2024-09-17T05:39:08.671568+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:PG 19.5 is not active+clean 2024-09-17T05:39:14.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.668496+0000', 'last_change': '2024-09-17T05:39:08.668496+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.668496+0000', 'last_undegraded': '2024-09-17T05:39:08.668496+0000', 'last_fullsized': '2024-09-17T05:39:08.668496+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:39:14.597 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2 is not active+clean 2024-09-17T05:39:14.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.673821+0000', 'last_change': '2024-09-17T05:39:08.673821+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.673821+0000', 'last_undegraded': '2024-09-17T05:39:08.673821+0000', 'last_fullsized': '2024-09-17T05:39:08.673821+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.597 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3 is not active+clean 2024-09-17T05:39:14.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.676324+0000', 'last_change': '2024-09-17T05:39:08.676324+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.676324+0000', 'last_undegraded': '2024-09-17T05:39:08.676324+0000', 'last_fullsized': '2024-09-17T05:39:08.676324+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:39:14.597 INFO:tasks.ceph.ceph_manager.ceph:PG 19.0 is not active+clean 2024-09-17T05:39:14.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.662397+0000', 'last_change': '2024-09-17T05:39:08.662397+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.662397+0000', 'last_undegraded': '2024-09-17T05:39:08.662397+0000', 'last_fullsized': '2024-09-17T05:39:08.662397+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:39:14.597 INFO:tasks.ceph.ceph_manager.ceph:PG 19.12 is not active+clean 2024-09-17T05:39:14.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 130, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:39:08.666672+0000', 'last_change': '2024-09-17T05:39:08.666672+0000', 'last_active': '2024-09-17T05:39:08.655235+0000', 'last_peered': '2024-09-17T05:39:08.655235+0000', 'last_clean': '2024-09-17T05:39:08.655235+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:39:08.666672+0000', 'last_undegraded': '2024-09-17T05:39:08.666672+0000', 'last_fullsized': '2024-09-17T05:39:08.666672+0000', 'mapping_epoch': 130, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 130, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'last_clean_scrub_stamp': '2024-09-17T05:39:08.655235+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:39:14.598 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:39:14.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.774+0000 7f4d36200640 1 -- 172.21.15.122:0/428039590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d300cfe80 msgr2=0x7f4d300d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.774+0000 7f4d36200640 1 --2- 172.21.15.122:0/428039590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d300cfe80 0x7f4d300d0260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009840 tx=0x7f4d2001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.774+0000 7f4d36200640 1 -- 172.21.15.122:0/428039590 shutdown_connections 2024-09-17T05:39:14.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.774+0000 7f4d36200640 1 --2- 172.21.15.122:0/428039590 >> v2:172.21.15.173:3300/0 conn(0x7f4d300b2760 0x7f4d301605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.774+0000 7f4d36200640 1 --2- 172.21.15.122:0/428039590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d300cfe80 0x7f4d300d0260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.774+0000 7f4d36200640 1 -- 172.21.15.122:0/428039590 >> 172.21.15.122:0/428039590 conn(0x7f4d300d1e40 msgr2=0x7f4d300d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:14.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.774+0000 7f4d36200640 1 -- 172.21.15.122:0/428039590 shutdown_connections 2024-09-17T05:39:14.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.774+0000 7f4d36200640 1 -- 172.21.15.122:0/428039590 wait complete. 2024-09-17T05:39:14.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 Processor -- start 2024-09-17T05:39:14.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 -- start start 2024-09-17T05:39:14.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d300b2760 0x7f4d300b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d300b2d00 0x7f4d300b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d300cfe80 0x7f4d30155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d300b0940 con 0x7f4d300cfe80 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d300b07c0 con 0x7f4d300b2d00 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4d300b0ac0 con 0x7f4d300b2760 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d300cfe80 0x7f4d30155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d300cfe80 0x7f4d30155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38846/0 (socket says 172.21.15.122:38846) 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 -- 172.21.15.122:0/3298864122 learned_addr learned my addr 172.21.15.122:0/3298864122 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 -- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d300b2760 msgr2=0x7f4d300b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d300b2760 0x7f4d300b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 -- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d300b2d00 msgr2=0x7f4d300b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d300b2d00 0x7f4d300b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 -- 172.21.15.122:0/3298864122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d300b6c30 con 0x7f4d300cfe80 2024-09-17T05:39:14.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d2fe00640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d300cfe80 0x7f4d30155040 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800eaf0 tx=0x7f4d2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:14.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d27400640 1 -- 172.21.15.122:0/3298864122 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28004b30 con 0x7f4d300cfe80 2024-09-17T05:39:14.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d27400640 1 -- 172.21.15.122:0/3298864122 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d28004cf0 con 0x7f4d300cfe80 2024-09-17T05:39:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d300b79a0 con 0x7f4d300cfe80 2024-09-17T05:39:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d27400640 1 -- 172.21.15.122:0/3298864122 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28005280 con 0x7f4d300cfe80 2024-09-17T05:39:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d30155d00 con 0x7f4d300cfe80 2024-09-17T05:39:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.778+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d30155810 con 0x7f4d300cfe80 2024-09-17T05:39:14.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.782+0000 7f4d27400640 1 -- 172.21.15.122:0/3298864122 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f4d28005420 con 0x7f4d300cfe80 2024-09-17T05:39:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.782+0000 7f4d27400640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d10075180 0x7f4d10077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.782+0000 7f4d27400640 1 -- 172.21.15.122:0/3298864122 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f4d28093f80 con 0x7f4d300cfe80 2024-09-17T05:39:14.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.782+0000 7f4d2f400640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d10075180 0x7f4d10077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:14.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.782+0000 7f4d2f400640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d10075180 0x7f4d10077640 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4d200ae000 tx=0x7f4d20019450 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:14.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:14.786+0000 7f4d27400640 1 -- 172.21.15.122:0/3298864122 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4d28060a90 con 0x7f4d300cfe80 2024-09-17T05:39:15.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.018+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4d300d0260 con 0x7f4d300cfe80 2024-09-17T05:39:15.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.018+0000 7f4d27400640 1 -- 172.21.15.122:0/3298864122 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1079 (secure 0 0 0) 0x7f4d28060390 con 0x7f4d300cfe80 2024-09-17T05:39:15.014 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:15.014 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":875,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550693,"num_in_osds":8,"osd_in_since":1726550682,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":55},{"state_name":"active+clean","count":42}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":1102090240,"bytes_avail":771992023040,"bytes_total":773094113280,"inactive_pgs_ratio":0.56701028347015381},"fsmap":{"epoch":86,"btime":"2024-09-17T05:39:12:702401+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":8916}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-09-17T05:38:50.731214+0000","services":{}},"progress_events":{}} 2024-09-17T05:39:15.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d10075180 msgr2=0x7f4d10077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d10075180 0x7f4d10077640 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4d200ae000 tx=0x7f4d20019450 comp rx=0 tx=0).stop 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d300cfe80 msgr2=0x7f4d30155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d300cfe80 0x7f4d30155040 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800eaf0 tx=0x7f4d2800c2d0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 shutdown_connections 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4d10075180 0x7f4d10077640 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d300cfe80 0x7f4d30155040 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d300b2d00 0x7f4d300b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 --2- 172.21.15.122:0/3298864122 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d300b2760 0x7f4d300b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 >> 172.21.15.122:0/3298864122 conn(0x7f4d300d1e40 msgr2=0x7f4d30070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 shutdown_connections 2024-09-17T05:39:15.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.022+0000 7f4d36200640 1 -- 172.21.15.122:0/3298864122 wait complete. 2024-09-17T05:39:15.035 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:39:15.035 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:39:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- 172.21.15.122:0/1425600223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a500b2e80 msgr2=0x7f9a500b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 --2- 172.21.15.122:0/1425600223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a500b2e80 0x7f9a500b1bd0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009900 tx=0x7f9a4001c820 comp rx=0 tx=0).stop 2024-09-17T05:39:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- 172.21.15.122:0/1425600223 shutdown_connections 2024-09-17T05:39:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 --2- 172.21.15.122:0/1425600223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a500afca0 0x7f9a500b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 --2- 172.21.15.122:0/1425600223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a500b2e80 0x7f9a500b1bd0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- 172.21.15.122:0/1425600223 >> 172.21.15.122:0/1425600223 conn(0x7f9a500d1e30 msgr2=0x7f9a500d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- 172.21.15.122:0/1425600223 shutdown_connections 2024-09-17T05:39:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- 172.21.15.122:0/1425600223 wait complete. 2024-09-17T05:39:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 Processor -- start 2024-09-17T05:39:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- start start 2024-09-17T05:39:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a500afca0 0x7f9a5014b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a500b2e80 0x7f9a501558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a5015f490 0x7f9a50155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50168e00 con 0x7f9a5015f490 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50168c80 con 0x7f9a500b2e80 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a58800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9a50168f80 con 0x7f9a500afca0 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a56800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a5015f490 0x7f9a50155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a56800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a5015f490 0x7f9a50155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38862/0 (socket says 172.21.15.122:38862) 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a56800640 1 -- 172.21.15.122:0/569454679 learned_addr learned my addr 172.21.15.122:0/569454679 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a55e00640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a500afca0 0x7f9a5014b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.182+0000 7f9a55400640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a500b2e80 0x7f9a501558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a56800640 1 -- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a500afca0 msgr2=0x7f9a5014b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a56800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a500afca0 0x7f9a5014b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a56800640 1 -- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a500b2e80 msgr2=0x7f9a501558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a56800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a500b2e80 0x7f9a501558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a56800640 1 -- 172.21.15.122:0/569454679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5014bc70 con 0x7f9a5015f490 2024-09-17T05:39:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a56800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a5015f490 0x7f9a50155de0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9a0 tx=0x7f9a4c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a4a000640 1 -- 172.21.15.122:0/569454679 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c004d30 con 0x7f9a5015f490 2024-09-17T05:39:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a4a000640 1 -- 172.21.15.122:0/569454679 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a4c00cb20 con 0x7f9a5015f490 2024-09-17T05:39:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a4a000640 1 -- 172.21.15.122:0/569454679 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c005020 con 0x7f9a5015f490 2024-09-17T05:39:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5014f8e0 con 0x7f9a5015f490 2024-09-17T05:39:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5014fea0 con 0x7f9a5015f490 2024-09-17T05:39:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a500b0110 con 0x7f9a5015f490 2024-09-17T05:39:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.186+0000 7f9a4a000640 1 -- 172.21.15.122:0/569454679 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f9a4c0051c0 con 0x7f9a5015f490 2024-09-17T05:39:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.190+0000 7f9a4a000640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a34075180 0x7f9a34077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:15.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.190+0000 7f9a4a000640 1 -- 172.21.15.122:0/569454679 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f9a4c093af0 con 0x7f9a5015f490 2024-09-17T05:39:15.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.190+0000 7f9a55e00640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a34075180 0x7f9a34077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:15.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.190+0000 7f9a55e00640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a34075180 0x7f9a34077640 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9a44007fd0 tx=0x7f9a44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:15.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.190+0000 7f9a4a000640 1 -- 172.21.15.122:0/569454679 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9a4c016030 con 0x7f9a5015f490 2024-09-17T05:39:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.358+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a5015c760 con 0x7f9a34075180 2024-09-17T05:39:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.362+0000 7f9a4a000640 1 -- 172.21.15.122:0/569454679 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252498 (secure 0 0 0) 0x7f9a5015c760 con 0x7f9a34075180 2024-09-17T05:39:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:39:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a34075180 msgr2=0x7f9a34077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a34075180 0x7f9a34077640 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9a44007fd0 tx=0x7f9a44009040 comp rx=0 tx=0).stop 2024-09-17T05:39:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a5015f490 msgr2=0x7f9a50155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a5015f490 0x7f9a50155de0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00e9a0 tx=0x7f9a4c00ee60 comp rx=0 tx=0).stop 2024-09-17T05:39:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 shutdown_connections 2024-09-17T05:39:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9a34075180 0x7f9a34077640 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9a5015f490 0x7f9a50155de0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a500b2e80 0x7f9a501558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 --2- 172.21.15.122:0/569454679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a500afca0 0x7f9a5014b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 >> 172.21.15.122:0/569454679 conn(0x7f9a500d1e30 msgr2=0x7f9a500b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 shutdown_connections 2024-09-17T05:39:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:15.366+0000 7f9a58800640 1 -- 172.21.15.122:0/569454679 wait complete. 2024-09-17T05:39:15.381 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":556,"stamp":"2024-09-17T05:39:14.736543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076516,"kb_used_data":14000,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898204,"statfs":{"total":773094113280,"available":771991760896,"internally_reserved":0,"allocated":14336000,"data_stored":6953425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1087819318},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.076878"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050018+0000","last_change":"2024-09-17T05:39:09.704682+0000","last_active":"2024-09-17T05:39:12.050018+0000","last_peered":"2024-09-17T05:39:12.050018+0000","last_clean":"2024-09-17T05:39:12.050018+0000","last_became_active":"2024-09-17T05:39:09.704406+0000","last_became_peered":"2024-09-17T05:39:09.704406+0000","last_unstale":"2024-09-17T05:39:12.050018+0000","last_undegraded":"2024-09-17T05:39:12.050018+0000","last_fullsized":"2024-09-17T05:39:12.050018+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:17.125213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049063+0000","last_change":"2024-09-17T05:39:09.701535+0000","last_active":"2024-09-17T05:39:12.049063+0000","last_peered":"2024-09-17T05:39:12.049063+0000","last_clean":"2024-09-17T05:39:12.049063+0000","last_became_active":"2024-09-17T05:39:09.700923+0000","last_became_peered":"2024-09-17T05:39:09.700923+0000","last_unstale":"2024-09-17T05:39:12.049063+0000","last_undegraded":"2024-09-17T05:39:12.049063+0000","last_fullsized":"2024-09-17T05:39:12.049063+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:36.746214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689709+0000","last_change":"2024-09-17T05:39:09.710429+0000","last_active":"2024-09-17T05:39:10.689709+0000","last_peered":"2024-09-17T05:39:10.689709+0000","last_clean":"2024-09-17T05:39:10.689709+0000","last_became_active":"2024-09-17T05:39:09.708138+0000","last_became_peered":"2024-09-17T05:39:09.708138+0000","last_unstale":"2024-09-17T05:39:10.689709+0000","last_undegraded":"2024-09-17T05:39:10.689709+0000","last_fullsized":"2024-09-17T05:39:10.689709+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:30.537275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049168+0000","last_change":"2024-09-17T05:39:09.696346+0000","last_active":"2024-09-17T05:39:12.049168+0000","last_peered":"2024-09-17T05:39:12.049168+0000","last_clean":"2024-09-17T05:39:12.049168+0000","last_became_active":"2024-09-17T05:39:09.696079+0000","last_became_peered":"2024-09-17T05:39:09.696079+0000","last_unstale":"2024-09-17T05:39:12.049168+0000","last_undegraded":"2024-09-17T05:39:12.049168+0000","last_fullsized":"2024-09-17T05:39:12.049168+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:20.246762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685522+0000","last_change":"2024-09-17T05:39:09.689996+0000","last_active":"2024-09-17T05:39:10.685522+0000","last_peered":"2024-09-17T05:39:10.685522+0000","last_clean":"2024-09-17T05:39:10.685522+0000","last_became_active":"2024-09-17T05:39:09.688528+0000","last_became_peered":"2024-09-17T05:39:09.688528+0000","last_unstale":"2024-09-17T05:39:10.685522+0000","last_undegraded":"2024-09-17T05:39:10.685522+0000","last_fullsized":"2024-09-17T05:39:10.685522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:16.440719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707109+0000","last_change":"2024-09-17T05:39:09.707106+0000","last_active":"2024-09-17T05:39:10.707109+0000","last_peered":"2024-09-17T05:39:10.707109+0000","last_clean":"2024-09-17T05:39:10.707109+0000","last_became_active":"2024-09-17T05:39:09.706836+0000","last_became_peered":"2024-09-17T05:39:09.706836+0000","last_unstale":"2024-09-17T05:39:10.707109+0000","last_undegraded":"2024-09-17T05:39:10.707109+0000","last_fullsized":"2024-09-17T05:39:10.707109+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:42.431203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685562+0000","last_change":"2024-09-17T05:39:09.698951+0000","last_active":"2024-09-17T05:39:10.685562+0000","last_peered":"2024-09-17T05:39:10.685562+0000","last_clean":"2024-09-17T05:39:10.685562+0000","last_became_active":"2024-09-17T05:39:09.698545+0000","last_became_peered":"2024-09-17T05:39:09.698545+0000","last_unstale":"2024-09-17T05:39:10.685562+0000","last_undegraded":"2024-09-17T05:39:10.685562+0000","last_fullsized":"2024-09-17T05:39:10.685562+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:10.630444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685198+0000","last_change":"2024-09-17T05:39:09.706086+0000","last_active":"2024-09-17T05:39:10.685198+0000","last_peered":"2024-09-17T05:39:10.685198+0000","last_clean":"2024-09-17T05:39:10.685198+0000","last_became_active":"2024-09-17T05:39:09.705904+0000","last_became_peered":"2024-09-17T05:39:09.705904+0000","last_unstale":"2024-09-17T05:39:10.685198+0000","last_undegraded":"2024-09-17T05:39:10.685198+0000","last_fullsized":"2024-09-17T05:39:10.685198+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:44.618638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689951+0000","last_change":"2024-09-17T05:39:09.708268+0000","last_active":"2024-09-17T05:39:10.689951+0000","last_peered":"2024-09-17T05:39:10.689951+0000","last_clean":"2024-09-17T05:39:10.689951+0000","last_became_active":"2024-09-17T05:39:09.707945+0000","last_became_peered":"2024-09-17T05:39:09.707945+0000","last_unstale":"2024-09-17T05:39:10.689951+0000","last_undegraded":"2024-09-17T05:39:10.689951+0000","last_fullsized":"2024-09-17T05:39:10.689951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:56.688630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049361+0000","last_change":"2024-09-17T05:39:09.697958+0000","last_active":"2024-09-17T05:39:12.049361+0000","last_peered":"2024-09-17T05:39:12.049361+0000","last_clean":"2024-09-17T05:39:12.049361+0000","last_became_active":"2024-09-17T05:39:09.697649+0000","last_became_peered":"2024-09-17T05:39:09.697649+0000","last_unstale":"2024-09-17T05:39:12.049361+0000","last_undegraded":"2024-09-17T05:39:12.049361+0000","last_fullsized":"2024-09-17T05:39:12.049361+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:37.853686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049467+0000","last_change":"2024-09-17T05:39:09.701372+0000","last_active":"2024-09-17T05:39:12.049467+0000","last_peered":"2024-09-17T05:39:12.049467+0000","last_clean":"2024-09-17T05:39:12.049467+0000","last_became_active":"2024-09-17T05:39:09.700339+0000","last_became_peered":"2024-09-17T05:39:09.700339+0000","last_unstale":"2024-09-17T05:39:12.049467+0000","last_undegraded":"2024-09-17T05:39:12.049467+0000","last_fullsized":"2024-09-17T05:39:12.049467+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:44.935564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707456+0000","last_change":"2024-09-17T05:39:09.688279+0000","last_active":"2024-09-17T05:39:10.707456+0000","last_peered":"2024-09-17T05:39:10.707456+0000","last_clean":"2024-09-17T05:39:10.707456+0000","last_became_active":"2024-09-17T05:39:09.687637+0000","last_became_peered":"2024-09-17T05:39:09.687637+0000","last_unstale":"2024-09-17T05:39:10.707456+0000","last_undegraded":"2024-09-17T05:39:10.707456+0000","last_fullsized":"2024-09-17T05:39:10.707456+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:36.746214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049429+0000","last_change":"2024-09-17T05:39:09.702005+0000","last_active":"2024-09-17T05:39:12.049429+0000","last_peered":"2024-09-17T05:39:12.049429+0000","last_clean":"2024-09-17T05:39:12.049429+0000","last_became_active":"2024-09-17T05:39:09.701415+0000","last_became_peered":"2024-09-17T05:39:09.701415+0000","last_unstale":"2024-09-17T05:39:12.049429+0000","last_undegraded":"2024-09-17T05:39:12.049429+0000","last_fullsized":"2024-09-17T05:39:12.049429+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:38.585539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.694612+0000","last_change":"2024-09-17T05:39:09.692704+0000","last_active":"2024-09-17T05:39:09.694612+0000","last_peered":"2024-09-17T05:39:09.694612+0000","last_clean":"2024-09-17T05:39:09.694612+0000","last_became_active":"2024-09-17T05:39:09.690032+0000","last_became_peered":"2024-09-17T05:39:09.690032+0000","last_unstale":"2024-09-17T05:39:09.694612+0000","last_undegraded":"2024-09-17T05:39:09.694612+0000","last_fullsized":"2024-09-17T05:39:09.694612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:10.630444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689821+0000","last_change":"2024-09-17T05:39:09.687863+0000","last_active":"2024-09-17T05:39:10.689821+0000","last_peered":"2024-09-17T05:39:10.689821+0000","last_clean":"2024-09-17T05:39:10.689821+0000","last_became_active":"2024-09-17T05:39:09.687453+0000","last_became_peered":"2024-09-17T05:39:09.687453+0000","last_unstale":"2024-09-17T05:39:10.689821+0000","last_undegraded":"2024-09-17T05:39:10.689821+0000","last_fullsized":"2024-09-17T05:39:10.689821+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:44.125738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702572+0000","last_change":"2024-09-17T05:39:09.698973+0000","last_active":"2024-09-17T05:39:09.702572+0000","last_peered":"2024-09-17T05:39:09.702572+0000","last_clean":"2024-09-17T05:39:09.702572+0000","last_became_active":"2024-09-17T05:39:09.698375+0000","last_became_peered":"2024-09-17T05:39:09.698375+0000","last_unstale":"2024-09-17T05:39:09.702572+0000","last_undegraded":"2024-09-17T05:39:09.702572+0000","last_fullsized":"2024-09-17T05:39:09.702572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:28.895178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690254+0000","last_change":"2024-09-17T05:39:09.705660+0000","last_active":"2024-09-17T05:39:10.690254+0000","last_peered":"2024-09-17T05:39:10.690254+0000","last_clean":"2024-09-17T05:39:10.690254+0000","last_became_active":"2024-09-17T05:39:09.705412+0000","last_became_peered":"2024-09-17T05:39:09.705412+0000","last_unstale":"2024-09-17T05:39:10.690254+0000","last_undegraded":"2024-09-17T05:39:10.690254+0000","last_fullsized":"2024-09-17T05:39:10.690254+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:23.159202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690473+0000","last_change":"2024-09-17T05:39:09.697274+0000","last_active":"2024-09-17T05:39:10.690473+0000","last_peered":"2024-09-17T05:39:10.690473+0000","last_clean":"2024-09-17T05:39:10.690473+0000","last_became_active":"2024-09-17T05:39:09.696899+0000","last_became_peered":"2024-09-17T05:39:09.696899+0000","last_unstale":"2024-09-17T05:39:10.690473+0000","last_undegraded":"2024-09-17T05:39:10.690473+0000","last_fullsized":"2024-09-17T05:39:10.690473+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:33.478612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049039+0000","last_change":"2024-09-17T05:39:09.704245+0000","last_active":"2024-09-17T05:39:12.049039+0000","last_peered":"2024-09-17T05:39:12.049039+0000","last_clean":"2024-09-17T05:39:12.049039+0000","last_became_active":"2024-09-17T05:39:09.703894+0000","last_became_peered":"2024-09-17T05:39:09.703894+0000","last_unstale":"2024-09-17T05:39:12.049039+0000","last_undegraded":"2024-09-17T05:39:12.049039+0000","last_fullsized":"2024-09-17T05:39:12.049039+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:15.709405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689908+0000","last_change":"2024-09-17T05:39:09.699326+0000","last_active":"2024-09-17T05:39:10.689908+0000","last_peered":"2024-09-17T05:39:10.689908+0000","last_clean":"2024-09-17T05:39:10.689908+0000","last_became_active":"2024-09-17T05:39:09.697050+0000","last_became_peered":"2024-09-17T05:39:09.697050+0000","last_unstale":"2024-09-17T05:39:10.689908+0000","last_undegraded":"2024-09-17T05:39:10.689908+0000","last_fullsized":"2024-09-17T05:39:10.689908+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:51.190185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707114+0000","last_change":"2024-09-17T05:39:09.687255+0000","last_active":"2024-09-17T05:39:10.707114+0000","last_peered":"2024-09-17T05:39:10.707114+0000","last_clean":"2024-09-17T05:39:10.707114+0000","last_became_active":"2024-09-17T05:39:09.686837+0000","last_became_peered":"2024-09-17T05:39:09.686837+0000","last_unstale":"2024-09-17T05:39:10.707114+0000","last_undegraded":"2024-09-17T05:39:10.707114+0000","last_fullsized":"2024-09-17T05:39:10.707114+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:34.045368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049149+0000","last_change":"2024-09-17T05:39:09.706086+0000","last_active":"2024-09-17T05:39:12.049149+0000","last_peered":"2024-09-17T05:39:12.049149+0000","last_clean":"2024-09-17T05:39:12.049149+0000","last_became_active":"2024-09-17T05:39:09.705785+0000","last_became_peered":"2024-09-17T05:39:09.705785+0000","last_unstale":"2024-09-17T05:39:12.049149+0000","last_undegraded":"2024-09-17T05:39:12.049149+0000","last_fullsized":"2024-09-17T05:39:12.049149+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:27.337857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462125+0000","last_change":"2024-09-17T05:39:09.684994+0000","last_active":"2024-09-17T05:39:11.462125+0000","last_peered":"2024-09-17T05:39:11.462125+0000","last_clean":"2024-09-17T05:39:11.462125+0000","last_became_active":"2024-09-17T05:39:09.684689+0000","last_became_peered":"2024-09-17T05:39:09.684689+0000","last_unstale":"2024-09-17T05:39:11.462125+0000","last_undegraded":"2024-09-17T05:39:11.462125+0000","last_fullsized":"2024-09-17T05:39:11.462125+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:59.788891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689711+0000","last_change":"2024-09-17T05:39:09.690410+0000","last_active":"2024-09-17T05:39:10.689711+0000","last_peered":"2024-09-17T05:39:10.689711+0000","last_clean":"2024-09-17T05:39:10.689711+0000","last_became_active":"2024-09-17T05:39:09.690127+0000","last_became_peered":"2024-09-17T05:39:09.690127+0000","last_unstale":"2024-09-17T05:39:10.689711+0000","last_undegraded":"2024-09-17T05:39:10.689711+0000","last_fullsized":"2024-09-17T05:39:10.689711+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:24.189535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050144+0000","last_change":"2024-09-17T05:39:09.700429+0000","last_active":"2024-09-17T05:39:12.050144+0000","last_peered":"2024-09-17T05:39:12.050144+0000","last_clean":"2024-09-17T05:39:12.050144+0000","last_became_active":"2024-09-17T05:39:09.700176+0000","last_became_peered":"2024-09-17T05:39:09.700176+0000","last_unstale":"2024-09-17T05:39:12.050144+0000","last_undegraded":"2024-09-17T05:39:12.050144+0000","last_fullsized":"2024-09-17T05:39:12.050144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:21.486551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049774+0000","last_change":"2024-09-17T05:39:09.701218+0000","last_active":"2024-09-17T05:39:12.049774+0000","last_peered":"2024-09-17T05:39:12.049774+0000","last_clean":"2024-09-17T05:39:12.049774+0000","last_became_active":"2024-09-17T05:39:09.700602+0000","last_became_peered":"2024-09-17T05:39:09.700602+0000","last_unstale":"2024-09-17T05:39:12.049774+0000","last_undegraded":"2024-09-17T05:39:12.049774+0000","last_fullsized":"2024-09-17T05:39:12.049774+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:31.026302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.699887+0000","last_change":"2024-09-17T05:39:09.693510+0000","last_active":"2024-09-17T05:39:09.699887+0000","last_peered":"2024-09-17T05:39:09.699887+0000","last_clean":"2024-09-17T05:39:09.699887+0000","last_became_active":"2024-09-17T05:39:09.693223+0000","last_became_peered":"2024-09-17T05:39:09.693223+0000","last_unstale":"2024-09-17T05:39:09.699887+0000","last_undegraded":"2024-09-17T05:39:09.699887+0000","last_fullsized":"2024-09-17T05:39:09.699887+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:54.958914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462230+0000","last_change":"2024-09-17T05:39:09.687403+0000","last_active":"2024-09-17T05:39:11.462230+0000","last_peered":"2024-09-17T05:39:11.462230+0000","last_clean":"2024-09-17T05:39:11.462230+0000","last_became_active":"2024-09-17T05:39:09.687119+0000","last_became_peered":"2024-09-17T05:39:09.687119+0000","last_unstale":"2024-09-17T05:39:11.462230+0000","last_undegraded":"2024-09-17T05:39:11.462230+0000","last_fullsized":"2024-09-17T05:39:11.462230+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:31.710346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049072+0000","last_change":"2024-09-17T05:39:09.703494+0000","last_active":"2024-09-17T05:39:12.049072+0000","last_peered":"2024-09-17T05:39:12.049072+0000","last_clean":"2024-09-17T05:39:12.049072+0000","last_became_active":"2024-09-17T05:39:09.703179+0000","last_became_peered":"2024-09-17T05:39:09.703179+0000","last_unstale":"2024-09-17T05:39:12.049072+0000","last_undegraded":"2024-09-17T05:39:12.049072+0000","last_fullsized":"2024-09-17T05:39:12.049072+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:48.869239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049892+0000","last_change":"2024-09-17T05:39:09.699512+0000","last_active":"2024-09-17T05:39:12.049892+0000","last_peered":"2024-09-17T05:39:12.049892+0000","last_clean":"2024-09-17T05:39:12.049892+0000","last_became_active":"2024-09-17T05:39:09.699240+0000","last_became_peered":"2024-09-17T05:39:09.699240+0000","last_unstale":"2024-09-17T05:39:12.049892+0000","last_undegraded":"2024-09-17T05:39:12.049892+0000","last_fullsized":"2024-09-17T05:39:12.049892+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:06.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462321+0000","last_change":"2024-09-17T05:39:09.683586+0000","last_active":"2024-09-17T05:39:11.462321+0000","last_peered":"2024-09-17T05:39:11.462321+0000","last_clean":"2024-09-17T05:39:11.462321+0000","last_became_active":"2024-09-17T05:39:09.683275+0000","last_became_peered":"2024-09-17T05:39:09.683275+0000","last_unstale":"2024-09-17T05:39:11.462321+0000","last_undegraded":"2024-09-17T05:39:11.462321+0000","last_fullsized":"2024-09-17T05:39:11.462321+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:09.051635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690081+0000","last_change":"2024-09-17T05:39:09.706818+0000","last_active":"2024-09-17T05:39:10.690081+0000","last_peered":"2024-09-17T05:39:10.690081+0000","last_clean":"2024-09-17T05:39:10.690081+0000","last_became_active":"2024-09-17T05:39:09.705441+0000","last_became_peered":"2024-09-17T05:39:09.705441+0000","last_unstale":"2024-09-17T05:39:10.690081+0000","last_undegraded":"2024-09-17T05:39:10.690081+0000","last_fullsized":"2024-09-17T05:39:10.690081+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:37.448980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.703188+0000","last_change":"2024-09-17T05:39:09.698329+0000","last_active":"2024-09-17T05:39:09.703188+0000","last_peered":"2024-09-17T05:39:09.703188+0000","last_clean":"2024-09-17T05:39:09.703188+0000","last_became_active":"2024-09-17T05:39:09.698084+0000","last_became_peered":"2024-09-17T05:39:09.698084+0000","last_unstale":"2024-09-17T05:39:09.703188+0000","last_undegraded":"2024-09-17T05:39:09.703188+0000","last_fullsized":"2024-09-17T05:39:09.703188+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:47.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":19,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.686013+0000","last_change":"2024-09-17T05:39:08.676570+0000","last_active":"2024-09-17T05:39:10.686013+0000","last_peered":"2024-09-17T05:39:10.686013+0000","last_clean":"2024-09-17T05:39:10.686013+0000","last_became_active":"2024-09-17T05:39:08.676332+0000","last_became_peered":"2024-09-17T05:39:08.676332+0000","last_unstale":"2024-09-17T05:39:10.686013+0000","last_undegraded":"2024-09-17T05:39:10.686013+0000","last_fullsized":"2024-09-17T05:39:10.686013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:52.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690193+0000","last_change":"2024-09-17T05:39:09.704965+0000","last_active":"2024-09-17T05:39:10.690193+0000","last_peered":"2024-09-17T05:39:10.690193+0000","last_clean":"2024-09-17T05:39:10.690193+0000","last_became_active":"2024-09-17T05:39:09.704735+0000","last_became_peered":"2024-09-17T05:39:09.704735+0000","last_unstale":"2024-09-17T05:39:10.690193+0000","last_undegraded":"2024-09-17T05:39:10.690193+0000","last_fullsized":"2024-09-17T05:39:10.690193+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:31.999643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689863+0000","last_change":"2024-09-17T05:39:08.681409+0000","last_active":"2024-09-17T05:39:10.689863+0000","last_peered":"2024-09-17T05:39:10.689863+0000","last_clean":"2024-09-17T05:39:10.689863+0000","last_became_active":"2024-09-17T05:39:08.681149+0000","last_became_peered":"2024-09-17T05:39:08.681149+0000","last_unstale":"2024-09-17T05:39:10.689863+0000","last_undegraded":"2024-09-17T05:39:10.689863+0000","last_fullsized":"2024-09-17T05:39:10.689863+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:50.068381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689662+0000","last_change":"2024-09-17T05:39:09.699757+0000","last_active":"2024-09-17T05:39:10.689662+0000","last_peered":"2024-09-17T05:39:10.689662+0000","last_clean":"2024-09-17T05:39:10.689662+0000","last_became_active":"2024-09-17T05:39:09.699479+0000","last_became_peered":"2024-09-17T05:39:09.699479+0000","last_unstale":"2024-09-17T05:39:10.689662+0000","last_undegraded":"2024-09-17T05:39:10.689662+0000","last_fullsized":"2024-09-17T05:39:10.689662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:42.172686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049530+0000","last_change":"2024-09-17T05:39:08.673698+0000","last_active":"2024-09-17T05:39:12.049530+0000","last_peered":"2024-09-17T05:39:12.049530+0000","last_clean":"2024-09-17T05:39:12.049530+0000","last_became_active":"2024-09-17T05:39:08.673385+0000","last_became_peered":"2024-09-17T05:39:08.673385+0000","last_unstale":"2024-09-17T05:39:12.049530+0000","last_undegraded":"2024-09-17T05:39:12.049530+0000","last_fullsized":"2024-09-17T05:39:12.049530+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:56.140398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707776+0000","last_change":"2024-09-17T05:39:09.688155+0000","last_active":"2024-09-17T05:39:10.707776+0000","last_peered":"2024-09-17T05:39:10.707776+0000","last_clean":"2024-09-17T05:39:10.707776+0000","last_became_active":"2024-09-17T05:39:09.687764+0000","last_became_peered":"2024-09-17T05:39:09.687764+0000","last_unstale":"2024-09-17T05:39:10.707776+0000","last_undegraded":"2024-09-17T05:39:10.707776+0000","last_fullsized":"2024-09-17T05:39:10.707776+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:07.017629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049638+0000","last_change":"2024-09-17T05:39:08.680638+0000","last_active":"2024-09-17T05:39:12.049638+0000","last_peered":"2024-09-17T05:39:12.049638+0000","last_clean":"2024-09-17T05:39:12.049638+0000","last_became_active":"2024-09-17T05:39:08.680162+0000","last_became_peered":"2024-09-17T05:39:08.680162+0000","last_unstale":"2024-09-17T05:39:12.049638+0000","last_undegraded":"2024-09-17T05:39:12.049638+0000","last_fullsized":"2024-09-17T05:39:12.049638+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:24.006992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690065+0000","last_change":"2024-09-17T05:39:09.690019+0000","last_active":"2024-09-17T05:39:10.690065+0000","last_peered":"2024-09-17T05:39:10.690065+0000","last_clean":"2024-09-17T05:39:10.690065+0000","last_became_active":"2024-09-17T05:39:09.689586+0000","last_became_peered":"2024-09-17T05:39:09.689586+0000","last_unstale":"2024-09-17T05:39:10.690065+0000","last_undegraded":"2024-09-17T05:39:10.690065+0000","last_fullsized":"2024-09-17T05:39:10.690065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:14.278198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049812+0000","last_change":"2024-09-17T05:39:08.679285+0000","last_active":"2024-09-17T05:39:12.049812+0000","last_peered":"2024-09-17T05:39:12.049812+0000","last_clean":"2024-09-17T05:39:12.049812+0000","last_became_active":"2024-09-17T05:39:08.679008+0000","last_became_peered":"2024-09-17T05:39:08.679008+0000","last_unstale":"2024-09-17T05:39:12.049812+0000","last_undegraded":"2024-09-17T05:39:12.049812+0000","last_fullsized":"2024-09-17T05:39:12.049812+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:13.783924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707215+0000","last_change":"2024-09-17T05:39:09.692770+0000","last_active":"2024-09-17T05:39:10.707215+0000","last_peered":"2024-09-17T05:39:10.707215+0000","last_clean":"2024-09-17T05:39:10.707215+0000","last_became_active":"2024-09-17T05:39:09.690591+0000","last_became_peered":"2024-09-17T05:39:09.690591+0000","last_unstale":"2024-09-17T05:39:10.707215+0000","last_undegraded":"2024-09-17T05:39:10.707215+0000","last_fullsized":"2024-09-17T05:39:10.707215+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:09.574102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689932+0000","last_change":"2024-09-17T05:39:08.682028+0000","last_active":"2024-09-17T05:39:10.689932+0000","last_peered":"2024-09-17T05:39:10.689932+0000","last_clean":"2024-09-17T05:39:10.689932+0000","last_became_active":"2024-09-17T05:39:08.680621+0000","last_became_peered":"2024-09-17T05:39:08.680621+0000","last_unstale":"2024-09-17T05:39:10.689932+0000","last_undegraded":"2024-09-17T05:39:10.689932+0000","last_fullsized":"2024-09-17T05:39:10.689932+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:26.453145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.706980+0000","last_change":"2024-09-17T05:39:09.686811+0000","last_active":"2024-09-17T05:39:10.706980+0000","last_peered":"2024-09-17T05:39:10.706980+0000","last_clean":"2024-09-17T05:39:10.706980+0000","last_became_active":"2024-09-17T05:39:09.685430+0000","last_became_peered":"2024-09-17T05:39:09.685430+0000","last_unstale":"2024-09-17T05:39:10.706980+0000","last_undegraded":"2024-09-17T05:39:10.706980+0000","last_fullsized":"2024-09-17T05:39:10.706980+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:30.455771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689518+0000","last_change":"2024-09-17T05:39:08.683990+0000","last_active":"2024-09-17T05:39:10.689518+0000","last_peered":"2024-09-17T05:39:10.689518+0000","last_clean":"2024-09-17T05:39:10.689518+0000","last_became_active":"2024-09-17T05:39:08.683688+0000","last_became_peered":"2024-09-17T05:39:08.683688+0000","last_unstale":"2024-09-17T05:39:10.689518+0000","last_undegraded":"2024-09-17T05:39:10.689518+0000","last_fullsized":"2024-09-17T05:39:10.689518+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:56.085990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049026+0000","last_change":"2024-09-17T05:39:09.709610+0000","last_active":"2024-09-17T05:39:12.049026+0000","last_peered":"2024-09-17T05:39:12.049026+0000","last_clean":"2024-09-17T05:39:12.049026+0000","last_became_active":"2024-09-17T05:39:09.709262+0000","last_became_peered":"2024-09-17T05:39:09.709262+0000","last_unstale":"2024-09-17T05:39:12.049026+0000","last_undegraded":"2024-09-17T05:39:12.049026+0000","last_fullsized":"2024-09-17T05:39:12.049026+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:42.431203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049602+0000","last_change":"2024-09-17T05:39:08.686070+0000","last_active":"2024-09-17T05:39:12.049602+0000","last_peered":"2024-09-17T05:39:12.049602+0000","last_clean":"2024-09-17T05:39:12.049602+0000","last_became_active":"2024-09-17T05:39:08.685786+0000","last_became_peered":"2024-09-17T05:39:08.685786+0000","last_unstale":"2024-09-17T05:39:12.049602+0000","last_undegraded":"2024-09-17T05:39:12.049602+0000","last_fullsized":"2024-09-17T05:39:12.049602+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:32.471998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050024+0000","last_change":"2024-09-17T05:39:09.704661+0000","last_active":"2024-09-17T05:39:12.050024+0000","last_peered":"2024-09-17T05:39:12.050024+0000","last_clean":"2024-09-17T05:39:12.050024+0000","last_became_active":"2024-09-17T05:39:09.704366+0000","last_became_peered":"2024-09-17T05:39:09.704366+0000","last_unstale":"2024-09-17T05:39:12.050024+0000","last_undegraded":"2024-09-17T05:39:12.050024+0000","last_fullsized":"2024-09-17T05:39:12.050024+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:35.933145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707796+0000","last_change":"2024-09-17T05:39:08.680218+0000","last_active":"2024-09-17T05:39:10.707796+0000","last_peered":"2024-09-17T05:39:10.707796+0000","last_clean":"2024-09-17T05:39:10.707796+0000","last_became_active":"2024-09-17T05:39:08.678856+0000","last_became_peered":"2024-09-17T05:39:08.678856+0000","last_unstale":"2024-09-17T05:39:10.707796+0000","last_undegraded":"2024-09-17T05:39:10.707796+0000","last_fullsized":"2024-09-17T05:39:10.707796+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.075618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685839+0000","last_change":"2024-09-17T05:39:09.689302+0000","last_active":"2024-09-17T05:39:10.685839+0000","last_peered":"2024-09-17T05:39:10.685839+0000","last_clean":"2024-09-17T05:39:10.685839+0000","last_became_active":"2024-09-17T05:39:09.689014+0000","last_became_peered":"2024-09-17T05:39:09.689014+0000","last_unstale":"2024-09-17T05:39:10.685839+0000","last_undegraded":"2024-09-17T05:39:10.685839+0000","last_fullsized":"2024-09-17T05:39:10.685839+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:47.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690647+0000","last_change":"2024-09-17T05:39:08.677955+0000","last_active":"2024-09-17T05:39:10.690647+0000","last_peered":"2024-09-17T05:39:10.690647+0000","last_clean":"2024-09-17T05:39:10.690647+0000","last_became_active":"2024-09-17T05:39:08.677694+0000","last_became_peered":"2024-09-17T05:39:08.677694+0000","last_unstale":"2024-09-17T05:39:10.690647+0000","last_undegraded":"2024-09-17T05:39:10.690647+0000","last_fullsized":"2024-09-17T05:39:10.690647+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:32:16.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707595+0000","last_change":"2024-09-17T05:39:09.688560+0000","last_active":"2024-09-17T05:39:10.707595+0000","last_peered":"2024-09-17T05:39:10.707595+0000","last_clean":"2024-09-17T05:39:10.707595+0000","last_became_active":"2024-09-17T05:39:09.688002+0000","last_became_peered":"2024-09-17T05:39:09.688002+0000","last_unstale":"2024-09-17T05:39:10.707595+0000","last_undegraded":"2024-09-17T05:39:10.707595+0000","last_fullsized":"2024-09-17T05:39:10.707595+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:27.223389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"131'3","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.686014+0000","last_change":"2024-09-17T05:39:08.672074+0000","last_active":"2024-09-17T05:39:10.686014+0000","last_peered":"2024-09-17T05:39:10.686014+0000","last_clean":"2024-09-17T05:39:10.686014+0000","last_became_active":"2024-09-17T05:39:08.671800+0000","last_became_peered":"2024-09-17T05:39:08.671800+0000","last_unstale":"2024-09-17T05:39:10.686014+0000","last_undegraded":"2024-09-17T05:39:10.686014+0000","last_fullsized":"2024-09-17T05:39:10.686014+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:54.394077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462405+0000","last_change":"2024-09-17T05:39:09.685796+0000","last_active":"2024-09-17T05:39:11.462405+0000","last_peered":"2024-09-17T05:39:11.462405+0000","last_clean":"2024-09-17T05:39:11.462405+0000","last_became_active":"2024-09-17T05:39:09.683964+0000","last_became_peered":"2024-09-17T05:39:09.683964+0000","last_unstale":"2024-09-17T05:39:11.462405+0000","last_undegraded":"2024-09-17T05:39:11.462405+0000","last_fullsized":"2024-09-17T05:39:11.462405+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:24.662471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"131'6","reported_seq":21,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462212+0000","last_change":"2024-09-17T05:39:08.686779+0000","last_active":"2024-09-17T05:39:11.462212+0000","last_peered":"2024-09-17T05:39:11.462212+0000","last_clean":"2024-09-17T05:39:11.462212+0000","last_became_active":"2024-09-17T05:39:08.686086+0000","last_became_peered":"2024-09-17T05:39:08.686086+0000","last_unstale":"2024-09-17T05:39:11.462212+0000","last_undegraded":"2024-09-17T05:39:11.462212+0000","last_fullsized":"2024-09-17T05:39:11.462212+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:46.349152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.707208+0000","last_change":"2024-09-17T05:39:09.702938+0000","last_active":"2024-09-17T05:39:09.707208+0000","last_peered":"2024-09-17T05:39:09.707208+0000","last_clean":"2024-09-17T05:39:09.707208+0000","last_became_active":"2024-09-17T05:39:09.702616+0000","last_became_peered":"2024-09-17T05:39:09.702616+0000","last_unstale":"2024-09-17T05:39:09.707208+0000","last_undegraded":"2024-09-17T05:39:09.707208+0000","last_fullsized":"2024-09-17T05:39:09.707208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:30.455771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683424+0000","last_change":"2024-09-17T05:39:08.674569+0000","last_active":"2024-09-17T05:39:09.683424+0000","last_peered":"2024-09-17T05:39:09.683424+0000","last_clean":"2024-09-17T05:39:09.683424+0000","last_became_active":"2024-09-17T05:39:08.674285+0000","last_became_peered":"2024-09-17T05:39:08.674285+0000","last_unstale":"2024-09-17T05:39:09.683424+0000","last_undegraded":"2024-09-17T05:39:09.683424+0000","last_fullsized":"2024-09-17T05:39:09.683424+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:13.783924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702018+0000","last_change":"2024-09-17T05:39:09.698643+0000","last_active":"2024-09-17T05:39:09.702018+0000","last_peered":"2024-09-17T05:39:09.702018+0000","last_clean":"2024-09-17T05:39:09.702018+0000","last_became_active":"2024-09-17T05:39:09.698013+0000","last_became_peered":"2024-09-17T05:39:09.698013+0000","last_unstale":"2024-09-17T05:39:09.702018+0000","last_undegraded":"2024-09-17T05:39:09.702018+0000","last_fullsized":"2024-09-17T05:39:09.702018+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:42.265017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049765+0000","last_change":"2024-09-17T05:39:08.684608+0000","last_active":"2024-09-17T05:39:12.049765+0000","last_peered":"2024-09-17T05:39:12.049765+0000","last_clean":"2024-09-17T05:39:12.049765+0000","last_became_active":"2024-09-17T05:39:08.684343+0000","last_became_peered":"2024-09-17T05:39:08.684343+0000","last_unstale":"2024-09-17T05:39:12.049765+0000","last_undegraded":"2024-09-17T05:39:12.049765+0000","last_fullsized":"2024-09-17T05:39:12.049765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:40.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.706960+0000","last_change":"2024-09-17T05:39:09.686193+0000","last_active":"2024-09-17T05:39:10.706960+0000","last_peered":"2024-09-17T05:39:10.706960+0000","last_clean":"2024-09-17T05:39:10.706960+0000","last_became_active":"2024-09-17T05:39:09.685938+0000","last_became_peered":"2024-09-17T05:39:09.685938+0000","last_unstale":"2024-09-17T05:39:10.706960+0000","last_undegraded":"2024-09-17T05:39:10.706960+0000","last_fullsized":"2024-09-17T05:39:10.706960+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:00.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":23,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.706921+0000","last_change":"2024-09-17T05:39:08.682503+0000","last_active":"2024-09-17T05:39:10.706921+0000","last_peered":"2024-09-17T05:39:10.706921+0000","last_clean":"2024-09-17T05:39:10.706921+0000","last_became_active":"2024-09-17T05:39:08.682225+0000","last_became_peered":"2024-09-17T05:39:08.682225+0000","last_unstale":"2024-09-17T05:39:10.706921+0000","last_undegraded":"2024-09-17T05:39:10.706921+0000","last_fullsized":"2024-09-17T05:39:10.706921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:49.204188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685806+0000","last_change":"2024-09-17T05:39:09.686773+0000","last_active":"2024-09-17T05:39:10.685806+0000","last_peered":"2024-09-17T05:39:10.685806+0000","last_clean":"2024-09-17T05:39:10.685806+0000","last_became_active":"2024-09-17T05:39:09.686467+0000","last_became_peered":"2024-09-17T05:39:09.686467+0000","last_unstale":"2024-09-17T05:39:10.685806+0000","last_undegraded":"2024-09-17T05:39:10.685806+0000","last_fullsized":"2024-09-17T05:39:10.685806+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:54.958914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707841+0000","last_change":"2024-09-17T05:39:08.685708+0000","last_active":"2024-09-17T05:39:10.707841+0000","last_peered":"2024-09-17T05:39:10.707841+0000","last_clean":"2024-09-17T05:39:10.707841+0000","last_became_active":"2024-09-17T05:39:08.685380+0000","last_became_peered":"2024-09-17T05:39:08.685380+0000","last_unstale":"2024-09-17T05:39:10.707841+0000","last_undegraded":"2024-09-17T05:39:10.707841+0000","last_fullsized":"2024-09-17T05:39:10.707841+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:52.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685862+0000","last_change":"2024-09-17T05:39:09.699827+0000","last_active":"2024-09-17T05:39:10.685862+0000","last_peered":"2024-09-17T05:39:10.685862+0000","last_clean":"2024-09-17T05:39:10.685862+0000","last_became_active":"2024-09-17T05:39:09.699467+0000","last_became_peered":"2024-09-17T05:39:09.699467+0000","last_unstale":"2024-09-17T05:39:10.685862+0000","last_undegraded":"2024-09-17T05:39:10.685862+0000","last_fullsized":"2024-09-17T05:39:10.685862+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:46.360228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049750+0000","last_change":"2024-09-17T05:39:08.679962+0000","last_active":"2024-09-17T05:39:12.049750+0000","last_peered":"2024-09-17T05:39:12.049750+0000","last_clean":"2024-09-17T05:39:12.049750+0000","last_became_active":"2024-09-17T05:39:08.679323+0000","last_became_peered":"2024-09-17T05:39:08.679323+0000","last_unstale":"2024-09-17T05:39:12.049750+0000","last_undegraded":"2024-09-17T05:39:12.049750+0000","last_fullsized":"2024-09-17T05:39:12.049750+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:40.800908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049566+0000","last_change":"2024-09-17T05:39:09.705201+0000","last_active":"2024-09-17T05:39:12.049566+0000","last_peered":"2024-09-17T05:39:12.049566+0000","last_clean":"2024-09-17T05:39:12.049566+0000","last_became_active":"2024-09-17T05:39:09.704923+0000","last_became_peered":"2024-09-17T05:39:09.704923+0000","last_unstale":"2024-09-17T05:39:12.049566+0000","last_undegraded":"2024-09-17T05:39:12.049566+0000","last_fullsized":"2024-09-17T05:39:12.049566+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:01.470180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689915+0000","last_change":"2024-09-17T05:39:08.679584+0000","last_active":"2024-09-17T05:39:10.689915+0000","last_peered":"2024-09-17T05:39:10.689915+0000","last_clean":"2024-09-17T05:39:10.689915+0000","last_became_active":"2024-09-17T05:39:08.679238+0000","last_became_peered":"2024-09-17T05:39:08.679238+0000","last_unstale":"2024-09-17T05:39:10.689915+0000","last_undegraded":"2024-09-17T05:39:10.689915+0000","last_fullsized":"2024-09-17T05:39:10.689915+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:07.052008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049349+0000","last_change":"2024-09-17T05:39:09.709707+0000","last_active":"2024-09-17T05:39:12.049349+0000","last_peered":"2024-09-17T05:39:12.049349+0000","last_clean":"2024-09-17T05:39:12.049349+0000","last_became_active":"2024-09-17T05:39:09.709396+0000","last_became_peered":"2024-09-17T05:39:09.709396+0000","last_unstale":"2024-09-17T05:39:12.049349+0000","last_undegraded":"2024-09-17T05:39:12.049349+0000","last_fullsized":"2024-09-17T05:39:12.049349+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:27.223389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685726+0000","last_change":"2024-09-17T05:39:08.683294+0000","last_active":"2024-09-17T05:39:10.685726+0000","last_peered":"2024-09-17T05:39:10.685726+0000","last_clean":"2024-09-17T05:39:10.685726+0000","last_became_active":"2024-09-17T05:39:08.683029+0000","last_became_peered":"2024-09-17T05:39:08.683029+0000","last_unstale":"2024-09-17T05:39:10.685726+0000","last_undegraded":"2024-09-17T05:39:10.685726+0000","last_fullsized":"2024-09-17T05:39:10.685726+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.075618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050111+0000","last_change":"2024-09-17T05:39:09.705343+0000","last_active":"2024-09-17T05:39:12.050111+0000","last_peered":"2024-09-17T05:39:12.050111+0000","last_clean":"2024-09-17T05:39:12.050111+0000","last_became_active":"2024-09-17T05:39:09.702948+0000","last_became_peered":"2024-09-17T05:39:09.702948+0000","last_unstale":"2024-09-17T05:39:12.050111+0000","last_undegraded":"2024-09-17T05:39:12.050111+0000","last_fullsized":"2024-09-17T05:39:12.050111+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:35.515218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689623+0000","last_change":"2024-09-17T05:39:08.677212+0000","last_active":"2024-09-17T05:39:10.689623+0000","last_peered":"2024-09-17T05:39:10.689623+0000","last_clean":"2024-09-17T05:39:10.689623+0000","last_became_active":"2024-09-17T05:39:08.676885+0000","last_became_peered":"2024-09-17T05:39:08.676885+0000","last_unstale":"2024-09-17T05:39:10.689623+0000","last_undegraded":"2024-09-17T05:39:10.689623+0000","last_fullsized":"2024-09-17T05:39:10.689623+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:25.782602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462133+0000","last_change":"2024-09-17T05:39:09.696737+0000","last_active":"2024-09-17T05:39:11.462133+0000","last_peered":"2024-09-17T05:39:11.462133+0000","last_clean":"2024-09-17T05:39:11.462133+0000","last_became_active":"2024-09-17T05:39:09.696473+0000","last_became_peered":"2024-09-17T05:39:09.696473+0000","last_unstale":"2024-09-17T05:39:11.462133+0000","last_undegraded":"2024-09-17T05:39:11.462133+0000","last_fullsized":"2024-09-17T05:39:11.462133+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:15.752583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690057+0000","last_change":"2024-09-17T05:39:08.681401+0000","last_active":"2024-09-17T05:39:10.690057+0000","last_peered":"2024-09-17T05:39:10.690057+0000","last_clean":"2024-09-17T05:39:10.690057+0000","last_became_active":"2024-09-17T05:39:08.681071+0000","last_became_peered":"2024-09-17T05:39:08.681071+0000","last_unstale":"2024-09-17T05:39:10.690057+0000","last_undegraded":"2024-09-17T05:39:10.690057+0000","last_fullsized":"2024-09-17T05:39:10.690057+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:43.581246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.690838+0000","last_change":"2024-09-17T05:39:09.688991+0000","last_active":"2024-09-17T05:39:09.690838+0000","last_peered":"2024-09-17T05:39:09.690838+0000","last_clean":"2024-09-17T05:39:09.690838+0000","last_became_active":"2024-09-17T05:39:09.688714+0000","last_became_peered":"2024-09-17T05:39:09.688714+0000","last_unstale":"2024-09-17T05:39:09.690838+0000","last_undegraded":"2024-09-17T05:39:09.690838+0000","last_fullsized":"2024-09-17T05:39:09.690838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:44.618638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683352+0000","last_change":"2024-09-17T05:39:08.670676+0000","last_active":"2024-09-17T05:39:09.683352+0000","last_peered":"2024-09-17T05:39:09.683352+0000","last_clean":"2024-09-17T05:39:09.683352+0000","last_became_active":"2024-09-17T05:39:08.669918+0000","last_became_peered":"2024-09-17T05:39:08.669918+0000","last_unstale":"2024-09-17T05:39:09.683352+0000","last_undegraded":"2024-09-17T05:39:09.683352+0000","last_fullsized":"2024-09-17T05:39:09.683352+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:23.308245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.706983+0000","last_change":"2024-09-17T05:39:09.689573+0000","last_active":"2024-09-17T05:39:10.706983+0000","last_peered":"2024-09-17T05:39:10.706983+0000","last_clean":"2024-09-17T05:39:10.706983+0000","last_became_active":"2024-09-17T05:39:09.689052+0000","last_became_peered":"2024-09-17T05:39:09.689052+0000","last_unstale":"2024-09-17T05:39:10.706983+0000","last_undegraded":"2024-09-17T05:39:10.706983+0000","last_fullsized":"2024-09-17T05:39:10.706983+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:18.145532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.048692+0000","last_change":"2024-09-17T05:39:08.676843+0000","last_active":"2024-09-17T05:39:12.048692+0000","last_peered":"2024-09-17T05:39:12.048692+0000","last_clean":"2024-09-17T05:39:12.048692+0000","last_became_active":"2024-09-17T05:39:08.676520+0000","last_became_peered":"2024-09-17T05:39:08.676520+0000","last_unstale":"2024-09-17T05:39:12.048692+0000","last_undegraded":"2024-09-17T05:39:12.048692+0000","last_fullsized":"2024-09-17T05:39:12.048692+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:02.570603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462310+0000","last_change":"2024-09-17T05:39:09.684664+0000","last_active":"2024-09-17T05:39:11.462310+0000","last_peered":"2024-09-17T05:39:11.462310+0000","last_clean":"2024-09-17T05:39:11.462310+0000","last_became_active":"2024-09-17T05:39:09.684045+0000","last_became_peered":"2024-09-17T05:39:09.684045+0000","last_unstale":"2024-09-17T05:39:11.462310+0000","last_undegraded":"2024-09-17T05:39:11.462310+0000","last_fullsized":"2024-09-17T05:39:11.462310+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:12.139121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690525+0000","last_change":"2024-09-17T05:39:08.681472+0000","last_active":"2024-09-17T05:39:10.690525+0000","last_peered":"2024-09-17T05:39:10.690525+0000","last_clean":"2024-09-17T05:39:10.690525+0000","last_became_active":"2024-09-17T05:39:08.680987+0000","last_became_peered":"2024-09-17T05:39:08.680987+0000","last_unstale":"2024-09-17T05:39:10.690525+0000","last_undegraded":"2024-09-17T05:39:10.690525+0000","last_fullsized":"2024-09-17T05:39:10.690525+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:34.886192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050277+0000","last_change":"2024-09-17T05:39:09.708694+0000","last_active":"2024-09-17T05:39:12.050277+0000","last_peered":"2024-09-17T05:39:12.050277+0000","last_clean":"2024-09-17T05:39:12.050277+0000","last_became_active":"2024-09-17T05:39:09.708388+0000","last_became_peered":"2024-09-17T05:39:09.708388+0000","last_unstale":"2024-09-17T05:39:12.050277+0000","last_undegraded":"2024-09-17T05:39:12.050277+0000","last_fullsized":"2024-09-17T05:39:12.050277+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:42.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"132'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685868+0000","last_change":"2024-09-17T05:39:08.674902+0000","last_active":"2024-09-17T05:39:10.685868+0000","last_peered":"2024-09-17T05:39:10.685868+0000","last_clean":"2024-09-17T05:39:10.685868+0000","last_became_active":"2024-09-17T05:39:08.674649+0000","last_became_peered":"2024-09-17T05:39:08.674649+0000","last_unstale":"2024-09-17T05:39:10.685868+0000","last_undegraded":"2024-09-17T05:39:10.685868+0000","last_fullsized":"2024-09-17T05:39:10.685868+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:50.216452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.702767+0000","last_change":"2024-09-17T05:39:09.699303+0000","last_active":"2024-09-17T05:39:09.702767+0000","last_peered":"2024-09-17T05:39:09.702767+0000","last_clean":"2024-09-17T05:39:09.702767+0000","last_became_active":"2024-09-17T05:39:09.698962+0000","last_became_peered":"2024-09-17T05:39:09.698962+0000","last_unstale":"2024-09-17T05:39:09.702767+0000","last_undegraded":"2024-09-17T05:39:09.702767+0000","last_fullsized":"2024-09-17T05:39:09.702767+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:00.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690369+0000","last_change":"2024-09-17T05:39:08.682089+0000","last_active":"2024-09-17T05:39:10.690369+0000","last_peered":"2024-09-17T05:39:10.690369+0000","last_clean":"2024-09-17T05:39:10.690369+0000","last_became_active":"2024-09-17T05:39:08.680263+0000","last_became_peered":"2024-09-17T05:39:08.680263+0000","last_unstale":"2024-09-17T05:39:10.690369+0000","last_undegraded":"2024-09-17T05:39:10.690369+0000","last_fullsized":"2024-09-17T05:39:10.690369+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:50.752563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689742+0000","last_change":"2024-09-17T05:39:09.703937+0000","last_active":"2024-09-17T05:39:10.689742+0000","last_peered":"2024-09-17T05:39:10.689742+0000","last_clean":"2024-09-17T05:39:10.689742+0000","last_became_active":"2024-09-17T05:39:09.703643+0000","last_became_peered":"2024-09-17T05:39:09.703643+0000","last_unstale":"2024-09-17T05:39:10.689742+0000","last_undegraded":"2024-09-17T05:39:10.689742+0000","last_fullsized":"2024-09-17T05:39:10.689742+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:22.554828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.684419+0000","last_change":"2024-09-17T05:39:08.673802+0000","last_active":"2024-09-17T05:39:09.684419+0000","last_peered":"2024-09-17T05:39:09.684419+0000","last_clean":"2024-09-17T05:39:09.684419+0000","last_became_active":"2024-09-17T05:39:08.673520+0000","last_became_peered":"2024-09-17T05:39:08.673520+0000","last_unstale":"2024-09-17T05:39:09.684419+0000","last_undegraded":"2024-09-17T05:39:09.684419+0000","last_fullsized":"2024-09-17T05:39:09.684419+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:57.829343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707470+0000","last_change":"2024-09-17T05:39:09.685986+0000","last_active":"2024-09-17T05:39:10.707470+0000","last_peered":"2024-09-17T05:39:10.707470+0000","last_clean":"2024-09-17T05:39:10.707470+0000","last_became_active":"2024-09-17T05:39:09.685714+0000","last_became_peered":"2024-09-17T05:39:09.685714+0000","last_unstale":"2024-09-17T05:39:10.707470+0000","last_undegraded":"2024-09-17T05:39:10.707470+0000","last_fullsized":"2024-09-17T05:39:10.707470+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:48.869239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049963+0000","last_change":"2024-09-17T05:39:08.683259+0000","last_active":"2024-09-17T05:39:12.049963+0000","last_peered":"2024-09-17T05:39:12.049963+0000","last_clean":"2024-09-17T05:39:12.049963+0000","last_became_active":"2024-09-17T05:39:08.682951+0000","last_became_peered":"2024-09-17T05:39:08.682951+0000","last_unstale":"2024-09-17T05:39:12.049963+0000","last_undegraded":"2024-09-17T05:39:12.049963+0000","last_fullsized":"2024-09-17T05:39:12.049963+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:38:46.002792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707321+0000","last_change":"2024-09-17T05:39:09.703634+0000","last_active":"2024-09-17T05:39:10.707321+0000","last_peered":"2024-09-17T05:39:10.707321+0000","last_clean":"2024-09-17T05:39:10.707321+0000","last_became_active":"2024-09-17T05:39:09.703280+0000","last_became_peered":"2024-09-17T05:39:09.703280+0000","last_unstale":"2024-09-17T05:39:10.707321+0000","last_undegraded":"2024-09-17T05:39:10.707321+0000","last_fullsized":"2024-09-17T05:39:10.707321+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:58.688537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707473+0000","last_change":"2024-09-17T05:39:08.683416+0000","last_active":"2024-09-17T05:39:10.707473+0000","last_peered":"2024-09-17T05:39:10.707473+0000","last_clean":"2024-09-17T05:39:10.707473+0000","last_became_active":"2024-09-17T05:39:08.682597+0000","last_became_peered":"2024-09-17T05:39:08.682597+0000","last_unstale":"2024-09-17T05:39:10.707473+0000","last_undegraded":"2024-09-17T05:39:10.707473+0000","last_fullsized":"2024-09-17T05:39:10.707473+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:50.216452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689976+0000","last_change":"2024-09-17T05:39:09.690617+0000","last_active":"2024-09-17T05:39:10.689976+0000","last_peered":"2024-09-17T05:39:10.689976+0000","last_clean":"2024-09-17T05:39:10.689976+0000","last_became_active":"2024-09-17T05:39:09.690358+0000","last_became_peered":"2024-09-17T05:39:09.690358+0000","last_unstale":"2024-09-17T05:39:10.689976+0000","last_undegraded":"2024-09-17T05:39:10.689976+0000","last_fullsized":"2024-09-17T05:39:10.689976+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:34.788898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.048827+0000","last_change":"2024-09-17T05:39:08.683995+0000","last_active":"2024-09-17T05:39:12.048827+0000","last_peered":"2024-09-17T05:39:12.048827+0000","last_clean":"2024-09-17T05:39:12.048827+0000","last_became_active":"2024-09-17T05:39:08.683726+0000","last_became_peered":"2024-09-17T05:39:08.683726+0000","last_unstale":"2024-09-17T05:39:12.048827+0000","last_undegraded":"2024-09-17T05:39:12.048827+0000","last_fullsized":"2024-09-17T05:39:12.048827+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:14.301339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.693873+0000","last_change":"2024-09-17T05:39:09.690892+0000","last_active":"2024-09-17T05:39:09.693873+0000","last_peered":"2024-09-17T05:39:09.693873+0000","last_clean":"2024-09-17T05:39:09.693873+0000","last_became_active":"2024-09-17T05:39:09.690610+0000","last_became_peered":"2024-09-17T05:39:09.690610+0000","last_unstale":"2024-09-17T05:39:09.693873+0000","last_undegraded":"2024-09-17T05:39:09.693873+0000","last_fullsized":"2024-09-17T05:39:09.693873+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:16.440719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-17T05:39:09.683840+0000","last_change":"2024-09-17T05:39:08.673866+0000","last_active":"2024-09-17T05:39:09.683840+0000","last_peered":"2024-09-17T05:39:09.683840+0000","last_clean":"2024-09-17T05:39:09.683840+0000","last_became_active":"2024-09-17T05:39:08.673542+0000","last_became_peered":"2024-09-17T05:39:08.673542+0000","last_unstale":"2024-09-17T05:39:09.683840+0000","last_undegraded":"2024-09-17T05:39:09.683840+0000","last_fullsized":"2024-09-17T05:39:09.683840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:11.138555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689741+0000","last_change":"2024-09-17T05:39:09.704967+0000","last_active":"2024-09-17T05:39:10.689741+0000","last_peered":"2024-09-17T05:39:10.689741+0000","last_clean":"2024-09-17T05:39:10.689741+0000","last_became_active":"2024-09-17T05:39:09.704618+0000","last_became_peered":"2024-09-17T05:39:09.704618+0000","last_unstale":"2024-09-17T05:39:10.689741+0000","last_undegraded":"2024-09-17T05:39:10.689741+0000","last_fullsized":"2024-09-17T05:39:10.689741+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:46.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707430+0000","last_change":"2024-09-17T05:39:08.680453+0000","last_active":"2024-09-17T05:39:10.707430+0000","last_peered":"2024-09-17T05:39:10.707430+0000","last_clean":"2024-09-17T05:39:10.707430+0000","last_became_active":"2024-09-17T05:39:08.680180+0000","last_became_peered":"2024-09-17T05:39:08.680180+0000","last_unstale":"2024-09-17T05:39:10.707430+0000","last_undegraded":"2024-09-17T05:39:10.707430+0000","last_fullsized":"2024-09-17T05:39:10.707430+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:15.575940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":321,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.461968+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:39:11.461968+0000","last_peered":"2024-09-17T05:39:11.461968+0000","last_clean":"2024-09-17T05:39:11.461968+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:39:11.461968+0000","last_undegraded":"2024-09-17T05:39:11.461968+0000","last_fullsized":"2024-09-17T05:39:11.461968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509615,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127128,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244712,"statfs":{"total":96636764160,"available":96506585088,"internally_reserved":0,"allocated":1642496,"data_stored":722673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136832,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235008,"statfs":{"total":96636764160,"available":96496648192,"internally_reserved":0,"allocated":1613824,"data_stored":709396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509615,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131284,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240556,"statfs":{"total":96636764160,"available":96502329344,"internally_reserved":0,"allocated":1638400,"data_stored":722639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509614,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131292,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240548,"statfs":{"total":96636764160,"available":96502321152,"internally_reserved":0,"allocated":1642496,"data_stored":722709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509615,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127136,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244704,"statfs":{"total":96636764160,"available":96506576896,"internally_reserved":0,"allocated":1646592,"data_stored":723311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509616,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145768,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226072,"statfs":{"total":96636764160,"available":96487497728,"internally_reserved":0,"allocated":2248704,"data_stored":1314641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509614,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145200,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226640,"statfs":{"total":96636764160,"available":96488079360,"internally_reserved":0,"allocated":1658880,"data_stored":724307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":146995659},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542320,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131876,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239964,"statfs":{"total":96636764160,"available":96501723136,"internally_reserved":0,"allocated":2244608,"data_stored":1313749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:39:15.640 INFO:tasks.ceph.mds.a.smithi122.stderr:2024-09-17T05:39:15.646+0000 7f6f6c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:39:18.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:39:18.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.538+0000 7fb11ee00640 1 -- 172.21.15.122:0/3272436098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1180d01b0 msgr2=0x7fb1180cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:18.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.538+0000 7fb11ee00640 1 --2- 172.21.15.122:0/3272436098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1180d01b0 0x7fb1180cefb0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fb104009900 tx=0x7fb10401c850 comp rx=0 tx=0).stop 2024-09-17T05:39:18.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 -- 172.21.15.122:0/3272436098 shutdown_connections 2024-09-17T05:39:18.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 --2- 172.21.15.122:0/3272436098 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1180b2590 0x7fb1180b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 --2- 172.21.15.122:0/3272436098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1180d01b0 0x7fb1180cefb0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 -- 172.21.15.122:0/3272436098 >> 172.21.15.122:0/3272436098 conn(0x7fb11809aa10 msgr2=0x7fb11809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:18.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 -- 172.21.15.122:0/3272436098 shutdown_connections 2024-09-17T05:39:18.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 -- 172.21.15.122:0/3272436098 wait complete. 2024-09-17T05:39:18.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 Processor -- start 2024-09-17T05:39:18.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 -- start start 2024-09-17T05:39:18.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1180b2590 0x7fb118151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:18.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1180d01b0 0x7fb1180b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:18.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb11815f000 0x7fb1180c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:18.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb118168f00 con 0x7fb11815f000 2024-09-17T05:39:18.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb118168d80 con 0x7fb1180d01b0 2024-09-17T05:39:18.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11ee00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb118169080 con 0x7fb1180b2590 2024-09-17T05:39:18.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb11815f000 0x7fb1180c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:18.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb11815f000 0x7fb1180c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38884/0 (socket says 172.21.15.122:38884) 2024-09-17T05:39:18.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 -- 172.21.15.122:0/595241157 learned_addr learned my addr 172.21.15.122:0/595241157 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:18.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11da00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1180b2590 0x7fb118151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:18.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11d000640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1180d01b0 0x7fb1180b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:18.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 -- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1180b2590 msgr2=0x7fb118151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:18.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1180b2590 0x7fb118151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 -- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1180d01b0 msgr2=0x7fb1180b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:18.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1180d01b0 0x7fb1180b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 -- 172.21.15.122:0/595241157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1180c6f20 con 0x7fb11815f000 2024-09-17T05:39:18.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb11e400640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb11815f000 0x7fb1180c69e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb11000bc80 tx=0x7fb110009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:18.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.542+0000 7fb115600640 1 -- 172.21.15.122:0/595241157 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110011250 con 0x7fb11815f000 2024-09-17T05:39:18.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb115600640 1 -- 172.21.15.122:0/595241157 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1100113f0 con 0x7fb11815f000 2024-09-17T05:39:18.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb115600640 1 -- 172.21.15.122:0/595241157 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110005020 con 0x7fb11815f000 2024-09-17T05:39:18.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1180b5b40 con 0x7fb11815f000 2024-09-17T05:39:18.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1180b6100 con 0x7fb11815f000 2024-09-17T05:39:18.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1180b29e0 con 0x7fb11815f000 2024-09-17T05:39:18.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb115600640 1 -- 172.21.15.122:0/595241157 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fb1100051c0 con 0x7fb11815f000 2024-09-17T05:39:18.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb115600640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb0f8075180 0x7fb0f8077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:18.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb115600640 1 -- 172.21.15.122:0/595241157 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7fb1100974b0 con 0x7fb11815f000 2024-09-17T05:39:18.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.546+0000 7fb11da00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb0f8075180 0x7fb0f8077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:18.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.550+0000 7fb11da00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb0f8075180 0x7fb0f8077640 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb10c001e40 tx=0x7fb10c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:18.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.550+0000 7fb115600640 1 -- 172.21.15.122:0/595241157 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb110016070 con 0x7fb11815f000 2024-09-17T05:39:18.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.714+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1180b6730 con 0x7fb0f8075180 2024-09-17T05:39:18.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.718+0000 7fb115600640 1 -- 172.21.15.122:0/595241157 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252499 (secure 0 0 0) 0x7fb1180b6730 con 0x7fb0f8075180 2024-09-17T05:39:18.717 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:18.719 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:39:18.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb0f8075180 msgr2=0x7fb0f8077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:18.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb0f8075180 0x7fb0f8077640 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb10c001e40 tx=0x7fb10c009040 comp rx=0 tx=0).stop 2024-09-17T05:39:18.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb11815f000 msgr2=0x7fb1180c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:18.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb11815f000 0x7fb1180c69e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb11000bc80 tx=0x7fb110009610 comp rx=0 tx=0).stop 2024-09-17T05:39:18.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 shutdown_connections 2024-09-17T05:39:18.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb0f8075180 0x7fb0f8077640 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb11815f000 0x7fb1180c69e0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1180d01b0 0x7fb1180b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 --2- 172.21.15.122:0/595241157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1180b2590 0x7fb118151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 >> 172.21.15.122:0/595241157 conn(0x7fb11809aa10 msgr2=0x7fb1180c8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:18.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 shutdown_connections 2024-09-17T05:39:18.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.722+0000 7fb11ee00640 1 -- 172.21.15.122:0/595241157 wait complete. 2024-09-17T05:39:18.738 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-09-17T05:39:16.737144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1076536,"kb_used_data":14024,"kb_used_omap":12,"kb_used_meta":1062323,"kb_avail":753898184,"statfs":{"total":773094113280,"available":771991740416,"internally_reserved":0,"allocated":14360576,"data_stored":6966668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1087819318},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001989"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050018+0000","last_change":"2024-09-17T05:39:09.704682+0000","last_active":"2024-09-17T05:39:12.050018+0000","last_peered":"2024-09-17T05:39:12.050018+0000","last_clean":"2024-09-17T05:39:12.050018+0000","last_became_active":"2024-09-17T05:39:09.704406+0000","last_became_peered":"2024-09-17T05:39:09.704406+0000","last_unstale":"2024-09-17T05:39:12.050018+0000","last_undegraded":"2024-09-17T05:39:12.050018+0000","last_fullsized":"2024-09-17T05:39:12.050018+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:17.125213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049063+0000","last_change":"2024-09-17T05:39:09.701535+0000","last_active":"2024-09-17T05:39:12.049063+0000","last_peered":"2024-09-17T05:39:12.049063+0000","last_clean":"2024-09-17T05:39:12.049063+0000","last_became_active":"2024-09-17T05:39:09.700923+0000","last_became_peered":"2024-09-17T05:39:09.700923+0000","last_unstale":"2024-09-17T05:39:12.049063+0000","last_undegraded":"2024-09-17T05:39:12.049063+0000","last_fullsized":"2024-09-17T05:39:12.049063+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:36.746214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689709+0000","last_change":"2024-09-17T05:39:09.710429+0000","last_active":"2024-09-17T05:39:10.689709+0000","last_peered":"2024-09-17T05:39:10.689709+0000","last_clean":"2024-09-17T05:39:10.689709+0000","last_became_active":"2024-09-17T05:39:09.708138+0000","last_became_peered":"2024-09-17T05:39:09.708138+0000","last_unstale":"2024-09-17T05:39:10.689709+0000","last_undegraded":"2024-09-17T05:39:10.689709+0000","last_fullsized":"2024-09-17T05:39:10.689709+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:30.537275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049168+0000","last_change":"2024-09-17T05:39:09.696346+0000","last_active":"2024-09-17T05:39:12.049168+0000","last_peered":"2024-09-17T05:39:12.049168+0000","last_clean":"2024-09-17T05:39:12.049168+0000","last_became_active":"2024-09-17T05:39:09.696079+0000","last_became_peered":"2024-09-17T05:39:09.696079+0000","last_unstale":"2024-09-17T05:39:12.049168+0000","last_undegraded":"2024-09-17T05:39:12.049168+0000","last_fullsized":"2024-09-17T05:39:12.049168+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:20.246762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685522+0000","last_change":"2024-09-17T05:39:09.689996+0000","last_active":"2024-09-17T05:39:10.685522+0000","last_peered":"2024-09-17T05:39:10.685522+0000","last_clean":"2024-09-17T05:39:10.685522+0000","last_became_active":"2024-09-17T05:39:09.688528+0000","last_became_peered":"2024-09-17T05:39:09.688528+0000","last_unstale":"2024-09-17T05:39:10.685522+0000","last_undegraded":"2024-09-17T05:39:10.685522+0000","last_fullsized":"2024-09-17T05:39:10.685522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:16.440719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707109+0000","last_change":"2024-09-17T05:39:09.707106+0000","last_active":"2024-09-17T05:39:10.707109+0000","last_peered":"2024-09-17T05:39:10.707109+0000","last_clean":"2024-09-17T05:39:10.707109+0000","last_became_active":"2024-09-17T05:39:09.706836+0000","last_became_peered":"2024-09-17T05:39:09.706836+0000","last_unstale":"2024-09-17T05:39:10.707109+0000","last_undegraded":"2024-09-17T05:39:10.707109+0000","last_fullsized":"2024-09-17T05:39:10.707109+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:42.431203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685562+0000","last_change":"2024-09-17T05:39:09.698951+0000","last_active":"2024-09-17T05:39:10.685562+0000","last_peered":"2024-09-17T05:39:10.685562+0000","last_clean":"2024-09-17T05:39:10.685562+0000","last_became_active":"2024-09-17T05:39:09.698545+0000","last_became_peered":"2024-09-17T05:39:09.698545+0000","last_unstale":"2024-09-17T05:39:10.685562+0000","last_undegraded":"2024-09-17T05:39:10.685562+0000","last_fullsized":"2024-09-17T05:39:10.685562+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:10.630444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685198+0000","last_change":"2024-09-17T05:39:09.706086+0000","last_active":"2024-09-17T05:39:10.685198+0000","last_peered":"2024-09-17T05:39:10.685198+0000","last_clean":"2024-09-17T05:39:10.685198+0000","last_became_active":"2024-09-17T05:39:09.705904+0000","last_became_peered":"2024-09-17T05:39:09.705904+0000","last_unstale":"2024-09-17T05:39:10.685198+0000","last_undegraded":"2024-09-17T05:39:10.685198+0000","last_fullsized":"2024-09-17T05:39:10.685198+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:44.618638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689951+0000","last_change":"2024-09-17T05:39:09.708268+0000","last_active":"2024-09-17T05:39:10.689951+0000","last_peered":"2024-09-17T05:39:10.689951+0000","last_clean":"2024-09-17T05:39:10.689951+0000","last_became_active":"2024-09-17T05:39:09.707945+0000","last_became_peered":"2024-09-17T05:39:09.707945+0000","last_unstale":"2024-09-17T05:39:10.689951+0000","last_undegraded":"2024-09-17T05:39:10.689951+0000","last_fullsized":"2024-09-17T05:39:10.689951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:56.688630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049361+0000","last_change":"2024-09-17T05:39:09.697958+0000","last_active":"2024-09-17T05:39:12.049361+0000","last_peered":"2024-09-17T05:39:12.049361+0000","last_clean":"2024-09-17T05:39:12.049361+0000","last_became_active":"2024-09-17T05:39:09.697649+0000","last_became_peered":"2024-09-17T05:39:09.697649+0000","last_unstale":"2024-09-17T05:39:12.049361+0000","last_undegraded":"2024-09-17T05:39:12.049361+0000","last_fullsized":"2024-09-17T05:39:12.049361+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:37.853686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049467+0000","last_change":"2024-09-17T05:39:09.701372+0000","last_active":"2024-09-17T05:39:12.049467+0000","last_peered":"2024-09-17T05:39:12.049467+0000","last_clean":"2024-09-17T05:39:12.049467+0000","last_became_active":"2024-09-17T05:39:09.700339+0000","last_became_peered":"2024-09-17T05:39:09.700339+0000","last_unstale":"2024-09-17T05:39:12.049467+0000","last_undegraded":"2024-09-17T05:39:12.049467+0000","last_fullsized":"2024-09-17T05:39:12.049467+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:44.935564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707456+0000","last_change":"2024-09-17T05:39:09.688279+0000","last_active":"2024-09-17T05:39:10.707456+0000","last_peered":"2024-09-17T05:39:10.707456+0000","last_clean":"2024-09-17T05:39:10.707456+0000","last_became_active":"2024-09-17T05:39:09.687637+0000","last_became_peered":"2024-09-17T05:39:09.687637+0000","last_unstale":"2024-09-17T05:39:10.707456+0000","last_undegraded":"2024-09-17T05:39:10.707456+0000","last_fullsized":"2024-09-17T05:39:10.707456+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:36.746214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049429+0000","last_change":"2024-09-17T05:39:09.702005+0000","last_active":"2024-09-17T05:39:12.049429+0000","last_peered":"2024-09-17T05:39:12.049429+0000","last_clean":"2024-09-17T05:39:12.049429+0000","last_became_active":"2024-09-17T05:39:09.701415+0000","last_became_peered":"2024-09-17T05:39:09.701415+0000","last_unstale":"2024-09-17T05:39:12.049429+0000","last_undegraded":"2024-09-17T05:39:12.049429+0000","last_fullsized":"2024-09-17T05:39:12.049429+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:38.585539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451321+0000","last_change":"2024-09-17T05:39:09.692704+0000","last_active":"2024-09-17T05:39:11.451321+0000","last_peered":"2024-09-17T05:39:11.451321+0000","last_clean":"2024-09-17T05:39:11.451321+0000","last_became_active":"2024-09-17T05:39:09.690032+0000","last_became_peered":"2024-09-17T05:39:09.690032+0000","last_unstale":"2024-09-17T05:39:11.451321+0000","last_undegraded":"2024-09-17T05:39:11.451321+0000","last_fullsized":"2024-09-17T05:39:11.451321+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:15.709405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689821+0000","last_change":"2024-09-17T05:39:09.687863+0000","last_active":"2024-09-17T05:39:10.689821+0000","last_peered":"2024-09-17T05:39:10.689821+0000","last_clean":"2024-09-17T05:39:10.689821+0000","last_became_active":"2024-09-17T05:39:09.687453+0000","last_became_peered":"2024-09-17T05:39:09.687453+0000","last_unstale":"2024-09-17T05:39:10.689821+0000","last_undegraded":"2024-09-17T05:39:10.689821+0000","last_fullsized":"2024-09-17T05:39:10.689821+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:44.125738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451141+0000","last_change":"2024-09-17T05:39:09.698973+0000","last_active":"2024-09-17T05:39:11.451141+0000","last_peered":"2024-09-17T05:39:11.451141+0000","last_clean":"2024-09-17T05:39:11.451141+0000","last_became_active":"2024-09-17T05:39:09.698375+0000","last_became_peered":"2024-09-17T05:39:09.698375+0000","last_unstale":"2024-09-17T05:39:11.451141+0000","last_undegraded":"2024-09-17T05:39:11.451141+0000","last_fullsized":"2024-09-17T05:39:11.451141+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:12.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690254+0000","last_change":"2024-09-17T05:39:09.705660+0000","last_active":"2024-09-17T05:39:10.690254+0000","last_peered":"2024-09-17T05:39:10.690254+0000","last_clean":"2024-09-17T05:39:10.690254+0000","last_became_active":"2024-09-17T05:39:09.705412+0000","last_became_peered":"2024-09-17T05:39:09.705412+0000","last_unstale":"2024-09-17T05:39:10.690254+0000","last_undegraded":"2024-09-17T05:39:10.690254+0000","last_fullsized":"2024-09-17T05:39:10.690254+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:23.159202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690473+0000","last_change":"2024-09-17T05:39:09.697274+0000","last_active":"2024-09-17T05:39:10.690473+0000","last_peered":"2024-09-17T05:39:10.690473+0000","last_clean":"2024-09-17T05:39:10.690473+0000","last_became_active":"2024-09-17T05:39:09.696899+0000","last_became_peered":"2024-09-17T05:39:09.696899+0000","last_unstale":"2024-09-17T05:39:10.690473+0000","last_undegraded":"2024-09-17T05:39:10.690473+0000","last_fullsized":"2024-09-17T05:39:10.690473+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:33.478612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049039+0000","last_change":"2024-09-17T05:39:09.704245+0000","last_active":"2024-09-17T05:39:12.049039+0000","last_peered":"2024-09-17T05:39:12.049039+0000","last_clean":"2024-09-17T05:39:12.049039+0000","last_became_active":"2024-09-17T05:39:09.703894+0000","last_became_peered":"2024-09-17T05:39:09.703894+0000","last_unstale":"2024-09-17T05:39:12.049039+0000","last_undegraded":"2024-09-17T05:39:12.049039+0000","last_fullsized":"2024-09-17T05:39:12.049039+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:15.709405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689908+0000","last_change":"2024-09-17T05:39:09.699326+0000","last_active":"2024-09-17T05:39:10.689908+0000","last_peered":"2024-09-17T05:39:10.689908+0000","last_clean":"2024-09-17T05:39:10.689908+0000","last_became_active":"2024-09-17T05:39:09.697050+0000","last_became_peered":"2024-09-17T05:39:09.697050+0000","last_unstale":"2024-09-17T05:39:10.689908+0000","last_undegraded":"2024-09-17T05:39:10.689908+0000","last_fullsized":"2024-09-17T05:39:10.689908+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:51.190185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707114+0000","last_change":"2024-09-17T05:39:09.687255+0000","last_active":"2024-09-17T05:39:10.707114+0000","last_peered":"2024-09-17T05:39:10.707114+0000","last_clean":"2024-09-17T05:39:10.707114+0000","last_became_active":"2024-09-17T05:39:09.686837+0000","last_became_peered":"2024-09-17T05:39:09.686837+0000","last_unstale":"2024-09-17T05:39:10.707114+0000","last_undegraded":"2024-09-17T05:39:10.707114+0000","last_fullsized":"2024-09-17T05:39:10.707114+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:34.045368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049149+0000","last_change":"2024-09-17T05:39:09.706086+0000","last_active":"2024-09-17T05:39:12.049149+0000","last_peered":"2024-09-17T05:39:12.049149+0000","last_clean":"2024-09-17T05:39:12.049149+0000","last_became_active":"2024-09-17T05:39:09.705785+0000","last_became_peered":"2024-09-17T05:39:09.705785+0000","last_unstale":"2024-09-17T05:39:12.049149+0000","last_undegraded":"2024-09-17T05:39:12.049149+0000","last_fullsized":"2024-09-17T05:39:12.049149+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:27.337857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462125+0000","last_change":"2024-09-17T05:39:09.684994+0000","last_active":"2024-09-17T05:39:11.462125+0000","last_peered":"2024-09-17T05:39:11.462125+0000","last_clean":"2024-09-17T05:39:11.462125+0000","last_became_active":"2024-09-17T05:39:09.684689+0000","last_became_peered":"2024-09-17T05:39:09.684689+0000","last_unstale":"2024-09-17T05:39:11.462125+0000","last_undegraded":"2024-09-17T05:39:11.462125+0000","last_fullsized":"2024-09-17T05:39:11.462125+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:59.788891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689711+0000","last_change":"2024-09-17T05:39:09.690410+0000","last_active":"2024-09-17T05:39:10.689711+0000","last_peered":"2024-09-17T05:39:10.689711+0000","last_clean":"2024-09-17T05:39:10.689711+0000","last_became_active":"2024-09-17T05:39:09.690127+0000","last_became_peered":"2024-09-17T05:39:09.690127+0000","last_unstale":"2024-09-17T05:39:10.689711+0000","last_undegraded":"2024-09-17T05:39:10.689711+0000","last_fullsized":"2024-09-17T05:39:10.689711+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:24.189535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050144+0000","last_change":"2024-09-17T05:39:09.700429+0000","last_active":"2024-09-17T05:39:12.050144+0000","last_peered":"2024-09-17T05:39:12.050144+0000","last_clean":"2024-09-17T05:39:12.050144+0000","last_became_active":"2024-09-17T05:39:09.700176+0000","last_became_peered":"2024-09-17T05:39:09.700176+0000","last_unstale":"2024-09-17T05:39:12.050144+0000","last_undegraded":"2024-09-17T05:39:12.050144+0000","last_fullsized":"2024-09-17T05:39:12.050144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:21.486551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049774+0000","last_change":"2024-09-17T05:39:09.701218+0000","last_active":"2024-09-17T05:39:12.049774+0000","last_peered":"2024-09-17T05:39:12.049774+0000","last_clean":"2024-09-17T05:39:12.049774+0000","last_became_active":"2024-09-17T05:39:09.700602+0000","last_became_peered":"2024-09-17T05:39:09.700602+0000","last_unstale":"2024-09-17T05:39:12.049774+0000","last_undegraded":"2024-09-17T05:39:12.049774+0000","last_fullsized":"2024-09-17T05:39:12.049774+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:12:31.026302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451793+0000","last_change":"2024-09-17T05:39:09.693510+0000","last_active":"2024-09-17T05:39:11.451793+0000","last_peered":"2024-09-17T05:39:11.451793+0000","last_clean":"2024-09-17T05:39:11.451793+0000","last_became_active":"2024-09-17T05:39:09.693223+0000","last_became_peered":"2024-09-17T05:39:09.693223+0000","last_unstale":"2024-09-17T05:39:11.451793+0000","last_undegraded":"2024-09-17T05:39:11.451793+0000","last_fullsized":"2024-09-17T05:39:11.451793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:29.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462230+0000","last_change":"2024-09-17T05:39:09.687403+0000","last_active":"2024-09-17T05:39:11.462230+0000","last_peered":"2024-09-17T05:39:11.462230+0000","last_clean":"2024-09-17T05:39:11.462230+0000","last_became_active":"2024-09-17T05:39:09.687119+0000","last_became_peered":"2024-09-17T05:39:09.687119+0000","last_unstale":"2024-09-17T05:39:11.462230+0000","last_undegraded":"2024-09-17T05:39:11.462230+0000","last_fullsized":"2024-09-17T05:39:11.462230+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:31.710346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049072+0000","last_change":"2024-09-17T05:39:09.703494+0000","last_active":"2024-09-17T05:39:12.049072+0000","last_peered":"2024-09-17T05:39:12.049072+0000","last_clean":"2024-09-17T05:39:12.049072+0000","last_became_active":"2024-09-17T05:39:09.703179+0000","last_became_peered":"2024-09-17T05:39:09.703179+0000","last_unstale":"2024-09-17T05:39:12.049072+0000","last_undegraded":"2024-09-17T05:39:12.049072+0000","last_fullsized":"2024-09-17T05:39:12.049072+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:48.869239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049892+0000","last_change":"2024-09-17T05:39:09.699512+0000","last_active":"2024-09-17T05:39:12.049892+0000","last_peered":"2024-09-17T05:39:12.049892+0000","last_clean":"2024-09-17T05:39:12.049892+0000","last_became_active":"2024-09-17T05:39:09.699240+0000","last_became_peered":"2024-09-17T05:39:09.699240+0000","last_unstale":"2024-09-17T05:39:12.049892+0000","last_undegraded":"2024-09-17T05:39:12.049892+0000","last_fullsized":"2024-09-17T05:39:12.049892+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:06.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462321+0000","last_change":"2024-09-17T05:39:09.683586+0000","last_active":"2024-09-17T05:39:11.462321+0000","last_peered":"2024-09-17T05:39:11.462321+0000","last_clean":"2024-09-17T05:39:11.462321+0000","last_became_active":"2024-09-17T05:39:09.683275+0000","last_became_peered":"2024-09-17T05:39:09.683275+0000","last_unstale":"2024-09-17T05:39:11.462321+0000","last_undegraded":"2024-09-17T05:39:11.462321+0000","last_fullsized":"2024-09-17T05:39:11.462321+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:09.051635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690081+0000","last_change":"2024-09-17T05:39:09.706818+0000","last_active":"2024-09-17T05:39:10.690081+0000","last_peered":"2024-09-17T05:39:10.690081+0000","last_clean":"2024-09-17T05:39:10.690081+0000","last_became_active":"2024-09-17T05:39:09.705441+0000","last_became_peered":"2024-09-17T05:39:09.705441+0000","last_unstale":"2024-09-17T05:39:10.690081+0000","last_undegraded":"2024-09-17T05:39:10.690081+0000","last_fullsized":"2024-09-17T05:39:10.690081+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:37.448980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451761+0000","last_change":"2024-09-17T05:39:09.698329+0000","last_active":"2024-09-17T05:39:11.451761+0000","last_peered":"2024-09-17T05:39:11.451761+0000","last_clean":"2024-09-17T05:39:11.451761+0000","last_became_active":"2024-09-17T05:39:09.698084+0000","last_became_peered":"2024-09-17T05:39:09.698084+0000","last_unstale":"2024-09-17T05:39:11.451761+0000","last_undegraded":"2024-09-17T05:39:11.451761+0000","last_fullsized":"2024-09-17T05:39:11.451761+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:37.853686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"131'4","reported_seq":19,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.686013+0000","last_change":"2024-09-17T05:39:08.676570+0000","last_active":"2024-09-17T05:39:10.686013+0000","last_peered":"2024-09-17T05:39:10.686013+0000","last_clean":"2024-09-17T05:39:10.686013+0000","last_became_active":"2024-09-17T05:39:08.676332+0000","last_became_peered":"2024-09-17T05:39:08.676332+0000","last_unstale":"2024-09-17T05:39:10.686013+0000","last_undegraded":"2024-09-17T05:39:10.686013+0000","last_fullsized":"2024-09-17T05:39:10.686013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:52.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690193+0000","last_change":"2024-09-17T05:39:09.704965+0000","last_active":"2024-09-17T05:39:10.690193+0000","last_peered":"2024-09-17T05:39:10.690193+0000","last_clean":"2024-09-17T05:39:10.690193+0000","last_became_active":"2024-09-17T05:39:09.704735+0000","last_became_peered":"2024-09-17T05:39:09.704735+0000","last_unstale":"2024-09-17T05:39:10.690193+0000","last_undegraded":"2024-09-17T05:39:10.690193+0000","last_fullsized":"2024-09-17T05:39:10.690193+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:31.999643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689863+0000","last_change":"2024-09-17T05:39:08.681409+0000","last_active":"2024-09-17T05:39:10.689863+0000","last_peered":"2024-09-17T05:39:10.689863+0000","last_clean":"2024-09-17T05:39:10.689863+0000","last_became_active":"2024-09-17T05:39:08.681149+0000","last_became_peered":"2024-09-17T05:39:08.681149+0000","last_unstale":"2024-09-17T05:39:10.689863+0000","last_undegraded":"2024-09-17T05:39:10.689863+0000","last_fullsized":"2024-09-17T05:39:10.689863+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:50.068381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689662+0000","last_change":"2024-09-17T05:39:09.699757+0000","last_active":"2024-09-17T05:39:10.689662+0000","last_peered":"2024-09-17T05:39:10.689662+0000","last_clean":"2024-09-17T05:39:10.689662+0000","last_became_active":"2024-09-17T05:39:09.699479+0000","last_became_peered":"2024-09-17T05:39:09.699479+0000","last_unstale":"2024-09-17T05:39:10.689662+0000","last_undegraded":"2024-09-17T05:39:10.689662+0000","last_fullsized":"2024-09-17T05:39:10.689662+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:19:42.172686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049530+0000","last_change":"2024-09-17T05:39:08.673698+0000","last_active":"2024-09-17T05:39:12.049530+0000","last_peered":"2024-09-17T05:39:12.049530+0000","last_clean":"2024-09-17T05:39:12.049530+0000","last_became_active":"2024-09-17T05:39:08.673385+0000","last_became_peered":"2024-09-17T05:39:08.673385+0000","last_unstale":"2024-09-17T05:39:12.049530+0000","last_undegraded":"2024-09-17T05:39:12.049530+0000","last_fullsized":"2024-09-17T05:39:12.049530+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:56.140398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707776+0000","last_change":"2024-09-17T05:39:09.688155+0000","last_active":"2024-09-17T05:39:10.707776+0000","last_peered":"2024-09-17T05:39:10.707776+0000","last_clean":"2024-09-17T05:39:10.707776+0000","last_became_active":"2024-09-17T05:39:09.687764+0000","last_became_peered":"2024-09-17T05:39:09.687764+0000","last_unstale":"2024-09-17T05:39:10.707776+0000","last_undegraded":"2024-09-17T05:39:10.707776+0000","last_fullsized":"2024-09-17T05:39:10.707776+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:07.017629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049638+0000","last_change":"2024-09-17T05:39:08.680638+0000","last_active":"2024-09-17T05:39:12.049638+0000","last_peered":"2024-09-17T05:39:12.049638+0000","last_clean":"2024-09-17T05:39:12.049638+0000","last_became_active":"2024-09-17T05:39:08.680162+0000","last_became_peered":"2024-09-17T05:39:08.680162+0000","last_unstale":"2024-09-17T05:39:12.049638+0000","last_undegraded":"2024-09-17T05:39:12.049638+0000","last_fullsized":"2024-09-17T05:39:12.049638+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:24.006992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690065+0000","last_change":"2024-09-17T05:39:09.690019+0000","last_active":"2024-09-17T05:39:10.690065+0000","last_peered":"2024-09-17T05:39:10.690065+0000","last_clean":"2024-09-17T05:39:10.690065+0000","last_became_active":"2024-09-17T05:39:09.689586+0000","last_became_peered":"2024-09-17T05:39:09.689586+0000","last_unstale":"2024-09-17T05:39:10.690065+0000","last_undegraded":"2024-09-17T05:39:10.690065+0000","last_fullsized":"2024-09-17T05:39:10.690065+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:14.278198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049812+0000","last_change":"2024-09-17T05:39:08.679285+0000","last_active":"2024-09-17T05:39:12.049812+0000","last_peered":"2024-09-17T05:39:12.049812+0000","last_clean":"2024-09-17T05:39:12.049812+0000","last_became_active":"2024-09-17T05:39:08.679008+0000","last_became_peered":"2024-09-17T05:39:08.679008+0000","last_unstale":"2024-09-17T05:39:12.049812+0000","last_undegraded":"2024-09-17T05:39:12.049812+0000","last_fullsized":"2024-09-17T05:39:12.049812+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:13.783924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707215+0000","last_change":"2024-09-17T05:39:09.692770+0000","last_active":"2024-09-17T05:39:10.707215+0000","last_peered":"2024-09-17T05:39:10.707215+0000","last_clean":"2024-09-17T05:39:10.707215+0000","last_became_active":"2024-09-17T05:39:09.690591+0000","last_became_peered":"2024-09-17T05:39:09.690591+0000","last_unstale":"2024-09-17T05:39:10.707215+0000","last_undegraded":"2024-09-17T05:39:10.707215+0000","last_fullsized":"2024-09-17T05:39:10.707215+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:09.574102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689932+0000","last_change":"2024-09-17T05:39:08.682028+0000","last_active":"2024-09-17T05:39:10.689932+0000","last_peered":"2024-09-17T05:39:10.689932+0000","last_clean":"2024-09-17T05:39:10.689932+0000","last_became_active":"2024-09-17T05:39:08.680621+0000","last_became_peered":"2024-09-17T05:39:08.680621+0000","last_unstale":"2024-09-17T05:39:10.689932+0000","last_undegraded":"2024-09-17T05:39:10.689932+0000","last_fullsized":"2024-09-17T05:39:10.689932+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:26.453145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.706980+0000","last_change":"2024-09-17T05:39:09.686811+0000","last_active":"2024-09-17T05:39:10.706980+0000","last_peered":"2024-09-17T05:39:10.706980+0000","last_clean":"2024-09-17T05:39:10.706980+0000","last_became_active":"2024-09-17T05:39:09.685430+0000","last_became_peered":"2024-09-17T05:39:09.685430+0000","last_unstale":"2024-09-17T05:39:10.706980+0000","last_undegraded":"2024-09-17T05:39:10.706980+0000","last_fullsized":"2024-09-17T05:39:10.706980+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:30.455771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689518+0000","last_change":"2024-09-17T05:39:08.683990+0000","last_active":"2024-09-17T05:39:10.689518+0000","last_peered":"2024-09-17T05:39:10.689518+0000","last_clean":"2024-09-17T05:39:10.689518+0000","last_became_active":"2024-09-17T05:39:08.683688+0000","last_became_peered":"2024-09-17T05:39:08.683688+0000","last_unstale":"2024-09-17T05:39:10.689518+0000","last_undegraded":"2024-09-17T05:39:10.689518+0000","last_fullsized":"2024-09-17T05:39:10.689518+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:56.085990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049026+0000","last_change":"2024-09-17T05:39:09.709610+0000","last_active":"2024-09-17T05:39:12.049026+0000","last_peered":"2024-09-17T05:39:12.049026+0000","last_clean":"2024-09-17T05:39:12.049026+0000","last_became_active":"2024-09-17T05:39:09.709262+0000","last_became_peered":"2024-09-17T05:39:09.709262+0000","last_unstale":"2024-09-17T05:39:12.049026+0000","last_undegraded":"2024-09-17T05:39:12.049026+0000","last_fullsized":"2024-09-17T05:39:12.049026+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:42.431203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049602+0000","last_change":"2024-09-17T05:39:08.686070+0000","last_active":"2024-09-17T05:39:12.049602+0000","last_peered":"2024-09-17T05:39:12.049602+0000","last_clean":"2024-09-17T05:39:12.049602+0000","last_became_active":"2024-09-17T05:39:08.685786+0000","last_became_peered":"2024-09-17T05:39:08.685786+0000","last_unstale":"2024-09-17T05:39:12.049602+0000","last_undegraded":"2024-09-17T05:39:12.049602+0000","last_fullsized":"2024-09-17T05:39:12.049602+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:32.471998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050024+0000","last_change":"2024-09-17T05:39:09.704661+0000","last_active":"2024-09-17T05:39:12.050024+0000","last_peered":"2024-09-17T05:39:12.050024+0000","last_clean":"2024-09-17T05:39:12.050024+0000","last_became_active":"2024-09-17T05:39:09.704366+0000","last_became_peered":"2024-09-17T05:39:09.704366+0000","last_unstale":"2024-09-17T05:39:12.050024+0000","last_undegraded":"2024-09-17T05:39:12.050024+0000","last_fullsized":"2024-09-17T05:39:12.050024+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:35.933145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707796+0000","last_change":"2024-09-17T05:39:08.680218+0000","last_active":"2024-09-17T05:39:10.707796+0000","last_peered":"2024-09-17T05:39:10.707796+0000","last_clean":"2024-09-17T05:39:10.707796+0000","last_became_active":"2024-09-17T05:39:08.678856+0000","last_became_peered":"2024-09-17T05:39:08.678856+0000","last_unstale":"2024-09-17T05:39:10.707796+0000","last_undegraded":"2024-09-17T05:39:10.707796+0000","last_fullsized":"2024-09-17T05:39:10.707796+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.075618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685839+0000","last_change":"2024-09-17T05:39:09.689302+0000","last_active":"2024-09-17T05:39:10.685839+0000","last_peered":"2024-09-17T05:39:10.685839+0000","last_clean":"2024-09-17T05:39:10.685839+0000","last_became_active":"2024-09-17T05:39:09.689014+0000","last_became_peered":"2024-09-17T05:39:09.689014+0000","last_unstale":"2024-09-17T05:39:10.685839+0000","last_undegraded":"2024-09-17T05:39:10.685839+0000","last_fullsized":"2024-09-17T05:39:10.685839+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:47.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690647+0000","last_change":"2024-09-17T05:39:08.677955+0000","last_active":"2024-09-17T05:39:10.690647+0000","last_peered":"2024-09-17T05:39:10.690647+0000","last_clean":"2024-09-17T05:39:10.690647+0000","last_became_active":"2024-09-17T05:39:08.677694+0000","last_became_peered":"2024-09-17T05:39:08.677694+0000","last_unstale":"2024-09-17T05:39:10.690647+0000","last_undegraded":"2024-09-17T05:39:10.690647+0000","last_fullsized":"2024-09-17T05:39:10.690647+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:32:16.221160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707595+0000","last_change":"2024-09-17T05:39:09.688560+0000","last_active":"2024-09-17T05:39:10.707595+0000","last_peered":"2024-09-17T05:39:10.707595+0000","last_clean":"2024-09-17T05:39:10.707595+0000","last_became_active":"2024-09-17T05:39:09.688002+0000","last_became_peered":"2024-09-17T05:39:09.688002+0000","last_unstale":"2024-09-17T05:39:10.707595+0000","last_undegraded":"2024-09-17T05:39:10.707595+0000","last_fullsized":"2024-09-17T05:39:10.707595+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:27.223389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"131'3","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.686014+0000","last_change":"2024-09-17T05:39:08.672074+0000","last_active":"2024-09-17T05:39:10.686014+0000","last_peered":"2024-09-17T05:39:10.686014+0000","last_clean":"2024-09-17T05:39:10.686014+0000","last_became_active":"2024-09-17T05:39:08.671800+0000","last_became_peered":"2024-09-17T05:39:08.671800+0000","last_unstale":"2024-09-17T05:39:10.686014+0000","last_undegraded":"2024-09-17T05:39:10.686014+0000","last_fullsized":"2024-09-17T05:39:10.686014+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:54.394077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462405+0000","last_change":"2024-09-17T05:39:09.685796+0000","last_active":"2024-09-17T05:39:11.462405+0000","last_peered":"2024-09-17T05:39:11.462405+0000","last_clean":"2024-09-17T05:39:11.462405+0000","last_became_active":"2024-09-17T05:39:09.683964+0000","last_became_peered":"2024-09-17T05:39:09.683964+0000","last_unstale":"2024-09-17T05:39:11.462405+0000","last_undegraded":"2024-09-17T05:39:11.462405+0000","last_fullsized":"2024-09-17T05:39:11.462405+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:24.662471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"131'6","reported_seq":21,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462212+0000","last_change":"2024-09-17T05:39:08.686779+0000","last_active":"2024-09-17T05:39:11.462212+0000","last_peered":"2024-09-17T05:39:11.462212+0000","last_clean":"2024-09-17T05:39:11.462212+0000","last_became_active":"2024-09-17T05:39:08.686086+0000","last_became_peered":"2024-09-17T05:39:08.686086+0000","last_unstale":"2024-09-17T05:39:11.462212+0000","last_undegraded":"2024-09-17T05:39:11.462212+0000","last_fullsized":"2024-09-17T05:39:11.462212+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:46.349152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451389+0000","last_change":"2024-09-17T05:39:09.702938+0000","last_active":"2024-09-17T05:39:11.451389+0000","last_peered":"2024-09-17T05:39:11.451389+0000","last_clean":"2024-09-17T05:39:11.451389+0000","last_became_active":"2024-09-17T05:39:09.702616+0000","last_became_peered":"2024-09-17T05:39:09.702616+0000","last_unstale":"2024-09-17T05:39:11.451389+0000","last_undegraded":"2024-09-17T05:39:11.451389+0000","last_fullsized":"2024-09-17T05:39:11.451389+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:33:45.921137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451287+0000","last_change":"2024-09-17T05:39:08.674569+0000","last_active":"2024-09-17T05:39:11.451287+0000","last_peered":"2024-09-17T05:39:11.451287+0000","last_clean":"2024-09-17T05:39:11.451287+0000","last_became_active":"2024-09-17T05:39:08.674285+0000","last_became_peered":"2024-09-17T05:39:08.674285+0000","last_unstale":"2024-09-17T05:39:11.451287+0000","last_undegraded":"2024-09-17T05:39:11.451287+0000","last_fullsized":"2024-09-17T05:39:11.451287+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:13.783924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451167+0000","last_change":"2024-09-17T05:39:09.698643+0000","last_active":"2024-09-17T05:39:11.451167+0000","last_peered":"2024-09-17T05:39:11.451167+0000","last_clean":"2024-09-17T05:39:11.451167+0000","last_became_active":"2024-09-17T05:39:09.698013+0000","last_became_peered":"2024-09-17T05:39:09.698013+0000","last_unstale":"2024-09-17T05:39:11.451167+0000","last_undegraded":"2024-09-17T05:39:11.451167+0000","last_fullsized":"2024-09-17T05:39:11.451167+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:49.704973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049765+0000","last_change":"2024-09-17T05:39:08.684608+0000","last_active":"2024-09-17T05:39:12.049765+0000","last_peered":"2024-09-17T05:39:12.049765+0000","last_clean":"2024-09-17T05:39:12.049765+0000","last_became_active":"2024-09-17T05:39:08.684343+0000","last_became_peered":"2024-09-17T05:39:08.684343+0000","last_unstale":"2024-09-17T05:39:12.049765+0000","last_undegraded":"2024-09-17T05:39:12.049765+0000","last_fullsized":"2024-09-17T05:39:12.049765+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:40.195253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.706960+0000","last_change":"2024-09-17T05:39:09.686193+0000","last_active":"2024-09-17T05:39:10.706960+0000","last_peered":"2024-09-17T05:39:10.706960+0000","last_clean":"2024-09-17T05:39:10.706960+0000","last_became_active":"2024-09-17T05:39:09.685938+0000","last_became_peered":"2024-09-17T05:39:09.685938+0000","last_unstale":"2024-09-17T05:39:10.706960+0000","last_undegraded":"2024-09-17T05:39:10.706960+0000","last_fullsized":"2024-09-17T05:39:10.706960+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:00.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"131'8","reported_seq":23,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.706921+0000","last_change":"2024-09-17T05:39:08.682503+0000","last_active":"2024-09-17T05:39:10.706921+0000","last_peered":"2024-09-17T05:39:10.706921+0000","last_clean":"2024-09-17T05:39:10.706921+0000","last_became_active":"2024-09-17T05:39:08.682225+0000","last_became_peered":"2024-09-17T05:39:08.682225+0000","last_unstale":"2024-09-17T05:39:10.706921+0000","last_undegraded":"2024-09-17T05:39:10.706921+0000","last_fullsized":"2024-09-17T05:39:10.706921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:49.204188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685806+0000","last_change":"2024-09-17T05:39:09.686773+0000","last_active":"2024-09-17T05:39:10.685806+0000","last_peered":"2024-09-17T05:39:10.685806+0000","last_clean":"2024-09-17T05:39:10.685806+0000","last_became_active":"2024-09-17T05:39:09.686467+0000","last_became_peered":"2024-09-17T05:39:09.686467+0000","last_unstale":"2024-09-17T05:39:10.685806+0000","last_undegraded":"2024-09-17T05:39:10.685806+0000","last_fullsized":"2024-09-17T05:39:10.685806+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:54.958914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707841+0000","last_change":"2024-09-17T05:39:08.685708+0000","last_active":"2024-09-17T05:39:10.707841+0000","last_peered":"2024-09-17T05:39:10.707841+0000","last_clean":"2024-09-17T05:39:10.707841+0000","last_became_active":"2024-09-17T05:39:08.685380+0000","last_became_peered":"2024-09-17T05:39:08.685380+0000","last_unstale":"2024-09-17T05:39:10.707841+0000","last_undegraded":"2024-09-17T05:39:10.707841+0000","last_fullsized":"2024-09-17T05:39:10.707841+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:52.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685862+0000","last_change":"2024-09-17T05:39:09.699827+0000","last_active":"2024-09-17T05:39:10.685862+0000","last_peered":"2024-09-17T05:39:10.685862+0000","last_clean":"2024-09-17T05:39:10.685862+0000","last_became_active":"2024-09-17T05:39:09.699467+0000","last_became_peered":"2024-09-17T05:39:09.699467+0000","last_unstale":"2024-09-17T05:39:10.685862+0000","last_undegraded":"2024-09-17T05:39:10.685862+0000","last_fullsized":"2024-09-17T05:39:10.685862+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:46.360228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049750+0000","last_change":"2024-09-17T05:39:08.679962+0000","last_active":"2024-09-17T05:39:12.049750+0000","last_peered":"2024-09-17T05:39:12.049750+0000","last_clean":"2024-09-17T05:39:12.049750+0000","last_became_active":"2024-09-17T05:39:08.679323+0000","last_became_peered":"2024-09-17T05:39:08.679323+0000","last_unstale":"2024-09-17T05:39:12.049750+0000","last_undegraded":"2024-09-17T05:39:12.049750+0000","last_fullsized":"2024-09-17T05:39:12.049750+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:40.800908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049566+0000","last_change":"2024-09-17T05:39:09.705201+0000","last_active":"2024-09-17T05:39:12.049566+0000","last_peered":"2024-09-17T05:39:12.049566+0000","last_clean":"2024-09-17T05:39:12.049566+0000","last_became_active":"2024-09-17T05:39:09.704923+0000","last_became_peered":"2024-09-17T05:39:09.704923+0000","last_unstale":"2024-09-17T05:39:12.049566+0000","last_undegraded":"2024-09-17T05:39:12.049566+0000","last_fullsized":"2024-09-17T05:39:12.049566+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:01.470180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689915+0000","last_change":"2024-09-17T05:39:08.679584+0000","last_active":"2024-09-17T05:39:10.689915+0000","last_peered":"2024-09-17T05:39:10.689915+0000","last_clean":"2024-09-17T05:39:10.689915+0000","last_became_active":"2024-09-17T05:39:08.679238+0000","last_became_peered":"2024-09-17T05:39:08.679238+0000","last_unstale":"2024-09-17T05:39:10.689915+0000","last_undegraded":"2024-09-17T05:39:10.689915+0000","last_fullsized":"2024-09-17T05:39:10.689915+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:07.052008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049349+0000","last_change":"2024-09-17T05:39:09.709707+0000","last_active":"2024-09-17T05:39:12.049349+0000","last_peered":"2024-09-17T05:39:12.049349+0000","last_clean":"2024-09-17T05:39:12.049349+0000","last_became_active":"2024-09-17T05:39:09.709396+0000","last_became_peered":"2024-09-17T05:39:09.709396+0000","last_unstale":"2024-09-17T05:39:12.049349+0000","last_undegraded":"2024-09-17T05:39:12.049349+0000","last_fullsized":"2024-09-17T05:39:12.049349+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:27.223389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685726+0000","last_change":"2024-09-17T05:39:08.683294+0000","last_active":"2024-09-17T05:39:10.685726+0000","last_peered":"2024-09-17T05:39:10.685726+0000","last_clean":"2024-09-17T05:39:10.685726+0000","last_became_active":"2024-09-17T05:39:08.683029+0000","last_became_peered":"2024-09-17T05:39:08.683029+0000","last_unstale":"2024-09-17T05:39:10.685726+0000","last_undegraded":"2024-09-17T05:39:10.685726+0000","last_fullsized":"2024-09-17T05:39:10.685726+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.075618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050111+0000","last_change":"2024-09-17T05:39:09.705343+0000","last_active":"2024-09-17T05:39:12.050111+0000","last_peered":"2024-09-17T05:39:12.050111+0000","last_clean":"2024-09-17T05:39:12.050111+0000","last_became_active":"2024-09-17T05:39:09.702948+0000","last_became_peered":"2024-09-17T05:39:09.702948+0000","last_unstale":"2024-09-17T05:39:12.050111+0000","last_undegraded":"2024-09-17T05:39:12.050111+0000","last_fullsized":"2024-09-17T05:39:12.050111+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:35.515218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689623+0000","last_change":"2024-09-17T05:39:08.677212+0000","last_active":"2024-09-17T05:39:10.689623+0000","last_peered":"2024-09-17T05:39:10.689623+0000","last_clean":"2024-09-17T05:39:10.689623+0000","last_became_active":"2024-09-17T05:39:08.676885+0000","last_became_peered":"2024-09-17T05:39:08.676885+0000","last_unstale":"2024-09-17T05:39:10.689623+0000","last_undegraded":"2024-09-17T05:39:10.689623+0000","last_fullsized":"2024-09-17T05:39:10.689623+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:25.782602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462133+0000","last_change":"2024-09-17T05:39:09.696737+0000","last_active":"2024-09-17T05:39:11.462133+0000","last_peered":"2024-09-17T05:39:11.462133+0000","last_clean":"2024-09-17T05:39:11.462133+0000","last_became_active":"2024-09-17T05:39:09.696473+0000","last_became_peered":"2024-09-17T05:39:09.696473+0000","last_unstale":"2024-09-17T05:39:11.462133+0000","last_undegraded":"2024-09-17T05:39:11.462133+0000","last_fullsized":"2024-09-17T05:39:11.462133+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:15.752583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690057+0000","last_change":"2024-09-17T05:39:08.681401+0000","last_active":"2024-09-17T05:39:10.690057+0000","last_peered":"2024-09-17T05:39:10.690057+0000","last_clean":"2024-09-17T05:39:10.690057+0000","last_became_active":"2024-09-17T05:39:08.681071+0000","last_became_peered":"2024-09-17T05:39:08.681071+0000","last_unstale":"2024-09-17T05:39:10.690057+0000","last_undegraded":"2024-09-17T05:39:10.690057+0000","last_fullsized":"2024-09-17T05:39:10.690057+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:25:43.581246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451394+0000","last_change":"2024-09-17T05:39:09.688991+0000","last_active":"2024-09-17T05:39:11.451394+0000","last_peered":"2024-09-17T05:39:11.451394+0000","last_clean":"2024-09-17T05:39:11.451394+0000","last_became_active":"2024-09-17T05:39:09.688714+0000","last_became_peered":"2024-09-17T05:39:09.688714+0000","last_unstale":"2024-09-17T05:39:11.451394+0000","last_undegraded":"2024-09-17T05:39:11.451394+0000","last_fullsized":"2024-09-17T05:39:11.451394+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:48.869239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451090+0000","last_change":"2024-09-17T05:39:08.670676+0000","last_active":"2024-09-17T05:39:11.451090+0000","last_peered":"2024-09-17T05:39:11.451090+0000","last_clean":"2024-09-17T05:39:11.451090+0000","last_became_active":"2024-09-17T05:39:08.669918+0000","last_became_peered":"2024-09-17T05:39:08.669918+0000","last_unstale":"2024-09-17T05:39:11.451090+0000","last_undegraded":"2024-09-17T05:39:11.451090+0000","last_fullsized":"2024-09-17T05:39:11.451090+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:23.308245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.706983+0000","last_change":"2024-09-17T05:39:09.689573+0000","last_active":"2024-09-17T05:39:10.706983+0000","last_peered":"2024-09-17T05:39:10.706983+0000","last_clean":"2024-09-17T05:39:10.706983+0000","last_became_active":"2024-09-17T05:39:09.689052+0000","last_became_peered":"2024-09-17T05:39:09.689052+0000","last_unstale":"2024-09-17T05:39:10.706983+0000","last_undegraded":"2024-09-17T05:39:10.706983+0000","last_fullsized":"2024-09-17T05:39:10.706983+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:18.145532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.048692+0000","last_change":"2024-09-17T05:39:08.676843+0000","last_active":"2024-09-17T05:39:12.048692+0000","last_peered":"2024-09-17T05:39:12.048692+0000","last_clean":"2024-09-17T05:39:12.048692+0000","last_became_active":"2024-09-17T05:39:08.676520+0000","last_became_peered":"2024-09-17T05:39:08.676520+0000","last_unstale":"2024-09-17T05:39:12.048692+0000","last_undegraded":"2024-09-17T05:39:12.048692+0000","last_fullsized":"2024-09-17T05:39:12.048692+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:02.570603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.462310+0000","last_change":"2024-09-17T05:39:09.684664+0000","last_active":"2024-09-17T05:39:11.462310+0000","last_peered":"2024-09-17T05:39:11.462310+0000","last_clean":"2024-09-17T05:39:11.462310+0000","last_became_active":"2024-09-17T05:39:09.684045+0000","last_became_peered":"2024-09-17T05:39:09.684045+0000","last_unstale":"2024-09-17T05:39:11.462310+0000","last_undegraded":"2024-09-17T05:39:11.462310+0000","last_fullsized":"2024-09-17T05:39:11.462310+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:12.139121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690525+0000","last_change":"2024-09-17T05:39:08.681472+0000","last_active":"2024-09-17T05:39:10.690525+0000","last_peered":"2024-09-17T05:39:10.690525+0000","last_clean":"2024-09-17T05:39:10.690525+0000","last_became_active":"2024-09-17T05:39:08.680987+0000","last_became_peered":"2024-09-17T05:39:08.680987+0000","last_unstale":"2024-09-17T05:39:10.690525+0000","last_undegraded":"2024-09-17T05:39:10.690525+0000","last_fullsized":"2024-09-17T05:39:10.690525+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:34.886192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.050277+0000","last_change":"2024-09-17T05:39:09.708694+0000","last_active":"2024-09-17T05:39:12.050277+0000","last_peered":"2024-09-17T05:39:12.050277+0000","last_clean":"2024-09-17T05:39:12.050277+0000","last_became_active":"2024-09-17T05:39:09.708388+0000","last_became_peered":"2024-09-17T05:39:09.708388+0000","last_unstale":"2024-09-17T05:39:12.050277+0000","last_undegraded":"2024-09-17T05:39:12.050277+0000","last_fullsized":"2024-09-17T05:39:12.050277+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:42.281853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"132'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.685868+0000","last_change":"2024-09-17T05:39:08.674902+0000","last_active":"2024-09-17T05:39:10.685868+0000","last_peered":"2024-09-17T05:39:10.685868+0000","last_clean":"2024-09-17T05:39:10.685868+0000","last_became_active":"2024-09-17T05:39:08.674649+0000","last_became_peered":"2024-09-17T05:39:08.674649+0000","last_unstale":"2024-09-17T05:39:10.685868+0000","last_undegraded":"2024-09-17T05:39:10.685868+0000","last_fullsized":"2024-09-17T05:39:10.685868+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:50.216452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451641+0000","last_change":"2024-09-17T05:39:09.699303+0000","last_active":"2024-09-17T05:39:11.451641+0000","last_peered":"2024-09-17T05:39:11.451641+0000","last_clean":"2024-09-17T05:39:11.451641+0000","last_became_active":"2024-09-17T05:39:09.698962+0000","last_became_peered":"2024-09-17T05:39:09.698962+0000","last_unstale":"2024-09-17T05:39:11.451641+0000","last_undegraded":"2024-09-17T05:39:11.451641+0000","last_fullsized":"2024-09-17T05:39:11.451641+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:12.709597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.690369+0000","last_change":"2024-09-17T05:39:08.682089+0000","last_active":"2024-09-17T05:39:10.690369+0000","last_peered":"2024-09-17T05:39:10.690369+0000","last_clean":"2024-09-17T05:39:10.690369+0000","last_became_active":"2024-09-17T05:39:08.680263+0000","last_became_peered":"2024-09-17T05:39:08.680263+0000","last_unstale":"2024-09-17T05:39:10.690369+0000","last_undegraded":"2024-09-17T05:39:10.690369+0000","last_fullsized":"2024-09-17T05:39:10.690369+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:37:50.752563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689742+0000","last_change":"2024-09-17T05:39:09.703937+0000","last_active":"2024-09-17T05:39:10.689742+0000","last_peered":"2024-09-17T05:39:10.689742+0000","last_clean":"2024-09-17T05:39:10.689742+0000","last_became_active":"2024-09-17T05:39:09.703643+0000","last_became_peered":"2024-09-17T05:39:09.703643+0000","last_unstale":"2024-09-17T05:39:10.689742+0000","last_undegraded":"2024-09-17T05:39:10.689742+0000","last_fullsized":"2024-09-17T05:39:10.689742+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:22.554828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451598+0000","last_change":"2024-09-17T05:39:08.673802+0000","last_active":"2024-09-17T05:39:11.451598+0000","last_peered":"2024-09-17T05:39:11.451598+0000","last_clean":"2024-09-17T05:39:11.451598+0000","last_became_active":"2024-09-17T05:39:08.673520+0000","last_became_peered":"2024-09-17T05:39:08.673520+0000","last_unstale":"2024-09-17T05:39:11.451598+0000","last_undegraded":"2024-09-17T05:39:11.451598+0000","last_fullsized":"2024-09-17T05:39:11.451598+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:57.829343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707470+0000","last_change":"2024-09-17T05:39:09.685986+0000","last_active":"2024-09-17T05:39:10.707470+0000","last_peered":"2024-09-17T05:39:10.707470+0000","last_clean":"2024-09-17T05:39:10.707470+0000","last_became_active":"2024-09-17T05:39:09.685714+0000","last_became_peered":"2024-09-17T05:39:09.685714+0000","last_unstale":"2024-09-17T05:39:10.707470+0000","last_undegraded":"2024-09-17T05:39:10.707470+0000","last_fullsized":"2024-09-17T05:39:10.707470+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:48.869239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.049963+0000","last_change":"2024-09-17T05:39:08.683259+0000","last_active":"2024-09-17T05:39:12.049963+0000","last_peered":"2024-09-17T05:39:12.049963+0000","last_clean":"2024-09-17T05:39:12.049963+0000","last_became_active":"2024-09-17T05:39:08.682951+0000","last_became_peered":"2024-09-17T05:39:08.682951+0000","last_unstale":"2024-09-17T05:39:12.049963+0000","last_undegraded":"2024-09-17T05:39:12.049963+0000","last_fullsized":"2024-09-17T05:39:12.049963+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:38:46.002792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707321+0000","last_change":"2024-09-17T05:39:09.703634+0000","last_active":"2024-09-17T05:39:10.707321+0000","last_peered":"2024-09-17T05:39:10.707321+0000","last_clean":"2024-09-17T05:39:10.707321+0000","last_became_active":"2024-09-17T05:39:09.703280+0000","last_became_peered":"2024-09-17T05:39:09.703280+0000","last_unstale":"2024-09-17T05:39:10.707321+0000","last_undegraded":"2024-09-17T05:39:10.707321+0000","last_fullsized":"2024-09-17T05:39:10.707321+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:58.688537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707473+0000","last_change":"2024-09-17T05:39:08.683416+0000","last_active":"2024-09-17T05:39:10.707473+0000","last_peered":"2024-09-17T05:39:10.707473+0000","last_clean":"2024-09-17T05:39:10.707473+0000","last_became_active":"2024-09-17T05:39:08.682597+0000","last_became_peered":"2024-09-17T05:39:08.682597+0000","last_unstale":"2024-09-17T05:39:10.707473+0000","last_undegraded":"2024-09-17T05:39:10.707473+0000","last_fullsized":"2024-09-17T05:39:10.707473+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:50.216452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689976+0000","last_change":"2024-09-17T05:39:09.690617+0000","last_active":"2024-09-17T05:39:10.689976+0000","last_peered":"2024-09-17T05:39:10.689976+0000","last_clean":"2024-09-17T05:39:10.689976+0000","last_became_active":"2024-09-17T05:39:09.690358+0000","last_became_peered":"2024-09-17T05:39:09.690358+0000","last_unstale":"2024-09-17T05:39:10.689976+0000","last_undegraded":"2024-09-17T05:39:10.689976+0000","last_fullsized":"2024-09-17T05:39:10.689976+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:34.788898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:12.048827+0000","last_change":"2024-09-17T05:39:08.683995+0000","last_active":"2024-09-17T05:39:12.048827+0000","last_peered":"2024-09-17T05:39:12.048827+0000","last_clean":"2024-09-17T05:39:12.048827+0000","last_became_active":"2024-09-17T05:39:08.683726+0000","last_became_peered":"2024-09-17T05:39:08.683726+0000","last_unstale":"2024-09-17T05:39:12.048827+0000","last_undegraded":"2024-09-17T05:39:12.048827+0000","last_fullsized":"2024-09-17T05:39:12.048827+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:14.301339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451138+0000","last_change":"2024-09-17T05:39:09.690892+0000","last_active":"2024-09-17T05:39:11.451138+0000","last_peered":"2024-09-17T05:39:11.451138+0000","last_clean":"2024-09-17T05:39:11.451138+0000","last_became_active":"2024-09-17T05:39:09.690610+0000","last_became_peered":"2024-09-17T05:39:09.690610+0000","last_unstale":"2024-09-17T05:39:11.451138+0000","last_undegraded":"2024-09-17T05:39:11.451138+0000","last_fullsized":"2024-09-17T05:39:11.451138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:34.045368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.451441+0000","last_change":"2024-09-17T05:39:08.673866+0000","last_active":"2024-09-17T05:39:11.451441+0000","last_peered":"2024-09-17T05:39:11.451441+0000","last_clean":"2024-09-17T05:39:11.451441+0000","last_became_active":"2024-09-17T05:39:08.673542+0000","last_became_peered":"2024-09-17T05:39:08.673542+0000","last_unstale":"2024-09-17T05:39:11.451441+0000","last_undegraded":"2024-09-17T05:39:11.451441+0000","last_fullsized":"2024-09-17T05:39:11.451441+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:05:11.138555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.689741+0000","last_change":"2024-09-17T05:39:09.704967+0000","last_active":"2024-09-17T05:39:10.689741+0000","last_peered":"2024-09-17T05:39:10.689741+0000","last_clean":"2024-09-17T05:39:10.689741+0000","last_became_active":"2024-09-17T05:39:09.704618+0000","last_became_peered":"2024-09-17T05:39:09.704618+0000","last_unstale":"2024-09-17T05:39:10.689741+0000","last_undegraded":"2024-09-17T05:39:10.689741+0000","last_fullsized":"2024-09-17T05:39:10.689741+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:08.655235+0000","last_clean_scrub_stamp":"2024-09-17T05:39:08.655235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:24:46.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:10.707430+0000","last_change":"2024-09-17T05:39:08.680453+0000","last_active":"2024-09-17T05:39:10.707430+0000","last_peered":"2024-09-17T05:39:10.707430+0000","last_clean":"2024-09-17T05:39:10.707430+0000","last_became_active":"2024-09-17T05:39:08.680180+0000","last_became_peered":"2024-09-17T05:39:08.680180+0000","last_unstale":"2024-09-17T05:39:10.707430+0000","last_undegraded":"2024-09-17T05:39:10.707430+0000","last_fullsized":"2024-09-17T05:39:10.707430+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:39:07.648621+0000","last_clean_scrub_stamp":"2024-09-17T05:39:07.648621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:15.575940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":321,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-17T05:39:11.461968+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:39:11.461968+0000","last_peered":"2024-09-17T05:39:11.461968+0000","last_clean":"2024-09-17T05:39:11.461968+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:39:11.461968+0000","last_undegraded":"2024-09-17T05:39:11.461968+0000","last_fullsized":"2024-09-17T05:39:11.461968+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509615,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127128,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244712,"statfs":{"total":96636764160,"available":96506585088,"internally_reserved":0,"allocated":1642496,"data_stored":722673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509615,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136852,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234988,"statfs":{"total":96636764160,"available":96496627712,"internally_reserved":0,"allocated":1638400,"data_stored":722639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509615,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131284,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240556,"statfs":{"total":96636764160,"available":96502329344,"internally_reserved":0,"allocated":1638400,"data_stored":722639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509614,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131292,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240548,"statfs":{"total":96636764160,"available":96502321152,"internally_reserved":0,"allocated":1642496,"data_stored":722709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509615,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127136,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244704,"statfs":{"total":96636764160,"available":96506576896,"internally_reserved":0,"allocated":1646592,"data_stored":723311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509616,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145768,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226072,"statfs":{"total":96636764160,"available":96487497728,"internally_reserved":0,"allocated":2248704,"data_stored":1314641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509614,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145200,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226640,"statfs":{"total":96636764160,"available":96488079360,"internally_reserved":0,"allocated":1658880,"data_stored":724307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":146995659},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542320,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131876,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239964,"statfs":{"total":96636764160,"available":96501723136,"internally_reserved":0,"allocated":2244608,"data_stored":1313749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:39:18.742 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:39:18.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:39:18.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- 172.21.15.122:0/3377737676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80b25e0 msgr2=0x7f48f80af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:18.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 --2- 172.21.15.122:0/3377737676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80b25e0 0x7f48f80af540 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f48f80cf7c0 tx=0x7f48f001c7d0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- 172.21.15.122:0/3377737676 shutdown_connections 2024-09-17T05:39:18.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 --2- 172.21.15.122:0/3377737676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80b25e0 0x7f48f80af540 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 --2- 172.21.15.122:0/3377737676 >> v2:172.21.15.173:3300/0 conn(0x7f48f80d2810 0x7f48f8160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- 172.21.15.122:0/3377737676 >> 172.21.15.122:0/3377737676 conn(0x7f48f80d1cf0 msgr2=0x7f48f80d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:18.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- 172.21.15.122:0/3377737676 shutdown_connections 2024-09-17T05:39:18.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- 172.21.15.122:0/3377737676 wait complete. 2024-09-17T05:39:18.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 Processor -- start 2024-09-17T05:39:18.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- start start 2024-09-17T05:39:18.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48f80b25e0 0x7f48f81fb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:18.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f80cfd30 0x7f48f814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80d2810 0x7f48f815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8168f10 con 0x7f48f80d2810 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f48f80b0750 con 0x7f48f80cfd30 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.938+0000 7f48f7200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f48f8169090 con 0x7f48f80b25e0 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80d2810 0x7f48f815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80d2810 0x7f48f815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38898/0 (socket says 172.21.15.122:38898) 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 -- 172.21.15.122:0/3500834121 learned_addr learned my addr 172.21.15.122:0/3500834121 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48ed400640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f80cfd30 0x7f48f814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f5e00640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48f80b25e0 0x7f48f81fb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 -- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48f80b25e0 msgr2=0x7f48f81fb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48f80b25e0 0x7f48f81fb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 -- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f80cfd30 msgr2=0x7f48f814e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:18.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f80cfd30 0x7f48f814e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:18.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 -- 172.21.15.122:0/3500834121 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f81fbda0 con 0x7f48f80d2810 2024-09-17T05:39:18.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f6800640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80d2810 0x7f48f815a040 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f48f001ccb0 tx=0x7f48f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:18.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48eea00640 1 -- 172.21.15.122:0/3500834121 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f00b2040 con 0x7f48f80d2810 2024-09-17T05:39:18.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48eea00640 1 -- 172.21.15.122:0/3500834121 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f00bb070 con 0x7f48f80d2810 2024-09-17T05:39:18.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f8156a20 con 0x7f48f80d2810 2024-09-17T05:39:18.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48eea00640 1 -- 172.21.15.122:0/3500834121 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0003800 con 0x7f48f80d2810 2024-09-17T05:39:18.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f81560f0 con 0x7f48f80d2810 2024-09-17T05:39:18.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f80af540 con 0x7f48f80d2810 2024-09-17T05:39:18.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.942+0000 7f48eea00640 1 -- 172.21.15.122:0/3500834121 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f48f00039a0 con 0x7f48f80d2810 2024-09-17T05:39:18.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.946+0000 7f48eea00640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f48d4075180 0x7f48d4077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:18.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.946+0000 7f48eea00640 1 -- 172.21.15.122:0/3500834121 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f48f012d860 con 0x7f48f80d2810 2024-09-17T05:39:18.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.946+0000 7f48f5e00640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f48d4075180 0x7f48d4077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:18.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.946+0000 7f48f5e00640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f48d4075180 0x7f48d4077640 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f48e80012f0 tx=0x7f48e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:18.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:18.950+0000 7f48eea00640 1 -- 172.21.15.122:0/3500834121 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f48f81fbda0 con 0x7f48f80d2810 2024-09-17T05:39:19.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.178+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f48f81f7930 con 0x7f48f80d2810 2024-09-17T05:39:19.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.182+0000 7f48eea00640 1 -- 172.21.15.122:0/3500834121 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f48f00163f0 con 0x7f48f80d2810 2024-09-17T05:39:19.178 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:39:19.178 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:39:19.178 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:39:19.178 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:39:19.178 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:39:19.178 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:39:19.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.182+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f48d4075180 msgr2=0x7f48d4077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:19.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.182+0000 7f48f7200640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f48d4075180 0x7f48d4077640 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f48e80012f0 tx=0x7f48e8009040 comp rx=0 tx=0).stop 2024-09-17T05:39:19.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.182+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80d2810 msgr2=0x7f48f815a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:19.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.182+0000 7f48f7200640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80d2810 0x7f48f815a040 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f48f001ccb0 tx=0x7f48f0004040 comp rx=0 tx=0).stop 2024-09-17T05:39:19.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.186+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 shutdown_connections 2024-09-17T05:39:19.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.186+0000 7f48f7200640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f48d4075180 0x7f48d4077640 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.186+0000 7f48f7200640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48f80d2810 0x7f48f815a040 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.186+0000 7f48f7200640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f80cfd30 0x7f48f814e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.186+0000 7f48f7200640 1 --2- 172.21.15.122:0/3500834121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48f80b25e0 0x7f48f81fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.186+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 >> 172.21.15.122:0/3500834121 conn(0x7f48f80d1cf0 msgr2=0x7f48f8070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:19.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.186+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 shutdown_connections 2024-09-17T05:39:19.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.186+0000 7f48f7200640 1 -- 172.21.15.122:0/3500834121 wait complete. 2024-09-17T05:39:19.199 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:39:19.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 -- 172.21.15.122:0/689648083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35200d28c0 msgr2=0x7f35200cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:19.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 --2- 172.21.15.122:0/689648083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35200d28c0 0x7f35200cea20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3508009840 tx=0x7f350801c820 comp rx=0 tx=0).stop 2024-09-17T05:39:19.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 -- 172.21.15.122:0/689648083 shutdown_connections 2024-09-17T05:39:19.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 --2- 172.21.15.122:0/689648083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35200d28c0 0x7f35200cea20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 --2- 172.21.15.122:0/689648083 >> v2:172.21.15.173:3300/0 conn(0x7f352015be80 0x7f35200ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 -- 172.21.15.122:0/689648083 >> 172.21.15.122:0/689648083 conn(0x7f35200d1da0 msgr2=0x7f35200d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:19.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 -- 172.21.15.122:0/689648083 shutdown_connections 2024-09-17T05:39:19.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 -- 172.21.15.122:0/689648083 wait complete. 2024-09-17T05:39:19.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 Processor -- start 2024-09-17T05:39:19.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.346+0000 7f3527400640 1 -- start start 2024-09-17T05:39:19.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35200b53f0 0x7f3520150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:19.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35200d28c0 0x7f35200c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:19.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f352015be80 0x7f35200c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:19.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3520169190 con 0x7f35200b53f0 2024-09-17T05:39:19.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3520169010 con 0x7f352015be80 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3520169310 con 0x7f35200d28c0 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35200b53f0 0x7f3520150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35200b53f0 0x7f3520150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38904/0 (socket says 172.21.15.122:38904) 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 -- 172.21.15.122:0/811593145 learned_addr learned my addr 172.21.15.122:0/811593145 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3525400640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f352015be80 0x7f35200c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3517e00640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35200d28c0 0x7f35200c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 -- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35200d28c0 msgr2=0x7f35200c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35200d28c0 0x7f35200c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 -- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f352015be80 msgr2=0x7f35200c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:19.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f352015be80 0x7f35200c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 -- 172.21.15.122:0/811593145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35200c80a0 con 0x7f35200b53f0 2024-09-17T05:39:19.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3525400640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f352015be80 0x7f35200c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:19.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3524a00640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35200b53f0 0x7f3520150940 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f351000e9a0 tx=0x7f351000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:19.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f351d600640 1 -- 172.21.15.122:0/811593145 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351000cd80 con 0x7f35200b53f0 2024-09-17T05:39:19.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f351d600640 1 -- 172.21.15.122:0/811593145 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3510004020 con 0x7f35200b53f0 2024-09-17T05:39:19.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f351d600640 1 -- 172.21.15.122:0/811593145 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510004450 con 0x7f35200b53f0 2024-09-17T05:39:19.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352006f660 con 0x7f35200b53f0 2024-09-17T05:39:19.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35200c78a0 con 0x7f35200b53f0 2024-09-17T05:39:19.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.350+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35200b5860 con 0x7f35200b53f0 2024-09-17T05:39:19.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.354+0000 7f351d600640 1 -- 172.21.15.122:0/811593145 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f3510003600 con 0x7f35200b53f0 2024-09-17T05:39:19.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.354+0000 7f351d600640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f34fc075180 0x7f34fc077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:19.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.354+0000 7f351d600640 1 -- 172.21.15.122:0/811593145 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f3510093ac0 con 0x7f35200b53f0 2024-09-17T05:39:19.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.354+0000 7f3517e00640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f34fc075180 0x7f34fc077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:19.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.354+0000 7f3517e00640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f34fc075180 0x7f34fc077640 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f35080af000 tx=0x7f350801b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:19.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.358+0000 7f351d600640 1 -- 172.21.15.122:0/811593145 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3510060670 con 0x7f35200b53f0 2024-09-17T05:39:19.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.558+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3520076410 con 0x7f35200b53f0 2024-09-17T05:39:19.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.558+0000 7f351d600640 1 -- 172.21.15.122:0/811593145 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5234 (secure 0 0 0) 0x7f351005ff70 con 0x7f35200b53f0 2024-09-17T05:39:19.556 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:19.556 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":86,"btime":"2024-09-17T05:39:12:702401+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/609693749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":609693749},{"type":"v1","addr":"172.21.15.122:6834","nonce":609693749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2826125653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2826125653},{"type":"v1","addr":"172.21.15.173:6836","nonce":2826125653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4138151052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4138151052},{"type":"v1","addr":"172.21.15.173:6837","nonce":4138151052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:39:09.055675+0000","modified":"2024-09-17T05:39:12.084809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2463174917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2463174917},{"type":"v1","addr":"172.21.15.122:6835","nonce":2463174917}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8916,"qdb_cluster":[8916]},"id":9}]} 2024-09-17T05:39:19.556 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 86 2024-09-17T05:39:19.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f34fc075180 msgr2=0x7f34fc077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:19.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f34fc075180 0x7f34fc077640 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f35080af000 tx=0x7f350801b060 comp rx=0 tx=0).stop 2024-09-17T05:39:19.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35200b53f0 msgr2=0x7f3520150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35200b53f0 0x7f3520150940 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f351000e9a0 tx=0x7f351000ee60 comp rx=0 tx=0).stop 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 shutdown_connections 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f34fc075180 0x7f34fc077640 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f352015be80 0x7f35200c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35200d28c0 0x7f35200c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 --2- 172.21.15.122:0/811593145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35200b53f0 0x7f3520150940 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 >> 172.21.15.122:0/811593145 conn(0x7f35200d1da0 msgr2=0x7f35200b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 shutdown_connections 2024-09-17T05:39:19.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:19.562+0000 7f3527400640 1 -- 172.21.15.122:0/811593145 wait complete. 2024-09-17T05:39:19.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:39:09.055675+0000', 'modified': '2024-09-17T05:39:12.084809+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8916}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8916': {'gid': 8916, 'name': 'a', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6835/2463174917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6832', 'nonce': 2463174917}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 2463174917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8916, 'qdb_cluster': [8916]} 2024-09-17T05:39:19.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:39:19.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T05:39:20.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.130+0000 7f39ce800640 1 -- 172.21.15.122:0/2801370778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80b49b0 msgr2=0x7f39c815ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.130+0000 7f39ce800640 1 --2- 172.21.15.122:0/2801370778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80b49b0 0x7f39c815ea30 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009900 tx=0x7f39b001c850 comp rx=0 tx=0).stop 2024-09-17T05:39:20.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.130+0000 7f39ce800640 1 -- 172.21.15.122:0/2801370778 shutdown_connections 2024-09-17T05:39:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.130+0000 7f39ce800640 1 --2- 172.21.15.122:0/2801370778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39c80703d0 0x7f39c815a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.130+0000 7f39ce800640 1 --2- 172.21.15.122:0/2801370778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80b49b0 0x7f39c815ea30 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.130+0000 7f39ce800640 1 -- 172.21.15.122:0/2801370778 >> 172.21.15.122:0/2801370778 conn(0x7f39c80b18f0 msgr2=0x7f39c80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.130+0000 7f39ce800640 1 -- 172.21.15.122:0/2801370778 shutdown_connections 2024-09-17T05:39:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.130+0000 7f39ce800640 1 -- 172.21.15.122:0/2801370778 wait complete. 2024-09-17T05:39:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 Processor -- start 2024-09-17T05:39:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 -- start start 2024-09-17T05:39:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80703d0 0x7f39c81f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c80b49b0 0x7f39c81564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39c815a4d0 0x7f39c809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39c80ccfe0 con 0x7f39c80703d0 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f39c80cce60 con 0x7f39c80b49b0 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f39c8165160 con 0x7f39c815a4d0 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80703d0 0x7f39c81f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80703d0 0x7f39c81f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38916/0 (socket says 172.21.15.122:38916) 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 -- 172.21.15.122:0/2070949634 learned_addr learned my addr 172.21.15.122:0/2070949634 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7e00640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39c815a4d0 0x7f39c809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39bea00640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c80b49b0 0x7f39c81564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 -- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39c815a4d0 msgr2=0x7f39c809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39c815a4d0 0x7f39c809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 -- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c80b49b0 msgr2=0x7f39c81564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c80b49b0 0x7f39c81564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 -- 172.21.15.122:0/2070949634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c81f9b40 con 0x7f39c80703d0 2024-09-17T05:39:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39c7400640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80703d0 0x7f39c81f9600 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c3c0 tx=0x7f39b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39bfe00640 1 -- 172.21.15.122:0/2070949634 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8017070 con 0x7f39c80703d0 2024-09-17T05:39:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39bfe00640 1 -- 172.21.15.122:0/2070949634 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39b80053c0 con 0x7f39c80703d0 2024-09-17T05:39:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 -- 172.21.15.122:0/2070949634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c8153c30 con 0x7f39c80703d0 2024-09-17T05:39:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39bfe00640 1 -- 172.21.15.122:0/2070949634 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8003860 con 0x7f39c80703d0 2024-09-17T05:39:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.134+0000 7f39ce800640 1 -- 172.21.15.122:0/2070949634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81541f0 con 0x7f39c80703d0 2024-09-17T05:39:20.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.138+0000 7f39bfe00640 1 -- 172.21.15.122:0/2070949634 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f39b801a070 con 0x7f39c80703d0 2024-09-17T05:39:20.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.138+0000 7f39d1b91000 1 Processor -- start 2024-09-17T05:39:20.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.138+0000 7f39bfe00640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39a4075180 0x7f39a4077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.138+0000 7f39d1b91000 1 -- start start 2024-09-17T05:39:20.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.138+0000 7f39bfe00640 1 -- 172.21.15.122:0/2070949634 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f39b80925a0 con 0x7f39c80703d0 2024-09-17T05:39:20.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.138+0000 7f39c7e00640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39a4075180 0x7f39a4077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7e00640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39a4075180 0x7f39a4077640 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f39c809c430 tx=0x7f39c00095d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bed4bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 0x5573becff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bed001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5573bec203f0 con 0x5573bed4b9d0 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5573beb5db20 con 0x5573bed4c2f0 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5573beb4cf50 con 0x5573becffe10 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bed4bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bed4bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38924/0 (socket says 172.21.15.122:38924) 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 -- 172.21.15.122:0/3675057195 learned_addr learned my addr 172.21.15.122:0/3675057195 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39bea00640 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 0x5573becff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7e00640 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bed001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 -- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 msgr2=0x5573bed001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bed001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 -- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 msgr2=0x5573becff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 0x5573becff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 -- 172.21.15.122:0/3675057195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573beb14590 con 0x5573bed4b9d0 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7e00640 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bed001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39c7400640 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bed4bdb0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f39b800cd80 tx=0x7f39b805e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a9a8cb766d703 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:20.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39b7e00640 1 -- 172.21.15.122:0/3675057195 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8063410 con 0x5573bed4b9d0 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39b7e00640 1 -- 172.21.15.122:0/3675057195 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39b80682c0 con 0x5573bed4b9d0 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39b7e00640 1 -- 172.21.15.122:0/3675057195 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b807eb30 con 0x5573bed4b9d0 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- 172.21.15.122:0/3675057195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 msgr2=0x5573bed4bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bed4bdb0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f39b800cd80 tx=0x7f39b805e460 comp rx=0 tx=0).stop 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- 172.21.15.122:0/3675057195 shutdown_connections 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bed001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 0x5573becff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 --2- 172.21.15.122:0/3675057195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bed4bdb0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- 172.21.15.122:0/3675057195 >> 172.21.15.122:0/3675057195 conn(0x5573bebe9130 msgr2=0x5573bed2ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- 172.21.15.122:0/3675057195 shutdown_connections 2024-09-17T05:39:20.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- 172.21.15.122:0/3675057195 wait complete. 2024-09-17T05:39:20.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 -- start start 2024-09-17T05:39:20.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.142+0000 7f39d1b91000 1 Processor -- start 2024-09-17T05:39:20.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 0x5573bed016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 0x5573bed01be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bed02120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5573beae75d0 con 0x5573bed4c2f0 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5573beb50d60 con 0x5573becffe10 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5573beaf3500 con 0x5573bed4b9d0 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 10 client.0 _ll_register_callbacks cb 0x5573beaf7810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bed02120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bed02120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38926/0 (socket says 172.21.15.122:38926) 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 -- 172.21.15.122:0/1768377419 learned_addr learned my addr 172.21.15.122:0/1768377419 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39bea00640 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 0x5573bed01be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7400640 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 0x5573bed016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 -- 172.21.15.122:0/1768377419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 msgr2=0x5573bed01be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 0x5573bed01be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 -- 172.21.15.122:0/1768377419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 msgr2=0x5573bed016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 0x5573bed016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 -- 172.21.15.122:0/1768377419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573beaeb170 con 0x5573bed4c2f0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7e00640 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bed02120 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009c60 tx=0x7f39c0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af07fa1a2d4c5a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 1 -- 172.21.15.122:0/1768377419 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0004d30 con 0x5573bed4c2f0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- 172.21.15.122:0/1768377419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5573beae50f0 con 0x5573bed4c2f0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 1 -- 172.21.15.122:0/1768377419 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f39c0012660 con 0x5573bed4c2f0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 1 -- 172.21.15.122:0/1768377419 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f39c0012860 con 0x5573bed4c2f0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 10 client.9063 fetch_fsmap learned FSMap version 86 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- 172.21.15.122:0/1768377419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x5573bebd1ef0 con 0x5573bed4c2f0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 1 -- 172.21.15.122:0/1768377419 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5573beae50f0 con 0x5573bed4c2f0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 1 -- 172.21.15.122:0/1768377419 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7f39c0032020 con 0x5573bed4c2f0 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 10 client.9063 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 10 client.9063 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:39:20.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 20 client.9063 populate_metadata read hostname 'smithi122' 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] conn(0x5573bed02b30 0x5573bed61370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 4 client.9063 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- 172.21.15.122:0/1768377419 --> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573beb5fa70 con 0x5573bed02b30 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7400640 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] conn(0x5573bed02b30 0x5573bed61370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 10 client.9063 ms_handle_connect on v2:172.21.15.122:6832/2463174917 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39c7400640 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] conn(0x5573bed02b30 0x5573bed61370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce44d5b0d96495e server_cookie=f9b1a11212c89776 in_seq=0 out_seq=0 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 1 -- 172.21.15.122:0/1768377419 <== mds.0 v2:172.21.15.122:6832/2463174917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573beb5fa70 con 0x5573bed02b30 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 10 client.9063 handle_command_reply: tid=0 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 client.9063 shutdown 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- 172.21.15.122:0/1768377419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 msgr2=0x5573bed02120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bed02120 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009c60 tx=0x7f39c0004ac0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- 172.21.15.122:0/1768377419 shutdown_connections 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] conn(0x5573bed02b30 0x5573bed61370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bed02120 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 0x5573bed01be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 --2- 172.21.15.122:0/1768377419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 0x5573bed016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39d1b91000 1 -- 172.21.15.122:0/1768377419 >> 172.21.15.122:0/1768377419 conn(0x5573bebe9130 msgr2=0x5573becfefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.146+0000 7f39b4c00640 0 client.9063 ms_handle_reset on v2:172.21.15.122:6832/2463174917 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.150+0000 7f39d1b91000 1 -- 172.21.15.122:0/1768377419 shutdown_connections 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.150+0000 7f39d1b91000 1 -- 172.21.15.122:0/1768377419 wait complete. 2024-09-17T05:39:20.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.150+0000 7f39d1b91000 20 client.9063 trim_cache size 0 max 16384 2024-09-17T05:39:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39d1b91000 1 Processor -- start 2024-09-17T05:39:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39d1b91000 1 -- start start 2024-09-17T05:39:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 0x5573bedc6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 0x5573bedc6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bedc6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39d1b91000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39c815d090 con 0x5573bed4c2f0 2024-09-17T05:39:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39d1b91000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f39c809d5b0 con 0x5573becffe10 2024-09-17T05:39:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39d1b91000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f39c809dc00 con 0x5573bed4b9d0 2024-09-17T05:39:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bedc6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39c7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bedc6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38930/0 (socket says 172.21.15.122:38930) 2024-09-17T05:39:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39c7e00640 1 -- 172.21.15.122:0/2784680623 learned_addr learned my addr 172.21.15.122:0/2784680623 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39c7e00640 1 -- 172.21.15.122:0/2784680623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 msgr2=0x5573bedc6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:39:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39c7e00640 1 --2- 172.21.15.122:0/2784680623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 0x5573bedc6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39c7e00640 1 -- 172.21.15.122:0/2784680623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 msgr2=0x5573bedc6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39c7e00640 1 --2- 172.21.15.122:0/2784680623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 0x5573bedc6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.166+0000 7f39c7e00640 1 -- 172.21.15.122:0/2784680623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573beba8d90 con 0x5573bed4c2f0 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39c7e00640 1 --2- 172.21.15.122:0/2784680623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bedc6ed0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f39c0000f90 tx=0x7f39c0011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144011b123930696 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39b7400640 1 -- 172.21.15.122:0/2784680623 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0004890 con 0x5573bed4c2f0 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39b7400640 1 -- 172.21.15.122:0/2784680623 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c0038020 con 0x5573bed4c2f0 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39b7400640 1 -- 172.21.15.122:0/2784680623 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0004b30 con 0x5573bed4c2f0 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- 172.21.15.122:0/2784680623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 msgr2=0x5573bedc6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2784680623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bedc6ed0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f39c0000f90 tx=0x7f39c0011f50 comp rx=0 tx=0).stop 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- 172.21.15.122:0/2784680623 shutdown_connections 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2784680623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4c2f0 0x5573bedc6ed0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2784680623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573bed4b9d0 0x5573bedc6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2784680623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573becffe10 0x5573bedc6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- 172.21.15.122:0/2784680623 >> 172.21.15.122:0/2784680623 conn(0x5573bebe9130 msgr2=0x5573bed01a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- 172.21.15.122:0/2784680623 shutdown_connections 2024-09-17T05:39:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- 172.21.15.122:0/2784680623 wait complete. 2024-09-17T05:39:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- start start 2024-09-17T05:39:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 Processor -- start 2024-09-17T05:39:20.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bedca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bedcaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 0x5573bedcaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5573bebd1a60 con 0x5573bed4b9d0 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5573beb18ec0 con 0x5573bed4c2f0 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5573beb55390 con 0x5573becffe10 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39d1b91000 10 client.0 _ll_register_callbacks cb 0x7f39c00094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bedcaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bedcaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38942/0 (socket says 172.21.15.122:38942) 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 -- 172.21.15.122:0/2234504657 learned_addr learned my addr 172.21.15.122:0/2234504657 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39c7e00640 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 0x5573bedcaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39c7400640 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bedca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 -- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 msgr2=0x5573bedca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bedca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 -- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 msgr2=0x5573bedcaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:39:20.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 0x5573bedcaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 -- 172.21.15.122:0/2234504657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573beb13a90 con 0x5573bed4b9d0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39c7400640 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bedca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39bea00640 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bedcaa40 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009900 tx=0x7f39b00041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19232ba843bd2ed server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39b4c00640 1 -- 172.21.15.122:0/2234504657 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b00bc020 con 0x5573bed4b9d0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.170+0000 7f39b4c00640 1 -- 172.21.15.122:0/2234504657 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f39b0005030 con 0x5573bed4b9d0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 -- 172.21.15.122:0/2234504657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c0038020 con 0x5573bed4b9d0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39b4c00640 1 -- 172.21.15.122:0/2234504657 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f39b00c3050 con 0x5573bed4b9d0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 10 client.9069 fetch_fsmap learned FSMap version 86 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 -- 172.21.15.122:0/2234504657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x5573beaf2050 con 0x5573bed4b9d0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39b4c00640 1 -- 172.21.15.122:0/2234504657 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c0038020 con 0x5573bed4b9d0 2024-09-17T05:39:20.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39b4c00640 1 -- 172.21.15.122:0/2234504657 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7f39b00c6020 con 0x5573bed4b9d0 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 10 client.9069 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 10 client.9069 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 20 client.9069 populate_metadata read hostname 'smithi122' 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] conn(0x5573bed57a10 0x5573bedd3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 4 client.9069 mds_command: new command op to 8916 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 -- 172.21.15.122:0/2234504657 --> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] -- command(tid 0: {"prefix": "status"}) -- 0x5573beb5b520 con 0x5573bed57a10 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39c7400640 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] conn(0x5573bed57a10 0x5573bedd3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39c7400640 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] conn(0x5573bed57a10 0x5573bedd3760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a83f9c8f2f8ce6c server_cookie=74455cdd9a93f009 in_seq=0 out_seq=0 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39b4c00640 10 client.9069 ms_handle_connect on v2:172.21.15.122:6832/2463174917 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39b4c00640 1 -- 172.21.15.122:0/2234504657 <== mds.0 v2:172.21.15.122:6832/2463174917 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x5573beb5b520 con 0x5573bed57a10 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39b4c00640 10 client.9069 handle_command_reply: tid=0 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 client.9069 shutdown 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 -- 172.21.15.122:0/2234504657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 msgr2=0x5573bedcaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bedcaa40 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009900 tx=0x7f39b00041a0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 -- 172.21.15.122:0/2234504657 shutdown_connections 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.122:6832/2463174917,v1:172.21.15.122:6835/2463174917] conn(0x5573bed57a10 0x5573bedd3760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573bed4c2f0 0x5573bedcaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573bed4b9d0 0x5573bedcaa40 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39b4c00640 0 client.9069 ms_handle_reset on v2:172.21.15.122:6832/2463174917 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 --2- 172.21.15.122:0/2234504657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5573becffe10 0x5573bedca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 -- 172.21.15.122:0/2234504657 >> 172.21.15.122:0/2234504657 conn(0x5573bebe9130 msgr2=0x5573becff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 -- 172.21.15.122:0/2234504657 shutdown_connections 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 1 -- 172.21.15.122:0/2234504657 wait complete. 2024-09-17T05:39:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39d1b91000 20 client.9069 trim_cache size 0 max 16384 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 8916, 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 10.48206731, 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 86, 2024-09-17T05:39:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 132, 2024-09-17T05:39:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 132, 2024-09-17T05:39:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 15.000271264 2024-09-17T05:39:20.177 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:39:20.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 -- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39a4075180 msgr2=0x7f39a4077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39a4075180 0x7f39a4077640 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f39c809c430 tx=0x7f39c00095d0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 -- 172.21.15.122:0/2070949634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80703d0 msgr2=0x7f39c81f9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80703d0 0x7f39c81f9600 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c3c0 tx=0x7f39b800c880 comp rx=0 tx=0).stop 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 -- 172.21.15.122:0/2070949634 shutdown_connections 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39a4075180 0x7f39a4077640 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39c815a4d0 0x7f39c809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c80b49b0 0x7f39c81564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 --2- 172.21.15.122:0/2070949634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80703d0 0x7f39c81f9600 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 -- 172.21.15.122:0/2070949634 >> 172.21.15.122:0/2070949634 conn(0x7f39c80b18f0 msgr2=0x7f39c8157a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 -- 172.21.15.122:0/2070949634 shutdown_connections 2024-09-17T05:39:20.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.174+0000 7f39ce800640 1 -- 172.21.15.122:0/2070949634 wait complete. 2024-09-17T05:39:20.189 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:39:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 -- 172.21.15.122:0/1489333982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac80d01b0 msgr2=0x7f1ac80cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 --2- 172.21.15.122:0/1489333982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac80d01b0 0x7f1ac80cefb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009840 tx=0x7f1ab401c790 comp rx=0 tx=0).stop 2024-09-17T05:39:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 -- 172.21.15.122:0/1489333982 shutdown_connections 2024-09-17T05:39:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 --2- 172.21.15.122:0/1489333982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac80d01b0 0x7f1ac80cefb0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 -- 172.21.15.122:0/1489333982 >> 172.21.15.122:0/1489333982 conn(0x7f1ac809aa10 msgr2=0x7f1ac809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 -- 172.21.15.122:0/1489333982 shutdown_connections 2024-09-17T05:39:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 -- 172.21.15.122:0/1489333982 wait complete. 2024-09-17T05:39:20.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 Processor -- start 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.334+0000 7f1acd800640 1 -- start start 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac80b2590 0x7f1ac8167020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ac80d01b0 0x7f1ac81552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac815f000 0x7f1ac80b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac816a0d0 con 0x7f1ac815f000 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8169f50 con 0x7f1ac80b2590 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac816a250 con 0x7f1ac80d01b0 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac815f000 0x7f1ac80b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac815f000 0x7f1ac80b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38966/0 (socket says 172.21.15.122:38966) 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 -- 172.21.15.122:0/2176395812 learned_addr learned my addr 172.21.15.122:0/2176395812 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:20.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1ac7400640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ac80d01b0 0x7f1ac81552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1ac7e00640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac80b2590 0x7f1ac8167020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 -- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ac80d01b0 msgr2=0x7f1ac81552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ac80d01b0 0x7f1ac81552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 -- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac80b2590 msgr2=0x7f1ac8167020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac80b2590 0x7f1ac8167020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 -- 172.21.15.122:0/2176395812 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac80aef70 con 0x7f1ac815f000 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acce00640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac815f000 0x7f1ac80b5410 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c780 tx=0x7f1ac000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1abfe00640 1 -- 172.21.15.122:0/2176395812 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004b70 con 0x7f1ac815f000 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1abfe00640 1 -- 172.21.15.122:0/2176395812 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ac0004e00 con 0x7f1ac815f000 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1abfe00640 1 -- 172.21.15.122:0/2176395812 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000aa90 con 0x7f1ac815f000 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac80b5e60 con 0x7f1ac815f000 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac806e1b0 con 0x7f1ac815f000 2024-09-17T05:39:20.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.338+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac80cefb0 con 0x7f1ac815f000 2024-09-17T05:39:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.342+0000 7f1abfe00640 1 -- 172.21.15.122:0/2176395812 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f1ac000ac30 con 0x7f1ac815f000 2024-09-17T05:39:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.342+0000 7f1abfe00640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1aa8075180 0x7f1aa8077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.342+0000 7f1abfe00640 1 -- 172.21.15.122:0/2176395812 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f1ac0093420 con 0x7f1ac815f000 2024-09-17T05:39:20.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.342+0000 7f1ac7e00640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1aa8075180 0x7f1aa8077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:20.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.342+0000 7f1ac7e00640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1aa8075180 0x7f1aa8077640 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8007fd0 tx=0x7f1ab8007ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:20.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.346+0000 7f1abfe00640 1 -- 172.21.15.122:0/2176395812 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1ac0020070 con 0x7f1ac815f000 2024-09-17T05:39:20.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ac80cd0a0 con 0x7f1ac815f000 2024-09-17T05:39:20.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1abfe00640 1 -- 172.21.15.122:0/2176395812 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7f1ac005f8e0 con 0x7f1ac815f000 2024-09-17T05:39:20.545 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:39:20.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1aa8075180 msgr2=0x7f1aa8077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1aa8075180 0x7f1aa8077640 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8007fd0 tx=0x7f1ab8007ea0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac815f000 msgr2=0x7f1ac80b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:20.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac815f000 0x7f1ac80b5410 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c780 tx=0x7f1ac000cc40 comp rx=0 tx=0).stop 2024-09-17T05:39:20.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 shutdown_connections 2024-09-17T05:39:20.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1aa8075180 0x7f1aa8077640 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ac815f000 0x7f1ac80b5410 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ac80d01b0 0x7f1ac81552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 --2- 172.21.15.122:0/2176395812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ac80b2590 0x7f1ac8167020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:20.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 >> 172.21.15.122:0/2176395812 conn(0x7f1ac809aa10 msgr2=0x7f1ac80ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:20.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 shutdown_connections 2024-09-17T05:39:20.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20.550+0000 7f1acd800640 1 -- 172.21.15.122:0/2176395812 wait complete. 2024-09-17T05:39:20.567 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:39:20.567 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:39:20.567 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:39:20.568 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:39:20.568 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:39:20.568 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:39:20.568 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:39:20.568 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:39:20.568 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:39:20.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:20.569 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:39:20.578 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:39:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:39:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:39:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 41607sec preferred_lft 41607sec 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:39:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:39:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:39:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:39:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:39:20.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:20.582 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:39:20.641 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:39:20.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:20.643 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:39:20.695 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:39:20.696 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:39:20.696 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:39:20.696 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:39:20.696 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:39:20.696 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:39:20.696 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:39:20.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:39:20.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:39:20.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:20.835 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:39:20.887 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:39:20.887 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:39:20.888 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:39:20.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:20.889 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:39:20.954 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:39:20.955 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:39:21.003 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:39:21.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:39:21.004 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:39:21.055 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:39:21.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:21.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:39:21.152 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:39:21.152 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:39:21.152 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:39:21.167 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:39:21.167 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:39:21.167 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:39:21.167 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:39:21.167 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:39:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:39:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:39:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:39:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:39:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:39:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:39:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:39:21.176 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:39:21.180 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:39:21.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:21.180 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:39:21.201 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:39:21.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:39:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:39:21.308 INFO:teuthology.orchestra.run.smithi122.stdout:client8170 2024-09-17T05:39:21.308 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:39:21.309 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:39:21.318 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:39:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b69a00640 1 -- 172.21.15.122:0/720711264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b48004010 msgr2=0x7f9b48004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b69a00640 1 --2- 172.21.15.122:0/720711264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b48004010 0x7f9b48004810 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000c310 tx=0x7f9b5001cf60 comp rx=0 tx=0).stop 2024-09-17T05:39:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b63400640 1 -- 172.21.15.122:0/720711264 reap_dead start 2024-09-17T05:39:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b69a00640 1 -- 172.21.15.122:0/720711264 shutdown_connections 2024-09-17T05:39:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b69a00640 1 -- 172.21.15.122:0/720711264 >> 172.21.15.122:0/720711264 conn(0x7f9b640d0000 msgr2=0x7f9b640d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:21.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b69a00640 1 -- 172.21.15.122:0/720711264 shutdown_connections 2024-09-17T05:39:21.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b69a00640 1 -- 172.21.15.122:0/720711264 wait complete. 2024-09-17T05:39:21.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b69a00640 1 Processor -- start 2024-09-17T05:39:21.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.510+0000 7f9b69a00640 1 -- start start 2024-09-17T05:39:21.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b48004010 0x7f9b640b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:21.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b641f81e0 0x7f9b640c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:21.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b640c70d0 0x7f9b6415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:21.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b640ce760 con 0x7f9b641f81e0 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9b640ce5e0 con 0x7f9b48004010 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9b640ce8e0 con 0x7f9b640c70d0 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b641f81e0 0x7f9b640c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b641f81e0 0x7f9b640c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48274/0 (socket says 172.21.15.122:48274) 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 -- 172.21.15.122:0/570876624 learned_addr learned my addr 172.21.15.122:0/570876624 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b48004010 0x7f9b640b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b63400640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b640c70d0 0x7f9b6415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 -- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b640c70d0 msgr2=0x7f9b6415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:21.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b640c70d0 0x7f9b6415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 -- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b48004010 msgr2=0x7f9b640b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:21.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b48004010 0x7f9b640b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 -- 172.21.15.122:0/570876624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b640b3820 con 0x7f9b641f81e0 2024-09-17T05:39:21.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b62000640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b641f81e0 0x7f9b640c94d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f9b5001b660 tx=0x7f9b50001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:21.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b5aa00640 1 -- 172.21.15.122:0/570876624 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50001540 con 0x7f9b641f81e0 2024-09-17T05:39:21.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b5aa00640 1 -- 172.21.15.122:0/570876624 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b500016e0 con 0x7f9b641f81e0 2024-09-17T05:39:21.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b640b3b00 con 0x7f9b641f81e0 2024-09-17T05:39:21.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b5aa00640 1 -- 172.21.15.122:0/570876624 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b500064e0 con 0x7f9b641f81e0 2024-09-17T05:39:21.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b640b45e0 con 0x7f9b641f81e0 2024-09-17T05:39:21.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.514+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6415f180 con 0x7f9b641f81e0 2024-09-17T05:39:21.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.518+0000 7f9b5aa00640 1 -- 172.21.15.122:0/570876624 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f9b50003600 con 0x7f9b641f81e0 2024-09-17T05:39:21.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.518+0000 7f9b5aa00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b44075180 0x7f9b44077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:21.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.518+0000 7f9b5aa00640 1 -- 172.21.15.122:0/570876624 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f9b5012caa0 con 0x7f9b641f81e0 2024-09-17T05:39:21.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.518+0000 7f9b62a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b44075180 0x7f9b44077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:21.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.518+0000 7f9b62a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b44075180 0x7f9b44077640 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9b54000f90 tx=0x7f9b54006ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:21.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.518+0000 7f9b5aa00640 1 -- 172.21.15.122:0/570876624 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9b50016d10 con 0x7f9b641f81e0 2024-09-17T05:39:21.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.722+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f9b640ce3c0 con 0x7f9b641f81e0 2024-09-17T05:39:21.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b5aa00640 1 -- 172.21.15.122:0/570876624 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v87) ==== 136+0+0 (secure 0 0 0) 0x7f9b500f8fe0 con 0x7f9b641f81e0 2024-09-17T05:39:21.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b44075180 msgr2=0x7f9b44077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:21.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b44075180 0x7f9b44077640 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9b54000f90 tx=0x7f9b54006ea0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b641f81e0 msgr2=0x7f9b640c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b641f81e0 0x7f9b640c94d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f9b5001b660 tx=0x7f9b50001d00 comp rx=0 tx=0).stop 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 shutdown_connections 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9b44075180 0x7f9b44077640 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b640c70d0 0x7f9b6415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b641f81e0 0x7f9b640c94d0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 --2- 172.21.15.122:0/570876624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b48004010 0x7f9b640b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 >> 172.21.15.122:0/570876624 conn(0x7f9b640d0000 msgr2=0x7f9b64164710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 shutdown_connections 2024-09-17T05:39:21.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.802+0000 7f9b69a00640 1 -- 172.21.15.122:0/570876624 wait complete. 2024-09-17T05:39:21.820 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:39:21.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- 172.21.15.122:0/4266003491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1614004010 msgr2=0x7f1614004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:21.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 --2- 172.21.15.122:0/4266003491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1614004010 0x7f1614004810 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f161800c260 tx=0x7f161801f6c0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 -- 172.21.15.122:0/4266003491 reap_dead start 2024-09-17T05:39:21.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- 172.21.15.122:0/4266003491 shutdown_connections 2024-09-17T05:39:21.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- 172.21.15.122:0/4266003491 >> 172.21.15.122:0/4266003491 conn(0x7f162c0cfa30 msgr2=0x7f162c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:21.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- 172.21.15.122:0/4266003491 shutdown_connections 2024-09-17T05:39:21.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- 172.21.15.122:0/4266003491 wait complete. 2024-09-17T05:39:21.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 Processor -- start 2024-09-17T05:39:21.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- start start 2024-09-17T05:39:21.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1614004010 0x7f162c0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:21.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c147670 0x7f162c0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:21.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c0b0fa0 0x7f162c1f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:21.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f162c070090 con 0x7f162c0b0fa0 2024-09-17T05:39:21.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0adff0 con 0x7f1614004010 2024-09-17T05:39:21.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1632800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f162c070210 con 0x7f162c147670 2024-09-17T05:39:21.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c0b0fa0 0x7f162c1f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:21.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c0b0fa0 0x7f162c1f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48282/0 (socket says 172.21.15.122:48282) 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 -- 172.21.15.122:0/2175302751 learned_addr learned my addr 172.21.15.122:0/2175302751 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631400640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1614004010 0x7f162c0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1630a00640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c147670 0x7f162c0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 -- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c147670 msgr2=0x7f162c0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c147670 0x7f162c0b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 -- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1614004010 msgr2=0x7f162c0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1614004010 0x7f162c0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 -- 172.21.15.122:0/2175302751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c0c54d0 con 0x7f162c0b0fa0 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631400640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1614004010 0x7f162c0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1630a00640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c147670 0x7f162c0b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1631e00640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c0b0fa0 0x7f162c1f7f30 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f162c161860 tx=0x7f1624001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.970+0000 7f1629600640 1 -- 172.21.15.122:0/2175302751 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624011630 con 0x7f162c0b0fa0 2024-09-17T05:39:21.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1629600640 1 -- 172.21.15.122:0/2175302751 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1624018070 con 0x7f162c0b0fa0 2024-09-17T05:39:21.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1629600640 1 -- 172.21.15.122:0/2175302751 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624011a70 con 0x7f162c0b0fa0 2024-09-17T05:39:21.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c0c5da0 con 0x7f162c0b0fa0 2024-09-17T05:39:21.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c0c6360 con 0x7f162c0b0fa0 2024-09-17T05:39:21.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c0cd600 con 0x7f162c0b0fa0 2024-09-17T05:39:21.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1629600640 1 -- 172.21.15.122:0/2175302751 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f1624011c10 con 0x7f162c0b0fa0 2024-09-17T05:39:21.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1629600640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0751b0 0x7f160c077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:21.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1629600640 1 -- 172.21.15.122:0/2175302751 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f16240937a0 con 0x7f162c0b0fa0 2024-09-17T05:39:21.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.974+0000 7f1631400640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0751b0 0x7f160c077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:21.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.978+0000 7f1631400640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0751b0 0x7f160c077670 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f162c06af90 tx=0x7f1620009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:21.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:21.978+0000 7f1629600640 1 -- 172.21.15.122:0/2175302751 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1624060350 con 0x7f162c0b0fa0 2024-09-17T05:39:22.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.166+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f162c0c6530 con 0x7f160c0751b0 2024-09-17T05:39:22.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.170+0000 7f1629600640 1 -- 172.21.15.122:0/2175302751 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f162c0c6530 con 0x7f160c0751b0 2024-09-17T05:39:22.167 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:39:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:39:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:39:22.167 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:39:22.168 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:39:22.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0751b0 msgr2=0x7f160c077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0751b0 0x7f160c077670 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f162c06af90 tx=0x7f1620009500 comp rx=0 tx=0).stop 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c0b0fa0 msgr2=0x7f162c1f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c0b0fa0 0x7f162c1f7f30 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f162c161860 tx=0x7f1624001640 comp rx=0 tx=0).stop 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 shutdown_connections 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0751b0 0x7f160c077670 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c0b0fa0 0x7f162c1f7f30 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c147670 0x7f162c0b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 --2- 172.21.15.122:0/2175302751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1614004010 0x7f162c0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 >> 172.21.15.122:0/2175302751 conn(0x7f162c0cfa30 msgr2=0x7f162c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 shutdown_connections 2024-09-17T05:39:22.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.174+0000 7f1632800640 1 -- 172.21.15.122:0/2175302751 wait complete. 2024-09-17T05:39:22.189 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:39:22.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 -- 172.21.15.122:0/397338660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefb415bd50 msgr2=0x7fefb40ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 --2- 172.21.15.122:0/397338660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefb415bd50 0x7fefb40ce5e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009c80 tx=0x7fefa801cc80 comp rx=0 tx=0).stop 2024-09-17T05:39:22.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 -- 172.21.15.122:0/397338660 shutdown_connections 2024-09-17T05:39:22.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 --2- 172.21.15.122:0/397338660 >> v2:172.21.15.173:3300/0 conn(0x7fefb415c130 0x7fefb40ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 --2- 172.21.15.122:0/397338660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefb415bd50 0x7fefb40ce5e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 -- 172.21.15.122:0/397338660 >> 172.21.15.122:0/397338660 conn(0x7fefb40d1ea0 msgr2=0x7fefb40d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:22.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 -- 172.21.15.122:0/397338660 shutdown_connections 2024-09-17T05:39:22.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 -- 172.21.15.122:0/397338660 wait complete. 2024-09-17T05:39:22.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.338+0000 7fefbc200640 1 Processor -- start 2024-09-17T05:39:22.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 -- start start 2024-09-17T05:39:22.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefb40cf4e0 0x7fefb414b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:22.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefb415bd50 0x7fefb41f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:22.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefb415c130 0x7fefb4071230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:22.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4168e50 con 0x7fefb415bd50 2024-09-17T05:39:22.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4168cd0 con 0x7fefb40cf4e0 2024-09-17T05:39:22.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fefb4168fd0 con 0x7fefb415c130 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefb415bd50 0x7fefb41f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb9800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefb40cf4e0 0x7fefb414b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefb415bd50 0x7fefb41f7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48286/0 (socket says 172.21.15.122:48286) 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 -- 172.21.15.122:0/2447527296 learned_addr learned my addr 172.21.15.122:0/2447527296 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefba200640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefb415c130 0x7fefb4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 -- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefb415c130 msgr2=0x7fefb4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefb415c130 0x7fefb4071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 -- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefb40cf4e0 msgr2=0x7fefb414b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefb40cf4e0 0x7fefb414b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 -- 172.21.15.122:0/2447527296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb409dd20 con 0x7fefb415bd50 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb8e00640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefb415bd50 0x7fefb41f7a30 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e7a0 tx=0x7fefa000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:22.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb1600640 1 -- 172.21.15.122:0/2447527296 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa000cc60 con 0x7fefb415bd50 2024-09-17T05:39:22.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb1600640 1 -- 172.21.15.122:0/2447527296 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefa0010430 con 0x7fefb415bd50 2024-09-17T05:39:22.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefb1600640 1 -- 172.21.15.122:0/2447527296 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0010860 con 0x7fefb415bd50 2024-09-17T05:39:22.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb414b990 con 0x7fefb415bd50 2024-09-17T05:39:22.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41f8210 con 0x7fefb415bd50 2024-09-17T05:39:22.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.342+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb40c5dd0 con 0x7fefb415bd50 2024-09-17T05:39:22.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.346+0000 7fefb1600640 1 -- 172.21.15.122:0/2447527296 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7fefa0010a00 con 0x7fefb415bd50 2024-09-17T05:39:22.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.346+0000 7fefb1600640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fef940751b0 0x7fef94077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:22.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.346+0000 7fefb1600640 1 -- 172.21.15.122:0/2447527296 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7fefa0093bd0 con 0x7fefb415bd50 2024-09-17T05:39:22.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.346+0000 7fefb9800640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fef940751b0 0x7fef94077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:22.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.346+0000 7fefb9800640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fef940751b0 0x7fef94077670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fefa80af000 tx=0x7fefa801b4c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:22.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.350+0000 7fefb1600640 1 -- 172.21.15.122:0/2447527296 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fefb409dd20 con 0x7fefb415bd50 2024-09-17T05:39:22.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.518+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fefb4072590 con 0x7fefb415bd50 2024-09-17T05:39:22.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.518+0000 7fefb1600640 1 -- 172.21.15.122:0/2447527296 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7fefa0060090 con 0x7fefb415bd50 2024-09-17T05:39:22.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fef940751b0 msgr2=0x7fef94077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fef940751b0 0x7fef94077670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fefa80af000 tx=0x7fefa801b4c0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefb415bd50 msgr2=0x7fefb41f7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefb415bd50 0x7fefb41f7a30 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e7a0 tx=0x7fefa000ec60 comp rx=0 tx=0).stop 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 shutdown_connections 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fef940751b0 0x7fef94077670 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefb415c130 0x7fefb4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefb415bd50 0x7fefb41f7a30 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 --2- 172.21.15.122:0/2447527296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefb40cf4e0 0x7fefb414b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 >> 172.21.15.122:0/2447527296 conn(0x7fefb40d1ea0 msgr2=0x7fefb40d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:22.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 shutdown_connections 2024-09-17T05:39:22.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.522+0000 7fefbc200640 1 -- 172.21.15.122:0/2447527296 wait complete. 2024-09-17T05:39:22.538 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:39:22.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 -- 172.21.15.122:0/3446631694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c15bd40 msgr2=0x7f7b5c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 --2- 172.21.15.122:0/3446631694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c15bd40 0x7f7b5c0d0240 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009840 tx=0x7f7b4c01c820 comp rx=0 tx=0).stop 2024-09-17T05:39:22.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 -- 172.21.15.122:0/3446631694 shutdown_connections 2024-09-17T05:39:22.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 --2- 172.21.15.122:0/3446631694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c15bd40 0x7f7b5c0d0240 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 -- 172.21.15.122:0/3446631694 >> 172.21.15.122:0/3446631694 conn(0x7f7b5c09aa10 msgr2=0x7f7b5c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:22.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 -- 172.21.15.122:0/3446631694 shutdown_connections 2024-09-17T05:39:22.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 -- 172.21.15.122:0/3446631694 wait complete. 2024-09-17T05:39:22.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 Processor -- start 2024-09-17T05:39:22.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.686+0000 7f7b5be00640 1 -- start start 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b5c0b0d10 0x7f7b5c14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c0d22c0 0x7f7b5c14d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b5c15bd40 0x7f7b5c152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c169e20 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5c169ca0 con 0x7f7b5c15bd40 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5c169fa0 con 0x7f7b5c0b0d10 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c0d22c0 0x7f7b5c14d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c0d22c0 0x7f7b5c14d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48302/0 (socket says 172.21.15.122:48302) 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 -- 172.21.15.122:0/1772097586 learned_addr learned my addr 172.21.15.122:0/1772097586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5aa00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b5c0b0d10 0x7f7b5c14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:22.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 -- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b5c0b0d10 msgr2=0x7f7b5c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b5c0b0d10 0x7f7b5c14dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 -- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b5c15bd40 msgr2=0x7f7b5c152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b5c15bd40 0x7f7b5c152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 -- 172.21.15.122:0/1772097586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5c14d870 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5a000640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c0d22c0 0x7f7b5c14d330 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c4b0 tx=0x7f7b4800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:22.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b52a00640 1 -- 172.21.15.122:0/1772097586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48016020 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b52a00640 1 -- 172.21.15.122:0/1772097586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b48012920 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5c071b80 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b52a00640 1 -- 172.21.15.122:0/1772097586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48012d90 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5c0cb6f0 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.690+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5c072030 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.694+0000 7f7b52a00640 1 -- 172.21.15.122:0/1772097586 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f7b48019020 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.694+0000 7f7b52a00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7b3c075180 0x7f7b3c077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:22.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.694+0000 7f7b52a00640 1 -- 172.21.15.122:0/1772097586 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f7b480932d0 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.694+0000 7f7b5aa00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7b3c075180 0x7f7b3c077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:22.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.694+0000 7f7b5aa00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7b3c075180 0x7f7b3c077640 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0af000 tx=0x7f7b4c01b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:22.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.698+0000 7f7b52a00640 1 -- 172.21.15.122:0/1772097586 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7b48020020 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.870+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f7b5c075d00 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b52a00640 1 -- 172.21.15.122:0/1772097586 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v51) ==== 116+0+2 (secure 0 0 0) 0x7f7b4805fde0 con 0x7f7b5c0d22c0 2024-09-17T05:39:22.868 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:39:22.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7b3c075180 msgr2=0x7f7b3c077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7b3c075180 0x7f7b3c077640 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0af000 tx=0x7f7b4c01b060 comp rx=0 tx=0).stop 2024-09-17T05:39:22.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c0d22c0 msgr2=0x7f7b5c14d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:22.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c0d22c0 0x7f7b5c14d330 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c4b0 tx=0x7f7b4800c970 comp rx=0 tx=0).stop 2024-09-17T05:39:22.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 shutdown_connections 2024-09-17T05:39:22.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7b3c075180 0x7f7b3c077640 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b5c15bd40 0x7f7b5c152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b5c0d22c0 0x7f7b5c14d330 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 --2- 172.21.15.122:0/1772097586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b5c0b0d10 0x7f7b5c14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:22.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 >> 172.21.15.122:0/1772097586 conn(0x7f7b5c09aa10 msgr2=0x7f7b5c0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:22.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 shutdown_connections 2024-09-17T05:39:22.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:22.874+0000 7f7b5be00640 1 -- 172.21.15.122:0/1772097586 wait complete. 2024-09-17T05:39:22.888 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:39:23.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.038+0000 7f3853600640 1 -- 172.21.15.122:0/4083718952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c1605d0 msgr2=0x7f384c0d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.038+0000 7f3853600640 1 --2- 172.21.15.122:0/4083718952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c1605d0 0x7f384c0d0690 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f384c0754a0 tx=0x7f383c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.038+0000 7f3853600640 1 -- 172.21.15.122:0/4083718952 shutdown_connections 2024-09-17T05:39:23.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.038+0000 7f3853600640 1 --2- 172.21.15.122:0/4083718952 >> v2:172.21.15.173:3300/0 conn(0x7f384c0afb10 0x7f384c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.038+0000 7f3853600640 1 --2- 172.21.15.122:0/4083718952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c1605d0 0x7f384c0d0690 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.038+0000 7f3853600640 1 -- 172.21.15.122:0/4083718952 >> 172.21.15.122:0/4083718952 conn(0x7f384c0d1dd0 msgr2=0x7f384c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:23.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.038+0000 7f3853600640 1 -- 172.21.15.122:0/4083718952 shutdown_connections 2024-09-17T05:39:23.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.038+0000 7f3853600640 1 -- 172.21.15.122:0/4083718952 wait complete. 2024-09-17T05:39:23.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 Processor -- start 2024-09-17T05:39:23.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 -- start start 2024-09-17T05:39:23.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f384c0afb10 0x7f384c0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c0b1650 0x7f384c0cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384c1605d0 0x7f384c0ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f384c169540 con 0x7f384c0b1650 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0b1430 con 0x7f384c1605d0 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f384c1696c0 con 0x7f384c0afb10 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c0b1650 0x7f384c0cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c0b1650 0x7f384c0cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48316/0 (socket says 172.21.15.122:48316) 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 -- 172.21.15.122:0/2508829740 learned_addr learned my addr 172.21.15.122:0/2508829740 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3850c00640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f384c0afb10 0x7f384c0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3851600640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384c1605d0 0x7f384c0ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 -- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f384c0afb10 msgr2=0x7f384c0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f384c0afb10 0x7f384c0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 -- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384c1605d0 msgr2=0x7f384c0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384c1605d0 0x7f384c0ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 -- 172.21.15.122:0/2508829740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c157350 con 0x7f384c0b1650 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f384be00640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c0b1650 0x7f384c0cdf20 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f383800c500 tx=0x7f383800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3848c00640 1 -- 172.21.15.122:0/2508829740 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838017070 con 0x7f384c0b1650 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3848c00640 1 -- 172.21.15.122:0/2508829740 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38380053c0 con 0x7f384c0b1650 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3848c00640 1 -- 172.21.15.122:0/2508829740 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838012890 con 0x7f384c0b1650 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c157630 con 0x7f384c0b1650 2024-09-17T05:39:23.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c157bf0 con 0x7f384c0b1650 2024-09-17T05:39:23.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.042+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384c0affc0 con 0x7f384c0b1650 2024-09-17T05:39:23.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.046+0000 7f3848c00640 1 -- 172.21.15.122:0/2508829740 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f383801a070 con 0x7f384c0b1650 2024-09-17T05:39:23.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.046+0000 7f3848c00640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f382c075180 0x7f382c077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.046+0000 7f3848c00640 1 -- 172.21.15.122:0/2508829740 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f3838092580 con 0x7f384c0b1650 2024-09-17T05:39:23.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.046+0000 7f3850c00640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f382c075180 0x7f382c077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.046+0000 7f3850c00640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f382c075180 0x7f382c077640 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f383c0af000 tx=0x7f383c0a7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:23.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.050+0000 7f3848c00640 1 -- 172.21.15.122:0/2508829740 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f383805e9c0 con 0x7f384c0b1650 2024-09-17T05:39:23.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.218+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f384c0d1080 con 0x7f384c0b1650 2024-09-17T05:39:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.222+0000 7f3848c00640 1 -- 172.21.15.122:0/2508829740 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7f38380633f0 con 0x7f384c0b1650 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f382c075180 msgr2=0x7f382c077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f382c075180 0x7f382c077640 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f383c0af000 tx=0x7f383c0a7040 comp rx=0 tx=0).stop 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c0b1650 msgr2=0x7f384c0cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c0b1650 0x7f384c0cdf20 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f383800c500 tx=0x7f383800c9c0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 shutdown_connections 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f382c075180 0x7f382c077640 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384c1605d0 0x7f384c0ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f384c0b1650 0x7f384c0cdf20 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 --2- 172.21.15.122:0/2508829740 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f384c0afb10 0x7f384c0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 >> 172.21.15.122:0/2508829740 conn(0x7f384c0d1dd0 msgr2=0x7f384c0cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:23.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 shutdown_connections 2024-09-17T05:39:23.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.226+0000 7f3853600640 1 -- 172.21.15.122:0/2508829740 wait complete. 2024-09-17T05:39:23.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T05:39:23.242 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.244+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.242 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.244+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.242 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.244+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.242 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.244+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.242 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.244+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.394+0000 7ff2b4c00640 1 -- 172.21.15.122:0/858568834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff294003a60 msgr2=0x7ff294003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.394+0000 7ff2b4c00640 1 --2- 172.21.15.122:0/858568834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff294003a60 0x7ff294003e80 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c2e0 tx=0x7ff2a001cf60 comp rx=0 tx=0).stop 2024-09-17T05:39:23.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.394+0000 7ff2ae800640 1 -- 172.21.15.122:0/858568834 reap_dead start 2024-09-17T05:39:23.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.394+0000 7ff2b4c00640 1 -- 172.21.15.122:0/858568834 shutdown_connections 2024-09-17T05:39:23.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.394+0000 7ff2b4c00640 1 -- 172.21.15.122:0/858568834 >> 172.21.15.122:0/858568834 conn(0x7ff2b006dd30 msgr2=0x7ff2b006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:23.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.394+0000 7ff2b4c00640 1 -- 172.21.15.122:0/858568834 shutdown_connections 2024-09-17T05:39:23.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.394+0000 7ff2b4c00640 1 -- 172.21.15.122:0/858568834 wait complete. 2024-09-17T05:39:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 Processor -- start 2024-09-17T05:39:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 -- start start 2024-09-17T05:39:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff294003a60 0x7ff2b0146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b01fbfd0 0x7ff2b0144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b01fb2e0 0x7ff2b01fb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0166cb0 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b0166b30 con 0x7ff2b01fb2e0 2024-09-17T05:39:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b0166e30 con 0x7ff294003a60 2024-09-17T05:39:23.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b01fbfd0 0x7ff2b0144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b01fbfd0 0x7ff2b0144050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48338/0 (socket says 172.21.15.122:48338) 2024-09-17T05:39:23.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 -- 172.21.15.122:0/4048287523 learned_addr learned my addr 172.21.15.122:0/4048287523 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:23.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ade00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff294003a60 0x7ff2b0146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ae800640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b01fb2e0 0x7ff2b01fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 -- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff294003a60 msgr2=0x7ff2b0146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff294003a60 0x7ff2b0146020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 -- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b01fb2e0 msgr2=0x7ff2b01fb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b01fb2e0 0x7ff2b01fb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 -- 172.21.15.122:0/4048287523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b0146780 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ae800640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b01fb2e0 0x7ff2b01fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:23.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2ad400640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b01fbfd0 0x7ff2b0144050 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff29c011ae0 tx=0x7ff29c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:23.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2a6000640 1 -- 172.21.15.122:0/4048287523 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c01e070 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2a6000640 1 -- 172.21.15.122:0/4048287523 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff29c00fc70 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2a6000640 1 -- 172.21.15.122:0/4048287523 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c019670 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b0146a60 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b0147020 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.398+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b00b5a90 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.402+0000 7ff2a6000640 1 -- 172.21.15.122:0/4048287523 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7ff29c021070 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.402+0000 7ff2a6000640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff2900751b0 0x7ff290077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.402+0000 7ff2a6000640 1 -- 172.21.15.122:0/4048287523 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7ff29c093990 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.402+0000 7ff2ade00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff2900751b0 0x7ff290077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.402+0000 7ff2ade00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff2900751b0 0x7ff290077670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c2e0 tx=0x7ff2a0019da0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:23.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.406+0000 7ff2a6000640 1 -- 172.21.15.122:0/4048287523 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff29c00fe60 con 0x7ff2b01fbfd0 2024-09-17T05:39:23.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.594+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7ff2b0147390 con 0x7ff2900751b0 2024-09-17T05:39:23.603 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.604+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.603 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.604+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.603 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.604+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.603 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.604+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.603 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:23.604+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:23.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.642+0000 7ff2a6000640 1 -- 172.21.15.122:0/4048287523 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff2b0147390 con 0x7ff2900751b0 2024-09-17T05:39:23.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff2900751b0 msgr2=0x7ff290077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff2900751b0 0x7ff290077670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c2e0 tx=0x7ff2a0019da0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b01fbfd0 msgr2=0x7ff2b0144050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b01fbfd0 0x7ff2b0144050 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff29c011ae0 tx=0x7ff29c009ec0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 shutdown_connections 2024-09-17T05:39:23.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff2900751b0 0x7ff290077670 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b01fb2e0 0x7ff2b01fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2b01fbfd0 0x7ff2b0144050 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 --2- 172.21.15.122:0/4048287523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff294003a60 0x7ff2b0146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 >> 172.21.15.122:0/4048287523 conn(0x7ff2b006dd30 msgr2=0x7ff2b006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:23.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 shutdown_connections 2024-09-17T05:39:23.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.646+0000 7ff2b4c00640 1 -- 172.21.15.122:0/4048287523 wait complete. 2024-09-17T05:39:23.660 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:39:23.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.810+0000 7feb38600640 1 -- 172.21.15.122:0/3057226373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb18003a60 msgr2=0x7feb18003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.810+0000 7feb38600640 1 --2- 172.21.15.122:0/3057226373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb18003a60 0x7feb18003e80 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c2e0 tx=0x7feb2401cf20 comp rx=0 tx=0).stop 2024-09-17T05:39:23.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.810+0000 7feb38600640 1 -- 172.21.15.122:0/3057226373 shutdown_connections 2024-09-17T05:39:23.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.810+0000 7feb38600640 1 --2- 172.21.15.122:0/3057226373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb18003a60 0x7feb18003e80 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.810+0000 7feb38600640 1 -- 172.21.15.122:0/3057226373 >> 172.21.15.122:0/3057226373 conn(0x7feb300d1dd0 msgr2=0x7feb300d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:23.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.810+0000 7feb38600640 1 -- 172.21.15.122:0/3057226373 shutdown_connections 2024-09-17T05:39:23.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.810+0000 7feb38600640 1 -- 172.21.15.122:0/3057226373 wait complete. 2024-09-17T05:39:23.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb38600640 1 Processor -- start 2024-09-17T05:39:23.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb38600640 1 -- start start 2024-09-17T05:39:23.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb38600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb18003a60 0x7feb300cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb38600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb301fba10 0x7feb300cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb38600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb300c49d0 0x7feb300c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb38600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb300b0e50 con 0x7feb301fba10 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb38600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb300b0cd0 con 0x7feb18003a60 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb38600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7feb300b1220 con 0x7feb300c49d0 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb301fba10 0x7feb300cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb301fba10 0x7feb300cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48358/0 (socket says 172.21.15.122:48358) 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 -- 172.21.15.122:0/619622697 learned_addr learned my addr 172.21.15.122:0/619622697 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35c00640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb18003a60 0x7feb300cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb36600640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb300c49d0 0x7feb300c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 -- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb300c49d0 msgr2=0x7feb300c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb300c49d0 0x7feb300c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 -- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb18003a60 msgr2=0x7feb300cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb18003a60 0x7feb300cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 -- 172.21.15.122:0/619622697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb301fbfd0 con 0x7feb301fba10 2024-09-17T05:39:23.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.814+0000 7feb35200640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb301fba10 0x7feb300cb9d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7feb1c004da0 tx=0x7feb1c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.818+0000 7feb2d600640 1 -- 172.21.15.122:0/619622697 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c001d50 con 0x7feb301fba10 2024-09-17T05:39:23.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.818+0000 7feb2d600640 1 -- 172.21.15.122:0/619622697 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb1c016070 con 0x7feb301fba10 2024-09-17T05:39:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.818+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb301fc2b0 con 0x7feb301fba10 2024-09-17T05:39:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.818+0000 7feb2d600640 1 -- 172.21.15.122:0/619622697 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c010b40 con 0x7feb301fba10 2024-09-17T05:39:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.818+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb301fc870 con 0x7feb301fba10 2024-09-17T05:39:23.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.818+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb300aeb00 con 0x7feb301fba10 2024-09-17T05:39:23.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.818+0000 7feb2d600640 1 -- 172.21.15.122:0/619622697 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7feb1c010ce0 con 0x7feb301fba10 2024-09-17T05:39:23.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.822+0000 7feb2d600640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb10075180 0x7feb10077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:23.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.822+0000 7feb2d600640 1 -- 172.21.15.122:0/619622697 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7feb1c093a00 con 0x7feb301fba10 2024-09-17T05:39:23.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.822+0000 7feb35c00640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb10075180 0x7feb10077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:23.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.822+0000 7feb35c00640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb10075180 0x7feb10077640 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7feb240ab000 tx=0x7feb24019d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:23.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:23.826+0000 7feb2d600640 1 -- 172.21.15.122:0/619622697 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7feb1c060510 con 0x7feb301fba10 2024-09-17T05:39:24.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.018+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7feb300d1080 con 0x7feb10075180 2024-09-17T05:39:24.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb2d600640 1 -- 172.21.15.122:0/619622697 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7feb300d1080 con 0x7feb10075180 2024-09-17T05:39:24.021 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:24.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb10075180 msgr2=0x7feb10077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:24.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb10075180 0x7feb10077640 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7feb240ab000 tx=0x7feb24019d60 comp rx=0 tx=0).stop 2024-09-17T05:39:24.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb301fba10 msgr2=0x7feb300cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:24.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb301fba10 0x7feb300cb9d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7feb1c004da0 tx=0x7feb1c001640 comp rx=0 tx=0).stop 2024-09-17T05:39:24.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 shutdown_connections 2024-09-17T05:39:24.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7feb10075180 0x7feb10077640 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb300c49d0 0x7feb300c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb301fba10 0x7feb300cb9d0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 --2- 172.21.15.122:0/619622697 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb18003a60 0x7feb300cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 >> 172.21.15.122:0/619622697 conn(0x7feb300d1dd0 msgr2=0x7feb30070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:24.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 shutdown_connections 2024-09-17T05:39:24.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.026+0000 7feb38600640 1 -- 172.21.15.122:0/619622697 wait complete. 2024-09-17T05:39:24.043 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:39:24.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- 172.21.15.122:0/3613570442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59b4004810 msgr2=0x7f59b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:24.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 --2- 172.21.15.122:0/3613570442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59b4004810 0x7f59b4004c90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00c2e0 tx=0x7f59bc01cc40 comp rx=0 tx=0).stop 2024-09-17T05:39:24.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- 172.21.15.122:0/3613570442 shutdown_connections 2024-09-17T05:39:24.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 --2- 172.21.15.122:0/3613570442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59b4004810 0x7f59b4004c90 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- 172.21.15.122:0/3613570442 >> 172.21.15.122:0/3613570442 conn(0x7f59d00d1ea0 msgr2=0x7f59d00d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:24.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- 172.21.15.122:0/3613570442 shutdown_connections 2024-09-17T05:39:24.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- 172.21.15.122:0/3613570442 wait complete. 2024-09-17T05:39:24.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 Processor -- start 2024-09-17T05:39:24.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- start start 2024-09-17T05:39:24.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59b4004810 0x7f59d0158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:24.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59d00c7e20 0x7f59d0156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:24.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59d006f2c0 0x7f59d006f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:24.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00cca80 con 0x7f59d006f2c0 2024-09-17T05:39:24.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59d00cc900 con 0x7f59d00c7e20 2024-09-17T05:39:24.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.194+0000 7f59d5000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f59d00ccc00 con 0x7f59b4004810 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59d006f2c0 0x7f59d006f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59d006f2c0 0x7f59d006f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48380/0 (socket says 172.21.15.122:48380) 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 -- 172.21.15.122:0/170590391 learned_addr learned my addr 172.21.15.122:0/170590391 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59cd400640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59d00c7e20 0x7f59d0156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59cde00640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59b4004810 0x7f59d0158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 -- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59b4004810 msgr2=0x7f59d0158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59b4004810 0x7f59d0158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 -- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59d00c7e20 msgr2=0x7f59d0156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59d00c7e20 0x7f59d0156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 -- 172.21.15.122:0/170590391 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d01fd140 con 0x7f59d006f2c0 2024-09-17T05:39:24.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59ce800640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59d006f2c0 0x7f59d006f720 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f59c8010910 tx=0x7f59c8010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:24.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59c6000640 1 -- 172.21.15.122:0/170590391 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c800ec60 con 0x7f59d006f2c0 2024-09-17T05:39:24.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59c6000640 1 -- 172.21.15.122:0/170590391 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59c8012450 con 0x7f59d006f2c0 2024-09-17T05:39:24.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59c6000640 1 -- 172.21.15.122:0/170590391 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80128a0 con 0x7f59d006f2c0 2024-09-17T05:39:24.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d01fc530 con 0x7f59d006f2c0 2024-09-17T05:39:24.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d01fcaf0 con 0x7f59d006f2c0 2024-09-17T05:39:24.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d00cd190 con 0x7f59d006f2c0 2024-09-17T05:39:24.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.198+0000 7f59c6000640 1 -- 172.21.15.122:0/170590391 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 48) ==== 100090+0+0 (secure 0 0 0) 0x7f59c801a020 con 0x7f59d006f2c0 2024-09-17T05:39:24.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.202+0000 7f59c6000640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59b00751b0 0x7f59b0077670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:24.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.202+0000 7f59c6000640 1 -- 172.21.15.122:0/170590391 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f59c8093aa0 con 0x7f59d006f2c0 2024-09-17T05:39:24.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.202+0000 7f59cde00640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59b00751b0 0x7f59b0077670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:24.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.202+0000 7f59cde00640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59b00751b0 0x7f59b0077670 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f59d01ec1d0 tx=0x7f59c0006d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:24.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.206+0000 7f59c6000640 1 -- 172.21.15.122:0/170590391 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f59c8060000 con 0x7f59d006f2c0 2024-09-17T05:39:24.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.398+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f59d01f8c80 con 0x7f59b00751b0 2024-09-17T05:39:24.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59c6000640 1 -- 172.21.15.122:0/170590391 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f59d01f8c80 con 0x7f59b00751b0 2024-09-17T05:39:24.397 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:24.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59b00751b0 msgr2=0x7f59b0077670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:24.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59b00751b0 0x7f59b0077670 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f59d01ec1d0 tx=0x7f59c0006d60 comp rx=0 tx=0).stop 2024-09-17T05:39:24.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59d006f2c0 msgr2=0x7f59d006f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:24.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59d006f2c0 0x7f59d006f720 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f59c8010910 tx=0x7f59c8010dd0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 shutdown_connections 2024-09-17T05:39:24.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59b00751b0 0x7f59b0077670 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59d006f2c0 0x7f59d006f720 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59d00c7e20 0x7f59d0156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 --2- 172.21.15.122:0/170590391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59b4004810 0x7f59d0158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:24.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.402+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 >> 172.21.15.122:0/170590391 conn(0x7f59d00d1ea0 msgr2=0x7f59d01ec390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:24.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.406+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 shutdown_connections 2024-09-17T05:39:24.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:24.406+0000 7f59d5000640 1 -- 172.21.15.122:0/170590391 wait complete. 2024-09-17T05:39:24.419 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:24.420 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:39:24.420 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:39:24.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:24.429 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:39:33.138 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:39:33.138 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:39:33.138 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.65189 s, 124 MB/s 2024-09-17T05:39:33.139 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:39:33.139 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:39:33.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:33.148 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:39:34.174 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:39:34.177 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:39:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:39:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:39:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.56025 s, 112 MB/s 2024-09-17T05:39:42.767 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:39:42.767 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:39:42.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:42.776 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:39:52.123 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:39:52.123 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:39:52.123 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.2938 s, 116 MB/s 2024-09-17T05:39:52.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T05:39:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.277+0000 7f3f24c00640 1 -- 172.21.15.122:0/2099354837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f200b0b60 msgr2=0x7f3f200ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.277+0000 7f3f24c00640 1 --2- 172.21.15.122:0/2099354837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f200b0b60 0x7f3f200ccbd0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3f200746b0 tx=0x7f3f1801eff0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.277+0000 7f3f24c00640 1 -- 172.21.15.122:0/2099354837 shutdown_connections 2024-09-17T05:39:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.277+0000 7f3f24c00640 1 --2- 172.21.15.122:0/2099354837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f200b0b60 0x7f3f200ccbd0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.277+0000 7f3f24c00640 1 --2- 172.21.15.122:0/2099354837 >> v2:172.21.15.173:3300/0 conn(0x7f3f2015c1f0 0x7f3f200cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.277+0000 7f3f24c00640 1 -- 172.21.15.122:0/2099354837 >> 172.21.15.122:0/2099354837 conn(0x7f3f200d1da0 msgr2=0x7f3f200d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.277+0000 7f3f24c00640 1 -- 172.21.15.122:0/2099354837 shutdown_connections 2024-09-17T05:39:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.277+0000 7f3f24c00640 1 -- 172.21.15.122:0/2099354837 wait complete. 2024-09-17T05:39:52.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 Processor -- start 2024-09-17T05:39:52.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 -- start start 2024-09-17T05:39:52.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f200b0b60 0x7f3f201f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:52.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f2015be10 0x7f3f200c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:52.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f2015c1f0 0x7f3f201f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:52.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200cd4a0 con 0x7f3f2015be10 2024-09-17T05:39:52.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200cd320 con 0x7f3f200b0b60 2024-09-17T05:39:52.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3f200cd620 con 0x7f3f2015c1f0 2024-09-17T05:39:52.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f2015be10 0x7f3f200c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:52.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f2015be10 0x7f3f200c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50474/0 (socket says 172.21.15.122:50474) 2024-09-17T05:39:52.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 -- 172.21.15.122:0/3605639296 learned_addr learned my addr 172.21.15.122:0/3605639296 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:52.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f1de00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f200b0b60 0x7f3f201f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:52.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f1e800640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f2015c1f0 0x7f3f201f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:52.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 -- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f2015c1f0 msgr2=0x7f3f201f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f2015c1f0 0x7f3f201f6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 -- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f200b0b60 msgr2=0x7f3f201f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f200b0b60 0x7f3f201f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 -- 172.21.15.122:0/3605639296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f201f3440 con 0x7f3f2015be10 2024-09-17T05:39:52.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f1de00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f200b0b60 0x7f3f201f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:39:52.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f15400640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f2015be10 0x7f3f200c8150 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800ca20 tx=0x7f3f0800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:52.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f16a00640 1 -- 172.21.15.122:0/3605639296 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08007c70 con 0x7f3f2015be10 2024-09-17T05:39:52.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f16a00640 1 -- 172.21.15.122:0/3605639296 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f0800f450 con 0x7f3f2015be10 2024-09-17T05:39:52.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f16a00640 1 -- 172.21.15.122:0/3605639296 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800f8a0 con 0x7f3f2015be10 2024-09-17T05:39:52.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f200b76a0 con 0x7f3f2015be10 2024-09-17T05:39:52.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.281+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201f2700 con 0x7f3f2015be10 2024-09-17T05:39:52.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.285+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f200cdb40 con 0x7f3f2015be10 2024-09-17T05:39:52.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.285+0000 7f3f16a00640 1 -- 172.21.15.122:0/3605639296 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 49) ==== 100192+0+0 (secure 0 0 0) 0x7f3f0800fa40 con 0x7f3f2015be10 2024-09-17T05:39:52.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.285+0000 7f3f16a00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3efc0752a0 0x7f3efc077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:52.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.285+0000 7f3f16a00640 1 -- 172.21.15.122:0/3605639296 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f3f08093970 con 0x7f3f2015be10 2024-09-17T05:39:52.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.285+0000 7f3f1de00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3efc0752a0 0x7f3efc077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:52.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.285+0000 7f3f1de00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3efc0752a0 0x7f3efc077760 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3f10001d20 tx=0x7f3f10001cb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:52.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.289+0000 7f3f16a00640 1 -- 172.21.15.122:0/3605639296 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3f201f3440 con 0x7f3f2015be10 2024-09-17T05:39:52.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.485+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f3f201f2550 con 0x7f3efc0752a0 2024-09-17T05:39:52.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f16a00640 1 -- 172.21.15.122:0/3605639296 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f201f2550 con 0x7f3efc0752a0 2024-09-17T05:39:52.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3efc0752a0 msgr2=0x7f3efc077760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3efc0752a0 0x7f3efc077760 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3f10001d20 tx=0x7f3f10001cb0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f2015be10 msgr2=0x7f3f200c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f2015be10 0x7f3f200c8150 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800ca20 tx=0x7f3f0800cee0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 shutdown_connections 2024-09-17T05:39:52.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3efc0752a0 0x7f3efc077760 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f2015c1f0 0x7f3f201f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f2015be10 0x7f3f200c8150 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 --2- 172.21.15.122:0/3605639296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f200b0b60 0x7f3f201f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 >> 172.21.15.122:0/3605639296 conn(0x7f3f200d1da0 msgr2=0x7f3f20070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:52.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 shutdown_connections 2024-09-17T05:39:52.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.501+0000 7f3f24c00640 1 -- 172.21.15.122:0/3605639296 wait complete. 2024-09-17T05:39:52.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:39:52.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 -- 172.21.15.122:0/753348223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7415bee0 msgr2=0x7f2a740cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 --2- 172.21.15.122:0/753348223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7415bee0 0x7f2a740cc330 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2a74070900 tx=0x7f2a6401f0b0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 -- 172.21.15.122:0/753348223 shutdown_connections 2024-09-17T05:39:52.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 --2- 172.21.15.122:0/753348223 >> v2:172.21.15.173:3300/0 conn(0x7f2a740cf400 0x7f2a740cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 --2- 172.21.15.122:0/753348223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a740ccdb0 0x7f2a7415fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 --2- 172.21.15.122:0/753348223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a7415bee0 0x7f2a740cc330 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 -- 172.21.15.122:0/753348223 >> 172.21.15.122:0/753348223 conn(0x7f2a740b18f0 msgr2=0x7f2a740b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 -- 172.21.15.122:0/753348223 shutdown_connections 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.665+0000 7f2a7b600640 1 -- 172.21.15.122:0/753348223 wait complete. 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 Processor -- start 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 -- start start 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a740ccdb0 0x7f2a741f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a740cf400 0x7f2a7415e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a7415bee0 0x7f2a741e5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a741601e0 con 0x7f2a740ccdb0 2024-09-17T05:39:52.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74160060 con 0x7f2a7415bee0 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2a74160360 con 0x7f2a740cf400 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a740ccdb0 0x7f2a741f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a740ccdb0 0x7f2a741f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50494/0 (socket says 172.21.15.122:50494) 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 -- 172.21.15.122:0/1069779247 learned_addr learned my addr 172.21.15.122:0/1069779247 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a6be00640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a740cf400 0x7f2a7415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a79600640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a7415bee0 0x7f2a741e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 -- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a740cf400 msgr2=0x7f2a7415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a740cf400 0x7f2a7415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 -- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a7415bee0 msgr2=0x7f2a741e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a7415bee0 0x7f2a741e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 -- 172.21.15.122:0/1069779247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a741f6c60 con 0x7f2a740ccdb0 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a78c00640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a740ccdb0 0x7f2a741f6720 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b3040 tx=0x7f2a64005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:52.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a71600640 1 -- 172.21.15.122:0/1069779247 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640b5040 con 0x7f2a740ccdb0 2024-09-17T05:39:52.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a71600640 1 -- 172.21.15.122:0/1069779247 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a64005140 con 0x7f2a740ccdb0 2024-09-17T05:39:52.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a71600640 1 -- 172.21.15.122:0/1069779247 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64005590 con 0x7f2a740ccdb0 2024-09-17T05:39:52.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7415d880 con 0x7f2a740ccdb0 2024-09-17T05:39:52.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7415de40 con 0x7f2a740ccdb0 2024-09-17T05:39:52.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.669+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7415ee60 con 0x7f2a740ccdb0 2024-09-17T05:39:52.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.673+0000 7f2a71600640 1 -- 172.21.15.122:0/1069779247 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 49) ==== 100192+0+0 (secure 0 0 0) 0x7f2a640c0060 con 0x7f2a740ccdb0 2024-09-17T05:39:52.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.673+0000 7f2a71600640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a500752a0 0x7f2a50077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:52.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.673+0000 7f2a71600640 1 -- 172.21.15.122:0/1069779247 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f2a6412fb10 con 0x7f2a740ccdb0 2024-09-17T05:39:52.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.673+0000 7f2a6be00640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a500752a0 0x7f2a50077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:52.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.673+0000 7f2a6be00640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a500752a0 0x7f2a50077760 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009600 tx=0x7f2a5c0091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:52.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.677+0000 7f2a71600640 1 -- 172.21.15.122:0/1069779247 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2a741f6c60 con 0x7f2a740ccdb0 2024-09-17T05:39:52.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.869+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2a74070a60 con 0x7f2a500752a0 2024-09-17T05:39:52.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.873+0000 7f2a71600640 1 -- 172.21.15.122:0/1069779247 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f2a74070a60 con 0x7f2a500752a0 2024-09-17T05:39:52.871 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a500752a0 msgr2=0x7f2a50077760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a500752a0 0x7f2a50077760 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009600 tx=0x7f2a5c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a740ccdb0 msgr2=0x7f2a741f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a740ccdb0 0x7f2a741f6720 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b3040 tx=0x7f2a64005780 comp rx=0 tx=0).stop 2024-09-17T05:39:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 shutdown_connections 2024-09-17T05:39:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2a500752a0 0x7f2a50077760 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a7415bee0 0x7f2a741e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a740cf400 0x7f2a7415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 --2- 172.21.15.122:0/1069779247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a740ccdb0 0x7f2a741f6720 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 >> 172.21.15.122:0/1069779247 conn(0x7f2a740b18f0 msgr2=0x7f2a740acd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 shutdown_connections 2024-09-17T05:39:52.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:52.877+0000 7f2a7b600640 1 -- 172.21.15.122:0/1069779247 wait complete. 2024-09-17T05:39:52.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:52.892 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:52.892 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:39:52.901 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:52.901 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T05:39:52.901 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:52.902 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T05:39:54.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:54.903 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:54.904 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:39:54.915 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:54.915 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3055190016" 2024-09-17T05:39:54.915 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:54.916 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-17T05:39:56.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:39:56.917 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:56.917 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:39:56.927 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:39:56.927 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T05:39:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:39:56.928 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T05:39:56.928 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-17T05:39:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.129+0000 7f6a73400640 1 -- 172.21.15.122:0/465530241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 msgr2=0x7f6a6c0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.129+0000 7f6a73400640 1 --2- 172.21.15.122:0/465530241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 0x7f6a6c0cebf0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c070d80 tx=0x7f6a5c01c5d0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.129+0000 7f6a73400640 1 -- 172.21.15.122:0/465530241 shutdown_connections 2024-09-17T05:39:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.129+0000 7f6a73400640 1 --2- 172.21.15.122:0/465530241 >> v2:172.21.15.173:3300/0 conn(0x7f6a6c06e810 0x7f6a6c0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.129+0000 7f6a73400640 1 --2- 172.21.15.122:0/465530241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a6c0b4bd0 0x7f6a6c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.129+0000 7f6a73400640 1 --2- 172.21.15.122:0/465530241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 0x7f6a6c0cebf0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.129+0000 7f6a73400640 1 -- 172.21.15.122:0/465530241 >> 172.21.15.122:0/465530241 conn(0x7f6a6c0b18f0 msgr2=0x7f6a6c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:57.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 -- 172.21.15.122:0/465530241 shutdown_connections 2024-09-17T05:39:57.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 -- 172.21.15.122:0/465530241 wait complete. 2024-09-17T05:39:57.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 Processor -- start 2024-09-17T05:39:57.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 -- start start 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a6c06e810 0x7f6a6c1f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a6c0b4bd0 0x7f6a6c09bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 0x7f6a6c0add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c06f100 con 0x7f6a6c15ade0 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c06ef80 con 0x7f6a6c06e810 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6a6c06f280 con 0x7f6a6c0b4bd0 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 0x7f6a6c0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 0x7f6a6c0add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50512/0 (socket says 172.21.15.122:50512) 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 -- 172.21.15.122:0/2643822582 learned_addr learned my addr 172.21.15.122:0/2643822582 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a70a00640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a6c06e810 0x7f6a6c1f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a63e00640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a6c0b4bd0 0x7f6a6c09bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 -- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a6c0b4bd0 msgr2=0x7f6a6c09bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a6c0b4bd0 0x7f6a6c09bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 -- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a6c06e810 msgr2=0x7f6a6c1f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a6c06e810 0x7f6a6c1f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 -- 172.21.15.122:0/2643822582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c1f8760 con 0x7f6a6c15ade0 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a71400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 0x7f6a6c0add90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400eb20 tx=0x7f6a6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a69600640 1 -- 172.21.15.122:0/2643822582 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64004b30 con 0x7f6a6c15ade0 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a69600640 1 -- 172.21.15.122:0/2643822582 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a64004cf0 con 0x7f6a6c15ade0 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a69600640 1 -- 172.21.15.122:0/2643822582 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64005280 con 0x7f6a6c15ade0 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.133+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6c152360 con 0x7f6a6c15ade0 2024-09-17T05:39:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.137+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c1528e0 con 0x7f6a6c15ade0 2024-09-17T05:39:57.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.137+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a6c15e900 con 0x7f6a6c15ade0 2024-09-17T05:39:57.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.137+0000 7f6a69600640 1 -- 172.21.15.122:0/2643822582 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 49) ==== 100192+0+0 (secure 0 0 0) 0x7f6a64005420 con 0x7f6a6c15ade0 2024-09-17T05:39:57.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.137+0000 7f6a69600640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a480752d0 0x7f6a48077790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:57.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.137+0000 7f6a69600640 1 -- 172.21.15.122:0/2643822582 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f6a64093ef0 con 0x7f6a6c15ade0 2024-09-17T05:39:57.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.137+0000 7f6a70a00640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a480752d0 0x7f6a48077790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:57.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.141+0000 7f6a70a00640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a480752d0 0x7f6a48077790 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0af000 tx=0x7f6a5c01ae10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:57.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.141+0000 7f6a69600640 1 -- 172.21.15.122:0/2643822582 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6a6c1f8760 con 0x7f6a6c15ade0 2024-09-17T05:39:57.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.345+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f6a6c06adb0 con 0x7f6a480752d0 2024-09-17T05:39:57.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a69600640 1 -- 172.21.15.122:0/2643822582 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a6c06adb0 con 0x7f6a480752d0 2024-09-17T05:39:57.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a480752d0 msgr2=0x7f6a48077790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:57.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a480752d0 0x7f6a48077790 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0af000 tx=0x7f6a5c01ae10 comp rx=0 tx=0).stop 2024-09-17T05:39:57.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 msgr2=0x7f6a6c0add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:57.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 0x7f6a6c0add90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400eb20 tx=0x7f6a6400c2d0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 shutdown_connections 2024-09-17T05:39:57.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a480752d0 0x7f6a48077790 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a6c15ade0 0x7f6a6c0add90 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a6c0b4bd0 0x7f6a6c09bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 --2- 172.21.15.122:0/2643822582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a6c06e810 0x7f6a6c1f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:57.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 >> 172.21.15.122:0/2643822582 conn(0x7f6a6c0b18f0 msgr2=0x7f6a6c0b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:57.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 shutdown_connections 2024-09-17T05:39:57.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:57.417+0000 7f6a73400640 1 -- 172.21.15.122:0/2643822582 wait complete. 2024-09-17T05:39:57.416 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.415+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.416 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.415+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.416 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.415+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.416 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.415+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.417 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.415+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.434 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.435+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.434 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.435+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.435 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.435+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.435 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.435+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:57.435 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:57.435+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:58.421 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:58.423+0000 7fc014e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:58.421 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:58.423+0000 7fc014e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:58.421 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:58.423+0000 7fc014e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:58.421 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:58.423+0000 7fc014e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:58.421 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:39:58.423+0000 7fc014e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:39:58.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-17T05:39:58.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- 172.21.15.122:0/3848443857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc0004810 msgr2=0x7f8dc0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:58.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 --2- 172.21.15.122:0/3848443857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc0004810 0x7f8dc0004c90 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009ee0 tx=0x7f8dc801d630 comp rx=0 tx=0).stop 2024-09-17T05:39:58.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de0e00640 1 -- 172.21.15.122:0/3848443857 reap_dead start 2024-09-17T05:39:58.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- 172.21.15.122:0/3848443857 shutdown_connections 2024-09-17T05:39:58.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- 172.21.15.122:0/3848443857 >> 172.21.15.122:0/3848443857 conn(0x7f8ddc0d1dd0 msgr2=0x7f8ddc0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:58.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- 172.21.15.122:0/3848443857 shutdown_connections 2024-09-17T05:39:58.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- 172.21.15.122:0/3848443857 wait complete. 2024-09-17T05:39:58.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 Processor -- start 2024-09-17T05:39:58.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- start start 2024-09-17T05:39:58.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc0004810 0x7f8ddc0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:58.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ddc1544a0 0x7f8ddc0cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:58.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ddc0c7aa0 0x7f8ddc0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:58.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc0b06d0 con 0x7f8ddc1544a0 2024-09-17T05:39:58.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc0b2b90 con 0x7f8ddc0c7aa0 2024-09-17T05:39:58.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8de2e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8ddc0b0cd0 con 0x7f8dc0004810 2024-09-17T05:39:58.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8ddb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ddc1544a0 0x7f8ddc0cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:58.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8ddb400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ddc1544a0 0x7f8ddc0cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50526/0 (socket says 172.21.15.122:50526) 2024-09-17T05:39:58.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.661+0000 7f8ddb400640 1 -- 172.21.15.122:0/4077328991 learned_addr learned my addr 172.21.15.122:0/4077328991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:58.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8ddbe00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc0004810 0x7f8ddc0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8ddb400640 1 -- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc0004810 msgr2=0x7f8ddc0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8ddb400640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc0004810 0x7f8ddc0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8ddb400640 1 -- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ddc0c7aa0 msgr2=0x7f8ddc0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8ddb400640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ddc0c7aa0 0x7f8ddc0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8ddb400640 1 -- 172.21.15.122:0/4077328991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc0b4e50 con 0x7f8ddc1544a0 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8ddb400640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ddc1544a0 0x7f8ddc0cbbb0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009ee0 tx=0x7f8dc8009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8dd3e00640 1 -- 172.21.15.122:0/4077328991 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80ad070 con 0x7f8ddc1544a0 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8dd3e00640 1 -- 172.21.15.122:0/4077328991 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dc8003c70 con 0x7f8ddc1544a0 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8dd3e00640 1 -- 172.21.15.122:0/4077328991 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8003e60 con 0x7f8ddc1544a0 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ddc0b5130 con 0x7f8ddc1544a0 2024-09-17T05:39:58.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.665+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ddc15b680 con 0x7f8ddc1544a0 2024-09-17T05:39:58.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.669+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db40052a0 con 0x7f8ddc1544a0 2024-09-17T05:39:58.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.669+0000 7f8dd3e00640 1 -- 172.21.15.122:0/4077328991 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 49) ==== 100192+0+0 (secure 0 0 0) 0x7f8dc80b0070 con 0x7f8ddc1544a0 2024-09-17T05:39:58.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.673+0000 7f8dd3e00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dbc0752d0 0x7f8dbc077790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:58.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.673+0000 7f8dd3e00640 1 -- 172.21.15.122:0/4077328991 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f8dc812e160 con 0x7f8ddc1544a0 2024-09-17T05:39:58.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.673+0000 7f8ddbe00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dbc0752d0 0x7f8dbc077790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:58.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.677+0000 7f8dd3e00640 1 -- 172.21.15.122:0/4077328991 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8dc8017340 con 0x7f8ddc1544a0 2024-09-17T05:39:58.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.681+0000 7f8ddbe00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dbc0752d0 0x7f8dbc077790 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4006fd0 tx=0x7f8dd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:58.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.957+0000 7f8dd3e00640 1 -- 172.21.15.122:0/4077328991 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7f8dc80a75d0 con 0x7f8ddc1544a0 2024-09-17T05:39:58.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:58.977+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f8db4000bd0 con 0x7f8dbc0752d0 2024-09-17T05:39:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8dd3e00640 1 -- 172.21.15.122:0/4077328991 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8db4000bd0 con 0x7f8dbc0752d0 2024-09-17T05:39:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dbc0752d0 msgr2=0x7f8dbc077790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dbc0752d0 0x7f8dbc077790 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4006fd0 tx=0x7f8dd4008040 comp rx=0 tx=0).stop 2024-09-17T05:39:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ddc1544a0 msgr2=0x7f8ddc0cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ddc1544a0 0x7f8ddc0cbbb0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009ee0 tx=0x7f8dc8009b40 comp rx=0 tx=0).stop 2024-09-17T05:39:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 shutdown_connections 2024-09-17T05:39:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8dbc0752d0 0x7f8dbc077790 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ddc0c7aa0 0x7f8ddc0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ddc1544a0 0x7f8ddc0cbbb0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:59.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 --2- 172.21.15.122:0/4077328991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc0004810 0x7f8ddc0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:59.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 >> 172.21.15.122:0/4077328991 conn(0x7f8ddc0d1dd0 msgr2=0x7f8ddc071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:59.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 shutdown_connections 2024-09-17T05:39:59.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.149+0000 7f8de2e00640 1 -- 172.21.15.122:0/4077328991 wait complete. 2024-09-17T05:39:59.165 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-17T05:39:59.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.313+0000 7f450b200640 1 -- 172.21.15.122:0/3229686620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 msgr2=0x7f45040d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:59.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.313+0000 7f450b200640 1 --2- 172.21.15.122:0/3229686620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 0x7f45040d0690 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009840 tx=0x7f44f401c820 comp rx=0 tx=0).stop 2024-09-17T05:39:59.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.313+0000 7f450b200640 1 -- 172.21.15.122:0/3229686620 shutdown_connections 2024-09-17T05:39:59.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.313+0000 7f450b200640 1 --2- 172.21.15.122:0/3229686620 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45040afb10 0x7f45040affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:59.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.313+0000 7f450b200640 1 --2- 172.21.15.122:0/3229686620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 0x7f45040d0690 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:59.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.313+0000 7f450b200640 1 -- 172.21.15.122:0/3229686620 >> 172.21.15.122:0/3229686620 conn(0x7f45040d1dd0 msgr2=0x7f45040d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:39:59.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.313+0000 7f450b200640 1 -- 172.21.15.122:0/3229686620 shutdown_connections 2024-09-17T05:39:59.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.313+0000 7f450b200640 1 -- 172.21.15.122:0/3229686620 wait complete. 2024-09-17T05:39:59.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 Processor -- start 2024-09-17T05:39:59.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 -- start start 2024-09-17T05:39:59.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45040afb10 0x7f45040b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:59.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45040b1650 0x7f450415a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:59.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 0x7f45040b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:59.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45040b13a0 con 0x7f45041605d0 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f45040b1220 con 0x7f45040afb10 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4504169540 con 0x7f45040b1650 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 0x7f45040b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 0x7f45040b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50530/0 (socket says 172.21.15.122:50530) 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4503e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45040afb10 0x7f45040b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 -- 172.21.15.122:0/1251719261 learned_addr learned my addr 172.21.15.122:0/1251719261 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4503400640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45040b1650 0x7f450415a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 -- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45040b1650 msgr2=0x7f450415a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45040b1650 0x7f450415a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 -- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45040afb10 msgr2=0x7f45040b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45040afb10 0x7f45040b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 -- 172.21.15.122:0/1251719261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45040cbe50 con 0x7f45041605d0 2024-09-17T05:39:59.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4503400640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45040b1650 0x7f450415a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:39:59.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f4509200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 0x7f45040b5dc0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00eb20 tx=0x7f44fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:59.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f44fbe00640 1 -- 172.21.15.122:0/1251719261 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004b30 con 0x7f45041605d0 2024-09-17T05:39:59.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f44fbe00640 1 -- 172.21.15.122:0/1251719261 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44fc004cf0 con 0x7f45041605d0 2024-09-17T05:39:59.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f44fbe00640 1 -- 172.21.15.122:0/1251719261 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc005280 con 0x7f45041605d0 2024-09-17T05:39:59.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45041f8ee0 con 0x7f45041605d0 2024-09-17T05:39:59.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.317+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45040b4cc0 con 0x7f45041605d0 2024-09-17T05:39:59.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.321+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45040affc0 con 0x7f45041605d0 2024-09-17T05:39:59.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.321+0000 7f44fbe00640 1 -- 172.21.15.122:0/1251719261 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7f44fc005420 con 0x7f45041605d0 2024-09-17T05:39:59.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.321+0000 7f44fbe00640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f44e4075450 0x7f44e4077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:39:59.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.321+0000 7f44fbe00640 1 -- 172.21.15.122:0/1251719261 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f44fc094060 con 0x7f45041605d0 2024-09-17T05:39:59.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.321+0000 7f4503e00640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f44e4075450 0x7f44e4077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:39:59.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.321+0000 7f4503e00640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f44e4075450 0x7f44e4077910 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f44f40af000 tx=0x7f44f401b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:39:59.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.325+0000 7f44fbe00640 1 -- 172.21.15.122:0/1251719261 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f44fc060af0 con 0x7f45041605d0 2024-09-17T05:39:59.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:39:59.529+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f450415a8f0 con 0x7f44e4075450 2024-09-17T05:40:00.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.185+0000 7f44fbe00640 1 -- 172.21.15.122:0/1251719261 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f450415a8f0 con 0x7f44e4075450 2024-09-17T05:40:00.183 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:00.184 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:40:00.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f44e4075450 msgr2=0x7f44e4077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f44e4075450 0x7f44e4077910 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f44f40af000 tx=0x7f44f401b060 comp rx=0 tx=0).stop 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 msgr2=0x7f45040b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 0x7f45040b5dc0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00eb20 tx=0x7f44fc00c2d0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 shutdown_connections 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f44e4075450 0x7f44e4077910 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45041605d0 0x7f45040b5dc0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45040b1650 0x7f450415a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 --2- 172.21.15.122:0/1251719261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45040afb10 0x7f45040b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 >> 172.21.15.122:0/1251719261 conn(0x7f45040d1dd0 msgr2=0x7f45040cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 shutdown_connections 2024-09-17T05:40:00.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.189+0000 7f450b200640 1 -- 172.21.15.122:0/1251719261 wait complete. 2024-09-17T05:40:00.205 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:40:00.205 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 -- 172.21.15.122:0/360791705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d140741c0 msgr2=0x7f0d1415e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 --2- 172.21.15.122:0/360791705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d140741c0 0x7f0d1415e680 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009c80 tx=0x7f0d0c01caa0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 -- 172.21.15.122:0/360791705 shutdown_connections 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 --2- 172.21.15.122:0/360791705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d140741c0 0x7f0d1415e680 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 --2- 172.21.15.122:0/360791705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d1415b090 0x7f0d14073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 --2- 172.21.15.122:0/360791705 >> v2:172.21.15.173:3300/0 conn(0x7f0d14158630 0x7f0d1406ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 -- 172.21.15.122:0/360791705 >> 172.21.15.122:0/360791705 conn(0x7f0d140cff00 msgr2=0x7f0d140d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 -- 172.21.15.122:0/360791705 shutdown_connections 2024-09-17T05:40:00.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.349+0000 7f0d13200640 1 -- 172.21.15.122:0/360791705 wait complete. 2024-09-17T05:40:00.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 Processor -- start 2024-09-17T05:40:00.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 -- start start 2024-09-17T05:40:00.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d140741c0 0x7f0d14151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:00.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d14158630 0x7f0d14155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d1415b090 0x7f0d141f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d140b3730 con 0x7f0d14158630 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d140b35b0 con 0x7f0d140741c0 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0d140b38b0 con 0x7f0d1415b090 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d14158630 0x7f0d14155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d14158630 0x7f0d14155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50552/0 (socket says 172.21.15.122:50552) 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 -- 172.21.15.122:0/4055612742 learned_addr learned my addr 172.21.15.122:0/4055612742 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d12800640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d1415b090 0x7f0d141f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d11e00640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d140741c0 0x7f0d14151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 -- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d1415b090 msgr2=0x7f0d141f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d1415b090 0x7f0d141f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 -- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d140741c0 msgr2=0x7f0d14151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d140741c0 0x7f0d14151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 -- 172.21.15.122:0/4055612742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d14152fa0 con 0x7f0d14158630 2024-09-17T05:40:00.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d09400640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d14158630 0x7f0d14155340 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b3f0 tx=0x7f0cfc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:00.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d0aa00640 1 -- 172.21.15.122:0/4055612742 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc015020 con 0x7f0d14158630 2024-09-17T05:40:00.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d0aa00640 1 -- 172.21.15.122:0/4055612742 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cfc011940 con 0x7f0d14158630 2024-09-17T05:40:00.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d14154580 con 0x7f0d14158630 2024-09-17T05:40:00.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d0aa00640 1 -- 172.21.15.122:0/4055612742 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc011d90 con 0x7f0d14158630 2024-09-17T05:40:00.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d14154b40 con 0x7f0d14158630 2024-09-17T05:40:00.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.353+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1406ff10 con 0x7f0d14158630 2024-09-17T05:40:00.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.357+0000 7f0d0aa00640 1 -- 172.21.15.122:0/4055612742 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7f0cfc018020 con 0x7f0d14158630 2024-09-17T05:40:00.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.357+0000 7f0d0aa00640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0cf0075450 0x7f0cf0077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:00.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.357+0000 7f0d0aa00640 1 -- 172.21.15.122:0/4055612742 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f0cfc093140 con 0x7f0d14158630 2024-09-17T05:40:00.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.357+0000 7f0d11e00640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0cf0075450 0x7f0cf0077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:00.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.357+0000 7f0d11e00640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0cf0075450 0x7f0cf0077910 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009990 tx=0x7f0d04009110 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:00.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.361+0000 7f0d0aa00640 1 -- 172.21.15.122:0/4055612742 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0cfc01f020 con 0x7f0d14158630 2024-09-17T05:40:00.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.589+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0d14074e10 con 0x7f0d14158630 2024-09-17T05:40:00.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.589+0000 7f0d0aa00640 1 -- 172.21.15.122:0/4055612742 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2106 (secure 0 0 0) 0x7f0cfc05f6a0 con 0x7f0d14158630 2024-09-17T05:40:00.586 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:00.586 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:00.587 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 921, 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:40:00.588 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 132, 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:40:00.589 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 793, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 3222011138, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 7536652288, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 765557460992, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 89458644, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 0, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 22 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:00.590 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 87, 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:39:21:803220+0000", 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 9, 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 9, 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "a", 2024-09-17T05:40:00.591 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 8916 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:40:00.592 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 20, 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:38:50.731214+0000", 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": { 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr-vol-ongoing-clones": { 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "progress": 0, 2024-09-17T05:40:00.593 INFO:teuthology.orchestra.run.smithi122.stdout: "add_to_ceph_s": true 2024-09-17T05:40:00.594 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:00.594 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:00.594 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0cf0075450 msgr2=0x7f0cf0077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0cf0075450 0x7f0cf0077910 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009990 tx=0x7f0d04009110 comp rx=0 tx=0).stop 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d14158630 msgr2=0x7f0d14155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d14158630 0x7f0d14155340 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00b3f0 tx=0x7f0cfc00b8b0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 shutdown_connections 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0cf0075450 0x7f0cf0077910 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d1415b090 0x7f0d141f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d14158630 0x7f0d14155340 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 --2- 172.21.15.122:0/4055612742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d140741c0 0x7f0d14151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 >> 172.21.15.122:0/4055612742 conn(0x7f0d140cff00 msgr2=0x7f0d14072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 shutdown_connections 2024-09-17T05:40:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:00.593+0000 7f0d13200640 1 -- 172.21.15.122:0/4055612742 wait complete. 2024-09-17T05:40:01.109 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:40:01.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.257+0000 7f6665a00640 1 -- 172.21.15.122:0/1859852240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6644003a60 msgr2=0x7f6644003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.257+0000 7f6665a00640 1 --2- 172.21.15.122:0/1859852240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6644003a60 0x7f6644003e80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f6650006fd0 tx=0x7f665001ca60 comp rx=0 tx=0).stop 2024-09-17T05:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.257+0000 7f6665a00640 1 -- 172.21.15.122:0/1859852240 shutdown_connections 2024-09-17T05:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.257+0000 7f6665a00640 1 --2- 172.21.15.122:0/1859852240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6644003a60 0x7f6644003e80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.257+0000 7f6665a00640 1 -- 172.21.15.122:0/1859852240 >> 172.21.15.122:0/1859852240 conn(0x7f66600d1e40 msgr2=0x7f66600d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.257+0000 7f6665a00640 1 -- 172.21.15.122:0/1859852240 shutdown_connections 2024-09-17T05:40:01.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.257+0000 7f6665a00640 1 -- 172.21.15.122:0/1859852240 wait complete. 2024-09-17T05:40:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 Processor -- start 2024-09-17T05:40:01.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 -- start start 2024-09-17T05:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6644003a60 0x7f666015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66601ecee0 0x7f66601ed340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666006e6a0 0x7f666006eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:01.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66600b18d0 con 0x7f666006e6a0 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66600b1750 con 0x7f6644003a60 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f66600b1a50 con 0x7f66601ecee0 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666006e6a0 0x7f666006eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666006e6a0 0x7f666006eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59702/0 (socket says 172.21.15.122:59702) 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 -- 172.21.15.122:0/3235661570 learned_addr learned my addr 172.21.15.122:0/3235661570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665ea00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6644003a60 0x7f666015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665e000640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66601ecee0 0x7f66601ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 -- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66601ecee0 msgr2=0x7f66601ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66601ecee0 0x7f66601ed340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 -- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6644003a60 msgr2=0x7f666015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6644003a60 0x7f666015bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 -- 172.21.15.122:0/3235661570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666015c5e0 con 0x7f666006e6a0 2024-09-17T05:40:01.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f665f400640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666006e6a0 0x7f666006eb20 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f66600b3150 tx=0x7f6658011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6656a00640 1 -- 172.21.15.122:0/3235661570 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658004440 con 0x7f666006e6a0 2024-09-17T05:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6656a00640 1 -- 172.21.15.122:0/3235661570 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6658016070 con 0x7f666006e6a0 2024-09-17T05:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6656a00640 1 -- 172.21.15.122:0/3235661570 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665800faf0 con 0x7f666006e6a0 2024-09-17T05:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66601eda10 con 0x7f666006e6a0 2024-09-17T05:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601fb4d0 con 0x7f666006e6a0 2024-09-17T05:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.261+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666006f060 con 0x7f666006e6a0 2024-09-17T05:40:01.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.265+0000 7f6656a00640 1 -- 172.21.15.122:0/3235661570 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7f665800fc90 con 0x7f666006e6a0 2024-09-17T05:40:01.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.265+0000 7f6656a00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6640075450 0x7f6640077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:01.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.265+0000 7f6656a00640 1 -- 172.21.15.122:0/3235661570 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f6658093b60 con 0x7f666006e6a0 2024-09-17T05:40:01.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.265+0000 7f665ea00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6640075450 0x7f6640077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:01.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.265+0000 7f665ea00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6640075450 0x7f6640077910 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f66500aa000 tx=0x7f66500b0040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:01.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.269+0000 7f6656a00640 1 -- 172.21.15.122:0/3235661570 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6658060020 con 0x7f666006e6a0 2024-09-17T05:40:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f66600b1fb0 con 0x7f666006e6a0 2024-09-17T05:40:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6656a00640 1 -- 172.21.15.122:0/3235661570 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1917 (secure 0 0 0) 0x7f6658064a50 con 0x7f666006e6a0 2024-09-17T05:40:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:01.493 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:40:01.493 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:40:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:40:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:40:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:40:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:40:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:01.494 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:40:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:40:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:40:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:40:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:40:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:01.495 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 922, 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:40:01.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 132, 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:01.497 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:40:01.498 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:40:01.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:01.498 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:40:01.498 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:40:01.498 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:01.498 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:01.499 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:40:01.499 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:40:01.499 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 856, 2024-09-17T05:40:01.499 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 3465411956, 2024-09-17T05:40:01.499 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 7666835456, 2024-09-17T05:40:01.499 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 765427277824, 2024-09-17T05:40:01.499 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:40:01.500 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 21665523, 2024-09-17T05:40:01.500 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 109739190, 2024-09-17T05:40:01.500 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 5, 2024-09-17T05:40:01.500 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 30 2024-09-17T05:40:01.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:01.500 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:40:01.500 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 87, 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:39:21:803220+0000", 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 9, 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 9, 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:40:01.501 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "a", 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 8916 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:40:01.502 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 20, 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:38:50.731214+0000", 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": {} 2024-09-17T05:40:01.503 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:40:01.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6640075450 msgr2=0x7f6640077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6640075450 0x7f6640077910 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f66500aa000 tx=0x7f66500b0040 comp rx=0 tx=0).stop 2024-09-17T05:40:01.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666006e6a0 msgr2=0x7f666006eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666006e6a0 0x7f666006eb20 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f66600b3150 tx=0x7f6658011be0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 shutdown_connections 2024-09-17T05:40:01.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6640075450 0x7f6640077910 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f666006e6a0 0x7f666006eb20 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66601ecee0 0x7f66601ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 --2- 172.21.15.122:0/3235661570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6644003a60 0x7f666015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 >> 172.21.15.122:0/3235661570 conn(0x7f66600d1e40 msgr2=0x7f666006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:01.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 shutdown_connections 2024-09-17T05:40:01.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.497+0000 7f6665a00640 1 -- 172.21.15.122:0/3235661570 wait complete. 2024-09-17T05:40:01.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:01.514 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/f24277a3-5997-41fe-b801-b8aaf6f65d82 2024-09-17T05:40:01.514 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:40:01.523 INFO:teuthology.orchestra.run.smithi122.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/f24277a3-5997-41fe-b801-b8aaf6f65d82': No such file or directory 2024-09-17T05:40:01.524 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T05:40:01.525 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:40:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.713+0000 7ff5f8a00640 1 -- 172.21.15.122:0/1875720471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5f40b2e80 msgr2=0x7ff5f40b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.713+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/1875720471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5f40b2e80 0x7ff5f40b1bd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009840 tx=0x7ff5e001c5f0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.713+0000 7ff5f8a00640 1 -- 172.21.15.122:0/1875720471 shutdown_connections 2024-09-17T05:40:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.713+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/1875720471 >> v2:172.21.15.173:3300/0 conn(0x7ff5f40afca0 0x7ff5f40b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.713+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/1875720471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5f40b2e80 0x7ff5f40b1bd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.713+0000 7ff5f8a00640 1 -- 172.21.15.122:0/1875720471 >> 172.21.15.122:0/1875720471 conn(0x7ff5f40d1e30 msgr2=0x7ff5f40d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.713+0000 7ff5f8a00640 1 -- 172.21.15.122:0/1875720471 shutdown_connections 2024-09-17T05:40:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.713+0000 7ff5f8a00640 1 -- 172.21.15.122:0/1875720471 wait complete. 2024-09-17T05:40:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 Processor -- start 2024-09-17T05:40:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 -- start start 2024-09-17T05:40:01.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f40afca0 0x7ff5f4157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:01.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5f40b2e80 0x7ff5f415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:01.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5f415f490 0x7ff5f415c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:01.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4168c80 con 0x7ff5f40afca0 2024-09-17T05:40:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f40b2320 con 0x7ff5f40b2e80 2024-09-17T05:40:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f4168e00 con 0x7ff5f415f490 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f40afca0 0x7ff5f4157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f40afca0 0x7ff5f4157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59710/0 (socket says 172.21.15.122:59710) 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 -- 172.21.15.122:0/556842455 learned_addr learned my addr 172.21.15.122:0/556842455 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 -- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5f415f490 msgr2=0x7ff5f415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5f415f490 0x7ff5f415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 -- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5f40b2e80 msgr2=0x7ff5f415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5f40b2e80 0x7ff5f415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 -- 172.21.15.122:0/556842455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f41fab10 con 0x7ff5f40afca0 2024-09-17T05:40:01.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f1e00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f40afca0 0x7ff5f4157c20 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c500 tx=0x7ff5e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:01.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5ea000640 1 -- 172.21.15.122:0/556842455 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4016020 con 0x7ff5f40afca0 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f41fadf0 con 0x7ff5f40afca0 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5ea000640 1 -- 172.21.15.122:0/556842455 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5e4012940 con 0x7ff5f40afca0 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5ea000640 1 -- 172.21.15.122:0/556842455 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4012d90 con 0x7ff5f40afca0 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f41fb3b0 con 0x7ff5f40afca0 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.717+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f41fb5b0 con 0x7ff5f40afca0 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.721+0000 7ff5ea000640 1 -- 172.21.15.122:0/556842455 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7ff5e4019020 con 0x7ff5f40afca0 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.721+0000 7ff5ea000640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff5d4075450 0x7ff5d4077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.721+0000 7ff5ea000640 1 -- 172.21.15.122:0/556842455 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7ff5e40934f0 con 0x7ff5f40afca0 2024-09-17T05:40:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.721+0000 7ff5f1400640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff5d4075450 0x7ff5d4077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.721+0000 7ff5f1400640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff5d4075450 0x7ff5d4077910 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00ae000 tx=0x7ff5e0019390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:01.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.725+0000 7ff5ea000640 1 -- 172.21.15.122:0/556842455 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff5f41fb5b0 con 0x7ff5f40afca0 2024-09-17T05:40:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.913+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff5f40cf5b0 con 0x7ff5d4075450 2024-09-17T05:40:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5ea000640 1 -- 172.21.15.122:0/556842455 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff5f40cf5b0 con 0x7ff5d4075450 2024-09-17T05:40:01.913 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:40:01.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:01.913 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:40:01.913 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:01.914 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:40:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff5d4075450 msgr2=0x7ff5d4077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff5d4075450 0x7ff5d4077910 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00ae000 tx=0x7ff5e0019390 comp rx=0 tx=0).stop 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f40afca0 msgr2=0x7ff5f4157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f40afca0 0x7ff5f4157c20 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c500 tx=0x7ff5e400c9c0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 shutdown_connections 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff5d4075450 0x7ff5d4077910 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5f415f490 0x7ff5f415c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5f40b2e80 0x7ff5f415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 --2- 172.21.15.122:0/556842455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f40afca0 0x7ff5f4157c20 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 >> 172.21.15.122:0/556842455 conn(0x7ff5f40d1e30 msgr2=0x7ff5f414eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 shutdown_connections 2024-09-17T05:40:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:01.917+0000 7ff5f8a00640 1 -- 172.21.15.122:0/556842455 wait complete. 2024-09-17T05:40:01.936 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:40:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81be00640 1 -- 172.21.15.122:0/107113255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 msgr2=0x7ff800004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81be00640 1 --2- 172.21.15.122:0/107113255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 0x7ff800004810 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff804006f90 tx=0x7ff80401cd60 comp rx=0 tx=0).stop 2024-09-17T05:40:02.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81b400640 1 -- 172.21.15.122:0/107113255 reap_dead start 2024-09-17T05:40:02.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81be00640 1 -- 172.21.15.122:0/107113255 shutdown_connections 2024-09-17T05:40:02.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81be00640 1 -- 172.21.15.122:0/107113255 >> 172.21.15.122:0/107113255 conn(0x7ff81c09aa10 msgr2=0x7ff81c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:02.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81be00640 1 -- 172.21.15.122:0/107113255 shutdown_connections 2024-09-17T05:40:02.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81be00640 1 -- 172.21.15.122:0/107113255 wait complete. 2024-09-17T05:40:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81be00640 1 Processor -- start 2024-09-17T05:40:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.085+0000 7ff81be00640 1 -- start start 2024-09-17T05:40:02.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 0x7ff81c0c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff81c159890 0x7ff81c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81c1f9050 0x7ff81c1f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c167f30 con 0x7ff800003fd0 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c167db0 con 0x7ff81c159890 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff81c1680b0 con 0x7ff81c1f9050 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 0x7ff81c0c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 0x7ff81c0c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59722/0 (socket says 172.21.15.122:59722) 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 -- 172.21.15.122:0/2242881554 learned_addr learned my addr 172.21.15.122:0/2242881554 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81b400640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81c1f9050 0x7ff81c1f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff812000640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff81c159890 0x7ff81c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 -- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81c1f9050 msgr2=0x7ff81c1f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81c1f9050 0x7ff81c1f94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 -- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff81c159890 msgr2=0x7ff81c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff81c159890 0x7ff81c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 -- 172.21.15.122:0/2242881554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c1f9d80 con 0x7ff800003fd0 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff812000640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff81c159890 0x7ff81c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81aa00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 0x7ff81c0c6fa0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff80c011a50 tx=0x7ff80c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff813400640 1 -- 172.21.15.122:0/2242881554 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c01f070 con 0x7ff800003fd0 2024-09-17T05:40:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff813400640 1 -- 172.21.15.122:0/2242881554 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff80c00fe80 con 0x7ff800003fd0 2024-09-17T05:40:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff813400640 1 -- 172.21.15.122:0/2242881554 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80c012690 con 0x7ff800003fd0 2024-09-17T05:40:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c1f5d20 con 0x7ff800003fd0 2024-09-17T05:40:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1f62e0 con 0x7ff800003fd0 2024-09-17T05:40:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81c0d1bb0 con 0x7ff800003fd0 2024-09-17T05:40:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.089+0000 7ff813400640 1 -- 172.21.15.122:0/2242881554 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7ff80c022070 con 0x7ff800003fd0 2024-09-17T05:40:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.093+0000 7ff813400640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff7f8075450 0x7ff7f8077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.093+0000 7ff813400640 1 -- 172.21.15.122:0/2242881554 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7ff80c093e40 con 0x7ff800003fd0 2024-09-17T05:40:02.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.093+0000 7ff812000640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff7f8075450 0x7ff7f8077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.093+0000 7ff812000640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff7f8075450 0x7ff7f8077910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff8040aa000 tx=0x7ff804019ba0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:02.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.097+0000 7ff813400640 1 -- 172.21.15.122:0/2242881554 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff80c060380 con 0x7ff800003fd0 2024-09-17T05:40:02.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.289+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff81c069e30 con 0x7ff7f8075450 2024-09-17T05:40:02.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff813400640 1 -- 172.21.15.122:0/2242881554 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7ff8040a9030 con 0x7ff7f8075450 2024-09-17T05:40:02.292 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:02.293 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:40:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sv1" 2024-09-17T05:40:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1clone1" 2024-09-17T05:40:02.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:02.293 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:40:02.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff7f8075450 msgr2=0x7ff7f8077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff7f8075450 0x7ff7f8077910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff8040aa000 tx=0x7ff804019ba0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 msgr2=0x7ff81c0c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 0x7ff81c0c6fa0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff80c011a50 tx=0x7ff80c011f10 comp rx=0 tx=0).stop 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 shutdown_connections 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff7f8075450 0x7ff7f8077910 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81c1f9050 0x7ff81c1f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff81c159890 0x7ff81c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 --2- 172.21.15.122:0/2242881554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff800003fd0 0x7ff81c0c6fa0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 >> 172.21.15.122:0/2242881554 conn(0x7ff81c09aa10 msgr2=0x7ff81c0b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 shutdown_connections 2024-09-17T05:40:02.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.297+0000 7ff81be00640 1 -- 172.21.15.122:0/2242881554 wait complete. 2024-09-17T05:40:02.314 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T05:40:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.461+0000 7efd22400640 1 -- 172.21.15.122:0/2016501797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c0d2540 msgr2=0x7efd1c0d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.461+0000 7efd22400640 1 --2- 172.21.15.122:0/2016501797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c0d2540 0x7efd1c0d29d0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7efd14009bc0 tx=0x7efd1401d600 comp rx=0 tx=0).stop 2024-09-17T05:40:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.461+0000 7efd22400640 1 -- 172.21.15.122:0/2016501797 shutdown_connections 2024-09-17T05:40:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.461+0000 7efd22400640 1 --2- 172.21.15.122:0/2016501797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c0d2540 0x7efd1c0d29d0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.461+0000 7efd22400640 1 --2- 172.21.15.122:0/2016501797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd1c15de40 0x7efd1c0b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.461+0000 7efd22400640 1 -- 172.21.15.122:0/2016501797 >> 172.21.15.122:0/2016501797 conn(0x7efd1c0d3f90 msgr2=0x7efd1c0d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.461+0000 7efd22400640 1 -- 172.21.15.122:0/2016501797 shutdown_connections 2024-09-17T05:40:02.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.461+0000 7efd22400640 1 -- 172.21.15.122:0/2016501797 wait complete. 2024-09-17T05:40:02.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 Processor -- start 2024-09-17T05:40:02.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 -- start start 2024-09-17T05:40:02.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd1c0d2540 0x7efd1c0b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c15de40 0x7efd1c155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c15e220 0x7efd1c156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c16b170 con 0x7efd1c15e220 2024-09-17T05:40:02.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0d0080 con 0x7efd1c15de40 2024-09-17T05:40:02.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c16b2f0 con 0x7efd1c0d2540 2024-09-17T05:40:02.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c15e220 0x7efd1c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c15e220 0x7efd1c156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59742/0 (socket says 172.21.15.122:59742) 2024-09-17T05:40:02.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 -- 172.21.15.122:0/1588732536 learned_addr learned my addr 172.21.15.122:0/1588732536 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:02.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1b400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd1c0d2540 0x7efd1c0b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1aa00640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c15de40 0x7efd1c155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 -- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd1c0d2540 msgr2=0x7efd1c0b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd1c0d2540 0x7efd1c0b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 -- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c15de40 msgr2=0x7efd1c155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c15de40 0x7efd1c155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 -- 172.21.15.122:0/1588732536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c0b11d0 con 0x7efd1c15e220 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1aa00640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c15de40 0x7efd1c155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd1be00640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c15e220 0x7efd1c156500 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7efd1401a7d0 tx=0x7efd140a5ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd13400640 1 -- 172.21.15.122:0/1588732536 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd140bb070 con 0x7efd1c15e220 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd13400640 1 -- 172.21.15.122:0/1588732536 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd14004e00 con 0x7efd1c15e220 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd13400640 1 -- 172.21.15.122:0/1588732536 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd140b2040 con 0x7efd1c15e220 2024-09-17T05:40:02.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c0b8e70 con 0x7efd1c15e220 2024-09-17T05:40:02.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c150a90 con 0x7efd1c15e220 2024-09-17T05:40:02.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.465+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1c0d29d0 con 0x7efd1c15e220 2024-09-17T05:40:02.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.469+0000 7efd13400640 1 -- 172.21.15.122:0/1588732536 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7efd140043a0 con 0x7efd1c15e220 2024-09-17T05:40:02.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.469+0000 7efd13400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efcfc075480 0x7efcfc077940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.469+0000 7efd13400640 1 -- 172.21.15.122:0/1588732536 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7efd1412d370 con 0x7efd1c15e220 2024-09-17T05:40:02.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.469+0000 7efd1b400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efcfc075480 0x7efcfc077940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.469+0000 7efd1b400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efcfc075480 0x7efcfc077940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c005db0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:02.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.473+0000 7efd13400640 1 -- 172.21.15.122:0/1588732536 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efd140a5e20 con 0x7efd1c15e220 2024-09-17T05:40:02.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.669+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd1c150d50 con 0x7efcfc075480 2024-09-17T05:40:02.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd13400640 1 -- 172.21.15.122:0/1588732536 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7efd1c150d50 con 0x7efcfc075480 2024-09-17T05:40:02.680 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:02.680 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:40:02.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:02.680 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1" 2024-09-17T05:40:02.680 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:02.680 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:40:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efcfc075480 msgr2=0x7efcfc077940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efcfc075480 0x7efcfc077940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c005db0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c15e220 msgr2=0x7efd1c156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c15e220 0x7efd1c156500 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7efd1401a7d0 tx=0x7efd140a5ae0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 shutdown_connections 2024-09-17T05:40:02.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efcfc075480 0x7efcfc077940 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd1c15e220 0x7efd1c156500 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd1c15de40 0x7efd1c155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 --2- 172.21.15.122:0/1588732536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd1c0d2540 0x7efd1c0b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 >> 172.21.15.122:0/1588732536 conn(0x7efd1c0d3f90 msgr2=0x7efd1c15cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:02.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 shutdown_connections 2024-09-17T05:40:02.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.685+0000 7efd22400640 1 -- 172.21.15.122:0/1588732536 wait complete. 2024-09-17T05:40:02.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T05:40:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 -- 172.21.15.122:0/3057529854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 msgr2=0x7f83580cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 --2- 172.21.15.122:0/3057529854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 0x7f83580cebf0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f834c009900 tx=0x7f834c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 -- 172.21.15.122:0/3057529854 shutdown_connections 2024-09-17T05:40:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 --2- 172.21.15.122:0/3057529854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f835806e810 0x7f83580cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 --2- 172.21.15.122:0/3057529854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 0x7f83580cebf0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 -- 172.21.15.122:0/3057529854 >> 172.21.15.122:0/3057529854 conn(0x7f83580b18f0 msgr2=0x7f83580b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 -- 172.21.15.122:0/3057529854 shutdown_connections 2024-09-17T05:40:02.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 -- 172.21.15.122:0/3057529854 wait complete. 2024-09-17T05:40:02.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 Processor -- start 2024-09-17T05:40:02.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.853+0000 7f835fc00640 1 -- start start 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f835806e810 0x7f835813e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83580b4bd0 0x7f835813e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 0x7f83581410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8358168090 con 0x7f835815ade0 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8358167f10 con 0x7f83580b4bd0 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8358168210 con 0x7f835806e810 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 0x7f83581410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 0x7f83581410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59766/0 (socket says 172.21.15.122:59766) 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 -- 172.21.15.122:0/48342840 learned_addr learned my addr 172.21.15.122:0/48342840 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835d200640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f835806e810 0x7f835813e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f8357e00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83580b4bd0 0x7f835813e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 -- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f835806e810 msgr2=0x7f835813e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f835806e810 0x7f835813e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 -- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83580b4bd0 msgr2=0x7f835813e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83580b4bd0 0x7f835813e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 -- 172.21.15.122:0/48342840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835809ca10 con 0x7f835815ade0 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f8357e00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83580b4bd0 0x7f835813e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835dc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 0x7f83581410b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f835000c730 tx=0x7f835000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f8354c00640 1 -- 172.21.15.122:0/48342840 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350003db0 con 0x7f835815ade0 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f8354c00640 1 -- 172.21.15.122:0/48342840 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f835000ce40 con 0x7f835815ade0 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f8354c00640 1 -- 172.21.15.122:0/48342840 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83500052d0 con 0x7f835815ade0 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835809ccf0 con 0x7f835815ade0 2024-09-17T05:40:02.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835813ee00 con 0x7f835815ade0 2024-09-17T05:40:02.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.857+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8358069410 con 0x7f835815ade0 2024-09-17T05:40:02.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.861+0000 7f8354c00640 1 -- 172.21.15.122:0/48342840 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7f8350005470 con 0x7f835815ade0 2024-09-17T05:40:02.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.861+0000 7f8354c00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8338075450 0x7f8338077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:02.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.861+0000 7f8354c00640 1 -- 172.21.15.122:0/48342840 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7f8350093200 con 0x7f835815ade0 2024-09-17T05:40:02.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.861+0000 7f835d200640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8338075450 0x7f8338077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:02.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.861+0000 7f835d200640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8338075450 0x7f8338077910 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f834c0af000 tx=0x7f834c01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:02.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:02.865+0000 7f8354c00640 1 -- 172.21.15.122:0/48342840 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f835005fd10 con 0x7f835815ade0 2024-09-17T05:40:03.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.065+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f835809cec0 con 0x7f8338075450 2024-09-17T05:40:03.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.093+0000 7f8354c00640 1 -- 172.21.15.122:0/48342840 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f835809cec0 con 0x7f8338075450 2024-09-17T05:40:03.090 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8338075450 msgr2=0x7f8338077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8338075450 0x7f8338077910 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f834c0af000 tx=0x7f834c01b120 comp rx=0 tx=0).stop 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 msgr2=0x7f83581410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 0x7f83581410b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f835000c730 tx=0x7f835000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 shutdown_connections 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8338075450 0x7f8338077910 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f835815ade0 0x7f83581410b0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83580b4bd0 0x7f835813e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 --2- 172.21.15.122:0/48342840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f835806e810 0x7f835813e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 >> 172.21.15.122:0/48342840 conn(0x7f83580b18f0 msgr2=0x7f83580b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:03.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 shutdown_connections 2024-09-17T05:40:03.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.097+0000 7f835fc00640 1 -- 172.21.15.122:0/48342840 wait complete. 2024-09-17T05:40:03.112 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T05:40:03.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 -- 172.21.15.122:0/3290542933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c15bd40 msgr2=0x7fe28c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 --2- 172.21.15.122:0/3290542933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c15bd40 0x7fe28c0d0240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009840 tx=0x7fe27c01c820 comp rx=0 tx=0).stop 2024-09-17T05:40:03.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 -- 172.21.15.122:0/3290542933 shutdown_connections 2024-09-17T05:40:03.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 --2- 172.21.15.122:0/3290542933 >> v2:172.21.15.173:3300/0 conn(0x7fe28c0d22c0 0x7fe28c0cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 --2- 172.21.15.122:0/3290542933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c15bd40 0x7fe28c0d0240 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 -- 172.21.15.122:0/3290542933 >> 172.21.15.122:0/3290542933 conn(0x7fe28c09aa10 msgr2=0x7fe28c09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:03.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 -- 172.21.15.122:0/3290542933 shutdown_connections 2024-09-17T05:40:03.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 -- 172.21.15.122:0/3290542933 wait complete. 2024-09-17T05:40:03.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 Processor -- start 2024-09-17T05:40:03.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.261+0000 7fe28be00640 1 -- start start 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe28c0b0d10 0x7fe28c0b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c0d22c0 0x7fe28c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c15bd40 0x7fe28c14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c168cb0 con 0x7fe28c0b0d10 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c168b30 con 0x7fe28c15bd40 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe28c168e30 con 0x7fe28c0d22c0 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe28c0b0d10 0x7fe28c0b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe28c0b0d10 0x7fe28c0b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59774/0 (socket says 172.21.15.122:59774) 2024-09-17T05:40:03.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 -- 172.21.15.122:0/1614853594 learned_addr learned my addr 172.21.15.122:0/1614853594 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28b400640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c15bd40 0x7fe28c14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe282000640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c0d22c0 0x7fe28c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 -- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c0d22c0 msgr2=0x7fe28c0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c0d22c0 0x7fe28c0cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 -- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c15bd40 msgr2=0x7fe28c14e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c15bd40 0x7fe28c14e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 -- 172.21.15.122:0/1614853594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c0b7730 con 0x7fe28c0b0d10 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe282000640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c0d22c0 0x7fe28c0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28aa00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe28c0b0d10 0x7fe28c0b71f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009790 tx=0x7fe27c0a6620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe283400640 1 -- 172.21.15.122:0/1614853594 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c004220 con 0x7fe28c0b0d10 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe283400640 1 -- 172.21.15.122:0/1614853594 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe27c0044b0 con 0x7fe28c0b0d10 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c0b4c50 con 0x7fe28c0b0d10 2024-09-17T05:40:03.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe283400640 1 -- 172.21.15.122:0/1614853594 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c003860 con 0x7fe28c0b0d10 2024-09-17T05:40:03.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c06ddd0 con 0x7fe28c0b0d10 2024-09-17T05:40:03.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.265+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe28c0cc620 con 0x7fe28c0b0d10 2024-09-17T05:40:03.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.269+0000 7fe283400640 1 -- 172.21.15.122:0/1614853594 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7fe27c0c0060 con 0x7fe28c0b0d10 2024-09-17T05:40:03.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.269+0000 7fe283400640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe268075480 0x7fe268077940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:03.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.269+0000 7fe283400640 1 -- 172.21.15.122:0/1614853594 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7fe27c12d680 con 0x7fe28c0b0d10 2024-09-17T05:40:03.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.269+0000 7fe282000640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe268075480 0x7fe268077940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:03.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.269+0000 7fe282000640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe268075480 0x7fe268077940 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe274001230 tx=0x7fe274009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:03.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.273+0000 7fe283400640 1 -- 172.21.15.122:0/1614853594 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe28c0b7730 con 0x7fe28c0b0d10 2024-09-17T05:40:03.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.465+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe28c09fba0 con 0x7fe268075480 2024-09-17T05:40:03.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe283400640 1 -- 172.21.15.122:0/1614853594 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe28c09fba0 con 0x7fe268075480 2024-09-17T05:40:03.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe268075480 msgr2=0x7fe268077940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe268075480 0x7fe268077940 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe274001230 tx=0x7fe274009040 comp rx=0 tx=0).stop 2024-09-17T05:40:03.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe28c0b0d10 msgr2=0x7fe28c0b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe28c0b0d10 0x7fe28c0b71f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009790 tx=0x7fe27c0a6620 comp rx=0 tx=0).stop 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 shutdown_connections 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe268075480 0x7fe268077940 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe28c15bd40 0x7fe28c14e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe28c0d22c0 0x7fe28c0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 --2- 172.21.15.122:0/1614853594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe28c0b0d10 0x7fe28c0b71f0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 >> 172.21.15.122:0/1614853594 conn(0x7fe28c09aa10 msgr2=0x7fe28c0b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 shutdown_connections 2024-09-17T05:40:03.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.489+0000 7fe28be00640 1 -- 172.21.15.122:0/1614853594 wait complete. 2024-09-17T05:40:03.496 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:03.499+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:03.496 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:03.499+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:03.496 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:03.499+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:03.497 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:03.499+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:03.497 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:03.499+0000 7fc09be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:03.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-17T05:40:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 -- 172.21.15.122:0/3156934757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c15c1f0 msgr2=0x7fbc6c0cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 --2- 172.21.15.122:0/3156934757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c15c1f0 0x7fbc6c0cee10 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0b0fb0 tx=0x7fbc5401eff0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 -- 172.21.15.122:0/3156934757 shutdown_connections 2024-09-17T05:40:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 --2- 172.21.15.122:0/3156934757 >> v2:172.21.15.173:3300/0 conn(0x7fbc6c0b0b60 0x7fbc6c0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 --2- 172.21.15.122:0/3156934757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c15c1f0 0x7fbc6c0cee10 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 -- 172.21.15.122:0/3156934757 >> 172.21.15.122:0/3156934757 conn(0x7fbc6c0d1da0 msgr2=0x7fbc6c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 -- 172.21.15.122:0/3156934757 shutdown_connections 2024-09-17T05:40:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 -- 172.21.15.122:0/3156934757 wait complete. 2024-09-17T05:40:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 Processor -- start 2024-09-17T05:40:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 -- start start 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c0b0b60 0x7fbc6c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c15be10 0x7fbc6c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.661+0000 7fbc70c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c15c1f0 0x7fbc6c0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc70c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c0cd4a0 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc70c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c0cd320 con 0x7fbc6c15be10 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc70c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c0cd620 con 0x7fbc6c15c1f0 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c0b0b60 0x7fbc6c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c0b0b60 0x7fbc6c06ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59786/0 (socket says 172.21.15.122:59786) 2024-09-17T05:40:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 -- 172.21.15.122:0/2387600823 learned_addr learned my addr 172.21.15.122:0/2387600823 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc63e00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c15be10 0x7fbc6c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc6a800640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c15c1f0 0x7fbc6c0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 -- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c15c1f0 msgr2=0x7fbc6c0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c15c1f0 0x7fbc6c0c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 -- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c15be10 msgr2=0x7fbc6c06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c15be10 0x7fbc6c06f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 -- 172.21.15.122:0/2387600823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c150fd0 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc69e00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c0b0b60 0x7fbc6c06ed80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00c500 tx=0x7fbc5c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc62000640 1 -- 172.21.15.122:0/2387600823 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c017070 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc62000640 1 -- 172.21.15.122:0/2387600823 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc5c003600 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c1512b0 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc62000640 1 -- 172.21.15.122:0/2387600823 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c003a90 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c151b50 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6c1516a0 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.665+0000 7fbc62000640 1 -- 172.21.15.122:0/2387600823 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 50) ==== 100345+0+0 (secure 0 0 0) 0x7fbc5c01a070 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.669+0000 7fbc62000640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc48075450 0x7fbc48077910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:03.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.669+0000 7fbc62000640 1 -- 172.21.15.122:0/2387600823 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5793+0+0 (secure 0 0 0) 0x7fbc5c0928d0 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.669+0000 7fbc63e00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc48075450 0x7fbc48077910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:03.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.669+0000 7fbc63e00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc48075450 0x7fbc48077910 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbc540b2000 tx=0x7fbc5401d830 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:03.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.669+0000 7fbc62000640 1 -- 172.21.15.122:0/2387600823 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fbc5c05f420 con 0x7fbc6c0b0b60 2024-09-17T05:40:03.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:03.897+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fbc6c0cdb40 con 0x7fbc48075450 2024-09-17T05:40:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.189+0000 7fbc62000640 1 -- 172.21.15.122:0/2387600823 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc6c0cdb40 con 0x7fbc48075450 2024-09-17T05:40:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc48075450 msgr2=0x7fbc48077910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc48075450 0x7fbc48077910 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbc540b2000 tx=0x7fbc5401d830 comp rx=0 tx=0).stop 2024-09-17T05:40:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c0b0b60 msgr2=0x7fbc6c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c0b0b60 0x7fbc6c06ed80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00c500 tx=0x7fbc5c00c9c0 comp rx=0 tx=0).stop 2024-09-17T05:40:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 shutdown_connections 2024-09-17T05:40:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fbc48075450 0x7fbc48077910 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c15c1f0 0x7fbc6c0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c15be10 0x7fbc6c06f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 --2- 172.21.15.122:0/2387600823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc6c0b0b60 0x7fbc6c06ed80 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 >> 172.21.15.122:0/2387600823 conn(0x7fbc6c0d1da0 msgr2=0x7fbc6c071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 shutdown_connections 2024-09-17T05:40:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04.193+0000 7fbc70c00640 1 -- 172.21.15.122:0/2387600823 wait complete. 2024-09-17T05:40:04.208 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:40:04.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:40:04.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:40:04.257 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:40:04.261 INFO:teuthology.orchestra.run.smithi122.stdout:3 2024-09-17T05:40:04.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:40:09.271 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:40:09.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:40:09.327 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:40:09.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:40:09.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:40:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.485+0000 7f248d200640 1 -- 172.21.15.122:0/252423995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24880d0700 msgr2=0x7f248815e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.485+0000 7f248d200640 1 --2- 172.21.15.122:0/252423995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24880d0700 0x7f248815e280 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f24880b2d60 tx=0x7f248001cb80 comp rx=0 tx=0).stop 2024-09-17T05:40:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.485+0000 7f248d200640 1 -- 172.21.15.122:0/252423995 shutdown_connections 2024-09-17T05:40:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.485+0000 7f248d200640 1 --2- 172.21.15.122:0/252423995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24880d0700 0x7f248815e280 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.485+0000 7f248d200640 1 --2- 172.21.15.122:0/252423995 >> v2:172.21.15.173:3300/0 conn(0x7f2488068960 0x7f24880d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.485+0000 7f248d200640 1 -- 172.21.15.122:0/252423995 >> 172.21.15.122:0/252423995 conn(0x7f24880742f0 msgr2=0x7f2488074700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.485+0000 7f248d200640 1 -- 172.21.15.122:0/252423995 shutdown_connections 2024-09-17T05:40:09.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.485+0000 7f248d200640 1 -- 172.21.15.122:0/252423995 wait complete. 2024-09-17T05:40:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 Processor -- start 2024-09-17T05:40:09.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 -- start start 2024-09-17T05:40:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2488068960 0x7f2488070910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24880cd860 0x7f2488165470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24880d0700 0x7f2488158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2488168e50 con 0x7f2488068960 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2488168cd0 con 0x7f24880cd860 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2488168fd0 con 0x7f24880d0700 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2488068960 0x7f2488070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2488068960 0x7f2488070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59802/0 (socket says 172.21.15.122:59802) 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 -- 172.21.15.122:0/874366745 learned_addr learned my addr 172.21.15.122:0/874366745 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f247ea00640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24880cd860 0x7f2488165470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 -- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24880d0700 msgr2=0x7f2488158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487e00640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24880d0700 0x7f2488158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24880d0700 0x7f2488158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 -- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24880cd860 msgr2=0x7f2488165470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24880cd860 0x7f2488165470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 -- 172.21.15.122:0/874366745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248806f1f0 con 0x7f2488068960 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f247ea00640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24880cd860 0x7f2488165470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487e00640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24880d0700 0x7f2488158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f2487400640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2488068960 0x7f2488070910 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f247800c730 tx=0x7f247800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f247fe00640 1 -- 172.21.15.122:0/874366745 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478017070 con 0x7f2488068960 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f247fe00640 1 -- 172.21.15.122:0/874366745 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24780051a0 con 0x7f2488068960 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f247fe00640 1 -- 172.21.15.122:0/874366745 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247800a640 con 0x7f2488068960 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248806f4d0 con 0x7f2488068960 2024-09-17T05:40:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248806fa90 con 0x7f2488068960 2024-09-17T05:40:09.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.489+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24880d01c0 con 0x7f2488068960 2024-09-17T05:40:09.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.493+0000 7f247fe00640 1 -- 172.21.15.122:0/874366745 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f247801a070 con 0x7f2488068960 2024-09-17T05:40:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.493+0000 7f247fe00640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f24640754e0 0x7f24640779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.493+0000 7f247fe00640 1 -- 172.21.15.122:0/874366745 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7f2478092d50 con 0x7f2488068960 2024-09-17T05:40:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.493+0000 7f247ea00640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f24640754e0 0x7f24640779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.497+0000 7f247ea00640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f24640754e0 0x7f24640779a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2470005fd0 tx=0x7f2470005e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.497+0000 7f247fe00640 1 -- 172.21.15.122:0/874366745 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f24880cbb00 con 0x7f2488068960 2024-09-17T05:40:09.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.665+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f24880b61b0 con 0x7f2488068960 2024-09-17T05:40:09.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.665+0000 7f247fe00640 1 -- 172.21.15.122:0/874366745 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7f247805f2d0 con 0x7f2488068960 2024-09-17T05:40:09.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f24640754e0 msgr2=0x7f24640779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f24640754e0 0x7f24640779a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2470005fd0 tx=0x7f2470005e60 comp rx=0 tx=0).stop 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2488068960 msgr2=0x7f2488070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2488068960 0x7f2488070910 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f247800c730 tx=0x7f247800cbf0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 shutdown_connections 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f24640754e0 0x7f24640779a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24880d0700 0x7f2488158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24880cd860 0x7f2488165470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 --2- 172.21.15.122:0/874366745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2488068960 0x7f2488070910 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 >> 172.21.15.122:0/874366745 conn(0x7f24880742f0 msgr2=0x7f24880ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 shutdown_connections 2024-09-17T05:40:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:09.669+0000 7f248d200640 1 -- 172.21.15.122:0/874366745 wait complete. 2024-09-17T05:40:09.685 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:40:09.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:40:09.770 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:40:09.863 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:40:09.863 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:40:09.864 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:09.864 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:09.873 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:09.873 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:09.928 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:40:10.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.121+0000 7f979ee00640 1 -- 172.21.15.122:0/3511397555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980ced80 msgr2=0x7f97980ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.121+0000 7f979ee00640 1 --2- 172.21.15.122:0/3511397555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980ced80 0x7f97980ae9b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9780009840 tx=0x7f978001c820 comp rx=0 tx=0).stop 2024-09-17T05:40:10.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.121+0000 7f979ee00640 1 -- 172.21.15.122:0/3511397555 shutdown_connections 2024-09-17T05:40:10.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.121+0000 7f979ee00640 1 --2- 172.21.15.122:0/3511397555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97980cdb30 0x7f97980aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.121+0000 7f979ee00640 1 --2- 172.21.15.122:0/3511397555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980ced80 0x7f97980ae9b0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.121+0000 7f979ee00640 1 -- 172.21.15.122:0/3511397555 >> 172.21.15.122:0/3511397555 conn(0x7f97980d1dc0 msgr2=0x7f97980d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:10.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.121+0000 7f979ee00640 1 -- 172.21.15.122:0/3511397555 shutdown_connections 2024-09-17T05:40:10.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.121+0000 7f979ee00640 1 -- 172.21.15.122:0/3511397555 wait complete. 2024-09-17T05:40:10.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 Processor -- start 2024-09-17T05:40:10.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 -- start start 2024-09-17T05:40:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980cdb30 0x7f9798151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97980ced80 0x7f97980b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f979815bea0 0x7f97980714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9798169340 con 0x7f97980cdb30 2024-09-17T05:40:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f97981691c0 con 0x7f979815bea0 2024-09-17T05:40:10.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f97981694c0 con 0x7f97980ced80 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980cdb30 0x7f9798151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980cdb30 0x7f9798151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59832/0 (socket says 172.21.15.122:59832) 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 -- 172.21.15.122:0/3364134236 learned_addr learned my addr 172.21.15.122:0/3364134236 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f978f400640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97980ced80 0x7f97980b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ce00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f979815bea0 0x7f97980714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 -- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97980ced80 msgr2=0x7f97980b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97980ced80 0x7f97980b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 -- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f979815bea0 msgr2=0x7f97980714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f979815bea0 0x7f97980714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 -- 172.21.15.122:0/3364134236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97980b5330 con 0x7f97980cdb30 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9797e00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980cdb30 0x7f9798151010 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9788001ec0 tx=0x7f978800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:10.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9794c00640 1 -- 172.21.15.122:0/3364134236 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788017070 con 0x7f97980cdb30 2024-09-17T05:40:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9794c00640 1 -- 172.21.15.122:0/3364134236 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9788003600 con 0x7f97980cdb30 2024-09-17T05:40:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f9794c00640 1 -- 172.21.15.122:0/3364134236 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788003a50 con 0x7f97980cdb30 2024-09-17T05:40:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97980b5610 con 0x7f97980cdb30 2024-09-17T05:40:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.125+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97980b5bd0 con 0x7f97980cdb30 2024-09-17T05:40:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.129+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97980d2500 con 0x7f97980cdb30 2024-09-17T05:40:10.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.129+0000 7f9794c00640 1 -- 172.21.15.122:0/3364134236 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f978801a070 con 0x7f97980cdb30 2024-09-17T05:40:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.129+0000 7f9794c00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9774075510 0x7f97740779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.129+0000 7f9794c00640 1 -- 172.21.15.122:0/3364134236 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7f9788092650 con 0x7f97980cdb30 2024-09-17T05:40:10.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.129+0000 7f978f400640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9774075510 0x7f97740779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:10.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.129+0000 7f978f400640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9774075510 0x7f97740779d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f97800af000 tx=0x7f978001b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:10.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.133+0000 7f9794c00640 1 -- 172.21.15.122:0/3364134236 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f978805f160 con 0x7f97980cdb30 2024-09-17T05:40:10.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.313+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f97981586f0 con 0x7f97980cdb30 2024-09-17T05:40:10.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.313+0000 7f9794c00640 1 -- 172.21.15.122:0/3364134236 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v133) ==== 80+0+0 (secure 0 0 0) 0x7f978805ea60 con 0x7f97980cdb30 2024-09-17T05:40:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:40:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9774075510 msgr2=0x7f97740779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9774075510 0x7f97740779d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f97800af000 tx=0x7f978001b060 comp rx=0 tx=0).stop 2024-09-17T05:40:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980cdb30 msgr2=0x7f9798151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980cdb30 0x7f9798151010 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9788001ec0 tx=0x7f978800c890 comp rx=0 tx=0).stop 2024-09-17T05:40:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 shutdown_connections 2024-09-17T05:40:10.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9774075510 0x7f97740779d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f979815bea0 0x7f97980714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97980ced80 0x7f97980b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 --2- 172.21.15.122:0/3364134236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97980cdb30 0x7f9798151010 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 >> 172.21.15.122:0/3364134236 conn(0x7f97980d1dc0 msgr2=0x7f9798157820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:10.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 shutdown_connections 2024-09-17T05:40:10.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.317+0000 7f979ee00640 1 -- 172.21.15.122:0/3364134236 wait complete. 2024-09-17T05:40:10.333 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:40:10.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:10.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:40:10.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:40:10.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:10.392 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:40:10.408 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:40:10.408 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:40:10.408 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:10.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:40:10.418 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:40:10.418 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:10.418 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:10.459 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:40:10.460 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:40:10.460 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:40:10.460 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:10.468 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:40:10.468 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:40:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.645+0000 7fc195000640 1 -- 172.21.15.122:0/3395304275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 msgr2=0x7fc1900b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.645+0000 7fc195000640 1 --2- 172.21.15.122:0/3395304275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 0x7fc1900b0130 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc188009c80 tx=0x7fc18801cc50 comp rx=0 tx=0).stop 2024-09-17T05:40:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.645+0000 7fc195000640 1 -- 172.21.15.122:0/3395304275 shutdown_connections 2024-09-17T05:40:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.645+0000 7fc195000640 1 --2- 172.21.15.122:0/3395304275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 0x7fc1900b0130 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.645+0000 7fc195000640 1 --2- 172.21.15.122:0/3395304275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1900b2e80 0x7fc1900b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.645+0000 7fc195000640 1 -- 172.21.15.122:0/3395304275 >> 172.21.15.122:0/3395304275 conn(0x7fc1900d1e30 msgr2=0x7fc1900d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.645+0000 7fc195000640 1 -- 172.21.15.122:0/3395304275 shutdown_connections 2024-09-17T05:40:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.645+0000 7fc195000640 1 -- 172.21.15.122:0/3395304275 wait complete. 2024-09-17T05:40:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 Processor -- start 2024-09-17T05:40:10.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 -- start start 2024-09-17T05:40:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 0x7fc19015d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1900b2e80 0x7fc19015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19015f490 0x7fc19014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc190168e00 con 0x7fc1900afca0 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc190168c80 con 0x7fc1900b2e80 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc190168f80 con 0x7fc19015f490 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 0x7fc19015d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 0x7fc19015d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59854/0 (socket says 172.21.15.122:59854) 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 -- 172.21.15.122:0/3471849968 learned_addr learned my addr 172.21.15.122:0/3471849968 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc185400640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1900b2e80 0x7fc19015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18e800640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19015f490 0x7fc19014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 -- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19015f490 msgr2=0x7fc19014ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19015f490 0x7fc19014ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 -- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1900b2e80 msgr2=0x7fc19015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1900b2e80 0x7fc19015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 -- 172.21.15.122:0/3471849968 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19015dd20 con 0x7fc1900afca0 2024-09-17T05:40:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc18de00640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 0x7fc19015d1d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c3c0 tx=0x7fc18000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc186a00640 1 -- 172.21.15.122:0/3471849968 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180017070 con 0x7fc1900afca0 2024-09-17T05:40:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc186a00640 1 -- 172.21.15.122:0/3471849968 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1800053c0 con 0x7fc1900afca0 2024-09-17T05:40:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc186a00640 1 -- 172.21.15.122:0/3471849968 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180003860 con 0x7fc1900afca0 2024-09-17T05:40:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19015e060 con 0x7fc1900afca0 2024-09-17T05:40:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.649+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19015d710 con 0x7fc1900afca0 2024-09-17T05:40:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.653+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1900b0130 con 0x7fc1900afca0 2024-09-17T05:40:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.653+0000 7fc186a00640 1 -- 172.21.15.122:0/3471849968 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fc18001a070 con 0x7fc1900afca0 2024-09-17T05:40:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.653+0000 7fc186a00640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc16c0754e0 0x7fc16c0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.653+0000 7fc186a00640 1 -- 172.21.15.122:0/3471849968 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7fc180092760 con 0x7fc1900afca0 2024-09-17T05:40:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.653+0000 7fc185400640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc16c0754e0 0x7fc16c0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:10.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.653+0000 7fc185400640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc16c0754e0 0x7fc16c0779a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc1780058d0 tx=0x7fc178005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:10.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.657+0000 7fc186a00640 1 -- 172.21.15.122:0/3471849968 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc19015e060 con 0x7fc1900afca0 2024-09-17T05:40:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1901593a0 con 0x7fc1900afca0 2024-09-17T05:40:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc186a00640 1 -- 172.21.15.122:0/3471849968 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5234 (secure 0 0 0) 0x7fc18005f270 con 0x7fc1900afca0 2024-09-17T05:40:10.854 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:10.855 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":87,"btime":"2024-09-17T05:39:21:803220+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/609693749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":609693749},{"type":"v1","addr":"172.21.15.122:6834","nonce":609693749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2826125653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2826125653},{"type":"v1","addr":"172.21.15.173:6836","nonce":2826125653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4138151052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4138151052},{"type":"v1","addr":"172.21.15.173:6837","nonce":4138151052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:39:09.055675+0000","modified":"2024-09-17T05:39:12.084809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2463174917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2463174917},{"type":"v1","addr":"172.21.15.122:6835","nonce":2463174917}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8916,"qdb_cluster":[8916]},"id":9}]} 2024-09-17T05:40:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 87 2024-09-17T05:40:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc16c0754e0 msgr2=0x7fc16c0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc16c0754e0 0x7fc16c0779a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc1780058d0 tx=0x7fc178005860 comp rx=0 tx=0).stop 2024-09-17T05:40:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 msgr2=0x7fc19015d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 0x7fc19015d1d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c3c0 tx=0x7fc18000c880 comp rx=0 tx=0).stop 2024-09-17T05:40:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 shutdown_connections 2024-09-17T05:40:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc16c0754e0 0x7fc16c0779a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19015f490 0x7fc19014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1900b2e80 0x7fc19015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.857+0000 7fc195000640 1 --2- 172.21.15.122:0/3471849968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1900afca0 0x7fc19015d1d0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.861+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 >> 172.21.15.122:0/3471849968 conn(0x7fc1900d1e30 msgr2=0x7fc1900ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.861+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 shutdown_connections 2024-09-17T05:40:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:10.861+0000 7fc195000640 1 -- 172.21.15.122:0/3471849968 wait complete. 2024-09-17T05:40:10.875 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:40:11.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 -- 172.21.15.122:0/1047020335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 msgr2=0x7fa478004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 --2- 172.21.15.122:0/1047020335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 0x7fa478004090 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa484009b60 tx=0x7fa48401d270 comp rx=0 tx=0).stop 2024-09-17T05:40:11.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 -- 172.21.15.122:0/1047020335 shutdown_connections 2024-09-17T05:40:11.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 --2- 172.21.15.122:0/1047020335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 0x7fa478004090 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 -- 172.21.15.122:0/1047020335 >> 172.21.15.122:0/1047020335 conn(0x7fa494073490 msgr2=0x7fa4940d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:11.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 -- 172.21.15.122:0/1047020335 shutdown_connections 2024-09-17T05:40:11.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 -- 172.21.15.122:0/1047020335 wait complete. 2024-09-17T05:40:11.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 Processor -- start 2024-09-17T05:40:11.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.021+0000 7fa493e00640 1 -- start start 2024-09-17T05:40:11.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 0x7fa4941ec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4941f9bf0 0x7fa4940c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4940c6af0 0x7fa4940b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4940ce6a0 con 0x7fa478003c90 2024-09-17T05:40:11.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4940cd240 con 0x7fa4941f9bf0 2024-09-17T05:40:11.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4940ce820 con 0x7fa4940c6af0 2024-09-17T05:40:11.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 0x7fa4941ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 0x7fa4941ec000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59876/0 (socket says 172.21.15.122:59876) 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 -- 172.21.15.122:0/1564394089 learned_addr learned my addr 172.21.15.122:0/1564394089 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492000640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4941f9bf0 0x7fa4940c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493400640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4940c6af0 0x7fa4940b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 -- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4940c6af0 msgr2=0x7fa4940b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4940c6af0 0x7fa4940b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 -- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4941f9bf0 msgr2=0x7fa4940c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4941f9bf0 0x7fa4940c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 -- 172.21.15.122:0/1564394089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4940c6f80 con 0x7fa478003c90 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493400640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4940c6af0 0x7fa4940b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa492a00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 0x7fa4941ec000 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fa4840af040 tx=0x7fa484005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa48aa00640 1 -- 172.21.15.122:0/1564394089 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840bd070 con 0x7fa478003c90 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa48aa00640 1 -- 172.21.15.122:0/1564394089 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa484004030 con 0x7fa478003c90 2024-09-17T05:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4941ec650 con 0x7fa478003c90 2024-09-17T05:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa48aa00640 1 -- 172.21.15.122:0/1564394089 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840b1040 con 0x7fa478003c90 2024-09-17T05:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4941f93f0 con 0x7fa478003c90 2024-09-17T05:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.025+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa494074340 con 0x7fa478003c90 2024-09-17T05:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.029+0000 7fa48aa00640 1 -- 172.21.15.122:0/1564394089 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fa4840c0070 con 0x7fa478003c90 2024-09-17T05:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.029+0000 7fa48aa00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4740754e0 0x7fa4740779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.029+0000 7fa48aa00640 1 -- 172.21.15.122:0/1564394089 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7fa48412d680 con 0x7fa478003c90 2024-09-17T05:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.029+0000 7fa492000640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4740754e0 0x7fa4740779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.029+0000 7fa492000640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4740754e0 0x7fa4740779a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa4941eac60 tx=0x7fa480006b80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.033+0000 7fa48aa00640 1 -- 172.21.15.122:0/1564394089 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa484016f80 con 0x7fa478003c90 2024-09-17T05:40:11.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.233+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa49415dc10 con 0x7fa478003c90 2024-09-17T05:40:11.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa48aa00640 1 -- 172.21.15.122:0/1564394089 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5234 (secure 0 0 0) 0x7fa4840f9b40 con 0x7fa478003c90 2024-09-17T05:40:11.232 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:11.233 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":87,"btime":"2024-09-17T05:39:21:803220+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/609693749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":609693749},{"type":"v1","addr":"172.21.15.122:6834","nonce":609693749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2826125653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2826125653},{"type":"v1","addr":"172.21.15.173:6836","nonce":2826125653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4138151052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4138151052},{"type":"v1","addr":"172.21.15.173:6837","nonce":4138151052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:39:09.055675+0000","modified":"2024-09-17T05:39:12.084809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2463174917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2463174917},{"type":"v1","addr":"172.21.15.122:6835","nonce":2463174917}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8916,"qdb_cluster":[8916]},"id":9}]} 2024-09-17T05:40:11.233 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 87 2024-09-17T05:40:11.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4740754e0 msgr2=0x7fa4740779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4740754e0 0x7fa4740779a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa4941eac60 tx=0x7fa480006b80 comp rx=0 tx=0).stop 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 msgr2=0x7fa4941ec000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 0x7fa4941ec000 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fa4840af040 tx=0x7fa484005520 comp rx=0 tx=0).stop 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 shutdown_connections 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa4740754e0 0x7fa4740779a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4940c6af0 0x7fa4940b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4941f9bf0 0x7fa4940c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 --2- 172.21.15.122:0/1564394089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa478003c90 0x7fa4941ec000 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 >> 172.21.15.122:0/1564394089 conn(0x7fa494073490 msgr2=0x7fa494074550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 shutdown_connections 2024-09-17T05:40:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.237+0000 7fa493e00640 1 -- 172.21.15.122:0/1564394089 wait complete. 2024-09-17T05:40:11.254 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:40:11.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a7e00640 1 -- 172.21.15.122:0/1309202343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2588004d50 msgr2=0x7f25880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a7e00640 1 --2- 172.21.15.122:0/1309202343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2588004d50 0x7f25880051d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f259800c310 tx=0x7f259801cfc0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a5e00640 1 -- 172.21.15.122:0/1309202343 reap_dead start 2024-09-17T05:40:11.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a7e00640 1 -- 172.21.15.122:0/1309202343 shutdown_connections 2024-09-17T05:40:11.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a7e00640 1 -- 172.21.15.122:0/1309202343 >> 172.21.15.122:0/1309202343 conn(0x7f25a00b18f0 msgr2=0x7f25a00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:11.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a7e00640 1 -- 172.21.15.122:0/1309202343 shutdown_connections 2024-09-17T05:40:11.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a7e00640 1 -- 172.21.15.122:0/1309202343 wait complete. 2024-09-17T05:40:11.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a7e00640 1 Processor -- start 2024-09-17T05:40:11.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.401+0000 7f25a7e00640 1 -- start start 2024-09-17T05:40:11.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588004d50 0x7f25a009c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a01f7f10 0x7f25a009aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25a01495f0 0x7f25a0158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25a006e920 con 0x7f25a01495f0 2024-09-17T05:40:11.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f25a006e7a0 con 0x7f2588004d50 2024-09-17T05:40:11.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f25a006eaa0 con 0x7f25a01f7f10 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25a01495f0 0x7f25a0158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25a01495f0 0x7f25a0158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43404/0 (socket says 172.21.15.122:43404) 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 -- 172.21.15.122:0/2006160837 learned_addr learned my addr 172.21.15.122:0/2006160837 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5400640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588004d50 0x7f25a009c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a4a00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a01f7f10 0x7f25a009aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 -- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a01f7f10 msgr2=0x7f25a009aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a01f7f10 0x7f25a009aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 -- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588004d50 msgr2=0x7f25a009c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588004d50 0x7f25a009c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 -- 172.21.15.122:0/2006160837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a009b240 con 0x7f25a01495f0 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a5e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25a01495f0 0x7f25a0158550 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f2598098b90 tx=0x7f2598001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:11.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f259d600640 1 -- 172.21.15.122:0/2006160837 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598000e50 con 0x7f25a01495f0 2024-09-17T05:40:11.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f259d600640 1 -- 172.21.15.122:0/2006160837 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2598006c60 con 0x7f25a01495f0 2024-09-17T05:40:11.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f259d600640 1 -- 172.21.15.122:0/2006160837 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980060d0 con 0x7f25a01495f0 2024-09-17T05:40:11.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a009b520 con 0x7f25a01495f0 2024-09-17T05:40:11.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a009bae0 con 0x7f25a01495f0 2024-09-17T05:40:11.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a0069410 con 0x7f25a01495f0 2024-09-17T05:40:11.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.405+0000 7f259d600640 1 -- 172.21.15.122:0/2006160837 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f2598003840 con 0x7f25a01495f0 2024-09-17T05:40:11.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.409+0000 7f259d600640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25800754e0 0x7f25800779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.409+0000 7f259d600640 1 -- 172.21.15.122:0/2006160837 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7f259812ce50 con 0x7f25a01495f0 2024-09-17T05:40:11.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.409+0000 7f25a5400640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25800754e0 0x7f25800779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.409+0000 7f25a5400640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25800754e0 0x7f25800779a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f25940124a0 tx=0x7f2594009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:11.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.413+0000 7f259d600640 1 -- 172.21.15.122:0/2006160837 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f25980bf030 con 0x7f25a01495f0 2024-09-17T05:40:11.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.581+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25a006ec80 con 0x7f25a01495f0 2024-09-17T05:40:11.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.581+0000 7f259d600640 1 -- 172.21.15.122:0/2006160837 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16991 (secure 0 0 0) 0x7f2598016d70 con 0x7f25a01495f0 2024-09-17T05:40:11.579 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:11.579 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":133,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:40:05.982895+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:39:06.760215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T05:39:08.026267+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:40:11.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25800754e0 msgr2=0x7f25800779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25800754e0 0x7f25800779a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f25940124a0 tx=0x7f2594009500 comp rx=0 tx=0).stop 2024-09-17T05:40:11.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25a01495f0 msgr2=0x7f25a0158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25a01495f0 0x7f25a0158550 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f2598098b90 tx=0x7f2598001a40 comp rx=0 tx=0).stop 2024-09-17T05:40:11.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 shutdown_connections 2024-09-17T05:40:11.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f25800754e0 0x7f25800779a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25a01495f0 0x7f25a0158550 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a01f7f10 0x7f25a009aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 --2- 172.21.15.122:0/2006160837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588004d50 0x7f25a009c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 >> 172.21.15.122:0/2006160837 conn(0x7f25a00b18f0 msgr2=0x7f25a0156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:11.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 shutdown_connections 2024-09-17T05:40:11.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.585+0000 7f25a7e00640 1 -- 172.21.15.122:0/2006160837 wait complete. 2024-09-17T05:40:11.601 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:40:11.602 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:40:11.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.745+0000 7f08f3c00640 1 -- 172.21.15.122:0/3027321666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 msgr2=0x7f08ec073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.745+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3027321666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 0x7f08ec073c20 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009840 tx=0x7f08d801c820 comp rx=0 tx=0).stop 2024-09-17T05:40:11.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.745+0000 7f08f3c00640 1 -- 172.21.15.122:0/3027321666 shutdown_connections 2024-09-17T05:40:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.745+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3027321666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 0x7f08ec073c20 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.745+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3027321666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ec15af60 0x7f08ec0cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.745+0000 7f08f3c00640 1 -- 172.21.15.122:0/3027321666 >> 172.21.15.122:0/3027321666 conn(0x7f08ec0d0c90 msgr2=0x7f08ec0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.745+0000 7f08f3c00640 1 -- 172.21.15.122:0/3027321666 shutdown_connections 2024-09-17T05:40:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.745+0000 7f08f3c00640 1 -- 172.21.15.122:0/3027321666 wait complete. 2024-09-17T05:40:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 Processor -- start 2024-09-17T05:40:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 -- start start 2024-09-17T05:40:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 0x7f08ec15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ec074160 0x7f08ec157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ec15af60 0x7f08ec158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec167b40 con 0x7f08ec0737c0 2024-09-17T05:40:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec070b70 con 0x7f08ec15af60 2024-09-17T05:40:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f08ec167cc0 con 0x7f08ec074160 2024-09-17T05:40:11.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 0x7f08ec15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 0x7f08ec15ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43422/0 (socket says 172.21.15.122:43422) 2024-09-17T05:40:11.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 -- 172.21.15.122:0/3447691451 learned_addr learned my addr 172.21.15.122:0/3447691451 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:11.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 -- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ec074160 msgr2=0x7f08ec157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ec074160 0x7f08ec157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 -- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ec15af60 msgr2=0x7f08ec158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ec15af60 0x7f08ec158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 -- 172.21.15.122:0/3447691451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ec1f7600 con 0x7f08ec0737c0 2024-09-17T05:40:11.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f1200640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 0x7f08ec15ca10 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00ead0 tx=0x7f08dc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:11.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08e8c00640 1 -- 172.21.15.122:0/3447691451 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc00ce40 con 0x7f08ec0737c0 2024-09-17T05:40:11.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08e8c00640 1 -- 172.21.15.122:0/3447691451 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08dc011470 con 0x7f08ec0737c0 2024-09-17T05:40:11.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ec1f78e0 con 0x7f08ec0737c0 2024-09-17T05:40:11.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08e8c00640 1 -- 172.21.15.122:0/3447691451 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0118a0 con 0x7f08ec0737c0 2024-09-17T05:40:11.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.749+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ec0c7320 con 0x7f08ec0737c0 2024-09-17T05:40:11.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.753+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ec073c20 con 0x7f08ec0737c0 2024-09-17T05:40:11.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.753+0000 7f08e8c00640 1 -- 172.21.15.122:0/3447691451 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f08dc011a40 con 0x7f08ec0737c0 2024-09-17T05:40:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.753+0000 7f08e8c00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08cc075510 0x7f08cc0779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.753+0000 7f08e8c00640 1 -- 172.21.15.122:0/3447691451 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7f08dc093fc0 con 0x7f08ec0737c0 2024-09-17T05:40:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.753+0000 7f08ebe00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08cc075510 0x7f08cc0779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.757+0000 7f08ebe00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08cc075510 0x7f08cc0779d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f08d80af000 tx=0x7f08d80195c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:11.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.757+0000 7f08e8c00640 1 -- 172.21.15.122:0/3447691451 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f08dc016030 con 0x7f08ec0737c0 2024-09-17T05:40:11.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.957+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08ec0b0230 con 0x7f08ec0737c0 2024-09-17T05:40:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.957+0000 7f08e8c00640 1 -- 172.21.15.122:0/3447691451 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5234 (secure 0 0 0) 0x7f08dc060b70 con 0x7f08ec0737c0 2024-09-17T05:40:11.955 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:11.955 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":87,"btime":"2024-09-17T05:39:21:803220+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/609693749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":609693749},{"type":"v1","addr":"172.21.15.122:6834","nonce":609693749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2826125653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2826125653},{"type":"v1","addr":"172.21.15.173:6836","nonce":2826125653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4138151052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4138151052},{"type":"v1","addr":"172.21.15.173:6837","nonce":4138151052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:39:09.055675+0000","modified":"2024-09-17T05:39:12.084809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2463174917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2463174917},{"type":"v1","addr":"172.21.15.122:6835","nonce":2463174917}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8916,"qdb_cluster":[8916]},"id":9}]} 2024-09-17T05:40:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 87 2024-09-17T05:40:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08cc075510 msgr2=0x7f08cc0779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08cc075510 0x7f08cc0779d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f08d80af000 tx=0x7f08d80195c0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 msgr2=0x7f08ec15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:11.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 0x7f08ec15ca10 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00ead0 tx=0x7f08dc00ef90 comp rx=0 tx=0).stop 2024-09-17T05:40:11.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 shutdown_connections 2024-09-17T05:40:11.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08cc075510 0x7f08cc0779d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ec15af60 0x7f08ec158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ec074160 0x7f08ec157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 --2- 172.21.15.122:0/3447691451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08ec0737c0 0x7f08ec15ca10 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:11.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 >> 172.21.15.122:0/3447691451 conn(0x7f08ec0d0c90 msgr2=0x7f08ec159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:11.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 shutdown_connections 2024-09-17T05:40:11.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:11.961+0000 7f08f3c00640 1 -- 172.21.15.122:0/3447691451 wait complete. 2024-09-17T05:40:11.977 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:40:12.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- 172.21.15.122:0/1420813577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47806e7a0 msgr2=0x7ff47806ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 --2- 172.21.15.122:0/1420813577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47806e7a0 0x7ff47806ec30 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff470009900 tx=0x7ff47001f0d0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- 172.21.15.122:0/1420813577 shutdown_connections 2024-09-17T05:40:12.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 --2- 172.21.15.122:0/1420813577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47806e7a0 0x7ff47806ec30 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 --2- 172.21.15.122:0/1420813577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff47815c3e0 0x7ff47815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- 172.21.15.122:0/1420813577 >> 172.21.15.122:0/1420813577 conn(0x7ff4780b18f0 msgr2=0x7ff4780b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:12.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- 172.21.15.122:0/1420813577 shutdown_connections 2024-09-17T05:40:12.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- 172.21.15.122:0/1420813577 wait complete. 2024-09-17T05:40:12.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 Processor -- start 2024-09-17T05:40:12.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- start start 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff47806e7a0 0x7ff478147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff47815bfa0 0x7ff478145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47815c3e0 0x7ff47809acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff47806f2f0 con 0x7ff47815c3e0 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff47806f170 con 0x7ff47806e7a0 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.125+0000 7ff47ce00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff47806f470 con 0x7ff47815bfa0 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47815c3e0 0x7ff47809acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47815c3e0 0x7ff47809acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43440/0 (socket says 172.21.15.122:43440) 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 -- 172.21.15.122:0/251152531 learned_addr learned my addr 172.21.15.122:0/251152531 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:12.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff46d400640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff47815bfa0 0x7ff478145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff475e00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff47806e7a0 0x7ff478147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 -- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff47815bfa0 msgr2=0x7ff478145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff47815bfa0 0x7ff478145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 -- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff47806e7a0 msgr2=0x7ff478147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff47806e7a0 0x7ff478147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 -- 172.21.15.122:0/251152531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47809b1f0 con 0x7ff47815c3e0 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff476800640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47815c3e0 0x7ff47809acb0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff4700b3040 tx=0x7ff470005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff46ea00640 1 -- 172.21.15.122:0/251152531 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4700b5040 con 0x7ff47815c3e0 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff46ea00640 1 -- 172.21.15.122:0/251152531 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff470005330 con 0x7ff47815c3e0 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4781481e0 con 0x7ff47815c3e0 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff46ea00640 1 -- 172.21.15.122:0/251152531 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4700046d0 con 0x7ff47815c3e0 2024-09-17T05:40:12.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4781487a0 con 0x7ff47815c3e0 2024-09-17T05:40:12.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff478069410 con 0x7ff47815c3e0 2024-09-17T05:40:12.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.129+0000 7ff46ea00640 1 -- 172.21.15.122:0/251152531 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7ff470004870 con 0x7ff47815c3e0 2024-09-17T05:40:12.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.133+0000 7ff46ea00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4540754e0 0x7ff4540779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.133+0000 7ff46ea00640 1 -- 172.21.15.122:0/251152531 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7ff470130470 con 0x7ff47815c3e0 2024-09-17T05:40:12.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.133+0000 7ff475e00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4540754e0 0x7ff4540779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.133+0000 7ff475e00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4540754e0 0x7ff4540779a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff468009ad0 tx=0x7ff4680091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:12.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.137+0000 7ff46ea00640 1 -- 172.21.15.122:0/251152531 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff4700bf060 con 0x7ff47815c3e0 2024-09-17T05:40:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.337+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4781489e0 con 0x7ff47815c3e0 2024-09-17T05:40:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.337+0000 7ff46ea00640 1 -- 172.21.15.122:0/251152531 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5234 (secure 0 0 0) 0x7ff4781489e0 con 0x7ff47815c3e0 2024-09-17T05:40:12.335 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:12.335 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":87,"btime":"2024-09-17T05:39:21:803220+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/609693749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":609693749},{"type":"v1","addr":"172.21.15.122:6834","nonce":609693749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2826125653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2826125653},{"type":"v1","addr":"172.21.15.173:6836","nonce":2826125653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4138151052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4138151052},{"type":"v1","addr":"172.21.15.173:6837","nonce":4138151052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:39:09.055675+0000","modified":"2024-09-17T05:39:12.084809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8916},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8916":{"gid":8916,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/2463174917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2463174917},{"type":"v1","addr":"172.21.15.122:6835","nonce":2463174917}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8916,"qdb_cluster":[8916]},"id":9}]} 2024-09-17T05:40:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 87 2024-09-17T05:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4540754e0 msgr2=0x7ff4540779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4540754e0 0x7ff4540779a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff468009ad0 tx=0x7ff4680091e0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47815c3e0 msgr2=0x7ff47809acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47815c3e0 0x7ff47809acb0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff4700b3040 tx=0x7ff470005d80 comp rx=0 tx=0).stop 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 shutdown_connections 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff4540754e0 0x7ff4540779a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff47815c3e0 0x7ff47809acb0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff47815bfa0 0x7ff478145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 --2- 172.21.15.122:0/251152531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff47806e7a0 0x7ff478147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 >> 172.21.15.122:0/251152531 conn(0x7ff4780b18f0 msgr2=0x7ff4780b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 shutdown_connections 2024-09-17T05:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.341+0000 7ff47ce00640 1 -- 172.21.15.122:0/251152531 wait complete. 2024-09-17T05:40:12.356 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:40:12.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.501+0000 7fcaf8600640 1 -- 172.21.15.122:0/904461961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf015bea0 msgr2=0x7fcaf00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.501+0000 7fcaf8600640 1 --2- 172.21.15.122:0/904461961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf015bea0 0x7fcaf00ce670 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009c80 tx=0x7fcae401cda0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.501+0000 7fcaf8600640 1 -- 172.21.15.122:0/904461961 shutdown_connections 2024-09-17T05:40:12.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.501+0000 7fcaf8600640 1 --2- 172.21.15.122:0/904461961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcaf00cdb30 0x7fcaf00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.501+0000 7fcaf8600640 1 --2- 172.21.15.122:0/904461961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf015bea0 0x7fcaf00ce670 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.501+0000 7fcaf8600640 1 -- 172.21.15.122:0/904461961 >> 172.21.15.122:0/904461961 conn(0x7fcaf00d1dc0 msgr2=0x7fcaf00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:12.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.501+0000 7fcaf8600640 1 -- 172.21.15.122:0/904461961 shutdown_connections 2024-09-17T05:40:12.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.501+0000 7fcaf8600640 1 -- 172.21.15.122:0/904461961 wait complete. 2024-09-17T05:40:12.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 Processor -- start 2024-09-17T05:40:12.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 -- start start 2024-09-17T05:40:12.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf00cdb30 0x7fcaf00c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcaf00d28e0 0x7fcaf00b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf015bea0 0x7fcaf00ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0168fc0 con 0x7fcaf015bea0 2024-09-17T05:40:12.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0168e40 con 0x7fcaf00cdb30 2024-09-17T05:40:12.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf0169140 con 0x7fcaf00d28e0 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf015bea0 0x7fcaf00ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf015bea0 0x7fcaf00ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43450/0 (socket says 172.21.15.122:43450) 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 -- 172.21.15.122:0/816879282 learned_addr learned my addr 172.21.15.122:0/816879282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf5200640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcaf00d28e0 0x7fcaf00b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf5c00640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf00cdb30 0x7fcaf00c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 -- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcaf00d28e0 msgr2=0x7fcaf00b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcaf00d28e0 0x7fcaf00b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 -- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf00cdb30 msgr2=0x7fcaf00c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf00cdb30 0x7fcaf00c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 -- 172.21.15.122:0/816879282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf00c8210 con 0x7fcaf015bea0 2024-09-17T05:40:12.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf6600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf015bea0 0x7fcaf00ca980 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fcae8001ec0 tx=0x7fcae800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:12.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaed600640 1 -- 172.21.15.122:0/816879282 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017070 con 0x7fcaf015bea0 2024-09-17T05:40:12.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaed600640 1 -- 172.21.15.122:0/816879282 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcae80053c0 con 0x7fcaf015bea0 2024-09-17T05:40:12.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf0155b80 con 0x7fcaf015bea0 2024-09-17T05:40:12.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaed600640 1 -- 172.21.15.122:0/816879282 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8003860 con 0x7fcaf015bea0 2024-09-17T05:40:12.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf0156140 con 0x7fcaf015bea0 2024-09-17T05:40:12.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.505+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf00cc9d0 con 0x7fcaf015bea0 2024-09-17T05:40:12.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.509+0000 7fcaed600640 1 -- 172.21.15.122:0/816879282 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fcae801a070 con 0x7fcaf015bea0 2024-09-17T05:40:12.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.509+0000 7fcaed600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcad00754e0 0x7fcad00779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.509+0000 7fcaed600640 1 -- 172.21.15.122:0/816879282 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7fcae8092b50 con 0x7fcaf015bea0 2024-09-17T05:40:12.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.509+0000 7fcaf5c00640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcad00754e0 0x7fcad00779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.509+0000 7fcaf5c00640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcad00754e0 0x7fcad00779a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcae40af000 tx=0x7fcae401b5e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:12.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.513+0000 7fcaed600640 1 -- 172.21.15.122:0/816879282 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcae8021020 con 0x7fcaf015bea0 2024-09-17T05:40:12.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcaf00aff50 con 0x7fcaf015bea0 2024-09-17T05:40:12.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaed600640 1 -- 172.21.15.122:0/816879282 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16991 (secure 0 0 0) 0x7fcae805f010 con 0x7fcaf015bea0 2024-09-17T05:40:12.682 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:12.682 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":133,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:40:05.982895+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:39:06.760215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-17T05:39:08.026267+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:40:12.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcad00754e0 msgr2=0x7fcad00779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcad00754e0 0x7fcad00779a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcae40af000 tx=0x7fcae401b5e0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf015bea0 msgr2=0x7fcaf00ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf015bea0 0x7fcaf00ca980 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fcae8001ec0 tx=0x7fcae800c890 comp rx=0 tx=0).stop 2024-09-17T05:40:12.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 shutdown_connections 2024-09-17T05:40:12.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcad00754e0 0x7fcad00779a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcaf015bea0 0x7fcaf00ca980 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcaf00d28e0 0x7fcaf00b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.685+0000 7fcaf8600640 1 --2- 172.21.15.122:0/816879282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaf00cdb30 0x7fcaf00c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.689+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 >> 172.21.15.122:0/816879282 conn(0x7fcaf00d1dc0 msgr2=0x7fcaf00cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:12.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.689+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 shutdown_connections 2024-09-17T05:40:12.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.689+0000 7fcaf8600640 1 -- 172.21.15.122:0/816879282 wait complete. 2024-09-17T05:40:12.703 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:40:12.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.849+0000 7faf1f200640 1 -- 172.21.15.122:0/3185830038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf04003a60 msgr2=0x7faf04003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.849+0000 7faf1f200640 1 --2- 172.21.15.122:0/3185830038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf04003a60 0x7faf04003e80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7faf1000bb10 tx=0x7faf1001ccf0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.849+0000 7faf1e800640 1 -- 172.21.15.122:0/3185830038 reap_dead start 2024-09-17T05:40:12.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.849+0000 7faf1f200640 1 -- 172.21.15.122:0/3185830038 shutdown_connections 2024-09-17T05:40:12.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.849+0000 7faf1f200640 1 -- 172.21.15.122:0/3185830038 >> 172.21.15.122:0/3185830038 conn(0x7faf20073490 msgr2=0x7faf200d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:12.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.849+0000 7faf1f200640 1 -- 172.21.15.122:0/3185830038 shutdown_connections 2024-09-17T05:40:12.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.849+0000 7faf1f200640 1 -- 172.21.15.122:0/3185830038 wait complete. 2024-09-17T05:40:12.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 Processor -- start 2024-09-17T05:40:12.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 -- start start 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf04003a60 0x7faf20099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf200afd20 0x7faf20159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf200b1fb0 0x7faf200b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faf20166ea0 con 0x7faf200afd20 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf20166d20 con 0x7faf200b1fb0 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faf20167020 con 0x7faf04003a60 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf200afd20 0x7faf20159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf200afd20 0x7faf20159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43478/0 (socket says 172.21.15.122:43478) 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 -- 172.21.15.122:0/69509099 learned_addr learned my addr 172.21.15.122:0/69509099 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:12.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1de00640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf04003a60 0x7faf20099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1e800640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf200b1fb0 0x7faf200b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 -- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf04003a60 msgr2=0x7faf20099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf04003a60 0x7faf20099850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 -- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf200b1fb0 msgr2=0x7faf200b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf200b1fb0 0x7faf200b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 -- 172.21.15.122:0/69509099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf200b0250 con 0x7faf200afd20 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1de00640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf04003a60 0x7faf20099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1e800640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf200b1fb0 0x7faf200b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf15400640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf200afd20 0x7faf20159620 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7faf08003f60 tx=0x7faf08003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf16a00640 1 -- 172.21.15.122:0/69509099 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08005220 con 0x7faf200afd20 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf200b0530 con 0x7faf200afd20 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf16a00640 1 -- 172.21.15.122:0/69509099 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf080053c0 con 0x7faf200afd20 2024-09-17T05:40:12.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf16a00640 1 -- 172.21.15.122:0/69509099 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080047b0 con 0x7faf200afd20 2024-09-17T05:40:12.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.853+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf200b0af0 con 0x7faf200afd20 2024-09-17T05:40:12.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.857+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf201602b0 con 0x7faf200afd20 2024-09-17T05:40:12.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.857+0000 7faf16a00640 1 -- 172.21.15.122:0/69509099 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7faf08004970 con 0x7faf200afd20 2024-09-17T05:40:12.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.857+0000 7faf16a00640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faefc0754e0 0x7faefc0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:12.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.857+0000 7faf16a00640 1 -- 172.21.15.122:0/69509099 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5849+0+0 (secure 0 0 0) 0x7faf08093f70 con 0x7faf200afd20 2024-09-17T05:40:12.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.857+0000 7faf1de00640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faefc0754e0 0x7faefc0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:12.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.861+0000 7faf1de00640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faefc0754e0 0x7faefc0779a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faf100b0000 tx=0x7faf100a7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:12.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:12.861+0000 7faf16a00640 1 -- 172.21.15.122:0/69509099 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7faf08060a00 con 0x7faf200afd20 2024-09-17T05:40:13.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.065+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7faf200cdc50 con 0x7faf200afd20 2024-09-17T05:40:13.063 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:40:13.065+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:40:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.069+0000 7faf16a00640 1 -- 172.21.15.122:0/69509099 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) ==== 161+0+0 (secure 0 0 0) 0x7faf08060300 con 0x7faf200afd20 2024-09-17T05:40:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:40:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faefc0754e0 msgr2=0x7faefc0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faefc0754e0 0x7faefc0779a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7faf100b0000 tx=0x7faf100a7040 comp rx=0 tx=0).stop 2024-09-17T05:40:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf200afd20 msgr2=0x7faf20159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf200afd20 0x7faf20159620 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7faf08003f60 tx=0x7faf08003f90 comp rx=0 tx=0).stop 2024-09-17T05:40:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 shutdown_connections 2024-09-17T05:40:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faefc0754e0 0x7faefc0779a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf200b1fb0 0x7faf200b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf200afd20 0x7faf20159620 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 --2- 172.21.15.122:0/69509099 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf04003a60 0x7faf20099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 >> 172.21.15.122:0/69509099 conn(0x7faf20073490 msgr2=0x7faf200759d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:13.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 shutdown_connections 2024-09-17T05:40:13.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.073+0000 7faf1f200640 1 -- 172.21.15.122:0/69509099 wait complete. 2024-09-17T05:40:13.090 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:40:13.134 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:40:13.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- 172.21.15.122:0/2064851895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3900d2150 msgr2=0x7fd39015c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:13.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 --2- 172.21.15.122:0/2064851895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3900d2150 0x7fd39015c800 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009bc0 tx=0x7fd38c01cae0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- 172.21.15.122:0/2064851895 shutdown_connections 2024-09-17T05:40:13.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 --2- 172.21.15.122:0/2064851895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3900d2150 0x7fd39015c800 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 --2- 172.21.15.122:0/2064851895 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd39015bc80 0x7fd3900cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- 172.21.15.122:0/2064851895 >> 172.21.15.122:0/2064851895 conn(0x7fd39009aa10 msgr2=0x7fd39009adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:13.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- 172.21.15.122:0/2064851895 shutdown_connections 2024-09-17T05:40:13.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- 172.21.15.122:0/2064851895 wait complete. 2024-09-17T05:40:13.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 Processor -- start 2024-09-17T05:40:13.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- start start 2024-09-17T05:40:13.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3900d2150 0x7fd390155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:13.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd39015bc80 0x7fd390156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:13.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39015c060 0x7fd390156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3900cd130 con 0x7fd39015c060 2024-09-17T05:40:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3900ccfb0 con 0x7fd3900d2150 2024-09-17T05:40:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd398800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd3900cd2b0 con 0x7fd39015bc80 2024-09-17T05:40:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd396800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39015c060 0x7fd390156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:13.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd396800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39015c060 0x7fd390156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43512/0 (socket says 172.21.15.122:43512) 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.241+0000 7fd396800640 1 -- 172.21.15.122:0/2303004607 learned_addr learned my addr 172.21.15.122:0/2303004607 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd395400640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd39015bc80 0x7fd390156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd395e00640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3900d2150 0x7fd390155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd396800640 1 -- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd39015bc80 msgr2=0x7fd390156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd396800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd39015bc80 0x7fd390156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd396800640 1 -- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3900d2150 msgr2=0x7fd390155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd396800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3900d2150 0x7fd390155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd396800640 1 -- 172.21.15.122:0/2303004607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3900aeb10 con 0x7fd39015c060 2024-09-17T05:40:13.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd395e00640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3900d2150 0x7fd390155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd396800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39015c060 0x7fd390156d90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd38c01a710 tx=0x7fd38c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd38a000640 1 -- 172.21.15.122:0/2303004607 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c0bb070 con 0x7fd39015c060 2024-09-17T05:40:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd38a000640 1 -- 172.21.15.122:0/2303004607 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd38c004d40 con 0x7fd39015c060 2024-09-17T05:40:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3901556d0 con 0x7fd39015c060 2024-09-17T05:40:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd38a000640 1 -- 172.21.15.122:0/2303004607 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c0b2040 con 0x7fd39015c060 2024-09-17T05:40:13.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3900aed80 con 0x7fd39015c060 2024-09-17T05:40:13.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39015ac60 con 0x7fd39015c060 2024-09-17T05:40:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd38a000640 1 -- 172.21.15.122:0/2303004607 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fd38c0be070 con 0x7fd39015c060 2024-09-17T05:40:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.245+0000 7fd38a000640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3740754e0 0x7fd3740779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.249+0000 7fd38a000640 1 -- 172.21.15.122:0/2303004607 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5907+0+0 (secure 0 0 0) 0x7fd38c12ce30 con 0x7fd39015c060 2024-09-17T05:40:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.249+0000 7fd395e00640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3740754e0 0x7fd3740779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.249+0000 7fd395e00640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3740754e0 0x7fd3740779a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd3900cf410 tx=0x7fd3840091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:13.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.249+0000 7fd38a000640 1 -- 172.21.15.122:0/2303004607 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd3901556d0 con 0x7fd39015c060 2024-09-17T05:40:13.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:13.449+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd39015c800 con 0x7fd39015c060 2024-09-17T05:40:14.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.069+0000 7fd38a000640 1 -- 172.21.15.122:0/2303004607 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) ==== 104+0+0 (secure 0 0 0) 0x7fd38c016790 con 0x7fd39015c060 2024-09-17T05:40:14.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3740754e0 msgr2=0x7fd3740779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:14.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3740754e0 0x7fd3740779a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd3900cf410 tx=0x7fd3840091e0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39015c060 msgr2=0x7fd390156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39015c060 0x7fd390156d90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd38c01a710 tx=0x7fd38c003f60 comp rx=0 tx=0).stop 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 shutdown_connections 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd3740754e0 0x7fd3740779a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39015c060 0x7fd390156d90 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd39015bc80 0x7fd390156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 --2- 172.21.15.122:0/2303004607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3900d2150 0x7fd390155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 >> 172.21.15.122:0/2303004607 conn(0x7fd39009aa10 msgr2=0x7fd39015e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 shutdown_connections 2024-09-17T05:40:14.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.073+0000 7fd398800640 1 -- 172.21.15.122:0/2303004607 wait complete. 2024-09-17T05:40:14.098 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:40:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.293+0000 7f3162e00640 1 -- 172.21.15.122:0/32470801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c0d28e0 msgr2=0x7f315c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.293+0000 7f3162e00640 1 --2- 172.21.15.122:0/32470801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c0d28e0 0x7f315c0d0170 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3148009840 tx=0x7f314801c820 comp rx=0 tx=0).stop 2024-09-17T05:40:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.293+0000 7f3162e00640 1 -- 172.21.15.122:0/32470801 shutdown_connections 2024-09-17T05:40:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.293+0000 7f3162e00640 1 --2- 172.21.15.122:0/32470801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c0d28e0 0x7f315c0d0170 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.293+0000 7f3162e00640 1 --2- 172.21.15.122:0/32470801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f315c15f390 0x7f315c0cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.293+0000 7f3162e00640 1 -- 172.21.15.122:0/32470801 >> 172.21.15.122:0/32470801 conn(0x7f315c0d1dc0 msgr2=0x7f315c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.293+0000 7f3162e00640 1 -- 172.21.15.122:0/32470801 shutdown_connections 2024-09-17T05:40:14.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.293+0000 7f3162e00640 1 -- 172.21.15.122:0/32470801 wait complete. 2024-09-17T05:40:14.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 Processor -- start 2024-09-17T05:40:14.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 -- start start 2024-09-17T05:40:14.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f315c0d06b0 0x7f315c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:14.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f315c0d28e0 0x7f315c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:14.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c15f390 0x7f315c150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:14.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f315c169110 con 0x7f315c15f390 2024-09-17T05:40:14.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f315c168f90 con 0x7f315c0d06b0 2024-09-17T05:40:14.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f315c169290 con 0x7f315c0d28e0 2024-09-17T05:40:14.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c15f390 0x7f315c150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:14.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c15f390 0x7f315c150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43528/0 (socket says 172.21.15.122:43528) 2024-09-17T05:40:14.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 -- 172.21.15.122:0/3163375053 learned_addr learned my addr 172.21.15.122:0/3163375053 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:14.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f315b400640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f315c0d28e0 0x7f315c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:14.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f315be00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f315c0d06b0 0x7f315c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 -- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f315c0d28e0 msgr2=0x7f315c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f315c0d28e0 0x7f315c14f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 -- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f315c0d06b0 msgr2=0x7f315c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f315c0d06b0 0x7f315c0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 -- 172.21.15.122:0/3163375053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c1501a0 con 0x7f315c15f390 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3160e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c15f390 0x7f315c150690 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f315400c410 tx=0x7f315400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3153e00640 1 -- 172.21.15.122:0/3163375053 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154011790 con 0x7f315c15f390 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3153e00640 1 -- 172.21.15.122:0/3163375053 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3154017070 con 0x7f315c15f390 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315c1f6100 con 0x7f315c15f390 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3153e00640 1 -- 172.21.15.122:0/3163375053 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154011cc0 con 0x7f315c15f390 2024-09-17T05:40:14.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.297+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315c1f66c0 con 0x7f315c15f390 2024-09-17T05:40:14.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.301+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f315c1f68c0 con 0x7f315c15f390 2024-09-17T05:40:14.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.301+0000 7f3153e00640 1 -- 172.21.15.122:0/3163375053 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f315401c460 con 0x7f315c15f390 2024-09-17T05:40:14.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.301+0000 7f3153e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f313c0754e0 0x7f313c0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:14.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.301+0000 7f3153e00640 1 -- 172.21.15.122:0/3163375053 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5907+0+0 (secure 0 0 0) 0x7f31540930b0 con 0x7f315c15f390 2024-09-17T05:40:14.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.301+0000 7f315be00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f313c0754e0 0x7f313c0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:14.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.301+0000 7f315be00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f313c0754e0 0x7f313c0779a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f314c009ec0 tx=0x7f314c009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:14.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.305+0000 7f3153e00640 1 -- 172.21.15.122:0/3163375053 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f315c1f68c0 con 0x7f315c15f390 2024-09-17T05:40:14.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:14.477+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f315c0b1350 con 0x7f315c15f390 2024-09-17T05:40:15.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.089+0000 7f3153e00640 1 -- 172.21.15.122:0/3163375053 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) ==== 181+0+0 (secure 0 0 0) 0x7f315405f570 con 0x7f315c15f390 2024-09-17T05:40:15.088 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:40:15.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f313c0754e0 msgr2=0x7f313c0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:15.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f313c0754e0 0x7f313c0779a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f314c009ec0 tx=0x7f314c009500 comp rx=0 tx=0).stop 2024-09-17T05:40:15.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c15f390 msgr2=0x7f315c150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:15.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c15f390 0x7f315c150690 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f315400c410 tx=0x7f315400c8d0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 shutdown_connections 2024-09-17T05:40:15.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f313c0754e0 0x7f313c0779a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f315c15f390 0x7f315c150690 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f315c0d28e0 0x7f315c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 --2- 172.21.15.122:0/3163375053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f315c0d06b0 0x7f315c0cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 >> 172.21.15.122:0/3163375053 conn(0x7f315c0d1dc0 msgr2=0x7f315c15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:15.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 shutdown_connections 2024-09-17T05:40:15.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.093+0000 7f3162e00640 1 -- 172.21.15.122:0/3163375053 wait complete. 2024-09-17T05:40:15.112 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:40:15.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.261+0000 7f9f35400640 1 -- 172.21.15.122:0/1716699767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3015be10 msgr2=0x7f9f300ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:15.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.261+0000 7f9f35400640 1 --2- 172.21.15.122:0/1716699767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3015be10 0x7f9f300ce700 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9f300746b0 tx=0x7f9f2001eff0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.261+0000 7f9f35400640 1 -- 172.21.15.122:0/1716699767 shutdown_connections 2024-09-17T05:40:15.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.261+0000 7f9f35400640 1 --2- 172.21.15.122:0/1716699767 >> v2:172.21.15.173:3300/0 conn(0x7f9f3015c1f0 0x7f9f300cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.261+0000 7f9f35400640 1 --2- 172.21.15.122:0/1716699767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3015be10 0x7f9f300ce700 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.261+0000 7f9f35400640 1 -- 172.21.15.122:0/1716699767 >> 172.21.15.122:0/1716699767 conn(0x7f9f300d1da0 msgr2=0x7f9f300d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:15.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.261+0000 7f9f35400640 1 -- 172.21.15.122:0/1716699767 shutdown_connections 2024-09-17T05:40:15.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.261+0000 7f9f35400640 1 -- 172.21.15.122:0/1716699767 wait complete. 2024-09-17T05:40:15.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 Processor -- start 2024-09-17T05:40:15.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 -- start start 2024-09-17T05:40:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f300b0b60 0x7f9f301f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3015be10 0x7f9f3006f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3015c1f0 0x7f9f30156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f300cf510 con 0x7f9f300b0b60 2024-09-17T05:40:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f300cf390 con 0x7f9f3015c1f0 2024-09-17T05:40:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9f30160290 con 0x7f9f3015be10 2024-09-17T05:40:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f300b0b60 0x7f9f301f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f300b0b60 0x7f9f301f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43536/0 (socket says 172.21.15.122:43536) 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 -- 172.21.15.122:0/3770391953 learned_addr learned my addr 172.21.15.122:0/3770391953 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f25e00640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3015be10 0x7f9f3006f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2f200640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3015c1f0 0x7f9f30156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 -- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3015be10 msgr2=0x7f9f3006f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3015be10 0x7f9f3006f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 -- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3015c1f0 msgr2=0x7f9f30156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3015c1f0 0x7f9f30156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 -- 172.21.15.122:0/3770391953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f301f2530 con 0x7f9f300b0b60 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f25e00640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3015be10 0x7f9f3006f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2f200640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3015c1f0 0x7f9f30156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f2e800640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f300b0b60 0x7f9f301f1ff0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9f20000c10 tx=0x7f9f200a8ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f27400640 1 -- 172.21.15.122:0/3770391953 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f200b5040 con 0x7f9f300b0b60 2024-09-17T05:40:15.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f27400640 1 -- 172.21.15.122:0/3770391953 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f200bf070 con 0x7f9f300b0b60 2024-09-17T05:40:15.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f27400640 1 -- 172.21.15.122:0/3770391953 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f200b1c50 con 0x7f9f300b0b60 2024-09-17T05:40:15.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f30159910 con 0x7f9f300b0b60 2024-09-17T05:40:15.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.265+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f30159ed0 con 0x7f9f300b0b60 2024-09-17T05:40:15.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.269+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f301607b0 con 0x7f9f300b0b60 2024-09-17T05:40:15.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.269+0000 7f9f27400640 1 -- 172.21.15.122:0/3770391953 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f9f20003600 con 0x7f9f300b0b60 2024-09-17T05:40:15.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.269+0000 7f9f27400640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f0c0754e0 0x7f9f0c0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:15.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.269+0000 7f9f27400640 1 -- 172.21.15.122:0/3770391953 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5441+0+0 (secure 0 0 0) 0x7f9f20130ed0 con 0x7f9f300b0b60 2024-09-17T05:40:15.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.269+0000 7f9f25e00640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f0c0754e0 0x7f9f0c0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:15.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.269+0000 7f9f25e00640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f0c0754e0 0x7f9f0c0779a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005fd0 tx=0x7f9f18005dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:15.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.273+0000 7f9f27400640 1 -- 172.21.15.122:0/3770391953 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9f20018b50 con 0x7f9f300b0b60 2024-09-17T05:40:15.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:15.441+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9f300af4e0 con 0x7f9f300b0b60 2024-09-17T05:40:16.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.097+0000 7f9f27400640 1 -- 172.21.15.122:0/3770391953 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) ==== 169+0+0 (secure 0 0 0) 0x7f9f200fdaf0 con 0x7f9f300b0b60 2024-09-17T05:40:16.095 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:40:16.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.101+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f0c0754e0 msgr2=0x7f9f0c0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.101+0000 7f9f35400640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f0c0754e0 0x7f9f0c0779a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9f18005fd0 tx=0x7f9f18005dc0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.101+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f300b0b60 msgr2=0x7f9f301f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.101+0000 7f9f35400640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f300b0b60 0x7f9f301f1ff0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9f20000c10 tx=0x7f9f200a8ca0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.101+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 shutdown_connections 2024-09-17T05:40:16.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.105+0000 7f9f35400640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f0c0754e0 0x7f9f0c0779a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.105+0000 7f9f35400640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3015c1f0 0x7f9f30156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.105+0000 7f9f35400640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3015be10 0x7f9f3006f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.105+0000 7f9f35400640 1 --2- 172.21.15.122:0/3770391953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f300b0b60 0x7f9f301f1ff0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.105+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 >> 172.21.15.122:0/3770391953 conn(0x7f9f300d1da0 msgr2=0x7f9f3014def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:16.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.105+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 shutdown_connections 2024-09-17T05:40:16.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.105+0000 7f9f35400640 1 -- 172.21.15.122:0/3770391953 wait complete. 2024-09-17T05:40:16.126 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:40:16.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- 172.21.15.122:0/1724039247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740ca950 msgr2=0x7f987406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 --2- 172.21.15.122:0/1724039247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740ca950 0x7f987406e960 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f98740b2650 tx=0x7f986401f580 comp rx=0 tx=0).stop 2024-09-17T05:40:16.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- 172.21.15.122:0/1724039247 shutdown_connections 2024-09-17T05:40:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 --2- 172.21.15.122:0/1724039247 >> v2:172.21.15.173:3300/0 conn(0x7f98740af120 0x7f98740cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 --2- 172.21.15.122:0/1724039247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98740707b0 0x7f9874072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 --2- 172.21.15.122:0/1724039247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740ca950 0x7f987406e960 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- 172.21.15.122:0/1724039247 >> 172.21.15.122:0/1724039247 conn(0x7f98740cfa30 msgr2=0x7f98740cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- 172.21.15.122:0/1724039247 shutdown_connections 2024-09-17T05:40:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- 172.21.15.122:0/1724039247 wait complete. 2024-09-17T05:40:16.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 Processor -- start 2024-09-17T05:40:16.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- start start 2024-09-17T05:40:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98740707b0 0x7f98741f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740af120 0x7f9874157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98740ca950 0x7f98740b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9874073810 con 0x7f98740af120 2024-09-17T05:40:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9874073690 con 0x7f98740ca950 2024-09-17T05:40:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.273+0000 7f9873e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9874073990 con 0x7f98740707b0 2024-09-17T05:40:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740af120 0x7f9874157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:16.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740af120 0x7f9874157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43556/0 (socket says 172.21.15.122:43556) 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 -- 172.21.15.122:0/322690031 learned_addr learned my addr 172.21.15.122:0/322690031 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f9873400640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98740ca950 0x7f98740b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f9872a00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98740707b0 0x7f98741f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 -- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98740707b0 msgr2=0x7f98741f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98740707b0 0x7f98741f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 -- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98740ca950 msgr2=0x7f98740b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98740ca950 0x7f98740b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 -- 172.21.15.122:0/322690031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98741f8ae0 con 0x7f98740af120 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986a000640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740af120 0x7f9874157df0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f985c00c970 tx=0x7f985c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:16.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986b400640 1 -- 172.21.15.122:0/322690031 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c007c70 con 0x7f98740af120 2024-09-17T05:40:16.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986b400640 1 -- 172.21.15.122:0/322690031 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f985c00f450 con 0x7f98740af120 2024-09-17T05:40:16.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98741f8240 con 0x7f98740af120 2024-09-17T05:40:16.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986b400640 1 -- 172.21.15.122:0/322690031 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c00f8a0 con 0x7f98740af120 2024-09-17T05:40:16.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98741f4780 con 0x7f98740af120 2024-09-17T05:40:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98741f8630 con 0x7f98740af120 2024-09-17T05:40:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.277+0000 7f986b400640 1 -- 172.21.15.122:0/322690031 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f985c00fa40 con 0x7f98740af120 2024-09-17T05:40:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.281+0000 7f986b400640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9850075510 0x7f98500779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.281+0000 7f986b400640 1 -- 172.21.15.122:0/322690031 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7f985c093a10 con 0x7f98740af120 2024-09-17T05:40:16.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.281+0000 7f9872a00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9850075510 0x7f98500779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:16.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.281+0000 7f9872a00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9850075510 0x7f98500779d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f98640b2000 tx=0x7f986401ddc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:16.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.285+0000 7f986b400640 1 -- 172.21.15.122:0/322690031 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f985c014030 con 0x7f98740af120 2024-09-17T05:40:16.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.453+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f98740cad30 con 0x7f98740af120 2024-09-17T05:40:16.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f986b400640 1 -- 172.21.15.122:0/322690031 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v53) ==== 71+0+0 (secure 0 0 0) 0x7f985c05ff70 con 0x7f98740af120 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9850075510 msgr2=0x7f98500779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9850075510 0x7f98500779d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f98640b2000 tx=0x7f986401ddc0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740af120 msgr2=0x7f9874157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740af120 0x7f9874157df0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f985c00c970 tx=0x7f985c00ce30 comp rx=0 tx=0).stop 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 shutdown_connections 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9850075510 0x7f98500779d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98740ca950 0x7f98740b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98740af120 0x7f9874157df0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 --2- 172.21.15.122:0/322690031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98740707b0 0x7f98741f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 >> 172.21.15.122:0/322690031 conn(0x7f98740cfa30 msgr2=0x7f9874073c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 shutdown_connections 2024-09-17T05:40:16.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.457+0000 7f9873e00640 1 -- 172.21.15.122:0/322690031 wait complete. 2024-09-17T05:40:16.475 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-17T05:40:16.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.625+0000 7f0e38c00640 1 -- 172.21.15.122:0/3796079698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3415c1c0 msgr2=0x7f0e3406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.625+0000 7f0e38c00640 1 --2- 172.21.15.122:0/3796079698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3415c1c0 0x7f0e3406d000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009840 tx=0x7f0e2001c710 comp rx=0 tx=0).stop 2024-09-17T05:40:16.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.625+0000 7f0e38c00640 1 -- 172.21.15.122:0/3796079698 shutdown_connections 2024-09-17T05:40:16.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.625+0000 7f0e38c00640 1 --2- 172.21.15.122:0/3796079698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3415c1c0 0x7f0e3406d000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.625+0000 7f0e38c00640 1 --2- 172.21.15.122:0/3796079698 >> v2:172.21.15.173:3300/0 conn(0x7f0e3406cc20 0x7f0e340737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.625+0000 7f0e38c00640 1 -- 172.21.15.122:0/3796079698 >> 172.21.15.122:0/3796079698 conn(0x7f0e340d0c90 msgr2=0x7f0e340d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:16.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.625+0000 7f0e38c00640 1 -- 172.21.15.122:0/3796079698 shutdown_connections 2024-09-17T05:40:16.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.625+0000 7f0e38c00640 1 -- 172.21.15.122:0/3796079698 wait complete. 2024-09-17T05:40:16.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 Processor -- start 2024-09-17T05:40:16.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 -- start start 2024-09-17T05:40:16.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e3406cc20 0x7f0e34154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:16.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3415c1c0 0x7f0e340c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:16.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3415f7b0 0x7f0e340b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:16.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34169360 con 0x7f0e3406cc20 2024-09-17T05:40:16.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e340721b0 con 0x7f0e3415c1c0 2024-09-17T05:40:16.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0e341694e0 con 0x7f0e3415f7b0 2024-09-17T05:40:16.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e3406cc20 0x7f0e34154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:16.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e3406cc20 0x7f0e34154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43572/0 (socket says 172.21.15.122:43572) 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 -- 172.21.15.122:0/2187208798 learned_addr learned my addr 172.21.15.122:0/2187208798 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31400640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3415c1c0 0x7f0e340c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e32800640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3415f7b0 0x7f0e340b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 -- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3415f7b0 msgr2=0x7f0e340b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3415f7b0 0x7f0e340b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 -- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3415c1c0 msgr2=0x7f0e340c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3415c1c0 0x7f0e340c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 -- 172.21.15.122:0/2187208798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e340b1970 con 0x7f0e3406cc20 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e31e00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e3406cc20 0x7f0e34154c70 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400ea40 tx=0x7f0e2400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e2a000640 1 -- 172.21.15.122:0/2187208798 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400ce40 con 0x7f0e3406cc20 2024-09-17T05:40:16.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e2a000640 1 -- 172.21.15.122:0/2187208798 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e24004020 con 0x7f0e3406cc20 2024-09-17T05:40:16.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e2a000640 1 -- 172.21.15.122:0/2187208798 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24004470 con 0x7f0e3406cc20 2024-09-17T05:40:16.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e340b2de0 con 0x7f0e3406cc20 2024-09-17T05:40:16.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.629+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e340b3360 con 0x7f0e3406cc20 2024-09-17T05:40:16.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.633+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e340737c0 con 0x7f0e3406cc20 2024-09-17T05:40:16.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.633+0000 7f0e2a000640 1 -- 172.21.15.122:0/2187208798 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f0e24003600 con 0x7f0e3406cc20 2024-09-17T05:40:16.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.633+0000 7f0e2a000640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e14075510 0x7f0e140779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:16.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.633+0000 7f0e2a000640 1 -- 172.21.15.122:0/2187208798 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7f0e24093cd0 con 0x7f0e3406cc20 2024-09-17T05:40:16.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.633+0000 7f0e31400640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e14075510 0x7f0e140779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:16.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.633+0000 7f0e31400640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e14075510 0x7f0e140779d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0e200af000 tx=0x7f0e2001af50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:16.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.637+0000 7f0e2a000640 1 -- 172.21.15.122:0/2187208798 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0e24060190 con 0x7f0e3406cc20 2024-09-17T05:40:16.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:16.861+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7f0e340c6840 con 0x7f0e3406cc20 2024-09-17T05:40:17.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e2a000640 1 -- 172.21.15.122:0/2187208798 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v692) ==== 186+0+0 (secure 0 0 0) 0x7f0e24064bc0 con 0x7f0e3406cc20 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e14075510 msgr2=0x7f0e140779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e14075510 0x7f0e140779d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0e200af000 tx=0x7f0e2001af50 comp rx=0 tx=0).stop 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e3406cc20 msgr2=0x7f0e34154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e3406cc20 0x7f0e34154c70 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400ea40 tx=0x7f0e2400ef00 comp rx=0 tx=0).stop 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 shutdown_connections 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0e14075510 0x7f0e140779d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3415f7b0 0x7f0e340b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3415c1c0 0x7f0e340c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 --2- 172.21.15.122:0/2187208798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e3406cc20 0x7f0e34154c70 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 >> 172.21.15.122:0/2187208798 conn(0x7f0e340d0c90 msgr2=0x7f0e3415e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:17.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.101+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 shutdown_connections 2024-09-17T05:40:17.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.105+0000 7f0e38c00640 1 -- 172.21.15.122:0/2187208798 wait complete. 2024-09-17T05:40:17.121 INFO:tasks.cephfs_test_runner:Test that the progress bar that is printed for 1 ongoing clone job is ... ok 2024-09-17T05:40:17.121 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-09-17T05:40:17.121 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-17T05:40:17.122 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:40:17.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- 172.21.15.122:0/92133271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 msgr2=0x7f5fac06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/92133271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 0x7f5fac06e7a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0b2c50 tx=0x7f5f9c01c890 comp rx=0 tx=0).stop 2024-09-17T05:40:17.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- 172.21.15.122:0/92133271 shutdown_connections 2024-09-17T05:40:17.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/92133271 >> v2:172.21.15.173:3300/0 conn(0x7f5fac0cf1d0 0x7f5fac15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/92133271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fac1602c0 0x7f5fac0ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/92133271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 0x7f5fac06e7a0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- 172.21.15.122:0/92133271 >> 172.21.15.122:0/92133271 conn(0x7f5fac0b18f0 msgr2=0x7f5fac0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:17.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- 172.21.15.122:0/92133271 shutdown_connections 2024-09-17T05:40:17.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- 172.21.15.122:0/92133271 wait complete. 2024-09-17T05:40:17.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 Processor -- start 2024-09-17T05:40:17.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- start start 2024-09-17T05:40:17.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac0cf1d0 0x7f5fac0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 0x7f5fac09bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fac1602c0 0x7f5fac09c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac15c790 con 0x7f5fac15f9a0 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac15c610 con 0x7f5fac0cf1d0 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fb1e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac15c910 con 0x7f5fac1602c0 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 0x7f5fac09bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 0x7f5fac09bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43594/0 (socket says 172.21.15.122:43594) 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 -- 172.21.15.122:0/827222251 learned_addr learned my addr 172.21.15.122:0/827222251 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fab400640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fac1602c0 0x7f5fac09c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5faaa00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac0cf1d0 0x7f5fac0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 -- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fac1602c0 msgr2=0x7f5fac09c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fac1602c0 0x7f5fac09c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 -- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac0cf1d0 msgr2=0x7f5fac0ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac0cf1d0 0x7f5fac0ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 -- 172.21.15.122:0/827222251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac145fd0 con 0x7f5fac15f9a0 2024-09-17T05:40:17.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.269+0000 7f5fa2000640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 0x7f5fac09bc50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400d7a0 tx=0x7f5f9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:17.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fa3e00640 1 -- 172.21.15.122:0/827222251 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94017070 con 0x7f5fac15f9a0 2024-09-17T05:40:17.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fa3e00640 1 -- 172.21.15.122:0/827222251 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f94009d20 con 0x7f5fac15f9a0 2024-09-17T05:40:17.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fa3e00640 1 -- 172.21.15.122:0/827222251 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f940106c0 con 0x7f5fac15f9a0 2024-09-17T05:40:17.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac15db10 con 0x7f5fac15f9a0 2024-09-17T05:40:17.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac0ad6e0 con 0x7f5fac15f9a0 2024-09-17T05:40:17.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac142eb0 con 0x7f5fac15f9a0 2024-09-17T05:40:17.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fa3e00640 1 -- 172.21.15.122:0/827222251 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f5f9401a070 con 0x7f5fac15f9a0 2024-09-17T05:40:17.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fa3e00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f880754e0 0x7f5f880779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5fa3e00640 1 -- 172.21.15.122:0/827222251 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7f5f94093540 con 0x7f5fac15f9a0 2024-09-17T05:40:17.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.273+0000 7f5faaa00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f880754e0 0x7f5f880779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.277+0000 7f5faaa00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f880754e0 0x7f5f880779a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0af000 tx=0x7f5f9c019630 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:17.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.277+0000 7f5fa3e00640 1 -- 172.21.15.122:0/827222251 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5fac142eb0 con 0x7f5fac15f9a0 2024-09-17T05:40:17.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.445+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fac06add0 con 0x7f5fac15f9a0 2024-09-17T05:40:17.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.445+0000 7f5fa3e00640 1 -- 172.21.15.122:0/827222251 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+13236 (secure 0 0 0) 0x7f5f94060050 con 0x7f5fac15f9a0 2024-09-17T05:40:17.443 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:17.443 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":137,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:40:16.096075+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6835/2463174917":"2024-09-18T05:40:13.070751+0000","172.21.15.122:6832/2463174917":"2024-09-18T05:40:13.070751+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:40:17.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f880754e0 msgr2=0x7f5f880779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f880754e0 0x7f5f880779a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0af000 tx=0x7f5f9c019630 comp rx=0 tx=0).stop 2024-09-17T05:40:17.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 msgr2=0x7f5fac09bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 0x7f5fac09bc50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400d7a0 tx=0x7f5f9400dc60 comp rx=0 tx=0).stop 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 shutdown_connections 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f880754e0 0x7f5f880779a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fac1602c0 0x7f5fac09c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fac15f9a0 0x7f5fac09bc50 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 --2- 172.21.15.122:0/827222251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fac0cf1d0 0x7f5fac0ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 >> 172.21.15.122:0/827222251 conn(0x7f5fac0b18f0 msgr2=0x7f5fac159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 shutdown_connections 2024-09-17T05:40:17.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.449+0000 7f5fb1e00640 1 -- 172.21.15.122:0/827222251 wait complete. 2024-09-17T05:40:17.465 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:24:54.466798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:40:17.466 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:40:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.613+0000 7fcf9b400640 1 -- 172.21.15.122:0/3757272117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf780039f0 msgr2=0x7fcf78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.613+0000 7fcf9b400640 1 --2- 172.21.15.122:0/3757272117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf780039f0 0x7fcf78003dd0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009b30 tx=0x7fcf8401d210 comp rx=0 tx=0).stop 2024-09-17T05:40:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.613+0000 7fcf99400640 1 -- 172.21.15.122:0/3757272117 reap_dead start 2024-09-17T05:40:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.613+0000 7fcf9b400640 1 -- 172.21.15.122:0/3757272117 shutdown_connections 2024-09-17T05:40:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.613+0000 7fcf9b400640 1 -- 172.21.15.122:0/3757272117 >> 172.21.15.122:0/3757272117 conn(0x7fcf940d1dd0 msgr2=0x7fcf940d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.613+0000 7fcf9b400640 1 -- 172.21.15.122:0/3757272117 shutdown_connections 2024-09-17T05:40:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.613+0000 7fcf9b400640 1 -- 172.21.15.122:0/3757272117 wait complete. 2024-09-17T05:40:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 Processor -- start 2024-09-17T05:40:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 -- start start 2024-09-17T05:40:17.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf780039f0 0x7fcf940c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf940ca850 0x7fcf940c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf940c9ab0 0x7fcf94158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf940b1220 con 0x7fcf940ca850 2024-09-17T05:40:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf940b0ee0 con 0x7fcf780039f0 2024-09-17T05:40:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcf940b13a0 con 0x7fcf940c9ab0 2024-09-17T05:40:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf940ca850 0x7fcf940c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf940ca850 0x7fcf940c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43616/0 (socket says 172.21.15.122:43616) 2024-09-17T05:40:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 -- 172.21.15.122:0/3407317388 learned_addr learned my addr 172.21.15.122:0/3407317388 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf98a00640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf780039f0 0x7fcf940c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf99400640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf940c9ab0 0x7fcf94158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 -- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf940c9ab0 msgr2=0x7fcf94158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf940c9ab0 0x7fcf94158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 -- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf780039f0 msgr2=0x7fcf940c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf780039f0 0x7fcf940c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 -- 172.21.15.122:0/3407317388 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf940cad10 con 0x7fcf940ca850 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf99400640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf940c9ab0 0x7fcf94158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf8be00640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf940ca850 0x7fcf940c8200 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c004e00 tx=0x7fcf7c001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf91600640 1 -- 172.21.15.122:0/3407317388 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0107d0 con 0x7fcf940ca850 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf91600640 1 -- 172.21.15.122:0/3407317388 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf7c016070 con 0x7fcf940ca850 2024-09-17T05:40:17.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf91600640 1 -- 172.21.15.122:0/3407317388 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c010d40 con 0x7fcf940ca850 2024-09-17T05:40:17.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf940caff0 con 0x7fcf940ca850 2024-09-17T05:40:17.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.617+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf940cb5b0 con 0x7fcf940ca850 2024-09-17T05:40:17.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.621+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf940d0b90 con 0x7fcf940ca850 2024-09-17T05:40:17.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.621+0000 7fcf91600640 1 -- 172.21.15.122:0/3407317388 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fcf7c01b460 con 0x7fcf940ca850 2024-09-17T05:40:17.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.621+0000 7fcf91600640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf700754e0 0x7fcf700779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.621+0000 7fcf91600640 1 -- 172.21.15.122:0/3407317388 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7fcf7c093c90 con 0x7fcf940ca850 2024-09-17T05:40:17.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.621+0000 7fcf98a00640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf700754e0 0x7fcf700779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.621+0000 7fcf98a00640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf700754e0 0x7fcf700779a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009b30 tx=0x7fcf84019fb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:17.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.625+0000 7fcf91600640 1 -- 172.21.15.122:0/3407317388 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcf7c060840 con 0x7fcf940ca850 2024-09-17T05:40:17.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.789+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf940cc1a0 con 0x7fcf940ca850 2024-09-17T05:40:17.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.789+0000 7fcf91600640 1 -- 172.21.15.122:0/3407317388 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7fcf7c060140 con 0x7fcf940ca850 2024-09-17T05:40:17.788 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-09-17T05:40:17.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf700754e0 msgr2=0x7fcf700779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf700754e0 0x7fcf700779a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009b30 tx=0x7fcf84019fb0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf940ca850 msgr2=0x7fcf940c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf940ca850 0x7fcf940c8200 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c004e00 tx=0x7fcf7c001720 comp rx=0 tx=0).stop 2024-09-17T05:40:17.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 shutdown_connections 2024-09-17T05:40:17.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcf700754e0 0x7fcf700779a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf940c9ab0 0x7fcf94158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf940ca850 0x7fcf940c8200 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 --2- 172.21.15.122:0/3407317388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf780039f0 0x7fcf940c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 >> 172.21.15.122:0/3407317388 conn(0x7fcf940d1dd0 msgr2=0x7fcf940cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:17.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 shutdown_connections 2024-09-17T05:40:17.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.793+0000 7fcf9b400640 1 -- 172.21.15.122:0/3407317388 wait complete. 2024-09-17T05:40:17.809 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-17T05:40:17.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.953+0000 7f1633800640 1 -- 172.21.15.122:0/4145071834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 msgr2=0x7f162c0aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.953+0000 7f1633800640 1 --2- 172.21.15.122:0/4145071834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 0x7f162c0aeef0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1624009bc0 tx=0x7f162401cba0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.953+0000 7f1633800640 1 -- 172.21.15.122:0/4145071834 shutdown_connections 2024-09-17T05:40:17.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.953+0000 7f1633800640 1 --2- 172.21.15.122:0/4145071834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 0x7f162c0aeef0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.953+0000 7f1633800640 1 --2- 172.21.15.122:0/4145071834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c0ced80 0x7f162c0ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.953+0000 7f1633800640 1 -- 172.21.15.122:0/4145071834 >> 172.21.15.122:0/4145071834 conn(0x7f162c0d1dc0 msgr2=0x7f162c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:17.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.953+0000 7f1633800640 1 -- 172.21.15.122:0/4145071834 shutdown_connections 2024-09-17T05:40:17.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.953+0000 7f1633800640 1 -- 172.21.15.122:0/4145071834 wait complete. 2024-09-17T05:40:17.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 Processor -- start 2024-09-17T05:40:17.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 -- start start 2024-09-17T05:40:17.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 0x7f162c148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c0ced80 0x7f162c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c15bea0 0x7f162c071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f162c169290 con 0x7f162c15bea0 2024-09-17T05:40:17.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f162c169110 con 0x7f162c0cdb30 2024-09-17T05:40:17.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f162c169410 con 0x7f162c0ced80 2024-09-17T05:40:17.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c15bea0 0x7f162c071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c15bea0 0x7f162c071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43622/0 (socket says 172.21.15.122:43622) 2024-09-17T05:40:17.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 -- 172.21.15.122:0/729003578 learned_addr learned my addr 172.21.15.122:0/729003578 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f162be00640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c0ced80 0x7f162c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1630e00640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 0x7f162c148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 -- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c0ced80 msgr2=0x7f162c0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c0ced80 0x7f162c0cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 -- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 msgr2=0x7f162c148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 0x7f162c148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 -- 172.21.15.122:0/729003578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1fb880 con 0x7f162c15bea0 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1630e00640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 0x7f162c148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1631800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c15bea0 0x7f162c071b00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f16240b0040 tx=0x7f1624005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:17.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1628c00640 1 -- 172.21.15.122:0/729003578 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240b2040 con 0x7f162c15bea0 2024-09-17T05:40:17.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1628c00640 1 -- 172.21.15.122:0/729003578 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1624004040 con 0x7f162c15bea0 2024-09-17T05:40:17.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1fbb60 con 0x7f162c15bea0 2024-09-17T05:40:17.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1628c00640 1 -- 172.21.15.122:0/729003578 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624004470 con 0x7f162c15bea0 2024-09-17T05:40:17.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.957+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1fc120 con 0x7f162c15bea0 2024-09-17T05:40:17.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.961+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c0ce670 con 0x7f162c15bea0 2024-09-17T05:40:17.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.961+0000 7f1628c00640 1 -- 172.21.15.122:0/729003578 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f16240bd030 con 0x7f162c15bea0 2024-09-17T05:40:17.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.961+0000 7f1628c00640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0754e0 0x7f160c0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:17.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.961+0000 7f1628c00640 1 -- 172.21.15.122:0/729003578 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7f162412d1d0 con 0x7f162c15bea0 2024-09-17T05:40:17.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.961+0000 7f1630e00640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0754e0 0x7f160c0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:17.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.961+0000 7f1630e00640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0754e0 0x7f160c0779a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f161c001230 tx=0x7f161c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:17.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:17.965+0000 7f1628c00640 1 -- 172.21.15.122:0/729003578 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1624016790 con 0x7f162c15bea0 2024-09-17T05:40:18.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:18.193+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f162c0b1030 con 0x7f162c15bea0 2024-09-17T05:40:19.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.109+0000 7f1628c00640 1 -- 172.21.15.122:0/729003578 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v694) ==== 189+0+0 (secure 0 0 0) 0x7f16240f9c60 con 0x7f162c15bea0 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0754e0 msgr2=0x7f160c0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0754e0 0x7f160c0779a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f161c001230 tx=0x7f161c009040 comp rx=0 tx=0).stop 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c15bea0 msgr2=0x7f162c071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c15bea0 0x7f162c071b00 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f16240b0040 tx=0x7f1624005a80 comp rx=0 tx=0).stop 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 shutdown_connections 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f160c0754e0 0x7f160c0779a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f162c15bea0 0x7f162c071b00 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162c0ced80 0x7f162c0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 --2- 172.21.15.122:0/729003578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f162c0cdb30 0x7f162c148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 >> 172.21.15.122:0/729003578 conn(0x7f162c0d1dc0 msgr2=0x7f162c15b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 shutdown_connections 2024-09-17T05:40:19.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.113+0000 7f1633800640 1 -- 172.21.15.122:0/729003578 wait complete. 2024-09-17T05:40:19.132 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:40:19.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 -- 172.21.15.122:0/3858553897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 msgr2=0x7f87180ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 --2- 172.21.15.122:0/3858553897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 0x7f87180ce670 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f87180cf310 tx=0x7f870c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 -- 172.21.15.122:0/3858553897 shutdown_connections 2024-09-17T05:40:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 --2- 172.21.15.122:0/3858553897 >> v2:172.21.15.173:3300/0 conn(0x7f87180d28e0 0x7f87180cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 --2- 172.21.15.122:0/3858553897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87180cdb30 0x7f87180cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 --2- 172.21.15.122:0/3858553897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 0x7f87180ce670 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 -- 172.21.15.122:0/3858553897 >> 172.21.15.122:0/3858553897 conn(0x7f87180d1dc0 msgr2=0x7f87180d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 -- 172.21.15.122:0/3858553897 shutdown_connections 2024-09-17T05:40:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 -- 172.21.15.122:0/3858553897 wait complete. 2024-09-17T05:40:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.281+0000 7f871f800640 1 Processor -- start 2024-09-17T05:40:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 -- start start 2024-09-17T05:40:19.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87180cdb30 0x7f87181f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:19.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87180d28e0 0x7f8718152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:19.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 0x7f8718157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87180af070 con 0x7f871815bea0 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f87180aeef0 con 0x7f87180d28e0 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f87180af1f0 con 0x7f87180cdb30 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 0x7f8718157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 0x7f8718157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43650/0 (socket says 172.21.15.122:43650) 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 -- 172.21.15.122:0/3570705748 learned_addr learned my addr 172.21.15.122:0/3570705748 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871ce00640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87180cdb30 0x7f87181f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f8717e00640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87180d28e0 0x7f8718152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 -- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87180cdb30 msgr2=0x7f87181f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87180cdb30 0x7f87181f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 -- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87180d28e0 msgr2=0x7f8718152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87180d28e0 0x7f8718152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 -- 172.21.15.122:0/3570705748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87181f6910 con 0x7f871815bea0 2024-09-17T05:40:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871d800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 0x7f8718157360 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f871000ea20 tx=0x7f871000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f8714c00640 1 -- 172.21.15.122:0/3570705748 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871000cde0 con 0x7f871815bea0 2024-09-17T05:40:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f8714c00640 1 -- 172.21.15.122:0/3570705748 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8710004c40 con 0x7f871815bea0 2024-09-17T05:40:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f8714c00640 1 -- 172.21.15.122:0/3570705748 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710005280 con 0x7f871815bea0 2024-09-17T05:40:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87181f94a0 con 0x7f871815bea0 2024-09-17T05:40:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87181f9a60 con 0x7f871815bea0 2024-09-17T05:40:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.285+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87180cc9d0 con 0x7f871815bea0 2024-09-17T05:40:19.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.289+0000 7f8714c00640 1 -- 172.21.15.122:0/3570705748 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f8710005420 con 0x7f871815bea0 2024-09-17T05:40:19.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.289+0000 7f8714c00640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86f80754e0 0x7f86f80779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:19.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.289+0000 7f8714c00640 1 -- 172.21.15.122:0/3570705748 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7f8710093c30 con 0x7f871815bea0 2024-09-17T05:40:19.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.289+0000 7f871ce00640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86f80754e0 0x7f86f80779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:19.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.289+0000 7f871ce00640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86f80754e0 0x7f86f80779a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f870c0af000 tx=0x7f870c0a7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:19.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.293+0000 7f8714c00640 1 -- 172.21.15.122:0/3570705748 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8710016030 con 0x7f871815bea0 2024-09-17T05:40:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.461+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f87181f9e00 con 0x7f871815bea0 2024-09-17T05:40:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.461+0000 7f8714c00640 1 -- 172.21.15.122:0/3570705748 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v53) ==== 59+0+1998 (secure 0 0 0) 0x7f8710060190 con 0x7f871815bea0 2024-09-17T05:40:19.459 INFO:teuthology.orchestra.run.smithi122.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:40:19.460 INFO:teuthology.orchestra.run.smithi122.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:40:19.460 INFO:teuthology.orchestra.run.smithi122.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-17T05:40:19.460 INFO:teuthology.orchestra.run.smithi122.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:40:19.460 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3966.725027 2024-09-17T05:40:19.461 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3991.290249 2024-09-17T05:40:19.461 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4048.491077 2024-09-17T05:40:19.461 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3894.303595 2024-09-17T05:40:19.461 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4659.648487 2024-09-17T05:40:19.461 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4795.974604 2024-09-17T05:40:19.461 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4821.292540 2024-09-17T05:40:19.462 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4733.982710 2024-09-17T05:40:19.462 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds 20/20 2024-09-17T05:40:19.462 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:40:19.462 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced debug_ms 2/2 2024-09-17T05:40:19.462 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_frag true 2024-09-17T05:40:19.462 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:40:19.463 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:40:19.463 INFO:teuthology.orchestra.run.smithi122.stdout:mds dev mds_verify_scatter true 2024-09-17T05:40:19.463 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:40:19.463 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:40:19.463 INFO:teuthology.orchestra.run.smithi122.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:40:19.464 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced client_mount_timeout 600 2024-09-17T05:40:19.464 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_client 20/20 2024-09-17T05:40:19.464 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced debug_ms 1/1 2024-09-17T05:40:19.464 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:40:19.464 INFO:teuthology.orchestra.run.smithi122.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:40:19.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86f80754e0 msgr2=0x7f86f80779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86f80754e0 0x7f86f80779a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f870c0af000 tx=0x7f870c0a7040 comp rx=0 tx=0).stop 2024-09-17T05:40:19.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 msgr2=0x7f8718157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 0x7f8718157360 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f871000ea20 tx=0x7f871000eee0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 shutdown_connections 2024-09-17T05:40:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f86f80754e0 0x7f86f80779a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f871815bea0 0x7f8718157360 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87180d28e0 0x7f8718152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 --2- 172.21.15.122:0/3570705748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87180cdb30 0x7f87181f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 >> 172.21.15.122:0/3570705748 conn(0x7f87180d1dc0 msgr2=0x7f87180af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 shutdown_connections 2024-09-17T05:40:19.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.465+0000 7f871f800640 1 -- 172.21.15.122:0/3570705748 wait complete. 2024-09-17T05:40:19.481 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:40:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.629+0000 7f088aa00640 1 -- 172.21.15.122:0/174416544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08840b4bd0 msgr2=0x7f08840b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.629+0000 7f088aa00640 1 --2- 172.21.15.122:0/174416544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08840b4bd0 0x7f08840b2bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f086c009900 tx=0x7f086c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.629+0000 7f088aa00640 1 -- 172.21.15.122:0/174416544 shutdown_connections 2024-09-17T05:40:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.629+0000 7f088aa00640 1 --2- 172.21.15.122:0/174416544 >> v2:172.21.15.173:3300/0 conn(0x7f08840b3130 0x7f08840cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.629+0000 7f088aa00640 1 --2- 172.21.15.122:0/174416544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08840b4bd0 0x7f08840b2bf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.629+0000 7f088aa00640 1 -- 172.21.15.122:0/174416544 >> 172.21.15.122:0/174416544 conn(0x7f08840b18f0 msgr2=0x7f08840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.629+0000 7f088aa00640 1 -- 172.21.15.122:0/174416544 shutdown_connections 2024-09-17T05:40:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.629+0000 7f088aa00640 1 -- 172.21.15.122:0/174416544 wait complete. 2024-09-17T05:40:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 Processor -- start 2024-09-17T05:40:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 -- start start 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08840b3130 0x7f088409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08840b4bd0 0x7f088409d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088415b580 0x7f08841f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0884167ae0 con 0x7f08840b3130 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0884167960 con 0x7f088415b580 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0884167c60 con 0x7f08840b4bd0 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0883e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08840b3130 0x7f088409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0888a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088415b580 0x7f08841f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f087be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08840b4bd0 0x7f088409d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:19.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0888a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088415b580 0x7f08841f8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:39866/0 (socket says 172.21.15.122:39866) 2024-09-17T05:40:19.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0888a00640 1 -- 172.21.15.122:0/2988507529 learned_addr learned my addr 172.21.15.122:0/2988507529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:19.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0883e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08840b3130 0x7f088409aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43652/0 (socket says 172.21.15.122:43652) 2024-09-17T05:40:19.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0883e00640 1 -- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08840b4bd0 msgr2=0x7f088409d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0883e00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08840b4bd0 0x7f088409d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0883e00640 1 -- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088415b580 msgr2=0x7f08841f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0883e00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088415b580 0x7f08841f8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0883e00640 1 -- 172.21.15.122:0/2988507529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08841f85c0 con 0x7f08840b3130 2024-09-17T05:40:19.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0883e00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08840b3130 0x7f088409aa10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f087400c3c0 tx=0x7f087400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:19.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0880c00640 1 -- 172.21.15.122:0/2988507529 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874017070 con 0x7f08840b3130 2024-09-17T05:40:19.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0880c00640 1 -- 172.21.15.122:0/2988507529 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0874003600 con 0x7f08840b3130 2024-09-17T05:40:19.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f0880c00640 1 -- 172.21.15.122:0/2988507529 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874003a90 con 0x7f08840b3130 2024-09-17T05:40:19.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088409b160 con 0x7f08840b3130 2024-09-17T05:40:19.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088409b720 con 0x7f08840b3130 2024-09-17T05:40:19.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.633+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0884069410 con 0x7f08840b3130 2024-09-17T05:40:19.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.637+0000 7f0880c00640 1 -- 172.21.15.122:0/2988507529 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f087401a070 con 0x7f08840b3130 2024-09-17T05:40:19.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.637+0000 7f0880c00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0860075510 0x7f08600779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:19.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.637+0000 7f0880c00640 1 -- 172.21.15.122:0/2988507529 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7f0874092520 con 0x7f08840b3130 2024-09-17T05:40:19.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.637+0000 7f087be00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0860075510 0x7f08600779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.637+0000 7f087be00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0860075510 0x7f08600779d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f086c0af000 tx=0x7f086c01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:19.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.641+0000 7f0880c00640 1 -- 172.21.15.122:0/2988507529 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f087405e9e0 con 0x7f08840b3130 2024-09-17T05:40:19.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.809+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f08840cc550 con 0x7f08840b3130 2024-09-17T05:40:19.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.813+0000 7f0880c00640 1 -- 172.21.15.122:0/2988507529 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f0874063410 con 0x7f08840b3130 2024-09-17T05:40:19.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.813+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0860075510 msgr2=0x7f08600779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.813+0000 7f088aa00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0860075510 0x7f08600779d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f086c0af000 tx=0x7f086c01b120 comp rx=0 tx=0).stop 2024-09-17T05:40:19.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.813+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08840b3130 msgr2=0x7f088409aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.813+0000 7f088aa00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08840b3130 0x7f088409aa10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f087400c3c0 tx=0x7f087400c880 comp rx=0 tx=0).stop 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.817+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 shutdown_connections 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.817+0000 7f088aa00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0860075510 0x7f08600779d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.817+0000 7f088aa00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f088415b580 0x7f08841f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.817+0000 7f088aa00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08840b4bd0 0x7f088409d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.817+0000 7f088aa00640 1 --2- 172.21.15.122:0/2988507529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08840b3130 0x7f088409aa10 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.817+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 >> 172.21.15.122:0/2988507529 conn(0x7f08840b18f0 msgr2=0x7f08840afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.817+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 shutdown_connections 2024-09-17T05:40:19.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:19.817+0000 7f088aa00640 1 -- 172.21.15.122:0/2988507529 wait complete. 2024-09-17T05:40:19.831 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:40:19.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:40:19.922 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:40:20.024 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:40:20.024 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:20.024 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:20.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:40:20.033 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:40:20.033 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:20.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:40:20.043 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:20.043 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:20.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:40:20.083 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:40:20.083 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:20.093 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:40:20.093 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:40:20.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:40:20.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- 172.21.15.122:0/742767399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff998072d00 msgr2=0x7ff998073190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:20.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 --2- 172.21.15.122:0/742767399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff998072d00 0x7ff998073190 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff9980d0c90 tx=0x7ff99001cd80 comp rx=0 tx=0).stop 2024-09-17T05:40:20.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- 172.21.15.122:0/742767399 shutdown_connections 2024-09-17T05:40:20.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 --2- 172.21.15.122:0/742767399 >> v2:172.21.15.173:3300/0 conn(0x7ff9980b42d0 0x7ff9980b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 --2- 172.21.15.122:0/742767399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff998072d00 0x7ff998073190 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 --2- 172.21.15.122:0/742767399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff99815a660 0x7ff9980cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- 172.21.15.122:0/742767399 >> 172.21.15.122:0/742767399 conn(0x7ff9980b18f0 msgr2=0x7ff9980b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:20.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- 172.21.15.122:0/742767399 shutdown_connections 2024-09-17T05:40:20.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- 172.21.15.122:0/742767399 wait complete. 2024-09-17T05:40:20.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 Processor -- start 2024-09-17T05:40:20.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- start start 2024-09-17T05:40:20.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff998072d00 0x7ff9980c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:20.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9980b42d0 0x7ff998158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff99815a660 0x7ff9980acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff998074000 con 0x7ff99815a660 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff998073e80 con 0x7ff9980b42d0 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff99cc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff998167190 con 0x7ff998072d00 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff996800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff99815a660 0x7ff9980acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff996800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff99815a660 0x7ff9980acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43668/0 (socket says 172.21.15.122:43668) 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff996800640 1 -- 172.21.15.122:0/2703439884 learned_addr learned my addr 172.21.15.122:0/2703439884 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.269+0000 7ff995e00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff998072d00 0x7ff9980c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff995400640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9980b42d0 0x7ff998158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff996800640 1 -- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff998072d00 msgr2=0x7ff9980c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff996800640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff998072d00 0x7ff9980c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff996800640 1 -- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9980b42d0 msgr2=0x7ff998158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff996800640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9980b42d0 0x7ff998158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff996800640 1 -- 172.21.15.122:0/2703439884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9980c90f0 con 0x7ff99815a660 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff995400640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9980b42d0 0x7ff998158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff996800640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff99815a660 0x7ff9980acd40 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff99001b430 tx=0x7ff990003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff98e000640 1 -- 172.21.15.122:0/2703439884 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900bb070 con 0x7ff99815a660 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff98e000640 1 -- 172.21.15.122:0/2703439884 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff990004e00 con 0x7ff99815a660 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9981f8a50 con 0x7ff99815a660 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff98e000640 1 -- 172.21.15.122:0/2703439884 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900b2040 con 0x7ff99815a660 2024-09-17T05:40:20.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9980c9690 con 0x7ff99815a660 2024-09-17T05:40:20.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff998069410 con 0x7ff99815a660 2024-09-17T05:40:20.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.273+0000 7ff98e000640 1 -- 172.21.15.122:0/2703439884 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7ff9900be070 con 0x7ff99815a660 2024-09-17T05:40:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.277+0000 7ff98e000640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff9780754e0 0x7ff9780779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.277+0000 7ff98e000640 1 -- 172.21.15.122:0/2703439884 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7ff99012d050 con 0x7ff99815a660 2024-09-17T05:40:20.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.277+0000 7ff995e00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff9780754e0 0x7ff9780779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:20.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.277+0000 7ff995e00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff9780754e0 0x7ff9780779a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff9880058d0 tx=0x7ff988005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:20.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.281+0000 7ff98e000640 1 -- 172.21.15.122:0/2703439884 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff990016970 con 0x7ff99815a660 2024-09-17T05:40:20.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9980b4710 con 0x7ff99815a660 2024-09-17T05:40:20.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff98e000640 1 -- 172.21.15.122:0/2703439884 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+3920 (secure 0 0 0) 0x7ff9900f9b60 con 0x7ff99815a660 2024-09-17T05:40:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:20.478 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":89,"btime":"2024-09-17T05:40:14:072611+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7337,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/609693749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":609693749},{"type":"v1","addr":"172.21.15.122:6834","nonce":609693749}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2826125653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2826125653},{"type":"v1","addr":"172.21.15.173:6836","nonce":2826125653}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7346,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/4138151052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":4138151052},{"type":"v1","addr":"172.21.15.173:6837","nonce":4138151052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3596715079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3596715079},{"type":"v1","addr":"172.21.15.122:6835","nonce":3596715079}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[]} 2024-09-17T05:40:20.478 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 89 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff9780754e0 msgr2=0x7ff9780779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff9780754e0 0x7ff9780779a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff9880058d0 tx=0x7ff988005860 comp rx=0 tx=0).stop 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff99815a660 msgr2=0x7ff9980acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff99815a660 0x7ff9980acd40 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff99001b430 tx=0x7ff990003f60 comp rx=0 tx=0).stop 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 shutdown_connections 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff9780754e0 0x7ff9780779a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff99815a660 0x7ff9980acd40 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9980b42d0 0x7ff998158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 --2- 172.21.15.122:0/2703439884 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff998072d00 0x7ff9980c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 >> 172.21.15.122:0/2703439884 conn(0x7ff9980b18f0 msgr2=0x7ff9980ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:20.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 shutdown_connections 2024-09-17T05:40:20.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.481+0000 7ff99cc00640 1 -- 172.21.15.122:0/2703439884 wait complete. 2024-09-17T05:40:20.499 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:40:20.499 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:40:20.499 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:40:20.499 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:40:20.499 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:40:20.499 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:40:20.500 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:40:20.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:40:20.500 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:40:20.500 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:40:20.500 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:40:20.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:40:20.500 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:40:20.501 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:40:20.501 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:40:20.501 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:40:20.515 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:40:20.515 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:40:20.516 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:40:20.516 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:40:20.517 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:40:20.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:40:20.518 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:40:20.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:40:20.562 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:40:20.563 DEBUG:teuthology.parallel:result is None 2024-09-17T05:40:20.564 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:40:20.564 DEBUG:teuthology.parallel:result is None 2024-09-17T05:40:20.564 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:40:20.565 DEBUG:teuthology.parallel:result is None 2024-09-17T05:40:20.565 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:40:20.565 DEBUG:teuthology.parallel:result is None 2024-09-17T05:40:20.566 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:40:20.647 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-09-17T05:40:20.647 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:40:20.659 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2024-09-17T05:40:20.659 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-09-17T05:40:20.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.721+0000 7f631d600640 1 -- 172.21.15.122:0/3603777818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6318073d10 msgr2=0x7f63180cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:20.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.721+0000 7f631d600640 1 --2- 172.21.15.122:0/3603777818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6318073d10 0x7f63180cd100 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f631806fcf0 tx=0x7f630001c820 comp rx=0 tx=0).stop 2024-09-17T05:40:20.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.721+0000 7f631d600640 1 -- 172.21.15.122:0/3603777818 shutdown_connections 2024-09-17T05:40:20.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.721+0000 7f631d600640 1 --2- 172.21.15.122:0/3603777818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63180b49e0 0x7f631806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.721+0000 7f631d600640 1 --2- 172.21.15.122:0/3603777818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6318073d10 0x7f63180cd100 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.721+0000 7f631d600640 1 -- 172.21.15.122:0/3603777818 >> 172.21.15.122:0/3603777818 conn(0x7f63180b18f0 msgr2=0x7f63180b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:20.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.721+0000 7f631d600640 1 -- 172.21.15.122:0/3603777818 shutdown_connections 2024-09-17T05:40:20.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.721+0000 7f631d600640 1 -- 172.21.15.122:0/3603777818 wait complete. 2024-09-17T05:40:20.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-17T05:40:20.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-17T05:40:20.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-17T05:40:20.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-17T05:40:20.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 Processor -- start 2024-09-17T05:40:20.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 -- start start 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6318073d10 0x7f63181f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63180b49e0 0x7f63181496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f631815a660 0x7f631809cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6318167310 con 0x7f6318073d10 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6318167190 con 0x7f631815a660 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6318167490 con 0x7f63180b49e0 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6318073d10 0x7f63181f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6318073d10 0x7f63181f7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43696/0 (socket says 172.21.15.122:43696) 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 -- 172.21.15.122:0/2099262496 learned_addr learned my addr 172.21.15.122:0/2099262496 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f630e000640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63180b49e0 0x7f63181496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:20.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6317400640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f631815a660 0x7f631809cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:20.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 -- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63180b49e0 msgr2=0x7f63181496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:20.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63180b49e0 0x7f63181496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 -- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f631815a660 msgr2=0x7f631809cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:20.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f631815a660 0x7f631809cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:20.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 -- 172.21.15.122:0/2099262496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63181637b0 con 0x7f6318073d10 2024-09-17T05:40:20.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f6316a00640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6318073d10 0x7f63181f7b40 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f630800c410 tx=0x7f630800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:20.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f630f400640 1 -- 172.21.15.122:0/2099262496 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308011790 con 0x7f6318073d10 2024-09-17T05:40:20.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f630f400640 1 -- 172.21.15.122:0/2099262496 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6308017070 con 0x7f6318073d10 2024-09-17T05:40:20.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f630f400640 1 -- 172.21.15.122:0/2099262496 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6308011ce0 con 0x7f6318073d10 2024-09-17T05:40:20.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631809bcb0 con 0x7f6318073d10 2024-09-17T05:40:20.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.725+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631809c270 con 0x7f6318073d10 2024-09-17T05:40:20.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.729+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6318069410 con 0x7f6318073d10 2024-09-17T05:40:20.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.729+0000 7f630f400640 1 -- 172.21.15.122:0/2099262496 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f630801c460 con 0x7f6318073d10 2024-09-17T05:40:20.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.729+0000 7f630f400640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f62f40754e0 0x7f62f40779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:20.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.729+0000 7f630f400640 1 -- 172.21.15.122:0/2099262496 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5027+0+0 (secure 0 0 0) 0x7f6308092ff0 con 0x7f6318073d10 2024-09-17T05:40:20.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.729+0000 7f630e000640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f62f40754e0 0x7f62f40779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:20.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.729+0000 7f630e000640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f62f40754e0 0x7f62f40779a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f63000ae000 tx=0x7f630001b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:20.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.733+0000 7f630f400640 1 -- 172.21.15.122:0/2099262496 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f630805f4b0 con 0x7f6318073d10 2024-09-17T05:40:20.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:20.913+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f63180cd100 con 0x7f6318073d10 2024-09-17T05:40:21.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f630f400640 1 -- 172.21.15.122:0/2099262496 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) ==== 97+0+0 (secure 0 0 0) 0x7f6308063ee0 con 0x7f6318073d10 2024-09-17T05:40:21.130 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f62f40754e0 msgr2=0x7f62f40779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f62f40754e0 0x7f62f40779a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f63000ae000 tx=0x7f630001b060 comp rx=0 tx=0).stop 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6318073d10 msgr2=0x7f63181f7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6318073d10 0x7f63181f7b40 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f630800c410 tx=0x7f630800c8d0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 shutdown_connections 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f62f40754e0 0x7f62f40779a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f631815a660 0x7f631809cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63180b49e0 0x7f63181496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 --2- 172.21.15.122:0/2099262496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6318073d10 0x7f63181f7b40 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.133+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 >> 172.21.15.122:0/2099262496 conn(0x7f63180b18f0 msgr2=0x7f6318155160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.137+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 shutdown_connections 2024-09-17T05:40:21.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.137+0000 7f631d600640 1 -- 172.21.15.122:0/2099262496 wait complete. 2024-09-17T05:40:21.154 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:40:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- 172.21.15.122:0/1049045199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85840b43d0 msgr2=0x7f85840b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 --2- 172.21.15.122:0/1049045199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85840b43d0 0x7f85840b47b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f857400a950 tx=0x7f857401db40 comp rx=0 tx=0).stop 2024-09-17T05:40:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- 172.21.15.122:0/1049045199 shutdown_connections 2024-09-17T05:40:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 --2- 172.21.15.122:0/1049045199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85840cd100 0x7f858406e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 --2- 172.21.15.122:0/1049045199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85840b43d0 0x7f85840b47b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- 172.21.15.122:0/1049045199 >> 172.21.15.122:0/1049045199 conn(0x7f85840b18f0 msgr2=0x7f85840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- 172.21.15.122:0/1049045199 shutdown_connections 2024-09-17T05:40:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- 172.21.15.122:0/1049045199 wait complete. 2024-09-17T05:40:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 Processor -- start 2024-09-17T05:40:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- start start 2024-09-17T05:40:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85840b2bf0 0x7f858415d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85840b43d0 0x7f858415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85840cd100 0x7f858415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8584167440 con 0x7f85840cd100 2024-09-17T05:40:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f85841672c0 con 0x7f85840b43d0 2024-09-17T05:40:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f858b800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f85841675c0 con 0x7f85840b2bf0 2024-09-17T05:40:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f8589800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85840cd100 0x7f858415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:21.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f8589800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85840cd100 0x7f858415c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39848/0 (socket says 172.21.15.122:39848) 2024-09-17T05:40:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.301+0000 7f8589800640 1 -- 172.21.15.122:0/115783346 learned_addr learned my addr 172.21.15.122:0/115783346 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f857be00640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85840b43d0 0x7f858415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8588e00640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85840b2bf0 0x7f858415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8589800640 1 -- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85840b2bf0 msgr2=0x7f858415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8589800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85840b2bf0 0x7f858415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8589800640 1 -- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85840b43d0 msgr2=0x7f858415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8589800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85840b43d0 0x7f858415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8589800640 1 -- 172.21.15.122:0/115783346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8584156100 con 0x7f85840cd100 2024-09-17T05:40:21.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8589800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85840cd100 0x7f858415c010 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f857c00cb10 tx=0x7f857c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:21.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8581600640 1 -- 172.21.15.122:0/115783346 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c016020 con 0x7f85840cd100 2024-09-17T05:40:21.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8581600640 1 -- 172.21.15.122:0/115783346 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f857c00ce50 con 0x7f85840cd100 2024-09-17T05:40:21.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8581600640 1 -- 172.21.15.122:0/115783346 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c00ade0 con 0x7f85840cd100 2024-09-17T05:40:21.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8584157510 con 0x7f85840cd100 2024-09-17T05:40:21.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85841566a0 con 0x7f85840cd100 2024-09-17T05:40:21.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8584069410 con 0x7f85840cd100 2024-09-17T05:40:21.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.305+0000 7f8581600640 1 -- 172.21.15.122:0/115783346 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f857c019020 con 0x7f85840cd100 2024-09-17T05:40:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.309+0000 7f8581600640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f85600754e0 0x7f85600779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.309+0000 7f8581600640 1 -- 172.21.15.122:0/115783346 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4941+0+0 (secure 0 0 0) 0x7f857c0934e0 con 0x7f85840cd100 2024-09-17T05:40:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.309+0000 7f8588e00640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f85600754e0 0x7f85600779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.309+0000 7f8588e00640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f85600754e0 0x7f85600779a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f85740b0000 tx=0x7f857401c380 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.313+0000 7f8581600640 1 -- 172.21.15.122:0/115783346 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f857c020020 con 0x7f85840cd100 2024-09-17T05:40:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.537+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f85840b47b0 con 0x7f85840cd100 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.537+0000 7f8581600640 1 -- 172.21.15.122:0/115783346 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7f857c05f9a0 con 0x7f85840cd100 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm4SWmGRAA8OMvaek++S89bD11D7dAug==", 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmiCKTJRAAvaQ3Hx0SNwptUS3lXju9pQ==", 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:40:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.c", 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmgyGsIBAAU4UOdN+JRQ26iD5mUtZbrA==", 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.d", 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmCiXlLBAAH8mo6XjW+akmn8fSvtLgkw==", 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB2Eulmnd1jMhAAcvfhxQGAe/32Q33dsyGRBA==", 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB6EulmQasQCxAAXu+LQl4wk1F5ny30MaCXvA==", 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:40:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-09-17T05:40:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB9EulmyCErLRAAgiF2oNvgMnobr1kDkiKRzA==", 2024-09-17T05:40:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:40:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCBEulmfxv8BhAA4JrlQDjY8zEDMW3qjz98Hw==", 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCHEulm6bfQBhAAqv4x4Pd9HaFPGFLA5tN/6Q==", 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCKEulmlihdGBAA29nmjUa1O44OCDGXb6/5DQ==", 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCNEulmG5YmKhAARps+J8p4USQRz+Mvrj4VBQ==", 2024-09-17T05:40:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCQEulmDJs0OxAArBl407YtZCij0yyETTnUfQ==", 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ==", 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow r", 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.1", 2024-09-17T05:40:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmFnZaOhAA0AMz1+BZsNEjexOdl4fqtQ==", 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.2", 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmGgzZNhAAtp8n/k/JLIYoyyO1VJ0n+A==", 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:40:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.3", 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQB0EulmmIHWBBAAcgcSDN/CZSN4b5IHNHgXgA==", 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-09-17T05:40:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQByEulm9KNnORAAEAkLwr+TXWaEBv8ECSpQnA==", 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmHKR8FRAAg+5OaCxBHAdFa85MoSzLdg==", 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQBzEulmrfKeERAABC3v/kgjVCCBPprGqnotMg==", 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-09-17T05:40:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-09-17T05:40:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-09-17T05:40:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-17T05:40:21.549 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.537+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f85600754e0 msgr2=0x7f85600779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.537+0000 7f858b800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f85600754e0 0x7f85600779a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f85740b0000 tx=0x7f857401c380 comp rx=0 tx=0).stop 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.537+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85840cd100 msgr2=0x7f858415c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.537+0000 7f858b800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85840cd100 0x7f858415c010 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f857c00cb10 tx=0x7f857c00cc00 comp rx=0 tx=0).stop 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.541+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 shutdown_connections 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.541+0000 7f858b800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f85600754e0 0x7f85600779a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.541+0000 7f858b800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85840cd100 0x7f858415c010 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.541+0000 7f858b800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85840b43d0 0x7f858415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.541+0000 7f858b800640 1 --2- 172.21.15.122:0/115783346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85840b2bf0 0x7f858415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.541+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 >> 172.21.15.122:0/115783346 conn(0x7f85840b18f0 msgr2=0x7f8584159540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:21.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.541+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 shutdown_connections 2024-09-17T05:40:21.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.541+0000 7f858b800640 1 -- 172.21.15.122:0/115783346 wait complete. 2024-09-17T05:40:21.556 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:40:21.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 -- 172.21.15.122:0/169738679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d00d29e0 msgr2=0x7f49d00b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 --2- 172.21.15.122:0/169738679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d00d29e0 0x7f49d00b6f80 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009c80 tx=0x7f49c801cd30 comp rx=0 tx=0).stop 2024-09-17T05:40:21.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 -- 172.21.15.122:0/169738679 shutdown_connections 2024-09-17T05:40:21.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 --2- 172.21.15.122:0/169738679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d00d29e0 0x7f49d00b6f80 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 --2- 172.21.15.122:0/169738679 >> v2:172.21.15.173:3300/0 conn(0x7f49d01617b0 0x7f49d00d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 -- 172.21.15.122:0/169738679 >> 172.21.15.122:0/169738679 conn(0x7f49d00d3e00 msgr2=0x7f49d00d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 -- 172.21.15.122:0/169738679 shutdown_connections 2024-09-17T05:40:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 -- 172.21.15.122:0/169738679 wait complete. 2024-09-17T05:40:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 Processor -- start 2024-09-17T05:40:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.701+0000 7f49d5c00640 1 -- start start 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d00d29e0 0x7f49d0159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d01613d0 0x7f49d00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d01617b0 0x7f49d01faea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49d016acc0 con 0x7f49d01617b0 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49d00b8240 con 0x7f49d00d29e0 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f49d016ae40 con 0x7f49d01613d0 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d01617b0 0x7f49d01faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d01617b0 0x7f49d01faea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39858/0 (socket says 172.21.15.122:39858) 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 -- 172.21.15.122:0/1505648876 learned_addr learned my addr 172.21.15.122:0/1505648876 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49cfe00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d00d29e0 0x7f49d0159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49cf400640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d01613d0 0x7f49d00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 -- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d01613d0 msgr2=0x7f49d00caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d01613d0 0x7f49d00caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 -- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d00d29e0 msgr2=0x7f49d0159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d00d29e0 0x7f49d0159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 -- 172.21.15.122:0/1505648876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d01512f0 con 0x7f49d01617b0 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49cfe00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d00d29e0 0x7f49d0159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49cf400640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d01613d0 0x7f49d00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5200640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d01617b0 0x7f49d01faea0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f49c801a960 tx=0x7f49c80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49c7e00640 1 -- 172.21.15.122:0/1505648876 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c80bb070 con 0x7f49d01617b0 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49c7e00640 1 -- 172.21.15.122:0/1505648876 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c8004c50 con 0x7f49d01617b0 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49c7e00640 1 -- 172.21.15.122:0/1505648876 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c80b2040 con 0x7f49d01617b0 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d0155690 con 0x7f49d01617b0 2024-09-17T05:40:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d0150d30 con 0x7f49d01617b0 2024-09-17T05:40:21.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.705+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d00b6f80 con 0x7f49d01617b0 2024-09-17T05:40:21.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.709+0000 7f49c7e00640 1 -- 172.21.15.122:0/1505648876 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f49c80ad850 con 0x7f49d01617b0 2024-09-17T05:40:21.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.709+0000 7f49c7e00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b00754e0 0x7f49b00779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:21.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.709+0000 7f49c7e00640 1 -- 172.21.15.122:0/1505648876 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4941+0+0 (secure 0 0 0) 0x7f49c812d1e0 con 0x7f49d01617b0 2024-09-17T05:40:21.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.709+0000 7f49cfe00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b00754e0 0x7f49b00779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:21.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.709+0000 7f49cfe00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b00754e0 0x7f49b00779a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f49c00016d0 tx=0x7f49c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:21.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.713+0000 7f49c7e00640 1 -- 172.21.15.122:0/1505648876 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f49c8016950 con 0x7f49d01617b0 2024-09-17T05:40:21.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.881+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f49d015e3c0 con 0x7f49d01617b0 2024-09-17T05:40:21.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.881+0000 7f49c7e00640 1 -- 172.21.15.122:0/1505648876 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) ==== 84+0+65 (secure 0 0 0) 0x7f49c8004e20 con 0x7f49d01617b0 2024-09-17T05:40:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:21.879 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:40:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-09-17T05:40:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-09-17T05:40:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:21.879 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:40:21.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b00754e0 msgr2=0x7f49b00779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b00754e0 0x7f49b00779a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f49c00016d0 tx=0x7f49c0009040 comp rx=0 tx=0).stop 2024-09-17T05:40:21.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d01617b0 msgr2=0x7f49d01faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:21.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d01617b0 0x7f49d01faea0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f49c801a960 tx=0x7f49c80092e0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 shutdown_connections 2024-09-17T05:40:21.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f49b00754e0 0x7f49b00779a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49d01617b0 0x7f49d01faea0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d01613d0 0x7f49d00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 --2- 172.21.15.122:0/1505648876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d00d29e0 0x7f49d0159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:21.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 >> 172.21.15.122:0/1505648876 conn(0x7f49d00d3e00 msgr2=0x7f49d006e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:21.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 shutdown_connections 2024-09-17T05:40:21.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:21.885+0000 7f49d5c00640 1 -- 172.21.15.122:0/1505648876 wait complete. 2024-09-17T05:40:21.901 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:40:21.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:40:22.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.049+0000 7fca8f200640 1 -- 172.21.15.122:0/2700769009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca9015f700 msgr2=0x7fca900d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:22.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.049+0000 7fca8f200640 1 --2- 172.21.15.122:0/2700769009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca9015f700 0x7fca900d0720 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fca88009bc0 tx=0x7fca8801cb70 comp rx=0 tx=0).stop 2024-09-17T05:40:22.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.049+0000 7fca8f200640 1 -- 172.21.15.122:0/2700769009 shutdown_connections 2024-09-17T05:40:22.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.049+0000 7fca8f200640 1 --2- 172.21.15.122:0/2700769009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca9015f700 0x7fca900d0720 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:22.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.049+0000 7fca8f200640 1 --2- 172.21.15.122:0/2700769009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca900cff00 0x7fca900d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:22.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.049+0000 7fca8f200640 1 -- 172.21.15.122:0/2700769009 >> 172.21.15.122:0/2700769009 conn(0x7fca900d1da0 msgr2=0x7fca900d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:22.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.049+0000 7fca8f200640 1 -- 172.21.15.122:0/2700769009 shutdown_connections 2024-09-17T05:40:22.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.049+0000 7fca8f200640 1 -- 172.21.15.122:0/2700769009 wait complete. 2024-09-17T05:40:22.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 Processor -- start 2024-09-17T05:40:22.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 -- start start 2024-09-17T05:40:22.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca900cff00 0x7fca900b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca900d02e0 0x7fca9014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca9015f700 0x7fca9014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca900b0ca0 con 0x7fca900d02e0 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fca900b0b20 con 0x7fca9015f700 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fca90168bf0 con 0x7fca900cff00 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca900d02e0 0x7fca9014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca900d02e0 0x7fca9014b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39868/0 (socket says 172.21.15.122:39868) 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 -- 172.21.15.122:0/4194013303 learned_addr learned my addr 172.21.15.122:0/4194013303 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8de00640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca900cff00 0x7fca900b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8e800640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca9015f700 0x7fca9014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 -- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca900cff00 msgr2=0x7fca900b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca900cff00 0x7fca900b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:22.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 -- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca9015f700 msgr2=0x7fca9014f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:22.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca9015f700 0x7fca9014f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:22.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 -- 172.21.15.122:0/4194013303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca900cc880 con 0x7fca900d02e0 2024-09-17T05:40:22.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca87e00640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca900d02e0 0x7fca9014b8f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fca7800c460 tx=0x7fca7800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:22.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca86000640 1 -- 172.21.15.122:0/4194013303 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78016020 con 0x7fca900d02e0 2024-09-17T05:40:22.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca86000640 1 -- 172.21.15.122:0/4194013303 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca78012920 con 0x7fca900d02e0 2024-09-17T05:40:22.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca86000640 1 -- 172.21.15.122:0/4194013303 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78012d50 con 0x7fca900d02e0 2024-09-17T05:40:22.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca900c94a0 con 0x7fca900d02e0 2024-09-17T05:40:22.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca900c9a60 con 0x7fca900d02e0 2024-09-17T05:40:22.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.053+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca900d28c0 con 0x7fca900d02e0 2024-09-17T05:40:22.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.057+0000 7fca86000640 1 -- 172.21.15.122:0/4194013303 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fca78019020 con 0x7fca900d02e0 2024-09-17T05:40:22.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.057+0000 7fca86000640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca6c0754e0 0x7fca6c0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:22.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.057+0000 7fca86000640 1 -- 172.21.15.122:0/4194013303 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4941+0+0 (secure 0 0 0) 0x7fca78093210 con 0x7fca900d02e0 2024-09-17T05:40:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.057+0000 7fca8de00640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca6c0754e0 0x7fca6c0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.057+0000 7fca8de00640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca6c0754e0 0x7fca6c0779a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fca80009600 tx=0x7fca800091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.061+0000 7fca86000640 1 -- 172.21.15.122:0/4194013303 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fca78020020 con 0x7fca900d02e0 2024-09-17T05:40:22.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:22.233+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fca9015d860 con 0x7fca900d02e0 2024-09-17T05:40:23.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca86000640 1 -- 172.21.15.122:0/4194013303 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) ==== 138+0+0 (secure 0 0 0) 0x7fca7805f6d0 con 0x7fca900d02e0 2024-09-17T05:40:23.137 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca6c0754e0 msgr2=0x7fca6c0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca6c0754e0 0x7fca6c0779a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fca80009600 tx=0x7fca800091e0 comp rx=0 tx=0).stop 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca900d02e0 msgr2=0x7fca9014b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca900d02e0 0x7fca9014b8f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fca7800c460 tx=0x7fca7800c920 comp rx=0 tx=0).stop 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 shutdown_connections 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fca6c0754e0 0x7fca6c0779a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca9015f700 0x7fca9014f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca900d02e0 0x7fca9014b8f0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:23.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 --2- 172.21.15.122:0/4194013303 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca900cff00 0x7fca900b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:23.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.141+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 >> 172.21.15.122:0/4194013303 conn(0x7fca900d1da0 msgr2=0x7fca901523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:23.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.145+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 shutdown_connections 2024-09-17T05:40:23.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.145+0000 7fca8f200640 1 -- 172.21.15.122:0/4194013303 wait complete. 2024-09-17T05:40:23.166 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:40:23.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- 172.21.15.122:0/2872407690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f746c004810 msgr2=0x7f746c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:23.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 --2- 172.21.15.122:0/2872407690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f746c004810 0x7f746c004c90 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f7474009b60 tx=0x7f747401d270 comp rx=0 tx=0).stop 2024-09-17T05:40:23.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- 172.21.15.122:0/2872407690 shutdown_connections 2024-09-17T05:40:23.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 --2- 172.21.15.122:0/2872407690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f746c004810 0x7f746c004c90 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:23.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- 172.21.15.122:0/2872407690 >> 172.21.15.122:0/2872407690 conn(0x7f74880d1dd0 msgr2=0x7f74880d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:23.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- 172.21.15.122:0/2872407690 shutdown_connections 2024-09-17T05:40:23.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- 172.21.15.122:0/2872407690 wait complete. 2024-09-17T05:40:23.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 Processor -- start 2024-09-17T05:40:23.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- start start 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f746c004810 0x7f74880b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74880c8850 0x7f74880cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f748815ab50 0x7f748815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74880b0e50 con 0x7f748815ab50 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74880b0cd0 con 0x7f74880c8850 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f748e800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f74880b1220 con 0x7f746c004810 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f748815ab50 0x7f748815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f748815ab50 0x7f748815afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39888/0 (socket says 172.21.15.122:39888) 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 -- 172.21.15.122:0/1597429963 learned_addr learned my addr 172.21.15.122:0/1597429963 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7486a00640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74880c8850 0x7f74880cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:23.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487400640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f746c004810 0x7f74880b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:23.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 -- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f746c004810 msgr2=0x7f74880b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f746c004810 0x7f74880b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 -- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74880c8850 msgr2=0x7f74880cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74880c8850 0x7f74880cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 -- 172.21.15.122:0/1597429963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74881f61f0 con 0x7f748815ab50 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.325+0000 7f7487e00640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f748815ab50 0x7f748815afd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7488069c10 tx=0x7f7480006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f747f400640 1 -- 172.21.15.122:0/1597429963 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480015070 con 0x7f748815ab50 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f747f400640 1 -- 172.21.15.122:0/1597429963 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f748000faa0 con 0x7f748815ab50 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74880c5ba0 con 0x7f748815ab50 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f747f400640 1 -- 172.21.15.122:0/1597429963 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480011430 con 0x7f748815ab50 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881f6790 con 0x7f748815ab50 2024-09-17T05:40:23.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74880b1a80 con 0x7f748815ab50 2024-09-17T05:40:23.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f747f400640 1 -- 172.21.15.122:0/1597429963 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f74800115d0 con 0x7f748815ab50 2024-09-17T05:40:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f747f400640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7468075510 0x7f74680779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f747f400640 1 -- 172.21.15.122:0/1597429963 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5311+0+0 (secure 0 0 0) 0x7f7480094090 con 0x7f748815ab50 2024-09-17T05:40:23.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.329+0000 7f7487400640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7468075510 0x7f74680779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:23.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.333+0000 7f7487400640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7468075510 0x7f74680779d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7478000c10 tx=0x7f7478006b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:23.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.333+0000 7f747f400640 1 -- 172.21.15.122:0/1597429963 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7480060550 con 0x7f748815ab50 2024-09-17T05:40:23.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:23.505+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f74880d1080 con 0x7f748815ab50 2024-09-17T05:40:24.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.145+0000 7f747f400640 1 -- 172.21.15.122:0/1597429963 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) ==== 170+0+0 (secure 0 0 0) 0x7f7480064f80 con 0x7f748815ab50 2024-09-17T05:40:24.140 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7468075510 msgr2=0x7f74680779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7468075510 0x7f74680779d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7478000c10 tx=0x7f7478006b20 comp rx=0 tx=0).stop 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f748815ab50 msgr2=0x7f748815afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f748815ab50 0x7f748815afd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7488069c10 tx=0x7f7480006720 comp rx=0 tx=0).stop 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 shutdown_connections 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7468075510 0x7f74680779d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f748815ab50 0x7f748815afd0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74880c8850 0x7f74880cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 --2- 172.21.15.122:0/1597429963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f746c004810 0x7f74880b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 >> 172.21.15.122:0/1597429963 conn(0x7f74880d1dd0 msgr2=0x7f7488071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 shutdown_connections 2024-09-17T05:40:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.149+0000 7f748e800640 1 -- 172.21.15.122:0/1597429963 wait complete. 2024-09-17T05:40:24.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:40:24.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 -- 172.21.15.122:0/367395598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72400b49e0 msgr2=0x7f724006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:24.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 --2- 172.21.15.122:0/367395598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72400b49e0 0x7f724006e7a0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f72400729f0 tx=0x7f723801c850 comp rx=0 tx=0).stop 2024-09-17T05:40:24.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 -- 172.21.15.122:0/367395598 shutdown_connections 2024-09-17T05:40:24.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 --2- 172.21.15.122:0/367395598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72400b49e0 0x7f724006e7a0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:24.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 --2- 172.21.15.122:0/367395598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7240073d10 0x7f72400cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:24.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 -- 172.21.15.122:0/367395598 >> 172.21.15.122:0/367395598 conn(0x7f72400b18f0 msgr2=0x7f72400b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:24.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 -- 172.21.15.122:0/367395598 shutdown_connections 2024-09-17T05:40:24.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 -- 172.21.15.122:0/367395598 wait complete. 2024-09-17T05:40:24.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 Processor -- start 2024-09-17T05:40:24.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.333+0000 7f7244a00640 1 -- start start 2024-09-17T05:40:24.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7240073d10 0x7f724015d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:24.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72400b49e0 0x7f724014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:24.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724015a660 0x7f7240159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7240167310 con 0x7f724015a660 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7240167190 con 0x7f7240073d10 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7240167490 con 0x7f72400b49e0 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724015a660 0x7f7240159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7240073d10 0x7f724015d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7235400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72400b49e0 0x7f724014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7240073d10 0x7f724015d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:52340/0 (socket says 172.21.15.122:52340) 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723de00640 1 -- 172.21.15.122:0/1430642818 learned_addr learned my addr 172.21.15.122:0/1430642818 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723e800640 1 -- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72400b49e0 msgr2=0x7f724014c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723e800640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72400b49e0 0x7f724014c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:24.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723e800640 1 -- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7240073d10 msgr2=0x7f724015d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:24.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723e800640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7240073d10 0x7f724015d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:24.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723e800640 1 -- 172.21.15.122:0/1430642818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72400af510 con 0x7f724015a660 2024-09-17T05:40:24.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7235400640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72400b49e0 0x7f724014c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:40:24.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f723e800640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724015a660 0x7f7240159280 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f72380ab000 tx=0x7f7238005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:24.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7236a00640 1 -- 172.21.15.122:0/1430642818 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72380ac070 con 0x7f724015a660 2024-09-17T05:40:24.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7236a00640 1 -- 172.21.15.122:0/1430642818 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7238005040 con 0x7f724015a660 2024-09-17T05:40:24.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724015d990 con 0x7f724015a660 2024-09-17T05:40:24.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7236a00640 1 -- 172.21.15.122:0/1430642818 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238005470 con 0x7f724015a660 2024-09-17T05:40:24.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724009ad90 con 0x7f724015a660 2024-09-17T05:40:24.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7240069410 con 0x7f724015a660 2024-09-17T05:40:24.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.337+0000 7f7236a00640 1 -- 172.21.15.122:0/1430642818 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f72380af070 con 0x7f724015a660 2024-09-17T05:40:24.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.341+0000 7f7236a00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f721c0754e0 0x7f721c0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:24.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.341+0000 7f7236a00640 1 -- 172.21.15.122:0/1430642818 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5693+0+0 (secure 0 0 0) 0x7f723812d960 con 0x7f724015a660 2024-09-17T05:40:24.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.341+0000 7f723de00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f721c0754e0 0x7f721c0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:24.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.341+0000 7f723de00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f721c0754e0 0x7f721c0779a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f72300012f0 tx=0x7f7230009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:24.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.345+0000 7f7236a00640 1 -- 172.21.15.122:0/1430642818 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f72380c1030 con 0x7f724015a660 2024-09-17T05:40:24.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:24.545+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f72400cd100 con 0x7f724015a660 2024-09-17T05:40:25.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.169+0000 7f7236a00640 1 -- 172.21.15.122:0/1430642818 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v92) ==== 174+0+0 (secure 0 0 0) 0x7f72380164d0 con 0x7f724015a660 2024-09-17T05:40:25.168 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 20 and data pool 21 2024-09-17T05:40:25.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f721c0754e0 msgr2=0x7f721c0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f721c0754e0 0x7f721c0779a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f72300012f0 tx=0x7f7230009040 comp rx=0 tx=0).stop 2024-09-17T05:40:25.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724015a660 msgr2=0x7f7240159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724015a660 0x7f7240159280 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f72380ab000 tx=0x7f7238005d60 comp rx=0 tx=0).stop 2024-09-17T05:40:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 shutdown_connections 2024-09-17T05:40:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f721c0754e0 0x7f721c0779a0 secure :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f72300012f0 tx=0x7f7230009040 comp rx=0 tx=0).stop 2024-09-17T05:40:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724015a660 0x7f7240159280 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72400b49e0 0x7f724014c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 --2- 172.21.15.122:0/1430642818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7240073d10 0x7f724015d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.177+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 >> 172.21.15.122:0/1430642818 conn(0x7f72400b18f0 msgr2=0x7f72401589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:25.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.185+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 shutdown_connections 2024-09-17T05:40:25.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.189+0000 7f7244a00640 1 -- 172.21.15.122:0/1430642818 wait complete. 2024-09-17T05:40:25.215 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:40:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 -- 172.21.15.122:0/1498171102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 msgr2=0x7fa2540ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 --2- 172.21.15.122:0/1498171102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 0x7fa2540ce4e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fa25415f940 tx=0x7fa24401cd80 comp rx=0 tx=0).stop 2024-09-17T05:40:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 -- 172.21.15.122:0/1498171102 shutdown_connections 2024-09-17T05:40:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 --2- 172.21.15.122:0/1498171102 >> v2:172.21.15.173:3300/0 conn(0x7fa2540d28c0 0x7fa2540cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 --2- 172.21.15.122:0/1498171102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 0x7fa2540ce4e0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 -- 172.21.15.122:0/1498171102 >> 172.21.15.122:0/1498171102 conn(0x7fa2540d1da0 msgr2=0x7fa2540d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 -- 172.21.15.122:0/1498171102 shutdown_connections 2024-09-17T05:40:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 -- 172.21.15.122:0/1498171102 wait complete. 2024-09-17T05:40:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 Processor -- start 2024-09-17T05:40:25.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.361+0000 7fa25b400640 1 -- start start 2024-09-17T05:40:25.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2540b53f0 0x7fa254155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:25.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2540d28c0 0x7fa25415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:25.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 0x7fa25406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:25.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa254169160 con 0x7fa25415be80 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa254168fe0 con 0x7fa2540b53f0 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa2541692e0 con 0x7fa2540d28c0 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 0x7fa25406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa258a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2540b53f0 0x7fa254155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 0x7fa25406dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39928/0 (socket says 172.21.15.122:39928) 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 -- 172.21.15.122:0/1923969591 learned_addr learned my addr 172.21.15.122:0/1923969591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa253e00640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2540d28c0 0x7fa25415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 -- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2540d28c0 msgr2=0x7fa25415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2540d28c0 0x7fa25415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 -- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2540b53f0 msgr2=0x7fa254155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2540b53f0 0x7fa254155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 -- 172.21.15.122:0/1923969591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa254159bf0 con 0x7fa25415be80 2024-09-17T05:40:25.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa258a00640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2540b53f0 0x7fa254155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:40:25.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa259400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 0x7fa25406dd30 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00c730 tx=0x7fa24c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:25.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa250c00640 1 -- 172.21.15.122:0/1923969591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c017070 con 0x7fa25415be80 2024-09-17T05:40:25.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa250c00640 1 -- 172.21.15.122:0/1923969591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa24c005070 con 0x7fa25415be80 2024-09-17T05:40:25.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25415b9a0 con 0x7fa25415be80 2024-09-17T05:40:25.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa250c00640 1 -- 172.21.15.122:0/1923969591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00a640 con 0x7fa25415be80 2024-09-17T05:40:25.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2541fbc30 con 0x7fa25415be80 2024-09-17T05:40:25.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2540b57f0 con 0x7fa25415be80 2024-09-17T05:40:25.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.365+0000 7fa250c00640 1 -- 172.21.15.122:0/1923969591 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fa24c01a070 con 0x7fa25415be80 2024-09-17T05:40:25.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.369+0000 7fa250c00640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa2340754e0 0x7fa2340779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:25.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.369+0000 7fa250c00640 1 -- 172.21.15.122:0/1923969591 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5793+0+0 (secure 0 0 0) 0x7fa24c092680 con 0x7fa25415be80 2024-09-17T05:40:25.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.369+0000 7fa258a00640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa2340754e0 0x7fa2340779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:25.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.369+0000 7fa258a00640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa2340754e0 0x7fa2340779a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa2440af000 tx=0x7fa24401b5c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:25.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.373+0000 7fa250c00640 1 -- 172.21.15.122:0/1923969591 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa2540d0350 con 0x7fa25415be80 2024-09-17T05:40:25.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.541+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2540b7a60 con 0x7fa25415be80 2024-09-17T05:40:25.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.541+0000 7fa250c00640 1 -- 172.21.15.122:0/1923969591 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16944 (secure 0 0 0) 0x7fa24c05eb40 con 0x7fa25415be80 2024-09-17T05:40:25.538 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:25.538 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":144,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:40:24.548865+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:40:22.246801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T05:40:23.519850+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:40:25.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.541+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa2340754e0 msgr2=0x7fa2340779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.541+0000 7fa25b400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa2340754e0 0x7fa2340779a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa2440af000 tx=0x7fa24401b5c0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 msgr2=0x7fa25406dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 0x7fa25406dd30 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00c730 tx=0x7fa24c00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 shutdown_connections 2024-09-17T05:40:25.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa2340754e0 0x7fa2340779a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa25415be80 0x7fa25406dd30 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2540d28c0 0x7fa25415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 --2- 172.21.15.122:0/1923969591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2540b53f0 0x7fa254155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 >> 172.21.15.122:0/1923969591 conn(0x7fa2540d1da0 msgr2=0x7fa2540ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:25.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 shutdown_connections 2024-09-17T05:40:25.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.545+0000 7fa25b400640 1 -- 172.21.15.122:0/1923969591 wait complete. 2024-09-17T05:40:25.559 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:40:25.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 -- 172.21.15.122:0/355380176 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39d415bea0 msgr2=0x7f39d40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 --2- 172.21.15.122:0/355380176 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39d415bea0 0x7f39d40ce670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009840 tx=0x7f39c401c820 comp rx=0 tx=0).stop 2024-09-17T05:40:25.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 -- 172.21.15.122:0/355380176 shutdown_connections 2024-09-17T05:40:25.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 --2- 172.21.15.122:0/355380176 >> v2:172.21.15.173:3300/0 conn(0x7f39d40cdb30 0x7f39d40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 --2- 172.21.15.122:0/355380176 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39d415bea0 0x7f39d40ce670 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 -- 172.21.15.122:0/355380176 >> 172.21.15.122:0/355380176 conn(0x7f39d40d1dc0 msgr2=0x7f39d40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:25.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 -- 172.21.15.122:0/355380176 shutdown_connections 2024-09-17T05:40:25.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 -- 172.21.15.122:0/355380176 wait complete. 2024-09-17T05:40:25.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 Processor -- start 2024-09-17T05:40:25.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.705+0000 7f39da800640 1 -- start start 2024-09-17T05:40:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39d40cdb30 0x7f39d409bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39d40d28e0 0x7f39d414c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d415bea0 0x7f39d40cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4168f10 con 0x7f39d415bea0 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4168d90 con 0x7f39d40cdb30 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f39d4169090 con 0x7f39d40d28e0 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d415bea0 0x7f39d40cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d415bea0 0x7f39d40cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39940/0 (socket says 172.21.15.122:39940) 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 -- 172.21.15.122:0/3361538675 learned_addr learned my addr 172.21.15.122:0/3361538675 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39caa00640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39d40d28e0 0x7f39d414c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3400640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39d40cdb30 0x7f39d409bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 -- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39d40d28e0 msgr2=0x7f39d414c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39d40d28e0 0x7f39d414c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 -- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39d40cdb30 msgr2=0x7f39d409bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39d40cdb30 0x7f39d409bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 -- 172.21.15.122:0/3361538675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d40b20d0 con 0x7f39d415bea0 2024-09-17T05:40:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39d3e00640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d415bea0 0x7f39d40cb060 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00cbf0 tx=0x7f39cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39cbe00640 1 -- 172.21.15.122:0/3361538675 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc016020 con 0x7f39d415bea0 2024-09-17T05:40:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39cbe00640 1 -- 172.21.15.122:0/3361538675 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39cc00ab00 con 0x7f39d415bea0 2024-09-17T05:40:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d40b1830 con 0x7f39d415bea0 2024-09-17T05:40:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39cbe00640 1 -- 172.21.15.122:0/3361538675 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc012420 con 0x7f39d415bea0 2024-09-17T05:40:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d40b1df0 con 0x7f39d415bea0 2024-09-17T05:40:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.709+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d40cc9d0 con 0x7f39d415bea0 2024-09-17T05:40:25.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.713+0000 7f39cbe00640 1 -- 172.21.15.122:0/3361538675 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f39cc019020 con 0x7f39d415bea0 2024-09-17T05:40:25.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.713+0000 7f39cbe00640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39b00754e0 0x7f39b00779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:25.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.713+0000 7f39cbe00640 1 -- 172.21.15.122:0/3361538675 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5793+0+0 (secure 0 0 0) 0x7f39cc0939b0 con 0x7f39d415bea0 2024-09-17T05:40:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.713+0000 7f39d3400640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39b00754e0 0x7f39b00779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.713+0000 7f39d3400640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39b00754e0 0x7f39b00779a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f39c40af000 tx=0x7f39c401b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:25.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.717+0000 7f39cbe00640 1 -- 172.21.15.122:0/3361538675 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f39cc020020 con 0x7f39d415bea0 2024-09-17T05:40:25.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.885+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39d415b610 con 0x7f39d415bea0 2024-09-17T05:40:25.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.885+0000 7f39cbe00640 1 -- 172.21.15.122:0/3361538675 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16944 (secure 0 0 0) 0x7f39cc05fef0 con 0x7f39d415bea0 2024-09-17T05:40:25.883 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:25.883 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":144,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:40:24.548865+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:40:22.246801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T05:40:23.519850+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:40:25.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39b00754e0 msgr2=0x7f39b00779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39b00754e0 0x7f39b00779a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f39c40af000 tx=0x7f39c401b060 comp rx=0 tx=0).stop 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d415bea0 msgr2=0x7f39d40cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d415bea0 0x7f39d40cb060 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00cbf0 tx=0x7f39cc00cce0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 shutdown_connections 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f39b00754e0 0x7f39b00779a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39d415bea0 0x7f39d40cb060 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39d40d28e0 0x7f39d414c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 --2- 172.21.15.122:0/3361538675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39d40cdb30 0x7f39d409bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 >> 172.21.15.122:0/3361538675 conn(0x7f39d40d1dc0 msgr2=0x7f39d406f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:25.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 shutdown_connections 2024-09-17T05:40:25.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:25.889+0000 7f39da800640 1 -- 172.21.15.122:0/3361538675 wait complete. 2024-09-17T05:40:25.906 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:40:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.053+0000 7fc166a00640 1 -- 172.21.15.122:0/1340124749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16006f8b0 msgr2=0x7fc1600acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.053+0000 7fc166a00640 1 --2- 172.21.15.122:0/1340124749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16006f8b0 0x7fc1600acd40 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fc160074c10 tx=0x7fc15001c620 comp rx=0 tx=0).stop 2024-09-17T05:40:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.053+0000 7fc166a00640 1 -- 172.21.15.122:0/1340124749 shutdown_connections 2024-09-17T05:40:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.053+0000 7fc166a00640 1 --2- 172.21.15.122:0/1340124749 >> v2:172.21.15.173:3300/0 conn(0x7fc16015db50 0x7fc1600b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.053+0000 7fc166a00640 1 --2- 172.21.15.122:0/1340124749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc16006f8b0 0x7fc1600acd40 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.053+0000 7fc166a00640 1 -- 172.21.15.122:0/1340124749 >> 172.21.15.122:0/1340124749 conn(0x7fc1600b18f0 msgr2=0x7fc1600b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.053+0000 7fc166a00640 1 -- 172.21.15.122:0/1340124749 shutdown_connections 2024-09-17T05:40:26.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.053+0000 7fc166a00640 1 -- 172.21.15.122:0/1340124749 wait complete. 2024-09-17T05:40:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 Processor -- start 2024-09-17T05:40:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 -- start start 2024-09-17T05:40:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc16006f8b0 0x7fc16014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1600cd100 0x7fc160154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc16015db50 0x7fc16015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1600cf4f0 con 0x7fc1600cd100 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1600cf370 con 0x7fc16015db50 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc160168130 con 0x7fc16006f8b0 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1600cd100 0x7fc160154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1600cd100 0x7fc160154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39944/0 (socket says 172.21.15.122:39944) 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 -- 172.21.15.122:0/2038466940 learned_addr learned my addr 172.21.15.122:0/2038466940 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc15fe00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc16006f8b0 0x7fc16014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc164a00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc16015db50 0x7fc16015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 -- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc16006f8b0 msgr2=0x7fc16014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc16006f8b0 0x7fc16014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:26.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 -- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc16015db50 msgr2=0x7fc16015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:26.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc16015db50 0x7fc16015c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:26.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 -- 172.21.15.122:0/2038466940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc160155640 con 0x7fc1600cd100 2024-09-17T05:40:26.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc157e00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1600cd100 0x7fc160154bf0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c4c0 tx=0x7fc148001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:26.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc15cc00640 1 -- 172.21.15.122:0/2038466940 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148017070 con 0x7fc1600cd100 2024-09-17T05:40:26.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc15cc00640 1 -- 172.21.15.122:0/2038466940 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc148003600 con 0x7fc1600cd100 2024-09-17T05:40:26.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1601f7bb0 con 0x7fc1600cd100 2024-09-17T05:40:26.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc15cc00640 1 -- 172.21.15.122:0/2038466940 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148003a50 con 0x7fc1600cd100 2024-09-17T05:40:26.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16014fa80 con 0x7fc1600cd100 2024-09-17T05:40:26.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.057+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc160069410 con 0x7fc1600cd100 2024-09-17T05:40:26.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.061+0000 7fc15cc00640 1 -- 172.21.15.122:0/2038466940 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fc14801a070 con 0x7fc1600cd100 2024-09-17T05:40:26.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.061+0000 7fc15cc00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc13c0754e0 0x7fc13c0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:26.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.061+0000 7fc15cc00640 1 -- 172.21.15.122:0/2038466940 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5793+0+0 (secure 0 0 0) 0x7fc148092710 con 0x7fc1600cd100 2024-09-17T05:40:26.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.061+0000 7fc15fe00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc13c0754e0 0x7fc13c0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:26.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.061+0000 7fc15fe00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc13c0754e0 0x7fc13c0779a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc1500af000 tx=0x7fc15001ae60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:26.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.065+0000 7fc15cc00640 1 -- 172.21.15.122:0/2038466940 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc14805f220 con 0x7fc1600cd100 2024-09-17T05:40:26.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:26.269+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc1600acd40 con 0x7fc1600cd100 2024-09-17T05:40:27.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.177+0000 7fc15cc00640 1 -- 172.21.15.122:0/2038466940 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) ==== 118+0+0 (secure 0 0 0) 0x7fc14805eb20 con 0x7fc1600cd100 2024-09-17T05:40:27.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc13c0754e0 msgr2=0x7fc13c0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:27.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc13c0754e0 0x7fc13c0779a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc1500af000 tx=0x7fc15001ae60 comp rx=0 tx=0).stop 2024-09-17T05:40:27.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1600cd100 msgr2=0x7fc160154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1600cd100 0x7fc160154bf0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fc14800c4c0 tx=0x7fc148001e80 comp rx=0 tx=0).stop 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 shutdown_connections 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc13c0754e0 0x7fc13c0779a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc16015db50 0x7fc16015c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1600cd100 0x7fc160154bf0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 --2- 172.21.15.122:0/2038466940 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc16006f8b0 0x7fc16014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 >> 172.21.15.122:0/2038466940 conn(0x7fc1600b18f0 msgr2=0x7fc160159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 shutdown_connections 2024-09-17T05:40:27.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.181+0000 7fc166a00640 1 -- 172.21.15.122:0/2038466940 wait complete. 2024-09-17T05:40:27.201 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:40:27.201 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:40:27.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.353+0000 7f425d200640 1 -- 172.21.15.122:0/1078795194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42580740c0 msgr2=0x7f4258074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:27.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.353+0000 7f425d200640 1 --2- 172.21.15.122:0/1078795194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42580740c0 0x7f4258074500 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4250009bc0 tx=0x7f425001cb70 comp rx=0 tx=0).stop 2024-09-17T05:40:27.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.353+0000 7f425d200640 1 -- 172.21.15.122:0/1078795194 shutdown_connections 2024-09-17T05:40:27.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.353+0000 7f425d200640 1 --2- 172.21.15.122:0/1078795194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42580740c0 0x7f4258074500 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:27.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.353+0000 7f425d200640 1 --2- 172.21.15.122:0/1078795194 >> v2:172.21.15.173:3300/0 conn(0x7f425815f220 0x7f425815b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:27.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.353+0000 7f425d200640 1 -- 172.21.15.122:0/1078795194 >> 172.21.15.122:0/1078795194 conn(0x7f425806ecb0 msgr2=0x7f425806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:27.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.353+0000 7f425d200640 1 -- 172.21.15.122:0/1078795194 shutdown_connections 2024-09-17T05:40:27.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.353+0000 7f425d200640 1 -- 172.21.15.122:0/1078795194 wait complete. 2024-09-17T05:40:27.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 Processor -- start 2024-09-17T05:40:27.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 -- start start 2024-09-17T05:40:27.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42580740c0 0x7f42580a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42581584d0 0x7f42581469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f425815f220 0x7f4258157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4258167960 con 0x7f425815f220 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f42581677e0 con 0x7f42581584d0 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4258167ae0 con 0x7f42580740c0 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f425815f220 0x7f4258157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f425815f220 0x7f4258157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39958/0 (socket says 172.21.15.122:39958) 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 -- 172.21.15.122:0/3936691502 learned_addr learned my addr 172.21.15.122:0/3936691502 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4255400640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42581584d0 0x7f42581469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4255e00640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42580740c0 0x7f42580a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 -- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42580740c0 msgr2=0x7f42580a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42580740c0 0x7f42580a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:27.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 -- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42581584d0 msgr2=0x7f42581469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42581584d0 0x7f42581469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 -- 172.21.15.122:0/3936691502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42580cda90 con 0x7f425815f220 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f4256800640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f425815f220 0x7f4258157620 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f425001a7a0 tx=0x7f4250009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f424e000640 1 -- 172.21.15.122:0/3936691502 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500b2040 con 0x7f425815f220 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f424e000640 1 -- 172.21.15.122:0/3936691502 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4250004d40 con 0x7f425815f220 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42580cdd70 con 0x7f425815f220 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f424e000640 1 -- 172.21.15.122:0/3936691502 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250004240 con 0x7f425815f220 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42580ce330 con 0x7f425815f220 2024-09-17T05:40:27.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.357+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4258074500 con 0x7f425815f220 2024-09-17T05:40:27.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.361+0000 7f424e000640 1 -- 172.21.15.122:0/3936691502 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f42500be060 con 0x7f425815f220 2024-09-17T05:40:27.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.361+0000 7f424e000640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42380754e0 0x7f42380779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:27.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.361+0000 7f424e000640 1 -- 172.21.15.122:0/3936691502 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f425012d120 con 0x7f425815f220 2024-09-17T05:40:27.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.361+0000 7f4255e00640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42380754e0 0x7f42380779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:27.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.361+0000 7f4255e00640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42380754e0 0x7f42380779a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f425815e5b0 tx=0x7f4248009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:27.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.365+0000 7f424e000640 1 -- 172.21.15.122:0/3936691502 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4250016790 con 0x7f425815f220 2024-09-17T05:40:27.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-17T05:40:27.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-17T05:40:27.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-17T05:40:27.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-17T05:40:27.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:27.565+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4258146f00 con 0x7f425815f220 2024-09-17T05:40:28.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f424e000640 1 -- 172.21.15.122:0/3936691502 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) ==== 122+0+0 (secure 0 0 0) 0x7f42500f9bb0 con 0x7f425815f220 2024-09-17T05:40:28.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42380754e0 msgr2=0x7f42380779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42380754e0 0x7f42380779a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f425815e5b0 tx=0x7f4248009040 comp rx=0 tx=0).stop 2024-09-17T05:40:28.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f425815f220 msgr2=0x7f4258157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f425815f220 0x7f4258157620 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f425001a7a0 tx=0x7f4250009300 comp rx=0 tx=0).stop 2024-09-17T05:40:28.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 shutdown_connections 2024-09-17T05:40:28.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f42380754e0 0x7f42380779a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f425815f220 0x7f4258157620 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42581584d0 0x7f42581469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 --2- 172.21.15.122:0/3936691502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42580740c0 0x7f42580a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 >> 172.21.15.122:0/3936691502 conn(0x7f425806ecb0 msgr2=0x7f425806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:28.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 shutdown_connections 2024-09-17T05:40:28.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.189+0000 7f425d200640 1 -- 172.21.15.122:0/3936691502 wait complete. 2024-09-17T05:40:28.209 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:40:28.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:40:28.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.357+0000 7fc8df200640 1 -- 172.21.15.122:0/888270618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8d80b0ce0 msgr2=0x7fc8d80b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.357+0000 7fc8df200640 1 --2- 172.21.15.122:0/888270618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8d80b0ce0 0x7fc8d80b5870 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009840 tx=0x7fc8c801c820 comp rx=0 tx=0).stop 2024-09-17T05:40:28.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.357+0000 7fc8df200640 1 -- 172.21.15.122:0/888270618 shutdown_connections 2024-09-17T05:40:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.357+0000 7fc8df200640 1 --2- 172.21.15.122:0/888270618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8d80b11a0 0x7fc8d8165670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.357+0000 7fc8df200640 1 --2- 172.21.15.122:0/888270618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8d80b0ce0 0x7fc8d80b5870 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.357+0000 7fc8df200640 1 -- 172.21.15.122:0/888270618 >> 172.21.15.122:0/888270618 conn(0x7fc8d809aa10 msgr2=0x7fc8d809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.357+0000 7fc8df200640 1 -- 172.21.15.122:0/888270618 shutdown_connections 2024-09-17T05:40:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.357+0000 7fc8df200640 1 -- 172.21.15.122:0/888270618 wait complete. 2024-09-17T05:40:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 Processor -- start 2024-09-17T05:40:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 -- start start 2024-09-17T05:40:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8d80b0ce0 0x7fc8d806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8d80b11a0 0x7fc8d8152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8d80b4d80 0x7fc8d8152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8168a10 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8168890 con 0x7fc8d80b11a0 2024-09-17T05:40:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d8168b90 con 0x7fc8d80b4d80 2024-09-17T05:40:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8d80b0ce0 0x7fc8d806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8d80b0ce0 0x7fc8d806dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39962/0 (socket says 172.21.15.122:39962) 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 -- 172.21.15.122:0/2622150618 learned_addr learned my addr 172.21.15.122:0/2622150618 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8de800640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8d80b4d80 0x7fc8d8152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dd400640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8d80b11a0 0x7fc8d8152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 -- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8d80b4d80 msgr2=0x7fc8d8152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8d80b4d80 0x7fc8d8152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 -- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8d80b11a0 msgr2=0x7fc8d8152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8d80b11a0 0x7fc8d8152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 -- 172.21.15.122:0/2622150618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d809f8b0 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8dde00640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8d80b0ce0 0x7fc8d806dd30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00c430 tx=0x7fc8cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8d2000640 1 -- 172.21.15.122:0/2622150618 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc011790 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8d2000640 1 -- 172.21.15.122:0/2622150618 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8cc017070 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d8159aa0 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8d2000640 1 -- 172.21.15.122:0/2622150618 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc011ce0 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d815a060 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.361+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d80b5160 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.365+0000 7fc8d2000640 1 -- 172.21.15.122:0/2622150618 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fc8cc01c460 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.365+0000 7fc8d2000640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc8bc0754e0 0x7fc8bc0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.365+0000 7fc8d2000640 1 -- 172.21.15.122:0/2622150618 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7fc8cc092fc0 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.365+0000 7fc8dd400640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc8bc0754e0 0x7fc8bc0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.365+0000 7fc8dd400640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc8bc0754e0 0x7fc8bc0779a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80af000 tx=0x7fc8c801b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:28.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.369+0000 7fc8d2000640 1 -- 172.21.15.122:0/2622150618 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc8cc05f5c0 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.569+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8d80ce0c0 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.569+0000 7fc8d2000640 1 -- 172.21.15.122:0/2622150618 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5233 (secure 0 0 0) 0x7fc8cc063ff0 con 0x7fc8d80b0ce0 2024-09-17T05:40:28.567 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:28.568 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":96,"btime":"2024-09-17T05:40:28:184035+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/223748651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":223748651},{"type":"v1","addr":"172.21.15.173:6837","nonce":223748651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/822031950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":822031950},{"type":"v1","addr":"172.21.15.173:6836","nonce":822031950}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2541492660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2541492660},{"type":"v1","addr":"172.21.15.122:6834","nonce":2541492660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:40:24.548833+0000","modified":"2024-09-17T05:40:27.570670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9399":{"gid":9399,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/4149068254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":4149068254},{"type":"v1","addr":"172.21.15.122:6835","nonce":4149068254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9399,"qdb_cluster":[9399]},"id":10}]} 2024-09-17T05:40:28.568 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 96 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc8bc0754e0 msgr2=0x7fc8bc0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc8bc0754e0 0x7fc8bc0779a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80af000 tx=0x7fc8c801b060 comp rx=0 tx=0).stop 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8d80b0ce0 msgr2=0x7fc8d806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8d80b0ce0 0x7fc8d806dd30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00c430 tx=0x7fc8cc00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 shutdown_connections 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc8bc0754e0 0x7fc8bc0779a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8d80b4d80 0x7fc8d8152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8d80b11a0 0x7fc8d8152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 --2- 172.21.15.122:0/2622150618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8d80b0ce0 0x7fc8d806dd30 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 >> 172.21.15.122:0/2622150618 conn(0x7fc8d809aa10 msgr2=0x7fc8d81562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:28.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 shutdown_connections 2024-09-17T05:40:28.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.573+0000 7fc8df200640 1 -- 172.21.15.122:0/2622150618 wait complete. 2024-09-17T05:40:28.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- 172.21.15.122:0/3609097956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8754003a10 msgr2=0x7f8754003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 --2- 172.21.15.122:0/3609097956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8754003a10 0x7f8754003e30 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8760009b60 tx=0x7f876001cf50 comp rx=0 tx=0).stop 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8775000640 1 -- 172.21.15.122:0/3609097956 reap_dead start 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- 172.21.15.122:0/3609097956 shutdown_connections 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- 172.21.15.122:0/3609097956 >> 172.21.15.122:0/3609097956 conn(0x7f87700d1ea0 msgr2=0x7f87700d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- 172.21.15.122:0/3609097956 shutdown_connections 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- 172.21.15.122:0/3609097956 wait complete. 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 Processor -- start 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- start start 2024-09-17T05:40:28.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8754003a10 0x7f877006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877014fea0 0x7f877014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87701fbc60 0x7f8770164c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8770166e90 con 0x7f877014fea0 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8770166d10 con 0x7f8754003a10 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8777000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8770167010 con 0x7f87701fbc60 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8767400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877014fea0 0x7f877014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8767400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877014fea0 0x7f877014f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39974/0 (socket says 172.21.15.122:39974) 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.737+0000 7f8767400640 1 -- 172.21.15.122:0/2884465507 learned_addr learned my addr 172.21.15.122:0/2884465507 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:28.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f876fe00640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8754003a10 0x7f877006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8775000640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87701fbc60 0x7f8770164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8767400640 1 -- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87701fbc60 msgr2=0x7f8770164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8767400640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87701fbc60 0x7f8770164c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8767400640 1 -- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8754003a10 msgr2=0x7f877006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8767400640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8754003a10 0x7f877006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8767400640 1 -- 172.21.15.122:0/2884465507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8770149f10 con 0x7f877014fea0 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8775000640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87701fbc60 0x7f8770164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f876fe00640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8754003a10 0x7f877006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8767400640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877014fea0 0x7f877014f730 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f87580017a0 tx=0x7f875800c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f876cc00640 1 -- 172.21.15.122:0/2884465507 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758015070 con 0x7f877014fea0 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f876cc00640 1 -- 172.21.15.122:0/2884465507 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8758011870 con 0x7f877014fea0 2024-09-17T05:40:28.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f876cc00640 1 -- 172.21.15.122:0/2884465507 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758011ca0 con 0x7f877014fea0 2024-09-17T05:40:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877014a1f0 con 0x7f877014fea0 2024-09-17T05:40:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877014a7b0 con 0x7f877014fea0 2024-09-17T05:40:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87700c5dd0 con 0x7f877014fea0 2024-09-17T05:40:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f876cc00640 1 -- 172.21.15.122:0/2884465507 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f8758018070 con 0x7f877014fea0 2024-09-17T05:40:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.741+0000 7f876cc00640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f874c075510 0x7f874c0779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.745+0000 7f876cc00640 1 -- 172.21.15.122:0/2884465507 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f8758094050 con 0x7f877014fea0 2024-09-17T05:40:28.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.745+0000 7f876fe00640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f874c075510 0x7f874c0779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:28.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.745+0000 7f876fe00640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f874c075510 0x7f874c0779d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f87600af000 tx=0x7f876001a010 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:28.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.745+0000 7f876cc00640 1 -- 172.21.15.122:0/2884465507 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f875800ce60 con 0x7f877014fea0 2024-09-17T05:40:28.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.917+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87700cd990 con 0x7f877014fea0 2024-09-17T05:40:28.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.917+0000 7f876cc00640 1 -- 172.21.15.122:0/2884465507 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16931 (secure 0 0 0) 0x7f8758060b70 con 0x7f877014fea0 2024-09-17T05:40:28.915 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:28.915 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":145,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:40:26.171613+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:40:22.246801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T05:40:23.519850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:40:28.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f874c075510 msgr2=0x7f874c0779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f874c075510 0x7f874c0779d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f87600af000 tx=0x7f876001a010 comp rx=0 tx=0).stop 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877014fea0 msgr2=0x7f877014f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877014fea0 0x7f877014f730 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f87580017a0 tx=0x7f875800c630 comp rx=0 tx=0).stop 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 shutdown_connections 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f874c075510 0x7f874c0779d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87701fbc60 0x7f8770164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f877014fea0 0x7f877014f730 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 --2- 172.21.15.122:0/2884465507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8754003a10 0x7f877006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 >> 172.21.15.122:0/2884465507 conn(0x7f87700d1ea0 msgr2=0x7f87700c8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 shutdown_connections 2024-09-17T05:40:28.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:28.921+0000 7f8777000640 1 -- 172.21.15.122:0/2884465507 wait complete. 2024-09-17T05:40:28.936 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:40:28.936 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:40:29.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 -- 172.21.15.122:0/1875290753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 msgr2=0x7f15180cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 --2- 172.21.15.122:0/1875290753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 0x7f15180cc5b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f1504009840 tx=0x7f150401c790 comp rx=0 tx=0).stop 2024-09-17T05:40:29.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 -- 172.21.15.122:0/1875290753 shutdown_connections 2024-09-17T05:40:29.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 --2- 172.21.15.122:0/1875290753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 0x7f15180cc5b0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 --2- 172.21.15.122:0/1875290753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151815f000 0x7f15180d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 -- 172.21.15.122:0/1875290753 >> 172.21.15.122:0/1875290753 conn(0x7f151809aa10 msgr2=0x7f151809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:29.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 -- 172.21.15.122:0/1875290753 shutdown_connections 2024-09-17T05:40:29.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 -- 172.21.15.122:0/1875290753 wait complete. 2024-09-17T05:40:29.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 Processor -- start 2024-09-17T05:40:29.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.081+0000 7f151de00640 1 -- start start 2024-09-17T05:40:29.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15180ccaf0 0x7f15181f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 0x7f1518071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151815f000 0x7f1518152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1518169290 con 0x7f15180cefb0 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1518169110 con 0x7f15180ccaf0 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1518169410 con 0x7f151815f000 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 0x7f1518071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 0x7f1518071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39992/0 (socket says 172.21.15.122:39992) 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 -- 172.21.15.122:0/2879118923 learned_addr learned my addr 172.21.15.122:0/2879118923 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 -- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151815f000 msgr2=0x7f1518152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151815f000 0x7f1518152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 -- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15180ccaf0 msgr2=0x7f15181f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15180ccaf0 0x7f15181f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 -- 172.21.15.122:0/2879118923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15180aeae0 con 0x7f15180cefb0 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1517e00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 0x7f1518071a10 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1504000c10 tx=0x7f15040a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1514c00640 1 -- 172.21.15.122:0/2879118923 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040ac070 con 0x7f15180cefb0 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1514c00640 1 -- 172.21.15.122:0/2879118923 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1504004310 con 0x7f15180cefb0 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f1514c00640 1 -- 172.21.15.122:0/2879118923 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15040053a0 con 0x7f15180cefb0 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151806d0f0 con 0x7f15180cefb0 2024-09-17T05:40:29.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1518166150 con 0x7f15180cefb0 2024-09-17T05:40:29.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.085+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15180cc5b0 con 0x7f15180cefb0 2024-09-17T05:40:29.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.089+0000 7f1514c00640 1 -- 172.21.15.122:0/2879118923 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f15040af070 con 0x7f15180cefb0 2024-09-17T05:40:29.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.089+0000 7f1514c00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f80754e0 0x7f14f80779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.089+0000 7f1514c00640 1 -- 172.21.15.122:0/2879118923 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f150412d950 con 0x7f15180cefb0 2024-09-17T05:40:29.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.089+0000 7f151ca00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f80754e0 0x7f14f80779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.089+0000 7f151ca00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f80754e0 0x7f14f80779a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f150c0058d0 tx=0x7f150c005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:29.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.093+0000 7f1514c00640 1 -- 172.21.15.122:0/2879118923 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f15040b6070 con 0x7f15180cefb0 2024-09-17T05:40:29.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.261+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1518164a40 con 0x7f14f80754e0 2024-09-17T05:40:29.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.265+0000 7f1514c00640 1 -- 172.21.15.122:0/2879118923 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243108 (secure 0 0 0) 0x7f1518164a40 con 0x7f14f80754e0 2024-09-17T05:40:29.264 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:29.265 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f80754e0 msgr2=0x7f14f80779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f80754e0 0x7f14f80779a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f150c0058d0 tx=0x7f150c005860 comp rx=0 tx=0).stop 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 msgr2=0x7f1518071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 0x7f1518071a10 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1504000c10 tx=0x7f15040a6570 comp rx=0 tx=0).stop 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 shutdown_connections 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14f80754e0 0x7f14f80779a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151815f000 0x7f1518152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15180cefb0 0x7f1518071a10 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 --2- 172.21.15.122:0/2879118923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15180ccaf0 0x7f15181f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 >> 172.21.15.122:0/2879118923 conn(0x7f151809aa10 msgr2=0x7f1518075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:29.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 shutdown_connections 2024-09-17T05:40:29.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.269+0000 7f151de00640 1 -- 172.21.15.122:0/2879118923 wait complete. 2024-09-17T05:40:29.284 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":606,"stamp":"2024-09-17T05:40:28.755744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592050,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1073428,"kb_used_data":15056,"kb_used_omap":12,"kb_used_meta":1058163,"kb_avail":753901292,"statfs":{"total":773094113280,"available":771994923008,"internally_reserved":0,"allocated":15417344,"data_stored":7446839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1083559478},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1682,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.608244"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163076+0000","last_change":"2024-09-17T05:40:24.163076+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163076+0000","last_undegraded":"2024-09-17T05:40:24.163076+0000","last_fullsized":"2024-09-17T05:40:24.163076+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155291+0000","last_change":"2024-09-17T05:40:24.155291+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155291+0000","last_undegraded":"2024-09-17T05:40:24.155291+0000","last_fullsized":"2024-09-17T05:40:24.155291+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182643+0000","last_change":"2024-09-17T05:40:25.183672+0000","last_active":"2024-09-17T05:40:26.182643+0000","last_peered":"2024-09-17T05:40:26.182643+0000","last_clean":"2024-09-17T05:40:26.182643+0000","last_became_active":"2024-09-17T05:40:25.183400+0000","last_became_peered":"2024-09-17T05:40:25.183400+0000","last_unstale":"2024-09-17T05:40:26.182643+0000","last_undegraded":"2024-09-17T05:40:26.182643+0000","last_fullsized":"2024-09-17T05:40:26.182643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:25.692977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.154825+0000","last_change":"2024-09-17T05:40:24.154825+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.154825+0000","last_undegraded":"2024-09-17T05:40:24.154825+0000","last_fullsized":"2024-09-17T05:40:24.154825+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155818+0000","last_change":"2024-09-17T05:40:24.155818+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155818+0000","last_undegraded":"2024-09-17T05:40:24.155818+0000","last_fullsized":"2024-09-17T05:40:24.155818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.162021+0000","last_change":"2024-09-17T05:40:24.162021+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.162021+0000","last_undegraded":"2024-09-17T05:40:24.162021+0000","last_fullsized":"2024-09-17T05:40:24.162021+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163566+0000","last_change":"2024-09-17T05:40:24.163566+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163566+0000","last_undegraded":"2024-09-17T05:40:24.163566+0000","last_fullsized":"2024-09-17T05:40:24.163566+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182207+0000","last_change":"2024-09-17T05:40:25.175534+0000","last_active":"2024-09-17T05:40:26.182207+0000","last_peered":"2024-09-17T05:40:26.182207+0000","last_clean":"2024-09-17T05:40:26.182207+0000","last_became_active":"2024-09-17T05:40:25.175183+0000","last_became_peered":"2024-09-17T05:40:25.175183+0000","last_unstale":"2024-09-17T05:40:26.182207+0000","last_undegraded":"2024-09-17T05:40:26.182207+0000","last_fullsized":"2024-09-17T05:40:26.182207+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:42.949912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.164790+0000","last_change":"2024-09-17T05:40:24.164790+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.164790+0000","last_undegraded":"2024-09-17T05:40:24.164790+0000","last_fullsized":"2024-09-17T05:40:24.164790+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.161537+0000","last_change":"2024-09-17T05:40:24.161537+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.161537+0000","last_undegraded":"2024-09-17T05:40:24.161537+0000","last_fullsized":"2024-09-17T05:40:24.161537+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163811+0000","last_change":"2024-09-17T05:40:24.163811+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163811+0000","last_undegraded":"2024-09-17T05:40:24.163811+0000","last_fullsized":"2024-09-17T05:40:24.163811+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.155652+0000","last_change":"2024-09-17T05:40:23.155652+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.155652+0000","last_undegraded":"2024-09-17T05:40:23.155652+0000","last_fullsized":"2024-09-17T05:40:23.155652+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183271+0000","last_change":"2024-09-17T05:40:25.179791+0000","last_active":"2024-09-17T05:40:26.183271+0000","last_peered":"2024-09-17T05:40:26.183271+0000","last_clean":"2024-09-17T05:40:26.183271+0000","last_became_active":"2024-09-17T05:40:25.179450+0000","last_became_peered":"2024-09-17T05:40:25.179450+0000","last_unstale":"2024-09-17T05:40:26.183271+0000","last_undegraded":"2024-09-17T05:40:26.183271+0000","last_fullsized":"2024-09-17T05:40:26.183271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.496633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.177062+0000","last_change":"2024-09-17T05:40:24.176345+0000","last_active":"2024-09-17T05:40:24.177062+0000","last_peered":"2024-09-17T05:40:24.177062+0000","last_clean":"2024-09-17T05:40:24.177062+0000","last_became_active":"2024-09-17T05:40:24.176106+0000","last_became_peered":"2024-09-17T05:40:24.176106+0000","last_unstale":"2024-09-17T05:40:24.177062+0000","last_undegraded":"2024-09-17T05:40:24.177062+0000","last_fullsized":"2024-09-17T05:40:24.177062+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:23.124357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.154383+0000","last_change":"2024-09-17T05:40:24.154383+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.154383+0000","last_undegraded":"2024-09-17T05:40:24.154383+0000","last_fullsized":"2024-09-17T05:40:24.154383+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154143+0000","last_change":"2024-09-17T05:40:23.154143+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154143+0000","last_undegraded":"2024-09-17T05:40:23.154143+0000","last_fullsized":"2024-09-17T05:40:23.154143+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.156957+0000","last_change":"2024-09-17T05:40:23.156957+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.156957+0000","last_undegraded":"2024-09-17T05:40:23.156957+0000","last_fullsized":"2024-09-17T05:40:23.156957+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153281+0000","last_change":"2024-09-17T05:40:23.153281+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153281+0000","last_undegraded":"2024-09-17T05:40:23.153281+0000","last_fullsized":"2024-09-17T05:40:23.153281+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.180564+0000","last_change":"2024-09-17T05:40:24.177481+0000","last_active":"2024-09-17T05:40:24.180564+0000","last_peered":"2024-09-17T05:40:24.180564+0000","last_clean":"2024-09-17T05:40:24.180564+0000","last_became_active":"2024-09-17T05:40:24.177186+0000","last_became_peered":"2024-09-17T05:40:24.177186+0000","last_unstale":"2024-09-17T05:40:24.180564+0000","last_undegraded":"2024-09-17T05:40:24.180564+0000","last_fullsized":"2024-09-17T05:40:24.180564+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:07.284037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.163654+0000","last_change":"2024-09-17T05:40:24.162381+0000","last_active":"2024-09-17T05:40:24.163654+0000","last_peered":"2024-09-17T05:40:24.163654+0000","last_clean":"2024-09-17T05:40:24.163654+0000","last_became_active":"2024-09-17T05:40:24.160458+0000","last_became_peered":"2024-09-17T05:40:24.160458+0000","last_unstale":"2024-09-17T05:40:24.163654+0000","last_undegraded":"2024-09-17T05:40:24.163654+0000","last_fullsized":"2024-09-17T05:40:24.163654+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:52.170344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154060+0000","last_change":"2024-09-17T05:40:23.154060+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154060+0000","last_undegraded":"2024-09-17T05:40:23.154060+0000","last_fullsized":"2024-09-17T05:40:23.154060+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.162838+0000","last_change":"2024-09-17T05:40:24.161649+0000","last_active":"2024-09-17T05:40:24.162838+0000","last_peered":"2024-09-17T05:40:24.162838+0000","last_clean":"2024-09-17T05:40:24.162838+0000","last_became_active":"2024-09-17T05:40:24.161393+0000","last_became_peered":"2024-09-17T05:40:24.161393+0000","last_unstale":"2024-09-17T05:40:24.162838+0000","last_undegraded":"2024-09-17T05:40:24.162838+0000","last_fullsized":"2024-09-17T05:40:24.162838+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:25.240554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.174326+0000","last_change":"2024-09-17T05:40:24.169609+0000","last_active":"2024-09-17T05:40:24.174326+0000","last_peered":"2024-09-17T05:40:24.174326+0000","last_clean":"2024-09-17T05:40:24.174326+0000","last_became_active":"2024-09-17T05:40:24.169317+0000","last_became_peered":"2024-09-17T05:40:24.169317+0000","last_unstale":"2024-09-17T05:40:24.174326+0000","last_undegraded":"2024-09-17T05:40:24.174326+0000","last_fullsized":"2024-09-17T05:40:24.174326+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:44.173740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183307+0000","last_change":"2024-09-17T05:40:25.184351+0000","last_active":"2024-09-17T05:40:26.183307+0000","last_peered":"2024-09-17T05:40:26.183307+0000","last_clean":"2024-09-17T05:40:26.183307+0000","last_became_active":"2024-09-17T05:40:25.183920+0000","last_became_peered":"2024-09-17T05:40:25.183920+0000","last_unstale":"2024-09-17T05:40:26.183307+0000","last_undegraded":"2024-09-17T05:40:26.183307+0000","last_fullsized":"2024-09-17T05:40:26.183307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:22.638149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182912+0000","last_change":"2024-09-17T05:40:24.169333+0000","last_active":"2024-09-17T05:40:26.182912+0000","last_peered":"2024-09-17T05:40:26.182912+0000","last_clean":"2024-09-17T05:40:26.182912+0000","last_became_active":"2024-09-17T05:40:24.168219+0000","last_became_peered":"2024-09-17T05:40:24.168219+0000","last_unstale":"2024-09-17T05:40:26.182912+0000","last_undegraded":"2024-09-17T05:40:26.182912+0000","last_fullsized":"2024-09-17T05:40:26.182912+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:58.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.156880+0000","last_change":"2024-09-17T05:40:23.156880+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.156880+0000","last_undegraded":"2024-09-17T05:40:23.156880+0000","last_fullsized":"2024-09-17T05:40:23.156880+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154467+0000","last_change":"2024-09-17T05:40:23.154467+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154467+0000","last_undegraded":"2024-09-17T05:40:23.154467+0000","last_fullsized":"2024-09-17T05:40:23.154467+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149174+0000","last_change":"2024-09-17T05:40:23.149174+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149174+0000","last_undegraded":"2024-09-17T05:40:23.149174+0000","last_fullsized":"2024-09-17T05:40:23.149174+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153142+0000","last_change":"2024-09-17T05:40:23.153142+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153142+0000","last_undegraded":"2024-09-17T05:40:23.153142+0000","last_fullsized":"2024-09-17T05:40:23.153142+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163295+0000","last_change":"2024-09-17T05:40:24.163295+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163295+0000","last_undegraded":"2024-09-17T05:40:24.163295+0000","last_fullsized":"2024-09-17T05:40:24.163295+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153655+0000","last_change":"2024-09-17T05:40:23.153655+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153655+0000","last_undegraded":"2024-09-17T05:40:23.153655+0000","last_fullsized":"2024-09-17T05:40:23.153655+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153751+0000","last_change":"2024-09-17T05:40:23.153751+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153751+0000","last_undegraded":"2024-09-17T05:40:23.153751+0000","last_fullsized":"2024-09-17T05:40:23.153751+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163036+0000","last_change":"2024-09-17T05:40:24.163036+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163036+0000","last_undegraded":"2024-09-17T05:40:24.163036+0000","last_fullsized":"2024-09-17T05:40:24.163036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183232+0000","last_change":"2024-09-17T05:40:24.173858+0000","last_active":"2024-09-17T05:40:26.183232+0000","last_peered":"2024-09-17T05:40:26.183232+0000","last_clean":"2024-09-17T05:40:26.183232+0000","last_became_active":"2024-09-17T05:40:24.173661+0000","last_became_peered":"2024-09-17T05:40:24.173661+0000","last_unstale":"2024-09-17T05:40:26.183232+0000","last_undegraded":"2024-09-17T05:40:26.183232+0000","last_fullsized":"2024-09-17T05:40:26.183232+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:21.200140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155393+0000","last_change":"2024-09-17T05:40:24.155393+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155393+0000","last_undegraded":"2024-09-17T05:40:24.155393+0000","last_fullsized":"2024-09-17T05:40:24.155393+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149919+0000","last_change":"2024-09-17T05:40:23.149919+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149919+0000","last_undegraded":"2024-09-17T05:40:23.149919+0000","last_fullsized":"2024-09-17T05:40:23.149919+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.175798+0000","last_change":"2024-09-17T05:40:24.174951+0000","last_active":"2024-09-17T05:40:24.175798+0000","last_peered":"2024-09-17T05:40:24.175798+0000","last_clean":"2024-09-17T05:40:24.175798+0000","last_became_active":"2024-09-17T05:40:24.174436+0000","last_became_peered":"2024-09-17T05:40:24.174436+0000","last_unstale":"2024-09-17T05:40:24.175798+0000","last_undegraded":"2024-09-17T05:40:24.175798+0000","last_fullsized":"2024-09-17T05:40:24.175798+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:29.137707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153136+0000","last_change":"2024-09-17T05:40:23.153136+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153136+0000","last_undegraded":"2024-09-17T05:40:23.153136+0000","last_fullsized":"2024-09-17T05:40:23.153136+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153092+0000","last_change":"2024-09-17T05:40:23.153092+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153092+0000","last_undegraded":"2024-09-17T05:40:23.153092+0000","last_fullsized":"2024-09-17T05:40:23.153092+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153455+0000","last_change":"2024-09-17T05:40:23.153455+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153455+0000","last_undegraded":"2024-09-17T05:40:23.153455+0000","last_fullsized":"2024-09-17T05:40:23.153455+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153170+0000","last_change":"2024-09-17T05:40:23.153170+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153170+0000","last_undegraded":"2024-09-17T05:40:23.153170+0000","last_fullsized":"2024-09-17T05:40:23.153170+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149222+0000","last_change":"2024-09-17T05:40:23.149222+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149222+0000","last_undegraded":"2024-09-17T05:40:23.149222+0000","last_fullsized":"2024-09-17T05:40:23.149222+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183134+0000","last_change":"2024-09-17T05:40:24.168536+0000","last_active":"2024-09-17T05:40:26.183134+0000","last_peered":"2024-09-17T05:40:26.183134+0000","last_clean":"2024-09-17T05:40:26.183134+0000","last_became_active":"2024-09-17T05:40:24.168121+0000","last_became_peered":"2024-09-17T05:40:24.168121+0000","last_unstale":"2024-09-17T05:40:26.183134+0000","last_undegraded":"2024-09-17T05:40:26.183134+0000","last_fullsized":"2024-09-17T05:40:26.183134+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:52.244624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183037+0000","last_change":"2024-09-17T05:40:24.167195+0000","last_active":"2024-09-17T05:40:26.183037+0000","last_peered":"2024-09-17T05:40:26.183037+0000","last_clean":"2024-09-17T05:40:26.183037+0000","last_became_active":"2024-09-17T05:40:24.166930+0000","last_became_peered":"2024-09-17T05:40:24.166930+0000","last_unstale":"2024-09-17T05:40:26.183037+0000","last_undegraded":"2024-09-17T05:40:26.183037+0000","last_fullsized":"2024-09-17T05:40:26.183037+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:47.316470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183088+0000","last_change":"2024-09-17T05:40:25.177997+0000","last_active":"2024-09-17T05:40:26.183088+0000","last_peered":"2024-09-17T05:40:26.183088+0000","last_clean":"2024-09-17T05:40:26.183088+0000","last_became_active":"2024-09-17T05:40:25.177737+0000","last_became_peered":"2024-09-17T05:40:25.177737+0000","last_unstale":"2024-09-17T05:40:26.183088+0000","last_undegraded":"2024-09-17T05:40:26.183088+0000","last_fullsized":"2024-09-17T05:40:26.183088+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.829551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149416+0000","last_change":"2024-09-17T05:40:23.149416+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149416+0000","last_undegraded":"2024-09-17T05:40:23.149416+0000","last_fullsized":"2024-09-17T05:40:23.149416+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.155191+0000","last_change":"2024-09-17T05:40:23.155191+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.155191+0000","last_undegraded":"2024-09-17T05:40:23.155191+0000","last_fullsized":"2024-09-17T05:40:23.155191+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154471+0000","last_change":"2024-09-17T05:40:23.154471+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154471+0000","last_undegraded":"2024-09-17T05:40:23.154471+0000","last_fullsized":"2024-09-17T05:40:23.154471+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.175381+0000","last_change":"2024-09-17T05:40:24.174394+0000","last_active":"2024-09-17T05:40:24.175381+0000","last_peered":"2024-09-17T05:40:24.175381+0000","last_clean":"2024-09-17T05:40:24.175381+0000","last_became_active":"2024-09-17T05:40:24.174122+0000","last_became_peered":"2024-09-17T05:40:24.174122+0000","last_unstale":"2024-09-17T05:40:24.175381+0000","last_undegraded":"2024-09-17T05:40:24.175381+0000","last_fullsized":"2024-09-17T05:40:24.175381+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:58.370544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183130+0000","last_change":"2024-09-17T05:40:25.185149+0000","last_active":"2024-09-17T05:40:26.183130+0000","last_peered":"2024-09-17T05:40:26.183130+0000","last_clean":"2024-09-17T05:40:26.183130+0000","last_became_active":"2024-09-17T05:40:25.184781+0000","last_became_peered":"2024-09-17T05:40:25.184781+0000","last_unstale":"2024-09-17T05:40:26.183130+0000","last_undegraded":"2024-09-17T05:40:26.183130+0000","last_fullsized":"2024-09-17T05:40:26.183130+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.885218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154548+0000","last_change":"2024-09-17T05:40:23.154548+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154548+0000","last_undegraded":"2024-09-17T05:40:23.154548+0000","last_fullsized":"2024-09-17T05:40:23.154548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":341,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.165229+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:40:24.165229+0000","last_peered":"2024-09-17T05:40:24.165229+0000","last_clean":"2024-09-17T05:40:24.165229+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:40:24.165229+0000","last_undegraded":"2024-09-17T05:40:24.165229+0000","last_fullsized":"2024-09-17T05:40:24.165229+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":3},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":1682,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509629,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127252,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244588,"statfs":{"total":96636764160,"available":96506458112,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509629,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137000,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234840,"statfs":{"total":96636764160,"available":96496476160,"internally_reserved":0,"allocated":1781760,"data_stored":786151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131416,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240424,"statfs":{"total":96636764160,"available":96502194176,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509628,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131412,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240428,"statfs":{"total":96636764160,"available":96502198272,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131420,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240420,"statfs":{"total":96636764160,"available":96502190080,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509630,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145880,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225960,"statfs":{"total":96636764160,"available":96487383040,"internally_reserved":0,"allocated":2363392,"data_stored":1370160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509628,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149468,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222372,"statfs":{"total":96636764160,"available":96483708928,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":151255499},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119580,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252260,"statfs":{"total":96636764160,"available":96514314240,"internally_reserved":0,"allocated":2424832,"data_stored":1391568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:40:29.288 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:40:29.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 -- 172.21.15.122:0/2231797560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff0004810 msgr2=0x7faff0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 --2- 172.21.15.122:0/2231797560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff0004810 0x7faff0004c90 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7faff4009b60 tx=0x7faff401d270 comp rx=0 tx=0).stop 2024-09-17T05:40:29.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 -- 172.21.15.122:0/2231797560 shutdown_connections 2024-09-17T05:40:29.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 --2- 172.21.15.122:0/2231797560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff0004810 0x7faff0004c90 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 -- 172.21.15.122:0/2231797560 >> 172.21.15.122:0/2231797560 conn(0x7fb00c0d1dc0 msgr2=0x7fb00c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:29.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 -- 172.21.15.122:0/2231797560 shutdown_connections 2024-09-17T05:40:29.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 -- 172.21.15.122:0/2231797560 wait complete. 2024-09-17T05:40:29.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 Processor -- start 2024-09-17T05:40:29.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.477+0000 7fb010c00640 1 -- start start 2024-09-17T05:40:29.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faff0004810 0x7fb00c1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb00c15db90 0x7fb00c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb00c0b6940 0x7fb00c0b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c15c490 con 0x7fb00c15db90 2024-09-17T05:40:29.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c0d2af0 con 0x7fb00c0b6940 2024-09-17T05:40:29.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb00c0ce670 con 0x7faff0004810 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb00c15db90 0x7fb00c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb00c15db90 0x7fb00c0cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40016/0 (socket says 172.21.15.122:40016) 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 -- 172.21.15.122:0/3955312231 learned_addr learned my addr 172.21.15.122:0/3955312231 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb009e00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faff0004810 0x7fb00c1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb00a800640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb00c0b6940 0x7fb00c0b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 -- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faff0004810 msgr2=0x7fb00c1522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faff0004810 0x7fb00c1522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 -- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb00c0b6940 msgr2=0x7fb00c0b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb00c0b6940 0x7fb00c0b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 -- 172.21.15.122:0/3955312231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c15c960 con 0x7fb00c15db90 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb003e00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb00c15db90 0x7fb00c0cba20 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7faff401d0a0 tx=0x7faff40056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb002000640 1 -- 172.21.15.122:0/3955312231 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff40b1040 con 0x7fb00c15db90 2024-09-17T05:40:29.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb002000640 1 -- 172.21.15.122:0/3955312231 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faff40bc070 con 0x7fb00c15db90 2024-09-17T05:40:29.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c15cc40 con 0x7fb00c15db90 2024-09-17T05:40:29.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb002000640 1 -- 172.21.15.122:0/3955312231 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff40acb00 con 0x7fb00c15db90 2024-09-17T05:40:29.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c15d200 con 0x7fb00c15db90 2024-09-17T05:40:29.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c0b7300 con 0x7fb00c15db90 2024-09-17T05:40:29.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.481+0000 7fb002000640 1 -- 172.21.15.122:0/3955312231 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7faff40acca0 con 0x7fb00c15db90 2024-09-17T05:40:29.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.485+0000 7fb002000640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fafe8075510 0x7fafe80779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.485+0000 7fb002000640 1 -- 172.21.15.122:0/3955312231 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7faff412e300 con 0x7fb00c15db90 2024-09-17T05:40:29.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.485+0000 7fb009e00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fafe8075510 0x7fafe80779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.485+0000 7fb009e00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fafe8075510 0x7fafe80779d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0c9180 tx=0x7faffc00abe0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:29.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.489+0000 7fb002000640 1 -- 172.21.15.122:0/3955312231 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7faff4136030 con 0x7fb00c15db90 2024-09-17T05:40:29.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.653+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb00c15d570 con 0x7fafe8075510 2024-09-17T05:40:29.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.657+0000 7fb002000640 1 -- 172.21.15.122:0/3955312231 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243108 (secure 0 0 0) 0x7fb00c15d570 con 0x7fafe8075510 2024-09-17T05:40:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:29.659 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fafe8075510 msgr2=0x7fafe80779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fafe8075510 0x7fafe80779d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0c9180 tx=0x7faffc00abe0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb00c15db90 msgr2=0x7fb00c0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb00c15db90 0x7fb00c0cba20 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7faff401d0a0 tx=0x7faff40056c0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 shutdown_connections 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fafe8075510 0x7fafe80779d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb00c0b6940 0x7fb00c0b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb00c15db90 0x7fb00c0cba20 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 --2- 172.21.15.122:0/3955312231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faff0004810 0x7fb00c1522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 >> 172.21.15.122:0/3955312231 conn(0x7fb00c0d1dc0 msgr2=0x7fb00c159940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:29.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 shutdown_connections 2024-09-17T05:40:29.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.661+0000 7fb010c00640 1 -- 172.21.15.122:0/3955312231 wait complete. 2024-09-17T05:40:29.678 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":606,"stamp":"2024-09-17T05:40:28.755744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592050,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1073428,"kb_used_data":15056,"kb_used_omap":12,"kb_used_meta":1058163,"kb_avail":753901292,"statfs":{"total":773094113280,"available":771994923008,"internally_reserved":0,"allocated":15417344,"data_stored":7446839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1083559478},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1682,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.608244"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163076+0000","last_change":"2024-09-17T05:40:24.163076+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163076+0000","last_undegraded":"2024-09-17T05:40:24.163076+0000","last_fullsized":"2024-09-17T05:40:24.163076+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155291+0000","last_change":"2024-09-17T05:40:24.155291+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155291+0000","last_undegraded":"2024-09-17T05:40:24.155291+0000","last_fullsized":"2024-09-17T05:40:24.155291+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182643+0000","last_change":"2024-09-17T05:40:25.183672+0000","last_active":"2024-09-17T05:40:26.182643+0000","last_peered":"2024-09-17T05:40:26.182643+0000","last_clean":"2024-09-17T05:40:26.182643+0000","last_became_active":"2024-09-17T05:40:25.183400+0000","last_became_peered":"2024-09-17T05:40:25.183400+0000","last_unstale":"2024-09-17T05:40:26.182643+0000","last_undegraded":"2024-09-17T05:40:26.182643+0000","last_fullsized":"2024-09-17T05:40:26.182643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:25.692977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.154825+0000","last_change":"2024-09-17T05:40:24.154825+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.154825+0000","last_undegraded":"2024-09-17T05:40:24.154825+0000","last_fullsized":"2024-09-17T05:40:24.154825+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155818+0000","last_change":"2024-09-17T05:40:24.155818+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155818+0000","last_undegraded":"2024-09-17T05:40:24.155818+0000","last_fullsized":"2024-09-17T05:40:24.155818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.162021+0000","last_change":"2024-09-17T05:40:24.162021+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.162021+0000","last_undegraded":"2024-09-17T05:40:24.162021+0000","last_fullsized":"2024-09-17T05:40:24.162021+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163566+0000","last_change":"2024-09-17T05:40:24.163566+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163566+0000","last_undegraded":"2024-09-17T05:40:24.163566+0000","last_fullsized":"2024-09-17T05:40:24.163566+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182207+0000","last_change":"2024-09-17T05:40:25.175534+0000","last_active":"2024-09-17T05:40:26.182207+0000","last_peered":"2024-09-17T05:40:26.182207+0000","last_clean":"2024-09-17T05:40:26.182207+0000","last_became_active":"2024-09-17T05:40:25.175183+0000","last_became_peered":"2024-09-17T05:40:25.175183+0000","last_unstale":"2024-09-17T05:40:26.182207+0000","last_undegraded":"2024-09-17T05:40:26.182207+0000","last_fullsized":"2024-09-17T05:40:26.182207+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:42.949912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.164790+0000","last_change":"2024-09-17T05:40:24.164790+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.164790+0000","last_undegraded":"2024-09-17T05:40:24.164790+0000","last_fullsized":"2024-09-17T05:40:24.164790+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.161537+0000","last_change":"2024-09-17T05:40:24.161537+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.161537+0000","last_undegraded":"2024-09-17T05:40:24.161537+0000","last_fullsized":"2024-09-17T05:40:24.161537+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163811+0000","last_change":"2024-09-17T05:40:24.163811+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163811+0000","last_undegraded":"2024-09-17T05:40:24.163811+0000","last_fullsized":"2024-09-17T05:40:24.163811+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.155652+0000","last_change":"2024-09-17T05:40:23.155652+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.155652+0000","last_undegraded":"2024-09-17T05:40:23.155652+0000","last_fullsized":"2024-09-17T05:40:23.155652+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183271+0000","last_change":"2024-09-17T05:40:25.179791+0000","last_active":"2024-09-17T05:40:26.183271+0000","last_peered":"2024-09-17T05:40:26.183271+0000","last_clean":"2024-09-17T05:40:26.183271+0000","last_became_active":"2024-09-17T05:40:25.179450+0000","last_became_peered":"2024-09-17T05:40:25.179450+0000","last_unstale":"2024-09-17T05:40:26.183271+0000","last_undegraded":"2024-09-17T05:40:26.183271+0000","last_fullsized":"2024-09-17T05:40:26.183271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.496633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.177062+0000","last_change":"2024-09-17T05:40:24.176345+0000","last_active":"2024-09-17T05:40:24.177062+0000","last_peered":"2024-09-17T05:40:24.177062+0000","last_clean":"2024-09-17T05:40:24.177062+0000","last_became_active":"2024-09-17T05:40:24.176106+0000","last_became_peered":"2024-09-17T05:40:24.176106+0000","last_unstale":"2024-09-17T05:40:24.177062+0000","last_undegraded":"2024-09-17T05:40:24.177062+0000","last_fullsized":"2024-09-17T05:40:24.177062+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:23.124357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.154383+0000","last_change":"2024-09-17T05:40:24.154383+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.154383+0000","last_undegraded":"2024-09-17T05:40:24.154383+0000","last_fullsized":"2024-09-17T05:40:24.154383+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154143+0000","last_change":"2024-09-17T05:40:23.154143+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154143+0000","last_undegraded":"2024-09-17T05:40:23.154143+0000","last_fullsized":"2024-09-17T05:40:23.154143+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.156957+0000","last_change":"2024-09-17T05:40:23.156957+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.156957+0000","last_undegraded":"2024-09-17T05:40:23.156957+0000","last_fullsized":"2024-09-17T05:40:23.156957+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153281+0000","last_change":"2024-09-17T05:40:23.153281+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153281+0000","last_undegraded":"2024-09-17T05:40:23.153281+0000","last_fullsized":"2024-09-17T05:40:23.153281+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.180564+0000","last_change":"2024-09-17T05:40:24.177481+0000","last_active":"2024-09-17T05:40:24.180564+0000","last_peered":"2024-09-17T05:40:24.180564+0000","last_clean":"2024-09-17T05:40:24.180564+0000","last_became_active":"2024-09-17T05:40:24.177186+0000","last_became_peered":"2024-09-17T05:40:24.177186+0000","last_unstale":"2024-09-17T05:40:24.180564+0000","last_undegraded":"2024-09-17T05:40:24.180564+0000","last_fullsized":"2024-09-17T05:40:24.180564+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:07.284037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.163654+0000","last_change":"2024-09-17T05:40:24.162381+0000","last_active":"2024-09-17T05:40:24.163654+0000","last_peered":"2024-09-17T05:40:24.163654+0000","last_clean":"2024-09-17T05:40:24.163654+0000","last_became_active":"2024-09-17T05:40:24.160458+0000","last_became_peered":"2024-09-17T05:40:24.160458+0000","last_unstale":"2024-09-17T05:40:24.163654+0000","last_undegraded":"2024-09-17T05:40:24.163654+0000","last_fullsized":"2024-09-17T05:40:24.163654+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:52.170344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154060+0000","last_change":"2024-09-17T05:40:23.154060+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154060+0000","last_undegraded":"2024-09-17T05:40:23.154060+0000","last_fullsized":"2024-09-17T05:40:23.154060+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.162838+0000","last_change":"2024-09-17T05:40:24.161649+0000","last_active":"2024-09-17T05:40:24.162838+0000","last_peered":"2024-09-17T05:40:24.162838+0000","last_clean":"2024-09-17T05:40:24.162838+0000","last_became_active":"2024-09-17T05:40:24.161393+0000","last_became_peered":"2024-09-17T05:40:24.161393+0000","last_unstale":"2024-09-17T05:40:24.162838+0000","last_undegraded":"2024-09-17T05:40:24.162838+0000","last_fullsized":"2024-09-17T05:40:24.162838+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:25.240554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.174326+0000","last_change":"2024-09-17T05:40:24.169609+0000","last_active":"2024-09-17T05:40:24.174326+0000","last_peered":"2024-09-17T05:40:24.174326+0000","last_clean":"2024-09-17T05:40:24.174326+0000","last_became_active":"2024-09-17T05:40:24.169317+0000","last_became_peered":"2024-09-17T05:40:24.169317+0000","last_unstale":"2024-09-17T05:40:24.174326+0000","last_undegraded":"2024-09-17T05:40:24.174326+0000","last_fullsized":"2024-09-17T05:40:24.174326+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:44.173740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183307+0000","last_change":"2024-09-17T05:40:25.184351+0000","last_active":"2024-09-17T05:40:26.183307+0000","last_peered":"2024-09-17T05:40:26.183307+0000","last_clean":"2024-09-17T05:40:26.183307+0000","last_became_active":"2024-09-17T05:40:25.183920+0000","last_became_peered":"2024-09-17T05:40:25.183920+0000","last_unstale":"2024-09-17T05:40:26.183307+0000","last_undegraded":"2024-09-17T05:40:26.183307+0000","last_fullsized":"2024-09-17T05:40:26.183307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:22.638149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182912+0000","last_change":"2024-09-17T05:40:24.169333+0000","last_active":"2024-09-17T05:40:26.182912+0000","last_peered":"2024-09-17T05:40:26.182912+0000","last_clean":"2024-09-17T05:40:26.182912+0000","last_became_active":"2024-09-17T05:40:24.168219+0000","last_became_peered":"2024-09-17T05:40:24.168219+0000","last_unstale":"2024-09-17T05:40:26.182912+0000","last_undegraded":"2024-09-17T05:40:26.182912+0000","last_fullsized":"2024-09-17T05:40:26.182912+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:58.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.156880+0000","last_change":"2024-09-17T05:40:23.156880+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.156880+0000","last_undegraded":"2024-09-17T05:40:23.156880+0000","last_fullsized":"2024-09-17T05:40:23.156880+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154467+0000","last_change":"2024-09-17T05:40:23.154467+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154467+0000","last_undegraded":"2024-09-17T05:40:23.154467+0000","last_fullsized":"2024-09-17T05:40:23.154467+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149174+0000","last_change":"2024-09-17T05:40:23.149174+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149174+0000","last_undegraded":"2024-09-17T05:40:23.149174+0000","last_fullsized":"2024-09-17T05:40:23.149174+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153142+0000","last_change":"2024-09-17T05:40:23.153142+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153142+0000","last_undegraded":"2024-09-17T05:40:23.153142+0000","last_fullsized":"2024-09-17T05:40:23.153142+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163295+0000","last_change":"2024-09-17T05:40:24.163295+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163295+0000","last_undegraded":"2024-09-17T05:40:24.163295+0000","last_fullsized":"2024-09-17T05:40:24.163295+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153655+0000","last_change":"2024-09-17T05:40:23.153655+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153655+0000","last_undegraded":"2024-09-17T05:40:23.153655+0000","last_fullsized":"2024-09-17T05:40:23.153655+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153751+0000","last_change":"2024-09-17T05:40:23.153751+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153751+0000","last_undegraded":"2024-09-17T05:40:23.153751+0000","last_fullsized":"2024-09-17T05:40:23.153751+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163036+0000","last_change":"2024-09-17T05:40:24.163036+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163036+0000","last_undegraded":"2024-09-17T05:40:24.163036+0000","last_fullsized":"2024-09-17T05:40:24.163036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183232+0000","last_change":"2024-09-17T05:40:24.173858+0000","last_active":"2024-09-17T05:40:26.183232+0000","last_peered":"2024-09-17T05:40:26.183232+0000","last_clean":"2024-09-17T05:40:26.183232+0000","last_became_active":"2024-09-17T05:40:24.173661+0000","last_became_peered":"2024-09-17T05:40:24.173661+0000","last_unstale":"2024-09-17T05:40:26.183232+0000","last_undegraded":"2024-09-17T05:40:26.183232+0000","last_fullsized":"2024-09-17T05:40:26.183232+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:21.200140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155393+0000","last_change":"2024-09-17T05:40:24.155393+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155393+0000","last_undegraded":"2024-09-17T05:40:24.155393+0000","last_fullsized":"2024-09-17T05:40:24.155393+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149919+0000","last_change":"2024-09-17T05:40:23.149919+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149919+0000","last_undegraded":"2024-09-17T05:40:23.149919+0000","last_fullsized":"2024-09-17T05:40:23.149919+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.175798+0000","last_change":"2024-09-17T05:40:24.174951+0000","last_active":"2024-09-17T05:40:24.175798+0000","last_peered":"2024-09-17T05:40:24.175798+0000","last_clean":"2024-09-17T05:40:24.175798+0000","last_became_active":"2024-09-17T05:40:24.174436+0000","last_became_peered":"2024-09-17T05:40:24.174436+0000","last_unstale":"2024-09-17T05:40:24.175798+0000","last_undegraded":"2024-09-17T05:40:24.175798+0000","last_fullsized":"2024-09-17T05:40:24.175798+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:29.137707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153136+0000","last_change":"2024-09-17T05:40:23.153136+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153136+0000","last_undegraded":"2024-09-17T05:40:23.153136+0000","last_fullsized":"2024-09-17T05:40:23.153136+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153092+0000","last_change":"2024-09-17T05:40:23.153092+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153092+0000","last_undegraded":"2024-09-17T05:40:23.153092+0000","last_fullsized":"2024-09-17T05:40:23.153092+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153455+0000","last_change":"2024-09-17T05:40:23.153455+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153455+0000","last_undegraded":"2024-09-17T05:40:23.153455+0000","last_fullsized":"2024-09-17T05:40:23.153455+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153170+0000","last_change":"2024-09-17T05:40:23.153170+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153170+0000","last_undegraded":"2024-09-17T05:40:23.153170+0000","last_fullsized":"2024-09-17T05:40:23.153170+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149222+0000","last_change":"2024-09-17T05:40:23.149222+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149222+0000","last_undegraded":"2024-09-17T05:40:23.149222+0000","last_fullsized":"2024-09-17T05:40:23.149222+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183134+0000","last_change":"2024-09-17T05:40:24.168536+0000","last_active":"2024-09-17T05:40:26.183134+0000","last_peered":"2024-09-17T05:40:26.183134+0000","last_clean":"2024-09-17T05:40:26.183134+0000","last_became_active":"2024-09-17T05:40:24.168121+0000","last_became_peered":"2024-09-17T05:40:24.168121+0000","last_unstale":"2024-09-17T05:40:26.183134+0000","last_undegraded":"2024-09-17T05:40:26.183134+0000","last_fullsized":"2024-09-17T05:40:26.183134+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:52.244624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183037+0000","last_change":"2024-09-17T05:40:24.167195+0000","last_active":"2024-09-17T05:40:26.183037+0000","last_peered":"2024-09-17T05:40:26.183037+0000","last_clean":"2024-09-17T05:40:26.183037+0000","last_became_active":"2024-09-17T05:40:24.166930+0000","last_became_peered":"2024-09-17T05:40:24.166930+0000","last_unstale":"2024-09-17T05:40:26.183037+0000","last_undegraded":"2024-09-17T05:40:26.183037+0000","last_fullsized":"2024-09-17T05:40:26.183037+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:47.316470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183088+0000","last_change":"2024-09-17T05:40:25.177997+0000","last_active":"2024-09-17T05:40:26.183088+0000","last_peered":"2024-09-17T05:40:26.183088+0000","last_clean":"2024-09-17T05:40:26.183088+0000","last_became_active":"2024-09-17T05:40:25.177737+0000","last_became_peered":"2024-09-17T05:40:25.177737+0000","last_unstale":"2024-09-17T05:40:26.183088+0000","last_undegraded":"2024-09-17T05:40:26.183088+0000","last_fullsized":"2024-09-17T05:40:26.183088+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.829551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149416+0000","last_change":"2024-09-17T05:40:23.149416+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149416+0000","last_undegraded":"2024-09-17T05:40:23.149416+0000","last_fullsized":"2024-09-17T05:40:23.149416+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.155191+0000","last_change":"2024-09-17T05:40:23.155191+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.155191+0000","last_undegraded":"2024-09-17T05:40:23.155191+0000","last_fullsized":"2024-09-17T05:40:23.155191+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154471+0000","last_change":"2024-09-17T05:40:23.154471+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154471+0000","last_undegraded":"2024-09-17T05:40:23.154471+0000","last_fullsized":"2024-09-17T05:40:23.154471+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.175381+0000","last_change":"2024-09-17T05:40:24.174394+0000","last_active":"2024-09-17T05:40:24.175381+0000","last_peered":"2024-09-17T05:40:24.175381+0000","last_clean":"2024-09-17T05:40:24.175381+0000","last_became_active":"2024-09-17T05:40:24.174122+0000","last_became_peered":"2024-09-17T05:40:24.174122+0000","last_unstale":"2024-09-17T05:40:24.175381+0000","last_undegraded":"2024-09-17T05:40:24.175381+0000","last_fullsized":"2024-09-17T05:40:24.175381+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:58.370544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183130+0000","last_change":"2024-09-17T05:40:25.185149+0000","last_active":"2024-09-17T05:40:26.183130+0000","last_peered":"2024-09-17T05:40:26.183130+0000","last_clean":"2024-09-17T05:40:26.183130+0000","last_became_active":"2024-09-17T05:40:25.184781+0000","last_became_peered":"2024-09-17T05:40:25.184781+0000","last_unstale":"2024-09-17T05:40:26.183130+0000","last_undegraded":"2024-09-17T05:40:26.183130+0000","last_fullsized":"2024-09-17T05:40:26.183130+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.885218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154548+0000","last_change":"2024-09-17T05:40:23.154548+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154548+0000","last_undegraded":"2024-09-17T05:40:23.154548+0000","last_fullsized":"2024-09-17T05:40:23.154548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":341,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.165229+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:40:24.165229+0000","last_peered":"2024-09-17T05:40:24.165229+0000","last_clean":"2024-09-17T05:40:24.165229+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:40:24.165229+0000","last_undegraded":"2024-09-17T05:40:24.165229+0000","last_fullsized":"2024-09-17T05:40:24.165229+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":3},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":1682,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509629,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127252,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244588,"statfs":{"total":96636764160,"available":96506458112,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509629,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137000,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234840,"statfs":{"total":96636764160,"available":96496476160,"internally_reserved":0,"allocated":1781760,"data_stored":786151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131416,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240424,"statfs":{"total":96636764160,"available":96502194176,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509628,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131412,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240428,"statfs":{"total":96636764160,"available":96502198272,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131420,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240420,"statfs":{"total":96636764160,"available":96502190080,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509630,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145880,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225960,"statfs":{"total":96636764160,"available":96487383040,"internally_reserved":0,"allocated":2363392,"data_stored":1370160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509628,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149468,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222372,"statfs":{"total":96636764160,"available":96483708928,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":151255499},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119580,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252260,"statfs":{"total":96636764160,"available":96514314240,"internally_reserved":0,"allocated":2424832,"data_stored":1391568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:40:29.682 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:40:29.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.861+0000 7f6019600640 1 -- 172.21.15.122:0/1708026140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60140b4cf0 msgr2=0x7f60140b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.861+0000 7f6019600640 1 --2- 172.21.15.122:0/1708026140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60140b4cf0 0x7f60140b3130 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f600c009c80 tx=0x7f600c01cd10 comp rx=0 tx=0).stop 2024-09-17T05:40:29.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.861+0000 7f6019600640 1 -- 172.21.15.122:0/1708026140 shutdown_connections 2024-09-17T05:40:29.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.861+0000 7f6019600640 1 --2- 172.21.15.122:0/1708026140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60140b4cf0 0x7f60140b3130 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.861+0000 7f6019600640 1 --2- 172.21.15.122:0/1708026140 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f601415e260 0x7f60140b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.861+0000 7f6019600640 1 -- 172.21.15.122:0/1708026140 >> 172.21.15.122:0/1708026140 conn(0x7f60140b18f0 msgr2=0x7f60140b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:29.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.861+0000 7f6019600640 1 -- 172.21.15.122:0/1708026140 shutdown_connections 2024-09-17T05:40:29.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.861+0000 7f6019600640 1 -- 172.21.15.122:0/1708026140 wait complete. 2024-09-17T05:40:29.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 Processor -- start 2024-09-17T05:40:29.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 -- start start 2024-09-17T05:40:29.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60140b4cf0 0x7f60141f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f601415e260 0x7f6014159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60141460e0 0x7f6014140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60141683f0 con 0x7f60141460e0 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6014168270 con 0x7f60140b4cf0 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6014168570 con 0x7f601415e260 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60141460e0 0x7f6014140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60141460e0 0x7f6014140040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40032/0 (socket says 172.21.15.122:40032) 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 -- 172.21.15.122:0/3313570878 learned_addr learned my addr 172.21.15.122:0/3313570878 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f600a000640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f601415e260 0x7f6014159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6012a00640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60140b4cf0 0x7f60141f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 -- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f601415e260 msgr2=0x7f6014159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f601415e260 0x7f6014159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 -- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60140b4cf0 msgr2=0x7f60141f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60140b4cf0 0x7f60141f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:29.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 -- 172.21.15.122:0/3313570878 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601415b820 con 0x7f60141460e0 2024-09-17T05:40:29.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6012a00640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60140b4cf0 0x7f60141f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:29.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f600a000640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f601415e260 0x7f6014159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:29.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6013400640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60141460e0 0x7f6014140040 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f600c009bd0 tx=0x7f600c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:29.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f600b400640 1 -- 172.21.15.122:0/3313570878 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0bb070 con 0x7f60141460e0 2024-09-17T05:40:29.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f600b400640 1 -- 172.21.15.122:0/3313570878 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f600c004e00 con 0x7f60141460e0 2024-09-17T05:40:29.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f600b400640 1 -- 172.21.15.122:0/3313570878 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0b2040 con 0x7f60141460e0 2024-09-17T05:40:29.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601415a4d0 con 0x7f60141460e0 2024-09-17T05:40:29.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.865+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60141f9ec0 con 0x7f60141460e0 2024-09-17T05:40:29.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.869+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6014069410 con 0x7f60141460e0 2024-09-17T05:40:29.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.869+0000 7f600b400640 1 -- 172.21.15.122:0/3313570878 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f600c004230 con 0x7f60141460e0 2024-09-17T05:40:29.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.869+0000 7f600b400640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ff00754e0 0x7f5ff00779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:29.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.869+0000 7f600b400640 1 -- 172.21.15.122:0/3313570878 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f600c12d670 con 0x7f60141460e0 2024-09-17T05:40:29.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.869+0000 7f6012a00640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ff00754e0 0x7f5ff00779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:29.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.873+0000 7f6012a00640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ff00754e0 0x7f5ff00779a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6004005fd0 tx=0x7f6004005dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:29.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:29.873+0000 7f600b400640 1 -- 172.21.15.122:0/3313570878 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f600c0c4020 con 0x7f60141460e0 2024-09-17T05:40:30.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.037+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60140ae070 con 0x7f5ff00754e0 2024-09-17T05:40:30.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.041+0000 7f600b400640 1 -- 172.21.15.122:0/3313570878 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243108 (secure 0 0 0) 0x7f60140ae070 con 0x7f5ff00754e0 2024-09-17T05:40:30.041 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:30.043 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:40:30.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ff00754e0 msgr2=0x7f5ff00779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ff00754e0 0x7f5ff00779a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6004005fd0 tx=0x7f6004005dc0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60141460e0 msgr2=0x7f6014140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60141460e0 0x7f6014140040 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f600c009bd0 tx=0x7f600c003f60 comp rx=0 tx=0).stop 2024-09-17T05:40:30.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 shutdown_connections 2024-09-17T05:40:30.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5ff00754e0 0x7f5ff00779a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60141460e0 0x7f6014140040 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f601415e260 0x7f6014159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 --2- 172.21.15.122:0/3313570878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60140b4cf0 0x7f60141f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 >> 172.21.15.122:0/3313570878 conn(0x7f60140b18f0 msgr2=0x7f60141546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:30.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 shutdown_connections 2024-09-17T05:40:30.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.045+0000 7f6019600640 1 -- 172.21.15.122:0/3313570878 wait complete. 2024-09-17T05:40:30.061 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":606,"stamp":"2024-09-17T05:40:28.755744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592050,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1073428,"kb_used_data":15056,"kb_used_omap":12,"kb_used_meta":1058163,"kb_avail":753901292,"statfs":{"total":773094113280,"available":771994923008,"internally_reserved":0,"allocated":15417344,"data_stored":7446839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1083559478},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1682,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.608244"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163076+0000","last_change":"2024-09-17T05:40:24.163076+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163076+0000","last_undegraded":"2024-09-17T05:40:24.163076+0000","last_fullsized":"2024-09-17T05:40:24.163076+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155291+0000","last_change":"2024-09-17T05:40:24.155291+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155291+0000","last_undegraded":"2024-09-17T05:40:24.155291+0000","last_fullsized":"2024-09-17T05:40:24.155291+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182643+0000","last_change":"2024-09-17T05:40:25.183672+0000","last_active":"2024-09-17T05:40:26.182643+0000","last_peered":"2024-09-17T05:40:26.182643+0000","last_clean":"2024-09-17T05:40:26.182643+0000","last_became_active":"2024-09-17T05:40:25.183400+0000","last_became_peered":"2024-09-17T05:40:25.183400+0000","last_unstale":"2024-09-17T05:40:26.182643+0000","last_undegraded":"2024-09-17T05:40:26.182643+0000","last_fullsized":"2024-09-17T05:40:26.182643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:25.692977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.154825+0000","last_change":"2024-09-17T05:40:24.154825+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.154825+0000","last_undegraded":"2024-09-17T05:40:24.154825+0000","last_fullsized":"2024-09-17T05:40:24.154825+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155818+0000","last_change":"2024-09-17T05:40:24.155818+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155818+0000","last_undegraded":"2024-09-17T05:40:24.155818+0000","last_fullsized":"2024-09-17T05:40:24.155818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.162021+0000","last_change":"2024-09-17T05:40:24.162021+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.162021+0000","last_undegraded":"2024-09-17T05:40:24.162021+0000","last_fullsized":"2024-09-17T05:40:24.162021+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163566+0000","last_change":"2024-09-17T05:40:24.163566+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163566+0000","last_undegraded":"2024-09-17T05:40:24.163566+0000","last_fullsized":"2024-09-17T05:40:24.163566+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182207+0000","last_change":"2024-09-17T05:40:25.175534+0000","last_active":"2024-09-17T05:40:26.182207+0000","last_peered":"2024-09-17T05:40:26.182207+0000","last_clean":"2024-09-17T05:40:26.182207+0000","last_became_active":"2024-09-17T05:40:25.175183+0000","last_became_peered":"2024-09-17T05:40:25.175183+0000","last_unstale":"2024-09-17T05:40:26.182207+0000","last_undegraded":"2024-09-17T05:40:26.182207+0000","last_fullsized":"2024-09-17T05:40:26.182207+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:42.949912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.164790+0000","last_change":"2024-09-17T05:40:24.164790+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.164790+0000","last_undegraded":"2024-09-17T05:40:24.164790+0000","last_fullsized":"2024-09-17T05:40:24.164790+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.161537+0000","last_change":"2024-09-17T05:40:24.161537+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.161537+0000","last_undegraded":"2024-09-17T05:40:24.161537+0000","last_fullsized":"2024-09-17T05:40:24.161537+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163811+0000","last_change":"2024-09-17T05:40:24.163811+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163811+0000","last_undegraded":"2024-09-17T05:40:24.163811+0000","last_fullsized":"2024-09-17T05:40:24.163811+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.155652+0000","last_change":"2024-09-17T05:40:23.155652+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.155652+0000","last_undegraded":"2024-09-17T05:40:23.155652+0000","last_fullsized":"2024-09-17T05:40:23.155652+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183271+0000","last_change":"2024-09-17T05:40:25.179791+0000","last_active":"2024-09-17T05:40:26.183271+0000","last_peered":"2024-09-17T05:40:26.183271+0000","last_clean":"2024-09-17T05:40:26.183271+0000","last_became_active":"2024-09-17T05:40:25.179450+0000","last_became_peered":"2024-09-17T05:40:25.179450+0000","last_unstale":"2024-09-17T05:40:26.183271+0000","last_undegraded":"2024-09-17T05:40:26.183271+0000","last_fullsized":"2024-09-17T05:40:26.183271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.496633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.177062+0000","last_change":"2024-09-17T05:40:24.176345+0000","last_active":"2024-09-17T05:40:24.177062+0000","last_peered":"2024-09-17T05:40:24.177062+0000","last_clean":"2024-09-17T05:40:24.177062+0000","last_became_active":"2024-09-17T05:40:24.176106+0000","last_became_peered":"2024-09-17T05:40:24.176106+0000","last_unstale":"2024-09-17T05:40:24.177062+0000","last_undegraded":"2024-09-17T05:40:24.177062+0000","last_fullsized":"2024-09-17T05:40:24.177062+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:23.124357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.154383+0000","last_change":"2024-09-17T05:40:24.154383+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.154383+0000","last_undegraded":"2024-09-17T05:40:24.154383+0000","last_fullsized":"2024-09-17T05:40:24.154383+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154143+0000","last_change":"2024-09-17T05:40:23.154143+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154143+0000","last_undegraded":"2024-09-17T05:40:23.154143+0000","last_fullsized":"2024-09-17T05:40:23.154143+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.156957+0000","last_change":"2024-09-17T05:40:23.156957+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.156957+0000","last_undegraded":"2024-09-17T05:40:23.156957+0000","last_fullsized":"2024-09-17T05:40:23.156957+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153281+0000","last_change":"2024-09-17T05:40:23.153281+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153281+0000","last_undegraded":"2024-09-17T05:40:23.153281+0000","last_fullsized":"2024-09-17T05:40:23.153281+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.180564+0000","last_change":"2024-09-17T05:40:24.177481+0000","last_active":"2024-09-17T05:40:24.180564+0000","last_peered":"2024-09-17T05:40:24.180564+0000","last_clean":"2024-09-17T05:40:24.180564+0000","last_became_active":"2024-09-17T05:40:24.177186+0000","last_became_peered":"2024-09-17T05:40:24.177186+0000","last_unstale":"2024-09-17T05:40:24.180564+0000","last_undegraded":"2024-09-17T05:40:24.180564+0000","last_fullsized":"2024-09-17T05:40:24.180564+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:07.284037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.163654+0000","last_change":"2024-09-17T05:40:24.162381+0000","last_active":"2024-09-17T05:40:24.163654+0000","last_peered":"2024-09-17T05:40:24.163654+0000","last_clean":"2024-09-17T05:40:24.163654+0000","last_became_active":"2024-09-17T05:40:24.160458+0000","last_became_peered":"2024-09-17T05:40:24.160458+0000","last_unstale":"2024-09-17T05:40:24.163654+0000","last_undegraded":"2024-09-17T05:40:24.163654+0000","last_fullsized":"2024-09-17T05:40:24.163654+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:52.170344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154060+0000","last_change":"2024-09-17T05:40:23.154060+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154060+0000","last_undegraded":"2024-09-17T05:40:23.154060+0000","last_fullsized":"2024-09-17T05:40:23.154060+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.162838+0000","last_change":"2024-09-17T05:40:24.161649+0000","last_active":"2024-09-17T05:40:24.162838+0000","last_peered":"2024-09-17T05:40:24.162838+0000","last_clean":"2024-09-17T05:40:24.162838+0000","last_became_active":"2024-09-17T05:40:24.161393+0000","last_became_peered":"2024-09-17T05:40:24.161393+0000","last_unstale":"2024-09-17T05:40:24.162838+0000","last_undegraded":"2024-09-17T05:40:24.162838+0000","last_fullsized":"2024-09-17T05:40:24.162838+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:25.240554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.174326+0000","last_change":"2024-09-17T05:40:24.169609+0000","last_active":"2024-09-17T05:40:24.174326+0000","last_peered":"2024-09-17T05:40:24.174326+0000","last_clean":"2024-09-17T05:40:24.174326+0000","last_became_active":"2024-09-17T05:40:24.169317+0000","last_became_peered":"2024-09-17T05:40:24.169317+0000","last_unstale":"2024-09-17T05:40:24.174326+0000","last_undegraded":"2024-09-17T05:40:24.174326+0000","last_fullsized":"2024-09-17T05:40:24.174326+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:44.173740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183307+0000","last_change":"2024-09-17T05:40:25.184351+0000","last_active":"2024-09-17T05:40:26.183307+0000","last_peered":"2024-09-17T05:40:26.183307+0000","last_clean":"2024-09-17T05:40:26.183307+0000","last_became_active":"2024-09-17T05:40:25.183920+0000","last_became_peered":"2024-09-17T05:40:25.183920+0000","last_unstale":"2024-09-17T05:40:26.183307+0000","last_undegraded":"2024-09-17T05:40:26.183307+0000","last_fullsized":"2024-09-17T05:40:26.183307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:22.638149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182912+0000","last_change":"2024-09-17T05:40:24.169333+0000","last_active":"2024-09-17T05:40:26.182912+0000","last_peered":"2024-09-17T05:40:26.182912+0000","last_clean":"2024-09-17T05:40:26.182912+0000","last_became_active":"2024-09-17T05:40:24.168219+0000","last_became_peered":"2024-09-17T05:40:24.168219+0000","last_unstale":"2024-09-17T05:40:26.182912+0000","last_undegraded":"2024-09-17T05:40:26.182912+0000","last_fullsized":"2024-09-17T05:40:26.182912+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:58.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.156880+0000","last_change":"2024-09-17T05:40:23.156880+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.156880+0000","last_undegraded":"2024-09-17T05:40:23.156880+0000","last_fullsized":"2024-09-17T05:40:23.156880+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154467+0000","last_change":"2024-09-17T05:40:23.154467+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154467+0000","last_undegraded":"2024-09-17T05:40:23.154467+0000","last_fullsized":"2024-09-17T05:40:23.154467+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149174+0000","last_change":"2024-09-17T05:40:23.149174+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149174+0000","last_undegraded":"2024-09-17T05:40:23.149174+0000","last_fullsized":"2024-09-17T05:40:23.149174+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153142+0000","last_change":"2024-09-17T05:40:23.153142+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153142+0000","last_undegraded":"2024-09-17T05:40:23.153142+0000","last_fullsized":"2024-09-17T05:40:23.153142+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163295+0000","last_change":"2024-09-17T05:40:24.163295+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163295+0000","last_undegraded":"2024-09-17T05:40:24.163295+0000","last_fullsized":"2024-09-17T05:40:24.163295+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153655+0000","last_change":"2024-09-17T05:40:23.153655+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153655+0000","last_undegraded":"2024-09-17T05:40:23.153655+0000","last_fullsized":"2024-09-17T05:40:23.153655+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153751+0000","last_change":"2024-09-17T05:40:23.153751+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153751+0000","last_undegraded":"2024-09-17T05:40:23.153751+0000","last_fullsized":"2024-09-17T05:40:23.153751+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.163036+0000","last_change":"2024-09-17T05:40:24.163036+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.163036+0000","last_undegraded":"2024-09-17T05:40:24.163036+0000","last_fullsized":"2024-09-17T05:40:24.163036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183232+0000","last_change":"2024-09-17T05:40:24.173858+0000","last_active":"2024-09-17T05:40:26.183232+0000","last_peered":"2024-09-17T05:40:26.183232+0000","last_clean":"2024-09-17T05:40:26.183232+0000","last_became_active":"2024-09-17T05:40:24.173661+0000","last_became_peered":"2024-09-17T05:40:24.173661+0000","last_unstale":"2024-09-17T05:40:26.183232+0000","last_undegraded":"2024-09-17T05:40:26.183232+0000","last_fullsized":"2024-09-17T05:40:26.183232+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:21.200140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-17T05:40:24.155393+0000","last_change":"2024-09-17T05:40:24.155393+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.155393+0000","last_undegraded":"2024-09-17T05:40:24.155393+0000","last_fullsized":"2024-09-17T05:40:24.155393+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149919+0000","last_change":"2024-09-17T05:40:23.149919+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149919+0000","last_undegraded":"2024-09-17T05:40:23.149919+0000","last_fullsized":"2024-09-17T05:40:23.149919+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.175798+0000","last_change":"2024-09-17T05:40:24.174951+0000","last_active":"2024-09-17T05:40:24.175798+0000","last_peered":"2024-09-17T05:40:24.175798+0000","last_clean":"2024-09-17T05:40:24.175798+0000","last_became_active":"2024-09-17T05:40:24.174436+0000","last_became_peered":"2024-09-17T05:40:24.174436+0000","last_unstale":"2024-09-17T05:40:24.175798+0000","last_undegraded":"2024-09-17T05:40:24.175798+0000","last_fullsized":"2024-09-17T05:40:24.175798+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:29.137707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153136+0000","last_change":"2024-09-17T05:40:23.153136+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153136+0000","last_undegraded":"2024-09-17T05:40:23.153136+0000","last_fullsized":"2024-09-17T05:40:23.153136+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153092+0000","last_change":"2024-09-17T05:40:23.153092+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153092+0000","last_undegraded":"2024-09-17T05:40:23.153092+0000","last_fullsized":"2024-09-17T05:40:23.153092+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153455+0000","last_change":"2024-09-17T05:40:23.153455+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153455+0000","last_undegraded":"2024-09-17T05:40:23.153455+0000","last_fullsized":"2024-09-17T05:40:23.153455+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.153170+0000","last_change":"2024-09-17T05:40:23.153170+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.153170+0000","last_undegraded":"2024-09-17T05:40:23.153170+0000","last_fullsized":"2024-09-17T05:40:23.153170+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149222+0000","last_change":"2024-09-17T05:40:23.149222+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149222+0000","last_undegraded":"2024-09-17T05:40:23.149222+0000","last_fullsized":"2024-09-17T05:40:23.149222+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183134+0000","last_change":"2024-09-17T05:40:24.168536+0000","last_active":"2024-09-17T05:40:26.183134+0000","last_peered":"2024-09-17T05:40:26.183134+0000","last_clean":"2024-09-17T05:40:26.183134+0000","last_became_active":"2024-09-17T05:40:24.168121+0000","last_became_peered":"2024-09-17T05:40:24.168121+0000","last_unstale":"2024-09-17T05:40:26.183134+0000","last_undegraded":"2024-09-17T05:40:26.183134+0000","last_fullsized":"2024-09-17T05:40:26.183134+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:52.244624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183037+0000","last_change":"2024-09-17T05:40:24.167195+0000","last_active":"2024-09-17T05:40:26.183037+0000","last_peered":"2024-09-17T05:40:26.183037+0000","last_clean":"2024-09-17T05:40:26.183037+0000","last_became_active":"2024-09-17T05:40:24.166930+0000","last_became_peered":"2024-09-17T05:40:24.166930+0000","last_unstale":"2024-09-17T05:40:26.183037+0000","last_undegraded":"2024-09-17T05:40:26.183037+0000","last_fullsized":"2024-09-17T05:40:26.183037+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:47.316470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183088+0000","last_change":"2024-09-17T05:40:25.177997+0000","last_active":"2024-09-17T05:40:26.183088+0000","last_peered":"2024-09-17T05:40:26.183088+0000","last_clean":"2024-09-17T05:40:26.183088+0000","last_became_active":"2024-09-17T05:40:25.177737+0000","last_became_peered":"2024-09-17T05:40:25.177737+0000","last_unstale":"2024-09-17T05:40:26.183088+0000","last_undegraded":"2024-09-17T05:40:26.183088+0000","last_fullsized":"2024-09-17T05:40:26.183088+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.829551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.149416+0000","last_change":"2024-09-17T05:40:23.149416+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.149416+0000","last_undegraded":"2024-09-17T05:40:23.149416+0000","last_fullsized":"2024-09-17T05:40:23.149416+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.155191+0000","last_change":"2024-09-17T05:40:23.155191+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.155191+0000","last_undegraded":"2024-09-17T05:40:23.155191+0000","last_fullsized":"2024-09-17T05:40:23.155191+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154471+0000","last_change":"2024-09-17T05:40:23.154471+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154471+0000","last_undegraded":"2024-09-17T05:40:23.154471+0000","last_fullsized":"2024-09-17T05:40:23.154471+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T05:40:24.144923+0000","last_change":"2024-09-17T05:40:24.144923+0000","last_active":"2024-09-17T05:40:24.144923+0000","last_peered":"2024-09-17T05:40:24.144923+0000","last_clean":"2024-09-17T05:40:24.144923+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:24.144923+0000","last_undegraded":"2024-09-17T05:40:24.144923+0000","last_fullsized":"2024-09-17T05:40:24.144923+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.175381+0000","last_change":"2024-09-17T05:40:24.174394+0000","last_active":"2024-09-17T05:40:24.175381+0000","last_peered":"2024-09-17T05:40:24.175381+0000","last_clean":"2024-09-17T05:40:24.175381+0000","last_became_active":"2024-09-17T05:40:24.174122+0000","last_became_peered":"2024-09-17T05:40:24.174122+0000","last_unstale":"2024-09-17T05:40:24.175381+0000","last_undegraded":"2024-09-17T05:40:24.175381+0000","last_fullsized":"2024-09-17T05:40:24.175381+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:58.370544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183130+0000","last_change":"2024-09-17T05:40:25.185149+0000","last_active":"2024-09-17T05:40:26.183130+0000","last_peered":"2024-09-17T05:40:26.183130+0000","last_clean":"2024-09-17T05:40:26.183130+0000","last_became_active":"2024-09-17T05:40:25.184781+0000","last_became_peered":"2024-09-17T05:40:25.184781+0000","last_unstale":"2024-09-17T05:40:26.183130+0000","last_undegraded":"2024-09-17T05:40:26.183130+0000","last_fullsized":"2024-09-17T05:40:26.183130+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.885218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-09-17T05:40:23.154548+0000","last_change":"2024-09-17T05:40:23.154548+0000","last_active":"2024-09-17T05:40:23.136931+0000","last_peered":"2024-09-17T05:40:23.136931+0000","last_clean":"2024-09-17T05:40:23.136931+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:40:23.154548+0000","last_undegraded":"2024-09-17T05:40:23.154548+0000","last_fullsized":"2024-09-17T05:40:23.154548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":341,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-17T05:40:24.165229+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:40:24.165229+0000","last_peered":"2024-09-17T05:40:24.165229+0000","last_clean":"2024-09-17T05:40:24.165229+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:40:24.165229+0000","last_undegraded":"2024-09-17T05:40:24.165229+0000","last_fullsized":"2024-09-17T05:40:24.165229+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":3},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":1682,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509629,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127252,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244588,"statfs":{"total":96636764160,"available":96506458112,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509629,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137000,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234840,"statfs":{"total":96636764160,"available":96496476160,"internally_reserved":0,"allocated":1781760,"data_stored":786151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131416,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240424,"statfs":{"total":96636764160,"available":96502194176,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509628,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131412,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240428,"statfs":{"total":96636764160,"available":96502198272,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131420,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240420,"statfs":{"total":96636764160,"available":96502190080,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509630,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145880,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225960,"statfs":{"total":96636764160,"available":96487383040,"internally_reserved":0,"allocated":2363392,"data_stored":1370160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509628,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149468,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222372,"statfs":{"total":96636764160,"available":96483708928,"internally_reserved":0,"allocated":1769472,"data_stored":779792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":151255499},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119580,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252260,"statfs":{"total":96636764160,"available":96514314240,"internally_reserved":0,"allocated":2424832,"data_stored":1391568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:40:30.065 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3f is not active+clean 2024-09-17T05:40:30.065 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3e is not active+clean 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3d is not active+clean 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3c is not active+clean 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3b is not active+clean 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.163076+0000', 'last_change': '2024-09-17T05:40:24.163076+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.163076+0000', 'last_undegraded': '2024-09-17T05:40:24.163076+0000', 'last_fullsized': '2024-09-17T05:40:24.163076+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3a is not active+clean 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.155291+0000', 'last_change': '2024-09-17T05:40:24.155291+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.155291+0000', 'last_undegraded': '2024-09-17T05:40:24.155291+0000', 'last_fullsized': '2024-09-17T05:40:24.155291+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:40:30.066 INFO:tasks.ceph.ceph_manager.ceph:PG 21.38 is not active+clean 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:PG 21.37 is not active+clean 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.154825+0000', 'last_change': '2024-09-17T05:40:24.154825+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.154825+0000', 'last_undegraded': '2024-09-17T05:40:24.154825+0000', 'last_fullsized': '2024-09-17T05:40:24.154825+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:PG 21.36 is not active+clean 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:PG 21.35 is not active+clean 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.155818+0000', 'last_change': '2024-09-17T05:40:24.155818+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.155818+0000', 'last_undegraded': '2024-09-17T05:40:24.155818+0000', 'last_fullsized': '2024-09-17T05:40:24.155818+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:PG 21.34 is not active+clean 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.067 INFO:tasks.ceph.ceph_manager.ceph:PG 21.33 is not active+clean 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.162021+0000', 'last_change': '2024-09-17T05:40:24.162021+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.162021+0000', 'last_undegraded': '2024-09-17T05:40:24.162021+0000', 'last_fullsized': '2024-09-17T05:40:24.162021+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:PG 21.32 is not active+clean 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:PG 21.31 is not active+clean 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.163566+0000', 'last_change': '2024-09-17T05:40:24.163566+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.163566+0000', 'last_undegraded': '2024-09-17T05:40:24.163566+0000', 'last_fullsized': '2024-09-17T05:40:24.163566+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2f is not active+clean 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2e is not active+clean 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.164790+0000', 'last_change': '2024-09-17T05:40:24.164790+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.164790+0000', 'last_undegraded': '2024-09-17T05:40:24.164790+0000', 'last_fullsized': '2024-09-17T05:40:24.164790+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:40:30.068 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2d is not active+clean 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2c is not active+clean 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2b is not active+clean 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2a is not active+clean 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:PG 21.29 is not active+clean 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:PG 21.28 is not active+clean 2024-09-17T05:40:30.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.161537+0000', 'last_change': '2024-09-17T05:40:24.161537+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.161537+0000', 'last_undegraded': '2024-09-17T05:40:24.161537+0000', 'last_fullsized': '2024-09-17T05:40:24.161537+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:PG 21.27 is not active+clean 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:PG 21.26 is not active+clean 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:PG 21.25 is not active+clean 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:PG 21.24 is not active+clean 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:PG 21.23 is not active+clean 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.070 INFO:tasks.ceph.ceph_manager.ceph:PG 21.22 is not active+clean 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:PG 21.21 is not active+clean 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:PG 21.20 is not active+clean 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1e is not active+clean 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.163811+0000', 'last_change': '2024-09-17T05:40:24.163811+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.163811+0000', 'last_undegraded': '2024-09-17T05:40:24.163811+0000', 'last_fullsized': '2024-09-17T05:40:24.163811+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1f is not active+clean 2024-09-17T05:40:30.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.155652+0000', 'last_change': '2024-09-17T05:40:23.155652+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.155652+0000', 'last_undegraded': '2024-09-17T05:40:23.155652+0000', 'last_fullsized': '2024-09-17T05:40:23.155652+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1c is not active+clean 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.154383+0000', 'last_change': '2024-09-17T05:40:24.154383+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.154383+0000', 'last_undegraded': '2024-09-17T05:40:24.154383+0000', 'last_fullsized': '2024-09-17T05:40:24.154383+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1d is not active+clean 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.154143+0000', 'last_change': '2024-09-17T05:40:23.154143+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.154143+0000', 'last_undegraded': '2024-09-17T05:40:23.154143+0000', 'last_fullsized': '2024-09-17T05:40:23.154143+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1d is not active+clean 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1c is not active+clean 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.156957+0000', 'last_change': '2024-09-17T05:40:23.156957+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.156957+0000', 'last_undegraded': '2024-09-17T05:40:23.156957+0000', 'last_fullsized': '2024-09-17T05:40:23.156957+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1a is not active+clean 2024-09-17T05:40:30.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1b is not active+clean 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.153281+0000', 'last_change': '2024-09-17T05:40:23.153281+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.153281+0000', 'last_undegraded': '2024-09-17T05:40:23.153281+0000', 'last_fullsized': '2024-09-17T05:40:23.153281+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1b is not active+clean 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:PG 21.18 is not active+clean 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:PG 21.19 is not active+clean 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:PG 20.18 is not active+clean 2024-09-17T05:40:30.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.154060+0000', 'last_change': '2024-09-17T05:40:23.154060+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.154060+0000', 'last_undegraded': '2024-09-17T05:40:23.154060+0000', 'last_fullsized': '2024-09-17T05:40:23.154060+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:40:30.074 INFO:tasks.ceph.ceph_manager.ceph:PG 21.16 is not active+clean 2024-09-17T05:40:30.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.074 INFO:tasks.ceph.ceph_manager.ceph:PG 21.17 is not active+clean 2024-09-17T05:40:30.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.074 INFO:tasks.ceph.ceph_manager.ceph:PG 21.12 is not active+clean 2024-09-17T05:40:30.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.074 INFO:tasks.ceph.ceph_manager.ceph:PG 20.13 is not active+clean 2024-09-17T05:40:30.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.156880+0000', 'last_change': '2024-09-17T05:40:23.156880+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.156880+0000', 'last_undegraded': '2024-09-17T05:40:23.156880+0000', 'last_fullsized': '2024-09-17T05:40:23.156880+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:PG 21.13 is not active+clean 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:PG 20.12 is not active+clean 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.154467+0000', 'last_change': '2024-09-17T05:40:23.154467+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.154467+0000', 'last_undegraded': '2024-09-17T05:40:23.154467+0000', 'last_fullsized': '2024-09-17T05:40:23.154467+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:PG 21.10 is not active+clean 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:PG 20.11 is not active+clean 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.149174+0000', 'last_change': '2024-09-17T05:40:23.149174+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.149174+0000', 'last_undegraded': '2024-09-17T05:40:23.149174+0000', 'last_fullsized': '2024-09-17T05:40:23.149174+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:PG 21.11 is not active+clean 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.075 INFO:tasks.ceph.ceph_manager.ceph:PG 20.10 is not active+clean 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.153142+0000', 'last_change': '2024-09-17T05:40:23.153142+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.153142+0000', 'last_undegraded': '2024-09-17T05:40:23.153142+0000', 'last_fullsized': '2024-09-17T05:40:23.153142+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:PG 21.e is not active+clean 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.163295+0000', 'last_change': '2024-09-17T05:40:24.163295+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.163295+0000', 'last_undegraded': '2024-09-17T05:40:24.163295+0000', 'last_fullsized': '2024-09-17T05:40:24.163295+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:PG 20.f is not active+clean 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.153655+0000', 'last_change': '2024-09-17T05:40:23.153655+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.153655+0000', 'last_undegraded': '2024-09-17T05:40:23.153655+0000', 'last_fullsized': '2024-09-17T05:40:23.153655+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:PG 21.f is not active+clean 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:PG 20.e is not active+clean 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.153751+0000', 'last_change': '2024-09-17T05:40:23.153751+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.153751+0000', 'last_undegraded': '2024-09-17T05:40:23.153751+0000', 'last_fullsized': '2024-09-17T05:40:23.153751+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:PG 21.c is not active+clean 2024-09-17T05:40:30.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.163036+0000', 'last_change': '2024-09-17T05:40:24.163036+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.163036+0000', 'last_undegraded': '2024-09-17T05:40:24.163036+0000', 'last_fullsized': '2024-09-17T05:40:24.163036+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:PG 21.d is not active+clean 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:24.155393+0000', 'last_change': '2024-09-17T05:40:24.155393+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.155393+0000', 'last_undegraded': '2024-09-17T05:40:24.155393+0000', 'last_fullsized': '2024-09-17T05:40:24.155393+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:PG 20.c is not active+clean 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.149919+0000', 'last_change': '2024-09-17T05:40:23.149919+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.149919+0000', 'last_undegraded': '2024-09-17T05:40:23.149919+0000', 'last_fullsized': '2024-09-17T05:40:23.149919+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:PG 21.a is not active+clean 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:PG 21.b is not active+clean 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:PG 20.a is not active+clean 2024-09-17T05:40:30.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.153136+0000', 'last_change': '2024-09-17T05:40:23.153136+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.153136+0000', 'last_undegraded': '2024-09-17T05:40:23.153136+0000', 'last_fullsized': '2024-09-17T05:40:23.153136+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:PG 21.8 is not active+clean 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:PG 20.9 is not active+clean 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.153092+0000', 'last_change': '2024-09-17T05:40:23.153092+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.153092+0000', 'last_undegraded': '2024-09-17T05:40:23.153092+0000', 'last_fullsized': '2024-09-17T05:40:23.153092+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:PG 21.9 is not active+clean 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:PG 20.8 is not active+clean 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.153455+0000', 'last_change': '2024-09-17T05:40:23.153455+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.153455+0000', 'last_undegraded': '2024-09-17T05:40:23.153455+0000', 'last_fullsized': '2024-09-17T05:40:23.153455+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:PG 21.6 is not active+clean 2024-09-17T05:40:30.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:PG 20.7 is not active+clean 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.153170+0000', 'last_change': '2024-09-17T05:40:23.153170+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.153170+0000', 'last_undegraded': '2024-09-17T05:40:23.153170+0000', 'last_fullsized': '2024-09-17T05:40:23.153170+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:PG 21.7 is not active+clean 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:PG 20.6 is not active+clean 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.149222+0000', 'last_change': '2024-09-17T05:40:23.149222+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.149222+0000', 'last_undegraded': '2024-09-17T05:40:23.149222+0000', 'last_fullsized': '2024-09-17T05:40:23.149222+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:PG 21.4 is not active+clean 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.079 INFO:tasks.ceph.ceph_manager.ceph:PG 21.5 is not active+clean 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3 is not active+clean 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.149416+0000', 'last_change': '2024-09-17T05:40:23.149416+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.149416+0000', 'last_undegraded': '2024-09-17T05:40:23.149416+0000', 'last_fullsized': '2024-09-17T05:40:23.149416+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3 is not active+clean 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2 is not active+clean 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.155191+0000', 'last_change': '2024-09-17T05:40:23.155191+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.155191+0000', 'last_undegraded': '2024-09-17T05:40:23.155191+0000', 'last_fullsized': '2024-09-17T05:40:23.155191+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:PG 21.0 is not active+clean 2024-09-17T05:40:30.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.081 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1 is not active+clean 2024-09-17T05:40:30.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.154471+0000', 'last_change': '2024-09-17T05:40:23.154471+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.154471+0000', 'last_undegraded': '2024-09-17T05:40:23.154471+0000', 'last_fullsized': '2024-09-17T05:40:23.154471+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:40:30.081 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1 is not active+clean 2024-09-17T05:40:30.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T05:40:24.144923+0000', 'last_change': '2024-09-17T05:40:24.144923+0000', 'last_active': '2024-09-17T05:40:24.144923+0000', 'last_peered': '2024-09-17T05:40:24.144923+0000', 'last_clean': '2024-09-17T05:40:24.144923+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:24.144923+0000', 'last_undegraded': '2024-09-17T05:40:24.144923+0000', 'last_fullsized': '2024-09-17T05:40:24.144923+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:24.144923+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T05:40:30.081 INFO:tasks.ceph.ceph_manager.ceph:PG 20.15 is not active+clean 2024-09-17T05:40:30.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:40:23.154548+0000', 'last_change': '2024-09-17T05:40:23.154548+0000', 'last_active': '2024-09-17T05:40:23.136931+0000', 'last_peered': '2024-09-17T05:40:23.136931+0000', 'last_clean': '2024-09-17T05:40:23.136931+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:40:23.154548+0000', 'last_undegraded': '2024-09-17T05:40:23.154548+0000', 'last_fullsized': '2024-09-17T05:40:23.154548+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'last_clean_scrub_stamp': '2024-09-17T05:40:23.136931+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:40:30.082 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:40:30.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- 172.21.15.122:0/855782298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 msgr2=0x7f1fd415fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/855782298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 0x7f1fd415fc40 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009900 tx=0x7f1fbc01c820 comp rx=0 tx=0).stop 2024-09-17T05:40:30.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- 172.21.15.122:0/855782298 shutdown_connections 2024-09-17T05:40:30.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/855782298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 0x7f1fd415fc40 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/855782298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fd4070d80 0x7f1fd4071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- 172.21.15.122:0/855782298 >> 172.21.15.122:0/855782298 conn(0x7f1fd40d1cf0 msgr2=0x7f1fd40d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:30.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- 172.21.15.122:0/855782298 shutdown_connections 2024-09-17T05:40:30.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- 172.21.15.122:0/855782298 wait complete. 2024-09-17T05:40:30.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 Processor -- start 2024-09-17T05:40:30.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- start start 2024-09-17T05:40:30.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fd4070d80 0x7f1fd406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:30.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 0x7f1fd41f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:30.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fd4160180 0x7f1fd4156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:30.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4169120 con 0x7f1fd40d2810 2024-09-17T05:40:30.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd40d0ef0 con 0x7f1fd4160180 2024-09-17T05:40:30.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.253+0000 7f1fd3e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd41692a0 con 0x7f1fd4070d80 2024-09-17T05:40:30.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 0x7f1fd41f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:30.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 0x7f1fd41f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40052/0 (socket says 172.21.15.122:40052) 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 -- 172.21.15.122:0/3120385477 learned_addr learned my addr 172.21.15.122:0/3120385477 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fd2a00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fd4070d80 0x7f1fd406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fd3400640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fd4160180 0x7f1fd4156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 -- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fd4070d80 msgr2=0x7f1fd406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fd4070d80 0x7f1fd406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 -- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fd4160180 msgr2=0x7f1fd4156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fd4160180 0x7f1fd4156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 -- 172.21.15.122:0/3120385477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd4159b40 con 0x7f1fd40d2810 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fca000640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 0x7f1fd41f7bb0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009850 tx=0x7f1fbc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fcb400640 1 -- 172.21.15.122:0/3120385477 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0b2040 con 0x7f1fd40d2810 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fcb400640 1 -- 172.21.15.122:0/3120385477 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fbc0044b0 con 0x7f1fd40d2810 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd41534f0 con 0x7f1fd40d2810 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fcb400640 1 -- 172.21.15.122:0/3120385477 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc005280 con 0x7f1fd40d2810 2024-09-17T05:40:30.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd41f8500 con 0x7f1fd40d2810 2024-09-17T05:40:30.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd4071160 con 0x7f1fd40d2810 2024-09-17T05:40:30.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.257+0000 7f1fcb400640 1 -- 172.21.15.122:0/3120385477 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f1fbc005420 con 0x7f1fd40d2810 2024-09-17T05:40:30.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.261+0000 7f1fcb400640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fb0075510 0x7f1fb00779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:30.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.261+0000 7f1fcb400640 1 -- 172.21.15.122:0/3120385477 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f1fbc12d280 con 0x7f1fd40d2810 2024-09-17T05:40:30.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.261+0000 7f1fd2a00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fb0075510 0x7f1fb00779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:30.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.261+0000 7f1fd2a00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fb0075510 0x7f1fb00779d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005fd0 tx=0x7f1fc4005db0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:30.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.261+0000 7f1fcb400640 1 -- 172.21.15.122:0/3120385477 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1fbc0164d0 con 0x7f1fd40d2810 2024-09-17T05:40:30.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1fd415fbb0 con 0x7f1fd40d2810 2024-09-17T05:40:30.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fcb400640 1 -- 172.21.15.122:0/3120385477 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2511 (secure 0 0 0) 0x7f1fbc0f9e10 con 0x7f1fd40d2810 2024-09-17T05:40:30.489 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:30.489 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":951,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":145,"num_osds":8,"num_up_osds":8,"osd_up_since":1726550693,"num_in_osds":8,"osd_in_since":1726550682,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":45},{"state_name":"creating+peering","count":34},{"state_name":"active+clean","count":18}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":592050,"bytes_used":1099190272,"bytes_avail":771994923008,"bytes_total":773094113280,"unknown_pgs_ratio":0.46391752362251282,"inactive_pgs_ratio":0.3505154550075531,"write_bytes_sec":730,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":96,"btime":"2024-09-17T05:40:28:184035+0000","id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9399}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-09-17T05:40:06.750588+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T05:40:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fb0075510 msgr2=0x7f1fb00779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fb0075510 0x7f1fb00779d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4005fd0 tx=0x7f1fc4005db0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 msgr2=0x7f1fd41f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 0x7f1fd41f7bb0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009850 tx=0x7f1fbc0040c0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 shutdown_connections 2024-09-17T05:40:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1fb0075510 0x7f1fb00779d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fd4160180 0x7f1fd4156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fd40d2810 0x7f1fd41f7bb0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 --2- 172.21.15.122:0/3120385477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fd4070d80 0x7f1fd406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 >> 172.21.15.122:0/3120385477 conn(0x7f1fd40d1cf0 msgr2=0x7f1fd40cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:30.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 shutdown_connections 2024-09-17T05:40:30.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.493+0000 7f1fd3e00640 1 -- 172.21.15.122:0/3120385477 wait complete. 2024-09-17T05:40:30.512 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:40:30.512 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:40:30.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 -- 172.21.15.122:0/1255921184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b4d80 msgr2=0x7f31bc0b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 --2- 172.21.15.122:0/1255921184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b4d80 0x7f31bc0b11a0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f31bc069e80 tx=0x7f31b001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 -- 172.21.15.122:0/1255921184 shutdown_connections 2024-09-17T05:40:30.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 --2- 172.21.15.122:0/1255921184 >> v2:172.21.15.173:3300/0 conn(0x7f31bc0b0ce0 0x7f31bc0b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 --2- 172.21.15.122:0/1255921184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b4d80 0x7f31bc0b11a0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 -- 172.21.15.122:0/1255921184 >> 172.21.15.122:0/1255921184 conn(0x7f31bc09aa10 msgr2=0x7f31bc09ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:30.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 -- 172.21.15.122:0/1255921184 shutdown_connections 2024-09-17T05:40:30.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 -- 172.21.15.122:0/1255921184 wait complete. 2024-09-17T05:40:30.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.657+0000 7f31c1e00640 1 Processor -- start 2024-09-17T05:40:30.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 -- start start 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b0ce0 0x7f31bc06f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31bc0b4d80 0x7f31bc1486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31bc0b56a0 0x7f31bc0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc168a20 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1688a0 con 0x7f31bc0b4d80 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f31bc168ba0 con 0x7f31bc0b56a0 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b0ce0 0x7f31bc06f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b0ce0 0x7f31bc06f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40066/0 (socket says 172.21.15.122:40066) 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 -- 172.21.15.122:0/2956391278 learned_addr learned my addr 172.21.15.122:0/2956391278 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:30.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 -- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31bc0b56a0 msgr2=0x7f31bc0b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31bc0b56a0 0x7f31bc0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 -- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31bc0b4d80 msgr2=0x7f31bc1486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31bc0b4d80 0x7f31bc1486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 -- 172.21.15.122:0/2956391278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc0cb040 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c0a00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b0ce0 0x7f31bc06f710 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f31b00b0040 tx=0x7f31b00a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31b8c00640 1 -- 172.21.15.122:0/2956391278 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00b2040 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31b8c00640 1 -- 172.21.15.122:0/2956391278 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31b0005020 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31b8c00640 1 -- 172.21.15.122:0/2956391278 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0005470 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc0cb2e0 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc0cb8a0 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.661+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31bc0b5160 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.665+0000 7f31b8c00640 1 -- 172.21.15.122:0/2956391278 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f31b00046d0 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.665+0000 7f31b8c00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f319c075510 0x7f319c0779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:30.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.665+0000 7f31b8c00640 1 -- 172.21.15.122:0/2956391278 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f31b012cef0 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.665+0000 7f31bbe00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f319c075510 0x7f319c0779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:30.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.665+0000 7f31bbe00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f319c075510 0x7f319c0779d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f31a80058d0 tx=0x7f31a8005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:30.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.669+0000 7f31b8c00640 1 -- 172.21.15.122:0/2956391278 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f31b00164b0 con 0x7f31bc0b0ce0 2024-09-17T05:40:30.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.833+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31bc151280 con 0x7f319c075510 2024-09-17T05:40:30.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.837+0000 7f31b8c00640 1 -- 172.21.15.122:0/2956391278 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252497 (secure 0 0 0) 0x7f31bc151280 con 0x7f319c075510 2024-09-17T05:40:30.837 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:30.838 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f319c075510 msgr2=0x7f319c0779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f319c075510 0x7f319c0779d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f31a80058d0 tx=0x7f31a8005860 comp rx=0 tx=0).stop 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b0ce0 msgr2=0x7f31bc06f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b0ce0 0x7f31bc06f710 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f31b00b0040 tx=0x7f31b00a57b0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 shutdown_connections 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f319c075510 0x7f319c0779d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31bc0b56a0 0x7f31bc0b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31bc0b4d80 0x7f31bc1486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 --2- 172.21.15.122:0/2956391278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31bc0b0ce0 0x7f31bc06f710 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 >> 172.21.15.122:0/2956391278 conn(0x7f31bc09aa10 msgr2=0x7f31bc152a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 shutdown_connections 2024-09-17T05:40:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:30.841+0000 7f31c1e00640 1 -- 172.21.15.122:0/2956391278 wait complete. 2024-09-17T05:40:30.859 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2024-09-17T05:40:30.756220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1095900,"kb_used_data":15336,"kb_used_omap":12,"kb_used_meta":1080371,"kb_avail":753878820,"statfs":{"total":773094113280,"available":771971911680,"internally_reserved":0,"allocated":15704064,"data_stored":7580708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1106300470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.608158"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257299+0000","last_change":"2024-09-17T05:40:25.183502+0000","last_active":"2024-09-17T05:40:27.257299+0000","last_peered":"2024-09-17T05:40:27.257299+0000","last_clean":"2024-09-17T05:40:27.257299+0000","last_became_active":"2024-09-17T05:40:25.183249+0000","last_became_peered":"2024-09-17T05:40:25.183249+0000","last_unstale":"2024-09-17T05:40:27.257299+0000","last_undegraded":"2024-09-17T05:40:27.257299+0000","last_fullsized":"2024-09-17T05:40:27.257299+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:52.938668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181051+0000","last_change":"2024-09-17T05:40:25.184193+0000","last_active":"2024-09-17T05:40:26.181051+0000","last_peered":"2024-09-17T05:40:26.181051+0000","last_clean":"2024-09-17T05:40:26.181051+0000","last_became_active":"2024-09-17T05:40:25.182207+0000","last_became_peered":"2024-09-17T05:40:25.182207+0000","last_unstale":"2024-09-17T05:40:26.181051+0000","last_undegraded":"2024-09-17T05:40:26.181051+0000","last_fullsized":"2024-09-17T05:40:26.181051+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:43.289435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059543+0000","last_change":"2024-09-17T05:40:25.190821+0000","last_active":"2024-09-17T05:40:27.059543+0000","last_peered":"2024-09-17T05:40:27.059543+0000","last_clean":"2024-09-17T05:40:27.059543+0000","last_became_active":"2024-09-17T05:40:25.189507+0000","last_became_peered":"2024-09-17T05:40:25.189507+0000","last_unstale":"2024-09-17T05:40:27.059543+0000","last_undegraded":"2024-09-17T05:40:27.059543+0000","last_fullsized":"2024-09-17T05:40:27.059543+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:05.796307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180298+0000","last_change":"2024-09-17T05:40:25.193673+0000","last_active":"2024-09-17T05:40:26.180298+0000","last_peered":"2024-09-17T05:40:26.180298+0000","last_clean":"2024-09-17T05:40:26.180298+0000","last_became_active":"2024-09-17T05:40:25.190809+0000","last_became_peered":"2024-09-17T05:40:25.190809+0000","last_unstale":"2024-09-17T05:40:26.180298+0000","last_undegraded":"2024-09-17T05:40:26.180298+0000","last_fullsized":"2024-09-17T05:40:26.180298+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:14.342301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257680+0000","last_change":"2024-09-17T05:40:25.192148+0000","last_active":"2024-09-17T05:40:27.257680+0000","last_peered":"2024-09-17T05:40:27.257680+0000","last_clean":"2024-09-17T05:40:27.257680+0000","last_became_active":"2024-09-17T05:40:25.191696+0000","last_became_peered":"2024-09-17T05:40:25.191696+0000","last_unstale":"2024-09-17T05:40:27.257680+0000","last_undegraded":"2024-09-17T05:40:27.257680+0000","last_fullsized":"2024-09-17T05:40:27.257680+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:35:01.410825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761849+0000","last_change":"2024-09-17T05:40:25.182263+0000","last_active":"2024-09-17T05:40:26.761849+0000","last_peered":"2024-09-17T05:40:26.761849+0000","last_clean":"2024-09-17T05:40:26.761849+0000","last_became_active":"2024-09-17T05:40:25.181213+0000","last_became_peered":"2024-09-17T05:40:25.181213+0000","last_unstale":"2024-09-17T05:40:26.761849+0000","last_undegraded":"2024-09-17T05:40:26.761849+0000","last_fullsized":"2024-09-17T05:40:26.761849+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:53.661119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182643+0000","last_change":"2024-09-17T05:40:25.183672+0000","last_active":"2024-09-17T05:40:26.182643+0000","last_peered":"2024-09-17T05:40:26.182643+0000","last_clean":"2024-09-17T05:40:26.182643+0000","last_became_active":"2024-09-17T05:40:25.183400+0000","last_became_peered":"2024-09-17T05:40:25.183400+0000","last_unstale":"2024-09-17T05:40:26.182643+0000","last_undegraded":"2024-09-17T05:40:26.182643+0000","last_fullsized":"2024-09-17T05:40:26.182643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:25.692977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256704+0000","last_change":"2024-09-17T05:40:25.188354+0000","last_active":"2024-09-17T05:40:27.256704+0000","last_peered":"2024-09-17T05:40:27.256704+0000","last_clean":"2024-09-17T05:40:27.256704+0000","last_became_active":"2024-09-17T05:40:25.188059+0000","last_became_peered":"2024-09-17T05:40:25.188059+0000","last_unstale":"2024-09-17T05:40:27.256704+0000","last_undegraded":"2024-09-17T05:40:27.256704+0000","last_fullsized":"2024-09-17T05:40:27.256704+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.904271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761948+0000","last_change":"2024-09-17T05:40:25.180756+0000","last_active":"2024-09-17T05:40:26.761948+0000","last_peered":"2024-09-17T05:40:26.761948+0000","last_clean":"2024-09-17T05:40:26.761948+0000","last_became_active":"2024-09-17T05:40:25.180399+0000","last_became_peered":"2024-09-17T05:40:25.180399+0000","last_unstale":"2024-09-17T05:40:26.761948+0000","last_undegraded":"2024-09-17T05:40:26.761948+0000","last_fullsized":"2024-09-17T05:40:26.761948+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:26.035276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059533+0000","last_change":"2024-09-17T05:40:25.174808+0000","last_active":"2024-09-17T05:40:27.059533+0000","last_peered":"2024-09-17T05:40:27.059533+0000","last_clean":"2024-09-17T05:40:27.059533+0000","last_became_active":"2024-09-17T05:40:25.174530+0000","last_became_peered":"2024-09-17T05:40:25.174530+0000","last_unstale":"2024-09-17T05:40:27.059533+0000","last_undegraded":"2024-09-17T05:40:27.059533+0000","last_fullsized":"2024-09-17T05:40:27.059533+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:28.976639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.762120+0000","last_change":"2024-09-17T05:40:25.185179+0000","last_active":"2024-09-17T05:40:26.762120+0000","last_peered":"2024-09-17T05:40:26.762120+0000","last_clean":"2024-09-17T05:40:26.762120+0000","last_became_active":"2024-09-17T05:40:25.184771+0000","last_became_peered":"2024-09-17T05:40:25.184771+0000","last_unstale":"2024-09-17T05:40:26.762120+0000","last_undegraded":"2024-09-17T05:40:26.762120+0000","last_fullsized":"2024-09-17T05:40:26.762120+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:14.342301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258498+0000","last_change":"2024-09-17T05:40:25.182688+0000","last_active":"2024-09-17T05:40:27.258498+0000","last_peered":"2024-09-17T05:40:27.258498+0000","last_clean":"2024-09-17T05:40:27.258498+0000","last_became_active":"2024-09-17T05:40:25.182418+0000","last_became_peered":"2024-09-17T05:40:25.182418+0000","last_unstale":"2024-09-17T05:40:27.258498+0000","last_undegraded":"2024-09-17T05:40:27.258498+0000","last_fullsized":"2024-09-17T05:40:27.258498+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:59.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257872+0000","last_change":"2024-09-17T05:40:25.184777+0000","last_active":"2024-09-17T05:40:27.257872+0000","last_peered":"2024-09-17T05:40:27.257872+0000","last_clean":"2024-09-17T05:40:27.257872+0000","last_became_active":"2024-09-17T05:40:25.184082+0000","last_became_peered":"2024-09-17T05:40:25.184082+0000","last_unstale":"2024-09-17T05:40:27.257872+0000","last_undegraded":"2024-09-17T05:40:27.257872+0000","last_fullsized":"2024-09-17T05:40:27.257872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:04.358926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258218+0000","last_change":"2024-09-17T05:40:25.179501+0000","last_active":"2024-09-17T05:40:27.258218+0000","last_peered":"2024-09-17T05:40:27.258218+0000","last_clean":"2024-09-17T05:40:27.258218+0000","last_became_active":"2024-09-17T05:40:25.179221+0000","last_became_peered":"2024-09-17T05:40:25.179221+0000","last_unstale":"2024-09-17T05:40:27.258218+0000","last_undegraded":"2024-09-17T05:40:27.258218+0000","last_fullsized":"2024-09-17T05:40:27.258218+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:08.540681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257617+0000","last_change":"2024-09-17T05:40:25.188964+0000","last_active":"2024-09-17T05:40:27.257617+0000","last_peered":"2024-09-17T05:40:27.257617+0000","last_clean":"2024-09-17T05:40:27.257617+0000","last_became_active":"2024-09-17T05:40:25.188669+0000","last_became_peered":"2024-09-17T05:40:25.188669+0000","last_unstale":"2024-09-17T05:40:27.257617+0000","last_undegraded":"2024-09-17T05:40:27.257617+0000","last_fullsized":"2024-09-17T05:40:27.257617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:31.199093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182207+0000","last_change":"2024-09-17T05:40:25.175534+0000","last_active":"2024-09-17T05:40:26.182207+0000","last_peered":"2024-09-17T05:40:26.182207+0000","last_clean":"2024-09-17T05:40:26.182207+0000","last_became_active":"2024-09-17T05:40:25.175183+0000","last_became_peered":"2024-09-17T05:40:25.175183+0000","last_unstale":"2024-09-17T05:40:26.182207+0000","last_undegraded":"2024-09-17T05:40:26.182207+0000","last_fullsized":"2024-09-17T05:40:26.182207+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:42.949912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059773+0000","last_change":"2024-09-17T05:40:25.189648+0000","last_active":"2024-09-17T05:40:27.059773+0000","last_peered":"2024-09-17T05:40:27.059773+0000","last_clean":"2024-09-17T05:40:27.059773+0000","last_became_active":"2024-09-17T05:40:25.189407+0000","last_became_peered":"2024-09-17T05:40:25.189407+0000","last_unstale":"2024-09-17T05:40:27.059773+0000","last_undegraded":"2024-09-17T05:40:27.059773+0000","last_fullsized":"2024-09-17T05:40:27.059773+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:01.239661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258367+0000","last_change":"2024-09-17T05:40:25.184991+0000","last_active":"2024-09-17T05:40:27.258367+0000","last_peered":"2024-09-17T05:40:27.258367+0000","last_clean":"2024-09-17T05:40:27.258367+0000","last_became_active":"2024-09-17T05:40:25.184229+0000","last_became_peered":"2024-09-17T05:40:25.184229+0000","last_unstale":"2024-09-17T05:40:27.258367+0000","last_undegraded":"2024-09-17T05:40:27.258367+0000","last_fullsized":"2024-09-17T05:40:27.258367+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:53.343374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256999+0000","last_change":"2024-09-17T05:40:25.180348+0000","last_active":"2024-09-17T05:40:27.256999+0000","last_peered":"2024-09-17T05:40:27.256999+0000","last_clean":"2024-09-17T05:40:27.256999+0000","last_became_active":"2024-09-17T05:40:25.180037+0000","last_became_peered":"2024-09-17T05:40:25.180037+0000","last_unstale":"2024-09-17T05:40:27.256999+0000","last_undegraded":"2024-09-17T05:40:27.256999+0000","last_fullsized":"2024-09-17T05:40:27.256999+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:38.044516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256872+0000","last_change":"2024-09-17T05:40:25.181664+0000","last_active":"2024-09-17T05:40:27.256872+0000","last_peered":"2024-09-17T05:40:27.256872+0000","last_clean":"2024-09-17T05:40:27.256872+0000","last_became_active":"2024-09-17T05:40:25.181358+0000","last_became_peered":"2024-09-17T05:40:25.181358+0000","last_unstale":"2024-09-17T05:40:27.256872+0000","last_undegraded":"2024-09-17T05:40:27.256872+0000","last_fullsized":"2024-09-17T05:40:27.256872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:57.662374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059705+0000","last_change":"2024-09-17T05:40:25.194561+0000","last_active":"2024-09-17T05:40:27.059705+0000","last_peered":"2024-09-17T05:40:27.059705+0000","last_clean":"2024-09-17T05:40:27.059705+0000","last_became_active":"2024-09-17T05:40:25.194294+0000","last_became_peered":"2024-09-17T05:40:25.194294+0000","last_unstale":"2024-09-17T05:40:27.059705+0000","last_undegraded":"2024-09-17T05:40:27.059705+0000","last_fullsized":"2024-09-17T05:40:27.059705+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:31.686698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258238+0000","last_change":"2024-09-17T05:40:25.186735+0000","last_active":"2024-09-17T05:40:27.258238+0000","last_peered":"2024-09-17T05:40:27.258238+0000","last_clean":"2024-09-17T05:40:27.258238+0000","last_became_active":"2024-09-17T05:40:25.183660+0000","last_became_peered":"2024-09-17T05:40:25.183660+0000","last_unstale":"2024-09-17T05:40:27.258238+0000","last_undegraded":"2024-09-17T05:40:27.258238+0000","last_fullsized":"2024-09-17T05:40:27.258238+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:31.164381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179462+0000","last_change":"2024-09-17T05:40:25.183059+0000","last_active":"2024-09-17T05:40:26.179462+0000","last_peered":"2024-09-17T05:40:26.179462+0000","last_clean":"2024-09-17T05:40:26.179462+0000","last_became_active":"2024-09-17T05:40:25.182536+0000","last_became_peered":"2024-09-17T05:40:25.182536+0000","last_unstale":"2024-09-17T05:40:26.179462+0000","last_undegraded":"2024-09-17T05:40:26.179462+0000","last_fullsized":"2024-09-17T05:40:26.179462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:19.801553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257485+0000","last_change":"2024-09-17T05:40:25.184874+0000","last_active":"2024-09-17T05:40:27.257485+0000","last_peered":"2024-09-17T05:40:27.257485+0000","last_clean":"2024-09-17T05:40:27.257485+0000","last_became_active":"2024-09-17T05:40:25.184586+0000","last_became_peered":"2024-09-17T05:40:25.184586+0000","last_unstale":"2024-09-17T05:40:27.257485+0000","last_undegraded":"2024-09-17T05:40:27.257485+0000","last_fullsized":"2024-09-17T05:40:27.257485+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:05.194660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181301+0000","last_change":"2024-09-17T05:40:25.181547+0000","last_active":"2024-09-17T05:40:26.181301+0000","last_peered":"2024-09-17T05:40:26.181301+0000","last_clean":"2024-09-17T05:40:26.181301+0000","last_became_active":"2024-09-17T05:40:25.178736+0000","last_became_peered":"2024-09-17T05:40:25.178736+0000","last_unstale":"2024-09-17T05:40:26.181301+0000","last_undegraded":"2024-09-17T05:40:26.181301+0000","last_fullsized":"2024-09-17T05:40:26.181301+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:26.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257174+0000","last_change":"2024-09-17T05:40:25.181568+0000","last_active":"2024-09-17T05:40:27.257174+0000","last_peered":"2024-09-17T05:40:27.257174+0000","last_clean":"2024-09-17T05:40:27.257174+0000","last_became_active":"2024-09-17T05:40:25.181282+0000","last_became_peered":"2024-09-17T05:40:25.181282+0000","last_unstale":"2024-09-17T05:40:27.257174+0000","last_undegraded":"2024-09-17T05:40:27.257174+0000","last_fullsized":"2024-09-17T05:40:27.257174+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:12.178318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180814+0000","last_change":"2024-09-17T05:40:25.178429+0000","last_active":"2024-09-17T05:40:26.180814+0000","last_peered":"2024-09-17T05:40:26.180814+0000","last_clean":"2024-09-17T05:40:26.180814+0000","last_became_active":"2024-09-17T05:40:25.178157+0000","last_became_peered":"2024-09-17T05:40:25.178157+0000","last_unstale":"2024-09-17T05:40:26.180814+0000","last_undegraded":"2024-09-17T05:40:26.180814+0000","last_fullsized":"2024-09-17T05:40:26.180814+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:46.260197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181602+0000","last_change":"2024-09-17T05:40:25.182320+0000","last_active":"2024-09-17T05:40:26.181602+0000","last_peered":"2024-09-17T05:40:26.181602+0000","last_clean":"2024-09-17T05:40:26.181602+0000","last_became_active":"2024-09-17T05:40:25.182041+0000","last_became_peered":"2024-09-17T05:40:25.182041+0000","last_unstale":"2024-09-17T05:40:26.181602+0000","last_undegraded":"2024-09-17T05:40:26.181602+0000","last_fullsized":"2024-09-17T05:40:26.181602+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:59.378535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256761+0000","last_change":"2024-09-17T05:40:25.188017+0000","last_active":"2024-09-17T05:40:27.256761+0000","last_peered":"2024-09-17T05:40:27.256761+0000","last_clean":"2024-09-17T05:40:27.256761+0000","last_became_active":"2024-09-17T05:40:25.187748+0000","last_became_peered":"2024-09-17T05:40:25.187748+0000","last_unstale":"2024-09-17T05:40:27.256761+0000","last_undegraded":"2024-09-17T05:40:27.256761+0000","last_fullsized":"2024-09-17T05:40:27.256761+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:44.136630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258494+0000","last_change":"2024-09-17T05:40:25.186435+0000","last_active":"2024-09-17T05:40:27.258494+0000","last_peered":"2024-09-17T05:40:27.258494+0000","last_clean":"2024-09-17T05:40:27.258494+0000","last_became_active":"2024-09-17T05:40:25.183302+0000","last_became_peered":"2024-09-17T05:40:25.183302+0000","last_unstale":"2024-09-17T05:40:27.258494+0000","last_undegraded":"2024-09-17T05:40:27.258494+0000","last_fullsized":"2024-09-17T05:40:27.258494+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:14.329706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180905+0000","last_change":"2024-09-17T05:40:25.170453+0000","last_active":"2024-09-17T05:40:26.180905+0000","last_peered":"2024-09-17T05:40:26.180905+0000","last_clean":"2024-09-17T05:40:26.180905+0000","last_became_active":"2024-09-17T05:40:25.169914+0000","last_became_peered":"2024-09-17T05:40:25.169914+0000","last_unstale":"2024-09-17T05:40:26.180905+0000","last_undegraded":"2024-09-17T05:40:26.180905+0000","last_fullsized":"2024-09-17T05:40:26.180905+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:12.798962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181155+0000","last_change":"2024-09-17T05:40:25.177605+0000","last_active":"2024-09-17T05:40:26.181155+0000","last_peered":"2024-09-17T05:40:26.181155+0000","last_clean":"2024-09-17T05:40:26.181155+0000","last_became_active":"2024-09-17T05:40:25.177331+0000","last_became_peered":"2024-09-17T05:40:25.177331+0000","last_unstale":"2024-09-17T05:40:26.181155+0000","last_undegraded":"2024-09-17T05:40:26.181155+0000","last_fullsized":"2024-09-17T05:40:26.181155+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:42.626670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258571+0000","last_change":"2024-09-17T05:40:25.197121+0000","last_active":"2024-09-17T05:40:27.258571+0000","last_peered":"2024-09-17T05:40:27.258571+0000","last_clean":"2024-09-17T05:40:27.258571+0000","last_became_active":"2024-09-17T05:40:25.196844+0000","last_became_peered":"2024-09-17T05:40:25.196844+0000","last_unstale":"2024-09-17T05:40:27.258571+0000","last_undegraded":"2024-09-17T05:40:27.258571+0000","last_fullsized":"2024-09-17T05:40:27.258571+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:44.688266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"144'4","reported_seq":19,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059703+0000","last_change":"2024-09-17T05:40:24.173930+0000","last_active":"2024-09-17T05:40:27.059703+0000","last_peered":"2024-09-17T05:40:27.059703+0000","last_clean":"2024-09-17T05:40:27.059703+0000","last_became_active":"2024-09-17T05:40:24.172704+0000","last_became_peered":"2024-09-17T05:40:24.172704+0000","last_unstale":"2024-09-17T05:40:27.059703+0000","last_undegraded":"2024-09-17T05:40:27.059703+0000","last_fullsized":"2024-09-17T05:40:27.059703+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:33.170502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183271+0000","last_change":"2024-09-17T05:40:25.179791+0000","last_active":"2024-09-17T05:40:26.183271+0000","last_peered":"2024-09-17T05:40:26.183271+0000","last_clean":"2024-09-17T05:40:26.183271+0000","last_became_active":"2024-09-17T05:40:25.179450+0000","last_became_peered":"2024-09-17T05:40:25.179450+0000","last_unstale":"2024-09-17T05:40:26.183271+0000","last_undegraded":"2024-09-17T05:40:26.183271+0000","last_fullsized":"2024-09-17T05:40:26.183271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.496633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.762120+0000","last_change":"2024-09-17T05:40:24.176345+0000","last_active":"2024-09-17T05:40:26.762120+0000","last_peered":"2024-09-17T05:40:26.762120+0000","last_clean":"2024-09-17T05:40:26.762120+0000","last_became_active":"2024-09-17T05:40:24.176106+0000","last_became_peered":"2024-09-17T05:40:24.176106+0000","last_unstale":"2024-09-17T05:40:26.762120+0000","last_undegraded":"2024-09-17T05:40:26.762120+0000","last_fullsized":"2024-09-17T05:40:26.762120+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:50.800729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761949+0000","last_change":"2024-09-17T05:40:25.188810+0000","last_active":"2024-09-17T05:40:26.761949+0000","last_peered":"2024-09-17T05:40:26.761949+0000","last_clean":"2024-09-17T05:40:26.761949+0000","last_became_active":"2024-09-17T05:40:25.188336+0000","last_became_peered":"2024-09-17T05:40:25.188336+0000","last_unstale":"2024-09-17T05:40:26.761949+0000","last_undegraded":"2024-09-17T05:40:26.761949+0000","last_fullsized":"2024-09-17T05:40:26.761949+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:08.540681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059517+0000","last_change":"2024-09-17T05:40:24.175670+0000","last_active":"2024-09-17T05:40:27.059517+0000","last_peered":"2024-09-17T05:40:27.059517+0000","last_clean":"2024-09-17T05:40:27.059517+0000","last_became_active":"2024-09-17T05:40:24.175367+0000","last_became_peered":"2024-09-17T05:40:24.175367+0000","last_unstale":"2024-09-17T05:40:27.059517+0000","last_undegraded":"2024-09-17T05:40:27.059517+0000","last_fullsized":"2024-09-17T05:40:27.059517+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:25.881116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180176+0000","last_change":"2024-09-17T05:40:25.181851+0000","last_active":"2024-09-17T05:40:26.180176+0000","last_peered":"2024-09-17T05:40:26.180176+0000","last_clean":"2024-09-17T05:40:26.180176+0000","last_became_active":"2024-09-17T05:40:25.181561+0000","last_became_peered":"2024-09-17T05:40:25.181561+0000","last_unstale":"2024-09-17T05:40:26.180176+0000","last_undegraded":"2024-09-17T05:40:26.180176+0000","last_fullsized":"2024-09-17T05:40:26.180176+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:26.035276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059396+0000","last_change":"2024-09-17T05:40:24.176368+0000","last_active":"2024-09-17T05:40:27.059396+0000","last_peered":"2024-09-17T05:40:27.059396+0000","last_clean":"2024-09-17T05:40:27.059396+0000","last_became_active":"2024-09-17T05:40:24.176061+0000","last_became_peered":"2024-09-17T05:40:24.176061+0000","last_unstale":"2024-09-17T05:40:27.059396+0000","last_undegraded":"2024-09-17T05:40:27.059396+0000","last_fullsized":"2024-09-17T05:40:27.059396+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:31.798405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180664+0000","last_change":"2024-09-17T05:40:25.182623+0000","last_active":"2024-09-17T05:40:26.180664+0000","last_peered":"2024-09-17T05:40:26.180664+0000","last_clean":"2024-09-17T05:40:26.180664+0000","last_became_active":"2024-09-17T05:40:25.180102+0000","last_became_peered":"2024-09-17T05:40:25.180102+0000","last_unstale":"2024-09-17T05:40:26.180664+0000","last_undegraded":"2024-09-17T05:40:26.180664+0000","last_fullsized":"2024-09-17T05:40:26.180664+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:33.835538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256777+0000","last_change":"2024-09-17T05:40:24.167785+0000","last_active":"2024-09-17T05:40:27.256777+0000","last_peered":"2024-09-17T05:40:27.256777+0000","last_clean":"2024-09-17T05:40:27.256777+0000","last_became_active":"2024-09-17T05:40:24.167502+0000","last_became_peered":"2024-09-17T05:40:24.167502+0000","last_unstale":"2024-09-17T05:40:27.256777+0000","last_undegraded":"2024-09-17T05:40:27.256777+0000","last_fullsized":"2024-09-17T05:40:27.256777+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:57.640098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179976+0000","last_change":"2024-09-17T05:40:25.190503+0000","last_active":"2024-09-17T05:40:26.179976+0000","last_peered":"2024-09-17T05:40:26.179976+0000","last_clean":"2024-09-17T05:40:26.179976+0000","last_became_active":"2024-09-17T05:40:25.190192+0000","last_became_peered":"2024-09-17T05:40:25.190192+0000","last_unstale":"2024-09-17T05:40:26.179976+0000","last_undegraded":"2024-09-17T05:40:26.179976+0000","last_fullsized":"2024-09-17T05:40:26.179976+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.507268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257552+0000","last_change":"2024-09-17T05:40:24.177481+0000","last_active":"2024-09-17T05:40:27.257552+0000","last_peered":"2024-09-17T05:40:27.257552+0000","last_clean":"2024-09-17T05:40:27.257552+0000","last_became_active":"2024-09-17T05:40:24.177186+0000","last_became_peered":"2024-09-17T05:40:24.177186+0000","last_unstale":"2024-09-17T05:40:27.257552+0000","last_undegraded":"2024-09-17T05:40:27.257552+0000","last_fullsized":"2024-09-17T05:40:27.257552+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:40:01.491102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180751+0000","last_change":"2024-09-17T05:40:25.177145+0000","last_active":"2024-09-17T05:40:26.180751+0000","last_peered":"2024-09-17T05:40:26.180751+0000","last_clean":"2024-09-17T05:40:26.180751+0000","last_became_active":"2024-09-17T05:40:25.176749+0000","last_became_peered":"2024-09-17T05:40:25.176749+0000","last_unstale":"2024-09-17T05:40:26.180751+0000","last_undegraded":"2024-09-17T05:40:26.180751+0000","last_fullsized":"2024-09-17T05:40:26.180751+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:50.785565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761817+0000","last_change":"2024-09-17T05:40:24.162381+0000","last_active":"2024-09-17T05:40:26.761817+0000","last_peered":"2024-09-17T05:40:26.761817+0000","last_clean":"2024-09-17T05:40:26.761817+0000","last_became_active":"2024-09-17T05:40:24.160458+0000","last_became_peered":"2024-09-17T05:40:24.160458+0000","last_unstale":"2024-09-17T05:40:26.761817+0000","last_undegraded":"2024-09-17T05:40:26.761817+0000","last_fullsized":"2024-09-17T05:40:26.761817+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:45.282829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179543+0000","last_change":"2024-09-17T05:40:25.187900+0000","last_active":"2024-09-17T05:40:26.179543+0000","last_peered":"2024-09-17T05:40:26.179543+0000","last_clean":"2024-09-17T05:40:26.179543+0000","last_became_active":"2024-09-17T05:40:25.187628+0000","last_became_peered":"2024-09-17T05:40:25.187628+0000","last_unstale":"2024-09-17T05:40:26.179543+0000","last_undegraded":"2024-09-17T05:40:26.179543+0000","last_fullsized":"2024-09-17T05:40:26.179543+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:29.075371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059111+0000","last_change":"2024-09-17T05:40:24.179128+0000","last_active":"2024-09-17T05:40:27.059111+0000","last_peered":"2024-09-17T05:40:27.059111+0000","last_clean":"2024-09-17T05:40:27.059111+0000","last_became_active":"2024-09-17T05:40:24.178552+0000","last_became_peered":"2024-09-17T05:40:24.178552+0000","last_unstale":"2024-09-17T05:40:27.059111+0000","last_undegraded":"2024-09-17T05:40:27.059111+0000","last_fullsized":"2024-09-17T05:40:27.059111+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:06.290224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180753+0000","last_change":"2024-09-17T05:40:25.192651+0000","last_active":"2024-09-17T05:40:26.180753+0000","last_peered":"2024-09-17T05:40:26.180753+0000","last_clean":"2024-09-17T05:40:26.180753+0000","last_became_active":"2024-09-17T05:40:25.184118+0000","last_became_peered":"2024-09-17T05:40:25.184118+0000","last_unstale":"2024-09-17T05:40:26.180753+0000","last_undegraded":"2024-09-17T05:40:26.180753+0000","last_fullsized":"2024-09-17T05:40:26.180753+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:58.648089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761975+0000","last_change":"2024-09-17T05:40:24.161649+0000","last_active":"2024-09-17T05:40:26.761975+0000","last_peered":"2024-09-17T05:40:26.761975+0000","last_clean":"2024-09-17T05:40:26.761975+0000","last_became_active":"2024-09-17T05:40:24.161393+0000","last_became_peered":"2024-09-17T05:40:24.161393+0000","last_unstale":"2024-09-17T05:40:26.761975+0000","last_undegraded":"2024-09-17T05:40:26.761975+0000","last_fullsized":"2024-09-17T05:40:26.761975+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:56.313784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258537+0000","last_change":"2024-09-17T05:40:25.182851+0000","last_active":"2024-09-17T05:40:27.258537+0000","last_peered":"2024-09-17T05:40:27.258537+0000","last_clean":"2024-09-17T05:40:27.258537+0000","last_became_active":"2024-09-17T05:40:25.182581+0000","last_became_peered":"2024-09-17T05:40:25.182581+0000","last_unstale":"2024-09-17T05:40:27.258537+0000","last_undegraded":"2024-09-17T05:40:27.258537+0000","last_fullsized":"2024-09-17T05:40:27.258537+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:34.728190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258890+0000","last_change":"2024-09-17T05:40:24.169609+0000","last_active":"2024-09-17T05:40:27.258890+0000","last_peered":"2024-09-17T05:40:27.258890+0000","last_clean":"2024-09-17T05:40:27.258890+0000","last_became_active":"2024-09-17T05:40:24.169317+0000","last_became_peered":"2024-09-17T05:40:24.169317+0000","last_unstale":"2024-09-17T05:40:27.258890+0000","last_undegraded":"2024-09-17T05:40:27.258890+0000","last_fullsized":"2024-09-17T05:40:27.258890+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:17.444963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183307+0000","last_change":"2024-09-17T05:40:25.184351+0000","last_active":"2024-09-17T05:40:26.183307+0000","last_peered":"2024-09-17T05:40:26.183307+0000","last_clean":"2024-09-17T05:40:26.183307+0000","last_became_active":"2024-09-17T05:40:25.183920+0000","last_became_peered":"2024-09-17T05:40:25.183920+0000","last_unstale":"2024-09-17T05:40:26.183307+0000","last_undegraded":"2024-09-17T05:40:26.183307+0000","last_fullsized":"2024-09-17T05:40:26.183307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:22.638149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182912+0000","last_change":"2024-09-17T05:40:24.169333+0000","last_active":"2024-09-17T05:40:26.182912+0000","last_peered":"2024-09-17T05:40:26.182912+0000","last_clean":"2024-09-17T05:40:26.182912+0000","last_became_active":"2024-09-17T05:40:24.168219+0000","last_became_peered":"2024-09-17T05:40:24.168219+0000","last_unstale":"2024-09-17T05:40:26.182912+0000","last_undegraded":"2024-09-17T05:40:26.182912+0000","last_fullsized":"2024-09-17T05:40:26.182912+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:58.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179534+0000","last_change":"2024-09-17T05:40:25.195715+0000","last_active":"2024-09-17T05:40:26.179534+0000","last_peered":"2024-09-17T05:40:26.179534+0000","last_clean":"2024-09-17T05:40:26.179534+0000","last_became_active":"2024-09-17T05:40:25.195398+0000","last_became_peered":"2024-09-17T05:40:25.195398+0000","last_unstale":"2024-09-17T05:40:26.179534+0000","last_undegraded":"2024-09-17T05:40:26.179534+0000","last_fullsized":"2024-09-17T05:40:26.179534+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:18.607272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059639+0000","last_change":"2024-09-17T05:40:24.177008+0000","last_active":"2024-09-17T05:40:27.059639+0000","last_peered":"2024-09-17T05:40:27.059639+0000","last_clean":"2024-09-17T05:40:27.059639+0000","last_became_active":"2024-09-17T05:40:24.175458+0000","last_became_peered":"2024-09-17T05:40:24.175458+0000","last_unstale":"2024-09-17T05:40:27.059639+0000","last_undegraded":"2024-09-17T05:40:27.059639+0000","last_fullsized":"2024-09-17T05:40:27.059639+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:34.355730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256708+0000","last_change":"2024-09-17T05:40:25.188264+0000","last_active":"2024-09-17T05:40:27.256708+0000","last_peered":"2024-09-17T05:40:27.256708+0000","last_clean":"2024-09-17T05:40:27.256708+0000","last_became_active":"2024-09-17T05:40:25.187942+0000","last_became_peered":"2024-09-17T05:40:25.187942+0000","last_unstale":"2024-09-17T05:40:27.256708+0000","last_undegraded":"2024-09-17T05:40:27.256708+0000","last_fullsized":"2024-09-17T05:40:27.256708+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:14.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258496+0000","last_change":"2024-09-17T05:40:24.171856+0000","last_active":"2024-09-17T05:40:27.258496+0000","last_peered":"2024-09-17T05:40:27.258496+0000","last_clean":"2024-09-17T05:40:27.258496+0000","last_became_active":"2024-09-17T05:40:24.171580+0000","last_became_peered":"2024-09-17T05:40:24.171580+0000","last_unstale":"2024-09-17T05:40:27.258496+0000","last_undegraded":"2024-09-17T05:40:27.258496+0000","last_fullsized":"2024-09-17T05:40:27.258496+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:47.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179401+0000","last_change":"2024-09-17T05:40:25.180695+0000","last_active":"2024-09-17T05:40:26.179401+0000","last_peered":"2024-09-17T05:40:26.179401+0000","last_clean":"2024-09-17T05:40:26.179401+0000","last_became_active":"2024-09-17T05:40:25.180073+0000","last_became_peered":"2024-09-17T05:40:25.180073+0000","last_unstale":"2024-09-17T05:40:26.179401+0000","last_undegraded":"2024-09-17T05:40:26.179401+0000","last_fullsized":"2024-09-17T05:40:26.179401+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.726434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180986+0000","last_change":"2024-09-17T05:40:24.163423+0000","last_active":"2024-09-17T05:40:26.180986+0000","last_peered":"2024-09-17T05:40:26.180986+0000","last_clean":"2024-09-17T05:40:26.180986+0000","last_became_active":"2024-09-17T05:40:24.163147+0000","last_became_peered":"2024-09-17T05:40:24.163147+0000","last_unstale":"2024-09-17T05:40:26.180986+0000","last_undegraded":"2024-09-17T05:40:26.180986+0000","last_fullsized":"2024-09-17T05:40:26.180986+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:03.350935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179660+0000","last_change":"2024-09-17T05:40:25.182786+0000","last_active":"2024-09-17T05:40:26.179660+0000","last_peered":"2024-09-17T05:40:26.179660+0000","last_clean":"2024-09-17T05:40:26.179660+0000","last_became_active":"2024-09-17T05:40:25.182186+0000","last_became_peered":"2024-09-17T05:40:25.182186+0000","last_unstale":"2024-09-17T05:40:26.179660+0000","last_undegraded":"2024-09-17T05:40:26.179660+0000","last_fullsized":"2024-09-17T05:40:26.179660+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:51.557517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"144'8","reported_seq":23,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256535+0000","last_change":"2024-09-17T05:40:24.159858+0000","last_active":"2024-09-17T05:40:27.256535+0000","last_peered":"2024-09-17T05:40:27.256535+0000","last_clean":"2024-09-17T05:40:27.256535+0000","last_became_active":"2024-09-17T05:40:24.158365+0000","last_became_peered":"2024-09-17T05:40:24.158365+0000","last_unstale":"2024-09-17T05:40:27.256535+0000","last_undegraded":"2024-09-17T05:40:27.256535+0000","last_fullsized":"2024-09-17T05:40:27.256535+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:25.027285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258758+0000","last_change":"2024-09-17T05:40:25.185576+0000","last_active":"2024-09-17T05:40:27.258758+0000","last_peered":"2024-09-17T05:40:27.258758+0000","last_clean":"2024-09-17T05:40:27.258758+0000","last_became_active":"2024-09-17T05:40:25.185029+0000","last_became_peered":"2024-09-17T05:40:25.185029+0000","last_unstale":"2024-09-17T05:40:27.258758+0000","last_undegraded":"2024-09-17T05:40:27.258758+0000","last_fullsized":"2024-09-17T05:40:27.258758+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:08.810996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180721+0000","last_change":"2024-09-17T05:40:24.168831+0000","last_active":"2024-09-17T05:40:26.180721+0000","last_peered":"2024-09-17T05:40:26.180721+0000","last_clean":"2024-09-17T05:40:26.180721+0000","last_became_active":"2024-09-17T05:40:24.168322+0000","last_became_peered":"2024-09-17T05:40:24.168322+0000","last_unstale":"2024-09-17T05:40:26.180721+0000","last_undegraded":"2024-09-17T05:40:26.180721+0000","last_fullsized":"2024-09-17T05:40:26.180721+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:11.790971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181516+0000","last_change":"2024-09-17T05:40:25.169953+0000","last_active":"2024-09-17T05:40:26.181516+0000","last_peered":"2024-09-17T05:40:26.181516+0000","last_clean":"2024-09-17T05:40:26.181516+0000","last_became_active":"2024-09-17T05:40:25.169667+0000","last_became_peered":"2024-09-17T05:40:25.169667+0000","last_unstale":"2024-09-17T05:40:26.181516+0000","last_undegraded":"2024-09-17T05:40:26.181516+0000","last_fullsized":"2024-09-17T05:40:26.181516+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:53.178335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257065+0000","last_change":"2024-09-17T05:40:24.175976+0000","last_active":"2024-09-17T05:40:27.257065+0000","last_peered":"2024-09-17T05:40:27.257065+0000","last_clean":"2024-09-17T05:40:27.257065+0000","last_became_active":"2024-09-17T05:40:24.175623+0000","last_became_peered":"2024-09-17T05:40:24.175623+0000","last_unstale":"2024-09-17T05:40:27.257065+0000","last_undegraded":"2024-09-17T05:40:27.257065+0000","last_fullsized":"2024-09-17T05:40:27.257065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:13.334310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258236+0000","last_change":"2024-09-17T05:40:25.180707+0000","last_active":"2024-09-17T05:40:27.258236+0000","last_peered":"2024-09-17T05:40:27.258236+0000","last_clean":"2024-09-17T05:40:27.258236+0000","last_became_active":"2024-09-17T05:40:25.180323+0000","last_became_peered":"2024-09-17T05:40:25.180323+0000","last_unstale":"2024-09-17T05:40:27.258236+0000","last_undegraded":"2024-09-17T05:40:27.258236+0000","last_fullsized":"2024-09-17T05:40:27.258236+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:28.199285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183232+0000","last_change":"2024-09-17T05:40:24.173858+0000","last_active":"2024-09-17T05:40:26.183232+0000","last_peered":"2024-09-17T05:40:26.183232+0000","last_clean":"2024-09-17T05:40:26.183232+0000","last_became_active":"2024-09-17T05:40:24.173661+0000","last_became_peered":"2024-09-17T05:40:24.173661+0000","last_unstale":"2024-09-17T05:40:26.183232+0000","last_undegraded":"2024-09-17T05:40:26.183232+0000","last_fullsized":"2024-09-17T05:40:26.183232+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:21.200140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.762248+0000","last_change":"2024-09-17T05:40:25.188672+0000","last_active":"2024-09-17T05:40:26.762248+0000","last_peered":"2024-09-17T05:40:26.762248+0000","last_clean":"2024-09-17T05:40:26.762248+0000","last_became_active":"2024-09-17T05:40:25.188365+0000","last_became_peered":"2024-09-17T05:40:25.188365+0000","last_unstale":"2024-09-17T05:40:26.762248+0000","last_undegraded":"2024-09-17T05:40:26.762248+0000","last_fullsized":"2024-09-17T05:40:26.762248+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:14.329706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181668+0000","last_change":"2024-09-17T05:40:24.169329+0000","last_active":"2024-09-17T05:40:26.181668+0000","last_peered":"2024-09-17T05:40:26.181668+0000","last_clean":"2024-09-17T05:40:26.181668+0000","last_became_active":"2024-09-17T05:40:24.169051+0000","last_became_peered":"2024-09-17T05:40:24.169051+0000","last_unstale":"2024-09-17T05:40:26.181668+0000","last_undegraded":"2024-09-17T05:40:26.181668+0000","last_fullsized":"2024-09-17T05:40:26.181668+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:52.335383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258437+0000","last_change":"2024-09-17T05:40:25.193334+0000","last_active":"2024-09-17T05:40:27.258437+0000","last_peered":"2024-09-17T05:40:27.258437+0000","last_clean":"2024-09-17T05:40:27.258437+0000","last_became_active":"2024-09-17T05:40:25.187108+0000","last_became_peered":"2024-09-17T05:40:25.187108+0000","last_unstale":"2024-09-17T05:40:27.258437+0000","last_undegraded":"2024-09-17T05:40:27.258437+0000","last_fullsized":"2024-09-17T05:40:27.258437+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:32.036031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761716+0000","last_change":"2024-09-17T05:40:24.174951+0000","last_active":"2024-09-17T05:40:26.761716+0000","last_peered":"2024-09-17T05:40:26.761716+0000","last_clean":"2024-09-17T05:40:26.761716+0000","last_became_active":"2024-09-17T05:40:24.174436+0000","last_became_peered":"2024-09-17T05:40:24.174436+0000","last_unstale":"2024-09-17T05:40:26.761716+0000","last_undegraded":"2024-09-17T05:40:26.761716+0000","last_fullsized":"2024-09-17T05:40:26.761716+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:23.920494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180846+0000","last_change":"2024-09-17T05:40:25.170463+0000","last_active":"2024-09-17T05:40:26.180846+0000","last_peered":"2024-09-17T05:40:26.180846+0000","last_clean":"2024-09-17T05:40:26.180846+0000","last_became_active":"2024-09-17T05:40:25.170138+0000","last_became_peered":"2024-09-17T05:40:25.170138+0000","last_unstale":"2024-09-17T05:40:26.180846+0000","last_undegraded":"2024-09-17T05:40:26.180846+0000","last_fullsized":"2024-09-17T05:40:26.180846+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:36.252315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179600+0000","last_change":"2024-09-17T05:40:24.169832+0000","last_active":"2024-09-17T05:40:26.179600+0000","last_peered":"2024-09-17T05:40:26.179600+0000","last_clean":"2024-09-17T05:40:26.179600+0000","last_became_active":"2024-09-17T05:40:24.169570+0000","last_became_peered":"2024-09-17T05:40:24.169570+0000","last_unstale":"2024-09-17T05:40:26.179600+0000","last_undegraded":"2024-09-17T05:40:26.179600+0000","last_fullsized":"2024-09-17T05:40:26.179600+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:52.170344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257866+0000","last_change":"2024-09-17T05:40:25.184998+0000","last_active":"2024-09-17T05:40:27.257866+0000","last_peered":"2024-09-17T05:40:27.257866+0000","last_clean":"2024-09-17T05:40:27.257866+0000","last_became_active":"2024-09-17T05:40:25.184718+0000","last_became_peered":"2024-09-17T05:40:25.184718+0000","last_unstale":"2024-09-17T05:40:27.257866+0000","last_undegraded":"2024-09-17T05:40:27.257866+0000","last_fullsized":"2024-09-17T05:40:27.257866+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:53.661119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179602+0000","last_change":"2024-09-17T05:40:24.167353+0000","last_active":"2024-09-17T05:40:26.179602+0000","last_peered":"2024-09-17T05:40:26.179602+0000","last_clean":"2024-09-17T05:40:26.179602+0000","last_became_active":"2024-09-17T05:40:24.167074+0000","last_became_peered":"2024-09-17T05:40:24.167074+0000","last_unstale":"2024-09-17T05:40:26.179602+0000","last_undegraded":"2024-09-17T05:40:26.179602+0000","last_fullsized":"2024-09-17T05:40:26.179602+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:41.618679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181002+0000","last_change":"2024-09-17T05:40:25.173599+0000","last_active":"2024-09-17T05:40:26.181002+0000","last_peered":"2024-09-17T05:40:26.181002+0000","last_clean":"2024-09-17T05:40:26.181002+0000","last_became_active":"2024-09-17T05:40:25.173358+0000","last_became_peered":"2024-09-17T05:40:25.173358+0000","last_unstale":"2024-09-17T05:40:26.181002+0000","last_undegraded":"2024-09-17T05:40:26.181002+0000","last_fullsized":"2024-09-17T05:40:26.181002+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:25.486183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179536+0000","last_change":"2024-09-17T05:40:24.173086+0000","last_active":"2024-09-17T05:40:26.179536+0000","last_peered":"2024-09-17T05:40:26.179536+0000","last_clean":"2024-09-17T05:40:26.179536+0000","last_became_active":"2024-09-17T05:40:24.171758+0000","last_became_peered":"2024-09-17T05:40:24.171758+0000","last_unstale":"2024-09-17T05:40:26.179536+0000","last_undegraded":"2024-09-17T05:40:26.179536+0000","last_fullsized":"2024-09-17T05:40:26.179536+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:42.281444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180826+0000","last_change":"2024-09-17T05:40:25.193176+0000","last_active":"2024-09-17T05:40:26.180826+0000","last_peered":"2024-09-17T05:40:26.180826+0000","last_clean":"2024-09-17T05:40:26.180826+0000","last_became_active":"2024-09-17T05:40:25.192893+0000","last_became_peered":"2024-09-17T05:40:25.192893+0000","last_unstale":"2024-09-17T05:40:26.180826+0000","last_undegraded":"2024-09-17T05:40:26.180826+0000","last_fullsized":"2024-09-17T05:40:26.180826+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:42.278904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180848+0000","last_change":"2024-09-17T05:40:24.168573+0000","last_active":"2024-09-17T05:40:26.180848+0000","last_peered":"2024-09-17T05:40:26.180848+0000","last_clean":"2024-09-17T05:40:26.180848+0000","last_became_active":"2024-09-17T05:40:24.167948+0000","last_became_peered":"2024-09-17T05:40:24.167948+0000","last_unstale":"2024-09-17T05:40:26.180848+0000","last_undegraded":"2024-09-17T05:40:26.180848+0000","last_fullsized":"2024-09-17T05:40:26.180848+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:24.478191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180780+0000","last_change":"2024-09-17T05:40:25.192972+0000","last_active":"2024-09-17T05:40:26.180780+0000","last_peered":"2024-09-17T05:40:26.180780+0000","last_clean":"2024-09-17T05:40:26.180780+0000","last_became_active":"2024-09-17T05:40:25.192566+0000","last_became_peered":"2024-09-17T05:40:25.192566+0000","last_unstale":"2024-09-17T05:40:26.180780+0000","last_undegraded":"2024-09-17T05:40:26.180780+0000","last_fullsized":"2024-09-17T05:40:26.180780+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:12.582292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180993+0000","last_change":"2024-09-17T05:40:24.175084+0000","last_active":"2024-09-17T05:40:26.180993+0000","last_peered":"2024-09-17T05:40:26.180993+0000","last_clean":"2024-09-17T05:40:26.180993+0000","last_became_active":"2024-09-17T05:40:24.174827+0000","last_became_peered":"2024-09-17T05:40:24.174827+0000","last_unstale":"2024-09-17T05:40:26.180993+0000","last_undegraded":"2024-09-17T05:40:26.180993+0000","last_fullsized":"2024-09-17T05:40:26.180993+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:43.680275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256953+0000","last_change":"2024-09-17T05:40:25.184916+0000","last_active":"2024-09-17T05:40:27.256953+0000","last_peered":"2024-09-17T05:40:27.256953+0000","last_clean":"2024-09-17T05:40:27.256953+0000","last_became_active":"2024-09-17T05:40:25.184598+0000","last_became_peered":"2024-09-17T05:40:25.184598+0000","last_unstale":"2024-09-17T05:40:27.256953+0000","last_undegraded":"2024-09-17T05:40:27.256953+0000","last_fullsized":"2024-09-17T05:40:27.256953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:02.388462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183134+0000","last_change":"2024-09-17T05:40:24.168536+0000","last_active":"2024-09-17T05:40:26.183134+0000","last_peered":"2024-09-17T05:40:26.183134+0000","last_clean":"2024-09-17T05:40:26.183134+0000","last_became_active":"2024-09-17T05:40:24.168121+0000","last_became_peered":"2024-09-17T05:40:24.168121+0000","last_unstale":"2024-09-17T05:40:26.183134+0000","last_undegraded":"2024-09-17T05:40:26.183134+0000","last_fullsized":"2024-09-17T05:40:26.183134+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:52.244624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256986+0000","last_change":"2024-09-17T05:40:25.184243+0000","last_active":"2024-09-17T05:40:27.256986+0000","last_peered":"2024-09-17T05:40:27.256986+0000","last_clean":"2024-09-17T05:40:27.256986+0000","last_became_active":"2024-09-17T05:40:25.183980+0000","last_became_peered":"2024-09-17T05:40:25.183980+0000","last_unstale":"2024-09-17T05:40:27.256986+0000","last_undegraded":"2024-09-17T05:40:27.256986+0000","last_fullsized":"2024-09-17T05:40:27.256986+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:53.918071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183037+0000","last_change":"2024-09-17T05:40:24.167195+0000","last_active":"2024-09-17T05:40:26.183037+0000","last_peered":"2024-09-17T05:40:26.183037+0000","last_clean":"2024-09-17T05:40:26.183037+0000","last_became_active":"2024-09-17T05:40:24.166930+0000","last_became_peered":"2024-09-17T05:40:24.166930+0000","last_unstale":"2024-09-17T05:40:26.183037+0000","last_undegraded":"2024-09-17T05:40:26.183037+0000","last_fullsized":"2024-09-17T05:40:26.183037+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:47.316470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183088+0000","last_change":"2024-09-17T05:40:25.177997+0000","last_active":"2024-09-17T05:40:26.183088+0000","last_peered":"2024-09-17T05:40:26.183088+0000","last_clean":"2024-09-17T05:40:26.183088+0000","last_became_active":"2024-09-17T05:40:25.177737+0000","last_became_peered":"2024-09-17T05:40:25.177737+0000","last_unstale":"2024-09-17T05:40:26.183088+0000","last_undegraded":"2024-09-17T05:40:26.183088+0000","last_fullsized":"2024-09-17T05:40:26.183088+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.829551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180907+0000","last_change":"2024-09-17T05:40:24.161046+0000","last_active":"2024-09-17T05:40:26.180907+0000","last_peered":"2024-09-17T05:40:26.180907+0000","last_clean":"2024-09-17T05:40:26.180907+0000","last_became_active":"2024-09-17T05:40:24.160629+0000","last_became_peered":"2024-09-17T05:40:24.160629+0000","last_unstale":"2024-09-17T05:40:26.180907+0000","last_undegraded":"2024-09-17T05:40:26.180907+0000","last_fullsized":"2024-09-17T05:40:26.180907+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:27.191294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180160+0000","last_change":"2024-09-17T05:40:25.190186+0000","last_active":"2024-09-17T05:40:26.180160+0000","last_peered":"2024-09-17T05:40:26.180160+0000","last_clean":"2024-09-17T05:40:26.180160+0000","last_became_active":"2024-09-17T05:40:25.189902+0000","last_became_peered":"2024-09-17T05:40:25.189902+0000","last_unstale":"2024-09-17T05:40:26.180160+0000","last_undegraded":"2024-09-17T05:40:26.180160+0000","last_fullsized":"2024-09-17T05:40:26.180160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:55.930768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059266+0000","last_change":"2024-09-17T05:40:24.176177+0000","last_active":"2024-09-17T05:40:27.059266+0000","last_peered":"2024-09-17T05:40:27.059266+0000","last_clean":"2024-09-17T05:40:27.059266+0000","last_became_active":"2024-09-17T05:40:24.175077+0000","last_became_peered":"2024-09-17T05:40:24.175077+0000","last_unstale":"2024-09-17T05:40:27.059266+0000","last_undegraded":"2024-09-17T05:40:27.059266+0000","last_fullsized":"2024-09-17T05:40:27.059266+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:08.803589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179495+0000","last_change":"2024-09-17T05:40:25.180978+0000","last_active":"2024-09-17T05:40:26.179495+0000","last_peered":"2024-09-17T05:40:26.179495+0000","last_clean":"2024-09-17T05:40:26.179495+0000","last_became_active":"2024-09-17T05:40:25.180431+0000","last_became_peered":"2024-09-17T05:40:25.180431+0000","last_unstale":"2024-09-17T05:40:26.179495+0000","last_undegraded":"2024-09-17T05:40:26.179495+0000","last_fullsized":"2024-09-17T05:40:26.179495+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:56.582632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257885+0000","last_change":"2024-09-17T05:40:24.172286+0000","last_active":"2024-09-17T05:40:27.257885+0000","last_peered":"2024-09-17T05:40:27.257885+0000","last_clean":"2024-09-17T05:40:27.257885+0000","last_became_active":"2024-09-17T05:40:24.171998+0000","last_became_peered":"2024-09-17T05:40:24.171998+0000","last_unstale":"2024-09-17T05:40:27.257885+0000","last_undegraded":"2024-09-17T05:40:27.257885+0000","last_fullsized":"2024-09-17T05:40:27.257885+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:51.086129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059184+0000","last_change":"2024-09-17T05:40:25.181389+0000","last_active":"2024-09-17T05:40:27.059184+0000","last_peered":"2024-09-17T05:40:27.059184+0000","last_clean":"2024-09-17T05:40:27.059184+0000","last_became_active":"2024-09-17T05:40:25.181136+0000","last_became_peered":"2024-09-17T05:40:25.181136+0000","last_unstale":"2024-09-17T05:40:27.059184+0000","last_undegraded":"2024-09-17T05:40:27.059184+0000","last_fullsized":"2024-09-17T05:40:27.059184+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:25.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761703+0000","last_change":"2024-09-17T05:40:24.174394+0000","last_active":"2024-09-17T05:40:26.761703+0000","last_peered":"2024-09-17T05:40:26.761703+0000","last_clean":"2024-09-17T05:40:26.761703+0000","last_became_active":"2024-09-17T05:40:24.174122+0000","last_became_peered":"2024-09-17T05:40:24.174122+0000","last_unstale":"2024-09-17T05:40:26.761703+0000","last_undegraded":"2024-09-17T05:40:26.761703+0000","last_fullsized":"2024-09-17T05:40:26.761703+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:08.718443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183130+0000","last_change":"2024-09-17T05:40:25.185149+0000","last_active":"2024-09-17T05:40:26.183130+0000","last_peered":"2024-09-17T05:40:26.183130+0000","last_clean":"2024-09-17T05:40:26.183130+0000","last_became_active":"2024-09-17T05:40:25.184781+0000","last_became_peered":"2024-09-17T05:40:25.184781+0000","last_unstale":"2024-09-17T05:40:26.183130+0000","last_undegraded":"2024-09-17T05:40:26.183130+0000","last_fullsized":"2024-09-17T05:40:26.183130+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.885218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"144'3","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258621+0000","last_change":"2024-09-17T05:40:24.173119+0000","last_active":"2024-09-17T05:40:27.258621+0000","last_peered":"2024-09-17T05:40:27.258621+0000","last_clean":"2024-09-17T05:40:27.258621+0000","last_became_active":"2024-09-17T05:40:24.172873+0000","last_became_peered":"2024-09-17T05:40:24.172873+0000","last_unstale":"2024-09-17T05:40:27.258621+0000","last_undegraded":"2024-09-17T05:40:27.258621+0000","last_fullsized":"2024-09-17T05:40:27.258621+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:54.973445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":345,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257437+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:40:27.257437+0000","last_peered":"2024-09-17T05:40:27.257437+0000","last_clean":"2024-09-17T05:40:27.257437+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:40:27.257437+0000","last_undegraded":"2024-09-17T05:40:27.257437+0000","last_fullsized":"2024-09-17T05:40:27.257437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127292,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244548,"statfs":{"total":96636764160,"available":96506417152,"internally_reserved":0,"allocated":1806336,"data_stored":799394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509630,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":155064,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216776,"statfs":{"total":96636764160,"available":96477978624,"internally_reserved":0,"allocated":1806336,"data_stored":799394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":156957126},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509630,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131468,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240372,"statfs":{"total":96636764160,"available":96502140928,"internally_reserved":0,"allocated":1826816,"data_stored":801610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135620,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236220,"statfs":{"total":96636764160,"available":96497889280,"internally_reserved":0,"allocated":1814528,"data_stored":800010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131456,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240384,"statfs":{"total":96636764160,"available":96502153216,"internally_reserved":0,"allocated":1810432,"data_stored":799416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509631,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145920,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225920,"statfs":{"total":96636764160,"available":96487342080,"internally_reserved":0,"allocated":2404352,"data_stored":1389832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509629,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149500,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222340,"statfs":{"total":96636764160,"available":96483676160,"internally_reserved":0,"allocated":1810432,"data_stored":799484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":151255499},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119580,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252260,"statfs":{"total":96636764160,"available":96514314240,"internally_reserved":0,"allocated":2424832,"data_stored":1391568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:40:31.129 INFO:tasks.ceph.mds.c.smithi122.stderr:2024-09-17T05:40:31.133+0000 7f5d83a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:40:33.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:40:34.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.016+0000 7f2d38c00640 1 -- 172.21.15.122:0/996471787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340ced80 msgr2=0x7f2d340ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.016+0000 7f2d38c00640 1 --2- 172.21.15.122:0/996471787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340ced80 0x7f2d340ae9b0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009900 tx=0x7f2d1c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.016+0000 7f2d38c00640 1 -- 172.21.15.122:0/996471787 shutdown_connections 2024-09-17T05:40:34.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.016+0000 7f2d38c00640 1 --2- 172.21.15.122:0/996471787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d340cdb30 0x7f2d340aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.016+0000 7f2d38c00640 1 --2- 172.21.15.122:0/996471787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340ced80 0x7f2d340ae9b0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.016+0000 7f2d38c00640 1 -- 172.21.15.122:0/996471787 >> 172.21.15.122:0/996471787 conn(0x7f2d340d1dc0 msgr2=0x7f2d340d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:34.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.016+0000 7f2d38c00640 1 -- 172.21.15.122:0/996471787 shutdown_connections 2024-09-17T05:40:34.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.016+0000 7f2d38c00640 1 -- 172.21.15.122:0/996471787 wait complete. 2024-09-17T05:40:34.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 Processor -- start 2024-09-17T05:40:34.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 -- start start 2024-09-17T05:40:34.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340cdb30 0x7f2d3406fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d340ced80 0x7f2d341fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d3415bea0 0x7f2d340c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34169340 con 0x7f2d340cdb30 2024-09-17T05:40:34.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341691c0 con 0x7f2d340ced80 2024-09-17T05:40:34.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2d341694c0 con 0x7f2d3415bea0 2024-09-17T05:40:34.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340cdb30 0x7f2d3406fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340cdb30 0x7f2d3406fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58760/0 (socket says 172.21.15.122:58760) 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 -- 172.21.15.122:0/2878306032 learned_addr learned my addr 172.21.15.122:0/2878306032 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d32800640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d3415bea0 0x7f2d340c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d29400640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d340ced80 0x7f2d341fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 -- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d3415bea0 msgr2=0x7f2d340c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d3415bea0 0x7f2d340c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 -- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d340ced80 msgr2=0x7f2d341fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d340ced80 0x7f2d341fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 -- 172.21.15.122:0/2878306032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d341fbe80 con 0x7f2d340cdb30 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d29400640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d340ced80 0x7f2d341fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d31e00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340cdb30 0x7f2d3406fa00 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c730 tx=0x7f2d2400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d2aa00640 1 -- 172.21.15.122:0/2878306032 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24003b80 con 0x7f2d340cdb30 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d2aa00640 1 -- 172.21.15.122:0/2878306032 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d24003d40 con 0x7f2d340cdb30 2024-09-17T05:40:34.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d2aa00640 1 -- 172.21.15.122:0/2878306032 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24005330 con 0x7f2d340cdb30 2024-09-17T05:40:34.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d341560d0 con 0x7f2d340cdb30 2024-09-17T05:40:34.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d340b1610 con 0x7f2d340cdb30 2024-09-17T05:40:34.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.020+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d341566b0 con 0x7f2d340cdb30 2024-09-17T05:40:34.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.024+0000 7f2d2aa00640 1 -- 172.21.15.122:0/2878306032 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f2d240054d0 con 0x7f2d340cdb30 2024-09-17T05:40:34.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.024+0000 7f2d2aa00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d100754e0 0x7f2d100779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.024+0000 7f2d2aa00640 1 -- 172.21.15.122:0/2878306032 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f2d24093170 con 0x7f2d340cdb30 2024-09-17T05:40:34.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.024+0000 7f2d29400640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d100754e0 0x7f2d100779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.024+0000 7f2d29400640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d100754e0 0x7f2d100779a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0af000 tx=0x7f2d1c01b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:34.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.028+0000 7f2d2aa00640 1 -- 172.21.15.122:0/2878306032 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2d341fbe80 con 0x7f2d340cdb30 2024-09-17T05:40:34.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.192+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d340ce670 con 0x7f2d100754e0 2024-09-17T05:40:34.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d2aa00640 1 -- 172.21.15.122:0/2878306032 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252497 (secure 0 0 0) 0x7f2d340ce670 con 0x7f2d100754e0 2024-09-17T05:40:34.198 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:34.200 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d100754e0 msgr2=0x7f2d100779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d100754e0 0x7f2d100779a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0af000 tx=0x7f2d1c01b120 comp rx=0 tx=0).stop 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340cdb30 msgr2=0x7f2d3406fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340cdb30 0x7f2d3406fa00 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c730 tx=0x7f2d2400cb30 comp rx=0 tx=0).stop 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 shutdown_connections 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2d100754e0 0x7f2d100779a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d3415bea0 0x7f2d340c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d340ced80 0x7f2d341fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 --2- 172.21.15.122:0/2878306032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d340cdb30 0x7f2d3406fa00 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 >> 172.21.15.122:0/2878306032 conn(0x7f2d340d1dc0 msgr2=0x7f2d340b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:34.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 shutdown_connections 2024-09-17T05:40:34.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.200+0000 7f2d38c00640 1 -- 172.21.15.122:0/2878306032 wait complete. 2024-09-17T05:40:34.220 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":608,"stamp":"2024-09-17T05:40:32.756841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1095900,"kb_used_data":15336,"kb_used_omap":12,"kb_used_meta":1080371,"kb_avail":753878820,"statfs":{"total":773094113280,"available":771971911680,"internally_reserved":0,"allocated":15704064,"data_stored":7580708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1106300470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001979"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257299+0000","last_change":"2024-09-17T05:40:25.183502+0000","last_active":"2024-09-17T05:40:27.257299+0000","last_peered":"2024-09-17T05:40:27.257299+0000","last_clean":"2024-09-17T05:40:27.257299+0000","last_became_active":"2024-09-17T05:40:25.183249+0000","last_became_peered":"2024-09-17T05:40:25.183249+0000","last_unstale":"2024-09-17T05:40:27.257299+0000","last_undegraded":"2024-09-17T05:40:27.257299+0000","last_fullsized":"2024-09-17T05:40:27.257299+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:52.938668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181051+0000","last_change":"2024-09-17T05:40:25.184193+0000","last_active":"2024-09-17T05:40:26.181051+0000","last_peered":"2024-09-17T05:40:26.181051+0000","last_clean":"2024-09-17T05:40:26.181051+0000","last_became_active":"2024-09-17T05:40:25.182207+0000","last_became_peered":"2024-09-17T05:40:25.182207+0000","last_unstale":"2024-09-17T05:40:26.181051+0000","last_undegraded":"2024-09-17T05:40:26.181051+0000","last_fullsized":"2024-09-17T05:40:26.181051+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:43.289435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059543+0000","last_change":"2024-09-17T05:40:25.190821+0000","last_active":"2024-09-17T05:40:27.059543+0000","last_peered":"2024-09-17T05:40:27.059543+0000","last_clean":"2024-09-17T05:40:27.059543+0000","last_became_active":"2024-09-17T05:40:25.189507+0000","last_became_peered":"2024-09-17T05:40:25.189507+0000","last_unstale":"2024-09-17T05:40:27.059543+0000","last_undegraded":"2024-09-17T05:40:27.059543+0000","last_fullsized":"2024-09-17T05:40:27.059543+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:05.796307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180298+0000","last_change":"2024-09-17T05:40:25.193673+0000","last_active":"2024-09-17T05:40:26.180298+0000","last_peered":"2024-09-17T05:40:26.180298+0000","last_clean":"2024-09-17T05:40:26.180298+0000","last_became_active":"2024-09-17T05:40:25.190809+0000","last_became_peered":"2024-09-17T05:40:25.190809+0000","last_unstale":"2024-09-17T05:40:26.180298+0000","last_undegraded":"2024-09-17T05:40:26.180298+0000","last_fullsized":"2024-09-17T05:40:26.180298+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:14.342301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257680+0000","last_change":"2024-09-17T05:40:25.192148+0000","last_active":"2024-09-17T05:40:27.257680+0000","last_peered":"2024-09-17T05:40:27.257680+0000","last_clean":"2024-09-17T05:40:27.257680+0000","last_became_active":"2024-09-17T05:40:25.191696+0000","last_became_peered":"2024-09-17T05:40:25.191696+0000","last_unstale":"2024-09-17T05:40:27.257680+0000","last_undegraded":"2024-09-17T05:40:27.257680+0000","last_fullsized":"2024-09-17T05:40:27.257680+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:35:01.410825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761849+0000","last_change":"2024-09-17T05:40:25.182263+0000","last_active":"2024-09-17T05:40:26.761849+0000","last_peered":"2024-09-17T05:40:26.761849+0000","last_clean":"2024-09-17T05:40:26.761849+0000","last_became_active":"2024-09-17T05:40:25.181213+0000","last_became_peered":"2024-09-17T05:40:25.181213+0000","last_unstale":"2024-09-17T05:40:26.761849+0000","last_undegraded":"2024-09-17T05:40:26.761849+0000","last_fullsized":"2024-09-17T05:40:26.761849+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:53.661119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182643+0000","last_change":"2024-09-17T05:40:25.183672+0000","last_active":"2024-09-17T05:40:26.182643+0000","last_peered":"2024-09-17T05:40:26.182643+0000","last_clean":"2024-09-17T05:40:26.182643+0000","last_became_active":"2024-09-17T05:40:25.183400+0000","last_became_peered":"2024-09-17T05:40:25.183400+0000","last_unstale":"2024-09-17T05:40:26.182643+0000","last_undegraded":"2024-09-17T05:40:26.182643+0000","last_fullsized":"2024-09-17T05:40:26.182643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:25.692977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256704+0000","last_change":"2024-09-17T05:40:25.188354+0000","last_active":"2024-09-17T05:40:27.256704+0000","last_peered":"2024-09-17T05:40:27.256704+0000","last_clean":"2024-09-17T05:40:27.256704+0000","last_became_active":"2024-09-17T05:40:25.188059+0000","last_became_peered":"2024-09-17T05:40:25.188059+0000","last_unstale":"2024-09-17T05:40:27.256704+0000","last_undegraded":"2024-09-17T05:40:27.256704+0000","last_fullsized":"2024-09-17T05:40:27.256704+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:57.904271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761948+0000","last_change":"2024-09-17T05:40:25.180756+0000","last_active":"2024-09-17T05:40:26.761948+0000","last_peered":"2024-09-17T05:40:26.761948+0000","last_clean":"2024-09-17T05:40:26.761948+0000","last_became_active":"2024-09-17T05:40:25.180399+0000","last_became_peered":"2024-09-17T05:40:25.180399+0000","last_unstale":"2024-09-17T05:40:26.761948+0000","last_undegraded":"2024-09-17T05:40:26.761948+0000","last_fullsized":"2024-09-17T05:40:26.761948+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:26.035276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059533+0000","last_change":"2024-09-17T05:40:25.174808+0000","last_active":"2024-09-17T05:40:27.059533+0000","last_peered":"2024-09-17T05:40:27.059533+0000","last_clean":"2024-09-17T05:40:27.059533+0000","last_became_active":"2024-09-17T05:40:25.174530+0000","last_became_peered":"2024-09-17T05:40:25.174530+0000","last_unstale":"2024-09-17T05:40:27.059533+0000","last_undegraded":"2024-09-17T05:40:27.059533+0000","last_fullsized":"2024-09-17T05:40:27.059533+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:28.976639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.762120+0000","last_change":"2024-09-17T05:40:25.185179+0000","last_active":"2024-09-17T05:40:26.762120+0000","last_peered":"2024-09-17T05:40:26.762120+0000","last_clean":"2024-09-17T05:40:26.762120+0000","last_became_active":"2024-09-17T05:40:25.184771+0000","last_became_peered":"2024-09-17T05:40:25.184771+0000","last_unstale":"2024-09-17T05:40:26.762120+0000","last_undegraded":"2024-09-17T05:40:26.762120+0000","last_fullsized":"2024-09-17T05:40:26.762120+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:14.342301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258498+0000","last_change":"2024-09-17T05:40:25.182688+0000","last_active":"2024-09-17T05:40:27.258498+0000","last_peered":"2024-09-17T05:40:27.258498+0000","last_clean":"2024-09-17T05:40:27.258498+0000","last_became_active":"2024-09-17T05:40:25.182418+0000","last_became_peered":"2024-09-17T05:40:25.182418+0000","last_unstale":"2024-09-17T05:40:27.258498+0000","last_undegraded":"2024-09-17T05:40:27.258498+0000","last_fullsized":"2024-09-17T05:40:27.258498+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:59.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257872+0000","last_change":"2024-09-17T05:40:25.184777+0000","last_active":"2024-09-17T05:40:27.257872+0000","last_peered":"2024-09-17T05:40:27.257872+0000","last_clean":"2024-09-17T05:40:27.257872+0000","last_became_active":"2024-09-17T05:40:25.184082+0000","last_became_peered":"2024-09-17T05:40:25.184082+0000","last_unstale":"2024-09-17T05:40:27.257872+0000","last_undegraded":"2024-09-17T05:40:27.257872+0000","last_fullsized":"2024-09-17T05:40:27.257872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:04.358926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258218+0000","last_change":"2024-09-17T05:40:25.179501+0000","last_active":"2024-09-17T05:40:27.258218+0000","last_peered":"2024-09-17T05:40:27.258218+0000","last_clean":"2024-09-17T05:40:27.258218+0000","last_became_active":"2024-09-17T05:40:25.179221+0000","last_became_peered":"2024-09-17T05:40:25.179221+0000","last_unstale":"2024-09-17T05:40:27.258218+0000","last_undegraded":"2024-09-17T05:40:27.258218+0000","last_fullsized":"2024-09-17T05:40:27.258218+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:08.540681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257617+0000","last_change":"2024-09-17T05:40:25.188964+0000","last_active":"2024-09-17T05:40:27.257617+0000","last_peered":"2024-09-17T05:40:27.257617+0000","last_clean":"2024-09-17T05:40:27.257617+0000","last_became_active":"2024-09-17T05:40:25.188669+0000","last_became_peered":"2024-09-17T05:40:25.188669+0000","last_unstale":"2024-09-17T05:40:27.257617+0000","last_undegraded":"2024-09-17T05:40:27.257617+0000","last_fullsized":"2024-09-17T05:40:27.257617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:31.199093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182207+0000","last_change":"2024-09-17T05:40:25.175534+0000","last_active":"2024-09-17T05:40:26.182207+0000","last_peered":"2024-09-17T05:40:26.182207+0000","last_clean":"2024-09-17T05:40:26.182207+0000","last_became_active":"2024-09-17T05:40:25.175183+0000","last_became_peered":"2024-09-17T05:40:25.175183+0000","last_unstale":"2024-09-17T05:40:26.182207+0000","last_undegraded":"2024-09-17T05:40:26.182207+0000","last_fullsized":"2024-09-17T05:40:26.182207+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:42.949912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059773+0000","last_change":"2024-09-17T05:40:25.189648+0000","last_active":"2024-09-17T05:40:27.059773+0000","last_peered":"2024-09-17T05:40:27.059773+0000","last_clean":"2024-09-17T05:40:27.059773+0000","last_became_active":"2024-09-17T05:40:25.189407+0000","last_became_peered":"2024-09-17T05:40:25.189407+0000","last_unstale":"2024-09-17T05:40:27.059773+0000","last_undegraded":"2024-09-17T05:40:27.059773+0000","last_fullsized":"2024-09-17T05:40:27.059773+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:01.239661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258367+0000","last_change":"2024-09-17T05:40:25.184991+0000","last_active":"2024-09-17T05:40:27.258367+0000","last_peered":"2024-09-17T05:40:27.258367+0000","last_clean":"2024-09-17T05:40:27.258367+0000","last_became_active":"2024-09-17T05:40:25.184229+0000","last_became_peered":"2024-09-17T05:40:25.184229+0000","last_unstale":"2024-09-17T05:40:27.258367+0000","last_undegraded":"2024-09-17T05:40:27.258367+0000","last_fullsized":"2024-09-17T05:40:27.258367+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:53.343374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256999+0000","last_change":"2024-09-17T05:40:25.180348+0000","last_active":"2024-09-17T05:40:27.256999+0000","last_peered":"2024-09-17T05:40:27.256999+0000","last_clean":"2024-09-17T05:40:27.256999+0000","last_became_active":"2024-09-17T05:40:25.180037+0000","last_became_peered":"2024-09-17T05:40:25.180037+0000","last_unstale":"2024-09-17T05:40:27.256999+0000","last_undegraded":"2024-09-17T05:40:27.256999+0000","last_fullsized":"2024-09-17T05:40:27.256999+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:38.044516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256872+0000","last_change":"2024-09-17T05:40:25.181664+0000","last_active":"2024-09-17T05:40:27.256872+0000","last_peered":"2024-09-17T05:40:27.256872+0000","last_clean":"2024-09-17T05:40:27.256872+0000","last_became_active":"2024-09-17T05:40:25.181358+0000","last_became_peered":"2024-09-17T05:40:25.181358+0000","last_unstale":"2024-09-17T05:40:27.256872+0000","last_undegraded":"2024-09-17T05:40:27.256872+0000","last_fullsized":"2024-09-17T05:40:27.256872+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:57.662374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059705+0000","last_change":"2024-09-17T05:40:25.194561+0000","last_active":"2024-09-17T05:40:27.059705+0000","last_peered":"2024-09-17T05:40:27.059705+0000","last_clean":"2024-09-17T05:40:27.059705+0000","last_became_active":"2024-09-17T05:40:25.194294+0000","last_became_peered":"2024-09-17T05:40:25.194294+0000","last_unstale":"2024-09-17T05:40:27.059705+0000","last_undegraded":"2024-09-17T05:40:27.059705+0000","last_fullsized":"2024-09-17T05:40:27.059705+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:31.686698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258238+0000","last_change":"2024-09-17T05:40:25.186735+0000","last_active":"2024-09-17T05:40:27.258238+0000","last_peered":"2024-09-17T05:40:27.258238+0000","last_clean":"2024-09-17T05:40:27.258238+0000","last_became_active":"2024-09-17T05:40:25.183660+0000","last_became_peered":"2024-09-17T05:40:25.183660+0000","last_unstale":"2024-09-17T05:40:27.258238+0000","last_undegraded":"2024-09-17T05:40:27.258238+0000","last_fullsized":"2024-09-17T05:40:27.258238+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:57:31.164381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179462+0000","last_change":"2024-09-17T05:40:25.183059+0000","last_active":"2024-09-17T05:40:26.179462+0000","last_peered":"2024-09-17T05:40:26.179462+0000","last_clean":"2024-09-17T05:40:26.179462+0000","last_became_active":"2024-09-17T05:40:25.182536+0000","last_became_peered":"2024-09-17T05:40:25.182536+0000","last_unstale":"2024-09-17T05:40:26.179462+0000","last_undegraded":"2024-09-17T05:40:26.179462+0000","last_fullsized":"2024-09-17T05:40:26.179462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:19.801553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257485+0000","last_change":"2024-09-17T05:40:25.184874+0000","last_active":"2024-09-17T05:40:27.257485+0000","last_peered":"2024-09-17T05:40:27.257485+0000","last_clean":"2024-09-17T05:40:27.257485+0000","last_became_active":"2024-09-17T05:40:25.184586+0000","last_became_peered":"2024-09-17T05:40:25.184586+0000","last_unstale":"2024-09-17T05:40:27.257485+0000","last_undegraded":"2024-09-17T05:40:27.257485+0000","last_fullsized":"2024-09-17T05:40:27.257485+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:05.194660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181301+0000","last_change":"2024-09-17T05:40:25.181547+0000","last_active":"2024-09-17T05:40:26.181301+0000","last_peered":"2024-09-17T05:40:26.181301+0000","last_clean":"2024-09-17T05:40:26.181301+0000","last_became_active":"2024-09-17T05:40:25.178736+0000","last_became_peered":"2024-09-17T05:40:25.178736+0000","last_unstale":"2024-09-17T05:40:26.181301+0000","last_undegraded":"2024-09-17T05:40:26.181301+0000","last_fullsized":"2024-09-17T05:40:26.181301+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:26.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257174+0000","last_change":"2024-09-17T05:40:25.181568+0000","last_active":"2024-09-17T05:40:27.257174+0000","last_peered":"2024-09-17T05:40:27.257174+0000","last_clean":"2024-09-17T05:40:27.257174+0000","last_became_active":"2024-09-17T05:40:25.181282+0000","last_became_peered":"2024-09-17T05:40:25.181282+0000","last_unstale":"2024-09-17T05:40:27.257174+0000","last_undegraded":"2024-09-17T05:40:27.257174+0000","last_fullsized":"2024-09-17T05:40:27.257174+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:12.178318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180814+0000","last_change":"2024-09-17T05:40:25.178429+0000","last_active":"2024-09-17T05:40:26.180814+0000","last_peered":"2024-09-17T05:40:26.180814+0000","last_clean":"2024-09-17T05:40:26.180814+0000","last_became_active":"2024-09-17T05:40:25.178157+0000","last_became_peered":"2024-09-17T05:40:25.178157+0000","last_unstale":"2024-09-17T05:40:26.180814+0000","last_undegraded":"2024-09-17T05:40:26.180814+0000","last_fullsized":"2024-09-17T05:40:26.180814+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:49:46.260197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181602+0000","last_change":"2024-09-17T05:40:25.182320+0000","last_active":"2024-09-17T05:40:26.181602+0000","last_peered":"2024-09-17T05:40:26.181602+0000","last_clean":"2024-09-17T05:40:26.181602+0000","last_became_active":"2024-09-17T05:40:25.182041+0000","last_became_peered":"2024-09-17T05:40:25.182041+0000","last_unstale":"2024-09-17T05:40:26.181602+0000","last_undegraded":"2024-09-17T05:40:26.181602+0000","last_fullsized":"2024-09-17T05:40:26.181602+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:59.378535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256761+0000","last_change":"2024-09-17T05:40:25.188017+0000","last_active":"2024-09-17T05:40:27.256761+0000","last_peered":"2024-09-17T05:40:27.256761+0000","last_clean":"2024-09-17T05:40:27.256761+0000","last_became_active":"2024-09-17T05:40:25.187748+0000","last_became_peered":"2024-09-17T05:40:25.187748+0000","last_unstale":"2024-09-17T05:40:27.256761+0000","last_undegraded":"2024-09-17T05:40:27.256761+0000","last_fullsized":"2024-09-17T05:40:27.256761+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:44.136630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258494+0000","last_change":"2024-09-17T05:40:25.186435+0000","last_active":"2024-09-17T05:40:27.258494+0000","last_peered":"2024-09-17T05:40:27.258494+0000","last_clean":"2024-09-17T05:40:27.258494+0000","last_became_active":"2024-09-17T05:40:25.183302+0000","last_became_peered":"2024-09-17T05:40:25.183302+0000","last_unstale":"2024-09-17T05:40:27.258494+0000","last_undegraded":"2024-09-17T05:40:27.258494+0000","last_fullsized":"2024-09-17T05:40:27.258494+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:14.329706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180905+0000","last_change":"2024-09-17T05:40:25.170453+0000","last_active":"2024-09-17T05:40:26.180905+0000","last_peered":"2024-09-17T05:40:26.180905+0000","last_clean":"2024-09-17T05:40:26.180905+0000","last_became_active":"2024-09-17T05:40:25.169914+0000","last_became_peered":"2024-09-17T05:40:25.169914+0000","last_unstale":"2024-09-17T05:40:26.180905+0000","last_undegraded":"2024-09-17T05:40:26.180905+0000","last_fullsized":"2024-09-17T05:40:26.180905+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:12.798962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181155+0000","last_change":"2024-09-17T05:40:25.177605+0000","last_active":"2024-09-17T05:40:26.181155+0000","last_peered":"2024-09-17T05:40:26.181155+0000","last_clean":"2024-09-17T05:40:26.181155+0000","last_became_active":"2024-09-17T05:40:25.177331+0000","last_became_peered":"2024-09-17T05:40:25.177331+0000","last_unstale":"2024-09-17T05:40:26.181155+0000","last_undegraded":"2024-09-17T05:40:26.181155+0000","last_fullsized":"2024-09-17T05:40:26.181155+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:42.626670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258571+0000","last_change":"2024-09-17T05:40:25.197121+0000","last_active":"2024-09-17T05:40:27.258571+0000","last_peered":"2024-09-17T05:40:27.258571+0000","last_clean":"2024-09-17T05:40:27.258571+0000","last_became_active":"2024-09-17T05:40:25.196844+0000","last_became_peered":"2024-09-17T05:40:25.196844+0000","last_unstale":"2024-09-17T05:40:27.258571+0000","last_undegraded":"2024-09-17T05:40:27.258571+0000","last_fullsized":"2024-09-17T05:40:27.258571+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:44.688266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"144'4","reported_seq":19,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059703+0000","last_change":"2024-09-17T05:40:24.173930+0000","last_active":"2024-09-17T05:40:27.059703+0000","last_peered":"2024-09-17T05:40:27.059703+0000","last_clean":"2024-09-17T05:40:27.059703+0000","last_became_active":"2024-09-17T05:40:24.172704+0000","last_became_peered":"2024-09-17T05:40:24.172704+0000","last_unstale":"2024-09-17T05:40:27.059703+0000","last_undegraded":"2024-09-17T05:40:27.059703+0000","last_fullsized":"2024-09-17T05:40:27.059703+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:33.170502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183271+0000","last_change":"2024-09-17T05:40:25.179791+0000","last_active":"2024-09-17T05:40:26.183271+0000","last_peered":"2024-09-17T05:40:26.183271+0000","last_clean":"2024-09-17T05:40:26.183271+0000","last_became_active":"2024-09-17T05:40:25.179450+0000","last_became_peered":"2024-09-17T05:40:25.179450+0000","last_unstale":"2024-09-17T05:40:26.183271+0000","last_undegraded":"2024-09-17T05:40:26.183271+0000","last_fullsized":"2024-09-17T05:40:26.183271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.496633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.762120+0000","last_change":"2024-09-17T05:40:24.176345+0000","last_active":"2024-09-17T05:40:26.762120+0000","last_peered":"2024-09-17T05:40:26.762120+0000","last_clean":"2024-09-17T05:40:26.762120+0000","last_became_active":"2024-09-17T05:40:24.176106+0000","last_became_peered":"2024-09-17T05:40:24.176106+0000","last_unstale":"2024-09-17T05:40:26.762120+0000","last_undegraded":"2024-09-17T05:40:26.762120+0000","last_fullsized":"2024-09-17T05:40:26.762120+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:50.800729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761949+0000","last_change":"2024-09-17T05:40:25.188810+0000","last_active":"2024-09-17T05:40:26.761949+0000","last_peered":"2024-09-17T05:40:26.761949+0000","last_clean":"2024-09-17T05:40:26.761949+0000","last_became_active":"2024-09-17T05:40:25.188336+0000","last_became_peered":"2024-09-17T05:40:25.188336+0000","last_unstale":"2024-09-17T05:40:26.761949+0000","last_undegraded":"2024-09-17T05:40:26.761949+0000","last_fullsized":"2024-09-17T05:40:26.761949+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:08.540681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059517+0000","last_change":"2024-09-17T05:40:24.175670+0000","last_active":"2024-09-17T05:40:27.059517+0000","last_peered":"2024-09-17T05:40:27.059517+0000","last_clean":"2024-09-17T05:40:27.059517+0000","last_became_active":"2024-09-17T05:40:24.175367+0000","last_became_peered":"2024-09-17T05:40:24.175367+0000","last_unstale":"2024-09-17T05:40:27.059517+0000","last_undegraded":"2024-09-17T05:40:27.059517+0000","last_fullsized":"2024-09-17T05:40:27.059517+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:25.881116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180176+0000","last_change":"2024-09-17T05:40:25.181851+0000","last_active":"2024-09-17T05:40:26.180176+0000","last_peered":"2024-09-17T05:40:26.180176+0000","last_clean":"2024-09-17T05:40:26.180176+0000","last_became_active":"2024-09-17T05:40:25.181561+0000","last_became_peered":"2024-09-17T05:40:25.181561+0000","last_unstale":"2024-09-17T05:40:26.180176+0000","last_undegraded":"2024-09-17T05:40:26.180176+0000","last_fullsized":"2024-09-17T05:40:26.180176+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:26.035276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059396+0000","last_change":"2024-09-17T05:40:24.176368+0000","last_active":"2024-09-17T05:40:27.059396+0000","last_peered":"2024-09-17T05:40:27.059396+0000","last_clean":"2024-09-17T05:40:27.059396+0000","last_became_active":"2024-09-17T05:40:24.176061+0000","last_became_peered":"2024-09-17T05:40:24.176061+0000","last_unstale":"2024-09-17T05:40:27.059396+0000","last_undegraded":"2024-09-17T05:40:27.059396+0000","last_fullsized":"2024-09-17T05:40:27.059396+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:31.798405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180664+0000","last_change":"2024-09-17T05:40:25.182623+0000","last_active":"2024-09-17T05:40:26.180664+0000","last_peered":"2024-09-17T05:40:26.180664+0000","last_clean":"2024-09-17T05:40:26.180664+0000","last_became_active":"2024-09-17T05:40:25.180102+0000","last_became_peered":"2024-09-17T05:40:25.180102+0000","last_unstale":"2024-09-17T05:40:26.180664+0000","last_undegraded":"2024-09-17T05:40:26.180664+0000","last_fullsized":"2024-09-17T05:40:26.180664+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:33.835538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256777+0000","last_change":"2024-09-17T05:40:24.167785+0000","last_active":"2024-09-17T05:40:27.256777+0000","last_peered":"2024-09-17T05:40:27.256777+0000","last_clean":"2024-09-17T05:40:27.256777+0000","last_became_active":"2024-09-17T05:40:24.167502+0000","last_became_peered":"2024-09-17T05:40:24.167502+0000","last_unstale":"2024-09-17T05:40:27.256777+0000","last_undegraded":"2024-09-17T05:40:27.256777+0000","last_fullsized":"2024-09-17T05:40:27.256777+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:57.640098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179976+0000","last_change":"2024-09-17T05:40:25.190503+0000","last_active":"2024-09-17T05:40:26.179976+0000","last_peered":"2024-09-17T05:40:26.179976+0000","last_clean":"2024-09-17T05:40:26.179976+0000","last_became_active":"2024-09-17T05:40:25.190192+0000","last_became_peered":"2024-09-17T05:40:25.190192+0000","last_unstale":"2024-09-17T05:40:26.179976+0000","last_undegraded":"2024-09-17T05:40:26.179976+0000","last_fullsized":"2024-09-17T05:40:26.179976+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:39.507268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257552+0000","last_change":"2024-09-17T05:40:24.177481+0000","last_active":"2024-09-17T05:40:27.257552+0000","last_peered":"2024-09-17T05:40:27.257552+0000","last_clean":"2024-09-17T05:40:27.257552+0000","last_became_active":"2024-09-17T05:40:24.177186+0000","last_became_peered":"2024-09-17T05:40:24.177186+0000","last_unstale":"2024-09-17T05:40:27.257552+0000","last_undegraded":"2024-09-17T05:40:27.257552+0000","last_fullsized":"2024-09-17T05:40:27.257552+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:40:01.491102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180751+0000","last_change":"2024-09-17T05:40:25.177145+0000","last_active":"2024-09-17T05:40:26.180751+0000","last_peered":"2024-09-17T05:40:26.180751+0000","last_clean":"2024-09-17T05:40:26.180751+0000","last_became_active":"2024-09-17T05:40:25.176749+0000","last_became_peered":"2024-09-17T05:40:25.176749+0000","last_unstale":"2024-09-17T05:40:26.180751+0000","last_undegraded":"2024-09-17T05:40:26.180751+0000","last_fullsized":"2024-09-17T05:40:26.180751+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:50.785565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761817+0000","last_change":"2024-09-17T05:40:24.162381+0000","last_active":"2024-09-17T05:40:26.761817+0000","last_peered":"2024-09-17T05:40:26.761817+0000","last_clean":"2024-09-17T05:40:26.761817+0000","last_became_active":"2024-09-17T05:40:24.160458+0000","last_became_peered":"2024-09-17T05:40:24.160458+0000","last_unstale":"2024-09-17T05:40:26.761817+0000","last_undegraded":"2024-09-17T05:40:26.761817+0000","last_fullsized":"2024-09-17T05:40:26.761817+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:45.282829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179543+0000","last_change":"2024-09-17T05:40:25.187900+0000","last_active":"2024-09-17T05:40:26.179543+0000","last_peered":"2024-09-17T05:40:26.179543+0000","last_clean":"2024-09-17T05:40:26.179543+0000","last_became_active":"2024-09-17T05:40:25.187628+0000","last_became_peered":"2024-09-17T05:40:25.187628+0000","last_unstale":"2024-09-17T05:40:26.179543+0000","last_undegraded":"2024-09-17T05:40:26.179543+0000","last_fullsized":"2024-09-17T05:40:26.179543+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:29.075371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059111+0000","last_change":"2024-09-17T05:40:24.179128+0000","last_active":"2024-09-17T05:40:27.059111+0000","last_peered":"2024-09-17T05:40:27.059111+0000","last_clean":"2024-09-17T05:40:27.059111+0000","last_became_active":"2024-09-17T05:40:24.178552+0000","last_became_peered":"2024-09-17T05:40:24.178552+0000","last_unstale":"2024-09-17T05:40:27.059111+0000","last_undegraded":"2024-09-17T05:40:27.059111+0000","last_fullsized":"2024-09-17T05:40:27.059111+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:06.290224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180753+0000","last_change":"2024-09-17T05:40:25.192651+0000","last_active":"2024-09-17T05:40:26.180753+0000","last_peered":"2024-09-17T05:40:26.180753+0000","last_clean":"2024-09-17T05:40:26.180753+0000","last_became_active":"2024-09-17T05:40:25.184118+0000","last_became_peered":"2024-09-17T05:40:25.184118+0000","last_unstale":"2024-09-17T05:40:26.180753+0000","last_undegraded":"2024-09-17T05:40:26.180753+0000","last_fullsized":"2024-09-17T05:40:26.180753+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:58.648089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761975+0000","last_change":"2024-09-17T05:40:24.161649+0000","last_active":"2024-09-17T05:40:26.761975+0000","last_peered":"2024-09-17T05:40:26.761975+0000","last_clean":"2024-09-17T05:40:26.761975+0000","last_became_active":"2024-09-17T05:40:24.161393+0000","last_became_peered":"2024-09-17T05:40:24.161393+0000","last_unstale":"2024-09-17T05:40:26.761975+0000","last_undegraded":"2024-09-17T05:40:26.761975+0000","last_fullsized":"2024-09-17T05:40:26.761975+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:56.313784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258537+0000","last_change":"2024-09-17T05:40:25.182851+0000","last_active":"2024-09-17T05:40:27.258537+0000","last_peered":"2024-09-17T05:40:27.258537+0000","last_clean":"2024-09-17T05:40:27.258537+0000","last_became_active":"2024-09-17T05:40:25.182581+0000","last_became_peered":"2024-09-17T05:40:25.182581+0000","last_unstale":"2024-09-17T05:40:27.258537+0000","last_undegraded":"2024-09-17T05:40:27.258537+0000","last_fullsized":"2024-09-17T05:40:27.258537+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:34.728190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258890+0000","last_change":"2024-09-17T05:40:24.169609+0000","last_active":"2024-09-17T05:40:27.258890+0000","last_peered":"2024-09-17T05:40:27.258890+0000","last_clean":"2024-09-17T05:40:27.258890+0000","last_became_active":"2024-09-17T05:40:24.169317+0000","last_became_peered":"2024-09-17T05:40:24.169317+0000","last_unstale":"2024-09-17T05:40:27.258890+0000","last_undegraded":"2024-09-17T05:40:27.258890+0000","last_fullsized":"2024-09-17T05:40:27.258890+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:17.444963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183307+0000","last_change":"2024-09-17T05:40:25.184351+0000","last_active":"2024-09-17T05:40:26.183307+0000","last_peered":"2024-09-17T05:40:26.183307+0000","last_clean":"2024-09-17T05:40:26.183307+0000","last_became_active":"2024-09-17T05:40:25.183920+0000","last_became_peered":"2024-09-17T05:40:25.183920+0000","last_unstale":"2024-09-17T05:40:26.183307+0000","last_undegraded":"2024-09-17T05:40:26.183307+0000","last_fullsized":"2024-09-17T05:40:26.183307+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:22.638149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.182912+0000","last_change":"2024-09-17T05:40:24.169333+0000","last_active":"2024-09-17T05:40:26.182912+0000","last_peered":"2024-09-17T05:40:26.182912+0000","last_clean":"2024-09-17T05:40:26.182912+0000","last_became_active":"2024-09-17T05:40:24.168219+0000","last_became_peered":"2024-09-17T05:40:24.168219+0000","last_unstale":"2024-09-17T05:40:26.182912+0000","last_undegraded":"2024-09-17T05:40:26.182912+0000","last_fullsized":"2024-09-17T05:40:26.182912+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:58.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179534+0000","last_change":"2024-09-17T05:40:25.195715+0000","last_active":"2024-09-17T05:40:26.179534+0000","last_peered":"2024-09-17T05:40:26.179534+0000","last_clean":"2024-09-17T05:40:26.179534+0000","last_became_active":"2024-09-17T05:40:25.195398+0000","last_became_peered":"2024-09-17T05:40:25.195398+0000","last_unstale":"2024-09-17T05:40:26.179534+0000","last_undegraded":"2024-09-17T05:40:26.179534+0000","last_fullsized":"2024-09-17T05:40:26.179534+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:18.607272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059639+0000","last_change":"2024-09-17T05:40:24.177008+0000","last_active":"2024-09-17T05:40:27.059639+0000","last_peered":"2024-09-17T05:40:27.059639+0000","last_clean":"2024-09-17T05:40:27.059639+0000","last_became_active":"2024-09-17T05:40:24.175458+0000","last_became_peered":"2024-09-17T05:40:24.175458+0000","last_unstale":"2024-09-17T05:40:27.059639+0000","last_undegraded":"2024-09-17T05:40:27.059639+0000","last_fullsized":"2024-09-17T05:40:27.059639+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:34.355730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256708+0000","last_change":"2024-09-17T05:40:25.188264+0000","last_active":"2024-09-17T05:40:27.256708+0000","last_peered":"2024-09-17T05:40:27.256708+0000","last_clean":"2024-09-17T05:40:27.256708+0000","last_became_active":"2024-09-17T05:40:25.187942+0000","last_became_peered":"2024-09-17T05:40:25.187942+0000","last_unstale":"2024-09-17T05:40:27.256708+0000","last_undegraded":"2024-09-17T05:40:27.256708+0000","last_fullsized":"2024-09-17T05:40:27.256708+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:14.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258496+0000","last_change":"2024-09-17T05:40:24.171856+0000","last_active":"2024-09-17T05:40:27.258496+0000","last_peered":"2024-09-17T05:40:27.258496+0000","last_clean":"2024-09-17T05:40:27.258496+0000","last_became_active":"2024-09-17T05:40:24.171580+0000","last_became_peered":"2024-09-17T05:40:24.171580+0000","last_unstale":"2024-09-17T05:40:27.258496+0000","last_undegraded":"2024-09-17T05:40:27.258496+0000","last_fullsized":"2024-09-17T05:40:27.258496+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:15:47.253610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179401+0000","last_change":"2024-09-17T05:40:25.180695+0000","last_active":"2024-09-17T05:40:26.179401+0000","last_peered":"2024-09-17T05:40:26.179401+0000","last_clean":"2024-09-17T05:40:26.179401+0000","last_became_active":"2024-09-17T05:40:25.180073+0000","last_became_peered":"2024-09-17T05:40:25.180073+0000","last_unstale":"2024-09-17T05:40:26.179401+0000","last_undegraded":"2024-09-17T05:40:26.179401+0000","last_fullsized":"2024-09-17T05:40:26.179401+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:09.726434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180986+0000","last_change":"2024-09-17T05:40:24.163423+0000","last_active":"2024-09-17T05:40:26.180986+0000","last_peered":"2024-09-17T05:40:26.180986+0000","last_clean":"2024-09-17T05:40:26.180986+0000","last_became_active":"2024-09-17T05:40:24.163147+0000","last_became_peered":"2024-09-17T05:40:24.163147+0000","last_unstale":"2024-09-17T05:40:26.180986+0000","last_undegraded":"2024-09-17T05:40:26.180986+0000","last_fullsized":"2024-09-17T05:40:26.180986+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:03.350935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179660+0000","last_change":"2024-09-17T05:40:25.182786+0000","last_active":"2024-09-17T05:40:26.179660+0000","last_peered":"2024-09-17T05:40:26.179660+0000","last_clean":"2024-09-17T05:40:26.179660+0000","last_became_active":"2024-09-17T05:40:25.182186+0000","last_became_peered":"2024-09-17T05:40:25.182186+0000","last_unstale":"2024-09-17T05:40:26.179660+0000","last_undegraded":"2024-09-17T05:40:26.179660+0000","last_fullsized":"2024-09-17T05:40:26.179660+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:51.557517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"144'8","reported_seq":23,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256535+0000","last_change":"2024-09-17T05:40:24.159858+0000","last_active":"2024-09-17T05:40:27.256535+0000","last_peered":"2024-09-17T05:40:27.256535+0000","last_clean":"2024-09-17T05:40:27.256535+0000","last_became_active":"2024-09-17T05:40:24.158365+0000","last_became_peered":"2024-09-17T05:40:24.158365+0000","last_unstale":"2024-09-17T05:40:27.256535+0000","last_undegraded":"2024-09-17T05:40:27.256535+0000","last_fullsized":"2024-09-17T05:40:27.256535+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:25.027285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258758+0000","last_change":"2024-09-17T05:40:25.185576+0000","last_active":"2024-09-17T05:40:27.258758+0000","last_peered":"2024-09-17T05:40:27.258758+0000","last_clean":"2024-09-17T05:40:27.258758+0000","last_became_active":"2024-09-17T05:40:25.185029+0000","last_became_peered":"2024-09-17T05:40:25.185029+0000","last_unstale":"2024-09-17T05:40:27.258758+0000","last_undegraded":"2024-09-17T05:40:27.258758+0000","last_fullsized":"2024-09-17T05:40:27.258758+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:08.810996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180721+0000","last_change":"2024-09-17T05:40:24.168831+0000","last_active":"2024-09-17T05:40:26.180721+0000","last_peered":"2024-09-17T05:40:26.180721+0000","last_clean":"2024-09-17T05:40:26.180721+0000","last_became_active":"2024-09-17T05:40:24.168322+0000","last_became_peered":"2024-09-17T05:40:24.168322+0000","last_unstale":"2024-09-17T05:40:26.180721+0000","last_undegraded":"2024-09-17T05:40:26.180721+0000","last_fullsized":"2024-09-17T05:40:26.180721+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:11.790971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181516+0000","last_change":"2024-09-17T05:40:25.169953+0000","last_active":"2024-09-17T05:40:26.181516+0000","last_peered":"2024-09-17T05:40:26.181516+0000","last_clean":"2024-09-17T05:40:26.181516+0000","last_became_active":"2024-09-17T05:40:25.169667+0000","last_became_peered":"2024-09-17T05:40:25.169667+0000","last_unstale":"2024-09-17T05:40:26.181516+0000","last_undegraded":"2024-09-17T05:40:26.181516+0000","last_fullsized":"2024-09-17T05:40:26.181516+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:53.178335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257065+0000","last_change":"2024-09-17T05:40:24.175976+0000","last_active":"2024-09-17T05:40:27.257065+0000","last_peered":"2024-09-17T05:40:27.257065+0000","last_clean":"2024-09-17T05:40:27.257065+0000","last_became_active":"2024-09-17T05:40:24.175623+0000","last_became_peered":"2024-09-17T05:40:24.175623+0000","last_unstale":"2024-09-17T05:40:27.257065+0000","last_undegraded":"2024-09-17T05:40:27.257065+0000","last_fullsized":"2024-09-17T05:40:27.257065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:13.334310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258236+0000","last_change":"2024-09-17T05:40:25.180707+0000","last_active":"2024-09-17T05:40:27.258236+0000","last_peered":"2024-09-17T05:40:27.258236+0000","last_clean":"2024-09-17T05:40:27.258236+0000","last_became_active":"2024-09-17T05:40:25.180323+0000","last_became_peered":"2024-09-17T05:40:25.180323+0000","last_unstale":"2024-09-17T05:40:27.258236+0000","last_undegraded":"2024-09-17T05:40:27.258236+0000","last_fullsized":"2024-09-17T05:40:27.258236+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:28.199285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183232+0000","last_change":"2024-09-17T05:40:24.173858+0000","last_active":"2024-09-17T05:40:26.183232+0000","last_peered":"2024-09-17T05:40:26.183232+0000","last_clean":"2024-09-17T05:40:26.183232+0000","last_became_active":"2024-09-17T05:40:24.173661+0000","last_became_peered":"2024-09-17T05:40:24.173661+0000","last_unstale":"2024-09-17T05:40:26.183232+0000","last_undegraded":"2024-09-17T05:40:26.183232+0000","last_fullsized":"2024-09-17T05:40:26.183232+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:21.200140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.762248+0000","last_change":"2024-09-17T05:40:25.188672+0000","last_active":"2024-09-17T05:40:26.762248+0000","last_peered":"2024-09-17T05:40:26.762248+0000","last_clean":"2024-09-17T05:40:26.762248+0000","last_became_active":"2024-09-17T05:40:25.188365+0000","last_became_peered":"2024-09-17T05:40:25.188365+0000","last_unstale":"2024-09-17T05:40:26.762248+0000","last_undegraded":"2024-09-17T05:40:26.762248+0000","last_fullsized":"2024-09-17T05:40:26.762248+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:14.329706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181668+0000","last_change":"2024-09-17T05:40:24.169329+0000","last_active":"2024-09-17T05:40:26.181668+0000","last_peered":"2024-09-17T05:40:26.181668+0000","last_clean":"2024-09-17T05:40:26.181668+0000","last_became_active":"2024-09-17T05:40:24.169051+0000","last_became_peered":"2024-09-17T05:40:24.169051+0000","last_unstale":"2024-09-17T05:40:26.181668+0000","last_undegraded":"2024-09-17T05:40:26.181668+0000","last_fullsized":"2024-09-17T05:40:26.181668+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:52.335383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258437+0000","last_change":"2024-09-17T05:40:25.193334+0000","last_active":"2024-09-17T05:40:27.258437+0000","last_peered":"2024-09-17T05:40:27.258437+0000","last_clean":"2024-09-17T05:40:27.258437+0000","last_became_active":"2024-09-17T05:40:25.187108+0000","last_became_peered":"2024-09-17T05:40:25.187108+0000","last_unstale":"2024-09-17T05:40:27.258437+0000","last_undegraded":"2024-09-17T05:40:27.258437+0000","last_fullsized":"2024-09-17T05:40:27.258437+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:32.036031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761716+0000","last_change":"2024-09-17T05:40:24.174951+0000","last_active":"2024-09-17T05:40:26.761716+0000","last_peered":"2024-09-17T05:40:26.761716+0000","last_clean":"2024-09-17T05:40:26.761716+0000","last_became_active":"2024-09-17T05:40:24.174436+0000","last_became_peered":"2024-09-17T05:40:24.174436+0000","last_unstale":"2024-09-17T05:40:26.761716+0000","last_undegraded":"2024-09-17T05:40:26.761716+0000","last_fullsized":"2024-09-17T05:40:26.761716+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:23.920494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180846+0000","last_change":"2024-09-17T05:40:25.170463+0000","last_active":"2024-09-17T05:40:26.180846+0000","last_peered":"2024-09-17T05:40:26.180846+0000","last_clean":"2024-09-17T05:40:26.180846+0000","last_became_active":"2024-09-17T05:40:25.170138+0000","last_became_peered":"2024-09-17T05:40:25.170138+0000","last_unstale":"2024-09-17T05:40:26.180846+0000","last_undegraded":"2024-09-17T05:40:26.180846+0000","last_fullsized":"2024-09-17T05:40:26.180846+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:36.252315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179600+0000","last_change":"2024-09-17T05:40:24.169832+0000","last_active":"2024-09-17T05:40:26.179600+0000","last_peered":"2024-09-17T05:40:26.179600+0000","last_clean":"2024-09-17T05:40:26.179600+0000","last_became_active":"2024-09-17T05:40:24.169570+0000","last_became_peered":"2024-09-17T05:40:24.169570+0000","last_unstale":"2024-09-17T05:40:26.179600+0000","last_undegraded":"2024-09-17T05:40:26.179600+0000","last_fullsized":"2024-09-17T05:40:26.179600+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:52.170344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257866+0000","last_change":"2024-09-17T05:40:25.184998+0000","last_active":"2024-09-17T05:40:27.257866+0000","last_peered":"2024-09-17T05:40:27.257866+0000","last_clean":"2024-09-17T05:40:27.257866+0000","last_became_active":"2024-09-17T05:40:25.184718+0000","last_became_peered":"2024-09-17T05:40:25.184718+0000","last_unstale":"2024-09-17T05:40:27.257866+0000","last_undegraded":"2024-09-17T05:40:27.257866+0000","last_fullsized":"2024-09-17T05:40:27.257866+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:15:53.661119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179602+0000","last_change":"2024-09-17T05:40:24.167353+0000","last_active":"2024-09-17T05:40:26.179602+0000","last_peered":"2024-09-17T05:40:26.179602+0000","last_clean":"2024-09-17T05:40:26.179602+0000","last_became_active":"2024-09-17T05:40:24.167074+0000","last_became_peered":"2024-09-17T05:40:24.167074+0000","last_unstale":"2024-09-17T05:40:26.179602+0000","last_undegraded":"2024-09-17T05:40:26.179602+0000","last_fullsized":"2024-09-17T05:40:26.179602+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:41.618679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.181002+0000","last_change":"2024-09-17T05:40:25.173599+0000","last_active":"2024-09-17T05:40:26.181002+0000","last_peered":"2024-09-17T05:40:26.181002+0000","last_clean":"2024-09-17T05:40:26.181002+0000","last_became_active":"2024-09-17T05:40:25.173358+0000","last_became_peered":"2024-09-17T05:40:25.173358+0000","last_unstale":"2024-09-17T05:40:26.181002+0000","last_undegraded":"2024-09-17T05:40:26.181002+0000","last_fullsized":"2024-09-17T05:40:26.181002+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:25.486183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179536+0000","last_change":"2024-09-17T05:40:24.173086+0000","last_active":"2024-09-17T05:40:26.179536+0000","last_peered":"2024-09-17T05:40:26.179536+0000","last_clean":"2024-09-17T05:40:26.179536+0000","last_became_active":"2024-09-17T05:40:24.171758+0000","last_became_peered":"2024-09-17T05:40:24.171758+0000","last_unstale":"2024-09-17T05:40:26.179536+0000","last_undegraded":"2024-09-17T05:40:26.179536+0000","last_fullsized":"2024-09-17T05:40:26.179536+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:42.281444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180826+0000","last_change":"2024-09-17T05:40:25.193176+0000","last_active":"2024-09-17T05:40:26.180826+0000","last_peered":"2024-09-17T05:40:26.180826+0000","last_clean":"2024-09-17T05:40:26.180826+0000","last_became_active":"2024-09-17T05:40:25.192893+0000","last_became_peered":"2024-09-17T05:40:25.192893+0000","last_unstale":"2024-09-17T05:40:26.180826+0000","last_undegraded":"2024-09-17T05:40:26.180826+0000","last_fullsized":"2024-09-17T05:40:26.180826+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:42.278904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180848+0000","last_change":"2024-09-17T05:40:24.168573+0000","last_active":"2024-09-17T05:40:26.180848+0000","last_peered":"2024-09-17T05:40:26.180848+0000","last_clean":"2024-09-17T05:40:26.180848+0000","last_became_active":"2024-09-17T05:40:24.167948+0000","last_became_peered":"2024-09-17T05:40:24.167948+0000","last_unstale":"2024-09-17T05:40:26.180848+0000","last_undegraded":"2024-09-17T05:40:26.180848+0000","last_fullsized":"2024-09-17T05:40:26.180848+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:24.478191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180780+0000","last_change":"2024-09-17T05:40:25.192972+0000","last_active":"2024-09-17T05:40:26.180780+0000","last_peered":"2024-09-17T05:40:26.180780+0000","last_clean":"2024-09-17T05:40:26.180780+0000","last_became_active":"2024-09-17T05:40:25.192566+0000","last_became_peered":"2024-09-17T05:40:25.192566+0000","last_unstale":"2024-09-17T05:40:26.180780+0000","last_undegraded":"2024-09-17T05:40:26.180780+0000","last_fullsized":"2024-09-17T05:40:26.180780+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:12.582292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180993+0000","last_change":"2024-09-17T05:40:24.175084+0000","last_active":"2024-09-17T05:40:26.180993+0000","last_peered":"2024-09-17T05:40:26.180993+0000","last_clean":"2024-09-17T05:40:26.180993+0000","last_became_active":"2024-09-17T05:40:24.174827+0000","last_became_peered":"2024-09-17T05:40:24.174827+0000","last_unstale":"2024-09-17T05:40:26.180993+0000","last_undegraded":"2024-09-17T05:40:26.180993+0000","last_fullsized":"2024-09-17T05:40:26.180993+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:43.680275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256953+0000","last_change":"2024-09-17T05:40:25.184916+0000","last_active":"2024-09-17T05:40:27.256953+0000","last_peered":"2024-09-17T05:40:27.256953+0000","last_clean":"2024-09-17T05:40:27.256953+0000","last_became_active":"2024-09-17T05:40:25.184598+0000","last_became_peered":"2024-09-17T05:40:25.184598+0000","last_unstale":"2024-09-17T05:40:27.256953+0000","last_undegraded":"2024-09-17T05:40:27.256953+0000","last_fullsized":"2024-09-17T05:40:27.256953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:26:02.388462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183134+0000","last_change":"2024-09-17T05:40:24.168536+0000","last_active":"2024-09-17T05:40:26.183134+0000","last_peered":"2024-09-17T05:40:26.183134+0000","last_clean":"2024-09-17T05:40:26.183134+0000","last_became_active":"2024-09-17T05:40:24.168121+0000","last_became_peered":"2024-09-17T05:40:24.168121+0000","last_unstale":"2024-09-17T05:40:26.183134+0000","last_undegraded":"2024-09-17T05:40:26.183134+0000","last_fullsized":"2024-09-17T05:40:26.183134+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:52.244624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.256986+0000","last_change":"2024-09-17T05:40:25.184243+0000","last_active":"2024-09-17T05:40:27.256986+0000","last_peered":"2024-09-17T05:40:27.256986+0000","last_clean":"2024-09-17T05:40:27.256986+0000","last_became_active":"2024-09-17T05:40:25.183980+0000","last_became_peered":"2024-09-17T05:40:25.183980+0000","last_unstale":"2024-09-17T05:40:27.256986+0000","last_undegraded":"2024-09-17T05:40:27.256986+0000","last_fullsized":"2024-09-17T05:40:27.256986+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:57:53.918071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183037+0000","last_change":"2024-09-17T05:40:24.167195+0000","last_active":"2024-09-17T05:40:26.183037+0000","last_peered":"2024-09-17T05:40:26.183037+0000","last_clean":"2024-09-17T05:40:26.183037+0000","last_became_active":"2024-09-17T05:40:24.166930+0000","last_became_peered":"2024-09-17T05:40:24.166930+0000","last_unstale":"2024-09-17T05:40:26.183037+0000","last_undegraded":"2024-09-17T05:40:26.183037+0000","last_fullsized":"2024-09-17T05:40:26.183037+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:47.316470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183088+0000","last_change":"2024-09-17T05:40:25.177997+0000","last_active":"2024-09-17T05:40:26.183088+0000","last_peered":"2024-09-17T05:40:26.183088+0000","last_clean":"2024-09-17T05:40:26.183088+0000","last_became_active":"2024-09-17T05:40:25.177737+0000","last_became_peered":"2024-09-17T05:40:25.177737+0000","last_unstale":"2024-09-17T05:40:26.183088+0000","last_undegraded":"2024-09-17T05:40:26.183088+0000","last_fullsized":"2024-09-17T05:40:26.183088+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:47.829551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180907+0000","last_change":"2024-09-17T05:40:24.161046+0000","last_active":"2024-09-17T05:40:26.180907+0000","last_peered":"2024-09-17T05:40:26.180907+0000","last_clean":"2024-09-17T05:40:26.180907+0000","last_became_active":"2024-09-17T05:40:24.160629+0000","last_became_peered":"2024-09-17T05:40:24.160629+0000","last_unstale":"2024-09-17T05:40:26.180907+0000","last_undegraded":"2024-09-17T05:40:26.180907+0000","last_fullsized":"2024-09-17T05:40:26.180907+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:27.191294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.180160+0000","last_change":"2024-09-17T05:40:25.190186+0000","last_active":"2024-09-17T05:40:26.180160+0000","last_peered":"2024-09-17T05:40:26.180160+0000","last_clean":"2024-09-17T05:40:26.180160+0000","last_became_active":"2024-09-17T05:40:25.189902+0000","last_became_peered":"2024-09-17T05:40:25.189902+0000","last_unstale":"2024-09-17T05:40:26.180160+0000","last_undegraded":"2024-09-17T05:40:26.180160+0000","last_fullsized":"2024-09-17T05:40:26.180160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:55.930768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059266+0000","last_change":"2024-09-17T05:40:24.176177+0000","last_active":"2024-09-17T05:40:27.059266+0000","last_peered":"2024-09-17T05:40:27.059266+0000","last_clean":"2024-09-17T05:40:27.059266+0000","last_became_active":"2024-09-17T05:40:24.175077+0000","last_became_peered":"2024-09-17T05:40:24.175077+0000","last_unstale":"2024-09-17T05:40:27.059266+0000","last_undegraded":"2024-09-17T05:40:27.059266+0000","last_fullsized":"2024-09-17T05:40:27.059266+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:08.803589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.179495+0000","last_change":"2024-09-17T05:40:25.180978+0000","last_active":"2024-09-17T05:40:26.179495+0000","last_peered":"2024-09-17T05:40:26.179495+0000","last_clean":"2024-09-17T05:40:26.179495+0000","last_became_active":"2024-09-17T05:40:25.180431+0000","last_became_peered":"2024-09-17T05:40:25.180431+0000","last_unstale":"2024-09-17T05:40:26.179495+0000","last_undegraded":"2024-09-17T05:40:26.179495+0000","last_fullsized":"2024-09-17T05:40:26.179495+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:56.582632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257885+0000","last_change":"2024-09-17T05:40:24.172286+0000","last_active":"2024-09-17T05:40:27.257885+0000","last_peered":"2024-09-17T05:40:27.257885+0000","last_clean":"2024-09-17T05:40:27.257885+0000","last_became_active":"2024-09-17T05:40:24.171998+0000","last_became_peered":"2024-09-17T05:40:24.171998+0000","last_unstale":"2024-09-17T05:40:27.257885+0000","last_undegraded":"2024-09-17T05:40:27.257885+0000","last_fullsized":"2024-09-17T05:40:27.257885+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:51.086129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.059184+0000","last_change":"2024-09-17T05:40:25.181389+0000","last_active":"2024-09-17T05:40:27.059184+0000","last_peered":"2024-09-17T05:40:27.059184+0000","last_clean":"2024-09-17T05:40:27.059184+0000","last_became_active":"2024-09-17T05:40:25.181136+0000","last_became_peered":"2024-09-17T05:40:25.181136+0000","last_unstale":"2024-09-17T05:40:27.059184+0000","last_undegraded":"2024-09-17T05:40:27.059184+0000","last_fullsized":"2024-09-17T05:40:27.059184+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:25.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.761703+0000","last_change":"2024-09-17T05:40:24.174394+0000","last_active":"2024-09-17T05:40:26.761703+0000","last_peered":"2024-09-17T05:40:26.761703+0000","last_clean":"2024-09-17T05:40:26.761703+0000","last_became_active":"2024-09-17T05:40:24.174122+0000","last_became_peered":"2024-09-17T05:40:24.174122+0000","last_unstale":"2024-09-17T05:40:26.761703+0000","last_undegraded":"2024-09-17T05:40:26.761703+0000","last_fullsized":"2024-09-17T05:40:26.761703+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:08.718443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:26.183130+0000","last_change":"2024-09-17T05:40:25.185149+0000","last_active":"2024-09-17T05:40:26.183130+0000","last_peered":"2024-09-17T05:40:26.183130+0000","last_clean":"2024-09-17T05:40:26.183130+0000","last_became_active":"2024-09-17T05:40:25.184781+0000","last_became_peered":"2024-09-17T05:40:25.184781+0000","last_unstale":"2024-09-17T05:40:26.183130+0000","last_undegraded":"2024-09-17T05:40:26.183130+0000","last_fullsized":"2024-09-17T05:40:26.183130+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:24.144923+0000","last_clean_scrub_stamp":"2024-09-17T05:40:24.144923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:04.885218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"144'3","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.258621+0000","last_change":"2024-09-17T05:40:24.173119+0000","last_active":"2024-09-17T05:40:27.258621+0000","last_peered":"2024-09-17T05:40:27.258621+0000","last_clean":"2024-09-17T05:40:27.258621+0000","last_became_active":"2024-09-17T05:40:24.172873+0000","last_became_peered":"2024-09-17T05:40:24.172873+0000","last_unstale":"2024-09-17T05:40:27.258621+0000","last_undegraded":"2024-09-17T05:40:27.258621+0000","last_fullsized":"2024-09-17T05:40:27.258621+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:40:23.136931+0000","last_clean_scrub_stamp":"2024-09-17T05:40:23.136931+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:54.973445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":345,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-17T05:40:27.257437+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:40:27.257437+0000","last_peered":"2024-09-17T05:40:27.257437+0000","last_clean":"2024-09-17T05:40:27.257437+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:40:27.257437+0000","last_undegraded":"2024-09-17T05:40:27.257437+0000","last_fullsized":"2024-09-17T05:40:27.257437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127292,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244548,"statfs":{"total":96636764160,"available":96506417152,"internally_reserved":0,"allocated":1806336,"data_stored":799394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128514506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509630,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":155064,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":153278,"kb_avail":94216776,"statfs":{"total":96636764160,"available":96477978624,"internally_reserved":0,"allocated":1806336,"data_stored":799394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":156957126},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509630,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131468,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240372,"statfs":{"total":96636764160,"available":96502140928,"internally_reserved":0,"allocated":1826816,"data_stored":801610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135620,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94236220,"statfs":{"total":96636764160,"available":96497889280,"internally_reserved":0,"allocated":1814528,"data_stored":800010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131456,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240384,"statfs":{"total":96636764160,"available":96502153216,"internally_reserved":0,"allocated":1810432,"data_stored":799416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509631,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145920,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225920,"statfs":{"total":96636764160,"available":96487342080,"internally_reserved":0,"allocated":2404352,"data_stored":1389832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509629,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149500,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222340,"statfs":{"total":96636764160,"available":96483676160,"internally_reserved":0,"allocated":1810432,"data_stored":799484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":151255499},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119580,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252260,"statfs":{"total":96636764160,"available":96514314240,"internally_reserved":0,"allocated":2424832,"data_stored":1391568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:40:34.224 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:40:34.225 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:40:34.314 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:40:34.317 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:40:34.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- 172.21.15.122:0/3214862764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 msgr2=0x7f9fe80cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 --2- 172.21.15.122:0/3214862764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 0x7f9fe80cd260 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009840 tx=0x7f9fd001c790 comp rx=0 tx=0).stop 2024-09-17T05:40:34.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- 172.21.15.122:0/3214862764 shutdown_connections 2024-09-17T05:40:34.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 --2- 172.21.15.122:0/3214862764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe80d2760 0x7f9fe80b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 --2- 172.21.15.122:0/3214862764 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 0x7f9fe80cd260 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- 172.21.15.122:0/3214862764 >> 172.21.15.122:0/3214862764 conn(0x7f9fe80d1c40 msgr2=0x7f9fe80d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:34.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- 172.21.15.122:0/3214862764 shutdown_connections 2024-09-17T05:40:34.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- 172.21.15.122:0/3214862764 wait complete. 2024-09-17T05:40:34.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 Processor -- start 2024-09-17T05:40:34.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- start start 2024-09-17T05:40:34.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 0x7f9fe814ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe80d2760 0x7f9fe815e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe815bdb0 0x7f9fe815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8168c80 con 0x7f9fe80d03d0 2024-09-17T05:40:34.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe8168b00 con 0x7f9fe815bdb0 2024-09-17T05:40:34.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9feca00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe8168e00 con 0x7f9fe80d2760 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9fe6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 0x7f9fe814ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9fe6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 0x7f9fe814ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58778/0 (socket says 172.21.15.122:58778) 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9fe6800640 1 -- 172.21.15.122:0/2252671362 learned_addr learned my addr 172.21.15.122:0/2252671362 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9fdfe00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe80d2760 0x7f9fe815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.420+0000 7f9fe7200640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe815bdb0 0x7f9fe815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fe6800640 1 -- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe80d2760 msgr2=0x7f9fe815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fe6800640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe80d2760 0x7f9fe815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fe6800640 1 -- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe815bdb0 msgr2=0x7f9fe815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fe6800640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe815bdb0 0x7f9fe815ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fe6800640 1 -- 172.21.15.122:0/2252671362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe80720f0 con 0x7f9fe80d03d0 2024-09-17T05:40:34.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fe7200640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe815bdb0 0x7f9fe815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:34.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fe6800640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 0x7f9fe814ddd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8001ec0 tx=0x7f9fd800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:34.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fdea00640 1 -- 172.21.15.122:0/2252671362 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8017070 con 0x7f9fe80d03d0 2024-09-17T05:40:34.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fdea00640 1 -- 172.21.15.122:0/2252671362 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fd8003600 con 0x7f9fe80d03d0 2024-09-17T05:40:34.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe815c430 con 0x7f9fe80d03d0 2024-09-17T05:40:34.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fdea00640 1 -- 172.21.15.122:0/2252671362 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8003a50 con 0x7f9fe80d03d0 2024-09-17T05:40:34.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe815c6b0 con 0x7f9fe80d03d0 2024-09-17T05:40:34.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe80cd260 con 0x7f9fe80d03d0 2024-09-17T05:40:34.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.424+0000 7f9fdea00640 1 -- 172.21.15.122:0/2252671362 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f9fd801a070 con 0x7f9fe80d03d0 2024-09-17T05:40:34.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.428+0000 7f9fdea00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc40754e0 0x7f9fc40779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.428+0000 7f9fdea00640 1 -- 172.21.15.122:0/2252671362 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f9fd80925d0 con 0x7f9fe80d03d0 2024-09-17T05:40:34.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.428+0000 7f9fdfe00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc40754e0 0x7f9fc40779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.428+0000 7f9fdfe00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc40754e0 0x7f9fc40779a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00af000 tx=0x7f9fd001afd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:34.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.432+0000 7f9fdea00640 1 -- 172.21.15.122:0/2252671362 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9fe815c430 con 0x7f9fe80d03d0 2024-09-17T05:40:34.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.656+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9fe80ce430 con 0x7f9fe80d03d0 2024-09-17T05:40:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.660+0000 7f9fdea00640 1 -- 172.21.15.122:0/2252671362 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f9fe80d0b20 con 0x7f9fe80d03d0 2024-09-17T05:40:34.658 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-17T05:40:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBzEulm8QfSMBAA953uy3hV56uLyvdt3TW0pQ== 2024-09-17T05:40:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-09-17T05:40:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-09-17T05:40:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:40:34.658 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-09-17T05:40:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.660+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc40754e0 msgr2=0x7f9fc40779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.660+0000 7f9feca00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc40754e0 0x7f9fc40779a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00af000 tx=0x7f9fd001afd0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 msgr2=0x7f9fe814ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 0x7f9fe814ddd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8001ec0 tx=0x7f9fd800c890 comp rx=0 tx=0).stop 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 shutdown_connections 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9fc40754e0 0x7f9fc40779a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe815bdb0 0x7f9fe815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fe80d2760 0x7f9fe815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 --2- 172.21.15.122:0/2252671362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fe80d03d0 0x7f9fe814ddd0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 >> 172.21.15.122:0/2252671362 conn(0x7f9fe80d1c40 msgr2=0x7f9fe80708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 shutdown_connections 2024-09-17T05:40:34.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.664+0000 7f9feca00640 1 -- 172.21.15.122:0/2252671362 wait complete. 2024-09-17T05:40:34.678 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:40:34.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.820+0000 7fa6d8000640 1 -- 172.21.15.122:0/2328444889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2760 msgr2=0x7fa6d00d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.820+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2328444889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2760 0x7fa6d00d0260 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00b33d0 tx=0x7fa6bc01c700 comp rx=0 tx=0).stop 2024-09-17T05:40:34.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.820+0000 7fa6d8000640 1 -- 172.21.15.122:0/2328444889 shutdown_connections 2024-09-17T05:40:34.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.820+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2328444889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6d00b2d00 0x7fa6d01605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.820+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2328444889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2760 0x7fa6d00d0260 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.820+0000 7fa6d8000640 1 -- 172.21.15.122:0/2328444889 >> 172.21.15.122:0/2328444889 conn(0x7fa6d00d1e40 msgr2=0x7fa6d00d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:34.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 -- 172.21.15.122:0/2328444889 shutdown_connections 2024-09-17T05:40:34.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 -- 172.21.15.122:0/2328444889 wait complete. 2024-09-17T05:40:34.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 Processor -- start 2024-09-17T05:40:34.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 -- start start 2024-09-17T05:40:34.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6d00b2760 0x7fa6d00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2d00 0x7fa6d01fad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6d00cfe80 0x7fa6d0149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d00b08f0 con 0x7fa6d00b2d00 2024-09-17T05:40:34.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d00b0770 con 0x7fa6d00cfe80 2024-09-17T05:40:34.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d8000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d00b0a70 con 0x7fa6d00b2760 2024-09-17T05:40:34.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2d00 0x7fa6d01fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2d00 0x7fa6d01fad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58788/0 (socket says 172.21.15.122:58788) 2024-09-17T05:40:34.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 -- 172.21.15.122:0/2646008127 learned_addr learned my addr 172.21.15.122:0/2646008127 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:34.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 -- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6d00b2760 msgr2=0x7fa6d00b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6d00b2760 0x7fa6d00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 -- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6d00cfe80 msgr2=0x7fa6d0149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:34.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6d00cfe80 0x7fa6d0149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:34.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 -- 172.21.15.122:0/2646008127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d01fb750 con 0x7fa6d00b2d00 2024-09-17T05:40:34.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6d4c00640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2d00 0x7fa6d01fad70 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc01adb0 tx=0x7fa6bc0a8540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:34.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.824+0000 7fa6cd600640 1 -- 172.21.15.122:0/2646008127 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0bf070 con 0x7fa6d00b2d00 2024-09-17T05:40:34.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6cd600640 1 -- 172.21.15.122:0/2646008127 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6bc005040 con 0x7fa6d00b2d00 2024-09-17T05:40:34.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d01fba30 con 0x7fa6d00b2d00 2024-09-17T05:40:34.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6cd600640 1 -- 172.21.15.122:0/2646008127 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc005430 con 0x7fa6d00b2d00 2024-09-17T05:40:34.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d01f8890 con 0x7fa6d00b2d00 2024-09-17T05:40:34.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d00d0260 con 0x7fa6d00b2d00 2024-09-17T05:40:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6cd600640 1 -- 172.21.15.122:0/2646008127 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fa6bc0c2070 con 0x7fa6d00b2d00 2024-09-17T05:40:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6cd600640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa6b00754e0 0x7fa6b00779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6cd600640 1 -- 172.21.15.122:0/2646008127 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7fa6bc12d500 con 0x7fa6d00b2d00 2024-09-17T05:40:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.828+0000 7fa6d5600640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa6b00754e0 0x7fa6b00779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.832+0000 7fa6d5600640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa6b00754e0 0x7fa6b00779a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4007fd0 tx=0x7fa6c4007940 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:34.832+0000 7fa6cd600640 1 -- 172.21.15.122:0/2646008127 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa6bc0ac030 con 0x7fa6d00b2d00 2024-09-17T05:40:35.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.032+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6d01fbc00 con 0x7fa6d00b2d00 2024-09-17T05:40:35.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.032+0000 7fa6cd600640 1 -- 172.21.15.122:0/2646008127 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5233 (secure 0 0 0) 0x7fa6bc016410 con 0x7fa6d00b2d00 2024-09-17T05:40:35.031 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:40:35.031 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":96,"btime":"2024-09-17T05:40:28:184035+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/223748651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":223748651},{"type":"v1","addr":"172.21.15.173:6837","nonce":223748651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/822031950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":822031950},{"type":"v1","addr":"172.21.15.173:6836","nonce":822031950}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2541492660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2541492660},{"type":"v1","addr":"172.21.15.122:6834","nonce":2541492660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:40:24.548833+0000","modified":"2024-09-17T05:40:27.570670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9399":{"gid":9399,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/4149068254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":4149068254},{"type":"v1","addr":"172.21.15.122:6835","nonce":4149068254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9399,"qdb_cluster":[9399]},"id":10}]} 2024-09-17T05:40:35.031 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 96 2024-09-17T05:40:35.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa6b00754e0 msgr2=0x7fa6b00779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa6b00754e0 0x7fa6b00779a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4007fd0 tx=0x7fa6c4007940 comp rx=0 tx=0).stop 2024-09-17T05:40:35.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2d00 msgr2=0x7fa6d01fad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2d00 0x7fa6d01fad70 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc01adb0 tx=0x7fa6bc0a8540 comp rx=0 tx=0).stop 2024-09-17T05:40:35.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 shutdown_connections 2024-09-17T05:40:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa6b00754e0 0x7fa6b00779a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6d00cfe80 0x7fa6d0149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6d00b2d00 0x7fa6d01fad70 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 --2- 172.21.15.122:0/2646008127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6d00b2760 0x7fa6d00b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 >> 172.21.15.122:0/2646008127 conn(0x7fa6d00d1e40 msgr2=0x7fa6d006e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 shutdown_connections 2024-09-17T05:40:35.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.036+0000 7fa6d8000640 1 -- 172.21.15.122:0/2646008127 wait complete. 2024-09-17T05:40:35.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:40:24.548833+0000', 'modified': '2024-09-17T05:40:27.570670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9399}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9399': {'gid': 9399, 'name': 'c', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6835/4149068254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6833', 'nonce': 4149068254}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 4149068254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9399, 'qdb_cluster': [9399]} 2024-09-17T05:40:35.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:40:35.456 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:40:35.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- 172.21.15.122:0/3039538743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 msgr2=0x7f02240d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 --2- 172.21.15.122:0/3039538743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 0x7f02240d01b0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0214009900 tx=0x7f021401c850 comp rx=0 tx=0).stop 2024-09-17T05:40:35.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- 172.21.15.122:0/3039538743 shutdown_connections 2024-09-17T05:40:35.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 --2- 172.21.15.122:0/3039538743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02240cefb0 0x7f02240cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 --2- 172.21.15.122:0/3039538743 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 0x7f02240d01b0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- 172.21.15.122:0/3039538743 >> 172.21.15.122:0/3039538743 conn(0x7f022409aa10 msgr2=0x7f022409ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:35.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- 172.21.15.122:0/3039538743 shutdown_connections 2024-09-17T05:40:35.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- 172.21.15.122:0/3039538743 wait complete. 2024-09-17T05:40:35.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 Processor -- start 2024-09-17T05:40:35.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- start start 2024-09-17T05:40:35.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02240ccaf0 0x7f02241fb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02240cefb0 0x7f02240b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 0x7f02240b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0224169290 con 0x7f022415f000 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0224169110 con 0x7f02240cefb0 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0229400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0224169410 con 0x7f02240ccaf0 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0228a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 0x7f02240b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0228a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 0x7f02240b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58792/0 (socket says 172.21.15.122:58792) 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0228a00640 1 -- 172.21.15.122:0/2694936924 learned_addr learned my addr 172.21.15.122:0/2694936924 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f021b400640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02240cefb0 0x7f02240b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0223e00640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02240ccaf0 0x7f02241fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0228a00640 1 -- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02240ccaf0 msgr2=0x7f02241fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0228a00640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02240ccaf0 0x7f02241fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0228a00640 1 -- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02240cefb0 msgr2=0x7f02240b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.604+0000 7f0228a00640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02240cefb0 0x7f02240b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0228a00640 1 -- 172.21.15.122:0/2694936924 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022406dd30 con 0x7f022415f000 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f021b400640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02240cefb0 0x7f02240b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0228a00640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 0x7f02240b4f40 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f021c00c730 tx=0x7f021c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0220c00640 1 -- 172.21.15.122:0/2694936924 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c003b80 con 0x7f022415f000 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0220c00640 1 -- 172.21.15.122:0/2694936924 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f021c003d20 con 0x7f022415f000 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0220c00640 1 -- 172.21.15.122:0/2694936924 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c00aab0 con 0x7f022415f000 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0229400640 1 -- 172.21.15.122:0/2694936924 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02241fbae0 con 0x7f022415f000 2024-09-17T05:40:35.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0229400640 1 -- 172.21.15.122:0/2694936924 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02241fc0a0 con 0x7f022415f000 2024-09-17T05:40:35.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0220c00640 1 -- 172.21.15.122:0/2694936924 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f021c00ac50 con 0x7f022415f000 2024-09-17T05:40:35.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0220c00640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0200075510 0x7f02000779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f022dd4e000 1 Processor -- start 2024-09-17T05:40:35.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.608+0000 7f0220c00640 1 -- 172.21.15.122:0/2694936924 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f021c093190 con 0x7f022415f000 2024-09-17T05:40:35.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- start start 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f0223e00640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0200075510 0x7f02000779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 0x563029a22db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x5630299d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 0x5630299d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5630298f73f0 con 0x563029a232f0 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563029834b20 con 0x563029a229d0 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563029823f50 con 0x5630299d6e10 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x5630299d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x5630299d68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58798/0 (socket says 172.21.15.122:58798) 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 -- 172.21.15.122:0/3954816655 learned_addr learned my addr 172.21.15.122:0/3954816655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f0223e00640 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 0x563029a22db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f0228a00640 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 0x5630299d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f0223e00640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0200075510 0x7f02000779d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f02140af000 tx=0x7f021401b090 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 -- 172.21.15.122:0/3954816655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 msgr2=0x5630299d71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 0x5630299d71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 -- 172.21.15.122:0/3954816655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 msgr2=0x563029a22db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 0x563029a22db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 -- 172.21.15.122:0/3954816655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630297eb590 con 0x563029a232f0 2024-09-17T05:40:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f021b400640 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x5630299d68d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f020c00bcb0 tx=0x7f020c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbc5c6551bc5d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:35.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f0218c00640 1 -- 172.21.15.122:0/3954816655 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c015070 con 0x563029a232f0 2024-09-17T05:40:35.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f0218c00640 1 -- 172.21.15.122:0/3954816655 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f020c005020 con 0x563029a232f0 2024-09-17T05:40:35.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f0218c00640 1 -- 172.21.15.122:0/3954816655 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c005430 con 0x563029a232f0 2024-09-17T05:40:35.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- 172.21.15.122:0/3954816655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 msgr2=0x5630299d68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x5630299d68d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f020c00bcb0 tx=0x7f020c00f730 comp rx=0 tx=0).stop 2024-09-17T05:40:35.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- 172.21.15.122:0/3954816655 shutdown_connections 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 0x5630299d71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x5630299d68d0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3954816655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 0x563029a22db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- 172.21.15.122:0/3954816655 >> 172.21.15.122:0/3954816655 conn(0x5630298c0130 msgr2=0x563029a05e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- 172.21.15.122:0/3954816655 shutdown_connections 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- 172.21.15.122:0/3954816655 wait complete. 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 -- start start 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.612+0000 7f022dd4e000 1 Processor -- start 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5630299d6e10 0x5630299d8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a229d0 0x5630299d8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a232f0 0x5630299d8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5630297be5d0 con 0x563029a229d0 2024-09-17T05:40:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563029827d60 con 0x5630299d6e10 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x5630297ca500 con 0x563029a232f0 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 10 client.0 _ll_register_callbacks cb 0x5630297ce810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a229d0 0x5630299d8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a229d0 0x5630299d8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58808/0 (socket says 172.21.15.122:58808) 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 -- 172.21.15.122:0/3208014231 learned_addr learned my addr 172.21.15.122:0/3208014231 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0223e00640 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5630299d6e10 0x5630299d8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0228a00640 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a232f0 0x5630299d8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 -- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a232f0 msgr2=0x5630299d8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:40:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a232f0 0x5630299d8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 -- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5630299d6e10 msgr2=0x5630299d8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5630299d6e10 0x5630299d8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 -- 172.21.15.122:0/3208014231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630297c2170 con 0x563029a229d0 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0223e00640 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5630299d6e10 0x5630299d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f021b400640 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a229d0 0x5630299d8780 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x5630297d5130 tx=0x7f020c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc17c81e8a4b4909 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0211600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0211600640 1 -- 172.21.15.122:0/3208014231 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c027060 con 0x563029a229d0 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0211600640 1 -- 172.21.15.122:0/3208014231 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f020c011750 con 0x563029a229d0 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 -- 172.21.15.122:0/3208014231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5630297bc0f0 con 0x563029a229d0 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0211600640 1 -- 172.21.15.122:0/3208014231 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f020c015040 con 0x563029a229d0 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 10 client.9543 fetch_fsmap learned FSMap version 96 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 -- 172.21.15.122:0/3208014231 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x5630298a8ef0 con 0x563029a229d0 2024-09-17T05:40:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0211600640 1 -- 172.21.15.122:0/3208014231 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5630297bc0f0 con 0x563029a229d0 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0211600640 1 -- 172.21.15.122:0/3208014231 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 96) ==== 3502+0+0 (secure 0 0 0) 0x7f020c011a40 con 0x563029a229d0 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 10 client.9543 fetch_fsmap finished waiting for FSMap version 96 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 10 client.9543 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 20 client.9543 populate_metadata read hostname 'smithi122' 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] conn(0x563029a31c70 0x563029a34060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 4 client.9543 mds_command: new command op to 9399 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f022dd4e000 1 -- 172.21.15.122:0/3208014231 --> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563029836a70 con 0x563029a31c70 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0223e00640 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] conn(0x563029a31c70 0x563029a34060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0223e00640 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] conn(0x563029a31c70 0x563029a34060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218e02fd5e96008 server_cookie=a84d4660315e3ed in_seq=0 out_seq=0 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.616+0000 7f0211600640 10 client.9543 ms_handle_connect on v2:172.21.15.122:6833/4149068254 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f0211600640 1 -- 172.21.15.122:0/3208014231 <== mds.0 v2:172.21.15.122:6833/4149068254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563029836a70 con 0x563029a31c70 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f0211600640 10 client.9543 handle_command_reply: tid=0 2024-09-17T05:40:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 client.9543 shutdown 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 -- 172.21.15.122:0/3208014231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a229d0 msgr2=0x5630299d8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a229d0 0x5630299d8780 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x5630297d5130 tx=0x7f020c00bfd0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 -- 172.21.15.122:0/3208014231 shutdown_connections 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a232f0 0x5630299d8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a229d0 0x5630299d8780 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] conn(0x563029a31c70 0x563029a34060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 --2- 172.21.15.122:0/3208014231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5630299d6e10 0x5630299d8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 -- 172.21.15.122:0/3208014231 >> 172.21.15.122:0/3208014231 conn(0x5630298c0130 msgr2=0x5630299d5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f0211600640 0 client.9543 ms_handle_reset on v2:172.21.15.122:6833/4149068254 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 -- 172.21.15.122:0/3208014231 shutdown_connections 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 1 -- 172.21.15.122:0/3208014231 wait complete. 2024-09-17T05:40:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.620+0000 7f022dd4e000 20 client.9543 trim_cache size 0 max 16384 2024-09-17T05:40:35.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.636+0000 7f022dd4e000 1 Processor -- start 2024-09-17T05:40:35.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.636+0000 7f022dd4e000 1 -- start start 2024-09-17T05:40:35.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.636+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5630299d6e10 0x563029a943c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.636+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a229d0 0x563029a94900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.636+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a232f0 0x563029a94e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.636+0000 7f022dd4e000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0224203810 con 0x5630299d6e10 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.636+0000 7f022dd4e000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f022406ee30 con 0x563029a232f0 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.636+0000 7f022dd4e000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f022406f450 con 0x563029a229d0 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5630299d6e10 0x563029a943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5630299d6e10 0x563029a943c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58820/0 (socket says 172.21.15.122:58820) 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 -- 172.21.15.122:0/2859953222 learned_addr learned my addr 172.21.15.122:0/2859953222 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f021b400640 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a229d0 0x563029a94900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0228a00640 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a232f0 0x563029a94e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 -- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a229d0 msgr2=0x563029a94900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a229d0 0x563029a94900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 -- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a232f0 msgr2=0x563029a94e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a232f0 0x563029a94e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 -- 172.21.15.122:0/2859953222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302987fd90 con 0x5630299d6e10 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f021b400640 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a229d0 0x563029a94900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0228a00640 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a232f0 0x563029a94e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5630299d6e10 0x563029a943c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0214009230 tx=0x7f02140b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ed9349d50bc77c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:35.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0213e00640 1 -- 172.21.15.122:0/2859953222 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02140b2040 con 0x5630299d6e10 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0213e00640 1 -- 172.21.15.122:0/2859953222 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02140051c0 con 0x5630299d6e10 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0213e00640 1 -- 172.21.15.122:0/2859953222 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02140055b0 con 0x5630299d6e10 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- 172.21.15.122:0/2859953222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5630299d6e10 msgr2=0x563029a943c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5630299d6e10 0x563029a943c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f0214009230 tx=0x7f02140b4000 comp rx=0 tx=0).stop 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- 172.21.15.122:0/2859953222 shutdown_connections 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a232f0 0x563029a94e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563029a229d0 0x563029a94900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 --2- 172.21.15.122:0/2859953222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5630299d6e10 0x563029a943c0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- 172.21.15.122:0/2859953222 >> 172.21.15.122:0/2859953222 conn(0x5630298c0130 msgr2=0x5630299d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:35.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- 172.21.15.122:0/2859953222 shutdown_connections 2024-09-17T05:40:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- 172.21.15.122:0/2859953222 wait complete. 2024-09-17T05:40:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- start start 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 Processor -- start 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 0x563029a38c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 0x563029a54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x563029a54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5630298338f0 con 0x563029a232f0 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5630297efec0 con 0x563029a229d0 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563029827070 con 0x5630299d6e10 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f022dd4e000 10 client.0 _ll_register_callbacks cb 0x7f02140b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0228a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x563029a54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0228a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x563029a54ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58822/0 (socket says 172.21.15.122:58822) 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0228a00640 1 -- 172.21.15.122:0/463813890 learned_addr learned my addr 172.21.15.122:0/463813890 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f0223e00640 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 0x563029a38c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.640+0000 7f021b400640 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 0x563029a54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0228a00640 1 -- 172.21.15.122:0/463813890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 msgr2=0x563029a38c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0228a00640 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 0x563029a38c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0228a00640 1 -- 172.21.15.122:0/463813890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 msgr2=0x563029a54580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0228a00640 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 0x563029a54580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0228a00640 1 -- 172.21.15.122:0/463813890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630297eaa90 con 0x563029a232f0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0228a00640 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x563029a54ac0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f021c00a430 tx=0x7f021c05bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e43915c731dd37 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 1 -- 172.21.15.122:0/463813890 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c079020 con 0x563029a232f0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 1 -- 172.21.15.122:0/463813890 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f021c05aef0 con 0x563029a232f0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 -- 172.21.15.122:0/463813890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02140051c0 con 0x563029a232f0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 1 -- 172.21.15.122:0/463813890 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f021c084070 con 0x563029a232f0 2024-09-17T05:40:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 10 client.9549 fetch_fsmap learned FSMap version 96 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 -- 172.21.15.122:0/463813890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x5630297c9050 con 0x563029a232f0 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 1 -- 172.21.15.122:0/463813890 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02140051c0 con 0x563029a232f0 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 1 -- 172.21.15.122:0/463813890 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 96) ==== 3502+0+0 (secure 0 0 0) 0x7f021c06d080 con 0x563029a232f0 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 10 client.9549 fetch_fsmap finished waiting for FSMap version 96 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 10 client.9549 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 20 client.9549 populate_metadata read hostname 'smithi122' 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] conn(0x563029a55000 0x563029a98400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 4 client.9549 mds_command: new command op to 9399 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 -- 172.21.15.122:0/463813890 --> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] -- command(tid 0: {"prefix": "status"}) -- 0x563029832520 con 0x563029a55000 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0223e00640 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] conn(0x563029a55000 0x563029a98400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0223e00640 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] conn(0x563029a55000 0x563029a98400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d73536fc72358e server_cookie=b282484bc4aa8e44 in_seq=0 out_seq=0 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 10 client.9549 ms_handle_connect on v2:172.21.15.122:6833/4149068254 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 1 -- 172.21.15.122:0/463813890 <== mds.0 v2:172.21.15.122:6833/4149068254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563029832520 con 0x563029a55000 2024-09-17T05:40:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 10 client.9549 handle_command_reply: tid=0 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 client.9549 shutdown 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 -- 172.21.15.122:0/463813890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 msgr2=0x563029a54ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x563029a54ac0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f021c00a430 tx=0x7f021c05bd20 comp rx=0 tx=0).stop 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 -- 172.21.15.122:0/463813890 shutdown_connections 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563029a232f0 0x563029a54ac0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.122:6833/4149068254,v1:172.21.15.122:6835/4149068254] conn(0x563029a55000 0x563029a98400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563029a229d0 0x563029a54580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 --2- 172.21.15.122:0/463813890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x5630299d6e10 0x563029a38c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 -- 172.21.15.122:0/463813890 >> 172.21.15.122:0/463813890 conn(0x5630298c0130 msgr2=0x563029a34060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0211600640 0 client.9549 ms_handle_reset on v2:172.21.15.122:6833/4149068254 2024-09-17T05:40:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 -- 172.21.15.122:0/463813890 shutdown_connections 2024-09-17T05:40:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 1 -- 172.21.15.122:0/463813890 wait complete. 2024-09-17T05:40:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f022dd4e000 20 client.9549 trim_cache size 0 max 16384 2024-09-17T05:40:35.649 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:40:35.649 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 9399, 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 10.477020827, 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 96, 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 145, 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 145, 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 14.991184197000001 2024-09-17T05:40:35.650 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:40:35.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 -- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0200075510 msgr2=0x7f02000779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0200075510 0x7f02000779d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f02140af000 tx=0x7f021401b090 comp rx=0 tx=0).stop 2024-09-17T05:40:35.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 -- 172.21.15.122:0/2694936924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 msgr2=0x7f02240b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 0x7f02240b4f40 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f021c00c730 tx=0x7f021c00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 -- 172.21.15.122:0/2694936924 shutdown_connections 2024-09-17T05:40:35.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0200075510 0x7f02000779d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f022415f000 0x7f02240b4f40 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02240cefb0 0x7f02240b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 --2- 172.21.15.122:0/2694936924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02240ccaf0 0x7f02241fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 -- 172.21.15.122:0/2694936924 >> 172.21.15.122:0/2694936924 conn(0x7f022409aa10 msgr2=0x7f02240ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:35.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 -- 172.21.15.122:0/2694936924 shutdown_connections 2024-09-17T05:40:35.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.644+0000 7f0229400640 1 -- 172.21.15.122:0/2694936924 wait complete. 2024-09-17T05:40:35.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:40:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.808+0000 7f529fe00640 1 -- 172.21.15.122:0/516323370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980cdb30 msgr2=0x7f52980af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.808+0000 7f529fe00640 1 --2- 172.21.15.122:0/516323370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980cdb30 0x7f52980af0c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f5290009bc0 tx=0x7f529001cba0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.808+0000 7f529fe00640 1 -- 172.21.15.122:0/516323370 shutdown_connections 2024-09-17T05:40:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.808+0000 7f529fe00640 1 --2- 172.21.15.122:0/516323370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980cdb30 0x7f52980af0c0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.808+0000 7f529fe00640 1 -- 172.21.15.122:0/516323370 >> 172.21.15.122:0/516323370 conn(0x7f52980d1dc0 msgr2=0x7f52980d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:35.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.808+0000 7f529fe00640 1 -- 172.21.15.122:0/516323370 shutdown_connections 2024-09-17T05:40:35.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.808+0000 7f529fe00640 1 -- 172.21.15.122:0/516323370 wait complete. 2024-09-17T05:40:35.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.808+0000 7f529fe00640 1 Processor -- start 2024-09-17T05:40:35.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 -- start start 2024-09-17T05:40:35.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52980cdb30 0x7f52981583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980ced80 0x7f52980b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f529815bea0 0x7f52980cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f529816a8a0 con 0x7f52980ced80 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f529816a720 con 0x7f529815bea0 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f529816aa20 con 0x7f52980cdb30 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980ced80 0x7f52980b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980ced80 0x7f52980b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58832/0 (socket says 172.21.15.122:58832) 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 -- 172.21.15.122:0/3318350699 learned_addr learned my addr 172.21.15.122:0/3318350699 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529de00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f529815bea0 0x7f52980cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529d400640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52980cdb30 0x7f52981583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 -- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52980cdb30 msgr2=0x7f52981583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52980cdb30 0x7f52981583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 -- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f529815bea0 msgr2=0x7f52980cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:35.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f529815bea0 0x7f52980cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:35.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 -- 172.21.15.122:0/3318350699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52980cbcc0 con 0x7f52980ced80 2024-09-17T05:40:35.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529ca00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980ced80 0x7f52980b2210 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f528400c4b0 tx=0x7f528400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:35.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f5295600640 1 -- 172.21.15.122:0/3318350699 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284016020 con 0x7f52980ced80 2024-09-17T05:40:35.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f5295600640 1 -- 172.21.15.122:0/3318350699 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5284012920 con 0x7f52980ced80 2024-09-17T05:40:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f5295600640 1 -- 172.21.15.122:0/3318350699 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284012d90 con 0x7f52980ced80 2024-09-17T05:40:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52980b1450 con 0x7f52980ced80 2024-09-17T05:40:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52980b1a10 con 0x7f52980ced80 2024-09-17T05:40:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.812+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52980ce670 con 0x7f52980ced80 2024-09-17T05:40:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.816+0000 7f5295600640 1 -- 172.21.15.122:0/3318350699 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f5284019020 con 0x7f52980ced80 2024-09-17T05:40:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.816+0000 7f5295600640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52780754e0 0x7f52780779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.816+0000 7f5295600640 1 -- 172.21.15.122:0/3318350699 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f5284092fa0 con 0x7f52980ced80 2024-09-17T05:40:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.816+0000 7f529d400640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52780754e0 0x7f52780779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.816+0000 7f529d400640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52780754e0 0x7f52780779a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f528c001610 tx=0x7f528c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:35.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:35.820+0000 7f5295600640 1 -- 172.21.15.122:0/3318350699 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5284020030 con 0x7f52980ced80 2024-09-17T05:40:36.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.028+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52980cb7f0 con 0x7f52980ced80 2024-09-17T05:40:36.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.028+0000 7f5295600640 1 -- 172.21.15.122:0/3318350699 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) ==== 53+0+73 (secure 0 0 0) 0x7f528405fab0 con 0x7f52980ced80 2024-09-17T05:40:36.026 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:40:36.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52780754e0 msgr2=0x7f52780779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:36.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52780754e0 0x7f52780779a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f528c001610 tx=0x7f528c009040 comp rx=0 tx=0).stop 2024-09-17T05:40:36.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980ced80 msgr2=0x7f52980b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:36.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980ced80 0x7f52980b2210 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f528400c4b0 tx=0x7f528400c970 comp rx=0 tx=0).stop 2024-09-17T05:40:36.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 shutdown_connections 2024-09-17T05:40:36.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52780754e0 0x7f52780779a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f529815bea0 0x7f52980cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52980ced80 0x7f52980b2210 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 --2- 172.21.15.122:0/3318350699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52980cdb30 0x7f52981583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 >> 172.21.15.122:0/3318350699 conn(0x7f52980d1dc0 msgr2=0x7f52980d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:36.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 shutdown_connections 2024-09-17T05:40:36.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.032+0000 7f529fe00640 1 -- 172.21.15.122:0/3318350699 wait complete. 2024-09-17T05:40:36.048 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:40:36.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:40:36.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-09-17T05:40:36.048 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:40:36.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:40:36.049 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:40:36.049 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:40:36.049 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:40:36.049 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:40:36.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:36.049 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-09-17T05:40:36.058 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:40:36.058 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:40:36.058 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:40:36.058 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:40:36.058 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-09-17T05:40:36.058 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 41531sec preferred_lft 41531sec 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-09-17T05:40:36.059 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:79:39:fe:f6 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:40:36.060 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:40:36.061 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:40:36.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:36.062 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:40:36.120 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-17T05:40:36.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:36.123 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-09-17T05:40:36.175 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-09-17T05:40:36.175 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:40:36.175 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:40:36.175 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:40:36.175 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:40:36.175 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:40:36.175 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:40:36.176 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:40:36.176 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:40:36.176 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:40:36.176 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:40:36.176 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:40:36.176 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-09-17T05:40:36.176 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:40:36.176 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:40:36.176 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:40:36.177 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-09-17T05:40:36.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:40:36.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:40:36.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:36.312 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:40:36.322 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:40:36.322 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:40:36.322 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:40:36.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:36.322 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-09-17T05:40:36.383 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:40:36.383 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:40:36.431 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:40:36.431 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:40:36.432 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:40:36.482 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:40:36.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:36.483 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:40:36.574 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:40:36.575 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:40:36.575 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:40:36.590 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:40:36.590 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:40:36.590 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:40:36.590 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:40:36.590 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:40:36.599 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:40:36.600 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:40:36.600 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-09-17T05:40:36.600 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:3300,172.21.15.173:3300,172.21.15.173:3301" 2024-09-17T05:40:36.600 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:40:36.600 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790" 2024-09-17T05:40:36.600 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.173:6789,172.21.15.173:6790:/ 2024-09-17T05:40:36.600 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=51547930-c98d-41c6-b8ce-f14d7623b707" will pass to kernel 2024-09-17T05:40:36.604 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:40:36.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:36.604 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:40:36.624 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:40:36.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:40:36.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:40:36.737 INFO:teuthology.orchestra.run.smithi122.stdout:client7799 2024-09-17T05:40:36.737 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:36.737 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:36.746 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-17T05:40:36.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.936+0000 7f6fc9a00640 1 -- 172.21.15.122:0/3911718678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc415a8c0 msgr2=0x7f6fc415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:36.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.936+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/3911718678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc415a8c0 0x7f6fc415f100 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40d02b0 tx=0x7f6fb401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.936+0000 7f6fc9a00640 1 -- 172.21.15.122:0/3911718678 shutdown_connections 2024-09-17T05:40:36.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.936+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/3911718678 >> v2:172.21.15.173:3300/0 conn(0x7f6fc4073830 0x7f6fc406fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.936+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/3911718678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc415a8c0 0x7f6fc415f100 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.936+0000 7f6fc9a00640 1 -- 172.21.15.122:0/3911718678 >> 172.21.15.122:0/3911718678 conn(0x7f6fc40b2140 msgr2=0x7f6fc40b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:36.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.936+0000 7f6fc9a00640 1 -- 172.21.15.122:0/3911718678 shutdown_connections 2024-09-17T05:40:36.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.936+0000 7f6fc9a00640 1 -- 172.21.15.122:0/3911718678 wait complete. 2024-09-17T05:40:36.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 Processor -- start 2024-09-17T05:40:36.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 -- start start 2024-09-17T05:40:36.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc4073830 0x7f6fc4150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fc415a8c0 0x7f6fc41544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fc415aca0 0x7f6fc40ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4167a90 con 0x7f6fc4073830 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4070dd0 con 0x7f6fc415aca0 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc4167c10 con 0x7f6fc415a8c0 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc4073830 0x7f6fc4150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc4073830 0x7f6fc4150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58846/0 (socket says 172.21.15.122:58846) 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 -- 172.21.15.122:0/2011836308 learned_addr learned my addr 172.21.15.122:0/2011836308 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 -- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fc415a8c0 msgr2=0x7f6fc41544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fc415a8c0 0x7f6fc41544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 -- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fc415aca0 msgr2=0x7f6fc40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fc415aca0 0x7f6fc40ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:36.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 -- 172.21.15.122:0/2011836308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc41f5b80 con 0x7f6fc4073830 2024-09-17T05:40:36.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc2a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc4073830 0x7f6fc4150b70 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40b0040 tx=0x7f6fb40a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:36.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fbaa00640 1 -- 172.21.15.122:0/2011836308 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40b2040 con 0x7f6fc4073830 2024-09-17T05:40:36.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fbaa00640 1 -- 172.21.15.122:0/2011836308 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb4005020 con 0x7f6fc4073830 2024-09-17T05:40:36.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc41f5e60 con 0x7f6fc4073830 2024-09-17T05:40:36.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fbaa00640 1 -- 172.21.15.122:0/2011836308 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4005470 con 0x7f6fc4073830 2024-09-17T05:40:36.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.940+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc41f6420 con 0x7f6fc4073830 2024-09-17T05:40:36.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.944+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc406fae0 con 0x7f6fc4073830 2024-09-17T05:40:36.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.944+0000 7f6fbaa00640 1 -- 172.21.15.122:0/2011836308 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f6fb40046d0 con 0x7f6fc4073830 2024-09-17T05:40:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.944+0000 7f6fbaa00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6fa40754e0 0x7f6fa40779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.944+0000 7f6fbaa00640 1 -- 172.21.15.122:0/2011836308 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f6fb412cfe0 con 0x7f6fc4073830 2024-09-17T05:40:36.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.944+0000 7f6fc2000640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6fa40754e0 0x7f6fa40779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:36.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.948+0000 7f6fc2000640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6fa40754e0 0x7f6fa40779a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0007fd0 tx=0x7f6fb0007940 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:36.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:36.948+0000 7f6fbaa00640 1 -- 172.21.15.122:0/2011836308 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6fb40164b0 con 0x7f6fc4073830 2024-09-17T05:40:37.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.152+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f6fc4157310 con 0x7f6fc4073830 2024-09-17T05:40:37.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.216+0000 7f6fbaa00640 1 -- 172.21.15.122:0/2011836308 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v97) ==== 136+0+0 (secure 0 0 0) 0x7f6fb40f9af0 con 0x7f6fc4073830 2024-09-17T05:40:37.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6fa40754e0 msgr2=0x7f6fa40779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6fa40754e0 0x7f6fa40779a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0007fd0 tx=0x7f6fb0007940 comp rx=0 tx=0).stop 2024-09-17T05:40:37.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc4073830 msgr2=0x7f6fc4150b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc4073830 0x7f6fc4150b70 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40b0040 tx=0x7f6fb40a57b0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 shutdown_connections 2024-09-17T05:40:37.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6fa40754e0 0x7f6fa40779a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fc415aca0 0x7f6fc40ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fc415a8c0 0x7f6fc41544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 --2- 172.21.15.122:0/2011836308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fc4073830 0x7f6fc4150b70 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 >> 172.21.15.122:0/2011836308 conn(0x7f6fc40b2140 msgr2=0x7f6fc4164f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:37.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 shutdown_connections 2024-09-17T05:40:37.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.220+0000 7f6fc9a00640 1 -- 172.21.15.122:0/2011836308 wait complete. 2024-09-17T05:40:37.238 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:40:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 -- 172.21.15.122:0/2634276490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c15db50 msgr2=0x7f203c0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 --2- 172.21.15.122:0/2634276490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c15db50 0x7f203c0acd40 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f202c00acd0 tx=0x7f202c01de00 comp rx=0 tx=0).stop 2024-09-17T05:40:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 -- 172.21.15.122:0/2634276490 shutdown_connections 2024-09-17T05:40:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 --2- 172.21.15.122:0/2634276490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203c0cd100 0x7f203c0b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 --2- 172.21.15.122:0/2634276490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c15db50 0x7f203c0acd40 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 -- 172.21.15.122:0/2634276490 >> 172.21.15.122:0/2634276490 conn(0x7f203c0b18f0 msgr2=0x7f203c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 -- 172.21.15.122:0/2634276490 shutdown_connections 2024-09-17T05:40:37.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 -- 172.21.15.122:0/2634276490 wait complete. 2024-09-17T05:40:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 Processor -- start 2024-09-17T05:40:37.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.384+0000 7f2040a00640 1 -- start start 2024-09-17T05:40:37.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c0b4880 0x7f203c1fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:37.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203c0cd100 0x7f203c1649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203c15db50 0x7f203c09c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f203c167120 con 0x7f203c0b4880 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f203c0ccd20 con 0x7f203c0cd100 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f203c1672a0 con 0x7f203c15db50 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c0b4880 0x7f203c1fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c0b4880 0x7f203c1fbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58858/0 (socket says 172.21.15.122:58858) 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 -- 172.21.15.122:0/1429349413 learned_addr learned my addr 172.21.15.122:0/1429349413 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 -- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203c15db50 msgr2=0x7f203c09c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203c15db50 0x7f203c09c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2031400640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203c0cd100 0x7f203c1649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 -- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203c0cd100 msgr2=0x7f203c1649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203c0cd100 0x7f203c1649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 -- 172.21.15.122:0/1429349413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203c164f30 con 0x7f203c0b4880 2024-09-17T05:40:37.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2031400640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203c0cd100 0x7f203c1649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:37.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2039e00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c0b4880 0x7f203c1fbc10 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f202c00ac20 tx=0x7f202c0a6a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:37.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2032a00640 1 -- 172.21.15.122:0/1429349413 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c0b3040 con 0x7f203c0b4880 2024-09-17T05:40:37.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2032a00640 1 -- 172.21.15.122:0/1429349413 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f202c004c50 con 0x7f203c0b4880 2024-09-17T05:40:37.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203c09d040 con 0x7f203c0b4880 2024-09-17T05:40:37.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2032a00640 1 -- 172.21.15.122:0/1429349413 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c004320 con 0x7f203c0b4880 2024-09-17T05:40:37.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f203c09d600 con 0x7f203c0b4880 2024-09-17T05:40:37.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.388+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203c0b4340 con 0x7f203c0b4880 2024-09-17T05:40:37.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.392+0000 7f2032a00640 1 -- 172.21.15.122:0/1429349413 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f202c0044c0 con 0x7f203c0b4880 2024-09-17T05:40:37.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.392+0000 7f2032a00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2018075510 0x7f20180779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:37.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.392+0000 7f2032a00640 1 -- 172.21.15.122:0/1429349413 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f202c12d9b0 con 0x7f203c0b4880 2024-09-17T05:40:37.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.392+0000 7f2031400640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2018075510 0x7f20180779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:37.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.392+0000 7f2031400640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2018075510 0x7f20180779d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2024005fd0 tx=0x7f2024005e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.396+0000 7f2032a00640 1 -- 172.21.15.122:0/1429349413 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f202c0c5060 con 0x7f203c0b4880 2024-09-17T05:40:37.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.584+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f203c0b1ea0 con 0x7f2018075510 2024-09-17T05:40:37.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.584+0000 7f2032a00640 1 -- 172.21.15.122:0/1429349413 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f203c0b1ea0 con 0x7f2018075510 2024-09-17T05:40:37.583 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:40:37.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:40:37.584 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:40:37.584 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:40:37.584 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2018075510 msgr2=0x7f20180779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2018075510 0x7f20180779d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2024005fd0 tx=0x7f2024005e80 comp rx=0 tx=0).stop 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c0b4880 msgr2=0x7f203c1fbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c0b4880 0x7f203c1fbc10 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f202c00ac20 tx=0x7f202c0a6a90 comp rx=0 tx=0).stop 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 shutdown_connections 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2018075510 0x7f20180779d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203c15db50 0x7f203c09c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203c0cd100 0x7f203c1649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 --2- 172.21.15.122:0/1429349413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f203c0b4880 0x7f203c1fbc10 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 >> 172.21.15.122:0/1429349413 conn(0x7f203c0b18f0 msgr2=0x7f203c156910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 shutdown_connections 2024-09-17T05:40:37.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.588+0000 7f2040a00640 1 -- 172.21.15.122:0/1429349413 wait complete. 2024-09-17T05:40:37.605 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:40:37.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 -- 172.21.15.122:0/1184930946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb53815f340 msgr2=0x7fb5380b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 --2- 172.21.15.122:0/1184930946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb53815f340 0x7fb5380b34a0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb528009840 tx=0x7fb52801c7f0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 -- 172.21.15.122:0/1184930946 shutdown_connections 2024-09-17T05:40:37.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 --2- 172.21.15.122:0/1184930946 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5380b2a00 0x7fb5380b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 --2- 172.21.15.122:0/1184930946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb53815f340 0x7fb5380b34a0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 -- 172.21.15.122:0/1184930946 >> 172.21.15.122:0/1184930946 conn(0x7fb5380d1e00 msgr2=0x7fb5380d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:37.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 -- 172.21.15.122:0/1184930946 shutdown_connections 2024-09-17T05:40:37.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 -- 172.21.15.122:0/1184930946 wait complete. 2024-09-17T05:40:37.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 Processor -- start 2024-09-17T05:40:37.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.752+0000 7fb53cc00640 1 -- start start 2024-09-17T05:40:37.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5380b2a00 0x7fb538157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:37.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5380ccf10 0x7fb5380ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:37.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb53815f340 0x7fb5381f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:37.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb538169020 con 0x7fb5380b2a00 2024-09-17T05:40:37.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb538168ea0 con 0x7fb53815f340 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb5381691a0 con 0x7fb5380ccf10 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5380b2a00 0x7fb538157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5380b2a00 0x7fb538157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58870/0 (socket says 172.21.15.122:58870) 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 -- 172.21.15.122:0/2022839972 learned_addr learned my addr 172.21.15.122:0/2022839972 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 -- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5380ccf10 msgr2=0x7fb5380ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb52fe00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5380ccf10 0x7fb5380ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb536800640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb53815f340 0x7fb5381f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5380ccf10 0x7fb5380ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 -- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb53815f340 msgr2=0x7fb5381f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb53815f340 0x7fb5381f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 -- 172.21.15.122:0/2022839972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb538165050 con 0x7fb5380b2a00 2024-09-17T05:40:37.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb52fe00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5380ccf10 0x7fb5380ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb536800640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb53815f340 0x7fb5381f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb535e00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5380b2a00 0x7fb538157470 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb528009790 tx=0x7fb528005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb52e000640 1 -- 172.21.15.122:0/2022839972 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5280b2040 con 0x7fb5380b2a00 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb52e000640 1 -- 172.21.15.122:0/2022839972 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5280044b0 con 0x7fb5380b2a00 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5381fbf50 con 0x7fb5380b2a00 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb52e000640 1 -- 172.21.15.122:0/2022839972 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528003800 con 0x7fb5380b2a00 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5381fbc70 con 0x7fb5380b2a00 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5381f79f0 con 0x7fb5380b2a00 2024-09-17T05:40:37.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.756+0000 7fb52e000640 1 -- 172.21.15.122:0/2022839972 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fb5280bd060 con 0x7fb5380b2a00 2024-09-17T05:40:37.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.760+0000 7fb52e000640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5140754e0 0x7fb5140779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:37.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.760+0000 7fb52e000640 1 -- 172.21.15.122:0/2022839972 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7fb52812cbf0 con 0x7fb5380b2a00 2024-09-17T05:40:37.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.760+0000 7fb52fe00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5140754e0 0x7fb5140779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:37.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.760+0000 7fb52fe00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5140754e0 0x7fb5140779a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb5380cbf40 tx=0x7fb520008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:37.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.764+0000 7fb52e000640 1 -- 172.21.15.122:0/2022839972 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb528016410 con 0x7fb5380b2a00 2024-09-17T05:40:37.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.928+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb5380cd3a0 con 0x7fb5380b2a00 2024-09-17T05:40:37.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb52e000640 1 -- 172.21.15.122:0/2022839972 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7fb5280f97a0 con 0x7fb5380b2a00 2024-09-17T05:40:37.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5140754e0 msgr2=0x7fb5140779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5140754e0 0x7fb5140779a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb5380cbf40 tx=0x7fb520008040 comp rx=0 tx=0).stop 2024-09-17T05:40:37.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5380b2a00 msgr2=0x7fb538157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:37.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5380b2a00 0x7fb538157470 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fb528009790 tx=0x7fb528005770 comp rx=0 tx=0).stop 2024-09-17T05:40:37.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 shutdown_connections 2024-09-17T05:40:37.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb5140754e0 0x7fb5140779a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb53815f340 0x7fb5381f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5380ccf10 0x7fb5380ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 --2- 172.21.15.122:0/2022839972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5380b2a00 0x7fb538157470 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:37.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 >> 172.21.15.122:0/2022839972 conn(0x7fb5380d1e00 msgr2=0x7fb53806e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:37.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 shutdown_connections 2024-09-17T05:40:37.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:37.932+0000 7fb53cc00640 1 -- 172.21.15.122:0/2022839972 wait complete. 2024-09-17T05:40:37.949 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-17T05:40:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 -- 172.21.15.122:0/318341663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66141603f0 msgr2=0x7f66140b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 --2- 172.21.15.122:0/318341663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66141603f0 0x7f66140b2390 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f66140b7d20 tx=0x7f660401d2c0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 -- 172.21.15.122:0/318341663 shutdown_connections 2024-09-17T05:40:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 --2- 172.21.15.122:0/318341663 >> v2:172.21.15.173:3300/0 conn(0x7f66140b28d0 0x7f66140ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 --2- 172.21.15.122:0/318341663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140b1190 0x7f66140ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 --2- 172.21.15.122:0/318341663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66141603f0 0x7f66140b2390 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 -- 172.21.15.122:0/318341663 >> 172.21.15.122:0/318341663 conn(0x7f66140d1df0 msgr2=0x7f66140d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 -- 172.21.15.122:0/318341663 shutdown_connections 2024-09-17T05:40:38.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 -- 172.21.15.122:0/318341663 wait complete. 2024-09-17T05:40:38.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 Processor -- start 2024-09-17T05:40:38.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.096+0000 7f6618e00640 1 -- start start 2024-09-17T05:40:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140b1190 0x7f66141572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66140b28d0 0x7f66141579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66141603f0 0x7f66140c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66140cce80 con 0x7f66140b28d0 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66140ccd00 con 0x7f66141603f0 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f66140cd000 con 0x7f66140b1190 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66140b28d0 0x7f66141579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66140b28d0 0x7f66141579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58882/0 (socket says 172.21.15.122:58882) 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 -- 172.21.15.122:0/2230741147 learned_addr learned my addr 172.21.15.122:0/2230741147 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140b1190 0x7f66141572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6612800640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66141603f0 0x7f66140c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 -- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140b1190 msgr2=0x7f66141572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140b1190 0x7f66141572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 -- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66141603f0 msgr2=0x7f66140c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66141603f0 0x7f66140c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 -- 172.21.15.122:0/2230741147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6614158350 con 0x7f66140b28d0 2024-09-17T05:40:38.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6611400640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66140b28d0 0x7f66141579e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f660000d980 tx=0x7f660000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:38.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f660a000640 1 -- 172.21.15.122:0/2230741147 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600017070 con 0x7f66140b28d0 2024-09-17T05:40:38.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f660a000640 1 -- 172.21.15.122:0/2230741147 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6600009d20 con 0x7f66140b28d0 2024-09-17T05:40:38.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6614158630 con 0x7f66140b28d0 2024-09-17T05:40:38.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f660a000640 1 -- 172.21.15.122:0/2230741147 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000106c0 con 0x7f66140b28d0 2024-09-17T05:40:38.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66140c9060 con 0x7f66140b28d0 2024-09-17T05:40:38.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6614158b50 con 0x7f66140b28d0 2024-09-17T05:40:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.100+0000 7f660a000640 1 -- 172.21.15.122:0/2230741147 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7f6600010860 con 0x7f66140b28d0 2024-09-17T05:40:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.104+0000 7f660a000640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65f4075510 0x7f65f40779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.104+0000 7f660a000640 1 -- 172.21.15.122:0/2230741147 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7f66000935e0 con 0x7f66140b28d0 2024-09-17T05:40:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.104+0000 7f6611e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65f4075510 0x7f65f40779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.104+0000 7f6611e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65f4075510 0x7f65f40779d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f66040af000 tx=0x7f660401b100 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:38.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.108+0000 7f660a000640 1 -- 172.21.15.122:0/2230741147 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f660005fb40 con 0x7f66140b28d0 2024-09-17T05:40:38.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.276+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f66140b2390 con 0x7f66140b28d0 2024-09-17T05:40:38.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.276+0000 7f660a000640 1 -- 172.21.15.122:0/2230741147 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v54) ==== 116+0+2 (secure 0 0 0) 0x7f6600064570 con 0x7f66140b28d0 2024-09-17T05:40:38.276 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65f4075510 msgr2=0x7f65f40779d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65f4075510 0x7f65f40779d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f66040af000 tx=0x7f660401b100 comp rx=0 tx=0).stop 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66140b28d0 msgr2=0x7f66141579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66140b28d0 0x7f66141579e0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f660000d980 tx=0x7f660000de40 comp rx=0 tx=0).stop 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 shutdown_connections 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f65f4075510 0x7f65f40779d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66141603f0 0x7f66140c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66140b28d0 0x7f66141579e0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 --2- 172.21.15.122:0/2230741147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140b1190 0x7f66141572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 >> 172.21.15.122:0/2230741147 conn(0x7f66140d1df0 msgr2=0x7f661415ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:38.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 shutdown_connections 2024-09-17T05:40:38.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.280+0000 7f6618e00640 1 -- 172.21.15.122:0/2230741147 wait complete. 2024-09-17T05:40:38.297 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:40:38.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.444+0000 7fd7f9a00640 1 -- 172.21.15.122:0/1965398037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 msgr2=0x7fd7d8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.444+0000 7fd7f9a00640 1 --2- 172.21.15.122:0/1965398037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 0x7fd7d8003e80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009b60 tx=0x7fd7e401d270 comp rx=0 tx=0).stop 2024-09-17T05:40:38.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.444+0000 7fd7f3400640 1 -- 172.21.15.122:0/1965398037 reap_dead start 2024-09-17T05:40:38.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.444+0000 7fd7f9a00640 1 -- 172.21.15.122:0/1965398037 shutdown_connections 2024-09-17T05:40:38.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.444+0000 7fd7f9a00640 1 -- 172.21.15.122:0/1965398037 >> 172.21.15.122:0/1965398037 conn(0x7fd7f40d1dc0 msgr2=0x7fd7f40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:38.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.444+0000 7fd7f9a00640 1 -- 172.21.15.122:0/1965398037 shutdown_connections 2024-09-17T05:40:38.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.444+0000 7fd7f9a00640 1 -- 172.21.15.122:0/1965398037 wait complete. 2024-09-17T05:40:38.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 Processor -- start 2024-09-17T05:40:38.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 -- start start 2024-09-17T05:40:38.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 0x7fd7f40ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7f40c86a0 0x7fd7f40ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7f40cf440 0x7fd7f415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40af650 con 0x7fd7d8003a60 2024-09-17T05:40:38.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f40d0590 con 0x7fd7f40c86a0 2024-09-17T05:40:38.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f40af7d0 con 0x7fd7f40cf440 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 0x7fd7f40ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 0x7fd7f40ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58892/0 (socket says 172.21.15.122:58892) 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 -- 172.21.15.122:0/3553636629 learned_addr learned my addr 172.21.15.122:0/3553636629 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7ea000640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7f40c86a0 0x7fd7f40ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f3400640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7f40cf440 0x7fd7f415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 -- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7f40cf440 msgr2=0x7fd7f415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7f40cf440 0x7fd7f415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 -- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7f40c86a0 msgr2=0x7fd7f40ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7f40c86a0 0x7fd7f40ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 -- 172.21.15.122:0/3553636629 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f40c9af0 con 0x7fd7d8003a60 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f3400640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7f40cf440 0x7fd7f415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:40:38.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f2a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 0x7fd7f40ce3d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd7e401d0a0 tx=0x7fd7e4005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:38.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7eb400640 1 -- 172.21.15.122:0/3553636629 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40b1040 con 0x7fd7d8003a60 2024-09-17T05:40:38.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7eb400640 1 -- 172.21.15.122:0/3553636629 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7e40bb070 con 0x7fd7d8003a60 2024-09-17T05:40:38.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7eb400640 1 -- 172.21.15.122:0/3553636629 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40043a0 con 0x7fd7d8003a60 2024-09-17T05:40:38.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f40c9dd0 con 0x7fd7d8003a60 2024-09-17T05:40:38.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.448+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f40cb4a0 con 0x7fd7d8003a60 2024-09-17T05:40:38.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.452+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f415fc20 con 0x7fd7d8003a60 2024-09-17T05:40:38.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.452+0000 7fd7eb400640 1 -- 172.21.15.122:0/3553636629 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fd7e40046d0 con 0x7fd7d8003a60 2024-09-17T05:40:38.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.452+0000 7fd7eb400640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd7d00754e0 0x7fd7d00779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.452+0000 7fd7eb400640 1 -- 172.21.15.122:0/3553636629 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7fd7e412e1c0 con 0x7fd7d8003a60 2024-09-17T05:40:38.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.452+0000 7fd7ea000640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd7d00754e0 0x7fd7d00779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.456+0000 7fd7ea000640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd7d00754e0 0x7fd7d00779a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000fc0 tx=0x7fd7dc006730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:38.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.456+0000 7fd7eb400640 1 -- 172.21.15.122:0/3553636629 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd7e4016f80 con 0x7fd7d8003a60 2024-09-17T05:40:38.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.624+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fd7f4074c70 con 0x7fd7d8003a60 2024-09-17T05:40:38.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.628+0000 7fd7eb400640 1 -- 172.21.15.122:0/3553636629 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v55)=0 v55) ==== 131+0+0 (secure 0 0 0) 0x7fd7e40fad70 con 0x7fd7d8003a60 2024-09-17T05:40:38.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.628+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd7d00754e0 msgr2=0x7fd7d00779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.628+0000 7fd7f9a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd7d00754e0 0x7fd7d00779a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000fc0 tx=0x7fd7dc006730 comp rx=0 tx=0).stop 2024-09-17T05:40:38.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.628+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 msgr2=0x7fd7f40ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.628+0000 7fd7f9a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 0x7fd7f40ce3d0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd7e401d0a0 tx=0x7fd7e4005660 comp rx=0 tx=0).stop 2024-09-17T05:40:38.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.632+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 shutdown_connections 2024-09-17T05:40:38.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.632+0000 7fd7f9a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd7d00754e0 0x7fd7d00779a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.632+0000 7fd7f9a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7f40cf440 0x7fd7f415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.632+0000 7fd7f9a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7f40c86a0 0x7fd7f40ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.632+0000 7fd7f9a00640 1 --2- 172.21.15.122:0/3553636629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d8003a60 0x7fd7f40ce3d0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.632+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 >> 172.21.15.122:0/3553636629 conn(0x7fd7f40d1dc0 msgr2=0x7fd7f40b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:38.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.632+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 shutdown_connections 2024-09-17T05:40:38.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.632+0000 7fd7f9a00640 1 -- 172.21.15.122:0/3553636629 wait complete. 2024-09-17T05:40:38.648 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:38.650+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:38.648 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:38.650+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:38.648 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:38.650+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:38.648 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:38.650+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:38.648 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:38.650+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:38.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-17T05:40:38.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 -- 172.21.15.122:0/4224792334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0b1650 msgr2=0x7faa9c0b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 --2- 172.21.15.122:0/4224792334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0b1650 0x7faa9c0b1a30 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7faa88009900 tx=0x7faa8801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 -- 172.21.15.122:0/4224792334 shutdown_connections 2024-09-17T05:40:38.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 --2- 172.21.15.122:0/4224792334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa9c0afb10 0x7faa9c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 --2- 172.21.15.122:0/4224792334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0b1650 0x7faa9c0b1a30 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 -- 172.21.15.122:0/4224792334 >> 172.21.15.122:0/4224792334 conn(0x7faa9c0d1dd0 msgr2=0x7faa9c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:38.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 -- 172.21.15.122:0/4224792334 shutdown_connections 2024-09-17T05:40:38.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 -- 172.21.15.122:0/4224792334 wait complete. 2024-09-17T05:40:38.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 Processor -- start 2024-09-17T05:40:38.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.796+0000 7faaa0a00640 1 -- start start 2024-09-17T05:40:38.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0afb10 0x7faa9c166760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa9c0b1650 0x7faa9c1fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9c1605d0 0x7faa9c14d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c169540 con 0x7faa9c0afb10 2024-09-17T05:40:38.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c0b1430 con 0x7faa9c1605d0 2024-09-17T05:40:38.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faa9c1696c0 con 0x7faa9c0b1650 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0afb10 0x7faa9c166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0afb10 0x7faa9c166760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58918/0 (socket says 172.21.15.122:58918) 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 -- 172.21.15.122:0/869938326 learned_addr learned my addr 172.21.15.122:0/869938326 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99400640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa9c0b1650 0x7faa9c1fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa9a800640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9c1605d0 0x7faa9c14d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 -- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa9c0b1650 msgr2=0x7faa9c1fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa9c0b1650 0x7faa9c1fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 -- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9c1605d0 msgr2=0x7faa9c14d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9c1605d0 0x7faa9c14d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 -- 172.21.15.122:0/869938326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c1fc3b0 con 0x7faa9c0afb10 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa99e00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0afb10 0x7faa9c166760 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c3f0 tx=0x7faa8c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa92000640 1 -- 172.21.15.122:0/869938326 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c017070 con 0x7faa9c0afb10 2024-09-17T05:40:38.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa92000640 1 -- 172.21.15.122:0/869938326 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa8c0053c0 con 0x7faa9c0afb10 2024-09-17T05:40:38.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c0b4cc0 con 0x7faa9c0afb10 2024-09-17T05:40:38.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa92000640 1 -- 172.21.15.122:0/869938326 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c003860 con 0x7faa9c0afb10 2024-09-17T05:40:38.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c0b5280 con 0x7faa9c0afb10 2024-09-17T05:40:38.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9c0affc0 con 0x7faa9c0afb10 2024-09-17T05:40:38.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.800+0000 7faa92000640 1 -- 172.21.15.122:0/869938326 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7faa8c01a070 con 0x7faa9c0afb10 2024-09-17T05:40:38.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.804+0000 7faa92000640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa7c0754e0 0x7faa7c0779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:38.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.804+0000 7faa92000640 1 -- 172.21.15.122:0/869938326 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7faa8c092680 con 0x7faa9c0afb10 2024-09-17T05:40:38.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.804+0000 7faa99400640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa7c0754e0 0x7faa7c0779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:38.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.804+0000 7faa99400640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa7c0754e0 0x7faa7c0779a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faa880af000 tx=0x7faa8801b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:38.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.808+0000 7faa92000640 1 -- 172.21.15.122:0/869938326 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7faa8c05f230 con 0x7faa9c0afb10 2024-09-17T05:40:38.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.972+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7faa9c0d2500 con 0x7faa9c0afb10 2024-09-17T05:40:38.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faa92000640 1 -- 172.21.15.122:0/869938326 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v56)=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7faa8c05eb30 con 0x7faa9c0afb10 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa7c0754e0 msgr2=0x7faa7c0779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa7c0754e0 0x7faa7c0779a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faa880af000 tx=0x7faa8801b120 comp rx=0 tx=0).stop 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0afb10 msgr2=0x7faa9c166760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0afb10 0x7faa9c166760 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c3f0 tx=0x7faa8c00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 shutdown_connections 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7faa7c0754e0 0x7faa7c0779a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa9c1605d0 0x7faa9c14d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa9c0b1650 0x7faa9c1fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 --2- 172.21.15.122:0/869938326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa9c0afb10 0x7faa9c166760 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 >> 172.21.15.122:0/869938326 conn(0x7faa9c0d1dd0 msgr2=0x7faa9c15c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:38.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 shutdown_connections 2024-09-17T05:40:38.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:38.980+0000 7faaa0a00640 1 -- 172.21.15.122:0/869938326 wait complete. 2024-09-17T05:40:38.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-17T05:40:39.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.144+0000 7fe03e600640 1 -- 172.21.15.122:0/1242621775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe038073660 msgr2=0x7fe03806fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.144+0000 7fe03e600640 1 --2- 172.21.15.122:0/1242621775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe038073660 0x7fe03806fae0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe0380b3780 tx=0x7fe03001cd80 comp rx=0 tx=0).stop 2024-09-17T05:40:39.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.144+0000 7fe03e600640 1 -- 172.21.15.122:0/1242621775 shutdown_connections 2024-09-17T05:40:39.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.144+0000 7fe03e600640 1 --2- 172.21.15.122:0/1242621775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe038073660 0x7fe03806fae0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.144+0000 7fe03e600640 1 --2- 172.21.15.122:0/1242621775 >> v2:172.21.15.173:3300/0 conn(0x7fe03815aca0 0x7fe038073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.144+0000 7fe03e600640 1 -- 172.21.15.122:0/1242621775 >> 172.21.15.122:0/1242621775 conn(0x7fe0380b2140 msgr2=0x7fe0380b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:39.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.144+0000 7fe03e600640 1 -- 172.21.15.122:0/1242621775 shutdown_connections 2024-09-17T05:40:39.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.144+0000 7fe03e600640 1 -- 172.21.15.122:0/1242621775 wait complete. 2024-09-17T05:40:39.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 Processor -- start 2024-09-17T05:40:39.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 -- start start 2024-09-17T05:40:39.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe038073660 0x7fe038165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe03815a8c0 0x7fe0380b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe03815aca0 0x7fe0380b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380cd220 con 0x7fe03815a8c0 2024-09-17T05:40:39.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380cd0a0 con 0x7fe03815aca0 2024-09-17T05:40:39.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe0380cd3a0 con 0x7fe038073660 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe03815a8c0 0x7fe0380b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe03815a8c0 0x7fe0380b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58940/0 (socket says 172.21.15.122:58940) 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 -- 172.21.15.122:0/759737931 learned_addr learned my addr 172.21.15.122:0/759737931 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe037e00640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe03815aca0 0x7fe0380b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe037400640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe038073660 0x7fe038165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 -- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe038073660 msgr2=0x7fe038165260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe038073660 0x7fe038165260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 -- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe03815aca0 msgr2=0x7fe0380b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe03815aca0 0x7fe0380b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 -- 172.21.15.122:0/759737931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03815ee60 con 0x7fe03815a8c0 2024-09-17T05:40:39.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe036a00640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe03815a8c0 0x7fe0380b3d40 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe02400e7f0 tx=0x7fe02400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:39.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe02f400640 1 -- 172.21.15.122:0/759737931 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02400ccc0 con 0x7fe03815a8c0 2024-09-17T05:40:39.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe02f400640 1 -- 172.21.15.122:0/759737931 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe02400ce80 con 0x7fe03815a8c0 2024-09-17T05:40:39.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe02f400640 1 -- 172.21.15.122:0/759737931 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024005280 con 0x7fe03815a8c0 2024-09-17T05:40:39.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03815f120 con 0x7fe03815a8c0 2024-09-17T05:40:39.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.148+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe038142a50 con 0x7fe03815a8c0 2024-09-17T05:40:39.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.152+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03806fae0 con 0x7fe03815a8c0 2024-09-17T05:40:39.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.152+0000 7fe02f400640 1 -- 172.21.15.122:0/759737931 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7fe024005420 con 0x7fe03815a8c0 2024-09-17T05:40:39.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.152+0000 7fe02f400640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe0180754e0 0x7fe0180779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.152+0000 7fe02f400640 1 -- 172.21.15.122:0/759737931 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7fe024093be0 con 0x7fe03815a8c0 2024-09-17T05:40:39.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.152+0000 7fe037400640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe0180754e0 0x7fe0180779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.152+0000 7fe037400640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe0180754e0 0x7fe0180779a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe028007fd0 tx=0x7fe028009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:39.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.156+0000 7fe02f400640 1 -- 172.21.15.122:0/759737931 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe0240600a0 con 0x7fe03815a8c0 2024-09-17T05:40:39.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.348+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fe038156b00 con 0x7fe0180754e0 2024-09-17T05:40:39.356 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:39.358+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:39.357 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:39.358+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:39.357 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:39.358+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:39.357 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:39.358+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:39.357 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:40:39.358+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:40:39.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.396+0000 7fe02f400640 1 -- 172.21.15.122:0/759737931 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe038156b00 con 0x7fe0180754e0 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe0180754e0 msgr2=0x7fe0180779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe0180754e0 0x7fe0180779a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe028007fd0 tx=0x7fe028009040 comp rx=0 tx=0).stop 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe03815a8c0 msgr2=0x7fe0380b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe03815a8c0 0x7fe0380b3d40 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe02400e7f0 tx=0x7fe02400ecb0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 shutdown_connections 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe0180754e0 0x7fe0180779a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe03815aca0 0x7fe0380b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe03815a8c0 0x7fe0380b3d40 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 --2- 172.21.15.122:0/759737931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe038073660 0x7fe038165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 >> 172.21.15.122:0/759737931 conn(0x7fe0380b2140 msgr2=0x7fe0380aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 shutdown_connections 2024-09-17T05:40:39.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.400+0000 7fe03e600640 1 -- 172.21.15.122:0/759737931 wait complete. 2024-09-17T05:40:39.417 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:40:39.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febdf200640 1 -- 172.21.15.122:0/2598993595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febc4003f80 msgr2=0x7febc4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febdf200640 1 --2- 172.21.15.122:0/2598993595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febc4003f80 0x7febc4004810 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7febcc001730 tx=0x7febcc01ef90 comp rx=0 tx=0).stop 2024-09-17T05:40:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febde800640 1 -- 172.21.15.122:0/2598993595 reap_dead start 2024-09-17T05:40:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febdf200640 1 -- 172.21.15.122:0/2598993595 shutdown_connections 2024-09-17T05:40:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febdf200640 1 -- 172.21.15.122:0/2598993595 >> 172.21.15.122:0/2598993595 conn(0x7febe00b2680 msgr2=0x7febe00b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febdf200640 1 -- 172.21.15.122:0/2598993595 shutdown_connections 2024-09-17T05:40:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febdf200640 1 -- 172.21.15.122:0/2598993595 wait complete. 2024-09-17T05:40:39.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febdf200640 1 Processor -- start 2024-09-17T05:40:39.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.568+0000 7febdf200640 1 -- start start 2024-09-17T05:40:39.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc4003f80 0x7febe01f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe00cfda0 0x7febe01e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe014c620 0x7febe014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7febe0074850 con 0x7febe014c620 2024-09-17T05:40:39.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7febe00746d0 con 0x7febe00cfda0 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7febe00749d0 con 0x7febc4003f80 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe014c620 0x7febe014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdde00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc4003f80 0x7febe01f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe014c620 0x7febe014caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58960/0 (socket says 172.21.15.122:58960) 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 -- 172.21.15.122:0/331417269 learned_addr learned my addr 172.21.15.122:0/331417269 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdd400640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe00cfda0 0x7febe01e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 -- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc4003f80 msgr2=0x7febe01f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc4003f80 0x7febe01f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 -- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe00cfda0 msgr2=0x7febe01e86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe00cfda0 0x7febe01e86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 -- 172.21.15.122:0/331417269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01f9050 con 0x7febe014c620 2024-09-17T05:40:39.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febde800640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe014c620 0x7febe014caa0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7febe00d1b70 tx=0x7febd800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:39.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febd6000640 1 -- 172.21.15.122:0/331417269 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8004b30 con 0x7febe014c620 2024-09-17T05:40:39.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febd6000640 1 -- 172.21.15.122:0/331417269 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febd8004cd0 con 0x7febe014c620 2024-09-17T05:40:39.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febd6000640 1 -- 172.21.15.122:0/331417269 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8003be0 con 0x7febe014c620 2024-09-17T05:40:39.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe014cfe0 con 0x7febe014c620 2024-09-17T05:40:39.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01f4f00 con 0x7febe014c620 2024-09-17T05:40:39.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe006cc20 con 0x7febe014c620 2024-09-17T05:40:39.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.572+0000 7febd6000640 1 -- 172.21.15.122:0/331417269 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 51) ==== 100396+0+0 (secure 0 0 0) 0x7febd8004030 con 0x7febe014c620 2024-09-17T05:40:39.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.576+0000 7febd6000640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc00754e0 0x7febc00779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.576+0000 7febd6000640 1 -- 172.21.15.122:0/331417269 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7febd8093e30 con 0x7febe014c620 2024-09-17T05:40:39.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.576+0000 7febdde00640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc00754e0 0x7febc00779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.576+0000 7febdde00640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc00754e0 0x7febc00779a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7febd0006b80 tx=0x7febd0006730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:39.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.580+0000 7febd6000640 1 -- 172.21.15.122:0/331417269 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7febe014cfe0 con 0x7febe014c620 2024-09-17T05:40:39.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.636+0000 7febd6000640 1 -- 172.21.15.122:0/331417269 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 52) ==== 100447+0+0 (secure 0 0 0) 0x7febd805c600 con 0x7febe014c620 2024-09-17T05:40:39.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.776+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7febe00b6a50 con 0x7febc00754e0 2024-09-17T05:40:39.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febd6000640 1 -- 172.21.15.122:0/331417269 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7febe00b6a50 con 0x7febc00754e0 2024-09-17T05:40:39.781 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:40:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc00754e0 msgr2=0x7febc00779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc00754e0 0x7febc00779a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7febd0006b80 tx=0x7febd0006730 comp rx=0 tx=0).stop 2024-09-17T05:40:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe014c620 msgr2=0x7febe014caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe014c620 0x7febe014caa0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7febe00d1b70 tx=0x7febd800f3c0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 shutdown_connections 2024-09-17T05:40:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7febc00754e0 0x7febc00779a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febe014c620 0x7febe014caa0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe00cfda0 0x7febe01e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 --2- 172.21.15.122:0/331417269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc4003f80 0x7febe01f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 >> 172.21.15.122:0/331417269 conn(0x7febe00b2680 msgr2=0x7febe015c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 shutdown_connections 2024-09-17T05:40:39.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.784+0000 7febdf200640 1 -- 172.21.15.122:0/331417269 wait complete. 2024-09-17T05:40:39.802 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:40:39.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.948+0000 7efe09e00640 1 -- 172.21.15.122:0/2893837653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe04152b90 msgr2=0x7efe0406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.948+0000 7efe09e00640 1 --2- 172.21.15.122:0/2893837653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe04152b90 0x7efe0406f090 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009930 tx=0x7efdf401c8a0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.948+0000 7efe09e00640 1 -- 172.21.15.122:0/2893837653 shutdown_connections 2024-09-17T05:40:39.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.948+0000 7efe09e00640 1 --2- 172.21.15.122:0/2893837653 >> v2:172.21.15.173:3300/0 conn(0x7efe0415ff50 0x7efe0406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.948+0000 7efe09e00640 1 --2- 172.21.15.122:0/2893837653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe04152b90 0x7efe0406f090 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.948+0000 7efe09e00640 1 -- 172.21.15.122:0/2893837653 >> 172.21.15.122:0/2893837653 conn(0x7efe040cce90 msgr2=0x7efe040cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:39.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.948+0000 7efe09e00640 1 -- 172.21.15.122:0/2893837653 shutdown_connections 2024-09-17T05:40:39.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.948+0000 7efe09e00640 1 -- 172.21.15.122:0/2893837653 wait complete. 2024-09-17T05:40:39.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 Processor -- start 2024-09-17T05:40:39.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 -- start start 2024-09-17T05:40:39.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe040c6b00 0x7efe0414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe04152b90 0x7efe041fb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe0415ff50 0x7efe040c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe04168b30 con 0x7efe0415ff50 2024-09-17T05:40:39.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efe041689b0 con 0x7efe04152b90 2024-09-17T05:40:39.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efe04168cb0 con 0x7efe040c6b00 2024-09-17T05:40:39.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe02a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe040c6b00 0x7efe0414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe04152b90 0x7efe041fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe04152b90 0x7efe041fb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.122:60660/0 (socket says 172.21.15.122:60660) 2024-09-17T05:40:39.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 -- 172.21.15.122:0/1329572721 learned_addr learned my addr 172.21.15.122:0/1329572721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:40:39.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe02a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe040c6b00 0x7efe0414a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:51438/0 (socket says 172.21.15.122:51438) 2024-09-17T05:40:39.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe03400640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe0415ff50 0x7efe040c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 -- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe040c6b00 msgr2=0x7efe0414a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe040c6b00 0x7efe0414a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 -- 172.21.15.122:0/1329572721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe0415ff50 msgr2=0x7efe040c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe0415ff50 0x7efe040c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 -- 172.21.15.122:0/1329572721 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe040c8b70 con 0x7efe04152b90 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe02a00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe040c6b00 0x7efe0414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe03400640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe0415ff50 0x7efe040c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfbe00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe04152b90 0x7efe041fb280 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c430 tx=0x7efdec00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfb400640 1 -- 172.21.15.122:0/1329572721 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec011790 con 0x7efe04152b90 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfb400640 1 -- 172.21.15.122:0/1329572721 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdec017070 con 0x7efe04152b90 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe041507c0 con 0x7efe04152b90 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efdfb400640 1 -- 172.21.15.122:0/1329572721 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec011d00 con 0x7efe04152b90 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe04151060 con 0x7efe04152b90 2024-09-17T05:40:39.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.952+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0406f090 con 0x7efe04152b90 2024-09-17T05:40:39.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.956+0000 7efdfb400640 1 -- 172.21.15.122:0/1329572721 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 52) ==== 100447+0+0 (secure 0 0 0) 0x7efdec03bb50 con 0x7efe04152b90 2024-09-17T05:40:39.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.956+0000 7efdfb400640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efde0075570 0x7efde0077a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:40:39.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.956+0000 7efdfb400640 1 -- 172.21.15.122:0/1329572721 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5793+0+0 (secure 0 0 0) 0x7efdec0933c0 con 0x7efe04152b90 2024-09-17T05:40:39.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.956+0000 7efe02a00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efde0075570 0x7efde0077a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:40:39.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.956+0000 7efe02a00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efde0075570 0x7efde0077a30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efdf40af000 tx=0x7efdf401b0e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:40:39.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:39.960+0000 7efdfb400640 1 -- 172.21.15.122:0/1329572721 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efdec05f900 con 0x7efe04152b90 2024-09-17T05:40:40.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.152+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7efe04151230 con 0x7efde0075570 2024-09-17T05:40:40.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.160+0000 7efdfb400640 1 -- 172.21.15.122:0/1329572721 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7efe04151230 con 0x7efde0075570 2024-09-17T05:40:40.158 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:40:40.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efde0075570 msgr2=0x7efde0077a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efde0075570 0x7efde0077a30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efdf40af000 tx=0x7efdf401b0e0 comp rx=0 tx=0).stop 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe04152b90 msgr2=0x7efe041fb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe04152b90 0x7efe041fb280 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c430 tx=0x7efdec00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 shutdown_connections 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efde0075570 0x7efde0077a30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe0415ff50 0x7efe040c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe04152b90 0x7efe041fb280 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 --2- 172.21.15.122:0/1329572721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe040c6b00 0x7efe0414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 >> 172.21.15.122:0/1329572721 conn(0x7efe040cce90 msgr2=0x7efe04070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 shutdown_connections 2024-09-17T05:40:40.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:40:40.164+0000 7efe09e00640 1 -- 172.21.15.122:0/1329572721 wait complete. 2024-09-17T05:40:40.181 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:40:40.181 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:40.181 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:40.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:40.190 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:40:49.295 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:40:49.295 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:40:49.295 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.04765 s, 119 MB/s 2024-09-17T05:40:49.296 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:49.297 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:49.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:49.306 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:40:58.421 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:40:58.422 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:40:58.422 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.06485 s, 118 MB/s 2024-09-17T05:40:58.423 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:40:58.423 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:40:58.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:40:58.432 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-17T05:41:04.373 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:41:04.377 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:41:07.423 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records in 2024-09-17T05:41:07.423 INFO:teuthology.orchestra.run.smithi122.stderr:1024+0 records out 2024-09-17T05:41:07.423 INFO:teuthology.orchestra.run.smithi122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.9379 s, 120 MB/s 2024-09-17T05:41:07.424 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-17T05:41:07.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.572+0000 7fa24a800640 1 -- 172.21.15.122:0/1373484837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2440d24d0 msgr2=0x7fa2440d2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:07.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.572+0000 7fa24a800640 1 --2- 172.21.15.122:0/1373484837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2440d24d0 0x7fa2440d2980 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009c80 tx=0x7fa23c01cda0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.572+0000 7fa24a800640 1 -- 172.21.15.122:0/1373484837 shutdown_connections 2024-09-17T05:41:07.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.572+0000 7fa24a800640 1 --2- 172.21.15.122:0/1373484837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2440d24d0 0x7fa2440d2980 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.572+0000 7fa24a800640 1 --2- 172.21.15.122:0/1373484837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa24415dff0 0x7fa2440d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.572+0000 7fa24a800640 1 -- 172.21.15.122:0/1373484837 >> 172.21.15.122:0/1373484837 conn(0x7fa2440d3f10 msgr2=0x7fa2440d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:07.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.572+0000 7fa24a800640 1 -- 172.21.15.122:0/1373484837 shutdown_connections 2024-09-17T05:41:07.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.572+0000 7fa24a800640 1 -- 172.21.15.122:0/1373484837 wait complete. 2024-09-17T05:41:07.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 Processor -- start 2024-09-17T05:41:07.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 -- start start 2024-09-17T05:41:07.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2440cf950 0x7fa244150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:07.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2440d24d0 0x7fa244152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:07.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa24415dff0 0x7fa24415cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa24416b030 con 0x7fa24415dff0 2024-09-17T05:41:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa24416aeb0 con 0x7fa2440cf950 2024-09-17T05:41:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa24416b1b0 con 0x7fa2440d24d0 2024-09-17T05:41:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa24415dff0 0x7fa24415cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa24415dff0 0x7fa24415cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57596/0 (socket says 172.21.15.122:57596) 2024-09-17T05:41:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 -- 172.21.15.122:0/4227339833 learned_addr learned my addr 172.21.15.122:0/4227339833 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249400640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2440cf950 0x7fa244150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa248a00640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2440d24d0 0x7fa244152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 -- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2440d24d0 msgr2=0x7fa244152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2440d24d0 0x7fa244152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 -- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2440cf950 msgr2=0x7fa244150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2440cf950 0x7fa244150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 -- 172.21.15.122:0/4227339833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24414fc70 con 0x7fa24415dff0 2024-09-17T05:41:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa249e00640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa24415dff0 0x7fa24415cf10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa23c01b450 tx=0x7fa23c009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa241600640 1 -- 172.21.15.122:0/4227339833 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0bb070 con 0x7fa24415dff0 2024-09-17T05:41:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa241600640 1 -- 172.21.15.122:0/4227339833 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa23c004e00 con 0x7fa24415dff0 2024-09-17T05:41:07.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24415a350 con 0x7fa24415dff0 2024-09-17T05:41:07.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa241600640 1 -- 172.21.15.122:0/4227339833 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0b2040 con 0x7fa24415dff0 2024-09-17T05:41:07.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.576+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24414ff10 con 0x7fa24415dff0 2024-09-17T05:41:07.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.580+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2440d2980 con 0x7fa24415dff0 2024-09-17T05:41:07.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.580+0000 7fa241600640 1 -- 172.21.15.122:0/4227339833 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 53) ==== 100498+0+0 (secure 0 0 0) 0x7fa23c0be070 con 0x7fa24415dff0 2024-09-17T05:41:07.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.580+0000 7fa241600640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa224075600 0x7fa224077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:07.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.580+0000 7fa241600640 1 -- 172.21.15.122:0/4227339833 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fa23c12d180 con 0x7fa24415dff0 2024-09-17T05:41:07.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.580+0000 7fa249400640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa224075600 0x7fa224077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:07.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.580+0000 7fa249400640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa224075600 0x7fa224077ac0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa23800bb50 tx=0x7fa23800b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:07.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.584+0000 7fa241600640 1 -- 172.21.15.122:0/4227339833 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa23c016990 con 0x7fa24415dff0 2024-09-17T05:41:07.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.780+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fa244069e60 con 0x7fa224075600 2024-09-17T05:41:07.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa241600640 1 -- 172.21.15.122:0/4227339833 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa238010bd0 con 0x7fa224075600 2024-09-17T05:41:07.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa224075600 msgr2=0x7fa224077ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:07.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa224075600 0x7fa224077ac0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa23800bb50 tx=0x7fa23800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa24415dff0 msgr2=0x7fa24415cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:07.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa24415dff0 0x7fa24415cf10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa23c01b450 tx=0x7fa23c009390 comp rx=0 tx=0).stop 2024-09-17T05:41:07.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 shutdown_connections 2024-09-17T05:41:07.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa224075600 0x7fa224077ac0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa24415dff0 0x7fa24415cf10 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2440d24d0 0x7fa244152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 --2- 172.21.15.122:0/4227339833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2440cf950 0x7fa244150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 >> 172.21.15.122:0/4227339833 conn(0x7fa2440d3f10 msgr2=0x7fa24415b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:07.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 shutdown_connections 2024-09-17T05:41:07.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.796+0000 7fa24a800640 1 -- 172.21.15.122:0/4227339833 wait complete. 2024-09-17T05:41:07.814 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-17T05:41:07.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.960+0000 7f08e2800640 1 -- 172.21.15.122:0/1000080121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc15dae0 msgr2=0x7f08dc0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:07.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.960+0000 7f08e2800640 1 --2- 172.21.15.122:0/1000080121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc15dae0 0x7f08dc0acd40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00ad00 tx=0x7f08cc01de80 comp rx=0 tx=0).stop 2024-09-17T05:41:07.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.960+0000 7f08e2800640 1 -- 172.21.15.122:0/1000080121 shutdown_connections 2024-09-17T05:41:07.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.960+0000 7f08e2800640 1 --2- 172.21.15.122:0/1000080121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc15dae0 0x7f08dc0acd40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.960+0000 7f08e2800640 1 -- 172.21.15.122:0/1000080121 >> 172.21.15.122:0/1000080121 conn(0x7f08dc0b18f0 msgr2=0x7f08dc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:07.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.960+0000 7f08e2800640 1 -- 172.21.15.122:0/1000080121 shutdown_connections 2024-09-17T05:41:07.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.960+0000 7f08e2800640 1 -- 172.21.15.122:0/1000080121 wait complete. 2024-09-17T05:41:07.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 Processor -- start 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 -- start start 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0b49e0 0x7f08dc09b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08dc0cd100 0x7f08dc14e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc15dae0 0x7f08dc06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc168580 con 0x7f08dc0cd100 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc168400 con 0x7f08dc15dae0 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f08dc168700 con 0x7f08dc0b49e0 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08dc0cd100 0x7f08dc14e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08dc0cd100 0x7f08dc14e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57606/0 (socket says 172.21.15.122:57606) 2024-09-17T05:41:07.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 -- 172.21.15.122:0/1106857930 learned_addr learned my addr 172.21.15.122:0/1106857930 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08db400640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0b49e0 0x7f08dc09b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08dbe00640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc15dae0 0x7f08dc06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 -- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0b49e0 msgr2=0x7f08dc09b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0b49e0 0x7f08dc09b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 -- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc15dae0 msgr2=0x7f08dc06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc15dae0 0x7f08dc06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 -- 172.21.15.122:0/1106857930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc09b960 con 0x7f08dc0cd100 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d2a00640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08dc0cd100 0x7f08dc14e530 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c530 tx=0x7f08c400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d3e00640 1 -- 172.21.15.122:0/1106857930 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c40052d0 con 0x7f08dc0cd100 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d3e00640 1 -- 172.21.15.122:0/1106857930 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08c4005490 con 0x7f08dc0cd100 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08d3e00640 1 -- 172.21.15.122:0/1106857930 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4012870 con 0x7f08dc0cd100 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc1532a0 con 0x7f08dc0cd100 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc1f9c50 con 0x7f08dc0cd100 2024-09-17T05:41:07.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.964+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08dc0acd40 con 0x7f08dc0cd100 2024-09-17T05:41:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.968+0000 7f08d3e00640 1 -- 172.21.15.122:0/1106857930 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 53) ==== 100498+0+0 (secure 0 0 0) 0x7f08c4012a10 con 0x7f08dc0cd100 2024-09-17T05:41:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.968+0000 7f08d3e00640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08b8075600 0x7f08b8077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.968+0000 7f08d3e00640 1 -- 172.21.15.122:0/1106857930 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f08c4092e10 con 0x7f08dc0cd100 2024-09-17T05:41:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.968+0000 7f08db400640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08b8075600 0x7f08b8077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:07.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.968+0000 7f08db400640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08b8075600 0x7f08b8077ac0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0b0000 tx=0x7f08cc01c6c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:07.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:07.972+0000 7f08d3e00640 1 -- 172.21.15.122:0/1106857930 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f08c405f370 con 0x7f08dc0cd100 2024-09-17T05:41:08.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.164+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f08dc159750 con 0x7f08b8075600 2024-09-17T05:41:08.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08d3e00640 1 -- 172.21.15.122:0/1106857930 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f08dc159750 con 0x7f08b8075600 2024-09-17T05:41:08.165 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08b8075600 msgr2=0x7f08b8077ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08b8075600 0x7f08b8077ac0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0b0000 tx=0x7f08cc01c6c0 comp rx=0 tx=0).stop 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08dc0cd100 msgr2=0x7f08dc14e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08dc0cd100 0x7f08dc14e530 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c530 tx=0x7f08c400c9f0 comp rx=0 tx=0).stop 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 shutdown_connections 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f08b8075600 0x7f08b8077ac0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc15dae0 0x7f08dc06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08dc0cd100 0x7f08dc14e530 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 --2- 172.21.15.122:0/1106857930 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0b49e0 0x7f08dc09b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 >> 172.21.15.122:0/1106857930 conn(0x7f08dc0b18f0 msgr2=0x7f08dc0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 shutdown_connections 2024-09-17T05:41:08.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:08.168+0000 7f08e2800640 1 -- 172.21.15.122:0/1106857930 wait complete. 2024-09-17T05:41:08.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:08.187 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:41:08.187 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:41:08.196 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:41:08.197 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="2972172288" 2024-09-17T05:41:08.197 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:08.198 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-17T05:41:10.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:10.199 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:41:10.199 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:41:10.210 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:41:10.210 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="2972172288" 2024-09-17T05:41:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:10.211 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-17T05:41:12.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:12.212 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:41:12.212 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:41:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:# file: volumes/_nogroup/sv1/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:41:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.rbytes="3221225472" 2024-09-17T05:41:12.222 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:12.223 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-17T05:41:12.223 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone51580 2024-09-17T05:41:12.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- 172.21.15.122:0/4028197140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200cf5a0 msgr2=0x7f1b200d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:12.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 --2- 172.21.15.122:0/4028197140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200cf5a0 0x7f1b200d22b0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f1b200698f0 tx=0x7f1b0c01c890 comp rx=0 tx=0).stop 2024-09-17T05:41:12.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- 172.21.15.122:0/4028197140 shutdown_connections 2024-09-17T05:41:12.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 --2- 172.21.15.122:0/4028197140 >> v2:172.21.15.173:3300/0 conn(0x7f1b200b42a0 0x7f1b200b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 --2- 172.21.15.122:0/4028197140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200cf5a0 0x7f1b200d22b0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- 172.21.15.122:0/4028197140 >> 172.21.15.122:0/4028197140 conn(0x7f1b200d3fd0 msgr2=0x7f1b200d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:12.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- 172.21.15.122:0/4028197140 shutdown_connections 2024-09-17T05:41:12.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- 172.21.15.122:0/4028197140 wait complete. 2024-09-17T05:41:12.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 Processor -- start 2024-09-17T05:41:12.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- start start 2024-09-17T05:41:12.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200b42a0 0x7f1b2014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:12.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b200cf5a0 0x7f1b201fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:12.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b2015e240 0x7f1b201553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:12.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2016b010 con 0x7f1b200b42a0 2024-09-17T05:41:12.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1b200d2a10 con 0x7f1b2015e240 2024-09-17T05:41:12.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b26e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2016b190 con 0x7f1b200cf5a0 2024-09-17T05:41:12.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200b42a0 0x7f1b2014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200b42a0 0x7f1b2014fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50116/0 (socket says 172.21.15.122:50116) 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1fe00640 1 -- 172.21.15.122:0/1679163422 learned_addr learned my addr 172.21.15.122:0/1679163422 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b24e00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b2015e240 0x7f1b201553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1f400640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b200cf5a0 0x7f1b201fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1fe00640 1 -- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b200cf5a0 msgr2=0x7f1b201fb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1fe00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b200cf5a0 0x7f1b201fb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1fe00640 1 -- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b2015e240 msgr2=0x7f1b201553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1fe00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b2015e240 0x7f1b201553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.428+0000 7f1b1fe00640 1 -- 172.21.15.122:0/1679163422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b201fbcf0 con 0x7f1b200b42a0 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b1fe00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200b42a0 0x7f1b2014fb40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000ba50 tx=0x7f1b1000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b17e00640 1 -- 172.21.15.122:0/1679163422 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10019070 con 0x7f1b200b42a0 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b17e00640 1 -- 172.21.15.122:0/1679163422 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b1000cd00 con 0x7f1b200b42a0 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b201519d0 con 0x7f1b200b42a0 2024-09-17T05:41:12.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b17e00640 1 -- 172.21.15.122:0/1679163422 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100146a0 con 0x7f1b200b42a0 2024-09-17T05:41:12.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b20156fb0 con 0x7f1b200b42a0 2024-09-17T05:41:12.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b200b4710 con 0x7f1b200b42a0 2024-09-17T05:41:12.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b17e00640 1 -- 172.21.15.122:0/1679163422 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 53) ==== 100498+0+0 (secure 0 0 0) 0x7f1b1001c070 con 0x7f1b200b42a0 2024-09-17T05:41:12.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b17e00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b00075600 0x7f1b00077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:12.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.432+0000 7f1b17e00640 1 -- 172.21.15.122:0/1679163422 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f1b100932c0 con 0x7f1b200b42a0 2024-09-17T05:41:12.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.436+0000 7f1b1f400640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b00075600 0x7f1b00077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:12.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.436+0000 7f1b1f400640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b00075600 0x7f1b00077ac0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0af000 tx=0x7f1b0c01b0d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:12.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.436+0000 7f1b17e00640 1 -- 172.21.15.122:0/1679163422 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1b201fbcf0 con 0x7f1b200b42a0 2024-09-17T05:41:12.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.636+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone51580", "target": ["mon-mgr", ""]}) -- 0x7f1b201583e0 con 0x7f1b00075600 2024-09-17T05:41:12.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b17e00640 1 -- 172.21.15.122:0/1679163422 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1b201583e0 con 0x7f1b00075600 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b00075600 msgr2=0x7f1b00077ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b00075600 0x7f1b00077ac0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0af000 tx=0x7f1b0c01b0d0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200b42a0 msgr2=0x7f1b2014fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200b42a0 0x7f1b2014fb40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000ba50 tx=0x7f1b1000bf10 comp rx=0 tx=0).stop 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 shutdown_connections 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f1b00075600 0x7f1b00077ac0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b2015e240 0x7f1b201553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b200cf5a0 0x7f1b201fb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 --2- 172.21.15.122:0/1679163422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b200b42a0 0x7f1b2014fb40 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 >> 172.21.15.122:0/1679163422 conn(0x7f1b200d3fd0 msgr2=0x7f1b200d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:12.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 shutdown_connections 2024-09-17T05:41:12.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.704+0000 7f1b26e00640 1 -- 172.21.15.122:0/1679163422 wait complete. 2024-09-17T05:41:12.707 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.708+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.707 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.708+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.708 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.708+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.708 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.708+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.708 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.708+0000 7fc09f000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.724 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone77831 2024-09-17T05:41:12.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.724+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.724+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.724+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.724+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:12.724+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:12.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.880+0000 7f5b26a00640 1 -- 172.21.15.122:0/4248954151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200b25e0 msgr2=0x7f5b200af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:12.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.880+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4248954151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200b25e0 0x7f5b200af540 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5b200cc2d0 tx=0x7f5b1801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.880+0000 7f5b26a00640 1 -- 172.21.15.122:0/4248954151 shutdown_connections 2024-09-17T05:41:12.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.880+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4248954151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200b25e0 0x7f5b200af540 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.880+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4248954151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b200cfd30 0x7f5b2015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.880+0000 7f5b26a00640 1 -- 172.21.15.122:0/4248954151 >> 172.21.15.122:0/4248954151 conn(0x7f5b200d1cf0 msgr2=0x7f5b200d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:12.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.880+0000 7f5b26a00640 1 -- 172.21.15.122:0/4248954151 shutdown_connections 2024-09-17T05:41:12.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.880+0000 7f5b26a00640 1 -- 172.21.15.122:0/4248954151 wait complete. 2024-09-17T05:41:12.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 Processor -- start 2024-09-17T05:41:12.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 -- start start 2024-09-17T05:41:12.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b200b25e0 0x7f5b2014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:12.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200cfd30 0x7f5b2006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:12.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b200d2810 0x7f5b201551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:12.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20168f10 con 0x7f5b200cfd30 2024-09-17T05:41:12.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b200b07a0 con 0x7f5b200d2810 2024-09-17T05:41:12.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5b20169090 con 0x7f5b200b25e0 2024-09-17T05:41:12.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200cfd30 0x7f5b2006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:12.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200cfd30 0x7f5b2006dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50126/0 (socket says 172.21.15.122:50126) 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 -- 172.21.15.122:0/4228484659 learned_addr learned my addr 172.21.15.122:0/4228484659 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26000640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b200d2810 0x7f5b201551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b25600640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b200b25e0 0x7f5b2014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 -- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b200b25e0 msgr2=0x7f5b2014cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b200b25e0 0x7f5b2014cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 -- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b200d2810 msgr2=0x7f5b201551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b200d2810 0x7f5b201551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 -- 172.21.15.122:0/4228484659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2006e7f0 con 0x7f5b200cfd30 2024-09-17T05:41:12.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b24c00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200cfd30 0x7f5b2006dd30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00b410 tx=0x7f5b0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:12.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b1d600640 1 -- 172.21.15.122:0/4228484659 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c015030 con 0x7f5b200cfd30 2024-09-17T05:41:12.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b1d600640 1 -- 172.21.15.122:0/4228484659 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b0c004020 con 0x7f5b200cfd30 2024-09-17T05:41:12.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b1d600640 1 -- 172.21.15.122:0/4228484659 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0044b0 con 0x7f5b200cfd30 2024-09-17T05:41:12.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.884+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2006ead0 con 0x7f5b200cfd30 2024-09-17T05:41:12.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.888+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2006f090 con 0x7f5b200cfd30 2024-09-17T05:41:12.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.888+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b200af540 con 0x7f5b200cfd30 2024-09-17T05:41:12.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.888+0000 7f5b1d600640 1 -- 172.21.15.122:0/4228484659 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 53) ==== 100498+0+0 (secure 0 0 0) 0x7f5b0c018070 con 0x7f5b200cfd30 2024-09-17T05:41:12.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.888+0000 7f5b1d600640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b00075600 0x7f5b00077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:12.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.888+0000 7f5b1d600640 1 -- 172.21.15.122:0/4228484659 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f5b0c092f60 con 0x7f5b200cfd30 2024-09-17T05:41:12.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.888+0000 7f5b25600640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b00075600 0x7f5b00077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:12.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.888+0000 7f5b25600640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b00075600 0x7f5b00077ac0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5b14005fd0 tx=0x7f5b14005db0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:12.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:12.892+0000 7f5b1d600640 1 -- 172.21.15.122:0/4228484659 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5b0c05fa70 con 0x7f5b200cfd30 2024-09-17T05:41:13.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.092+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone77831", "target": ["mon-mgr", ""]}) -- 0x7f5b201567e0 con 0x7f5b00075600 2024-09-17T05:41:13.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.360+0000 7f5b1d600640 1 -- 172.21.15.122:0/4228484659 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 54) ==== 100600+0+0 (secure 0 0 0) 0x7f5b0c05b5e0 con 0x7f5b200cfd30 2024-09-17T05:41:13.706 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:13.708+0000 7fc01d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:13.706 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:13.708+0000 7fc01d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:13.706 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:13.708+0000 7fc01d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:13.706 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:13.708+0000 7fc01d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:13.706 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:13.708+0000 7fc01d000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:13.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.796+0000 7f5b1d600640 1 -- 172.21.15.122:0/4228484659 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b201567e0 con 0x7f5b00075600 2024-09-17T05:41:13.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b00075600 msgr2=0x7f5b00077ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:13.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b00075600 0x7f5b00077ac0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5b14005fd0 tx=0x7f5b14005db0 comp rx=0 tx=0).stop 2024-09-17T05:41:13.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200cfd30 msgr2=0x7f5b2006dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:13.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200cfd30 0x7f5b2006dd30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00b410 tx=0x7f5b0c00b8d0 comp rx=0 tx=0).stop 2024-09-17T05:41:13.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 shutdown_connections 2024-09-17T05:41:13.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5b00075600 0x7f5b00077ac0 secure :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5b14005fd0 tx=0x7f5b14005db0 comp rx=0 tx=0).stop 2024-09-17T05:41:13.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b200d2810 0x7f5b201551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:13.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b200cfd30 0x7f5b2006dd30 secure :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00b410 tx=0x7f5b0c00b8d0 comp rx=0 tx=0).stop 2024-09-17T05:41:13.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 --2- 172.21.15.122:0/4228484659 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b200b25e0 0x7f5b2014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:13.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.800+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 >> 172.21.15.122:0/4228484659 conn(0x7f5b200d1cf0 msgr2=0x7f5b200d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:13.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.804+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 shutdown_connections 2024-09-17T05:41:13.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:13.804+0000 7f5b26a00640 1 -- 172.21.15.122:0/4228484659 wait complete. 2024-09-17T05:41:13.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone11792 2024-09-17T05:41:14.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3764042733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c94003fe0 msgr2=0x7f7c94004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:14.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cb4e00640 1 --2- 172.21.15.122:0/3764042733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c94003fe0 0x7f7c94004810 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009b90 tx=0x7f7c9c01d270 comp rx=0 tx=0).stop 2024-09-17T05:41:14.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cafe00640 1 -- 172.21.15.122:0/3764042733 reap_dead start 2024-09-17T05:41:14.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3764042733 shutdown_connections 2024-09-17T05:41:14.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3764042733 >> 172.21.15.122:0/3764042733 conn(0x7f7cb00d1cf0 msgr2=0x7f7cb00d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:14.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3764042733 shutdown_connections 2024-09-17T05:41:14.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3764042733 wait complete. 2024-09-17T05:41:14.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cb4e00640 1 Processor -- start 2024-09-17T05:41:14.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.056+0000 7f7cb4e00640 1 -- start start 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c94003fe0 0x7f7cb01f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb006dd30 0x7f7cb015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb00cade0 0x7f7cb00cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb0160180 con 0x7f7cb00cade0 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb00b0870 con 0x7f7c94003fe0 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb0160300 con 0x7f7cb006dd30 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb00cade0 0x7f7cb00cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb00cade0 0x7f7cb00cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50144/0 (socket says 172.21.15.122:50144) 2024-09-17T05:41:14.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 -- 172.21.15.122:0/3249417181 learned_addr learned my addr 172.21.15.122:0/3249417181 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:14.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7caf400640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c94003fe0 0x7f7cb01f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:14.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7caea00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb006dd30 0x7f7cb015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:14.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 -- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb006dd30 msgr2=0x7f7cb015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:14.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb006dd30 0x7f7cb015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:14.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 -- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c94003fe0 msgr2=0x7f7cb01f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:14.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c94003fe0 0x7f7cb01f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:14.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 -- 172.21.15.122:0/3249417181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb006e490 con 0x7f7cb00cade0 2024-09-17T05:41:14.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cafe00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb00cade0 0x7f7cb00cb260 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8001380 tx=0x7f7ca800f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:14.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7ca7400640 1 -- 172.21.15.122:0/3249417181 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800a5a0 con 0x7f7cb00cade0 2024-09-17T05:41:14.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7ca7400640 1 -- 172.21.15.122:0/3249417181 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ca800a850 con 0x7f7cb00cade0 2024-09-17T05:41:14.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb01f50f0 con 0x7f7cb00cade0 2024-09-17T05:41:14.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7ca7400640 1 -- 172.21.15.122:0/3249417181 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8017420 con 0x7f7cb00cade0 2024-09-17T05:41:14.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb00cb7a0 con 0x7f7cb00cade0 2024-09-17T05:41:14.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.060+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb0160890 con 0x7f7cb00cade0 2024-09-17T05:41:14.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.064+0000 7f7ca7400640 1 -- 172.21.15.122:0/3249417181 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 54) ==== 100600+0+0 (secure 0 0 0) 0x7f7ca8014070 con 0x7f7cb00cade0 2024-09-17T05:41:14.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.064+0000 7f7ca7400640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075720 0x7f7c90077be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:14.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.064+0000 7f7ca7400640 1 -- 172.21.15.122:0/3249417181 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f7ca8095d90 con 0x7f7cb00cade0 2024-09-17T05:41:14.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.068+0000 7f7caf400640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075720 0x7f7c90077be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:14.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.072+0000 7f7ca7400640 1 -- 172.21.15.122:0/3249417181 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7ca801d060 con 0x7f7cb00cade0 2024-09-17T05:41:14.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.080+0000 7f7caf400640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075720 0x7f7c90077be0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00cf4b0 tx=0x7f7ca00091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:14.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:14.412+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone11792", "target": ["mon-mgr", ""]}) -- 0x7f7cb00b6ff0 con 0x7f7c90075720 2024-09-17T05:41:15.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.064+0000 7f7ca7400640 1 -- 172.21.15.122:0/3249417181 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7cb00b6ff0 con 0x7f7c90075720 2024-09-17T05:41:15.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075720 msgr2=0x7f7c90077be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:15.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075720 0x7f7c90077be0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00cf4b0 tx=0x7f7ca00091e0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb00cade0 msgr2=0x7f7cb00cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:15.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb00cade0 0x7f7cb00cb260 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8001380 tx=0x7f7ca800f8f0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 shutdown_connections 2024-09-17T05:41:15.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075720 0x7f7c90077be0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb00cade0 0x7f7cb00cb260 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb006dd30 0x7f7cb015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 --2- 172.21.15.122:0/3249417181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c94003fe0 0x7f7cb01f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 >> 172.21.15.122:0/3249417181 conn(0x7f7cb00d1cf0 msgr2=0x7f7cb00b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:15.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 shutdown_connections 2024-09-17T05:41:15.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.068+0000 7f7cb4e00640 1 -- 172.21.15.122:0/3249417181 wait complete. 2024-09-17T05:41:15.100 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:15.100+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:15.101 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:15.100+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:15.101 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:15.100+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:15.101 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:15.100+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:15.101 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:15.100+0000 7fc0a0400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:15.115 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81883 2024-09-17T05:41:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.328+0000 7efc79c00640 1 -- 172.21.15.122:0/3965707707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 msgr2=0x7efc58004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.328+0000 7efc79c00640 1 --2- 172.21.15.122:0/3965707707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 0x7efc58004c90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c700 tx=0x7efc6001c6c0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.328+0000 7efc73400640 1 -- 172.21.15.122:0/3965707707 reap_dead start 2024-09-17T05:41:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.328+0000 7efc79c00640 1 -- 172.21.15.122:0/3965707707 shutdown_connections 2024-09-17T05:41:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.328+0000 7efc79c00640 1 -- 172.21.15.122:0/3965707707 >> 172.21.15.122:0/3965707707 conn(0x7efc740cd740 msgr2=0x7efc740cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.328+0000 7efc79c00640 1 -- 172.21.15.122:0/3965707707 shutdown_connections 2024-09-17T05:41:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.328+0000 7efc79c00640 1 -- 172.21.15.122:0/3965707707 wait complete. 2024-09-17T05:41:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 Processor -- start 2024-09-17T05:41:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 -- start start 2024-09-17T05:41:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 0x7efc7415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc741f1750 0x7efc741f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc741f57d0 0x7efc741f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc74070000 con 0x7efc58004810 2024-09-17T05:41:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc740cf530 con 0x7efc741f57d0 2024-09-17T05:41:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efc74070180 con 0x7efc741f1750 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 0x7efc7415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 0x7efc7415dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50158/0 (socket says 172.21.15.122:50158) 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 -- 172.21.15.122:0/2984624695 learned_addr learned my addr 172.21.15.122:0/2984624695 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 -- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc741f1750 msgr2=0x7efc741f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc741f1750 0x7efc741f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 -- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc741f57d0 msgr2=0x7efc741f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc741f57d0 0x7efc741f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 -- 172.21.15.122:0/2984624695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc740c9920 con 0x7efc58004810 2024-09-17T05:41:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc72a00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 0x7efc7415dc30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7efc6400e9f0 tx=0x7efc6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc6aa00640 1 -- 172.21.15.122:0/2984624695 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6400ad60 con 0x7efc58004810 2024-09-17T05:41:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc6aa00640 1 -- 172.21.15.122:0/2984624695 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc64010430 con 0x7efc58004810 2024-09-17T05:41:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc740c9c00 con 0x7efc58004810 2024-09-17T05:41:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc6aa00640 1 -- 172.21.15.122:0/2984624695 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64010840 con 0x7efc58004810 2024-09-17T05:41:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc740ca1c0 con 0x7efc58004810 2024-09-17T05:41:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.332+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc740706e0 con 0x7efc58004810 2024-09-17T05:41:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.336+0000 7efc6aa00640 1 -- 172.21.15.122:0/2984624695 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 54) ==== 100600+0+0 (secure 0 0 0) 0x7efc640109e0 con 0x7efc58004810 2024-09-17T05:41:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.336+0000 7efc6aa00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc54075720 0x7efc54077be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.336+0000 7efc6aa00640 1 -- 172.21.15.122:0/2984624695 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7efc64093c60 con 0x7efc58004810 2024-09-17T05:41:15.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.336+0000 7efc72000640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc54075720 0x7efc54077be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.340+0000 7efc72000640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc54075720 0x7efc54077be0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efc740b3320 tx=0x7efc58005840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.340+0000 7efc6aa00640 1 -- 172.21.15.122:0/2984624695 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efc64060770 con 0x7efc58004810 2024-09-17T05:41:15.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.420+0000 7efc6aa00640 1 -- 172.21.15.122:0/2984624695 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 55) ==== 100651+0+0 (secure 0 0 0) 0x7efc64010cd0 con 0x7efc58004810 2024-09-17T05:41:15.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:15.684+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81883", "target": ["mon-mgr", ""]}) -- 0x7efc740c4590 con 0x7efc54075720 2024-09-17T05:41:16.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.444+0000 7efc6aa00640 1 -- 172.21.15.122:0/2984624695 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc740c4590 con 0x7efc54075720 2024-09-17T05:41:16.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc54075720 msgr2=0x7efc54077be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:16.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc54075720 0x7efc54077be0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efc740b3320 tx=0x7efc58005840 comp rx=0 tx=0).stop 2024-09-17T05:41:16.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 msgr2=0x7efc7415dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:16.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 0x7efc7415dc30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7efc6400e9f0 tx=0x7efc6400eeb0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 shutdown_connections 2024-09-17T05:41:16.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc54075720 0x7efc54077be0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc741f57d0 0x7efc741f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc741f1750 0x7efc741f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 --2- 172.21.15.122:0/2984624695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc58004810 0x7efc7415dc30 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 >> 172.21.15.122:0/2984624695 conn(0x7efc740cd740 msgr2=0x7efc74073690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:16.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 shutdown_connections 2024-09-17T05:41:16.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.448+0000 7efc79c00640 1 -- 172.21.15.122:0/2984624695 wait complete. 2024-09-17T05:41:16.493 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone40724 2024-09-17T05:41:16.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.808+0000 7fb86ae00640 1 -- 172.21.15.122:0/2997171403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415ac00 msgr2=0x7fb8640b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:16.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.808+0000 7fb86ae00640 1 --2- 172.21.15.122:0/2997171403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415ac00 0x7fb8640b4650 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb854009900 tx=0x7fb85401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.808+0000 7fb86ae00640 1 -- 172.21.15.122:0/2997171403 shutdown_connections 2024-09-17T05:41:16.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.808+0000 7fb86ae00640 1 --2- 172.21.15.122:0/2997171403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415ac00 0x7fb8640b4650 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.808+0000 7fb86ae00640 1 -- 172.21.15.122:0/2997171403 >> 172.21.15.122:0/2997171403 conn(0x7fb86406ecd0 msgr2=0x7fb86406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:16.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.808+0000 7fb86ae00640 1 -- 172.21.15.122:0/2997171403 shutdown_connections 2024-09-17T05:41:16.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.808+0000 7fb86ae00640 1 -- 172.21.15.122:0/2997171403 wait complete. 2024-09-17T05:41:16.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb86ae00640 1 Processor -- start 2024-09-17T05:41:16.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb86ae00640 1 -- start start 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb86ae00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8640cfee0 0x7fb8641fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb86ae00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb86415ac00 0x7fb86414c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb86ae00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415b6f0 0x7fb8640ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb86ae00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb864169480 con 0x7fb86415b6f0 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb86ae00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb864169300 con 0x7fb8640cfee0 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb86ae00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb864169600 con 0x7fb86415ac00 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb868e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415b6f0 0x7fb8640ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb868e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415b6f0 0x7fb8640ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50186/0 (socket says 172.21.15.122:50186) 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb868e00640 1 -- 172.21.15.122:0/2895489841 learned_addr learned my addr 172.21.15.122:0/2895489841 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:16.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb863e00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8640cfee0 0x7fb8641fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:16.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb863400640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb86415ac00 0x7fb86414c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:16.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb868e00640 1 -- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb86415ac00 msgr2=0x7fb86414c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb868e00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb86415ac00 0x7fb86414c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb868e00640 1 -- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8640cfee0 msgr2=0x7fb8641fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb868e00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8640cfee0 0x7fb8641fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.812+0000 7fb868e00640 1 -- 172.21.15.122:0/2895489841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8641f8280 con 0x7fb86415b6f0 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.816+0000 7fb868e00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415b6f0 0x7fb8640ae4e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00b970 tx=0x7fb85c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.816+0000 7fb85be00640 1 -- 172.21.15.122:0/2895489841 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c019070 con 0x7fb86415b6f0 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.816+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86414cb80 con 0x7fb86415b6f0 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.816+0000 7fb85be00640 1 -- 172.21.15.122:0/2895489841 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb85c00cbd0 con 0x7fb86415b6f0 2024-09-17T05:41:16.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.816+0000 7fb85be00640 1 -- 172.21.15.122:0/2895489841 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0146a0 con 0x7fb86415b6f0 2024-09-17T05:41:16.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.816+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8641fc300 con 0x7fb86415b6f0 2024-09-17T05:41:16.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.816+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8640b4650 con 0x7fb86415b6f0 2024-09-17T05:41:16.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.816+0000 7fb85be00640 1 -- 172.21.15.122:0/2895489841 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 55) ==== 100651+0+0 (secure 0 0 0) 0x7fb85c01c070 con 0x7fb86415b6f0 2024-09-17T05:41:16.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.820+0000 7fb85be00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8440757b0 0x7fb844077c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:16.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.820+0000 7fb85be00640 1 -- 172.21.15.122:0/2895489841 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fb85c0931c0 con 0x7fb86415b6f0 2024-09-17T05:41:16.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.828+0000 7fb863e00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8440757b0 0x7fb844077c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:16.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.832+0000 7fb863e00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8440757b0 0x7fb844077c70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb8540af000 tx=0x7fb85401b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:16.832+0000 7fb85be00640 1 -- 172.21.15.122:0/2895489841 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb85c05fc50 con 0x7fb86415b6f0 2024-09-17T05:41:16.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:16.924+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:16.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:16.924+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:16.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:16.924+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:16.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:16.924+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:16.923 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:16.924+0000 7fc09e600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:17.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:17.136+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone40724", "target": ["mon-mgr", ""]}) -- 0x7fb86415a410 con 0x7fb8440757b0 2024-09-17T05:41:17.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:17.496+0000 7fb85be00640 1 -- 172.21.15.122:0/2895489841 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 56) ==== 100702+0+0 (secure 0 0 0) 0x7fb85c006d10 con 0x7fb86415b6f0 2024-09-17T05:41:18.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.376+0000 7fb85be00640 1 -- 172.21.15.122:0/2895489841 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb86415a410 con 0x7fb8440757b0 2024-09-17T05:41:18.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8440757b0 msgr2=0x7fb844077c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:18.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8440757b0 0x7fb844077c70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb8540af000 tx=0x7fb85401b120 comp rx=0 tx=0).stop 2024-09-17T05:41:18.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415b6f0 msgr2=0x7fb8640ae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:18.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415b6f0 0x7fb8640ae4e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00b970 tx=0x7fb85c00be30 comp rx=0 tx=0).stop 2024-09-17T05:41:18.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 shutdown_connections 2024-09-17T05:41:18.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8440757b0 0x7fb844077c70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:18.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86415b6f0 0x7fb8640ae4e0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:18.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb86415ac00 0x7fb86414c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:18.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 --2- 172.21.15.122:0/2895489841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8640cfee0 0x7fb8641fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:18.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 >> 172.21.15.122:0/2895489841 conn(0x7fb86406ecd0 msgr2=0x7fb864072be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:18.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 shutdown_connections 2024-09-17T05:41:18.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.384+0000 7fb86ae00640 1 -- 172.21.15.122:0/2895489841 wait complete. 2024-09-17T05:41:18.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone12085 2024-09-17T05:41:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 -- 172.21.15.122:0/3758528942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0b2e50 msgr2=0x7f83dc06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 --2- 172.21.15.122:0/3758528942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0b2e50 0x7f83dc06f8f0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0d06f0 tx=0x7f83d401cb70 comp rx=0 tx=0).stop 2024-09-17T05:41:18.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 -- 172.21.15.122:0/3758528942 shutdown_connections 2024-09-17T05:41:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 --2- 172.21.15.122:0/3758528942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0b2e50 0x7f83dc06f8f0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 --2- 172.21.15.122:0/3758528942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83dc15ac70 0x7f83dc0cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 -- 172.21.15.122:0/3758528942 >> 172.21.15.122:0/3758528942 conn(0x7f83dc0cd740 msgr2=0x7f83dc0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 -- 172.21.15.122:0/3758528942 shutdown_connections 2024-09-17T05:41:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 -- 172.21.15.122:0/3758528942 wait complete. 2024-09-17T05:41:18.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 Processor -- start 2024-09-17T05:41:18.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.676+0000 7f83e4600640 1 -- start start 2024-09-17T05:41:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e4600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83dc0b2e50 0x7f83dc0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e4600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0cbdb0 0x7f83dc0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:18.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e4600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83dc15ac70 0x7f83dc0b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e4600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc167b50 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e4600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1679d0 con 0x7f83dc0b2e50 2024-09-17T05:41:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e4600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f83dc167cd0 con 0x7f83dc15ac70 2024-09-17T05:41:18.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0cbdb0 0x7f83dc0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0cbdb0 0x7f83dc0b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50190/0 (socket says 172.21.15.122:50190) 2024-09-17T05:41:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 -- 172.21.15.122:0/2216104881 learned_addr learned my addr 172.21.15.122:0/2216104881 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1c00640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83dc0b2e50 0x7f83dc0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 -- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83dc15ac70 msgr2=0x7f83dc0b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83dc15ac70 0x7f83dc0b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 -- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83dc0b2e50 msgr2=0x7f83dc0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83dc0b2e50 0x7f83dc0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:18.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 -- 172.21.15.122:0/2216104881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc0c62b0 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e1200640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0cbdb0 0x7f83dc0b5d60 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c450 tx=0x7f83c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83d9600640 1 -- 172.21.15.122:0/2216104881 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8017070 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83d9600640 1 -- 172.21.15.122:0/2216104881 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83c8003600 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc163e30 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83d9600640 1 -- 172.21.15.122:0/2216104881 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8003a50 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.680+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc164110 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.684+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83dc06f8f0 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.684+0000 7f83d9600640 1 -- 172.21.15.122:0/2216104881 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 56) ==== 100702+0+0 (secure 0 0 0) 0x7f83c801a070 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.688+0000 7f83d9600640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83bc075840 0x7f83bc077d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.688+0000 7f83d9600640 1 -- 172.21.15.122:0/2216104881 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f83c8092800 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.696+0000 7f83d9600640 1 -- 172.21.15.122:0/2216104881 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f83c805f470 con 0x7f83dc0cbdb0 2024-09-17T05:41:18.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.716+0000 7f83e1c00640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83bc075840 0x7f83bc077d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:18.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:18.744+0000 7f83e1c00640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83bc075840 0x7f83bc077d00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cf2c0 tx=0x7f83d00091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:19.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:19.024+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone12085", "target": ["mon-mgr", ""]}) -- 0x7f83dc0713d0 con 0x7f83bc075840 2024-09-17T05:41:19.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:19.568+0000 7f83d9600640 1 -- 172.21.15.122:0/2216104881 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 57) ==== 100753+0+0 (secure 0 0 0) 0x7f83c805afe0 con 0x7f83dc0cbdb0 2024-09-17T05:41:20.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.032+0000 7f83d9600640 1 -- 172.21.15.122:0/2216104881 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f83dc0713d0 con 0x7f83bc075840 2024-09-17T05:41:20.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.036+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83bc075840 msgr2=0x7f83bc077d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:20.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.036+0000 7f83e4600640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83bc075840 0x7f83bc077d00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cf2c0 tx=0x7f83d00091e0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.036+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0cbdb0 msgr2=0x7f83dc0b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:20.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.036+0000 7f83e4600640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0cbdb0 0x7f83dc0b5d60 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c450 tx=0x7f83c800c910 comp rx=0 tx=0).stop 2024-09-17T05:41:20.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.040+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 shutdown_connections 2024-09-17T05:41:20.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.040+0000 7f83e4600640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f83bc075840 0x7f83bc077d00 secure :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cf2c0 tx=0x7f83d00091e0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.040+0000 7f83e4600640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83dc15ac70 0x7f83dc0b62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.040+0000 7f83e4600640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83dc0cbdb0 0x7f83dc0b5d60 secure :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c450 tx=0x7f83c800c910 comp rx=0 tx=0).stop 2024-09-17T05:41:20.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.040+0000 7f83e4600640 1 --2- 172.21.15.122:0/2216104881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83dc0b2e50 0x7f83dc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.040+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 >> 172.21.15.122:0/2216104881 conn(0x7f83dc0cd740 msgr2=0x7f83dc157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:20.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.040+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 shutdown_connections 2024-09-17T05:41:20.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.040+0000 7f83e4600640 1 -- 172.21.15.122:0/2216104881 wait complete. 2024-09-17T05:41:20.088 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone46056 2024-09-17T05:41:20.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 -- 172.21.15.122:0/1321114498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00cbdb0 msgr2=0x7feff00b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:20.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 --2- 172.21.15.122:0/1321114498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00cbdb0 0x7feff00b2740 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7feff006ed00 tx=0x7fefe001c600 comp rx=0 tx=0).stop 2024-09-17T05:41:20.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 -- 172.21.15.122:0/1321114498 shutdown_connections 2024-09-17T05:41:20.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 --2- 172.21.15.122:0/1321114498 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feff00b2c80 0x7feff0073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 --2- 172.21.15.122:0/1321114498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00cbdb0 0x7feff00b2740 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 --2- 172.21.15.122:0/1321114498 >> v2:172.21.15.173:3300/0 conn(0x7feff015ac70 0x7feff00cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 -- 172.21.15.122:0/1321114498 >> 172.21.15.122:0/1321114498 conn(0x7feff00cd740 msgr2=0x7feff00cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:20.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 -- 172.21.15.122:0/1321114498 shutdown_connections 2024-09-17T05:41:20.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.440+0000 7feff6800640 1 -- 172.21.15.122:0/1321114498 wait complete. 2024-09-17T05:41:20.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.444+0000 7feff6800640 1 Processor -- start 2024-09-17T05:41:20.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.444+0000 7feff6800640 1 -- start start 2024-09-17T05:41:20.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00b2c80 0x7feff00c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff00cbdb0 0x7feff00b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feff015ac70 0x7feff0158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feff006d250 con 0x7feff00b2c80 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feff006d0d0 con 0x7feff00cbdb0 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7feff006d3d0 con 0x7feff015ac70 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00b2c80 0x7feff00c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00b2c80 0x7feff00c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50208/0 (socket says 172.21.15.122:50208) 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 -- 172.21.15.122:0/2495836119 learned_addr learned my addr 172.21.15.122:0/2495836119 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 -- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feff015ac70 msgr2=0x7feff0158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feff015ac70 0x7feff0158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 -- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff00cbdb0 msgr2=0x7feff00b53e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:41:20.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff00cbdb0 0x7feff00b53e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:20.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 -- 172.21.15.122:0/2495836119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff0159f00 con 0x7feff00b2c80 2024-09-17T05:41:20.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefef400640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00b2c80 0x7feff00c9790 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0058d0 tx=0x7fefdc00ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:20.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefe7400640 1 -- 172.21.15.122:0/2495836119 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc019070 con 0x7feff00b2c80 2024-09-17T05:41:20.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefe7400640 1 -- 172.21.15.122:0/2495836119 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefdc0079b0 con 0x7feff00b2c80 2024-09-17T05:41:20.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff00c9e90 con 0x7feff00b2c80 2024-09-17T05:41:20.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7fefe7400640 1 -- 172.21.15.122:0/2495836119 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc007de0 con 0x7feff00b2c80 2024-09-17T05:41:20.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff00b4be0 con 0x7feff00b2c80 2024-09-17T05:41:20.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.448+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff006d910 con 0x7feff00b2c80 2024-09-17T05:41:20.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.452+0000 7fefe7400640 1 -- 172.21.15.122:0/2495836119 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 57) ==== 100753+0+0 (secure 0 0 0) 0x7fefdc01c070 con 0x7feff00b2c80 2024-09-17T05:41:20.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.452+0000 7fefe7400640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefd00758d0 0x7fefd0077d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:20.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.452+0000 7fefe7400640 1 -- 172.21.15.122:0/2495836119 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fefdc092db0 con 0x7feff00b2c80 2024-09-17T05:41:20.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.464+0000 7fefe7400640 1 -- 172.21.15.122:0/2495836119 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7feff0159f00 con 0x7feff00b2c80 2024-09-17T05:41:20.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.496+0000 7fefeea00640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefd00758d0 0x7fefd0077d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.508+0000 7fefeea00640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefd00758d0 0x7fefd0077d90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fefe00af000 tx=0x7fefe001ae40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:20.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:20.796+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone46056", "target": ["mon-mgr", ""]}) -- 0x7feff00cb7e0 con 0x7fefd00758d0 2024-09-17T05:41:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.748+0000 7fefe7400640 1 -- 172.21.15.122:0/2495836119 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feff00cb7e0 con 0x7fefd00758d0 2024-09-17T05:41:21.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefd00758d0 msgr2=0x7fefd0077d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:21.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefd00758d0 0x7fefd0077d90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fefe00af000 tx=0x7fefe001ae40 comp rx=0 tx=0).stop 2024-09-17T05:41:21.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00b2c80 msgr2=0x7feff00c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:21.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00b2c80 0x7feff00c9790 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0058d0 tx=0x7fefdc00ae10 comp rx=0 tx=0).stop 2024-09-17T05:41:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 shutdown_connections 2024-09-17T05:41:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fefd00758d0 0x7fefd0077d90 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feff015ac70 0x7feff0158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff00cbdb0 0x7feff00b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 --2- 172.21.15.122:0/2495836119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff00b2c80 0x7feff00c9790 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 >> 172.21.15.122:0/2495836119 conn(0x7feff00cd740 msgr2=0x7feff0071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 shutdown_connections 2024-09-17T05:41:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:21.760+0000 7feff6800640 1 -- 172.21.15.122:0/2495836119 wait complete. 2024-09-17T05:41:22.794 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone51580 2024-09-17T05:41:23.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.059+0000 7f6a34c00640 1 -- 172.21.15.122:0/955464449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 msgr2=0x7f6a300acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:23.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.059+0000 7f6a34c00640 1 --2- 172.21.15.122:0/955464449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 0x7f6a300acd40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009900 tx=0x7f6a2001d980 comp rx=0 tx=0).stop 2024-09-17T05:41:23.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.059+0000 7f6a34c00640 1 -- 172.21.15.122:0/955464449 shutdown_connections 2024-09-17T05:41:23.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.059+0000 7f6a34c00640 1 --2- 172.21.15.122:0/955464449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a300b49e0 0x7f6a3006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:23.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.059+0000 7f6a34c00640 1 --2- 172.21.15.122:0/955464449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 0x7f6a300acd40 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:23.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.059+0000 7f6a34c00640 1 -- 172.21.15.122:0/955464449 >> 172.21.15.122:0/955464449 conn(0x7f6a300b18f0 msgr2=0x7f6a300b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:23.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.059+0000 7f6a34c00640 1 -- 172.21.15.122:0/955464449 shutdown_connections 2024-09-17T05:41:23.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.059+0000 7f6a34c00640 1 -- 172.21.15.122:0/955464449 wait complete. 2024-09-17T05:41:23.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.063+0000 7f6a34c00640 1 Processor -- start 2024-09-17T05:41:23.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.063+0000 7f6a34c00640 1 -- start start 2024-09-17T05:41:23.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.067+0000 7f6a34c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a300b49e0 0x7f6a30149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:23.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.067+0000 7f6a34c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a300cd100 0x7f6a30156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:23.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.067+0000 7f6a34c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 0x7f6a301ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:23.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.067+0000 7f6a34c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a300cf1e0 con 0x7f6a3015dae0 2024-09-17T05:41:23.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.067+0000 7f6a2e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 0x7f6a301ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:23.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.067+0000 7f6a2e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 0x7f6a301ffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59846/0 (socket says 172.21.15.122:59846) 2024-09-17T05:41:23.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.067+0000 7f6a2e800640 1 -- 172.21.15.122:0/1118594917 learned_addr learned my addr 172.21.15.122:0/1118594917 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:23.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a2de00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a300b49e0 0x7f6a30149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:23.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a300cf060 con 0x7f6a300cd100 2024-09-17T05:41:23.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6a300cf360 con 0x7f6a300b49e0 2024-09-17T05:41:23.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a2d400640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a300cd100 0x7f6a30156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:23.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a2e800640 1 -- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a300b49e0 msgr2=0x7f6a30149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:23.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a2e800640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a300b49e0 0x7f6a30149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:23.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a2e800640 1 -- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a300cd100 msgr2=0x7f6a30156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:23.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a2e800640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a300cd100 0x7f6a30156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:23.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a2e800640 1 -- 172.21.15.122:0/1118594917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a302005c0 con 0x7f6a3015dae0 2024-09-17T05:41:23.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.071+0000 7f6a2e800640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 0x7f6a301ffa50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800ba10 tx=0x7f6a2800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:23.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.075+0000 7f6a26000640 1 -- 172.21.15.122:0/1118594917 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28019070 con 0x7f6a3015dae0 2024-09-17T05:41:23.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.075+0000 7f6a26000640 1 -- 172.21.15.122:0/1118594917 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a2800cc90 con 0x7f6a3015dae0 2024-09-17T05:41:23.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.075+0000 7f6a26000640 1 -- 172.21.15.122:0/1118594917 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280146a0 con 0x7f6a3015dae0 2024-09-17T05:41:23.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.075+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a302008a0 con 0x7f6a3015dae0 2024-09-17T05:41:23.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.075+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a30148b40 con 0x7f6a3015dae0 2024-09-17T05:41:23.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.079+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a080052a0 con 0x7f6a3015dae0 2024-09-17T05:41:23.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.083+0000 7f6a26000640 1 -- 172.21.15.122:0/1118594917 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 57) ==== 100753+0+0 (secure 0 0 0) 0x7f6a2801c070 con 0x7f6a3015dae0 2024-09-17T05:41:23.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.087+0000 7f6a26000640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a100759a0 0x7f6a10077e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:23.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.091+0000 7f6a26000640 1 -- 172.21.15.122:0/1118594917 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f6a28093470 con 0x7f6a3015dae0 2024-09-17T05:41:23.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.091+0000 7f6a26000640 1 -- 172.21.15.122:0/1118594917 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f6a28023020 con 0x7f6a3015dae0 2024-09-17T05:41:23.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.111+0000 7f6a2de00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a100759a0 0x7f6a10077e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:23.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.187+0000 7f6a2de00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a100759a0 0x7f6a10077e60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6a200b0000 tx=0x7f6a2001c1c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:23.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:23.427+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone51580", "target": ["mon-mgr", ""]}) -- 0x7f6a08000bd0 con 0x7f6a100759a0 2024-09-17T05:41:23.686 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:23.688+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:23.686 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:23.688+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:23.687 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:23.688+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:23.687 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:23.688+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:23.687 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:23.688+0000 7fc098c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:25.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:25.979+0000 7f6a26000640 1 -- 172.21.15.122:0/1118594917 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f6a2805bc20 con 0x7f6a3015dae0 2024-09-17T05:41:26.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.115+0000 7f6a26000640 1 -- 172.21.15.122:0/1118594917 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a08000bd0 con 0x7f6a100759a0 2024-09-17T05:41:26.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a100759a0 msgr2=0x7f6a10077e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:26.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a100759a0 0x7f6a10077e60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6a200b0000 tx=0x7f6a2001c1c0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 msgr2=0x7f6a301ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:26.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 0x7f6a301ffa50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800ba10 tx=0x7f6a2800bed0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 shutdown_connections 2024-09-17T05:41:26.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6a100759a0 0x7f6a10077e60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3015dae0 0x7f6a301ffa50 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a300cd100 0x7f6a30156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 --2- 172.21.15.122:0/1118594917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a300b49e0 0x7f6a30149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 >> 172.21.15.122:0/1118594917 conn(0x7f6a300b18f0 msgr2=0x7f6a30164620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:26.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 shutdown_connections 2024-09-17T05:41:26.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.119+0000 7f6a34c00640 1 -- 172.21.15.122:0/1118594917 wait complete. 2024-09-17T05:41:26.164 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone77831 2024-09-17T05:41:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.407+0000 7f059c600640 1 -- 172.21.15.122:0/1244826951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940afb10 msgr2=0x7f05940affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.407+0000 7f059c600640 1 --2- 172.21.15.122:0/1244826951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940afb10 0x7f05940affc0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f058c009900 tx=0x7f058c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.411+0000 7f059c600640 1 -- 172.21.15.122:0/1244826951 shutdown_connections 2024-09-17T05:41:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.411+0000 7f059c600640 1 --2- 172.21.15.122:0/1244826951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940afb10 0x7f05940affc0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.411+0000 7f059c600640 1 --2- 172.21.15.122:0/1244826951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05940b1650 0x7f05940b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.411+0000 7f059c600640 1 -- 172.21.15.122:0/1244826951 >> 172.21.15.122:0/1244826951 conn(0x7f05940d1dd0 msgr2=0x7f05940d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.411+0000 7f059c600640 1 -- 172.21.15.122:0/1244826951 shutdown_connections 2024-09-17T05:41:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.411+0000 7f059c600640 1 -- 172.21.15.122:0/1244826951 wait complete. 2024-09-17T05:41:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.411+0000 7f059c600640 1 Processor -- start 2024-09-17T05:41:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.411+0000 7f059c600640 1 -- start start 2024-09-17T05:41:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05940afb10 0x7f059406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940b1650 0x7f0594158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05941605d0 0x7f05941fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0594169540 con 0x7f05940b1650 2024-09-17T05:41:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05940b1430 con 0x7f05941605d0 2024-09-17T05:41:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f05941696c0 con 0x7f05940afb10 2024-09-17T05:41:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940b1650 0x7f0594158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940b1650 0x7f0594158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59864/0 (socket says 172.21.15.122:59864) 2024-09-17T05:41:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 -- 172.21.15.122:0/3884671929 learned_addr learned my addr 172.21.15.122:0/3884671929 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:26.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 -- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05940afb10 msgr2=0x7f059406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:26.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05940afb10 0x7f059406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 -- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05941605d0 msgr2=0x7f05941fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05941605d0 0x7f05941fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 -- 172.21.15.122:0/3884671929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059406fb40 con 0x7f05940b1650 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0599200640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940b1650 0x7f0594158230 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f058000b550 tx=0x7f058000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0591600640 1 -- 172.21.15.122:0/3884671929 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580014070 con 0x7f05940b1650 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0591600640 1 -- 172.21.15.122:0/3884671929 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0580010980 con 0x7f05940b1650 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059406f860 con 0x7f05940b1650 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f0591600640 1 -- 172.21.15.122:0/3884671929 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580010dd0 con 0x7f05940b1650 2024-09-17T05:41:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059406e090 con 0x7f05940b1650 2024-09-17T05:41:26.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.415+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05940affc0 con 0x7f05940b1650 2024-09-17T05:41:26.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.419+0000 7f0591600640 1 -- 172.21.15.122:0/3884671929 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f0580017070 con 0x7f05940b1650 2024-09-17T05:41:26.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.419+0000 7f0591600640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0574075960 0x7f0574077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:26.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.419+0000 7f0591600640 1 -- 172.21.15.122:0/3884671929 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f0580093290 con 0x7f05940b1650 2024-09-17T05:41:26.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.423+0000 7f0599c00640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0574075960 0x7f0574077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:26.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.427+0000 7f0591600640 1 -- 172.21.15.122:0/3884671929 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f058005fe40 con 0x7f05940b1650 2024-09-17T05:41:26.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.431+0000 7f0599c00640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0574075960 0x7f0574077e20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f058800abb0 tx=0x7f0588009250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:26.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:26.843+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone77831", "target": ["mon-mgr", ""]}) -- 0x7f05940d1080 con 0x7f0574075960 2024-09-17T05:41:27.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f0591600640 1 -- 172.21.15.122:0/3884671929 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f05940d1080 con 0x7f0574075960 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0574075960 msgr2=0x7f0574077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0574075960 0x7f0574077e20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f058800abb0 tx=0x7f0588009250 comp rx=0 tx=0).stop 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940b1650 msgr2=0x7f0594158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940b1650 0x7f0594158230 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f058000b550 tx=0x7f058000ba10 comp rx=0 tx=0).stop 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 shutdown_connections 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0574075960 0x7f0574077e20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05941605d0 0x7f05941fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05940b1650 0x7f0594158230 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 --2- 172.21.15.122:0/3884671929 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05940afb10 0x7f059406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:27.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 >> 172.21.15.122:0/3884671929 conn(0x7f05940d1dd0 msgr2=0x7f059415b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:27.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 shutdown_connections 2024-09-17T05:41:27.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.639+0000 7f059c600640 1 -- 172.21.15.122:0/3884671929 wait complete. 2024-09-17T05:41:27.691 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone11792 2024-09-17T05:41:27.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.983+0000 7fcd77000640 1 -- 172.21.15.122:0/3058570933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd701585f0 msgr2=0x7fcd70073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:27.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.983+0000 7fcd77000640 1 --2- 172.21.15.122:0/3058570933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd701585f0 0x7fcd70073a10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009840 tx=0x7fcd5c01c820 comp rx=0 tx=0).stop 2024-09-17T05:41:27.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.983+0000 7fcd77000640 1 -- 172.21.15.122:0/3058570933 shutdown_connections 2024-09-17T05:41:27.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.983+0000 7fcd77000640 1 --2- 172.21.15.122:0/3058570933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd7015eea0 0x7fcd700ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:27.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.983+0000 7fcd77000640 1 --2- 172.21.15.122:0/3058570933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd701585f0 0x7fcd70073a10 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:27.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.983+0000 7fcd77000640 1 -- 172.21.15.122:0/3058570933 >> 172.21.15.122:0/3058570933 conn(0x7fcd700cce90 msgr2=0x7fcd700cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:27.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.987+0000 7fcd77000640 1 -- 172.21.15.122:0/3058570933 shutdown_connections 2024-09-17T05:41:27.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.987+0000 7fcd77000640 1 -- 172.21.15.122:0/3058570933 wait complete. 2024-09-17T05:41:27.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.987+0000 7fcd77000640 1 Processor -- start 2024-09-17T05:41:27.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.987+0000 7fcd77000640 1 -- start start 2024-09-17T05:41:27.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd70073f50 0x7fcd70154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:27.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd7015eea0 0x7fcd70155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd7014ba20 0x7fcd701399a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7006cda0 con 0x7fcd7014ba20 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7006cc20 con 0x7fcd7015eea0 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7006cf20 con 0x7fcd70073f50 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd7014ba20 0x7fcd701399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd7014ba20 0x7fcd701399a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59880/0 (socket says 172.21.15.122:59880) 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 -- 172.21.15.122:0/2188312516 learned_addr learned my addr 172.21.15.122:0/2188312516 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd6fe00640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd70073f50 0x7fcd70154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:27.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd6f400640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd7015eea0 0x7fcd70155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:27.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 -- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd70073f50 msgr2=0x7fcd70154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:27.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd70073f50 0x7fcd70154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:27.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 -- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd7015eea0 msgr2=0x7fcd70155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd7015eea0 0x7fcd70155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 -- 172.21.15.122:0/2188312516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7014bf20 con 0x7fcd7014ba20 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd75000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd7014ba20 0x7fcd701399a0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800cce0 tx=0x7fcd6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd67e00640 1 -- 172.21.15.122:0/2188312516 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800f450 con 0x7fcd7014ba20 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd67e00640 1 -- 172.21.15.122:0/2188312516 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd6800f6e0 con 0x7fcd7014ba20 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7014c200 con 0x7fcd7014ba20 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd67e00640 1 -- 172.21.15.122:0/2188312516 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800fb30 con 0x7fcd7014ba20 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7014c7c0 con 0x7fcd7014ba20 2024-09-17T05:41:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.991+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd70068e70 con 0x7fcd7014ba20 2024-09-17T05:41:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.995+0000 7fcd67e00640 1 -- 172.21.15.122:0/2188312516 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7fcd6800fcd0 con 0x7fcd7014ba20 2024-09-17T05:41:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.995+0000 7fcd67e00640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd50075a60 0x7fcd50077f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.995+0000 7fcd67e00640 1 -- 172.21.15.122:0/2188312516 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fcd680940c0 con 0x7fcd7014ba20 2024-09-17T05:41:27.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:27.999+0000 7fcd67e00640 1 -- 172.21.15.122:0/2188312516 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fcd68014030 con 0x7fcd7014ba20 2024-09-17T05:41:27.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:28.003+0000 7fcd6fe00640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd50075a60 0x7fcd50077f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:28.031+0000 7fcd6fe00640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd50075a60 0x7fcd50077f20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0af000 tx=0x7fcd5c01b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:28.323+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone11792", "target": ["mon-mgr", ""]}) -- 0x7fcd700cc6d0 con 0x7fcd50075a60 2024-09-17T05:41:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd67e00640 1 -- 172.21.15.122:0/2188312516 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd700cc6d0 con 0x7fcd50075a60 2024-09-17T05:41:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd50075a60 msgr2=0x7fcd50077f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd50075a60 0x7fcd50077f20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0af000 tx=0x7fcd5c01b060 comp rx=0 tx=0).stop 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd7014ba20 msgr2=0x7fcd701399a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd7014ba20 0x7fcd701399a0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800cce0 tx=0x7fcd6800a620 comp rx=0 tx=0).stop 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 shutdown_connections 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fcd50075a60 0x7fcd50077f20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd7014ba20 0x7fcd701399a0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd7015eea0 0x7fcd70155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 --2- 172.21.15.122:0/2188312516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd70073f50 0x7fcd70154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 >> 172.21.15.122:0/2188312516 conn(0x7fcd700cce90 msgr2=0x7fcd7006f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 shutdown_connections 2024-09-17T05:41:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:29.971+0000 7fcd77000640 1 -- 172.21.15.122:0/2188312516 wait complete. 2024-09-17T05:41:30.002 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81883 2024-09-17T05:41:30.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 -- 172.21.15.122:0/3091386130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc0ca590 msgr2=0x7fb9bc0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:30.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3091386130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc0ca590 0x7fb9bc0cda20 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0cd490 tx=0x7fb9b401d880 comp rx=0 tx=0).stop 2024-09-17T05:41:30.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 -- 172.21.15.122:0/3091386130 shutdown_connections 2024-09-17T05:41:30.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3091386130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc0ca590 0x7fb9bc0cda20 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:30.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3091386130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9bc0d0520 0x7fb9bc0ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:30.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3091386130 >> v2:172.21.15.173:3300/0 conn(0x7fb9bc157c70 0x7fb9bc15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:30.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 -- 172.21.15.122:0/3091386130 >> 172.21.15.122:0/3091386130 conn(0x7fb9bc0cfa00 msgr2=0x7fb9bc0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:30.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 -- 172.21.15.122:0/3091386130 shutdown_connections 2024-09-17T05:41:30.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 -- 172.21.15.122:0/3091386130 wait complete. 2024-09-17T05:41:30.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 Processor -- start 2024-09-17T05:41:30.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.275+0000 7fb9bb200640 1 -- start start 2024-09-17T05:41:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9bb200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9bc0ca590 0x7fb9bc1f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9bb200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9bc0d0520 0x7fb9bc0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9bb200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc157c70 0x7fb9bc1fb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9bb200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc06eb50 con 0x7fb9bc157c70 2024-09-17T05:41:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9bb200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc06e9d0 con 0x7fb9bc0ca590 2024-09-17T05:41:30.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9bb200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb9bc06ecd0 con 0x7fb9bc0d0520 2024-09-17T05:41:30.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9ba800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc157c70 0x7fb9bc1fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:30.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9ba800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc157c70 0x7fb9bc1fb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59898/0 (socket says 172.21.15.122:59898) 2024-09-17T05:41:30.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9ba800640 1 -- 172.21.15.122:0/3743202085 learned_addr learned my addr 172.21.15.122:0/3743202085 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:30.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9ba800640 1 -- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9bc0d0520 msgr2=0x7fb9bc0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:30.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9ba800640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9bc0d0520 0x7fb9bc0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:30.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9ba800640 1 -- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9bc0ca590 msgr2=0x7fb9bc1f85b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:41:30.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9ba800640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9bc0ca590 0x7fb9bc1f85b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:30.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.279+0000 7fb9ba800640 1 -- 172.21.15.122:0/3743202085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc1fbad0 con 0x7fb9bc157c70 2024-09-17T05:41:30.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.283+0000 7fb9ba800640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc157c70 0x7fb9bc1fb050 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4007740 tx=0x7fb9b40a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:30.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.283+0000 7fb9b2000640 1 -- 172.21.15.122:0/3743202085 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4007c60 con 0x7fb9bc157c70 2024-09-17T05:41:30.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.283+0000 7fb9b2000640 1 -- 172.21.15.122:0/3743202085 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9b4003980 con 0x7fb9bc157c70 2024-09-17T05:41:30.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.283+0000 7fb9b2000640 1 -- 172.21.15.122:0/3743202085 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4003dd0 con 0x7fb9bc157c70 2024-09-17T05:41:30.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.283+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc1fbdb0 con 0x7fb9bc157c70 2024-09-17T05:41:30.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.283+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1f8090 con 0x7fb9bc157c70 2024-09-17T05:41:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.287+0000 7fb9b2000640 1 -- 172.21.15.122:0/3743202085 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7fb9b40a68e0 con 0x7fb9bc157c70 2024-09-17T05:41:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.287+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9940052a0 con 0x7fb9bc157c70 2024-09-17T05:41:30.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.287+0000 7fb9b2000640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb99c075960 0x7fb99c077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:30.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.287+0000 7fb9b2000640 1 -- 172.21.15.122:0/3743202085 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fb9b412d230 con 0x7fb9bc157c70 2024-09-17T05:41:30.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.295+0000 7fb9b2000640 1 -- 172.21.15.122:0/3743202085 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb9b40174a0 con 0x7fb9bc157c70 2024-09-17T05:41:30.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.327+0000 7fb9b9e00640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb99c075960 0x7fb99c077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:30.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.367+0000 7fb9b9e00640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb99c075960 0x7fb99c077e20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00b8f0 tx=0x7fb9ac00b2f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:30.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:30.671+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81883", "target": ["mon-mgr", ""]}) -- 0x7fb994000bd0 con 0x7fb99c075960 2024-09-17T05:41:32.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.263+0000 7fb9b2000640 1 -- 172.21.15.122:0/3743202085 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb994000bd0 con 0x7fb99c075960 2024-09-17T05:41:32.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb99c075960 msgr2=0x7fb99c077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:32.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb99c075960 0x7fb99c077e20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00b8f0 tx=0x7fb9ac00b2f0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc157c70 msgr2=0x7fb9bc1fb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:32.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc157c70 0x7fb9bc1fb050 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4007740 tx=0x7fb9b40a66f0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 shutdown_connections 2024-09-17T05:41:32.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb99c075960 0x7fb99c077e20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9bc157c70 0x7fb9bc1fb050 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9bc0d0520 0x7fb9bc0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 --2- 172.21.15.122:0/3743202085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9bc0ca590 0x7fb9bc1f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 >> 172.21.15.122:0/3743202085 conn(0x7fb9bc0cfa00 msgr2=0x7fb9bc0cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:32.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 shutdown_connections 2024-09-17T05:41:32.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.267+0000 7fb9bb200640 1 -- 172.21.15.122:0/3743202085 wait complete. 2024-09-17T05:41:32.295 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone40724 2024-09-17T05:41:32.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.535+0000 7f5f99400640 1 -- 172.21.15.122:0/219252598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f9415bb90 msgr2=0x7f5f94160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:32.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.535+0000 7f5f99400640 1 --2- 172.21.15.122:0/219252598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f9415bb90 0x7f5f94160060 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f5f94070410 tx=0x7f5f8401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.535+0000 7f5f99400640 1 -- 172.21.15.122:0/219252598 shutdown_connections 2024-09-17T05:41:32.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.535+0000 7f5f99400640 1 --2- 172.21.15.122:0/219252598 >> v2:172.21.15.173:3300/0 conn(0x7f5f941605a0 0x7f5f940ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.535+0000 7f5f99400640 1 --2- 172.21.15.122:0/219252598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f9415bb90 0x7f5f94160060 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.535+0000 7f5f99400640 1 -- 172.21.15.122:0/219252598 >> 172.21.15.122:0/219252598 conn(0x7f5f940cfa00 msgr2=0x7f5f940cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:32.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.535+0000 7f5f99400640 1 -- 172.21.15.122:0/219252598 shutdown_connections 2024-09-17T05:41:32.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.535+0000 7f5f99400640 1 -- 172.21.15.122:0/219252598 wait complete. 2024-09-17T05:41:32.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f99400640 1 Processor -- start 2024-09-17T05:41:32.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f99400640 1 -- start start 2024-09-17T05:41:32.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f99400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f940cbd80 0x7f5f940c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:32.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f99400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f9415bb90 0x7f5f94159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:32.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f99400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f940b4b10 0x7f5f9415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:32.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f99400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f940729a0 con 0x7f5f940cbd80 2024-09-17T05:41:32.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f99400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94072820 con 0x7f5f940b4b10 2024-09-17T05:41:32.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f99400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5f940ca590 con 0x7f5f9415bb90 2024-09-17T05:41:32.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f940cbd80 0x7f5f940c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:32.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f940cbd80 0x7f5f940c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56338/0 (socket says 172.21.15.122:56338) 2024-09-17T05:41:32.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.539+0000 7f5f93e00640 1 -- 172.21.15.122:0/3001827545 learned_addr learned my addr 172.21.15.122:0/3001827545 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:32.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f93e00640 1 -- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f9415bb90 msgr2=0x7f5f94159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:32.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f93e00640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f9415bb90 0x7f5f94159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f93e00640 1 -- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f940b4b10 msgr2=0x7f5f9415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:32.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f93e00640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f940b4b10 0x7f5f9415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:32.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f93e00640 1 -- 172.21.15.122:0/3001827545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9415b020 con 0x7f5f940cbd80 2024-09-17T05:41:32.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f93e00640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f940cbd80 0x7f5f940c5b60 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f5f8401af70 tx=0x7f5f840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:32.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f8be00640 1 -- 172.21.15.122:0/3001827545 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f840bb070 con 0x7f5f940cbd80 2024-09-17T05:41:32.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f8be00640 1 -- 172.21.15.122:0/3001827545 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f840044e0 con 0x7f5f940cbd80 2024-09-17T05:41:32.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9415ad40 con 0x7f5f940cbd80 2024-09-17T05:41:32.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f8be00640 1 -- 172.21.15.122:0/3001827545 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f840b2040 con 0x7f5f940cbd80 2024-09-17T05:41:32.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f940adf70 con 0x7f5f940cbd80 2024-09-17T05:41:32.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f940ae1b0 con 0x7f5f940cbd80 2024-09-17T05:41:32.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.543+0000 7f5f8be00640 1 -- 172.21.15.122:0/3001827545 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f5f840be070 con 0x7f5f940cbd80 2024-09-17T05:41:32.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.547+0000 7f5f8be00640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f740759c0 0x7f5f74077e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:32.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.547+0000 7f5f8be00640 1 -- 172.21.15.122:0/3001827545 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f5f8412cf80 con 0x7f5f940cbd80 2024-09-17T05:41:32.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.551+0000 7f5f8be00640 1 -- 172.21.15.122:0/3001827545 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5f940ae1b0 con 0x7f5f940cbd80 2024-09-17T05:41:32.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.567+0000 7f5f93400640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f740759c0 0x7f5f74077e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:32.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.575+0000 7f5f93400640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f740759c0 0x7f5f74077e80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5f9415e550 tx=0x7f5f80006c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:32.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:32.947+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone40724", "target": ["mon-mgr", ""]}) -- 0x7f5f9415e580 con 0x7f5f740759c0 2024-09-17T05:41:33.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f8be00640 1 -- 172.21.15.122:0/3001827545 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5f9415e580 con 0x7f5f740759c0 2024-09-17T05:41:33.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f740759c0 msgr2=0x7f5f74077e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:33.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f740759c0 0x7f5f74077e80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5f9415e550 tx=0x7f5f80006c00 comp rx=0 tx=0).stop 2024-09-17T05:41:33.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f940cbd80 msgr2=0x7f5f940c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:33.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f940cbd80 0x7f5f940c5b60 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f5f8401af70 tx=0x7f5f840042f0 comp rx=0 tx=0).stop 2024-09-17T05:41:33.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 shutdown_connections 2024-09-17T05:41:33.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5f740759c0 0x7f5f74077e80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:33.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f940b4b10 0x7f5f9415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:33.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f9415bb90 0x7f5f94159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:33.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 --2- 172.21.15.122:0/3001827545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f940cbd80 0x7f5f940c5b60 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:33.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 >> 172.21.15.122:0/3001827545 conn(0x7f5f940cfa00 msgr2=0x7f5f940af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:33.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.719+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 shutdown_connections 2024-09-17T05:41:33.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:33.723+0000 7f5f99400640 1 -- 172.21.15.122:0/3001827545 wait complete. 2024-09-17T05:41:33.761 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone12085 2024-09-17T05:41:34.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.055+0000 7f2229e00640 1 -- 172.21.15.122:0/1674045490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2224155240 msgr2=0x7f2224068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:34.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.055+0000 7f2229e00640 1 --2- 172.21.15.122:0/1674045490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2224155240 0x7f2224068960 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f2210009840 tx=0x7f221001c820 comp rx=0 tx=0).stop 2024-09-17T05:41:34.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.055+0000 7f2229e00640 1 -- 172.21.15.122:0/1674045490 shutdown_connections 2024-09-17T05:41:34.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.055+0000 7f2229e00640 1 --2- 172.21.15.122:0/1674045490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2224155240 0x7f2224068960 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:34.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.055+0000 7f2229e00640 1 -- 172.21.15.122:0/1674045490 >> 172.21.15.122:0/1674045490 conn(0x7f22240cced0 msgr2=0x7f22240cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:34.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.055+0000 7f2229e00640 1 -- 172.21.15.122:0/1674045490 shutdown_connections 2024-09-17T05:41:34.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.055+0000 7f2229e00640 1 -- 172.21.15.122:0/1674045490 wait complete. 2024-09-17T05:41:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 Processor -- start 2024-09-17T05:41:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 -- start start 2024-09-17T05:41:34.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22240cf0d0 0x7f2224150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:34.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222415f000 0x7f2224159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:34.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22241598c0 0x7f222414c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:34.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2224168de0 con 0x7f22240cf0d0 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2224168c60 con 0x7f22241598c0 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2224168f60 con 0x7f222415f000 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22240cf0d0 0x7f2224150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22240cf0d0 0x7f2224150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56354/0 (socket says 172.21.15.122:56354) 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 -- 172.21.15.122:0/3235210847 learned_addr learned my addr 172.21.15.122:0/3235210847 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 -- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222415f000 msgr2=0x7f2224159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222415f000 0x7f2224159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 -- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22241598c0 msgr2=0x7f222414c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22241598c0 0x7f222414c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 -- 172.21.15.122:0/3235210847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222414d480 con 0x7f22240cf0d0 2024-09-17T05:41:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2228a00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22240cf0d0 0x7f2224150bc0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f221400c490 tx=0x7f2214001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2220c00640 1 -- 172.21.15.122:0/3235210847 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214017070 con 0x7f22240cf0d0 2024-09-17T05:41:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2220c00640 1 -- 172.21.15.122:0/3235210847 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2214003600 con 0x7f22240cf0d0 2024-09-17T05:41:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222414dca0 con 0x7f22240cf0d0 2024-09-17T05:41:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2220c00640 1 -- 172.21.15.122:0/3235210847 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214003a50 con 0x7f22240cf0d0 2024-09-17T05:41:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.059+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222414e260 con 0x7f22240cf0d0 2024-09-17T05:41:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.063+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2224068960 con 0x7f22240cf0d0 2024-09-17T05:41:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.063+0000 7f2220c00640 1 -- 172.21.15.122:0/3235210847 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f221401a070 con 0x7f22240cf0d0 2024-09-17T05:41:34.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.067+0000 7f2220c00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2204075960 0x7f2204077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:34.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.067+0000 7f2220c00640 1 -- 172.21.15.122:0/3235210847 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f2214092820 con 0x7f22240cf0d0 2024-09-17T05:41:34.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.071+0000 7f2223e00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2204075960 0x7f2204077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:34.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.075+0000 7f2220c00640 1 -- 172.21.15.122:0/3235210847 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f221405f3d0 con 0x7f22240cf0d0 2024-09-17T05:41:34.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.087+0000 7f2223e00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2204075960 0x7f2204077e20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f22100af000 tx=0x7f22100195c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:34.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:34.427+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone12085", "target": ["mon-mgr", ""]}) -- 0x7f222414da40 con 0x7f2204075960 2024-09-17T05:41:34.430 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:41:34.434 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:41:35.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.091+0000 7f2220c00640 1 -- 172.21.15.122:0/3235210847 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f22100040c0 con 0x7f2204075960 2024-09-17T05:41:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2204075960 msgr2=0x7f2204077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2204075960 0x7f2204077e20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f22100af000 tx=0x7f22100195c0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22240cf0d0 msgr2=0x7f2224150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22240cf0d0 0x7f2224150bc0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f221400c490 tx=0x7f2214001e80 comp rx=0 tx=0).stop 2024-09-17T05:41:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 shutdown_connections 2024-09-17T05:41:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2204075960 0x7f2204077e20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22241598c0 0x7f222414c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222415f000 0x7f2224159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 --2- 172.21.15.122:0/3235210847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22240cf0d0 0x7f2224150bc0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 >> 172.21.15.122:0/3235210847 conn(0x7f22240cced0 msgr2=0x7f222414fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 shutdown_connections 2024-09-17T05:41:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.095+0000 7f2229e00640 1 -- 172.21.15.122:0/3235210847 wait complete. 2024-09-17T05:41:35.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone46056 2024-09-17T05:41:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.327+0000 7fe048e00640 1 -- 172.21.15.122:0/118129357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0441605d0 msgr2=0x7fe0440d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.327+0000 7fe048e00640 1 --2- 172.21.15.122:0/118129357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0441605d0 0x7fe0440d0690 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fe034009900 tx=0x7fe03401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.327+0000 7fe048e00640 1 -- 172.21.15.122:0/118129357 shutdown_connections 2024-09-17T05:41:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.327+0000 7fe048e00640 1 --2- 172.21.15.122:0/118129357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0440afb10 0x7fe0440affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.327+0000 7fe048e00640 1 --2- 172.21.15.122:0/118129357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0441605d0 0x7fe0440d0690 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.327+0000 7fe048e00640 1 -- 172.21.15.122:0/118129357 >> 172.21.15.122:0/118129357 conn(0x7fe0440d1dd0 msgr2=0x7fe0440d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.327+0000 7fe048e00640 1 -- 172.21.15.122:0/118129357 shutdown_connections 2024-09-17T05:41:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.327+0000 7fe048e00640 1 -- 172.21.15.122:0/118129357 wait complete. 2024-09-17T05:41:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.331+0000 7fe048e00640 1 Processor -- start 2024-09-17T05:41:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.331+0000 7fe048e00640 1 -- start start 2024-09-17T05:41:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.331+0000 7fe048e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0440afb10 0x7fe044155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe048e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0440b1650 0x7fe044071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe048e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0441605d0 0x7fe0440b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe048e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe044169540 con 0x7fe0440b1650 2024-09-17T05:41:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe048e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe0440b1430 con 0x7fe0441605d0 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe048e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe0441696c0 con 0x7fe0440afb10 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0440b1650 0x7fe044071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0440b1650 0x7fe044071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56372/0 (socket says 172.21.15.122:56372) 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 -- 172.21.15.122:0/2793068519 learned_addr learned my addr 172.21.15.122:0/2793068519 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe041e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0440afb10 0x7fe044155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 -- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0440afb10 msgr2=0x7fe044155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0440afb10 0x7fe044155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 -- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0441605d0 msgr2=0x7fe0440b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0441605d0 0x7fe0440b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 -- 172.21.15.122:0/2793068519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0440718a0 con 0x7fe0440b1650 2024-09-17T05:41:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03be00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0440b1650 0x7fe044071360 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c490 tx=0x7fe02c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03a000640 1 -- 172.21.15.122:0/2793068519 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c017070 con 0x7fe0440b1650 2024-09-17T05:41:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03a000640 1 -- 172.21.15.122:0/2793068519 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe02c003600 con 0x7fe0440b1650 2024-09-17T05:41:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0440b6ae0 con 0x7fe0440b1650 2024-09-17T05:41:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe03a000640 1 -- 172.21.15.122:0/2793068519 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c003a90 con 0x7fe0440b1650 2024-09-17T05:41:35.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe044071f10 con 0x7fe0440b1650 2024-09-17T05:41:35.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.335+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0440affc0 con 0x7fe0440b1650 2024-09-17T05:41:35.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.339+0000 7fe03a000640 1 -- 172.21.15.122:0/2793068519 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7fe02c01a070 con 0x7fe0440b1650 2024-09-17T05:41:35.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.339+0000 7fe03a000640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe020075960 0x7fe020077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:35.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.339+0000 7fe03a000640 1 -- 172.21.15.122:0/2793068519 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fe02c0928e0 con 0x7fe0440b1650 2024-09-17T05:41:35.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.339+0000 7fe041e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe020075960 0x7fe020077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:35.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.343+0000 7fe041e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe020075960 0x7fe020077e20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe0340af000 tx=0x7fe03401b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:35.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.347+0000 7fe03a000640 1 -- 172.21.15.122:0/2793068519 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe0440718a0 con 0x7fe0440b1650 2024-09-17T05:41:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.655+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone46056", "target": ["mon-mgr", ""]}) -- 0x7fe0440cc1a0 con 0x7fe020075960 2024-09-17T05:41:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.815+0000 7fe03a000640 1 -- 172.21.15.122:0/2793068519 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe0440cc1a0 con 0x7fe020075960 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe020075960 msgr2=0x7fe020077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe020075960 0x7fe020077e20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe0340af000 tx=0x7fe03401b120 comp rx=0 tx=0).stop 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0440b1650 msgr2=0x7fe044071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0440b1650 0x7fe044071360 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c490 tx=0x7fe02c001e80 comp rx=0 tx=0).stop 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 shutdown_connections 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe020075960 0x7fe020077e20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0441605d0 0x7fe0440b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0440b1650 0x7fe044071360 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 --2- 172.21.15.122:0/2793068519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0440afb10 0x7fe044155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 >> 172.21.15.122:0/2793068519 conn(0x7fe0440d1dd0 msgr2=0x7fe0440b6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 shutdown_connections 2024-09-17T05:41:35.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.819+0000 7fe048e00640 1 -- 172.21.15.122:0/2793068519 wait complete. 2024-09-17T05:41:35.833 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone51580 2024-09-17T05:41:35.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 -- 172.21.15.122:0/1655224627 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0d2950 msgr2=0x7f55ec0d09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 --2- 172.21.15.122:0/1655224627 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0d2950 0x7f55ec0d09d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009bc0 tx=0x7f55e801cae0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 -- 172.21.15.122:0/1655224627 shutdown_connections 2024-09-17T05:41:35.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 --2- 172.21.15.122:0/1655224627 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0d2950 0x7f55ec0d09d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 --2- 172.21.15.122:0/1655224627 >> v2:172.21.15.173:3300/0 conn(0x7f55ec161230 0x7f55ec0d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 -- 172.21.15.122:0/1655224627 >> 172.21.15.122:0/1655224627 conn(0x7f55ec09d0c0 msgr2=0x7f55ec0d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:35.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 -- 172.21.15.122:0/1655224627 shutdown_connections 2024-09-17T05:41:35.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 -- 172.21.15.122:0/1655224627 wait complete. 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.987+0000 7f55f3200640 1 Processor -- start 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 -- start start 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0b0a90 0x7f55ec0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55ec0d2950 0x7f55ec0b8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55ec161230 0x7f55ec1f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec16af60 con 0x7f55ec0d2950 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec0d4c40 con 0x7f55ec161230 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f55ec16b0e0 con 0x7f55ec0b0a90 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55ec0d2950 0x7f55ec0b8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0b0a90 0x7f55ec0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:35.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f2800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55ec161230 0x7f55ec1f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:35.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0b0a90 0x7f55ec0cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:59320/0 (socket says 172.21.15.122:59320) 2024-09-17T05:41:35.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 -- 172.21.15.122:0/119951487 learned_addr learned my addr 172.21.15.122:0/119951487 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:35.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1400640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55ec0d2950 0x7f55ec0b8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56388/0 (socket says 172.21.15.122:56388) 2024-09-17T05:41:35.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 -- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55ec161230 msgr2=0x7f55ec1f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55ec161230 0x7f55ec1f79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 -- 172.21.15.122:0/119951487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55ec0d2950 msgr2=0x7f55ec0b8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:35.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55ec0d2950 0x7f55ec0b8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:35.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 -- 172.21.15.122:0/119951487 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec1f83a0 con 0x7f55ec0b0a90 2024-09-17T05:41:35.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f1e00640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0b0a90 0x7f55ec0cb870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f55e000e7a0 tx=0x7f55e000ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:35.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55e6000640 1 -- 172.21.15.122:0/119951487 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e000ccc0 con 0x7f55ec0b0a90 2024-09-17T05:41:35.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55e6000640 1 -- 172.21.15.122:0/119951487 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e0010430 con 0x7f55ec0b0a90 2024-09-17T05:41:35.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec1f8680 con 0x7f55ec0b0a90 2024-09-17T05:41:35.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55e6000640 1 -- 172.21.15.122:0/119951487 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0010860 con 0x7f55ec0b0a90 2024-09-17T05:41:35.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec0cb350 con 0x7f55ec0b0a90 2024-09-17T05:41:35.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.991+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ec0d09d0 con 0x7f55ec0b0a90 2024-09-17T05:41:35.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.995+0000 7f55e6000640 1 -- 172.21.15.122:0/119951487 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f55e0010a00 con 0x7f55ec0b0a90 2024-09-17T05:41:35.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.995+0000 7f55e6000640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d0075960 0x7f55d0077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:35.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.995+0000 7f55e6000640 1 -- 172.21.15.122:0/119951487 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f55e0093f00 con 0x7f55ec0b0a90 2024-09-17T05:41:35.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.995+0000 7f55f1400640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d0075960 0x7f55d0077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:35.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.995+0000 7f55f1400640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d0075960 0x7f55d0077e20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0ce210 tx=0x7f55dc001440 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:35.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:35.999+0000 7f55e6000640 1 -- 172.21.15.122:0/119951487 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f55e00603c0 con 0x7f55ec0b0a90 2024-09-17T05:41:36.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.207+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone51580", "target": ["mon-mgr", ""]}) -- 0x7f55ec069e30 con 0x7f55d0075960 2024-09-17T05:41:36.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.211+0000 7f55e6000640 1 -- 172.21.15.122:0/119951487 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f55dc00f680 con 0x7f55d0075960 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:41:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:41:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:41:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:36.209 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d0075960 msgr2=0x7f55d0077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d0075960 0x7f55d0077e20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0ce210 tx=0x7f55dc001440 comp rx=0 tx=0).stop 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0b0a90 msgr2=0x7f55ec0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0b0a90 0x7f55ec0cb870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f55e000e7a0 tx=0x7f55e000ec60 comp rx=0 tx=0).stop 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 shutdown_connections 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f55d0075960 0x7f55d0077e20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55ec161230 0x7f55ec1f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55ec0d2950 0x7f55ec0b8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 --2- 172.21.15.122:0/119951487 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55ec0b0a90 0x7f55ec0cb870 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 >> 172.21.15.122:0/119951487 conn(0x7f55ec09d0c0 msgr2=0x7f55ec15a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:36.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 shutdown_connections 2024-09-17T05:41:36.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.215+0000 7f55f3200640 1 -- 172.21.15.122:0/119951487 wait complete. 2024-09-17T05:41:36.229 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:41:36.229 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone77831 2024-09-17T05:41:36.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- 172.21.15.122:0/4254569941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80b2cf0 msgr2=0x7f32c8160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 --2- 172.21.15.122:0/4254569941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80b2cf0 0x7f32c8160330 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f32c8073b00 tx=0x7f32c001f580 comp rx=0 tx=0).stop 2024-09-17T05:41:36.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- 172.21.15.122:0/4254569941 shutdown_connections 2024-09-17T05:41:36.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 --2- 172.21.15.122:0/4254569941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80b2cf0 0x7f32c8160330 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 --2- 172.21.15.122:0/4254569941 >> v2:172.21.15.173:3300/0 conn(0x7f32c80cfe90 0x7f32c80d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- 172.21.15.122:0/4254569941 >> 172.21.15.122:0/4254569941 conn(0x7f32c80d1dc0 msgr2=0x7f32c80d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:36.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- 172.21.15.122:0/4254569941 shutdown_connections 2024-09-17T05:41:36.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- 172.21.15.122:0/4254569941 wait complete. 2024-09-17T05:41:36.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 Processor -- start 2024-09-17T05:41:36.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- start start 2024-09-17T05:41:36.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32c80b2cf0 0x7f32c81f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:36.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32c80cfe90 0x7f32c80c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:36.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80d07b0 0x7f32c80c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:36.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80afb30 con 0x7f32c80d07b0 2024-09-17T05:41:36.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80af9b0 con 0x7f32c80cfe90 2024-09-17T05:41:36.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32ccc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f32c80afcb0 con 0x7f32c80b2cf0 2024-09-17T05:41:36.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80d07b0 0x7f32c80c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:36.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80d07b0 0x7f32c80c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56406/0 (socket says 172.21.15.122:56406) 2024-09-17T05:41:36.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 -- 172.21.15.122:0/3777316291 learned_addr learned my addr 172.21.15.122:0/3777316291 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:36.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 -- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32c80b2cf0 msgr2=0x7f32c81f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32c80b2cf0 0x7f32c81f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 -- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32c80cfe90 msgr2=0x7f32c80c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32c80cfe90 0x7f32c80c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 -- 172.21.15.122:0/3777316291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c8149cb0 con 0x7f32c80d07b0 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32c6800640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80d07b0 0x7f32c80c93d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f32c001dc30 tx=0x7f32c00093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.383+0000 7f32be000640 1 -- 172.21.15.122:0/3777316291 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00bf070 con 0x7f32c80d07b0 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32be000640 1 -- 172.21.15.122:0/3777316291 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32c0004b40 con 0x7f32c80d07b0 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32be000640 1 -- 172.21.15.122:0/3777316291 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00b5040 con 0x7f32c80d07b0 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c80caf20 con 0x7f32c80d07b0 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c80cb4e0 con 0x7f32c80d07b0 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c8146a30 con 0x7f32c80d07b0 2024-09-17T05:41:36.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32be000640 1 -- 172.21.15.122:0/3777316291 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f32c00b0830 con 0x7f32c80d07b0 2024-09-17T05:41:36.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32be000640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f32a8075990 0x7f32a8077e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:36.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32be000640 1 -- 172.21.15.122:0/3777316291 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f32c01307b0 con 0x7f32c80d07b0 2024-09-17T05:41:36.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.387+0000 7f32c5e00640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f32a8075990 0x7f32a8077e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:36.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.391+0000 7f32c5e00640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f32a8075990 0x7f32a8077e50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009730 tx=0x7f32b8006ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:36.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.391+0000 7f32be000640 1 -- 172.21.15.122:0/3777316291 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f32c8146a30 con 0x7f32c80d07b0 2024-09-17T05:41:36.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.591+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone77831", "target": ["mon-mgr", ""]}) -- 0x7f32c80d0270 con 0x7f32a8075990 2024-09-17T05:41:36.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.599+0000 7f32be000640 1 -- 172.21.15.122:0/3777316291 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f32c80d0270 con 0x7f32a8075990 2024-09-17T05:41:36.595 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:41:36.595 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:41:36.595 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:36.596 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:41:36.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f32a8075990 msgr2=0x7f32a8077e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f32a8075990 0x7f32a8077e50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009730 tx=0x7f32b8006ca0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80d07b0 msgr2=0x7f32c80c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80d07b0 0x7f32c80c93d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f32c001dc30 tx=0x7f32c00093b0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 shutdown_connections 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f32a8075990 0x7f32a8077e50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32c80d07b0 0x7f32c80c93d0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32c80cfe90 0x7f32c80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 --2- 172.21.15.122:0/3777316291 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32c80b2cf0 0x7f32c81f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 >> 172.21.15.122:0/3777316291 conn(0x7f32c80d1dc0 msgr2=0x7f32c80b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 shutdown_connections 2024-09-17T05:41:36.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.603+0000 7f32ccc00640 1 -- 172.21.15.122:0/3777316291 wait complete. 2024-09-17T05:41:36.617 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:41:36.617 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone11792 2024-09-17T05:41:36.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.767+0000 7f0408800640 1 -- 172.21.15.122:0/3807778878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0400164ab0 msgr2=0x7f0400166ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.767+0000 7f0408800640 1 --2- 172.21.15.122:0/3807778878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0400164ab0 0x7f0400166ea0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009bf0 tx=0x7f03fc01f2f0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.767+0000 7f0408800640 1 -- 172.21.15.122:0/3807778878 shutdown_connections 2024-09-17T05:41:36.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.767+0000 7f0408800640 1 --2- 172.21.15.122:0/3807778878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0400164ab0 0x7f0400166ea0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.767+0000 7f0408800640 1 -- 172.21.15.122:0/3807778878 >> 172.21.15.122:0/3807778878 conn(0x7f04000b18f0 msgr2=0x7f04000b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:36.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.767+0000 7f0408800640 1 -- 172.21.15.122:0/3807778878 shutdown_connections 2024-09-17T05:41:36.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.767+0000 7f0408800640 1 -- 172.21.15.122:0/3807778878 wait complete. 2024-09-17T05:41:36.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 Processor -- start 2024-09-17T05:41:36.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 -- start start 2024-09-17T05:41:36.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f040015bfa0 0x7f0400148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f040015c3e0 0x7f04000b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0400164ab0 0x7f0400157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04001673e0 con 0x7f0400164ab0 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f040006f8c0 con 0x7f040015bfa0 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0400167560 con 0x7f040015c3e0 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0400164ab0 0x7f0400157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0400164ab0 0x7f0400157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56422/0 (socket says 172.21.15.122:56422) 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 -- 172.21.15.122:0/1110164818 learned_addr learned my addr 172.21.15.122:0/1110164818 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0405e00640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f040015bfa0 0x7f0400148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0405400640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f040015c3e0 0x7f04000b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:36.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 -- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f040015c3e0 msgr2=0x7f04000b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f040015c3e0 0x7f04000b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 -- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f040015bfa0 msgr2=0x7f0400148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f040015bfa0 0x7f0400148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 -- 172.21.15.122:0/1110164818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04000b2e90 con 0x7f0400164ab0 2024-09-17T05:41:36.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0406800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0400164ab0 0x7f0400157180 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f03fc01d9a0 tx=0x7f03fc005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:36.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f03fa000640 1 -- 172.21.15.122:0/1110164818 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0b4040 con 0x7f0400164ab0 2024-09-17T05:41:36.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f03fa000640 1 -- 172.21.15.122:0/1110164818 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03fc004b40 con 0x7f0400164ab0 2024-09-17T05:41:36.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04000b3170 con 0x7f0400164ab0 2024-09-17T05:41:36.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f03fa000640 1 -- 172.21.15.122:0/1110164818 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0af6a0 con 0x7f0400164ab0 2024-09-17T05:41:36.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.771+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04000b3730 con 0x7f0400164ab0 2024-09-17T05:41:36.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.775+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0400069410 con 0x7f0400164ab0 2024-09-17T05:41:36.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.775+0000 7f03fa000640 1 -- 172.21.15.122:0/1110164818 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f03fc0c1020 con 0x7f0400164ab0 2024-09-17T05:41:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.775+0000 7f03fa000640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03e4075960 0x7f03e4077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:36.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.775+0000 7f03fa000640 1 -- 172.21.15.122:0/1110164818 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f03fc130cd0 con 0x7f0400164ab0 2024-09-17T05:41:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.775+0000 7f0405e00640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03e4075960 0x7f03e4077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:36.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.779+0000 7f0405e00640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03e4075960 0x7f03e4077e20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f03f4001820 tx=0x7f03f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:36.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.779+0000 7f03fa000640 1 -- 172.21.15.122:0/1110164818 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f03fc018e50 con 0x7f0400164ab0 2024-09-17T05:41:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.983+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone11792", "target": ["mon-mgr", ""]}) -- 0x7f04000cd000 con 0x7f03e4075960 2024-09-17T05:41:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f03fa000640 1 -- 172.21.15.122:0/1110164818 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f04000cd000 con 0x7f03e4075960 2024-09-17T05:41:36.984 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:41:36.984 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:36.985 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:41:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03e4075960 msgr2=0x7f03e4077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03e4075960 0x7f03e4077e20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f03f4001820 tx=0x7f03f4009040 comp rx=0 tx=0).stop 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0400164ab0 msgr2=0x7f0400157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0400164ab0 0x7f0400157180 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f03fc01d9a0 tx=0x7f03fc005aa0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 shutdown_connections 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f03e4075960 0x7f03e4077e20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0400164ab0 0x7f0400157180 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f040015c3e0 0x7f04000b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 --2- 172.21.15.122:0/1110164818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f040015bfa0 0x7f0400148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 >> 172.21.15.122:0/1110164818 conn(0x7f04000b18f0 msgr2=0x7f040015edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 shutdown_connections 2024-09-17T05:41:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:36.991+0000 7f0408800640 1 -- 172.21.15.122:0/1110164818 wait complete. 2024-09-17T05:41:37.009 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:41:37.010 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone81883 2024-09-17T05:41:37.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.159+0000 7f63bc600640 1 -- 172.21.15.122:0/929220690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40b49b0 msgr2=0x7f63b415ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:37.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.159+0000 7f63bc600640 1 --2- 172.21.15.122:0/929220690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40b49b0 0x7f63b415ea30 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f63b415e160 tx=0x7f63a001dac0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.159+0000 7f63bc600640 1 -- 172.21.15.122:0/929220690 shutdown_connections 2024-09-17T05:41:37.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.159+0000 7f63bc600640 1 --2- 172.21.15.122:0/929220690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40b49b0 0x7f63b415ea30 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.159+0000 7f63bc600640 1 -- 172.21.15.122:0/929220690 >> 172.21.15.122:0/929220690 conn(0x7f63b40b18f0 msgr2=0x7f63b40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:37.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.159+0000 7f63bc600640 1 -- 172.21.15.122:0/929220690 shutdown_connections 2024-09-17T05:41:37.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.159+0000 7f63bc600640 1 -- 172.21.15.122:0/929220690 wait complete. 2024-09-17T05:41:37.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 Processor -- start 2024-09-17T05:41:37.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 -- start start 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40703d0 0x7f63b409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63b40b49b0 0x7f63b41fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b415a4d0 0x7f63b4158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40cd080 con 0x7f63b40703d0 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40ccf00 con 0x7f63b415a4d0 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f63b4168180 con 0x7f63b40b49b0 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40703d0 0x7f63b409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40703d0 0x7f63b409aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56438/0 (socket says 172.21.15.122:56438) 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 -- 172.21.15.122:0/2795353414 learned_addr learned my addr 172.21.15.122:0/2795353414 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:37.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63ba600640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b415a4d0 0x7f63b4158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9200640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63b40b49b0 0x7f63b41fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 -- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63b40b49b0 msgr2=0x7f63b41fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63b40b49b0 0x7f63b41fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 -- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b415a4d0 msgr2=0x7f63b4158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b415a4d0 0x7f63b4158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 -- 172.21.15.122:0/2795353414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b409b500 con 0x7f63b40703d0 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b9c00640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40703d0 0x7f63b409aa10 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f63a800c3f0 tx=0x7f63a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b1600640 1 -- 172.21.15.122:0/2795353414 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8017070 con 0x7f63b40703d0 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b1600640 1 -- 172.21.15.122:0/2795353414 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63a80053c0 con 0x7f63b40703d0 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63b1600640 1 -- 172.21.15.122:0/2795353414 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8003860 con 0x7f63b40703d0 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b409b7e0 con 0x7f63b40703d0 2024-09-17T05:41:37.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b41ed260 con 0x7f63b40703d0 2024-09-17T05:41:37.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.163+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b4069410 con 0x7f63b40703d0 2024-09-17T05:41:37.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.167+0000 7f63b1600640 1 -- 172.21.15.122:0/2795353414 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f63a801a070 con 0x7f63b40703d0 2024-09-17T05:41:37.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.167+0000 7f63b1600640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6394075960 0x7f6394077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:37.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.167+0000 7f63b1600640 1 -- 172.21.15.122:0/2795353414 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f63a80928c0 con 0x7f63b40703d0 2024-09-17T05:41:37.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.167+0000 7f63b9200640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6394075960 0x7f6394077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:37.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.167+0000 7f63b9200640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6394075960 0x7f6394077e20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f63a00b0000 tx=0x7f63a001c300 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:37.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.171+0000 7f63b1600640 1 -- 172.21.15.122:0/2795353414 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f63a805f3d0 con 0x7f63b40703d0 2024-09-17T05:41:37.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.379+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone81883", "target": ["mon-mgr", ""]}) -- 0x7f63b409b0f0 con 0x7f6394075960 2024-09-17T05:41:37.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.391+0000 7f63b1600640 1 -- 172.21.15.122:0/2795353414 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f63b409b0f0 con 0x7f6394075960 2024-09-17T05:41:37.387 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:37.388 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:37.389 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:41:37.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6394075960 msgr2=0x7f6394077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:37.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6394075960 0x7f6394077e20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f63a00b0000 tx=0x7f63a001c300 comp rx=0 tx=0).stop 2024-09-17T05:41:37.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40703d0 msgr2=0x7f63b409aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:37.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40703d0 0x7f63b409aa10 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f63a800c3f0 tx=0x7f63a800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 shutdown_connections 2024-09-17T05:41:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6394075960 0x7f6394077e20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63b415a4d0 0x7f63b4158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63b40b49b0 0x7f63b41fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 --2- 172.21.15.122:0/2795353414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63b40703d0 0x7f63b409aa10 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 >> 172.21.15.122:0/2795353414 conn(0x7f63b40b18f0 msgr2=0x7f63b40b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 shutdown_connections 2024-09-17T05:41:37.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.395+0000 7f63bc600640 1 -- 172.21.15.122:0/2795353414 wait complete. 2024-09-17T05:41:37.408 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:41:37.408 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone40724 2024-09-17T05:41:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 -- 172.21.15.122:0/1232934890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 msgr2=0x7f536c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 --2- 172.21.15.122:0/1232934890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 0x7f536c004c90 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f5370009b30 tx=0x7f537001d230 comp rx=0 tx=0).stop 2024-09-17T05:41:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 -- 172.21.15.122:0/1232934890 shutdown_connections 2024-09-17T05:41:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 --2- 172.21.15.122:0/1232934890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 0x7f536c004c90 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 -- 172.21.15.122:0/1232934890 >> 172.21.15.122:0/1232934890 conn(0x7f538809aa10 msgr2=0x7f538809ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:37.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 -- 172.21.15.122:0/1232934890 shutdown_connections 2024-09-17T05:41:37.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 -- 172.21.15.122:0/1232934890 wait complete. 2024-09-17T05:41:37.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 Processor -- start 2024-09-17T05:41:37.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.559+0000 7f5387e00640 1 -- start start 2024-09-17T05:41:37.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 0x7f538815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:37.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5388158170 0x7f5388152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:37.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53880718c0 0x7f5388071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:37.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53880d03c0 con 0x7f536c004810 2024-09-17T05:41:37.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53880d0240 con 0x7f53880718c0 2024-09-17T05:41:37.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f53880d0540 con 0x7f5388158170 2024-09-17T05:41:37.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 0x7f538815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 0x7f538815e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56458/0 (socket says 172.21.15.122:56458) 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 -- 172.21.15.122:0/532349675 learned_addr learned my addr 172.21.15.122:0/532349675 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f537e000640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5388158170 0x7f5388152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387400640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53880718c0 0x7f5388071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 -- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5388158170 msgr2=0x7f5388152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5388158170 0x7f5388152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 -- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53880718c0 msgr2=0x7f5388071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53880718c0 0x7f5388071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 -- 172.21.15.122:0/532349675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53881f8270 con 0x7f536c004810 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387400640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53880718c0 0x7f5388071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5386a00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 0x7f538815e810 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f537800e310 tx=0x7f537800e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:37.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f537f400640 1 -- 172.21.15.122:0/532349675 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378012850 con 0x7f536c004810 2024-09-17T05:41:37.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f537f400640 1 -- 172.21.15.122:0/532349675 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5378012b00 con 0x7f536c004810 2024-09-17T05:41:37.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53881f8510 con 0x7f536c004810 2024-09-17T05:41:37.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f537f400640 1 -- 172.21.15.122:0/532349675 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537801a400 con 0x7f536c004810 2024-09-17T05:41:37.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53881f8ad0 con 0x7f536c004810 2024-09-17T05:41:37.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.563+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53880c8ac0 con 0x7f536c004810 2024-09-17T05:41:37.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.567+0000 7f537f400640 1 -- 172.21.15.122:0/532349675 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f537800d030 con 0x7f536c004810 2024-09-17T05:41:37.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.567+0000 7f537f400640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5364075990 0x7f5364077e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:37.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.567+0000 7f537f400640 1 -- 172.21.15.122:0/532349675 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f53780967a0 con 0x7f536c004810 2024-09-17T05:41:37.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.567+0000 7f537e000640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5364075990 0x7f5364077e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:37.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.567+0000 7f537e000640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5364075990 0x7f5364077e50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f53700ae000 tx=0x7f5370019fd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:37.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.571+0000 7f537f400640 1 -- 172.21.15.122:0/532349675 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f53780a0070 con 0x7f536c004810 2024-09-17T05:41:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:37.771+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone40724", "target": ["mon-mgr", ""]}) -- 0x7f5388069e30 con 0x7f5364075990 2024-09-17T05:41:40.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f537f400640 1 -- 172.21.15.122:0/532349675 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f5370004140 con 0x7f5364075990 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:41:40.669 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:41:40.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:40.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:40.670 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:41:40.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5364075990 msgr2=0x7f5364077e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5364075990 0x7f5364077e50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f53700ae000 tx=0x7f5370019fd0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 msgr2=0x7f538815e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 0x7f538815e810 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f537800e310 tx=0x7f537800e7d0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 shutdown_connections 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f5364075990 0x7f5364077e50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53880718c0 0x7f5388071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5388158170 0x7f5388152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 --2- 172.21.15.122:0/532349675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f536c004810 0x7f538815e810 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 >> 172.21.15.122:0/532349675 conn(0x7f538809aa10 msgr2=0x7f53880707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 shutdown_connections 2024-09-17T05:41:40.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.675+0000 7f5387e00640 1 -- 172.21.15.122:0/532349675 wait complete. 2024-09-17T05:41:40.694 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:41:40.695 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone12085 2024-09-17T05:41:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- 172.21.15.122:0/3566684874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f815e260 msgr2=0x7f23f80b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:40.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 --2- 172.21.15.122:0/3566684874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f815e260 0x7f23f80b2bf0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f23f80af700 tx=0x7f23ec01c570 comp rx=0 tx=0).stop 2024-09-17T05:41:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- 172.21.15.122:0/3566684874 shutdown_connections 2024-09-17T05:41:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 --2- 172.21.15.122:0/3566684874 >> v2:172.21.15.173:3300/0 conn(0x7f23f80b4cf0 0x7f23f815e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 --2- 172.21.15.122:0/3566684874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f80cc2a0 0x7f23f80b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 --2- 172.21.15.122:0/3566684874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f815e260 0x7f23f80b2bf0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- 172.21.15.122:0/3566684874 >> 172.21.15.122:0/3566684874 conn(0x7f23f80b18f0 msgr2=0x7f23f80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- 172.21.15.122:0/3566684874 shutdown_connections 2024-09-17T05:41:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- 172.21.15.122:0/3566684874 wait complete. 2024-09-17T05:41:40.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 Processor -- start 2024-09-17T05:41:40.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- start start 2024-09-17T05:41:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f80b4cf0 0x7f23f8152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f815e260 0x7f23f81f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f8153750 0x7f23f81f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23f806fd60 con 0x7f23f8153750 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23f806fbe0 con 0x7f23f815e260 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f2400400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f23f80cdae0 con 0x7f23f80b4cf0 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f23fe400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f8153750 0x7f23f81f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f23fda00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f80b4cf0 0x7f23f8152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f23fd000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f815e260 0x7f23f81f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f23fe400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f8153750 0x7f23f81f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56464/0 (socket says 172.21.15.122:56464) 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.851+0000 7f23fe400640 1 -- 172.21.15.122:0/2833943171 learned_addr learned my addr 172.21.15.122:0/2833943171 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23fe400640 1 -- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f80b4cf0 msgr2=0x7f23f8152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23fe400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f80b4cf0 0x7f23f8152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23fe400640 1 -- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f815e260 msgr2=0x7f23f81f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23fe400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f815e260 0x7f23f81f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23fe400640 1 -- 172.21.15.122:0/2833943171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f81590e0 con 0x7f23f8153750 2024-09-17T05:41:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23fe400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f8153750 0x7f23f81f8460 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f23f000cd10 tx=0x7f23f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23f5600640 1 -- 172.21.15.122:0/2833943171 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000f450 con 0x7f23f8153750 2024-09-17T05:41:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23f5600640 1 -- 172.21.15.122:0/2833943171 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23f000f700 con 0x7f23f8153750 2024-09-17T05:41:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23f5600640 1 -- 172.21.15.122:0/2833943171 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f000fb50 con 0x7f23f8153750 2024-09-17T05:41:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f8159360 con 0x7f23f8153750 2024-09-17T05:41:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f8072ee0 con 0x7f23f8153750 2024-09-17T05:41:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f8069410 con 0x7f23f8153750 2024-09-17T05:41:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.855+0000 7f23f5600640 1 -- 172.21.15.122:0/2833943171 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f23f000fcf0 con 0x7f23f8153750 2024-09-17T05:41:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.859+0000 7f23f5600640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f23d8075960 0x7f23d8077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.859+0000 7f23f5600640 1 -- 172.21.15.122:0/2833943171 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f23f0093f50 con 0x7f23f8153750 2024-09-17T05:41:40.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.859+0000 7f23fda00640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f23d8075960 0x7f23d8077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:40.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.859+0000 7f23fda00640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f23d8075960 0x7f23d8077e20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0ae000 tx=0x7f23ec0b3040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:40.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:40.863+0000 7f23f5600640 1 -- 172.21.15.122:0/2833943171 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f23f81590e0 con 0x7f23f8153750 2024-09-17T05:41:41.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.059+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone12085", "target": ["mon-mgr", ""]}) -- 0x7f23f8159880 con 0x7f23d8075960 2024-09-17T05:41:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f23f5600640 1 -- 172.21.15.122:0/2833943171 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f23ec005140 con 0x7f23d8075960 2024-09-17T05:41:41.069 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:41:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:41.070 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f23d8075960 msgr2=0x7f23d8077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f23d8075960 0x7f23d8077e20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0ae000 tx=0x7f23ec0b3040 comp rx=0 tx=0).stop 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f8153750 msgr2=0x7f23f81f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f8153750 0x7f23f81f8460 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f23f000cd10 tx=0x7f23f000a620 comp rx=0 tx=0).stop 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 shutdown_connections 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f23d8075960 0x7f23d8077e20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f8153750 0x7f23f81f8460 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f815e260 0x7f23f81f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 --2- 172.21.15.122:0/2833943171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f80b4cf0 0x7f23f8152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.075+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 >> 172.21.15.122:0/2833943171 conn(0x7f23f80b18f0 msgr2=0x7f23f8159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.079+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 shutdown_connections 2024-09-17T05:41:41.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.079+0000 7f2400400640 1 -- 172.21.15.122:0/2833943171 wait complete. 2024-09-17T05:41:41.091 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:41:41.091 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone46056 2024-09-17T05:41:41.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 -- 172.21.15.122:0/1143390031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd0157d40 msgr2=0x7f4cd0071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:41.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 --2- 172.21.15.122:0/1143390031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd0157d40 0x7f4cd0071060 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00d0e40 tx=0x7f4cc401c870 comp rx=0 tx=0).stop 2024-09-17T05:41:41.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 -- 172.21.15.122:0/1143390031 shutdown_connections 2024-09-17T05:41:41.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 --2- 172.21.15.122:0/1143390031 >> v2:172.21.15.173:3300/0 conn(0x7f4cd015ea20 0x7f4cd00b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 --2- 172.21.15.122:0/1143390031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cd015dda0 0x7f4cd015e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 --2- 172.21.15.122:0/1143390031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd0157d40 0x7f4cd0071060 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 -- 172.21.15.122:0/1143390031 >> 172.21.15.122:0/1143390031 conn(0x7f4cd006ecd0 msgr2=0x7f4cd006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 -- 172.21.15.122:0/1143390031 shutdown_connections 2024-09-17T05:41:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 -- 172.21.15.122:0/1143390031 wait complete. 2024-09-17T05:41:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 Processor -- start 2024-09-17T05:41:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.243+0000 7f4cd7600640 1 -- start start 2024-09-17T05:41:41.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cd0157d40 0x7f4cd0141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd015dda0 0x7f4cd0144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cd015ea20 0x7f4cd01477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0073220 con 0x7f4cd015dda0 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd00730a0 con 0x7f4cd0157d40 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd00733a0 con 0x7f4cd015ea20 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd015dda0 0x7f4cd0144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd015dda0 0x7f4cd0144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56490/0 (socket says 172.21.15.122:56490) 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 -- 172.21.15.122:0/2126690469 learned_addr learned my addr 172.21.15.122:0/2126690469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd4c00640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cd0157d40 0x7f4cd0141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd5600640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cd015ea20 0x7f4cd01477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 -- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cd015ea20 msgr2=0x7f4cd01477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cd015ea20 0x7f4cd01477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 -- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cd0157d40 msgr2=0x7f4cd0141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cd0157d40 0x7f4cd0141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 -- 172.21.15.122:0/2126690469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd0143540 con 0x7f4cd015dda0 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4ccfe00640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd015dda0 0x7f4cd0144150 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00dbc0 tx=0x7f4cbc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cccc00640 1 -- 172.21.15.122:0/2126690469 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc004020 con 0x7f4cd015dda0 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cccc00640 1 -- 172.21.15.122:0/2126690469 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cbc0042b0 con 0x7f4cd015dda0 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd0141b30 con 0x7f4cd015dda0 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cccc00640 1 -- 172.21.15.122:0/2126690469 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc005280 con 0x7f4cd015dda0 2024-09-17T05:41:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd01420f0 con 0x7f4cd015dda0 2024-09-17T05:41:41.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.247+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd015e210 con 0x7f4cd015dda0 2024-09-17T05:41:41.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.251+0000 7f4cccc00640 1 -- 172.21.15.122:0/2126690469 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f4cbc01d020 con 0x7f4cd015dda0 2024-09-17T05:41:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.251+0000 7f4cccc00640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4cb0075960 0x7f4cb0077e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.251+0000 7f4cccc00640 1 -- 172.21.15.122:0/2126690469 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f4cbc093830 con 0x7f4cd015dda0 2024-09-17T05:41:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.251+0000 7f4cd4c00640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4cb0075960 0x7f4cb0077e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.251+0000 7f4cd4c00640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4cb0075960 0x7f4cb0077e20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40af000 tx=0x7f4cc401b0b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:41.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.255+0000 7f4cccc00640 1 -- 172.21.15.122:0/2126690469 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4cbc05fc70 con 0x7f4cd015dda0 2024-09-17T05:41:41.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:41.459+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone46056", "target": ["mon-mgr", ""]}) -- 0x7f4cd00cf270 con 0x7f4cb0075960 2024-09-17T05:41:44.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.559+0000 7f4cccc00640 1 -- 172.21.15.122:0/2126690469 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f4cd00cf270 con 0x7f4cb0075960 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "status": { 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "canceled", 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "source": { 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "volume": "cephfs", 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "subvolume": "sv1", 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot": "ss1" 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "failure": { 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "errno": "4", 2024-09-17T05:41:44.556 INFO:teuthology.orchestra.run.smithi122.stdout: "error_msg": "user interrupted clone operation" 2024-09-17T05:41:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:44.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:44.557 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4cb0075960 msgr2=0x7f4cb0077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4cb0075960 0x7f4cb0077e20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40af000 tx=0x7f4cc401b0b0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd015dda0 msgr2=0x7f4cd0144150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd015dda0 0x7f4cd0144150 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00dbc0 tx=0x7f4cbc00a620 comp rx=0 tx=0).stop 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 shutdown_connections 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4cb0075960 0x7f4cb0077e20 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cd015ea20 0x7f4cd01477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4cd015dda0 0x7f4cd0144150 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 --2- 172.21.15.122:0/2126690469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cd0157d40 0x7f4cd0141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 >> 172.21.15.122:0/2126690469 conn(0x7f4cd006ecd0 msgr2=0x7f4cd00cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:44.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 shutdown_connections 2024-09-17T05:41:44.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.563+0000 7f4cd7600640 1 -- 172.21.15.122:0/2126690469 wait complete. 2024-09-17T05:41:44.577 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-17T05:41:44.577 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-17T05:41:44.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.723+0000 7fd296800640 1 -- 172.21.15.122:0/3783061383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd278003c70 msgr2=0x7fd278004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:44.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.723+0000 7fd296800640 1 --2- 172.21.15.122:0/3783061383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd278003c70 0x7fd278004070 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fd284009b60 tx=0x7fd28401d270 comp rx=0 tx=0).stop 2024-09-17T05:41:44.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.723+0000 7fd295e00640 1 -- 172.21.15.122:0/3783061383 reap_dead start 2024-09-17T05:41:44.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.723+0000 7fd296800640 1 -- 172.21.15.122:0/3783061383 shutdown_connections 2024-09-17T05:41:44.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.723+0000 7fd296800640 1 -- 172.21.15.122:0/3783061383 >> 172.21.15.122:0/3783061383 conn(0x7fd2900d1c30 msgr2=0x7fd2900d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:44.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.723+0000 7fd296800640 1 -- 172.21.15.122:0/3783061383 shutdown_connections 2024-09-17T05:41:44.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.723+0000 7fd296800640 1 -- 172.21.15.122:0/3783061383 wait complete. 2024-09-17T05:41:44.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 Processor -- start 2024-09-17T05:41:44.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 -- start start 2024-09-17T05:41:44.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd278003c70 0x7fd29015e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:44.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2900b6db0 0x7fd290148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:44.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2900b7230 0x7fd290149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:44.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd290169db0 con 0x7fd2900b6db0 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd290169c30 con 0x7fd2900b7230 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd290169f30 con 0x7fd278003c70 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2900b6db0 0x7fd290148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2900b6db0 0x7fd290148b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48692/0 (socket says 172.21.15.122:48692) 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 -- 172.21.15.122:0/3595085955 learned_addr learned my addr 172.21.15.122:0/3595085955 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd295400640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd278003c70 0x7fd29015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd295e00640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2900b7230 0x7fd290149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 -- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd278003c70 msgr2=0x7fd29015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd278003c70 0x7fd29015e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 -- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2900b7230 msgr2=0x7fd290149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2900b7230 0x7fd290149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 -- 172.21.15.122:0/3595085955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2900b65f0 con 0x7fd2900b6db0 2024-09-17T05:41:44.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd294a00640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2900b6db0 0x7fd290148b10 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fd27c006460 tx=0x7fd27c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:44.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd28d600640 1 -- 172.21.15.122:0/3595085955 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c015070 con 0x7fd2900b6db0 2024-09-17T05:41:44.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd28d600640 1 -- 172.21.15.122:0/3595085955 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd27c011900 con 0x7fd2900b6db0 2024-09-17T05:41:44.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd28d600640 1 -- 172.21.15.122:0/3595085955 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c011d50 con 0x7fd2900b6db0 2024-09-17T05:41:44.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29015e960 con 0x7fd2900b6db0 2024-09-17T05:41:44.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.727+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd290161280 con 0x7fd2900b6db0 2024-09-17T05:41:44.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.731+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2900cce00 con 0x7fd2900b6db0 2024-09-17T05:41:44.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.731+0000 7fd28d600640 1 -- 172.21.15.122:0/3595085955 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7fd27c018070 con 0x7fd2900b6db0 2024-09-17T05:41:44.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.731+0000 7fd28d600640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd270075990 0x7fd270077e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:44.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.731+0000 7fd28d600640 1 -- 172.21.15.122:0/3595085955 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fd27c094680 con 0x7fd2900b6db0 2024-09-17T05:41:44.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.731+0000 7fd295400640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd270075990 0x7fd270077e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:44.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.731+0000 7fd295400640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd270075990 0x7fd270077e50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd2840ae000 tx=0x7fd28401a010 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:44.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.735+0000 7fd28d600640 1 -- 172.21.15.122:0/3595085955 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fd27c060bc0 con 0x7fd2900b6db0 2024-09-17T05:41:44.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd290071c10 con 0x7fd2900b6db0 2024-09-17T05:41:44.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd28d600640 1 -- 172.21.15.122:0/3595085955 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1926 (secure 0 0 0) 0x7fd27c0655f0 con 0x7fd2900b6db0 2024-09-17T05:41:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:44.956 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:41:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: "fsid": "51547930-c98d-41c6-b8ce-f14d7623b707", 2024-09-17T05:41:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: "health": { 2024-09-17T05:41:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "HEALTH_OK", 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "checks": {}, 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "mutes": [] 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "election_epoch": 4, 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum": [ 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: 0, 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: 1, 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: 2 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_names": [ 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "a", 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "b", 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: "c" 2024-09-17T05:41:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "quorum_age": 1025, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "monmap": { 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 1, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "min_mon_release_name": "squid", 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "num_mons": 3 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap": { 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 147, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "num_osds": 8, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "num_up_osds": 8, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_up_since": 1726550693, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "num_in_osds": 8, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_in_since": 1726550682, 2024-09-17T05:41:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: "num_remapped_pgs": 0 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap": { 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "pgs_by_state": [ 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "state_name": "active+clean", 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "count": 97 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pgs": 97, 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "num_pools": 3, 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "num_objects": 4386, 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "data_bytes": 18247371776, 2024-09-17T05:41:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_used": 37660086272, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_avail": 735434027008, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "bytes_total": 773094113280, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "read_bytes_sec": 367579760, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "write_bytes_sec": 370423127, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "read_op_per_sec": 91, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "write_op_per_sec": 105 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "fsmap": { 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 97, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "btime": "2024-09-17T05:40:37:216813+0000", 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 10, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "up": 1, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "in": 1, 2024-09-17T05:41:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1, 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "by_rank": [ 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "filesystem_id": 10, 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "rank": 0, 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "c", 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "up:active", 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 9399 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "up:standby": 3 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "mgrmap": { 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-09-17T05:41:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: "num_standbys": 1, 2024-09-17T05:41:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-09-17T05:41:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-09-17T05:41:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-09-17T05:41:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: "restful" 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "servicemap": { 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 25, 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "modified": "2024-09-17T05:41:32.787308+0000", 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "services": {} 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: "progress_events": {} 2024-09-17T05:41:44.964 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd270075990 msgr2=0x7fd270077e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd270075990 0x7fd270077e50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd2840ae000 tx=0x7fd28401a010 comp rx=0 tx=0).stop 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2900b6db0 msgr2=0x7fd290148b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2900b6db0 0x7fd290148b10 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fd27c006460 tx=0x7fd27c0039d0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 shutdown_connections 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fd270075990 0x7fd270077e50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2900b7230 0x7fd290149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2900b6db0 0x7fd290148b10 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 --2- 172.21.15.122:0/3595085955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd278003c70 0x7fd29015e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 >> 172.21.15.122:0/3595085955 conn(0x7fd2900d1c30 msgr2=0x7fd290168aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 shutdown_connections 2024-09-17T05:41:44.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:44.963+0000 7fd296800640 1 -- 172.21.15.122:0/3595085955 wait complete. 2024-09-17T05:41:44.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:44.978 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone51580/0138f012-94c8-47e9-8930-26f234dfaeb6 2024-09-17T05:41:44.978 DEBUG:teuthology.orchestra.run.smithi122:> ) 2024-09-17T05:41:44.988 INFO:teuthology.orchestra.run.smithi122.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone51580/0138f012-94c8-47e9-8930-26f234dfaeb6': No such file or directory 2024-09-17T05:41:44.988 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T05:41:44.989 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-17T05:41:45.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 -- 172.21.15.122:0/571352295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 msgr2=0x7f18f8004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 --2- 172.21.15.122:0/571352295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 0x7f18f8004040 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f190400af30 tx=0x7f190401d110 comp rx=0 tx=0).stop 2024-09-17T05:41:45.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 -- 172.21.15.122:0/571352295 shutdown_connections 2024-09-17T05:41:45.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 --2- 172.21.15.122:0/571352295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 0x7f18f8004040 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 -- 172.21.15.122:0/571352295 >> 172.21.15.122:0/571352295 conn(0x7f19140b18f0 msgr2=0x7f19140b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:45.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 -- 172.21.15.122:0/571352295 shutdown_connections 2024-09-17T05:41:45.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 -- 172.21.15.122:0/571352295 wait complete. 2024-09-17T05:41:45.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 Processor -- start 2024-09-17T05:41:45.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.179+0000 7f191ba00640 1 -- start start 2024-09-17T05:41:45.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 0x7f19141f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1914156440 0x7f19141568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f191406f640 0x7f191406fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f19140cec20 con 0x7f18f8003c40 2024-09-17T05:41:45.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f19140ceaa0 con 0x7f191406f640 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f19140ceda0 con 0x7f1914156440 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 0x7f19141f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 0x7f19141f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48712/0 (socket says 172.21.15.122:48712) 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 -- 172.21.15.122:0/4131866103 learned_addr learned my addr 172.21.15.122:0/4131866103 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1913e00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1914156440 0x7f19141568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919a00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f191406f640 0x7f191406fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 -- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1914156440 msgr2=0x7f19141568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1914156440 0x7f19141568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 -- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f191406f640 msgr2=0x7f191406fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f191406f640 0x7f191406fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 -- 172.21.15.122:0/4131866103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1914156ee0 con 0x7f18f8003c40 2024-09-17T05:41:45.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919a00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f191406f640 0x7f191406fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:41:45.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1919000640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 0x7f19141f8a60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f19040b0040 tx=0x7f190401b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:45.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1910c00640 1 -- 172.21.15.122:0/4131866103 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040bb070 con 0x7f18f8003c40 2024-09-17T05:41:45.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1910c00640 1 -- 172.21.15.122:0/4131866103 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1904003b80 con 0x7f18f8003c40 2024-09-17T05:41:45.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19140c91c0 con 0x7f18f8003c40 2024-09-17T05:41:45.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f1910c00640 1 -- 172.21.15.122:0/4131866103 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040b2040 con 0x7f18f8003c40 2024-09-17T05:41:45.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19140c9780 con 0x7f18f8003c40 2024-09-17T05:41:45.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.183+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1914069410 con 0x7f18f8003c40 2024-09-17T05:41:45.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.187+0000 7f1910c00640 1 -- 172.21.15.122:0/4131866103 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 58) ==== 100804+0+0 (secure 0 0 0) 0x7f19040be070 con 0x7f18f8003c40 2024-09-17T05:41:45.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.187+0000 7f1910c00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18f4075990 0x7f18f4077e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.187+0000 7f1910c00640 1 -- 172.21.15.122:0/4131866103 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f190412d6c0 con 0x7f18f8003c40 2024-09-17T05:41:45.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.187+0000 7f1913e00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18f4075990 0x7f18f4077e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.187+0000 7f1913e00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18f4075990 0x7f18f4077e50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1900001ec0 tx=0x7f1900006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:45.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.191+0000 7f1910c00640 1 -- 172.21.15.122:0/4131866103 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f1904016e20 con 0x7f18f8003c40 2024-09-17T05:41:45.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.287+0000 7f1910c00640 1 -- 172.21.15.122:0/4131866103 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 59) ==== 100753+0+0 (secure 0 0 0) 0x7f19040f5df0 con 0x7f18f8003c40 2024-09-17T05:41:45.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.387+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f19140cf300 con 0x7f18f4075990 2024-09-17T05:41:45.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.387+0000 7f1910c00640 1 -- 172.21.15.122:0/4131866103 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f19140cf300 con 0x7f18f4075990 2024-09-17T05:41:45.382 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:41:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephfs" 2024-09-17T05:41:45.382 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:45.382 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:41:45.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18f4075990 msgr2=0x7f18f4077e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18f4075990 0x7f18f4077e50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1900001ec0 tx=0x7f1900006040 comp rx=0 tx=0).stop 2024-09-17T05:41:45.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 msgr2=0x7f19141f8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 0x7f19141f8a60 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f19040b0040 tx=0x7f190401b950 comp rx=0 tx=0).stop 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 shutdown_connections 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f18f4075990 0x7f18f4077e50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f191406f640 0x7f191406fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1914156440 0x7f19141568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 --2- 172.21.15.122:0/4131866103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18f8003c40 0x7f19141f8a60 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 >> 172.21.15.122:0/4131866103 conn(0x7f19140b18f0 msgr2=0x7f1914072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 shutdown_connections 2024-09-17T05:41:45.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.391+0000 7f191ba00640 1 -- 172.21.15.122:0/4131866103 wait complete. 2024-09-17T05:41:45.405 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-17T05:41:45.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 -- 172.21.15.122:0/2707990182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb4074760 msgr2=0x7f7cb40cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/2707990182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb4074760 0x7f7cb40cdb20 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40cd740 tx=0x7f7cac01cb10 comp rx=0 tx=0).stop 2024-09-17T05:41:45.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 -- 172.21.15.122:0/2707990182 shutdown_connections 2024-09-17T05:41:45.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/2707990182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb4074760 0x7f7cb40cdb20 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/2707990182 >> v2:172.21.15.173:3300/0 conn(0x7f7cb41564d0 0x7f7cb415cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 -- 172.21.15.122:0/2707990182 >> 172.21.15.122:0/2707990182 conn(0x7f7cb40cfc20 msgr2=0x7f7cb40d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:45.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 -- 172.21.15.122:0/2707990182 shutdown_connections 2024-09-17T05:41:45.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 -- 172.21.15.122:0/2707990182 wait complete. 2024-09-17T05:41:45.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 Processor -- start 2024-09-17T05:41:45.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.555+0000 7f7cbbc00640 1 -- start start 2024-09-17T05:41:45.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb4074760 0x7f7cb414da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb40cadf0 0x7f7cb414bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb41564d0 0x7f7cb414e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb40cec00 con 0x7f7cb40cadf0 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb40cea80 con 0x7f7cb41564d0 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb40ced80 con 0x7f7cb4074760 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb40cadf0 0x7f7cb414bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb40cadf0 0x7f7cb414bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48732/0 (socket says 172.21.15.122:48732) 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 -- 172.21.15.122:0/4184748519 learned_addr learned my addr 172.21.15.122:0/4184748519 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cb9200640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb4074760 0x7f7cb414da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cb9c00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb41564d0 0x7f7cb414e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 -- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb4074760 msgr2=0x7f7cb414da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb4074760 0x7f7cb414da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 -- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb41564d0 msgr2=0x7f7cb414e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb41564d0 0x7f7cb414e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 -- 172.21.15.122:0/4184748519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb4152a80 con 0x7f7cb40cadf0 2024-09-17T05:41:45.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cb9c00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb41564d0 0x7f7cb414e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:41:45.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cabe00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb40cadf0 0x7f7cb414bd30 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00c450 tx=0x7f7c9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:45.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cb1600640 1 -- 172.21.15.122:0/4184748519 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c012830 con 0x7f7cb40cadf0 2024-09-17T05:41:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cb1600640 1 -- 172.21.15.122:0/4184748519 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c9c017070 con 0x7f7cb40cadf0 2024-09-17T05:41:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb414d170 con 0x7f7cb40cadf0 2024-09-17T05:41:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cb1600640 1 -- 172.21.15.122:0/4184748519 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c012d80 con 0x7f7cb40cadf0 2024-09-17T05:41:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb414d730 con 0x7f7cb40cadf0 2024-09-17T05:41:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb40cf300 con 0x7f7cb40cadf0 2024-09-17T05:41:45.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.559+0000 7f7cb1600640 1 -- 172.21.15.122:0/4184748519 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 59) ==== 100753+0+0 (secure 0 0 0) 0x7f7c9c01c460 con 0x7f7cb40cadf0 2024-09-17T05:41:45.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.563+0000 7f7cb1600640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075900 0x7f7c90077dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.563+0000 7f7cb1600640 1 -- 172.21.15.122:0/4184748519 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f7c9c093150 con 0x7f7cb40cadf0 2024-09-17T05:41:45.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.563+0000 7f7cb9200640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075900 0x7f7c90077dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.563+0000 7f7cb9200640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075900 0x7f7c90077dc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4151870 tx=0x7f7ca4006f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:45.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.567+0000 7f7cb1600640 1 -- 172.21.15.122:0/4184748519 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7c9c05fd00 con 0x7f7cb40cadf0 2024-09-17T05:41:45.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.759+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7cb415db50 con 0x7f7c90075900 2024-09-17T05:41:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.771+0000 7f7cb1600640 1 -- 172.21.15.122:0/4184748519 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f7cb415db50 con 0x7f7c90075900 2024-09-17T05:41:45.766 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:45.766 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:41:45.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone12085" 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone77831" 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sv1" 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone11792" 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.767 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone40724" 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone46056" 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone81883" 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subvol_clone51580" 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:45.768 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:41:45.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075900 msgr2=0x7f7c90077dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075900 0x7f7c90077dc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4151870 tx=0x7f7ca4006f90 comp rx=0 tx=0).stop 2024-09-17T05:41:45.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb40cadf0 msgr2=0x7f7cb414bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb40cadf0 0x7f7cb414bd30 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00c450 tx=0x7f7c9c00c910 comp rx=0 tx=0).stop 2024-09-17T05:41:45.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 shutdown_connections 2024-09-17T05:41:45.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f7c90075900 0x7f7c90077dc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb41564d0 0x7f7cb414e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7cb40cadf0 0x7f7cb414bd30 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 --2- 172.21.15.122:0/4184748519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb4074760 0x7f7cb414da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 >> 172.21.15.122:0/4184748519 conn(0x7f7cb40cfc20 msgr2=0x7f7cb415e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:45.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 shutdown_connections 2024-09-17T05:41:45.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.775+0000 7f7cbbc00640 1 -- 172.21.15.122:0/4184748519 wait complete. 2024-09-17T05:41:45.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone12085 2024-09-17T05:41:45.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 -- 172.21.15.122:0/2528089861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c015be10 msgr2=0x7f08c00ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 --2- 172.21.15.122:0/2528089861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c015be10 0x7f08c00ce700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009bc0 tx=0x7f08b001f3b0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 -- 172.21.15.122:0/2528089861 shutdown_connections 2024-09-17T05:41:45.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 --2- 172.21.15.122:0/2528089861 >> v2:172.21.15.173:3300/0 conn(0x7f08c00b0b60 0x7f08c00ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 --2- 172.21.15.122:0/2528089861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c015be10 0x7f08c00ce700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 -- 172.21.15.122:0/2528089861 >> 172.21.15.122:0/2528089861 conn(0x7f08c00d1da0 msgr2=0x7f08c00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:45.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 -- 172.21.15.122:0/2528089861 shutdown_connections 2024-09-17T05:41:45.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 -- 172.21.15.122:0/2528089861 wait complete. 2024-09-17T05:41:45.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 Processor -- start 2024-09-17T05:41:45.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.939+0000 7f08c5a00640 1 -- start start 2024-09-17T05:41:45.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08c00b0b60 0x7f08c014f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c015be10 0x7f08c00c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c015c1f0 0x7f08c01562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08c00cd320 con 0x7f08c015be10 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08c00cf590 con 0x7f08c00b0b60 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f08c00cd4a0 con 0x7f08c015c1f0 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c015be10 0x7f08c00c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c015be10 0x7f08c00c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48736/0 (socket says 172.21.15.122:48736) 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 -- 172.21.15.122:0/3585577245 learned_addr learned my addr 172.21.15.122:0/3585577245 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 -- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c015c1f0 msgr2=0x7f08c01562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c015c1f0 0x7f08c01562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 -- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08c00b0b60 msgr2=0x7f08c014f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08c00b0b60 0x7f08c014f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 -- 172.21.15.122:0/3585577245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c00c6a50 con 0x7f08c015be10 2024-09-17T05:41:45.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b6000640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c015be10 0x7f08c00c7ab0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c460 tx=0x7f08a800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b7400640 1 -- 172.21.15.122:0/3585577245 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8016020 con 0x7f08c015be10 2024-09-17T05:41:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b7400640 1 -- 172.21.15.122:0/3585577245 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08a8012940 con 0x7f08c015be10 2024-09-17T05:41:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b7400640 1 -- 172.21.15.122:0/3585577245 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8012d90 con 0x7f08c015be10 2024-09-17T05:41:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c00c6cf0 con 0x7f08c015be10 2024-09-17T05:41:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c00c72b0 con 0x7f08c015be10 2024-09-17T05:41:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c00ccbd0 con 0x7f08c015be10 2024-09-17T05:41:45.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.943+0000 7f08b7400640 1 -- 172.21.15.122:0/3585577245 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 59) ==== 100753+0+0 (secure 0 0 0) 0x7f08a8019020 con 0x7f08c015be10 2024-09-17T05:41:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.947+0000 7f08b7400640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f089c0758d0 0x7f089c077d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:45.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.947+0000 7f08b7400640 1 -- 172.21.15.122:0/3585577245 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f08a8093490 con 0x7f08c015be10 2024-09-17T05:41:45.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.947+0000 7f08bea00640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f089c0758d0 0x7f089c077d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:45.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.947+0000 7f08bea00640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f089c0758d0 0x7f089c077d90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f08b00b1000 tx=0x7f08b001c150 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:45.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:45.951+0000 7f08b7400640 1 -- 172.21.15.122:0/3585577245 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f08a8020020 con 0x7f08c015be10 2024-09-17T05:41:46.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.143+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone12085", "target": ["mon-mgr", ""]}) -- 0x7f08c0070350 con 0x7f089c0758d0 2024-09-17T05:41:46.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08b7400640 1 -- 172.21.15.122:0/3585577245 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f08c0070350 con 0x7f089c0758d0 2024-09-17T05:41:46.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f089c0758d0 msgr2=0x7f089c077d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f089c0758d0 0x7f089c077d90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f08b00b1000 tx=0x7f08b001c150 comp rx=0 tx=0).stop 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c015be10 msgr2=0x7f08c00c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c015be10 0x7f08c00c7ab0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c460 tx=0x7f08a800c920 comp rx=0 tx=0).stop 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 shutdown_connections 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f089c0758d0 0x7f089c077d90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c015c1f0 0x7f08c01562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08c015be10 0x7f08c00c7ab0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 --2- 172.21.15.122:0/3585577245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08c00b0b60 0x7f08c014f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 >> 172.21.15.122:0/3585577245 conn(0x7f08c00d1da0 msgr2=0x7f08c0071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 shutdown_connections 2024-09-17T05:41:46.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.163+0000 7f08c5a00640 1 -- 172.21.15.122:0/3585577245 wait complete. 2024-09-17T05:41:46.162 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.163+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.162 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.163+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.162 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.163+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.162 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.163+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.162 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.163+0000 7fc09aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.179 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone77831 2024-09-17T05:41:46.180 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.179+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.180 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.179+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.180 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.179+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.180 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.179+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.180 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-09-17T05:41:46.179+0000 7fc09c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-17T05:41:46.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 -- 172.21.15.122:0/4012717861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80cf440 msgr2=0x7fb9e80ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 --2- 172.21.15.122:0/4012717861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80cf440 0x7fb9e80ce350 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8051480 tx=0x7fb9d401dbc0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 -- 172.21.15.122:0/4012717861 shutdown_connections 2024-09-17T05:41:46.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 --2- 172.21.15.122:0/4012717861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80cf440 0x7fb9e80ce350 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 --2- 172.21.15.122:0/4012717861 >> v2:172.21.15.173:3300/0 conn(0x7fb9e80734f0 0x7fb9e815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 -- 172.21.15.122:0/4012717861 >> 172.21.15.122:0/4012717861 conn(0x7fb9e80cfbc0 msgr2=0x7fb9e80cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:46.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 -- 172.21.15.122:0/4012717861 shutdown_connections 2024-09-17T05:41:46.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 -- 172.21.15.122:0/4012717861 wait complete. 2024-09-17T05:41:46.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.335+0000 7fb9ed200640 1 Processor -- start 2024-09-17T05:41:46.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 -- start start 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80734f0 0x7fb9e8158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9e80ce890 0x7fb9e8152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9e80cf440 0x7fb9e8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e815ed80 con 0x7fb9e80734f0 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e815ec00 con 0x7fb9e80cf440 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb9e8167360 con 0x7fb9e80ce890 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80734f0 0x7fb9e8158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80734f0 0x7fb9e8158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48766/0 (socket says 172.21.15.122:48766) 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 -- 172.21.15.122:0/2515574573 learned_addr learned my addr 172.21.15.122:0/2515574573 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e6800640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9e80cf440 0x7fb9e8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:46.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5400640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9e80ce890 0x7fb9e8152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:46.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 -- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9e80ce890 msgr2=0x7fb9e8152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9e80ce890 0x7fb9e8152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 -- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9e80cf440 msgr2=0x7fb9e8156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9e80cf440 0x7fb9e8156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 -- 172.21.15.122:0/2515574573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e8153100 con 0x7fb9e80734f0 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9e5e00640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80734f0 0x7fb9e8158ba0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800d980 tx=0x7fb9d800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9de000640 1 -- 172.21.15.122:0/2515574573 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8017070 con 0x7fb9e80734f0 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9de000640 1 -- 172.21.15.122:0/2515574573 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9d8009d60 con 0x7fb9e80734f0 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9de000640 1 -- 172.21.15.122:0/2515574573 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8005280 con 0x7fb9e80734f0 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e81fde20 con 0x7fb9e80734f0 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e81fe3e0 con 0x7fb9e80734f0 2024-09-17T05:41:46.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.339+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e8069410 con 0x7fb9e80734f0 2024-09-17T05:41:46.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.343+0000 7fb9de000640 1 -- 172.21.15.122:0/2515574573 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 59) ==== 100753+0+0 (secure 0 0 0) 0x7fb9d8005420 con 0x7fb9e80734f0 2024-09-17T05:41:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.343+0000 7fb9de000640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9c80758d0 0x7fb9c8077d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.343+0000 7fb9de000640 1 -- 172.21.15.122:0/2515574573 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fb9d8093610 con 0x7fb9e80734f0 2024-09-17T05:41:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.343+0000 7fb9e5400640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9c80758d0 0x7fb9c8077d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:46.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.343+0000 7fb9e5400640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9c80758d0 0x7fb9c8077d90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40b0000 tx=0x7fb9d401c400 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:46.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.347+0000 7fb9de000640 1 -- 172.21.15.122:0/2515574573 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb9e8153100 con 0x7fb9e80734f0 2024-09-17T05:41:46.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.539+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone77831", "target": ["mon-mgr", ""]}) -- 0x7fb9e815a9d0 con 0x7fb9c80758d0 2024-09-17T05:41:46.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9de000640 1 -- 172.21.15.122:0/2515574573 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb9e815a9d0 con 0x7fb9c80758d0 2024-09-17T05:41:46.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9c80758d0 msgr2=0x7fb9c8077d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9c80758d0 0x7fb9c8077d90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40b0000 tx=0x7fb9d401c400 comp rx=0 tx=0).stop 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80734f0 msgr2=0x7fb9e8158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80734f0 0x7fb9e8158ba0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800d980 tx=0x7fb9d800de40 comp rx=0 tx=0).stop 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 shutdown_connections 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb9c80758d0 0x7fb9c8077d90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9e80cf440 0x7fb9e8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9e80ce890 0x7fb9e8152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 --2- 172.21.15.122:0/2515574573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9e80734f0 0x7fb9e8158ba0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 >> 172.21.15.122:0/2515574573 conn(0x7fb9e80cfbc0 msgr2=0x7fb9e80af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 shutdown_connections 2024-09-17T05:41:46.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.567+0000 7fb9ed200640 1 -- 172.21.15.122:0/2515574573 wait complete. 2024-09-17T05:41:46.582 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-17T05:41:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 -- 172.21.15.122:0/1167628602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 msgr2=0x7fe9bc0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 --2- 172.21.15.122:0/1167628602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 0x7fe9bc0d28c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009840 tx=0x7fe9b001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 -- 172.21.15.122:0/1167628602 shutdown_connections 2024-09-17T05:41:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 --2- 172.21.15.122:0/1167628602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 0x7fe9bc0d28c0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 -- 172.21.15.122:0/1167628602 >> 172.21.15.122:0/1167628602 conn(0x7fe9bc0d1da0 msgr2=0x7fe9bc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 -- 172.21.15.122:0/1167628602 shutdown_connections 2024-09-17T05:41:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 -- 172.21.15.122:0/1167628602 wait complete. 2024-09-17T05:41:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 Processor -- start 2024-09-17T05:41:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.735+0000 7fe9c3000640 1 -- start start 2024-09-17T05:41:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 0x7fe9bc0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9bc0d02e0 0x7fe9bc152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9bc15f700 0x7fe9bc154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc169f90 con 0x7fe9bc0cff00 2024-09-17T05:41:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc169e10 con 0x7fe9bc0d02e0 2024-09-17T05:41:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe9bc16a110 con 0x7fe9bc15f700 2024-09-17T05:41:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 0x7fe9bc0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 0x7fe9bc0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48782/0 (socket says 172.21.15.122:48782) 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 -- 172.21.15.122:0/3236549015 learned_addr learned my addr 172.21.15.122:0/3236549015 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1200640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9bc0d02e0 0x7fe9bc152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c2600640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9bc15f700 0x7fe9bc154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 -- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9bc15f700 msgr2=0x7fe9bc154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9bc15f700 0x7fe9bc154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 -- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9bc0d02e0 msgr2=0x7fe9bc152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9bc0d02e0 0x7fe9bc152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 -- 172.21.15.122:0/3236549015 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc0ce680 con 0x7fe9bc0cff00 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c2600640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9bc15f700 0x7fe9bc154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c1c00640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 0x7fe9bc0cdff0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009790 tx=0x7fe9b00a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9b9600640 1 -- 172.21.15.122:0/3236549015 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00b2040 con 0x7fe9bc0cff00 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9b9600640 1 -- 172.21.15.122:0/3236549015 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9b0004470 con 0x7fe9bc0cff00 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9b9600640 1 -- 172.21.15.122:0/3236549015 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0005220 con 0x7fe9bc0cff00 2024-09-17T05:41:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc0b4bf0 con 0x7fe9bc0cff00 2024-09-17T05:41:46.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc0b51b0 con 0x7fe9bc0cff00 2024-09-17T05:41:46.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.739+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc0d28c0 con 0x7fe9bc0cff00 2024-09-17T05:41:46.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.743+0000 7fe9b9600640 1 -- 172.21.15.122:0/3236549015 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 59) ==== 100753+0+0 (secure 0 0 0) 0x7fe9b00bd060 con 0x7fe9bc0cff00 2024-09-17T05:41:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.743+0000 7fe9b9600640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe99c0758d0 0x7fe99c077d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.743+0000 7fe9b9600640 1 -- 172.21.15.122:0/3236549015 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7fe9b012cc90 con 0x7fe9bc0cff00 2024-09-17T05:41:46.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.743+0000 7fe9c1200640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe99c0758d0 0x7fe99c077d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:46.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.743+0000 7fe9c1200640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe99c0758d0 0x7fe99c077d90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0c81b0 tx=0x7fe9a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:46.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.747+0000 7fe9b9600640 1 -- 172.21.15.122:0/3236549015 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fe9b0016410 con 0x7fe9bc0cff00 2024-09-17T05:41:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:46.939+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9bc0b26d0 con 0x7fe99c0758d0 2024-09-17T05:41:47.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.299+0000 7fe9b9600640 1 -- 172.21.15.122:0/3236549015 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fe9b00f5310 con 0x7fe9bc0cff00 2024-09-17T05:41:47.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.575+0000 7fe9b9600640 1 -- 172.21.15.122:0/3236549015 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe9bc0b26d0 con 0x7fe99c0758d0 2024-09-17T05:41:47.570 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:47.570 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-09-17T05:41:47.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-17T05:41:47.571 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ss1" 2024-09-17T05:41:47.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-17T05:41:47.571 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-09-17T05:41:47.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe99c0758d0 msgr2=0x7fe99c077d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:47.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe99c0758d0 0x7fe99c077d90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0c81b0 tx=0x7fe9a8009040 comp rx=0 tx=0).stop 2024-09-17T05:41:47.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 msgr2=0x7fe9bc0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:47.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 0x7fe9bc0cdff0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009790 tx=0x7fe9b00a5740 comp rx=0 tx=0).stop 2024-09-17T05:41:47.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 shutdown_connections 2024-09-17T05:41:47.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fe99c0758d0 0x7fe99c077d90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9bc15f700 0x7fe9bc154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9bc0d02e0 0x7fe9bc152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 --2- 172.21.15.122:0/3236549015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9bc0cff00 0x7fe9bc0cdff0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 >> 172.21.15.122:0/3236549015 conn(0x7fe9bc0d1da0 msgr2=0x7fe9bc071e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 shutdown_connections 2024-09-17T05:41:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.579+0000 7fe9c3000640 1 -- 172.21.15.122:0/3236549015 wait complete. 2024-09-17T05:41:47.597 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-17T05:41:47.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.751+0000 7f58b2c00640 1 -- 172.21.15.122:0/1781714824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac15c2d0 msgr2=0x7f58ac074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:47.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.751+0000 7f58b2c00640 1 --2- 172.21.15.122:0/1781714824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac15c2d0 0x7f58ac074060 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0ce850 tx=0x7f589401cac0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.751+0000 7f58b2c00640 1 -- 172.21.15.122:0/1781714824 shutdown_connections 2024-09-17T05:41:47.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.751+0000 7f58b2c00640 1 --2- 172.21.15.122:0/1781714824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac15c2d0 0x7f58ac074060 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.751+0000 7f58b2c00640 1 --2- 172.21.15.122:0/1781714824 >> v2:172.21.15.173:3300/0 conn(0x7f58ac0b6350 0x7f58ac06fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.751+0000 7f58b2c00640 1 -- 172.21.15.122:0/1781714824 >> 172.21.15.122:0/1781714824 conn(0x7f58ac0d0cd0 msgr2=0x7f58ac0d10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:47.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.751+0000 7f58b2c00640 1 -- 172.21.15.122:0/1781714824 shutdown_connections 2024-09-17T05:41:47.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.751+0000 7f58b2c00640 1 -- 172.21.15.122:0/1781714824 wait complete. 2024-09-17T05:41:47.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 Processor -- start 2024-09-17T05:41:47.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 -- start start 2024-09-17T05:41:47.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac0b6350 0x7f58ac15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:47.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58ac15c2d0 0x7f58ac0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:47.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ac153300 0x7f58ac0c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:47.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac168da0 con 0x7f58ac0b6350 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac168c20 con 0x7f58ac153300 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac168f20 con 0x7f58ac15c2d0 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac0b6350 0x7f58ac15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac0b6350 0x7f58ac15dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48800/0 (socket says 172.21.15.122:48800) 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 -- 172.21.15.122:0/2297189913 learned_addr learned my addr 172.21.15.122:0/2297189913 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b0c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ac153300 0x7f58ac0c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 -- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58ac15c2d0 msgr2=0x7f58ac0cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58a3400640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58ac15c2d0 0x7f58ac0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58ac15c2d0 0x7f58ac0cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 -- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ac153300 msgr2=0x7f58ac0c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ac153300 0x7f58ac0c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 -- 172.21.15.122:0/2297189913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac0ceda0 con 0x7f58ac0b6350 2024-09-17T05:41:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b0c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ac153300 0x7f58ac0c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:41:47.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58a3400640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58ac15c2d0 0x7f58ac0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:41:47.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58abe00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac0b6350 0x7f58ac15dd90 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f589c00d980 tx=0x7f589c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:47.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58a8c00640 1 -- 172.21.15.122:0/2297189913 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c017070 con 0x7f58ac0b6350 2024-09-17T05:41:47.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58a8c00640 1 -- 172.21.15.122:0/2297189913 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f589c009d60 con 0x7f58ac0b6350 2024-09-17T05:41:47.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58a8c00640 1 -- 172.21.15.122:0/2297189913 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c005280 con 0x7f58ac0b6350 2024-09-17T05:41:47.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac0cf360 con 0x7f58ac0b6350 2024-09-17T05:41:47.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac0cf920 con 0x7f58ac0b6350 2024-09-17T05:41:47.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.755+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ac06fff0 con 0x7f58ac0b6350 2024-09-17T05:41:47.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.759+0000 7f58a8c00640 1 -- 172.21.15.122:0/2297189913 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f589c005420 con 0x7f58ac0b6350 2024-09-17T05:41:47.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.759+0000 7f58a8c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58880759f0 0x7f5888077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:47.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.759+0000 7f58a8c00640 1 -- 172.21.15.122:0/2297189913 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f589c0937a0 con 0x7f58ac0b6350 2024-09-17T05:41:47.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.759+0000 7f58a3400640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58880759f0 0x7f5888077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:47.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.759+0000 7f58a3400640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58880759f0 0x7f5888077eb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f58940af000 tx=0x7f589401b300 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:47.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.763+0000 7f58a8c00640 1 -- 172.21.15.122:0/2297189913 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f589c0602b0 con 0x7f58ac0b6350 2024-09-17T05:41:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.959+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58ac0d1b00 con 0x7f58880759f0 2024-09-17T05:41:47.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.987+0000 7f58a8c00640 1 -- 172.21.15.122:0/2297189913 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f58ac0d1b00 con 0x7f58880759f0 2024-09-17T05:41:47.983 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:47.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58880759f0 msgr2=0x7f5888077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:47.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58880759f0 0x7f5888077eb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f58940af000 tx=0x7f589401b300 comp rx=0 tx=0).stop 2024-09-17T05:41:47.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac0b6350 msgr2=0x7f58ac15dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:47.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac0b6350 0x7f58ac15dd90 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f589c00d980 tx=0x7f589c00de40 comp rx=0 tx=0).stop 2024-09-17T05:41:47.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 shutdown_connections 2024-09-17T05:41:47.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f58880759f0 0x7f5888077eb0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ac153300 0x7f58ac0c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58ac15c2d0 0x7f58ac0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 --2- 172.21.15.122:0/2297189913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58ac0b6350 0x7f58ac15dd90 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:47.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 >> 172.21.15.122:0/2297189913 conn(0x7f58ac0d0cd0 msgr2=0x7f58ac0cd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:47.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 shutdown_connections 2024-09-17T05:41:47.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:47.991+0000 7f58b2c00640 1 -- 172.21.15.122:0/2297189913 wait complete. 2024-09-17T05:41:48.004 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-17T05:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- 172.21.15.122:0/3154372027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 msgr2=0x7f22680cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 --2- 172.21.15.122:0/3154372027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 0x7f22680cee30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2250009840 tx=0x7f225001c820 comp rx=0 tx=0).stop 2024-09-17T05:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- 172.21.15.122:0/3154372027 shutdown_connections 2024-09-17T05:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 --2- 172.21.15.122:0/3154372027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22680cd150 0x7f22680cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 --2- 172.21.15.122:0/3154372027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 0x7f22680cee30 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- 172.21.15.122:0/3154372027 >> 172.21.15.122:0/3154372027 conn(0x7f22680d1ea0 msgr2=0x7f22680d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- 172.21.15.122:0/3154372027 shutdown_connections 2024-09-17T05:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- 172.21.15.122:0/3154372027 wait complete. 2024-09-17T05:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 Processor -- start 2024-09-17T05:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- start start 2024-09-17T05:41:48.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 0x7f2268154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:48.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22680cd150 0x7f226815cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:48.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f226815e130 0x7f2268150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:48.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f226816ae90 con 0x7f226806e740 2024-09-17T05:41:48.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22680cf5b0 con 0x7f22680cd150 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f226d200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f226816b010 con 0x7f226815e130 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2265e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 0x7f2268154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2265e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 0x7f2268154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48832/0 (socket says 172.21.15.122:48832) 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2265e00640 1 -- 172.21.15.122:0/292644338 learned_addr learned my addr 172.21.15.122:0/292644338 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2266800640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f226815e130 0x7f2268150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f225d400640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22680cd150 0x7f226815cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2265e00640 1 -- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f226815e130 msgr2=0x7f2268150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2265e00640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f226815e130 0x7f2268150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2265e00640 1 -- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22680cd150 msgr2=0x7f226815cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2265e00640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22680cd150 0x7f226815cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2265e00640 1 -- 172.21.15.122:0/292644338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22680b5cb0 con 0x7f226806e740 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f2266800640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f226815e130 0x7f2268150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.155+0000 7f225d400640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22680cd150 0x7f226815cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f2265e00640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 0x7f2268154590 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f225800c760 tx=0x7f225800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f225ea00640 1 -- 172.21.15.122:0/292644338 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258016020 con 0x7f226806e740 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f225ea00640 1 -- 172.21.15.122:0/292644338 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f225800a8f0 con 0x7f226806e740 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22680b5410 con 0x7f226806e740 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f225ea00640 1 -- 172.21.15.122:0/292644338 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225800ad40 con 0x7f226806e740 2024-09-17T05:41:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22680b59d0 con 0x7f226806e740 2024-09-17T05:41:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22680cd5e0 con 0x7f226806e740 2024-09-17T05:41:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f225ea00640 1 -- 172.21.15.122:0/292644338 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f2258019020 con 0x7f226806e740 2024-09-17T05:41:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f225ea00640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22440759f0 0x7f2244077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f225ea00640 1 -- 172.21.15.122:0/292644338 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f2258093440 con 0x7f226806e740 2024-09-17T05:41:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.159+0000 7f225d400640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22440759f0 0x7f2244077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.163+0000 7f225d400640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22440759f0 0x7f2244077eb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f22500af000 tx=0x7f225001b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.163+0000 7f225ea00640 1 -- 172.21.15.122:0/292644338 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2258020020 con 0x7f226806e740 2024-09-17T05:41:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.467+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f22680b35a0 con 0x7f22440759f0 2024-09-17T05:41:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.515+0000 7f225ea00640 1 -- 172.21.15.122:0/292644338 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f22680b35a0 con 0x7f22440759f0 2024-09-17T05:41:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22440759f0 msgr2=0x7f2244077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22440759f0 0x7f2244077eb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f22500af000 tx=0x7f225001b060 comp rx=0 tx=0).stop 2024-09-17T05:41:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 msgr2=0x7f2268154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 0x7f2268154590 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f225800c760 tx=0x7f225800cc20 comp rx=0 tx=0).stop 2024-09-17T05:41:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 shutdown_connections 2024-09-17T05:41:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22440759f0 0x7f2244077eb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f226815e130 0x7f2268150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22680cd150 0x7f226815cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 --2- 172.21.15.122:0/292644338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f226806e740 0x7f2268154590 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 >> 172.21.15.122:0/292644338 conn(0x7f22680d1ea0 msgr2=0x7f22680b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:48.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 shutdown_connections 2024-09-17T05:41:48.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.519+0000 7f226d200640 1 -- 172.21.15.122:0/292644338 wait complete. 2024-09-17T05:41:48.558 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone11792 2024-09-17T05:41:48.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.819+0000 7f2c57e00640 1 -- 172.21.15.122:0/4095757857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815e620 msgr2=0x7f2c5815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:48.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.819+0000 7f2c57e00640 1 --2- 172.21.15.122:0/4095757857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815e620 0x7f2c5815c140 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009870 tx=0x7f2c4401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 -- 172.21.15.122:0/4095757857 shutdown_connections 2024-09-17T05:41:48.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 --2- 172.21.15.122:0/4095757857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815e620 0x7f2c5815c140 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 --2- 172.21.15.122:0/4095757857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c58158620 0x7f2c5815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 -- 172.21.15.122:0/4095757857 >> 172.21.15.122:0/4095757857 conn(0x7f2c5806ec90 msgr2=0x7f2c5806f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:48.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 -- 172.21.15.122:0/4095757857 shutdown_connections 2024-09-17T05:41:48.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 -- 172.21.15.122:0/4095757857 wait complete. 2024-09-17T05:41:48.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 Processor -- start 2024-09-17T05:41:48.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 -- start start 2024-09-17T05:41:48.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c58158620 0x7f2c580cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:48.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c5815e620 0x7f2c58163af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:48.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815ea40 0x7f2c581fb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:48.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c58167910 con 0x7f2c58158620 2024-09-17T05:41:48.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2c58167790 con 0x7f2c5815ea40 2024-09-17T05:41:48.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2c58167a90 con 0x7f2c5815e620 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c58158620 0x7f2c580cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c58158620 0x7f2c580cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48848/0 (socket says 172.21.15.122:48848) 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 -- 172.21.15.122:0/175317718 learned_addr learned my addr 172.21.15.122:0/175317718 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56000640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c5815e620 0x7f2c58163af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57400640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815ea40 0x7f2c581fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 -- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c5815e620 msgr2=0x7f2c58163af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c5815e620 0x7f2c58163af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 -- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815ea40 msgr2=0x7f2c581fb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815ea40 0x7f2c581fb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 -- 172.21.15.122:0/175317718 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5815c4b0 con 0x7f2c58158620 2024-09-17T05:41:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c57400640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815ea40 0x7f2c581fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:41:48.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.823+0000 7f2c56a00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c58158620 0x7f2c580cd5e0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800e9d0 tx=0x7f2c4800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:48.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c4ea00640 1 -- 172.21.15.122:0/175317718 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800cde0 con 0x7f2c58158620 2024-09-17T05:41:48.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c4ea00640 1 -- 172.21.15.122:0/175317718 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c48004020 con 0x7f2c58158620 2024-09-17T05:41:48.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c4ea00640 1 -- 172.21.15.122:0/175317718 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48004450 con 0x7f2c58158620 2024-09-17T05:41:48.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5814dc60 con 0x7f2c58158620 2024-09-17T05:41:48.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c58150420 con 0x7f2c58158620 2024-09-17T05:41:48.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5815bb20 con 0x7f2c58158620 2024-09-17T05:41:48.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c4ea00640 1 -- 172.21.15.122:0/175317718 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f2c48003600 con 0x7f2c58158620 2024-09-17T05:41:48.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c4ea00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c380759f0 0x7f2c38077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:48.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c4ea00640 1 -- 172.21.15.122:0/175317718 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f2c48093ee0 con 0x7f2c58158620 2024-09-17T05:41:48.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.827+0000 7f2c56000640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c380759f0 0x7f2c38077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.831+0000 7f2c56000640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c380759f0 0x7f2c38077eb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2c440af000 tx=0x7f2c4401b030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:48.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:48.831+0000 7f2c4ea00640 1 -- 172.21.15.122:0/175317718 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2c480609f0 con 0x7f2c58158620 2024-09-17T05:41:49.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.023+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone11792", "target": ["mon-mgr", ""]}) -- 0x7f2c581fad50 con 0x7f2c380759f0 2024-09-17T05:41:49.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c4ea00640 1 -- 172.21.15.122:0/175317718 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c581fad50 con 0x7f2c380759f0 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c380759f0 msgr2=0x7f2c38077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c380759f0 0x7f2c38077eb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2c440af000 tx=0x7f2c4401b030 comp rx=0 tx=0).stop 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c58158620 msgr2=0x7f2c580cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c58158620 0x7f2c580cd5e0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800e9d0 tx=0x7f2c4800ee90 comp rx=0 tx=0).stop 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 shutdown_connections 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2c380759f0 0x7f2c38077eb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c5815ea40 0x7f2c581fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c5815e620 0x7f2c58163af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 --2- 172.21.15.122:0/175317718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c58158620 0x7f2c580cd5e0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 >> 172.21.15.122:0/175317718 conn(0x7f2c5806ec90 msgr2=0x7f2c580cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:49.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 shutdown_connections 2024-09-17T05:41:49.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.043+0000 7f2c57e00640 1 -- 172.21.15.122:0/175317718 wait complete. 2024-09-17T05:41:49.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone40724 2024-09-17T05:41:49.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.215+0000 7f1436400640 1 -- 172.21.15.122:0/2583389645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14300b0b60 msgr2=0x7f14300ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.215+0000 7f1436400640 1 --2- 172.21.15.122:0/2583389645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14300b0b60 0x7f14300ccbd0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f1428009c80 tx=0x7f142801f5a0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.215+0000 7f1436400640 1 -- 172.21.15.122:0/2583389645 shutdown_connections 2024-09-17T05:41:49.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.215+0000 7f1436400640 1 --2- 172.21.15.122:0/2583389645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14300b0b60 0x7f14300ccbd0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.215+0000 7f1436400640 1 --2- 172.21.15.122:0/2583389645 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f143015be10 0x7f14300ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.215+0000 7f1436400640 1 -- 172.21.15.122:0/2583389645 >> 172.21.15.122:0/2583389645 conn(0x7f14300d1da0 msgr2=0x7f14300d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:49.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.215+0000 7f1436400640 1 -- 172.21.15.122:0/2583389645 shutdown_connections 2024-09-17T05:41:49.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.215+0000 7f1436400640 1 -- 172.21.15.122:0/2583389645 wait complete. 2024-09-17T05:41:49.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 Processor -- start 2024-09-17T05:41:49.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 -- start start 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14300b0b60 0x7f14300b7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f143015be10 0x7f1430159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143015c1f0 0x7f1430156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14300cd4a0 con 0x7f143015c1f0 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f14300cd320 con 0x7f14300b0b60 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f14300cd620 con 0x7f143015be10 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143015c1f0 0x7f1430156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143015c1f0 0x7f1430156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48862/0 (socket says 172.21.15.122:48862) 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 -- 172.21.15.122:0/3766426405 learned_addr learned my addr 172.21.15.122:0/3766426405 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142f400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14300b0b60 0x7f14300b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:49.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142ea00640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f143015be10 0x7f1430159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:49.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 -- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f143015be10 msgr2=0x7f1430159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f143015be10 0x7f1430159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 -- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14300b0b60 msgr2=0x7f14300b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14300b0b60 0x7f14300b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 -- 172.21.15.122:0/3766426405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14300c6950 con 0x7f143015c1f0 2024-09-17T05:41:49.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142f400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14300b0b60 0x7f14300b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:41:49.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f142fe00640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143015c1f0 0x7f1430156010 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f1428009bd0 tx=0x7f1428005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:49.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1427400640 1 -- 172.21.15.122:0/3766426405 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14280af610 con 0x7f143015c1f0 2024-09-17T05:41:49.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1427400640 1 -- 172.21.15.122:0/3766426405 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14280af990 con 0x7f143015c1f0 2024-09-17T05:41:49.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1427400640 1 -- 172.21.15.122:0/3766426405 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14280afd40 con 0x7f143015c1f0 2024-09-17T05:41:49.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14301f6a60 con 0x7f143015c1f0 2024-09-17T05:41:49.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.219+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14301f6260 con 0x7f143015c1f0 2024-09-17T05:41:49.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.223+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14301f5da0 con 0x7f143015c1f0 2024-09-17T05:41:49.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.223+0000 7f1427400640 1 -- 172.21.15.122:0/3766426405 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f1428004040 con 0x7f143015c1f0 2024-09-17T05:41:49.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.223+0000 7f1427400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14100759f0 0x7f1410077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:49.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.223+0000 7f1427400640 1 -- 172.21.15.122:0/3766426405 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f1428132030 con 0x7f143015c1f0 2024-09-17T05:41:49.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.223+0000 7f142f400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14100759f0 0x7f1410077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:49.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.223+0000 7f142f400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14100759f0 0x7f1410077eb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f14300cc2d0 tx=0x7f14200091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:49.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.227+0000 7f1427400640 1 -- 172.21.15.122:0/3766426405 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f14301f5da0 con 0x7f143015c1f0 2024-09-17T05:41:49.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.423+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone40724", "target": ["mon-mgr", ""]}) -- 0x7f1430070350 con 0x7f14100759f0 2024-09-17T05:41:49.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.443+0000 7f1427400640 1 -- 172.21.15.122:0/3766426405 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1430070350 con 0x7f14100759f0 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14100759f0 msgr2=0x7f1410077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14100759f0 0x7f1410077eb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f14300cc2d0 tx=0x7f14200091e0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143015c1f0 msgr2=0x7f1430156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143015c1f0 0x7f1430156010 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f1428009bd0 tx=0x7f1428005a60 comp rx=0 tx=0).stop 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 shutdown_connections 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f14100759f0 0x7f1410077eb0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143015c1f0 0x7f1430156010 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f143015be10 0x7f1430159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 --2- 172.21.15.122:0/3766426405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14300b0b60 0x7f14300b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 >> 172.21.15.122:0/3766426405 conn(0x7f14300d1da0 msgr2=0x7f14300b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 shutdown_connections 2024-09-17T05:41:49.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.447+0000 7f1436400640 1 -- 172.21.15.122:0/3766426405 wait complete. 2024-09-17T05:41:49.463 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone46056 2024-09-17T05:41:49.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- 172.21.15.122:0/2152225405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900cf440 msgr2=0x7ff0900ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 --2- 172.21.15.122:0/2152225405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900cf440 0x7ff0900ce350 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7ff090070c30 tx=0x7ff07c01dbf0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- 172.21.15.122:0/2152225405 shutdown_connections 2024-09-17T05:41:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 --2- 172.21.15.122:0/2152225405 >> v2:172.21.15.173:3300/0 conn(0x7ff0900ce890 0x7ff09015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 --2- 172.21.15.122:0/2152225405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900cf440 0x7ff0900ce350 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 --2- 172.21.15.122:0/2152225405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0900734f0 0x7ff09015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- 172.21.15.122:0/2152225405 >> 172.21.15.122:0/2152225405 conn(0x7ff0900cfbc0 msgr2=0x7ff0900cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- 172.21.15.122:0/2152225405 shutdown_connections 2024-09-17T05:41:49.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- 172.21.15.122:0/2152225405 wait complete. 2024-09-17T05:41:49.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 Processor -- start 2024-09-17T05:41:49.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- start start 2024-09-17T05:41:49.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0900734f0 0x7ff09009b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:49.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900ce890 0x7ff0900b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0900cf440 0x7ff090150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff09015ee00 con 0x7ff0900ce890 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff09015ec80 con 0x7ff0900cf440 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff097e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff09006eb40 con 0x7ff0900734f0 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff094a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900ce890 0x7ff0900b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff094a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900ce890 0x7ff0900b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48872/0 (socket says 172.21.15.122:48872) 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff094a00640 1 -- 172.21.15.122:0/2774814357 learned_addr learned my addr 172.21.15.122:0/2774814357 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff095400640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0900734f0 0x7ff09009b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff095e00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0900cf440 0x7ff090150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff094a00640 1 -- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0900734f0 msgr2=0x7ff09009b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff094a00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0900734f0 0x7ff09009b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff094a00640 1 -- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0900cf440 msgr2=0x7ff090150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff094a00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0900cf440 0x7ff090150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.619+0000 7ff094a00640 1 -- 172.21.15.122:0/2774814357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0900b1110 con 0x7ff0900ce890 2024-09-17T05:41:49.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff094a00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900ce890 0x7ff0900b08e0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0b1040 tx=0x7ff07c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:49.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff08d600640 1 -- 172.21.15.122:0/2774814357 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0b3070 con 0x7ff0900ce890 2024-09-17T05:41:49.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff08d600640 1 -- 172.21.15.122:0/2774814357 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff07c005330 con 0x7ff0900ce890 2024-09-17T05:41:49.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0901fb3f0 con 0x7ff0900ce890 2024-09-17T05:41:49.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff08d600640 1 -- 172.21.15.122:0/2774814357 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0046d0 con 0x7ff0900ce890 2024-09-17T05:41:49.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0900b0e20 con 0x7ff0900ce890 2024-09-17T05:41:49.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff090069410 con 0x7ff0900ce890 2024-09-17T05:41:49.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff08d600640 1 -- 172.21.15.122:0/2774814357 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7ff07c0be060 con 0x7ff0900ce890 2024-09-17T05:41:49.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff08d600640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0700759f0 0x7ff070077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:49.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.623+0000 7ff08d600640 1 -- 172.21.15.122:0/2774814357 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7ff07c12d570 con 0x7ff0900ce890 2024-09-17T05:41:49.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.627+0000 7ff095400640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0700759f0 0x7ff070077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:49.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.627+0000 7ff095400640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0700759f0 0x7ff070077eb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff084009b60 tx=0x7ff0840091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:49.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.627+0000 7ff08d600640 1 -- 172.21.15.122:0/2774814357 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff07c0177e0 con 0x7ff0900ce890 2024-09-17T05:41:49.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.835+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone46056", "target": ["mon-mgr", ""]}) -- 0x7ff09015df70 con 0x7ff0700759f0 2024-09-17T05:41:49.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff08d600640 1 -- 172.21.15.122:0/2774814357 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff09015df70 con 0x7ff0700759f0 2024-09-17T05:41:49.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0700759f0 msgr2=0x7ff070077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0700759f0 0x7ff070077eb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff084009b60 tx=0x7ff0840091e0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900ce890 msgr2=0x7ff0900b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:49.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900ce890 0x7ff0900b08e0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0b1040 tx=0x7ff07c004600 comp rx=0 tx=0).stop 2024-09-17T05:41:49.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 shutdown_connections 2024-09-17T05:41:49.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff0700759f0 0x7ff070077eb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff0900cf440 0x7ff090150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0900ce890 0x7ff0900b08e0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 --2- 172.21.15.122:0/2774814357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0900734f0 0x7ff09009b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:49.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 >> 172.21.15.122:0/2774814357 conn(0x7ff0900cfbc0 msgr2=0x7ff09015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:49.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 shutdown_connections 2024-09-17T05:41:49.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:49.851+0000 7ff097e00640 1 -- 172.21.15.122:0/2774814357 wait complete. 2024-09-17T05:41:49.867 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81883 2024-09-17T05:41:50.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 -- 172.21.15.122:0/3325756161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54400cf2b0 msgr2=0x7f544006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 --2- 172.21.15.122:0/3325756161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54400cf2b0 0x7f544006fcc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5438009bf0 tx=0x7f543801cbe0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 -- 172.21.15.122:0/3325756161 shutdown_connections 2024-09-17T05:41:50.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 --2- 172.21.15.122:0/3325756161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54400cf2b0 0x7f544006fcc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 -- 172.21.15.122:0/3325756161 >> 172.21.15.122:0/3325756161 conn(0x7f54400cfbc0 msgr2=0x7f54400cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:50.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 -- 172.21.15.122:0/3325756161 shutdown_connections 2024-09-17T05:41:50.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 -- 172.21.15.122:0/3325756161 wait complete. 2024-09-17T05:41:50.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 Processor -- start 2024-09-17T05:41:50.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.019+0000 7f5445e00640 1 -- start start 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5440070960 0x7f5440138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5440072f80 0x7f5440158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54400cf2b0 0x7f54401591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5440169840 con 0x7f5440070960 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f54401699c0 con 0x7f5440072f80 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5440169b40 con 0x7f54400cf2b0 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5440070960 0x7f5440138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5440070960 0x7f5440138ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48884/0 (socket says 172.21.15.122:48884) 2024-09-17T05:41:50.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 -- 172.21.15.122:0/660793254 learned_addr learned my addr 172.21.15.122:0/660793254 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543e000640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5440072f80 0x7f5440158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543f400640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54400cf2b0 0x7f54401591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 -- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54400cf2b0 msgr2=0x7f54401591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54400cf2b0 0x7f54401591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 -- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5440072f80 msgr2=0x7f5440158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5440072f80 0x7f5440158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 -- 172.21.15.122:0/660793254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544009cd50 con 0x7f5440070960 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f543ea00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5440070960 0x7f5440138ff0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f543000c680 tx=0x7f543000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5437400640 1 -- 172.21.15.122:0/660793254 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430016020 con 0x7f5440070960 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5437400640 1 -- 172.21.15.122:0/660793254 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f543000a930 con 0x7f5440070960 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544013c810 con 0x7f5440070960 2024-09-17T05:41:50.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5437400640 1 -- 172.21.15.122:0/660793254 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543000ada0 con 0x7f5440070960 2024-09-17T05:41:50.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401f9bf0 con 0x7f5440070960 2024-09-17T05:41:50.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.023+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5440069410 con 0x7f5440070960 2024-09-17T05:41:50.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.027+0000 7f5437400640 1 -- 172.21.15.122:0/660793254 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f5430019020 con 0x7f5440070960 2024-09-17T05:41:50.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.027+0000 7f5437400640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54200759f0 0x7f5420077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:50.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.027+0000 7f5437400640 1 -- 172.21.15.122:0/660793254 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f54300932e0 con 0x7f5440070960 2024-09-17T05:41:50.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.027+0000 7f543e000640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54200759f0 0x7f5420077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:50.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.027+0000 7f543e000640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54200759f0 0x7f5420077eb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f542c000c10 tx=0x7f542c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:50.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.031+0000 7f5437400640 1 -- 172.21.15.122:0/660793254 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5430020030 con 0x7f5440070960 2024-09-17T05:41:50.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.223+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81883", "target": ["mon-mgr", ""]}) -- 0x7f54401fa350 con 0x7f54200759f0 2024-09-17T05:41:50.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.235+0000 7f5437400640 1 -- 172.21.15.122:0/660793254 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f54401fa350 con 0x7f54200759f0 2024-09-17T05:41:50.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54200759f0 msgr2=0x7f5420077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54200759f0 0x7f5420077eb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f542c000c10 tx=0x7f542c009040 comp rx=0 tx=0).stop 2024-09-17T05:41:50.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5440070960 msgr2=0x7f5440138ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5440070960 0x7f5440138ff0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f543000c680 tx=0x7f543000cb40 comp rx=0 tx=0).stop 2024-09-17T05:41:50.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 shutdown_connections 2024-09-17T05:41:50.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f54200759f0 0x7f5420077eb0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54400cf2b0 0x7f54401591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5440072f80 0x7f5440158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 --2- 172.21.15.122:0/660793254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5440070960 0x7f5440138ff0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 >> 172.21.15.122:0/660793254 conn(0x7f54400cfbc0 msgr2=0x7f54401581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:50.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 shutdown_connections 2024-09-17T05:41:50.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.239+0000 7f5445e00640 1 -- 172.21.15.122:0/660793254 wait complete. 2024-09-17T05:41:50.254 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone51580 2024-09-17T05:41:50.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.511+0000 7f9f59000640 1 -- 172.21.15.122:0/650341671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f38004ca0 msgr2=0x7f9f380050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.511+0000 7f9f59000640 1 --2- 172.21.15.122:0/650341671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f38004ca0 0x7f9f380050a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9f44001850 tx=0x7f9f4401c880 comp rx=0 tx=0).stop 2024-09-17T05:41:50.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.511+0000 7f9f59000640 1 -- 172.21.15.122:0/650341671 shutdown_connections 2024-09-17T05:41:50.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.511+0000 7f9f59000640 1 --2- 172.21.15.122:0/650341671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f38004ca0 0x7f9f380050a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.511+0000 7f9f59000640 1 -- 172.21.15.122:0/650341671 >> 172.21.15.122:0/650341671 conn(0x7f9f540d1da0 msgr2=0x7f9f540d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:50.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.511+0000 7f9f59000640 1 -- 172.21.15.122:0/650341671 shutdown_connections 2024-09-17T05:41:50.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.511+0000 7f9f59000640 1 -- 172.21.15.122:0/650341671 wait complete. 2024-09-17T05:41:50.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f59000640 1 Processor -- start 2024-09-17T05:41:50.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f59000640 1 -- start start 2024-09-17T05:41:50.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f59000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f38004ca0 0x7f9f541542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:50.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f59000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f540cd500 0x7f9f540b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:50.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f59000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f540b5340 0x7f9f540b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:50.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f53400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f38004ca0 0x7f9f541542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:50.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f53400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f38004ca0 0x7f9f541542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48916/0 (socket says 172.21.15.122:48916) 2024-09-17T05:41:50.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f53400640 1 -- 172.21.15.122:0/3261450986 learned_addr learned my addr 172.21.15.122:0/3261450986 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:50.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540d0460 con 0x7f9f38004ca0 2024-09-17T05:41:50.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540d02e0 con 0x7f9f540cd500 2024-09-17T05:41:50.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9f540d05e0 con 0x7f9f540b5340 2024-09-17T05:41:50.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.515+0000 7f9f52a00640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f540cd500 0x7f9f540b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:50.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f53e00640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f540b5340 0x7f9f540b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:50.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f52a00640 1 -- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f540b5340 msgr2=0x7f9f540b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f52a00640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f540b5340 0x7f9f540b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f52a00640 1 -- 172.21.15.122:0/3261450986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f38004ca0 msgr2=0x7f9f541542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f52a00640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f38004ca0 0x7f9f541542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f52a00640 1 -- 172.21.15.122:0/3261450986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f540cf5d0 con 0x7f9f540cd500 2024-09-17T05:41:50.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f53400640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f38004ca0 0x7f9f541542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:41:50.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f52a00640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f540cd500 0x7f9f540b4e00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00d5e0 tx=0x7f9f3c00daa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:50.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f4b400640 1 -- 172.21.15.122:0/3261450986 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c017070 con 0x7f9f540cd500 2024-09-17T05:41:50.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f540b5950 con 0x7f9f540cd500 2024-09-17T05:41:50.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f4b400640 1 -- 172.21.15.122:0/3261450986 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f3c005310 con 0x7f9f540cd500 2024-09-17T05:41:50.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f4b400640 1 -- 172.21.15.122:0/3261450986 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c017070 con 0x7f9f540cd500 2024-09-17T05:41:50.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.519+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f540cca20 con 0x7f9f540cd500 2024-09-17T05:41:50.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.523+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5415f580 con 0x7f9f540cd500 2024-09-17T05:41:50.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.523+0000 7f9f4b400640 1 -- 172.21.15.122:0/3261450986 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f9f3c0047b0 con 0x7f9f540cd500 2024-09-17T05:41:50.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.527+0000 7f9f4b400640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f30075bf0 0x7f9f300780b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:50.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.527+0000 7f9f4b400640 1 -- 172.21.15.122:0/3261450986 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7f9f3c092ff0 con 0x7f9f540cd500 2024-09-17T05:41:50.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.527+0000 7f9f53400640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f30075bf0 0x7f9f300780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:50.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.527+0000 7f9f53400640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f30075bf0 0x7f9f300780b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9f4401b150 tx=0x7f9f440196c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:50.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.531+0000 7f9f4b400640 1 -- 172.21.15.122:0/3261450986 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9f3c05f4b0 con 0x7f9f540cd500 2024-09-17T05:41:50.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.923+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone51580", "target": ["mon-mgr", ""]}) -- 0x7f9f54069e60 con 0x7f9f30075bf0 2024-09-17T05:41:50.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.939+0000 7f9f4b400640 1 -- 172.21.15.122:0/3261450986 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f44006150 con 0x7f9f30075bf0 2024-09-17T05:41:50.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f30075bf0 msgr2=0x7f9f300780b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f30075bf0 0x7f9f300780b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9f4401b150 tx=0x7f9f440196c0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f540cd500 msgr2=0x7f9f540b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:50.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f540cd500 0x7f9f540b4e00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00d5e0 tx=0x7f9f3c00daa0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 shutdown_connections 2024-09-17T05:41:50.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9f30075bf0 0x7f9f300780b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f540b5340 0x7f9f540b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f540cd500 0x7f9f540b4e00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 --2- 172.21.15.122:0/3261450986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f38004ca0 0x7f9f541542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:50.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 >> 172.21.15.122:0/3261450986 conn(0x7f9f540d1da0 msgr2=0x7f9f5406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:50.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 shutdown_connections 2024-09-17T05:41:50.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:50.943+0000 7f9f59000640 1 -- 172.21.15.122:0/3261450986 wait complete. 2024-09-17T05:41:50.958 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-17T05:41:51.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:41:51.012 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-17T05:41:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:41:51.019 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-17T05:41:51.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 -- 172.21.15.122:0/3092397162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815c130 msgr2=0x7efc780cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:51.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 --2- 172.21.15.122:0/3092397162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815c130 0x7efc780cd050 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7efc780d0780 tx=0x7efc7001c700 comp rx=0 tx=0).stop 2024-09-17T05:41:51.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 -- 172.21.15.122:0/3092397162 shutdown_connections 2024-09-17T05:41:51.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 --2- 172.21.15.122:0/3092397162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815c130 0x7efc780cd050 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 -- 172.21.15.122:0/3092397162 >> 172.21.15.122:0/3092397162 conn(0x7efc780d1ea0 msgr2=0x7efc780d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:51.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 -- 172.21.15.122:0/3092397162 shutdown_connections 2024-09-17T05:41:51.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 -- 172.21.15.122:0/3092397162 wait complete. 2024-09-17T05:41:51.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 Processor -- start 2024-09-17T05:41:51.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.171+0000 7efc7d400640 1 -- start start 2024-09-17T05:41:51.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc780cecf0 0x7efc780c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815bd50 0x7efc780b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7815c130 0x7efc7814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc7816a5b0 con 0x7efc7815bd50 2024-09-17T05:41:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc7816a430 con 0x7efc7815c130 2024-09-17T05:41:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7efc7816a730 con 0x7efc780cecf0 2024-09-17T05:41:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815bd50 0x7efc780b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815bd50 0x7efc780b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48942/0 (socket says 172.21.15.122:48942) 2024-09-17T05:41:51.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 -- 172.21.15.122:0/2941068737 learned_addr learned my addr 172.21.15.122:0/2941068737 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:51.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc76800640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc780cecf0 0x7efc780c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:51.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc77200640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7815c130 0x7efc7814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:51.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 -- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc780cecf0 msgr2=0x7efc780c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:51.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc780cecf0 0x7efc780c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 -- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7815c130 msgr2=0x7efc7814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:51.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7815c130 0x7efc7814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 -- 172.21.15.122:0/2941068737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc780b5790 con 0x7efc7815bd50 2024-09-17T05:41:51.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6de00640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815bd50 0x7efc780b4cd0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c450 tx=0x7efc6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:51.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6f400640 1 -- 172.21.15.122:0/2941068737 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60012830 con 0x7efc7815bd50 2024-09-17T05:41:51.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6f400640 1 -- 172.21.15.122:0/2941068737 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc60017070 con 0x7efc7815bd50 2024-09-17T05:41:51.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc780b5a70 con 0x7efc7815bd50 2024-09-17T05:41:51.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6f400640 1 -- 172.21.15.122:0/2941068737 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60012d80 con 0x7efc7815bd50 2024-09-17T05:41:51.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc780cbc50 con 0x7efc7815bd50 2024-09-17T05:41:51.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc780b5e60 con 0x7efc7815bd50 2024-09-17T05:41:51.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.175+0000 7efc6f400640 1 -- 172.21.15.122:0/2941068737 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7efc6001c460 con 0x7efc7815bd50 2024-09-17T05:41:51.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.179+0000 7efc6f400640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc540759f0 0x7efc54077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:51.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.179+0000 7efc6f400640 1 -- 172.21.15.122:0/2941068737 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5851+0+0 (secure 0 0 0) 0x7efc60093310 con 0x7efc7815bd50 2024-09-17T05:41:51.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.179+0000 7efc76800640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc540759f0 0x7efc54077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:51.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.179+0000 7efc76800640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc540759f0 0x7efc54077eb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7efc6800f290 tx=0x7efc68001aa0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:51.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.183+0000 7efc6f400640 1 -- 172.21.15.122:0/2941068737 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7efc6009ae20 con 0x7efc7815bd50 2024-09-17T05:41:51.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.443+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7efc780ce5e0 con 0x7efc7815bd50 2024-09-17T05:41:51.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.443+0000 7efc6f400640 1 -- 172.21.15.122:0/2941068737 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7efc6005f7d0 con 0x7efc7815bd50 2024-09-17T05:41:51.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc540759f0 msgr2=0x7efc54077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:51.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc540759f0 0x7efc54077eb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7efc6800f290 tx=0x7efc68001aa0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815bd50 msgr2=0x7efc780b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:51.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815bd50 0x7efc780b4cd0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c450 tx=0x7efc6000c910 comp rx=0 tx=0).stop 2024-09-17T05:41:51.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 shutdown_connections 2024-09-17T05:41:51.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7efc540759f0 0x7efc54077eb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7815c130 0x7efc7814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc7815bd50 0x7efc780b4cd0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 --2- 172.21.15.122:0/2941068737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc780cecf0 0x7efc780c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.447+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 >> 172.21.15.122:0/2941068737 conn(0x7efc780d1ea0 msgr2=0x7efc780709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:51.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.451+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 shutdown_connections 2024-09-17T05:41:51.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.451+0000 7efc7d400640 1 -- 172.21.15.122:0/2941068737 wait complete. 2024-09-17T05:41:51.475 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:41:51.476 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:41:51.598 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:41:51.700 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:41:51.701 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:41:51.701 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:41:51.701 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:51.710 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:41:51.710 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:51.764 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:41:51.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.955+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3898188034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40af120 msgr2=0x7f2ed406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:51.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.955+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3898188034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40af120 0x7f2ed406e960 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009840 tx=0x7f2ebc01f010 comp rx=0 tx=0).stop 2024-09-17T05:41:51.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.955+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3898188034 shutdown_connections 2024-09-17T05:41:51.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.955+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3898188034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ed40707b0 0x7f2ed4072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.955+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3898188034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40af120 0x7f2ed406e960 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.955+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3898188034 >> 172.21.15.122:0/3898188034 conn(0x7f2ed40cfa30 msgr2=0x7f2ed40cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:51.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.955+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3898188034 shutdown_connections 2024-09-17T05:41:51.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.955+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3898188034 wait complete. 2024-09-17T05:41:51.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 Processor -- start 2024-09-17T05:41:51.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 -- start start 2024-09-17T05:41:51.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ed40707b0 0x7f2ed415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:51.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ed40af120 0x7f2ed4152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:51.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40ca950 0x7f2ed4159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:51.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed40739e0 con 0x7f2ed40707b0 2024-09-17T05:41:51.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4073860 con 0x7f2ed40ca950 2024-09-17T05:41:51.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed415eb30 con 0x7f2ed40af120 2024-09-17T05:41:51.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ed40707b0 0x7f2ed415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:51.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ecaa00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ed40af120 0x7f2ed4152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:51.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ed40707b0 0x7f2ed415c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54560/0 (socket says 172.21.15.122:54560) 2024-09-17T05:41:51.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 -- 172.21.15.122:0/3975722211 learned_addr learned my addr 172.21.15.122:0/3975722211 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:51.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3e00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40ca950 0x7f2ed4159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:51.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 -- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ed40af120 msgr2=0x7f2ed4152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:51.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ed40af120 0x7f2ed4152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 -- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40ca950 msgr2=0x7f2ed4159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:51.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40ca950 0x7f2ed4159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:51.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 -- 172.21.15.122:0/3975722211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed415a080 con 0x7f2ed40707b0 2024-09-17T05:41:51.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ecaa00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ed40af120 0x7f2ed4152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:41:51.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3e00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40ca950 0x7f2ed4159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:41:51.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed3400640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ed40707b0 0x7f2ed415c5c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400c430 tx=0x7f2ec400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:51.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ecbe00640 1 -- 172.21.15.122:0/3975722211 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4011790 con 0x7f2ed40707b0 2024-09-17T05:41:51.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ecbe00640 1 -- 172.21.15.122:0/3975722211 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ec4017070 con 0x7f2ed40707b0 2024-09-17T05:41:51.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ecbe00640 1 -- 172.21.15.122:0/3975722211 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4011ce0 con 0x7f2ed40707b0 2024-09-17T05:41:51.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed41f7de0 con 0x7f2ed40707b0 2024-09-17T05:41:51.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.959+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed41f83a0 con 0x7f2ed40707b0 2024-09-17T05:41:51.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.963+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed415d440 con 0x7f2ed40707b0 2024-09-17T05:41:51.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.963+0000 7f2ecbe00640 1 -- 172.21.15.122:0/3975722211 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f2ec401c460 con 0x7f2ed40707b0 2024-09-17T05:41:51.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.963+0000 7f2ecbe00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2eb00759f0 0x7f2eb0077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:51.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.963+0000 7f2ecbe00640 1 -- 172.21.15.122:0/3975722211 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5907+0+0 (secure 0 0 0) 0x7f2ec4093100 con 0x7f2ed40707b0 2024-09-17T05:41:51.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.963+0000 7f2ecaa00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2eb00759f0 0x7f2eb0077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:51.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.963+0000 7f2ecaa00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2eb00759f0 0x7f2eb0077eb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0b2000 tx=0x7f2ebc01d850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:51.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:51.967+0000 7f2ecbe00640 1 -- 172.21.15.122:0/3975722211 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2ec405fd70 con 0x7f2ed40707b0 2024-09-17T05:41:52.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.147+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ed4072d20 con 0x7f2ed40707b0 2024-09-17T05:41:52.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.147+0000 7f2ecbe00640 1 -- 172.21.15.122:0/3975722211 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v148) ==== 80+0+0 (secure 0 0 0) 0x7f2ec405f670 con 0x7f2ed40707b0 2024-09-17T05:41:52.141 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-09-17T05:41:52.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.147+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2eb00759f0 msgr2=0x7f2eb0077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.147+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2eb00759f0 0x7f2eb0077eb0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0b2000 tx=0x7f2ebc01d850 comp rx=0 tx=0).stop 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.147+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ed40707b0 msgr2=0x7f2ed415c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.147+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ed40707b0 0x7f2ed415c5c0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400c430 tx=0x7f2ec400c830 comp rx=0 tx=0).stop 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.151+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 shutdown_connections 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.151+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2eb00759f0 0x7f2eb0077eb0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.151+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ed40ca950 0x7f2ed4159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.151+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ed40af120 0x7f2ed4152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.151+0000 7f2ed8e00640 1 --2- 172.21.15.122:0/3975722211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ed40707b0 0x7f2ed415c5c0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.151+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 >> 172.21.15.122:0/3975722211 conn(0x7f2ed40cfa30 msgr2=0x7f2ed415dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.151+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 shutdown_connections 2024-09-17T05:41:52.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.151+0000 7f2ed8e00640 1 -- 172.21.15.122:0/3975722211 wait complete. 2024-09-17T05:41:52.164 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:41:52.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:52.165 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:41:52.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:41:52.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:52.215 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:41:52.235 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:41:52.236 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:41:52.236 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:52.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:41:52.246 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:41:52.246 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:41:52.246 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:52.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:41:52.291 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:41:52.291 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:41:52.291 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:52.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:41:52.306 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:41:52.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.587+0000 7ff580200640 1 -- 172.21.15.122:0/3285333695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57806e7a0 msgr2=0x7ff57806ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.587+0000 7ff580200640 1 --2- 172.21.15.122:0/3285333695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57806e7a0 0x7ff57806ec30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff570009bc0 tx=0x7ff57001f3b0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.587+0000 7ff580200640 1 -- 172.21.15.122:0/3285333695 shutdown_connections 2024-09-17T05:41:52.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.587+0000 7ff580200640 1 --2- 172.21.15.122:0/3285333695 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57806e7a0 0x7ff57806ec30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.587+0000 7ff580200640 1 --2- 172.21.15.122:0/3285333695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff57815c3e0 0x7ff57815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.587+0000 7ff580200640 1 -- 172.21.15.122:0/3285333695 >> 172.21.15.122:0/3285333695 conn(0x7ff5780b18f0 msgr2=0x7ff5780b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:52.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.587+0000 7ff580200640 1 -- 172.21.15.122:0/3285333695 shutdown_connections 2024-09-17T05:41:52.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.587+0000 7ff580200640 1 -- 172.21.15.122:0/3285333695 wait complete. 2024-09-17T05:41:52.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff580200640 1 Processor -- start 2024-09-17T05:41:52.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff580200640 1 -- start start 2024-09-17T05:41:52.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff580200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff57806e7a0 0x7ff57815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:52.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff580200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57815bfa0 0x7ff5780acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:52.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff580200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815c3e0 0x7ff5781555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:52.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff580200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff57806f2f0 con 0x7ff57815c3e0 2024-09-17T05:41:52.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff580200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff57806f170 con 0x7ff57815bfa0 2024-09-17T05:41:52.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff580200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff57806f470 con 0x7ff57806e7a0 2024-09-17T05:41:52.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.591+0000 7ff57ce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57815bfa0 0x7ff5780acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:52.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57d800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff57806e7a0 0x7ff57815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:52.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815c3e0 0x7ff5781555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:52.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57e200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815c3e0 0x7ff5781555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54568/0 (socket says 172.21.15.122:54568) 2024-09-17T05:41:52.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57e200640 1 -- 172.21.15.122:0/3998309739 learned_addr learned my addr 172.21.15.122:0/3998309739 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57d800640 1 -- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57815bfa0 msgr2=0x7ff5780acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57d800640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57815bfa0 0x7ff5780acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57d800640 1 -- 172.21.15.122:0/3998309739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815c3e0 msgr2=0x7ff5781555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57d800640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815c3e0 0x7ff5781555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57d800640 1 -- 172.21.15.122:0/3998309739 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5780ad280 con 0x7ff57806e7a0 2024-09-17T05:41:52.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57ce00640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57815bfa0 0x7ff5780acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:41:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff57d800640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff57806e7a0 0x7ff57815a880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00e6c0 tx=0x7ff56c00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff575600640 1 -- 172.21.15.122:0/3998309739 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00cab0 con 0x7ff57806e7a0 2024-09-17T05:41:52.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff575600640 1 -- 172.21.15.122:0/3998309739 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff56c00cd40 con 0x7ff57806e7a0 2024-09-17T05:41:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff575600640 1 -- 172.21.15.122:0/3998309739 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c010690 con 0x7ff57806e7a0 2024-09-17T05:41:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5781fc830 con 0x7ff57806e7a0 2024-09-17T05:41:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.595+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5781fcdf0 con 0x7ff57806e7a0 2024-09-17T05:41:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.599+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff57806ec30 con 0x7ff57806e7a0 2024-09-17T05:41:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.603+0000 7ff575600640 1 -- 172.21.15.122:0/3998309739 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7ff56c010830 con 0x7ff57806e7a0 2024-09-17T05:41:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.603+0000 7ff575600640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff558075ac0 0x7ff558077f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.603+0000 7ff575600640 1 -- 172.21.15.122:0/3998309739 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(148..148 src has 1..148) ==== 5907+0+0 (secure 0 0 0) 0x7ff56c093db0 con 0x7ff57806e7a0 2024-09-17T05:41:52.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.603+0000 7ff57ce00640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff558075ac0 0x7ff558077f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:52.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.607+0000 7ff57ce00640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff558075ac0 0x7ff558077f80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff57815b970 tx=0x7ff564009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:52.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.607+0000 7ff575600640 1 -- 172.21.15.122:0/3998309739 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7ff56c0602f0 con 0x7ff57806e7a0 2024-09-17T05:41:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5780cd070 con 0x7ff57806e7a0 2024-09-17T05:41:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff575600640 1 -- 172.21.15.122:0/3998309739 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7ff56c064d20 con 0x7ff57806e7a0 2024-09-17T05:41:52.805 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:52.805 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":97,"btime":"2024-09-17T05:40:37:216813+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/223748651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":223748651},{"type":"v1","addr":"172.21.15.173:6837","nonce":223748651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/822031950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":822031950},{"type":"v1","addr":"172.21.15.173:6836","nonce":822031950}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2541492660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2541492660},{"type":"v1","addr":"172.21.15.122:6834","nonce":2541492660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:40:24.548833+0000","modified":"2024-09-17T05:40:27.570670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9399":{"gid":9399,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/4149068254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":4149068254},{"type":"v1","addr":"172.21.15.122:6835","nonce":4149068254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9399,"qdb_cluster":[9399]},"id":10}]} 2024-09-17T05:41:52.805 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 97 2024-09-17T05:41:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff558075ac0 msgr2=0x7ff558077f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff558075ac0 0x7ff558077f80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff57815b970 tx=0x7ff564009040 comp rx=0 tx=0).stop 2024-09-17T05:41:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff57806e7a0 msgr2=0x7ff57815a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff57806e7a0 0x7ff57815a880 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00e6c0 tx=0x7ff56c00eb80 comp rx=0 tx=0).stop 2024-09-17T05:41:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 shutdown_connections 2024-09-17T05:41:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7ff558075ac0 0x7ff558077f80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff57815c3e0 0x7ff5781555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff57815bfa0 0x7ff5780acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 --2- 172.21.15.122:0/3998309739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff57806e7a0 0x7ff57815a880 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 >> 172.21.15.122:0/3998309739 conn(0x7ff5780b18f0 msgr2=0x7ff5780cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 shutdown_connections 2024-09-17T05:41:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.811+0000 7ff580200640 1 -- 172.21.15.122:0/3998309739 wait complete. 2024-09-17T05:41:52.828 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:41:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.975+0000 7f89e3200640 1 -- 172.21.15.122:0/3197410114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 msgr2=0x7f89e40cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.975+0000 7f89e3200640 1 --2- 172.21.15.122:0/3197410114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 0x7f89e40cc9a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f89d4009840 tx=0x7f89d401c790 comp rx=0 tx=0).stop 2024-09-17T05:41:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.975+0000 7f89e3200640 1 -- 172.21.15.122:0/3197410114 shutdown_connections 2024-09-17T05:41:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.975+0000 7f89e3200640 1 --2- 172.21.15.122:0/3197410114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 0x7f89e40cc9a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.975+0000 7f89e3200640 1 -- 172.21.15.122:0/3197410114 >> 172.21.15.122:0/3197410114 conn(0x7f89e40742d0 msgr2=0x7f89e40746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.975+0000 7f89e3200640 1 -- 172.21.15.122:0/3197410114 shutdown_connections 2024-09-17T05:41:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.975+0000 7f89e3200640 1 -- 172.21.15.122:0/3197410114 wait complete. 2024-09-17T05:41:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.975+0000 7f89e3200640 1 Processor -- start 2024-09-17T05:41:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 -- start start 2024-09-17T05:41:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e4070240 0x7f89e4157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89e40cb360 0x7f89e40c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 0x7f89e40cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89e4169010 con 0x7f89e4070240 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89e4168e90 con 0x7f89e415f2e0 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f89e4169190 con 0x7f89e40cb360 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e4070240 0x7f89e4157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e4070240 0x7f89e4157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54584/0 (socket says 172.21.15.122:54584) 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 -- 172.21.15.122:0/1814615069 learned_addr learned my addr 172.21.15.122:0/1814615069 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 -- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89e40cb360 msgr2=0x7f89e40c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89e40cb360 0x7f89e40c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 -- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 msgr2=0x7f89e40cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e2800640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 0x7f89e40cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 0x7f89e40cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 -- 172.21.15.122:0/1814615069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e40c6cf0 con 0x7f89e4070240 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e2800640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 0x7f89e40cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e1e00640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e4070240 0x7f89e4157d60 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f89d401a3c0 tx=0x7f89d4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89da000640 1 -- 172.21.15.122:0/1814615069 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d40b2040 con 0x7f89e4070240 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89da000640 1 -- 172.21.15.122:0/1814615069 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89d40042f0 con 0x7f89e4070240 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e415b280 con 0x7f89e4070240 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89da000640 1 -- 172.21.15.122:0/1814615069 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d40052d0 con 0x7f89e4070240 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e4157560 con 0x7f89e4070240 2024-09-17T05:41:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.979+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e40cb780 con 0x7f89e4070240 2024-09-17T05:41:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.983+0000 7f89da000640 1 -- 172.21.15.122:0/1814615069 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f89d40a56e0 con 0x7f89e4070240 2024-09-17T05:41:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.983+0000 7f89da000640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89c00759f0 0x7f89c0077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.983+0000 7f89da000640 1 -- 172.21.15.122:0/1814615069 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5907+0+0 (secure 0 0 0) 0x7f89d412d200 con 0x7f89e4070240 2024-09-17T05:41:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.983+0000 7f89dbe00640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89c00759f0 0x7f89c0077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.983+0000 7f89dbe00640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89c00759f0 0x7f89c0077eb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f89cc007fd0 tx=0x7f89cc007980 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:52.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:52.987+0000 7f89da000640 1 -- 172.21.15.122:0/1814615069 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f89d4016410 con 0x7f89e4070240 2024-09-17T05:41:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.191+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89e4075100 con 0x7f89e4070240 2024-09-17T05:41:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.191+0000 7f89da000640 1 -- 172.21.15.122:0/1814615069 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7f89d40f9db0 con 0x7f89e4070240 2024-09-17T05:41:53.189 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:53.189 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":97,"btime":"2024-09-17T05:40:37:216813+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/223748651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":223748651},{"type":"v1","addr":"172.21.15.173:6837","nonce":223748651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/822031950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":822031950},{"type":"v1","addr":"172.21.15.173:6836","nonce":822031950}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2541492660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2541492660},{"type":"v1","addr":"172.21.15.122:6834","nonce":2541492660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:40:24.548833+0000","modified":"2024-09-17T05:40:27.570670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9399":{"gid":9399,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/4149068254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":4149068254},{"type":"v1","addr":"172.21.15.122:6835","nonce":4149068254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9399,"qdb_cluster":[9399]},"id":10}]} 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 97 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89c00759f0 msgr2=0x7f89c0077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89c00759f0 0x7f89c0077eb0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f89cc007fd0 tx=0x7f89cc007980 comp rx=0 tx=0).stop 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e4070240 msgr2=0x7f89e4157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e4070240 0x7f89e4157d60 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f89d401a3c0 tx=0x7f89d4005db0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 shutdown_connections 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f89c00759f0 0x7f89c0077eb0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89e415f2e0 0x7f89e40cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89e40cb360 0x7f89e40c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 --2- 172.21.15.122:0/1814615069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89e4070240 0x7f89e4157d60 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 >> 172.21.15.122:0/1814615069 conn(0x7f89e40742d0 msgr2=0x7f89e40b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 shutdown_connections 2024-09-17T05:41:53.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.195+0000 7f89e3200640 1 -- 172.21.15.122:0/1814615069 wait complete. 2024-09-17T05:41:53.210 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:41:53.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.423+0000 7f56c3000640 1 -- 172.21.15.122:0/1525585388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc159cc0 msgr2=0x7f56bc072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.423+0000 7f56c3000640 1 --2- 172.21.15.122:0/1525585388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc159cc0 0x7f56bc072680 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f56b00074e0 tx=0x7f56b001d570 comp rx=0 tx=0).stop 2024-09-17T05:41:53.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.423+0000 7f56c3000640 1 -- 172.21.15.122:0/1525585388 shutdown_connections 2024-09-17T05:41:53.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.423+0000 7f56c3000640 1 --2- 172.21.15.122:0/1525585388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc0cd600 0x7f56bc0cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.423+0000 7f56c3000640 1 --2- 172.21.15.122:0/1525585388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc159cc0 0x7f56bc072680 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.423+0000 7f56c3000640 1 -- 172.21.15.122:0/1525585388 >> 172.21.15.122:0/1525585388 conn(0x7f56bc0cfa30 msgr2=0x7f56bc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:53.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.423+0000 7f56c3000640 1 -- 172.21.15.122:0/1525585388 shutdown_connections 2024-09-17T05:41:53.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.423+0000 7f56c3000640 1 -- 172.21.15.122:0/1525585388 wait complete. 2024-09-17T05:41:53.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.427+0000 7f56c3000640 1 Processor -- start 2024-09-17T05:41:53.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.427+0000 7f56c3000640 1 -- start start 2024-09-17T05:41:53.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c3000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc0cd600 0x7f56bc1ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:53.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c3000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc0cf570 0x7f56bc0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:53.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c3000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc152550 0x7f56bc0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:53.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c3000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc06f610 con 0x7f56bc152550 2024-09-17T05:41:53.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c3000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc06f490 con 0x7f56bc0cf570 2024-09-17T05:41:53.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c3000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f56bc06f790 con 0x7f56bc0cd600 2024-09-17T05:41:53.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc152550 0x7f56bc0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:53.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc152550 0x7f56bc0cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54602/0 (socket says 172.21.15.122:54602) 2024-09-17T05:41:53.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 -- 172.21.15.122:0/3016202788 learned_addr learned my addr 172.21.15.122:0/3016202788 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:53.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c1c00640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc0cd600 0x7f56bc1ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c1200640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc0cf570 0x7f56bc0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 -- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc0cd600 msgr2=0x7f56bc1ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc0cd600 0x7f56bc1ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 -- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc0cf570 msgr2=0x7f56bc0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc0cf570 0x7f56bc0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 -- 172.21.15.122:0/3016202788 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc1ffe70 con 0x7f56bc152550 2024-09-17T05:41:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c1200640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc0cf570 0x7f56bc0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:41:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c1c00640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc0cd600 0x7f56bc1ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:41:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56c2600640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc152550 0x7f56bc0cc9f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f56b400ea20 tx=0x7f56b400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56b9600640 1 -- 172.21.15.122:0/3016202788 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b400cde0 con 0x7f56bc152550 2024-09-17T05:41:53.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56b9600640 1 -- 172.21.15.122:0/3016202788 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b4004c40 con 0x7f56bc152550 2024-09-17T05:41:53.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.431+0000 7f56b9600640 1 -- 172.21.15.122:0/3016202788 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4005280 con 0x7f56bc152550 2024-09-17T05:41:53.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.435+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc2000f0 con 0x7f56bc152550 2024-09-17T05:41:53.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.435+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc2006b0 con 0x7f56bc152550 2024-09-17T05:41:53.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.435+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc06fd00 con 0x7f56bc152550 2024-09-17T05:41:53.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.439+0000 7f56b9600640 1 -- 172.21.15.122:0/3016202788 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f56b4005420 con 0x7f56bc152550 2024-09-17T05:41:53.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.439+0000 7f56b9600640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0759f0 0x7f569c077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.439+0000 7f56b9600640 1 -- 172.21.15.122:0/3016202788 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5907+0+0 (secure 0 0 0) 0x7f56b4094060 con 0x7f56bc152550 2024-09-17T05:41:53.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.439+0000 7f56c1c00640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0759f0 0x7f569c077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.443+0000 7f56c1c00640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0759f0 0x7f569c077eb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f56b00af000 tx=0x7f56b001bdb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:53.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.447+0000 7f56b9600640 1 -- 172.21.15.122:0/3016202788 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f56bc1ffe70 con 0x7f56bc152550 2024-09-17T05:41:53.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.675+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56bc0c9cc0 con 0x7f56bc152550 2024-09-17T05:41:53.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.675+0000 7f56b9600640 1 -- 172.21.15.122:0/3016202788 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+17140 (secure 0 0 0) 0x7f56b4016030 con 0x7f56bc152550 2024-09-17T05:41:53.670 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:53.671 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":148,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:41:51.311072+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:40:22.246801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T05:40:23.519850+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.16","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:41:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0759f0 msgr2=0x7f569c077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0759f0 0x7f569c077eb0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f56b00af000 tx=0x7f56b001bdb0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc152550 msgr2=0x7f56bc0cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc152550 0x7f56bc0cc9f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f56b400ea20 tx=0x7f56b400eee0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 shutdown_connections 2024-09-17T05:41:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f569c0759f0 0x7f569c077eb0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56bc152550 0x7f56bc0cc9f0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56bc0cf570 0x7f56bc0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 --2- 172.21.15.122:0/3016202788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56bc0cd600 0x7f56bc1ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 >> 172.21.15.122:0/3016202788 conn(0x7f56bc0cfa30 msgr2=0x7f56bc0af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:53.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 shutdown_connections 2024-09-17T05:41:53.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.679+0000 7f56c3000640 1 -- 172.21.15.122:0/3016202788 wait complete. 2024-09-17T05:41:53.693 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:41:53.693 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:41:53.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.839+0000 7f7197e00640 1 -- 172.21.15.122:0/1985465042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71900b49e0 msgr2=0x7f719006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.839+0000 7f7197e00640 1 --2- 172.21.15.122:0/1985465042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71900b49e0 0x7f719006f7b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f718800ad90 tx=0x7f718801dc20 comp rx=0 tx=0).stop 2024-09-17T05:41:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.839+0000 7f7197e00640 1 -- 172.21.15.122:0/1985465042 shutdown_connections 2024-09-17T05:41:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.839+0000 7f7197e00640 1 --2- 172.21.15.122:0/1985465042 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71900b49e0 0x7f719006f7b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.839+0000 7f7197e00640 1 --2- 172.21.15.122:0/1985465042 >> v2:172.21.15.173:3300/0 conn(0x7f719015dae0 0x7f71900acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.839+0000 7f7197e00640 1 -- 172.21.15.122:0/1985465042 >> 172.21.15.122:0/1985465042 conn(0x7f71900b18f0 msgr2=0x7f71900b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.839+0000 7f7197e00640 1 -- 172.21.15.122:0/1985465042 shutdown_connections 2024-09-17T05:41:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.839+0000 7f7197e00640 1 -- 172.21.15.122:0/1985465042 wait complete. 2024-09-17T05:41:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7197e00640 1 Processor -- start 2024-09-17T05:41:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7197e00640 1 -- start start 2024-09-17T05:41:53.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7197e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71900b49e0 0x7f71901585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:53.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7197e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71900cd100 0x7f719015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7197e00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719015dae0 0x7f7190155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7197e00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71900cf1c0 con 0x7f71900cd100 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7197e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f71900cf040 con 0x7f71900b49e0 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7197e00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f71900cf340 con 0x7f719015dae0 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7194a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71900cd100 0x7f719015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7194a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71900cd100 0x7f719015a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54616/0 (socket says 172.21.15.122:54616) 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7194a00640 1 -- 172.21.15.122:0/2697614272 learned_addr learned my addr 172.21.15.122:0/2697614272 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7195e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719015dae0 0x7f7190155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7195400640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71900b49e0 0x7f71901585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:53.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7195e00640 1 -- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71900b49e0 msgr2=0x7f71901585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7195e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71900b49e0 0x7f71901585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7195e00640 1 -- 172.21.15.122:0/2697614272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71900cd100 msgr2=0x7f719015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:53.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7195e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71900cd100 0x7f719015a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:53.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7195e00640 1 -- 172.21.15.122:0/2697614272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71901553c0 con 0x7f719015dae0 2024-09-17T05:41:53.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7194a00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71900cd100 0x7f719015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:41:53.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f7195e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719015dae0 0x7f7190155fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71880ad000 tx=0x7f718800a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:53.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f718d600640 1 -- 172.21.15.122:0/2697614272 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71880bb020 con 0x7f719015dae0 2024-09-17T05:41:53.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f718d600640 1 -- 172.21.15.122:0/2697614272 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7188004a50 con 0x7f719015dae0 2024-09-17T05:41:53.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.843+0000 7f718d600640 1 -- 172.21.15.122:0/2697614272 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71880b3040 con 0x7f719015dae0 2024-09-17T05:41:53.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.847+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71901556a0 con 0x7f719015dae0 2024-09-17T05:41:53.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.847+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7190157db0 con 0x7f719015dae0 2024-09-17T05:41:53.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.847+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71900acd40 con 0x7f719015dae0 2024-09-17T05:41:53.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.847+0000 7f718d600640 1 -- 172.21.15.122:0/2697614272 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f7188004000 con 0x7f719015dae0 2024-09-17T05:41:53.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.847+0000 7f718d600640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71700759f0 0x7f7170077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:53.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.847+0000 7f718d600640 1 -- 172.21.15.122:0/2697614272 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(148..148 src has 1..148) ==== 5907+0+0 (secure 0 0 0) 0x7f718812d720 con 0x7f719015dae0 2024-09-17T05:41:53.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.847+0000 7f7195400640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71700759f0 0x7f7170077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:53.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.851+0000 7f7195400640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71700759f0 0x7f7170077eb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7184009600 tx=0x7f71840091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:53.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:53.851+0000 7f718d600640 1 -- 172.21.15.122:0/2697614272 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f71880c5060 con 0x7f719015dae0 2024-09-17T05:41:54.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.051+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7190158240 con 0x7f719015dae0 2024-09-17T05:41:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f718d600640 1 -- 172.21.15.122:0/2697614272 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7f71880178a0 con 0x7f719015dae0 2024-09-17T05:41:54.048 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":97,"btime":"2024-09-17T05:40:37:216813+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/223748651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":223748651},{"type":"v1","addr":"172.21.15.173:6837","nonce":223748651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/822031950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":822031950},{"type":"v1","addr":"172.21.15.173:6836","nonce":822031950}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2541492660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2541492660},{"type":"v1","addr":"172.21.15.122:6834","nonce":2541492660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:40:24.548833+0000","modified":"2024-09-17T05:40:27.570670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9399":{"gid":9399,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/4149068254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":4149068254},{"type":"v1","addr":"172.21.15.122:6835","nonce":4149068254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9399,"qdb_cluster":[9399]},"id":10}]} 2024-09-17T05:41:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 97 2024-09-17T05:41:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71700759f0 msgr2=0x7f7170077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71700759f0 0x7f7170077eb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7184009600 tx=0x7f71840091e0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719015dae0 msgr2=0x7f7190155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719015dae0 0x7f7190155fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71880ad000 tx=0x7f718800a440 comp rx=0 tx=0).stop 2024-09-17T05:41:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 shutdown_connections 2024-09-17T05:41:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f71700759f0 0x7f7170077eb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719015dae0 0x7f7190155fd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71900cd100 0x7f719015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 --2- 172.21.15.122:0/2697614272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71900b49e0 0x7f71901585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 >> 172.21.15.122:0/2697614272 conn(0x7f71900b18f0 msgr2=0x7f7190150720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:54.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 shutdown_connections 2024-09-17T05:41:54.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.055+0000 7f7197e00640 1 -- 172.21.15.122:0/2697614272 wait complete. 2024-09-17T05:41:54.069 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:41:54.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 -- 172.21.15.122:0/563256199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b4be0 msgr2=0x7f51a80738f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 --2- 172.21.15.122:0/563256199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b4be0 0x7f51a80738f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f51a80749f0 tx=0x7f51a001cac0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 -- 172.21.15.122:0/563256199 shutdown_connections 2024-09-17T05:41:54.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 --2- 172.21.15.122:0/563256199 >> v2:172.21.15.173:3300/0 conn(0x7f51a80b3dc0 0x7f51a80b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 --2- 172.21.15.122:0/563256199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b4be0 0x7f51a80738f0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 --2- 172.21.15.122:0/563256199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a815a980 0x7f51a80d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 -- 172.21.15.122:0/563256199 >> 172.21.15.122:0/563256199 conn(0x7f51a80b2890 msgr2=0x7f51a80b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:54.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 -- 172.21.15.122:0/563256199 shutdown_connections 2024-09-17T05:41:54.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.215+0000 7f51ada00640 1 -- 172.21.15.122:0/563256199 wait complete. 2024-09-17T05:41:54.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 Processor -- start 2024-09-17T05:41:54.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 -- start start 2024-09-17T05:41:54.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b3dc0 0x7f51a814f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a80b4be0 0x7f51a81f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a815a980 0x7f51a81faa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8070910 con 0x7f51a80b3dc0 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8070790 con 0x7f51a815a980 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f51a8070a90 con 0x7f51a80b4be0 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b3dc0 0x7f51a814f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b3dc0 0x7f51a814f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54624/0 (socket says 172.21.15.122:54624) 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 -- 172.21.15.122:0/2603897175 learned_addr learned my addr 172.21.15.122:0/2603897175 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a7400640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a815a980 0x7f51a81faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f519e000640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a80b4be0 0x7f51a81f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 -- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a80b4be0 msgr2=0x7f51a81f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a80b4be0 0x7f51a81f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 -- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a815a980 msgr2=0x7f51a81faa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a815a980 0x7f51a81faa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 -- 172.21.15.122:0/2603897175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a81fb310 con 0x7f51a80b3dc0 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51a6a00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b3dc0 0x7f51a814f610 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f519800d980 tx=0x7f519800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f519f400640 1 -- 172.21.15.122:0/2603897175 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198017070 con 0x7f51a80b3dc0 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f519f400640 1 -- 172.21.15.122:0/2603897175 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5198009d20 con 0x7f51a80b3dc0 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f519f400640 1 -- 172.21.15.122:0/2603897175 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198004280 con 0x7f51a80b3dc0 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a81fb5f0 con 0x7f51a80b3dc0 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.219+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a81f7880 con 0x7f51a80b3dc0 2024-09-17T05:41:54.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.223+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a80738f0 con 0x7f51a80b3dc0 2024-09-17T05:41:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.223+0000 7f519f400640 1 -- 172.21.15.122:0/2603897175 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f5198005020 con 0x7f51a80b3dc0 2024-09-17T05:41:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.223+0000 7f519f400640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51840759f0 0x7f5184077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.223+0000 7f519f400640 1 -- 172.21.15.122:0/2603897175 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5907+0+0 (secure 0 0 0) 0x7f51980937c0 con 0x7f51a80b3dc0 2024-09-17T05:41:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.223+0000 7f519e000640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51840759f0 0x7f5184077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.223+0000 7f519e000640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51840759f0 0x7f5184077eb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f51a8165780 tx=0x7f519000b410 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:54.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.227+0000 7f519f400640 1 -- 172.21.15.122:0/2603897175 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5198020020 con 0x7f51a80b3dc0 2024-09-17T05:41:54.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.431+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51a809efe0 con 0x7f51a80b3dc0 2024-09-17T05:41:54.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.431+0000 7f519f400640 1 -- 172.21.15.122:0/2603897175 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5233 (secure 0 0 0) 0x7f519805fc80 con 0x7f51a80b3dc0 2024-09-17T05:41:54.426 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:54.426 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":97,"btime":"2024-09-17T05:40:37:216813+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/223748651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":223748651},{"type":"v1","addr":"172.21.15.173:6837","nonce":223748651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/822031950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":822031950},{"type":"v1","addr":"172.21.15.173:6836","nonce":822031950}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2541492660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2541492660},{"type":"v1","addr":"172.21.15.122:6834","nonce":2541492660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:40:24.548833+0000","modified":"2024-09-17T05:40:27.570670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9399":{"gid":9399,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6835/4149068254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":4149068254},{"type":"v1","addr":"172.21.15.122:6835","nonce":4149068254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9399,"qdb_cluster":[9399]},"id":10}]} 2024-09-17T05:41:54.426 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 97 2024-09-17T05:41:54.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.431+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51840759f0 msgr2=0x7f5184077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.431+0000 7f51ada00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51840759f0 0x7f5184077eb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f51a8165780 tx=0x7f519000b410 comp rx=0 tx=0).stop 2024-09-17T05:41:54.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.431+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b3dc0 msgr2=0x7f51a814f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.431+0000 7f51ada00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b3dc0 0x7f51a814f610 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f519800d980 tx=0x7f519800de40 comp rx=0 tx=0).stop 2024-09-17T05:41:54.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.435+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 shutdown_connections 2024-09-17T05:41:54.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.435+0000 7f51ada00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f51840759f0 0x7f5184077eb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.435+0000 7f51ada00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51a815a980 0x7f51a81faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.435+0000 7f51ada00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f51a80b4be0 0x7f51a81f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.435+0000 7f51ada00640 1 --2- 172.21.15.122:0/2603897175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a80b3dc0 0x7f51a814f610 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.435+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 >> 172.21.15.122:0/2603897175 conn(0x7f51a80b2890 msgr2=0x7f51a815d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:54.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.435+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 shutdown_connections 2024-09-17T05:41:54.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.435+0000 7f51ada00640 1 -- 172.21.15.122:0/2603897175 wait complete. 2024-09-17T05:41:54.447 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:41:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 -- 172.21.15.122:0/295706282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f894015db50 msgr2=0x7f89400acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 --2- 172.21.15.122:0/295706282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f894015db50 0x7f89400acd40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8930009870 tx=0x7f893001c6f0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 -- 172.21.15.122:0/295706282 shutdown_connections 2024-09-17T05:41:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 --2- 172.21.15.122:0/295706282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f894015db50 0x7f89400acd40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 -- 172.21.15.122:0/295706282 >> 172.21.15.122:0/295706282 conn(0x7f89400b18f0 msgr2=0x7f89400b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 -- 172.21.15.122:0/295706282 shutdown_connections 2024-09-17T05:41:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 -- 172.21.15.122:0/295706282 wait complete. 2024-09-17T05:41:54.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 Processor -- start 2024-09-17T05:41:54.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 -- start start 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89400b4a50 0x7f8940140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89400cd100 0x7f89400b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.595+0000 7f8946200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f894015db50 0x7f89401557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8946200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8940168130 con 0x7f894015db50 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8946200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89400cccd0 con 0x7f89400b4a50 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8946200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f89401682b0 con 0x7f89400cd100 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f894015db50 0x7f89401557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f894015db50 0x7f89401557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54626/0 (socket says 172.21.15.122:54626) 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 -- 172.21.15.122:0/3195074577 learned_addr learned my addr 172.21.15.122:0/3195074577 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893f400640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89400b4a50 0x7f8940140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8936a00640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89400cd100 0x7f89400b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 -- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89400cd100 msgr2=0x7f89400b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89400cd100 0x7f89400b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 -- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89400b4a50 msgr2=0x7f8940140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89400b4a50 0x7f8940140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 -- 172.21.15.122:0/3195074577 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89400b0440 con 0x7f894015db50 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893f400640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89400b4a50 0x7f8940140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f893fe00640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f894015db50 0x7f89401557c0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f893800cfd0 tx=0x7f8938002000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8937e00640 1 -- 172.21.15.122:0/3195074577 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938016020 con 0x7f894015db50 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8937e00640 1 -- 172.21.15.122:0/3195074577 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f893800ab40 con 0x7f894015db50 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8937e00640 1 -- 172.21.15.122:0/3195074577 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938012420 con 0x7f894015db50 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8940153320 con 0x7f894015db50 2024-09-17T05:41:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89400b09c0 con 0x7f894015db50 2024-09-17T05:41:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8940069410 con 0x7f894015db50 2024-09-17T05:41:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.599+0000 7f8937e00640 1 -- 172.21.15.122:0/3195074577 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f8938019020 con 0x7f894015db50 2024-09-17T05:41:54.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.603+0000 7f8937e00640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f891c0759f0 0x7f891c077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.603+0000 7f8937e00640 1 -- 172.21.15.122:0/3195074577 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5907+0+0 (secure 0 0 0) 0x7f8938093930 con 0x7f894015db50 2024-09-17T05:41:54.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.603+0000 7f893f400640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f891c0759f0 0x7f891c077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.603+0000 7f893f400640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f891c0759f0 0x7f891c077eb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f89300af000 tx=0x7f893001af30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.607+0000 7f8937e00640 1 -- 172.21.15.122:0/3195074577 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f8938020020 con 0x7f894015db50 2024-09-17T05:41:54.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.775+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89400adec0 con 0x7f894015db50 2024-09-17T05:41:54.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.775+0000 7f8937e00640 1 -- 172.21.15.122:0/3195074577 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+17140 (secure 0 0 0) 0x7f8938060440 con 0x7f894015db50 2024-09-17T05:41:54.771 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:41:54.771 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":148,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:41:51.311072+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:40:22.246801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-17T05:40:23.519850+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.16","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:41:54.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f891c0759f0 msgr2=0x7f891c077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f891c0759f0 0x7f891c077eb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f89300af000 tx=0x7f893001af30 comp rx=0 tx=0).stop 2024-09-17T05:41:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f894015db50 msgr2=0x7f89401557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f894015db50 0x7f89401557c0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f893800cfd0 tx=0x7f8938002000 comp rx=0 tx=0).stop 2024-09-17T05:41:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 shutdown_connections 2024-09-17T05:41:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f891c0759f0 0x7f891c077eb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f894015db50 0x7f89401557c0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89400cd100 0x7f89400b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 --2- 172.21.15.122:0/3195074577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89400b4a50 0x7f8940140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 >> 172.21.15.122:0/3195074577 conn(0x7f89400b18f0 msgr2=0x7f89400b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:54.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 shutdown_connections 2024-09-17T05:41:54.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.779+0000 7f8946200640 1 -- 172.21.15.122:0/3195074577 wait complete. 2024-09-17T05:41:54.793 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:41:54.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- 172.21.15.122:0/1791824253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7710003a10 msgr2=0x7f7710003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 --2- 172.21.15.122:0/1791824253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7710003a10 0x7f7710003df0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f771c001770 tx=0x7f771c01c820 comp rx=0 tx=0).stop 2024-09-17T05:41:54.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- 172.21.15.122:0/1791824253 shutdown_connections 2024-09-17T05:41:54.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 --2- 172.21.15.122:0/1791824253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7710003a10 0x7f7710003df0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- 172.21.15.122:0/1791824253 >> 172.21.15.122:0/1791824253 conn(0x7f772c0b18f0 msgr2=0x7f772c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:54.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- 172.21.15.122:0/1791824253 shutdown_connections 2024-09-17T05:41:54.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- 172.21.15.122:0/1791824253 wait complete. 2024-09-17T05:41:54.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 Processor -- start 2024-09-17T05:41:54.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- start start 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7710003a10 0x7f772c1ff150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f772c0addc0 0x7f772c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f772c09b810 0x7f772c09bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f772c0cc8e0 con 0x7f772c09b810 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f772c0cc760 con 0x7f7710003a10 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f7732000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f772c0cca60 con 0x7f772c0addc0 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f772c09b810 0x7f772c09bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f772c09b810 0x7f772c09bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54648/0 (socket says 172.21.15.122:54648) 2024-09-17T05:41:54.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772be00640 1 -- 172.21.15.122:0/915827096 learned_addr learned my addr 172.21.15.122:0/915827096 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772aa00640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f772c0addc0 0x7f772c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772b400640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7710003a10 0x7f772c1ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772be00640 1 -- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f772c0addc0 msgr2=0x7f772c0ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772be00640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f772c0addc0 0x7f772c0ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772be00640 1 -- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7710003a10 msgr2=0x7f772c1ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772be00640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7710003a10 0x7f772c1ff150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.943+0000 7f772be00640 1 -- 172.21.15.122:0/915827096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c1fc700 con 0x7f772c09b810 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f772be00640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f772c09b810 0x7f772c09bc90 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f772c0cbe80 tx=0x7f7724010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7723400640 1 -- 172.21.15.122:0/915827096 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724004b30 con 0x7f772c09b810 2024-09-17T05:41:54.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7723400640 1 -- 172.21.15.122:0/915827096 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7724015070 con 0x7f772c09b810 2024-09-17T05:41:54.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7723400640 1 -- 172.21.15.122:0/915827096 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724003be0 con 0x7f772c09b810 2024-09-17T05:41:54.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c1fc9e0 con 0x7f772c09b810 2024-09-17T05:41:54.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c1fc3d0 con 0x7f772c09b810 2024-09-17T05:41:54.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772c0ccfc0 con 0x7f772c09b810 2024-09-17T05:41:54.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7723400640 1 -- 172.21.15.122:0/915827096 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f7724004030 con 0x7f772c09b810 2024-09-17T05:41:54.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7723400640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f770c0759f0 0x7f770c077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:54.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f7723400640 1 -- 172.21.15.122:0/915827096 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5907+0+0 (secure 0 0 0) 0x7f7724093d10 con 0x7f772c09b810 2024-09-17T05:41:54.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.947+0000 7f772b400640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f770c0759f0 0x7f770c077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:54.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.951+0000 7f772b400640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f770c0759f0 0x7f770c077eb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f771c0a9000 tx=0x7f771c0195c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:54.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:54.951+0000 7f7723400640 1 -- 172.21.15.122:0/915827096 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f7724060820 con 0x7f772c09b810 2024-09-17T05:41:55.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.151+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f772c15ad20 con 0x7f772c09b810 2024-09-17T05:41:55.147 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-17T05:41:55.151+0000 7ff1f0000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:41:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.155+0000 7f7723400640 1 -- 172.21.15.122:0/915827096 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) ==== 161+0+0 (secure 0 0 0) 0x7f7724060120 con 0x7f772c09b810 2024-09-17T05:41:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:41:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f770c0759f0 msgr2=0x7f770c077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f770c0759f0 0x7f770c077eb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f771c0a9000 tx=0x7f771c0195c0 comp rx=0 tx=0).stop 2024-09-17T05:41:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f772c09b810 msgr2=0x7f772c09bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f772c09b810 0x7f772c09bc90 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f772c0cbe80 tx=0x7f7724010f40 comp rx=0 tx=0).stop 2024-09-17T05:41:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 shutdown_connections 2024-09-17T05:41:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f770c0759f0 0x7f770c077eb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f772c09b810 0x7f772c09bc90 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f772c0addc0 0x7f772c0ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 --2- 172.21.15.122:0/915827096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7710003a10 0x7f772c1ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 >> 172.21.15.122:0/915827096 conn(0x7f772c0b18f0 msgr2=0x7f772c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 shutdown_connections 2024-09-17T05:41:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.159+0000 7f7732000640 1 -- 172.21.15.122:0/915827096 wait complete. 2024-09-17T05:41:55.175 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:41:55.219 INFO:tasks.ceph.mds.c.smithi122.stdout:starting mds.c at 2024-09-17T05:41:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 -- 172.21.15.122:0/1893981443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 msgr2=0x7f69b80ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 --2- 172.21.15.122:0/1893981443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 0x7f69b80ce810 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f69b000acd0 tx=0x7f69b001de60 comp rx=0 tx=0).stop 2024-09-17T05:41:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 -- 172.21.15.122:0/1893981443 shutdown_connections 2024-09-17T05:41:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 --2- 172.21.15.122:0/1893981443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 0x7f69b80ce810 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 -- 172.21.15.122:0/1893981443 >> 172.21.15.122:0/1893981443 conn(0x7f69b80cfc20 msgr2=0x7f69b80d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 -- 172.21.15.122:0/1893981443 shutdown_connections 2024-09-17T05:41:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 -- 172.21.15.122:0/1893981443 wait complete. 2024-09-17T05:41:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 Processor -- start 2024-09-17T05:41:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.327+0000 7f69bd800640 1 -- start start 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 0x7f69b8146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b80ce3b0 0x7f69b809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b8151410 0x7f69b80af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8168600 con 0x7f69b80ae0c0 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8168480 con 0x7f69b80ce3b0 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f69b8168780 con 0x7f69b8151410 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 0x7f69b8146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 0x7f69b8146820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54668/0 (socket says 172.21.15.122:54668) 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 -- 172.21.15.122:0/508060778 learned_addr learned my addr 172.21.15.122:0/508060778 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6000640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b80ce3b0 0x7f69b809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b7400640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b8151410 0x7f69b80af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 -- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b8151410 msgr2=0x7f69b80af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b8151410 0x7f69b80af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 -- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b80ce3b0 msgr2=0x7f69b809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b80ce3b0 0x7f69b809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 -- 172.21.15.122:0/508060778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b81f8ac0 con 0x7f69b80ae0c0 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b7400640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b8151410 0x7f69b80af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6000640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b80ce3b0 0x7f69b809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69b6a00640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 0x7f69b8146820 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f69a800c490 tx=0x7f69a800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69aea00640 1 -- 172.21.15.122:0/508060778 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8016020 con 0x7f69b80ae0c0 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69aea00640 1 -- 172.21.15.122:0/508060778 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69a8012940 con 0x7f69b80ae0c0 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69aea00640 1 -- 172.21.15.122:0/508060778 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8012d90 con 0x7f69b80ae0c0 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b80b3ce0 con 0x7f69b80ae0c0 2024-09-17T05:41:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b809af50 con 0x7f69b80ae0c0 2024-09-17T05:41:55.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.331+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b80d0810 con 0x7f69b80ae0c0 2024-09-17T05:41:55.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.335+0000 7f69aea00640 1 -- 172.21.15.122:0/508060778 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f69a8019020 con 0x7f69b80ae0c0 2024-09-17T05:41:55.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.335+0000 7f69aea00640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6998075a20 0x7f6998077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:55.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.335+0000 7f69aea00640 1 -- 172.21.15.122:0/508060778 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5965+0+0 (secure 0 0 0) 0x7f69a80934f0 con 0x7f69b80ae0c0 2024-09-17T05:41:55.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.335+0000 7f69b6000640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6998075a20 0x7f6998077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:55.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.335+0000 7f69b6000640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6998075a20 0x7f6998077ee0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f69b80c94d0 tx=0x7f69a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:55.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.339+0000 7f69aea00640 1 -- 172.21.15.122:0/508060778 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f69a8020020 con 0x7f69b80ae0c0 2024-09-17T05:41:55.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:55.543+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f69b80ce7f0 con 0x7f69b80ae0c0 2024-09-17T05:41:56.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69aea00640 1 -- 172.21.15.122:0/508060778 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v99) ==== 104+0+0 (secure 0 0 0) 0x7f69a80600c0 con 0x7f69b80ae0c0 2024-09-17T05:41:56.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6998075a20 msgr2=0x7f6998077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:56.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6998075a20 0x7f6998077ee0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f69b80c94d0 tx=0x7f69a4009040 comp rx=0 tx=0).stop 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 msgr2=0x7f69b8146820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 0x7f69b8146820 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f69a800c490 tx=0x7f69a800c950 comp rx=0 tx=0).stop 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 shutdown_connections 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f6998075a20 0x7f6998077ee0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69b8151410 0x7f69b80af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69b80ce3b0 0x7f69b809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 --2- 172.21.15.122:0/508060778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b80ae0c0 0x7f69b8146820 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 >> 172.21.15.122:0/508060778 conn(0x7f69b80cfc20 msgr2=0x7f69b80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:56.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.159+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 shutdown_connections 2024-09-17T05:41:56.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.163+0000 7f69bd800640 1 -- 172.21.15.122:0/508060778 wait complete. 2024-09-17T05:41:56.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:41:56.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.375+0000 7f2261c00640 1 -- 172.21.15.122:0/1886921329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c15e220 msgr2=0x7f225c0d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:56.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.375+0000 7f2261c00640 1 --2- 172.21.15.122:0/1886921329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c15e220 0x7f225c0d1b60 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f2244009840 tx=0x7f224401c820 comp rx=0 tx=0).stop 2024-09-17T05:41:56.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.375+0000 7f2261c00640 1 -- 172.21.15.122:0/1886921329 shutdown_connections 2024-09-17T05:41:56.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.375+0000 7f2261c00640 1 --2- 172.21.15.122:0/1886921329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c15e220 0x7f225c0d1b60 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:56.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.375+0000 7f2261c00640 1 -- 172.21.15.122:0/1886921329 >> 172.21.15.122:0/1886921329 conn(0x7f225c0d3f90 msgr2=0x7f225c0d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:56.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.375+0000 7f2261c00640 1 -- 172.21.15.122:0/1886921329 shutdown_connections 2024-09-17T05:41:56.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.375+0000 7f2261c00640 1 -- 172.21.15.122:0/1886921329 wait complete. 2024-09-17T05:41:56.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 Processor -- start 2024-09-17T05:41:56.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 -- start start 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c0d20a0 0x7f225c0b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f225c15de40 0x7f225c1fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f225c15e220 0x7f225c1fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f225c16c360 con 0x7f225c0d20a0 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f225c16c1e0 con 0x7f225c15de40 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f225c16c4e0 con 0x7f225c15e220 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c0d20a0 0x7f225c0b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c0d20a0 0x7f225c0b8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54690/0 (socket says 172.21.15.122:54690) 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 -- 172.21.15.122:0/428792523 learned_addr learned my addr 172.21.15.122:0/428792523 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:56.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2252000640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f225c15de40 0x7f225c1fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:56.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225b400640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f225c15e220 0x7f225c1fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:56.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 -- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f225c15e220 msgr2=0x7f225c1fb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:56.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f225c15e220 0x7f225c1fb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:56.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 -- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f225c15de40 msgr2=0x7f225c1fb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:56.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f225c15de40 0x7f225c1fb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:56.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 -- 172.21.15.122:0/428792523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c0b9430 con 0x7f225c0d20a0 2024-09-17T05:41:56.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f225aa00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c0d20a0 0x7f225c0b8ef0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f224c00c480 tx=0x7f224c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:56.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2253400640 1 -- 172.21.15.122:0/428792523 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c012830 con 0x7f225c0d20a0 2024-09-17T05:41:56.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2253400640 1 -- 172.21.15.122:0/428792523 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f224c017070 con 0x7f225c0d20a0 2024-09-17T05:41:56.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c15e7d0 con 0x7f225c0d20a0 2024-09-17T05:41:56.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2253400640 1 -- 172.21.15.122:0/428792523 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c012d80 con 0x7f225c0d20a0 2024-09-17T05:41:56.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.379+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c15ed90 con 0x7f225c0d20a0 2024-09-17T05:41:56.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.383+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c0b9660 con 0x7f225c0d20a0 2024-09-17T05:41:56.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.383+0000 7f2253400640 1 -- 172.21.15.122:0/428792523 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f224c01c460 con 0x7f225c0d20a0 2024-09-17T05:41:56.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.383+0000 7f2253400640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22380759f0 0x7f2238077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:56.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.383+0000 7f2253400640 1 -- 172.21.15.122:0/428792523 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5965+0+0 (secure 0 0 0) 0x7f224c093320 con 0x7f225c0d20a0 2024-09-17T05:41:56.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.383+0000 7f2252000640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22380759f0 0x7f2238077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:56.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.383+0000 7f2252000640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22380759f0 0x7f2238077eb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f22440af000 tx=0x7f224401b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:56.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.387+0000 7f2253400640 1 -- 172.21.15.122:0/428792523 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f225c0b9430 con 0x7f225c0d20a0 2024-09-17T05:41:56.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:56.555+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f225c0ce2b0 con 0x7f225c0d20a0 2024-09-17T05:41:57.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2253400640 1 -- 172.21.15.122:0/428792523 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v150) ==== 181+0+0 (secure 0 0 0) 0x7f224c05f7e0 con 0x7f225c0d20a0 2024-09-17T05:41:57.357 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:41:57.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22380759f0 msgr2=0x7f2238077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:57.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22380759f0 0x7f2238077eb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f22440af000 tx=0x7f224401b060 comp rx=0 tx=0).stop 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c0d20a0 msgr2=0x7f225c0b8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c0d20a0 0x7f225c0b8ef0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f224c00c480 tx=0x7f224c00c940 comp rx=0 tx=0).stop 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 shutdown_connections 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f22380759f0 0x7f2238077eb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f225c15e220 0x7f225c1fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f225c15de40 0x7f225c1fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 --2- 172.21.15.122:0/428792523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f225c0d20a0 0x7f225c0b8ef0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.363+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 >> 172.21.15.122:0/428792523 conn(0x7f225c0d3f90 msgr2=0x7f225c1620e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.367+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 shutdown_connections 2024-09-17T05:41:57.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.367+0000 7f2261c00640 1 -- 172.21.15.122:0/428792523 wait complete. 2024-09-17T05:41:57.388 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:41:57.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.543+0000 7f2fd1200640 1 -- 172.21.15.122:0/2005546113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc0cff00 msgr2=0x7f2fcc0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:57.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.543+0000 7f2fd1200640 1 --2- 172.21.15.122:0/2005546113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc0cff00 0x7f2fcc0d28c0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009840 tx=0x7f2fbc01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:41:57.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.543+0000 7f2fd1200640 1 -- 172.21.15.122:0/2005546113 shutdown_connections 2024-09-17T05:41:57.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.543+0000 7f2fd1200640 1 --2- 172.21.15.122:0/2005546113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc0cff00 0x7f2fcc0d28c0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.543+0000 7f2fd1200640 1 -- 172.21.15.122:0/2005546113 >> 172.21.15.122:0/2005546113 conn(0x7f2fcc0d1da0 msgr2=0x7f2fcc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.543+0000 7f2fd1200640 1 -- 172.21.15.122:0/2005546113 shutdown_connections 2024-09-17T05:41:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.543+0000 7f2fd1200640 1 -- 172.21.15.122:0/2005546113 wait complete. 2024-09-17T05:41:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 Processor -- start 2024-09-17T05:41:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 -- start start 2024-09-17T05:41:57.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2fcc0cff00 0x7f2fcc1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:57.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2fcc0d02e0 0x7f2fcc1faa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:57.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc15f700 0x7f2fcc1fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:57.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc169f90 con 0x7f2fcc15f700 2024-09-17T05:41:57.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc169e10 con 0x7f2fcc0d02e0 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2fcc16a110 con 0x7f2fcc0cff00 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc15f700 0x7f2fcc1fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc15f700 0x7f2fcc1fafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54708/0 (socket says 172.21.15.122:54708) 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 -- 172.21.15.122:0/1339053886 learned_addr learned my addr 172.21.15.122:0/1339053886 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 -- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2fcc0cff00 msgr2=0x7f2fcc1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2fcc0cff00 0x7f2fcc1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 -- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2fcc0d02e0 msgr2=0x7f2fcc1faa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2fcc0d02e0 0x7f2fcc1faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 -- 172.21.15.122:0/1339053886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc154f60 con 0x7f2fcc15f700 2024-09-17T05:41:57.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fcbe00640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc15f700 0x7f2fcc1fafc0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400cae0 tx=0x7f2fc400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:57.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fc3e00640 1 -- 172.21.15.122:0/1339053886 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4016020 con 0x7f2fcc15f700 2024-09-17T05:41:57.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fc3e00640 1 -- 172.21.15.122:0/1339053886 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fc400ab40 con 0x7f2fcc15f700 2024-09-17T05:41:57.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc155240 con 0x7f2fcc15f700 2024-09-17T05:41:57.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fc3e00640 1 -- 172.21.15.122:0/1339053886 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400ce20 con 0x7f2fcc15f700 2024-09-17T05:41:57.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.547+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc155800 con 0x7f2fcc15f700 2024-09-17T05:41:57.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.551+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc0d28c0 con 0x7f2fcc15f700 2024-09-17T05:41:57.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.551+0000 7f2fc3e00640 1 -- 172.21.15.122:0/1339053886 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f2fc4019020 con 0x7f2fcc15f700 2024-09-17T05:41:57.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.551+0000 7f2fc3e00640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fa8075a20 0x7f2fa8077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:57.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.551+0000 7f2fc3e00640 1 -- 172.21.15.122:0/1339053886 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5527+0+0 (secure 0 0 0) 0x7f2fc40934c0 con 0x7f2fcc15f700 2024-09-17T05:41:57.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.551+0000 7f2fcb400640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fa8075a20 0x7f2fa8077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:57.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.551+0000 7f2fcb400640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fa8075a20 0x7f2fa8077ee0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0af000 tx=0x7f2fbc019590 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:57.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.555+0000 7f2fc3e00640 1 -- 172.21.15.122:0/1339053886 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f2fc4020030 con 0x7f2fcc15f700 2024-09-17T05:41:57.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:57.723+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2fcc071220 con 0x7f2fcc15f700 2024-09-17T05:41:58.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.371+0000 7f2fc3e00640 1 -- 172.21.15.122:0/1339053886 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) ==== 169+0+0 (secure 0 0 0) 0x7f2fc405ffd0 con 0x7f2fcc15f700 2024-09-17T05:41:58.366 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-09-17T05:41:58.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fa8075a20 msgr2=0x7f2fa8077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fa8075a20 0x7f2fa8077ee0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0af000 tx=0x7f2fbc019590 comp rx=0 tx=0).stop 2024-09-17T05:41:58.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc15f700 msgr2=0x7f2fcc1fafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc15f700 0x7f2fcc1fafc0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400cae0 tx=0x7f2fc400cbd0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 shutdown_connections 2024-09-17T05:41:58.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2fa8075a20 0x7f2fa8077ee0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fcc15f700 0x7f2fcc1fafc0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2fcc0d02e0 0x7f2fcc1faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 --2- 172.21.15.122:0/1339053886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2fcc0cff00 0x7f2fcc1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 >> 172.21.15.122:0/1339053886 conn(0x7f2fcc0d1da0 msgr2=0x7f2fcc071850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:58.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.375+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 shutdown_connections 2024-09-17T05:41:58.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.379+0000 7f2fd1200640 1 -- 172.21.15.122:0/1339053886 wait complete. 2024-09-17T05:41:58.398 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-17T05:41:58.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.547+0000 7f8129200640 1 -- 172.21.15.122:0/4040402543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8108003ab0 msgr2=0x7f8108003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.547+0000 7f8129200640 1 --2- 172.21.15.122:0/4040402543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8108003ab0 0x7f8108003ed0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f811400bb10 tx=0x7f811401cdc0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.547+0000 7f8129200640 1 -- 172.21.15.122:0/4040402543 shutdown_connections 2024-09-17T05:41:58.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.547+0000 7f8129200640 1 --2- 172.21.15.122:0/4040402543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8108003ab0 0x7f8108003ed0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.547+0000 7f8129200640 1 -- 172.21.15.122:0/4040402543 >> 172.21.15.122:0/4040402543 conn(0x7f81240b18f0 msgr2=0x7f81240b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:58.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.547+0000 7f8129200640 1 -- 172.21.15.122:0/4040402543 shutdown_connections 2024-09-17T05:41:58.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.547+0000 7f8129200640 1 -- 172.21.15.122:0/4040402543 wait complete. 2024-09-17T05:41:58.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 Processor -- start 2024-09-17T05:41:58.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 -- start start 2024-09-17T05:41:58.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8108003ab0 0x7f812415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:58.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f812415a730 0x7f812415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:58.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81241f81e0 0x7f81241f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:58.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f812415f1c0 con 0x7f81241f81e0 2024-09-17T05:41:58.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f812415f040 con 0x7f8108003ab0 2024-09-17T05:41:58.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f812415f340 con 0x7f812415a730 2024-09-17T05:41:58.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81241f81e0 0x7f81241f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:58.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81241f81e0 0x7f81241f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54732/0 (socket says 172.21.15.122:54732) 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 -- 172.21.15.122:0/25575110 learned_addr learned my addr 172.21.15.122:0/25575110 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8119400640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f812415a730 0x7f812415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8121e00640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8108003ab0 0x7f812415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 -- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f812415a730 msgr2=0x7f812415abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f812415a730 0x7f812415abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 -- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8108003ab0 msgr2=0x7f812415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8108003ab0 0x7f812415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 -- 172.21.15.122:0/25575110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81241f8ef0 con 0x7f81241f81e0 2024-09-17T05:41:58.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8122800640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81241f81e0 0x7f81241f8660 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f811c006fd0 tx=0x7f811c0064e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:58.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f811aa00640 1 -- 172.21.15.122:0/25575110 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c010890 con 0x7f81241f81e0 2024-09-17T05:41:58.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f811aa00640 1 -- 172.21.15.122:0/25575110 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f811c017070 con 0x7f81241f81e0 2024-09-17T05:41:58.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f811aa00640 1 -- 172.21.15.122:0/25575110 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c010de0 con 0x7f81241f81e0 2024-09-17T05:41:58.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812415ca00 con 0x7f81241f81e0 2024-09-17T05:41:58.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.551+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241579d0 con 0x7f81241f81e0 2024-09-17T05:41:58.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.555+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8124069410 con 0x7f81241f81e0 2024-09-17T05:41:58.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.555+0000 7f811aa00640 1 -- 172.21.15.122:0/25575110 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f811c01c460 con 0x7f81241f81e0 2024-09-17T05:41:58.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.555+0000 7f811aa00640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81000759f0 0x7f8100077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:58.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.555+0000 7f811aa00640 1 -- 172.21.15.122:0/25575110 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f811c093d80 con 0x7f81241f81e0 2024-09-17T05:41:58.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.555+0000 7f8121e00640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81000759f0 0x7f8100077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:58.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.555+0000 7f8121e00640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81000759f0 0x7f8100077eb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f81140b0000 tx=0x7f81140a7040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:58.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.559+0000 7f811aa00640 1 -- 172.21.15.122:0/25575110 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f811c0601c0 con 0x7f81241f81e0 2024-09-17T05:41:58.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.735+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f812415f5b0 con 0x7f81241f81e0 2024-09-17T05:41:58.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.739+0000 7f811aa00640 1 -- 172.21.15.122:0/25575110 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v57) ==== 71+0+0 (secure 0 0 0) 0x7f811c064bf0 con 0x7f81241f81e0 2024-09-17T05:41:58.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81000759f0 msgr2=0x7f8100077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81000759f0 0x7f8100077eb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f81140b0000 tx=0x7f81140a7040 comp rx=0 tx=0).stop 2024-09-17T05:41:58.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81241f81e0 msgr2=0x7f81241f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81241f81e0 0x7f81241f8660 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f811c006fd0 tx=0x7f811c0064e0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 shutdown_connections 2024-09-17T05:41:58.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f81000759f0 0x7f8100077eb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81241f81e0 0x7f81241f8660 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f812415a730 0x7f812415abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 --2- 172.21.15.122:0/25575110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8108003ab0 0x7f812415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 >> 172.21.15.122:0/25575110 conn(0x7f81240b18f0 msgr2=0x7f81240d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 shutdown_connections 2024-09-17T05:41:58.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.743+0000 7f8129200640 1 -- 172.21.15.122:0/25575110 wait complete. 2024-09-17T05:41:58.758 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-17T05:41:58.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- 172.21.15.122:0/825484005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40acd40 msgr2=0x7f0fc40b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 --2- 172.21.15.122:0/825484005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40acd40 0x7f0fc40b42d0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009c80 tx=0x7f0fbc01cc80 comp rx=0 tx=0).stop 2024-09-17T05:41:58.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- 172.21.15.122:0/825484005 shutdown_connections 2024-09-17T05:41:58.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 --2- 172.21.15.122:0/825484005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40acd40 0x7f0fc40b42d0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 --2- 172.21.15.122:0/825484005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fc40cd100 0x7f0fc415dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- 172.21.15.122:0/825484005 >> 172.21.15.122:0/825484005 conn(0x7f0fc40b18f0 msgr2=0x7f0fc40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:58.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- 172.21.15.122:0/825484005 shutdown_connections 2024-09-17T05:41:58.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- 172.21.15.122:0/825484005 wait complete. 2024-09-17T05:41:58.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 Processor -- start 2024-09-17T05:41:58.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- start start 2024-09-17T05:41:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc406f8b0 0x7f0fc4153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fc40acd40 0x7f0fc414f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40cd100 0x7f0fc415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41682b0 con 0x7f0fc40cd100 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4168130 con 0x7f0fc406f8b0 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.907+0000 7f0fc9000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc4168430 con 0x7f0fc40acd40 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40cd100 0x7f0fc415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40cd100 0x7f0fc415afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54748/0 (socket says 172.21.15.122:54748) 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 -- 172.21.15.122:0/1593437672 learned_addr learned my addr 172.21.15.122:0/1593437672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fb9400640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fc40acd40 0x7f0fc414f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc1e00640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc406f8b0 0x7f0fc4153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 -- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fc40acd40 msgr2=0x7f0fc414f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fc40acd40 0x7f0fc414f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 -- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc406f8b0 msgr2=0x7f0fc4153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc406f8b0 0x7f0fc4153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 -- 172.21.15.122:0/1593437672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc4150bc0 con 0x7f0fc40cd100 2024-09-17T05:41:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc2800640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40cd100 0x7f0fc415afc0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009bd0 tx=0x7f0fbc009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fbaa00640 1 -- 172.21.15.122:0/1593437672 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0bb070 con 0x7f0fc40cd100 2024-09-17T05:41:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fbaa00640 1 -- 172.21.15.122:0/1593437672 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fbc004e00 con 0x7f0fc40cd100 2024-09-17T05:41:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fbaa00640 1 -- 172.21.15.122:0/1593437672 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0b2040 con 0x7f0fc40cd100 2024-09-17T05:41:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc4150ea0 con 0x7f0fc40cd100 2024-09-17T05:41:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc4150870 con 0x7f0fc40cd100 2024-09-17T05:41:58.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc40b42d0 con 0x7f0fc40cd100 2024-09-17T05:41:58.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.911+0000 7f0fbaa00640 1 -- 172.21.15.122:0/1593437672 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f0fbc0a5ad0 con 0x7f0fc40cd100 2024-09-17T05:41:58.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.915+0000 7f0fbaa00640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0fa0075a20 0x7f0fa0077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:41:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.915+0000 7f0fbaa00640 1 -- 172.21.15.122:0/1593437672 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f0fbc12d320 con 0x7f0fc40cd100 2024-09-17T05:41:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.915+0000 7f0fc1e00640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0fa0075a20 0x7f0fa0077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:41:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.915+0000 7f0fc1e00640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0fa0075a20 0x7f0fa0077ee0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4007fd0 tx=0x7f0fb4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:41:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:58.919+0000 7f0fbaa00640 1 -- 172.21.15.122:0/1593437672 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f0fbc016990 con 0x7f0fc40cd100 2024-09-17T05:41:59.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.143+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7f0fc41595f0 con 0x7f0fc40cd100 2024-09-17T05:41:59.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fbaa00640 1 -- 172.21.15.122:0/1593437672 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v772) ==== 186+0+0 (secure 0 0 0) 0x7f0fbc0f97e0 con 0x7f0fc40cd100 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0fa0075a20 msgr2=0x7f0fa0077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0fa0075a20 0x7f0fa0077ee0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4007fd0 tx=0x7f0fb4009040 comp rx=0 tx=0).stop 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40cd100 msgr2=0x7f0fc415afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40cd100 0x7f0fc415afc0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009bd0 tx=0x7f0fbc009390 comp rx=0 tx=0).stop 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 shutdown_connections 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f0fa0075a20 0x7f0fa0077ee0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc40cd100 0x7f0fc415afc0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fc40acd40 0x7f0fc414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 --2- 172.21.15.122:0/1593437672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fc406f8b0 0x7f0fc4153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 >> 172.21.15.122:0/1593437672 conn(0x7f0fc40b18f0 msgr2=0x7f0fc406fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 shutdown_connections 2024-09-17T05:41:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:41:59.375+0000 7f0fc9000640 1 -- 172.21.15.122:0/1593437672 wait complete. 2024-09-17T05:41:59.390 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ok 2024-09-17T05:41:59.390 INFO:tasks.cephfs_test_runner: 2024-09-17T05:41:59.390 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T05:41:59.390 INFO:tasks.cephfs_test_runner:Ran 9 tests in 989.406s 2024-09-17T05:41:59.390 INFO:tasks.cephfs_test_runner: 2024-09-17T05:41:59.390 INFO:tasks.cephfs_test_runner:OK 2024-09-17T05:41:59.391 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-17T05:41:59.399 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-17T05:41:59.423 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-17T05:41:59.423 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:41:59.423 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:59.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:41:59.433 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-17T05:41:59.433 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:59.487 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350080k,nr_inodes=4087520,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3131 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274948k,nr_inodes=818737,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274952k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:41:59.487 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:41:59.487 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:59.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:41:59.498 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-09-17T05:41:59.498 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:41:59.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350256k,nr_inodes=4087564,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3466 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274984k,nr_inodes=818746,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274988k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:41:59.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:59.555 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:41:59.563 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:41:59.564 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:41:59.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:59.565 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:41:59.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:41:59.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:42:00.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:00.117 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-17T05:42:00.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:00.242 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-09-17T05:42:00.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:00.251 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-09-17T05:42:00.702 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-09-17T05:42:00.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:42:00.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:00.705 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2024-09-17T05:42:00.714 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:42:00.714 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:42:00.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:00.715 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:42:00.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:00.784 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:42:01.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:01.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2024-09-17T05:42:01.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:01.361 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2024-09-17T05:42:01.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:42:01.370 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2024-09-17T05:42:01.838 INFO:teuthology.orchestra.run.smithi173.stderr:Cannot find device "ceph-brx" 2024-09-17T05:42:01.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:42:01.841 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T05:42:01.849 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:42:01.849 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:42:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- 172.21.15.122:0/3293428161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415abc0 msgr2=0x7f4e540d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3293428161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415abc0 0x7f4e540d0c40 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009840 tx=0x7f4e4001d280 comp rx=0 tx=0).stop 2024-09-17T05:42:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- 172.21.15.122:0/3293428161 shutdown_connections 2024-09-17T05:42:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3293428161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e540d1180 0x7f4e540b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3293428161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415abc0 0x7f4e540d0c40 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- 172.21.15.122:0/3293428161 >> 172.21.15.122:0/3293428161 conn(0x7f4e5406dd30 msgr2=0x7f4e5406e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- 172.21.15.122:0/3293428161 shutdown_connections 2024-09-17T05:42:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- 172.21.15.122:0/3293428161 wait complete. 2024-09-17T05:42:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 Processor -- start 2024-09-17T05:42:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- start start 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e540d1180 0x7f4e5409e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415a7e0 0x7f4e54164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e5415abc0 0x7f4e54162ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54167c80 con 0x7f4e5415a7e0 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54167b00 con 0x7f4e540d1180 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5c200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4e54167e00 con 0x7f4e5415abc0 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415a7e0 0x7f4e54164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415a7e0 0x7f4e54164ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58592/0 (socket says 172.21.15.122:58592) 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 -- 172.21.15.122:0/3642517702 learned_addr learned my addr 172.21.15.122:0/3642517702 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e5a200640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e5415abc0 0x7f4e54162ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 -- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e5415abc0 msgr2=0x7f4e54162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e5415abc0 0x7f4e54162ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 -- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e540d1180 msgr2=0x7f4e5409e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e540d1180 0x7f4e5409e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 -- 172.21.15.122:0/3642517702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e54146850 con 0x7f4e5415a7e0 2024-09-17T05:42:02.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e58e00640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415a7e0 0x7f4e54164ef0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009790 tx=0x7f4e40005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:02.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e51600640 1 -- 172.21.15.122:0/3642517702 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e400b2040 con 0x7f4e5415a7e0 2024-09-17T05:42:02.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e51600640 1 -- 172.21.15.122:0/3642517702 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e400043f0 con 0x7f4e5415a7e0 2024-09-17T05:42:02.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.007+0000 7f4e51600640 1 -- 172.21.15.122:0/3642517702 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40003800 con 0x7f4e5415a7e0 2024-09-17T05:42:02.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.011+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e54146b30 con 0x7f4e5415a7e0 2024-09-17T05:42:02.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.011+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541470f0 con 0x7f4e5415a7e0 2024-09-17T05:42:02.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.011+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e540b48f0 con 0x7f4e5415a7e0 2024-09-17T05:42:02.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.011+0000 7f4e51600640 1 -- 172.21.15.122:0/3642517702 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f4e400bd060 con 0x7f4e5415a7e0 2024-09-17T05:42:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.011+0000 7f4e51600640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e340759f0 0x7f4e34077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.011+0000 7f4e51600640 1 -- 172.21.15.122:0/3642517702 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f4e4012cd80 con 0x7f4e5415a7e0 2024-09-17T05:42:02.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.015+0000 7f4e59800640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e340759f0 0x7f4e34077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.015+0000 7f4e59800640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e340759f0 0x7f4e34077eb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4e48007fd0 tx=0x7f4e480078e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:02.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.015+0000 7f4e51600640 1 -- 172.21.15.122:0/3642517702 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f4e40016410 con 0x7f4e5415a7e0 2024-09-17T05:42:02.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.183+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e54165430 con 0x7f4e340759f0 2024-09-17T05:42:02.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.183+0000 7f4e51600640 1 -- 172.21.15.122:0/3642517702 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13549 (secure 0 0 0) 0x7f4e54165430 con 0x7f4e340759f0 2024-09-17T05:42:02.181 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:42:02.182 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:42:02.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e340759f0 msgr2=0x7f4e34077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e340759f0 0x7f4e34077eb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4e48007fd0 tx=0x7f4e480078e0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415a7e0 msgr2=0x7f4e54164ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415a7e0 0x7f4e54164ef0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009790 tx=0x7f4e40005db0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 shutdown_connections 2024-09-17T05:42:02.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f4e340759f0 0x7f4e34077eb0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e5415abc0 0x7f4e54162ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e5415a7e0 0x7f4e54164ef0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 --2- 172.21.15.122:0/3642517702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e540d1180 0x7f4e5409e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 >> 172.21.15.122:0/3642517702 conn(0x7f4e5406dd30 msgr2=0x7f4e540b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:02.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 shutdown_connections 2024-09-17T05:42:02.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.187+0000 7f4e5c200640 1 -- 172.21.15.122:0/3642517702 wait complete. 2024-09-17T05:42:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":658,"stamp":"2024-09-17T05:42:00.795045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":1143240,"kb_used_data":19604,"kb_used_omap":12,"kb_used_meta":1123443,"kb_avail":753831480,"statfs":{"total":773094113280,"available":771923435520,"internally_reserved":0,"allocated":20074496,"data_stored":11705569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1150406198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1423817707,"num_objects":-364,"num_object_clones":0,"num_object_copies":-728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-364,"num_whiteouts":0,"num_read":-3724,"num_read_kb":-14811244,"num_write":-9110,"num_write_kb":-17819810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":-12582912,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001499"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":358,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T05:41:58.681897+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:41:58.681897+0000","last_peered":"2024-09-17T05:41:58.681897+0000","last_clean":"2024-09-17T05:41:58.681897+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:41:58.681897+0000","last_undegraded":"2024-09-17T05:41:58.681897+0000","last_fullsized":"2024-09-17T05:41:58.681897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509648,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123192,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248648,"statfs":{"total":96636764160,"available":96510615552,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509648,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163452,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208388,"statfs":{"total":96636764160,"available":96469389312,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":147296,"kb_used_data":3724,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94224544,"statfs":{"total":96636764160,"available":96485933056,"internally_reserved":0,"allocated":3813376,"data_stored":2764186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509647,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509648,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509649,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137680,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234160,"statfs":{"total":96636764160,"available":96495779840,"internally_reserved":0,"allocated":2482176,"data_stored":1426591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509647,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":142735819},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147888,"kb_used_data":4316,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94223952,"statfs":{"total":96636764160,"available":96485326848,"internally_reserved":0,"allocated":4419584,"data_stored":3362232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:42:02.202 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:42:02.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.347+0000 7fb8d8000640 1 -- 172.21.15.122:0/1318409901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d015dae0 msgr2=0x7fb8d00acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.347+0000 7fb8d8000640 1 --2- 172.21.15.122:0/1318409901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d015dae0 0x7fb8d00acd40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400a950 tx=0x7fb8c401da00 comp rx=0 tx=0).stop 2024-09-17T05:42:02.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.347+0000 7fb8d8000640 1 -- 172.21.15.122:0/1318409901 shutdown_connections 2024-09-17T05:42:02.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 --2- 172.21.15.122:0/1318409901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d00b49e0 0x7fb8d006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 --2- 172.21.15.122:0/1318409901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d015dae0 0x7fb8d00acd40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 -- 172.21.15.122:0/1318409901 >> 172.21.15.122:0/1318409901 conn(0x7fb8d00b18f0 msgr2=0x7fb8d00b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:02.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 -- 172.21.15.122:0/1318409901 shutdown_connections 2024-09-17T05:42:02.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 -- 172.21.15.122:0/1318409901 wait complete. 2024-09-17T05:42:02.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 Processor -- start 2024-09-17T05:42:02.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 -- start start 2024-09-17T05:42:02.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d00b49e0 0x7fb8d0164880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d00cd100 0x7fb8d00c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d015dae0 0x7fb8d014b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d00cf230 con 0x7fb8d00b49e0 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d00cf0b0 con 0x7fb8d015dae0 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d00cf3b0 con 0x7fb8d00cd100 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d00b49e0 0x7fb8d0164880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d00b49e0 0x7fb8d0164880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58606/0 (socket says 172.21.15.122:58606) 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 -- 172.21.15.122:0/3165763795 learned_addr learned my addr 172.21.15.122:0/3165763795 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d4c00640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d00cd100 0x7fb8d00c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d6000640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d015dae0 0x7fb8d014b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 -- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d00cd100 msgr2=0x7fb8d00c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d00cd100 0x7fb8d00c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 -- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d015dae0 msgr2=0x7fb8d014b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d015dae0 0x7fb8d014b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 -- 172.21.15.122:0/3165763795 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d00c50a0 con 0x7fb8d00b49e0 2024-09-17T05:42:02.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d5600640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d00b49e0 0x7fb8d0164880 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fb8c401c0b0 tx=0x7fb8c4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:02.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8cd600640 1 -- 172.21.15.122:0/3165763795 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40b3040 con 0x7fb8d00b49e0 2024-09-17T05:42:02.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8cd600640 1 -- 172.21.15.122:0/3165763795 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8c40a6c70 con 0x7fb8d00b49e0 2024-09-17T05:42:02.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8cd600640 1 -- 172.21.15.122:0/3165763795 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4003800 con 0x7fb8d00b49e0 2024-09-17T05:42:02.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.351+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d009c4d0 con 0x7fb8d00b49e0 2024-09-17T05:42:02.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.355+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d009ca90 con 0x7fb8d00b49e0 2024-09-17T05:42:02.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.355+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d00acd40 con 0x7fb8d00b49e0 2024-09-17T05:42:02.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.355+0000 7fb8cd600640 1 -- 172.21.15.122:0/3165763795 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fb8c40be060 con 0x7fb8d00b49e0 2024-09-17T05:42:02.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.355+0000 7fb8cd600640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b0075a20 0x7fb8b0077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.355+0000 7fb8cd600640 1 -- 172.21.15.122:0/3165763795 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fb8c412ce70 con 0x7fb8d00b49e0 2024-09-17T05:42:02.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.355+0000 7fb8d4c00640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b0075a20 0x7fb8b0077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.359+0000 7fb8d4c00640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b0075a20 0x7fb8b0077ee0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb8d006e6a0 tx=0x7fb8bc001e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:02.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.359+0000 7fb8cd600640 1 -- 172.21.15.122:0/3165763795 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb8c4017680 con 0x7fb8d00b49e0 2024-09-17T05:42:02.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.527+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8d00b4290 con 0x7fb8b0075a20 2024-09-17T05:42:02.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.527+0000 7fb8cd600640 1 -- 172.21.15.122:0/3165763795 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13549 (secure 0 0 0) 0x7fb8d00b4290 con 0x7fb8b0075a20 2024-09-17T05:42:02.522 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:42:02.522 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:42:02.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b0075a20 msgr2=0x7fb8b0077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b0075a20 0x7fb8b0077ee0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb8d006e6a0 tx=0x7fb8bc001e40 comp rx=0 tx=0).stop 2024-09-17T05:42:02.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d00b49e0 msgr2=0x7fb8d0164880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d00b49e0 0x7fb8d0164880 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fb8c401c0b0 tx=0x7fb8c4004620 comp rx=0 tx=0).stop 2024-09-17T05:42:02.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 shutdown_connections 2024-09-17T05:42:02.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb8b0075a20 0x7fb8b0077ee0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d015dae0 0x7fb8d014b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d00cd100 0x7fb8d00c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 --2- 172.21.15.122:0/3165763795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8d00b49e0 0x7fb8d0164880 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 >> 172.21.15.122:0/3165763795 conn(0x7fb8d00b18f0 msgr2=0x7fb8d00aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:02.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 shutdown_connections 2024-09-17T05:42:02.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.531+0000 7fb8d8000640 1 -- 172.21.15.122:0/3165763795 wait complete. 2024-09-17T05:42:02.544 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":658,"stamp":"2024-09-17T05:42:00.795045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":1143240,"kb_used_data":19604,"kb_used_omap":12,"kb_used_meta":1123443,"kb_avail":753831480,"statfs":{"total":773094113280,"available":771923435520,"internally_reserved":0,"allocated":20074496,"data_stored":11705569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1150406198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1423817707,"num_objects":-364,"num_object_clones":0,"num_object_copies":-728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-364,"num_whiteouts":0,"num_read":-3724,"num_read_kb":-14811244,"num_write":-9110,"num_write_kb":-17819810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":-12582912,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001499"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":358,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T05:41:58.681897+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:41:58.681897+0000","last_peered":"2024-09-17T05:41:58.681897+0000","last_clean":"2024-09-17T05:41:58.681897+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:41:58.681897+0000","last_undegraded":"2024-09-17T05:41:58.681897+0000","last_fullsized":"2024-09-17T05:41:58.681897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509648,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123192,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248648,"statfs":{"total":96636764160,"available":96510615552,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509648,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163452,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208388,"statfs":{"total":96636764160,"available":96469389312,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":147296,"kb_used_data":3724,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94224544,"statfs":{"total":96636764160,"available":96485933056,"internally_reserved":0,"allocated":3813376,"data_stored":2764186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509647,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509648,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509649,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137680,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234160,"statfs":{"total":96636764160,"available":96495779840,"internally_reserved":0,"allocated":2482176,"data_stored":1426591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509647,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":142735819},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147888,"kb_used_data":4316,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94223952,"statfs":{"total":96636764160,"available":96485326848,"internally_reserved":0,"allocated":4419584,"data_stored":3362232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:42:02.545 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:42:02.545 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:42:02.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 -- 172.21.15.122:0/1145040355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0cdb30 msgr2=0x7f63bc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 --2- 172.21.15.122:0/1145040355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0cdb30 0x7f63bc0cebb0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0745d0 tx=0x7f63a401c830 comp rx=0 tx=0).stop 2024-09-17T05:42:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 -- 172.21.15.122:0/1145040355 shutdown_connections 2024-09-17T05:42:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 --2- 172.21.15.122:0/1145040355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0cdb30 0x7f63bc0cebb0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 --2- 172.21.15.122:0/1145040355 >> v2:172.21.15.173:3300/0 conn(0x7f63bc15bea0 0x7f63bc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 -- 172.21.15.122:0/1145040355 >> 172.21.15.122:0/1145040355 conn(0x7f63bc0d1dc0 msgr2=0x7f63bc0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 -- 172.21.15.122:0/1145040355 shutdown_connections 2024-09-17T05:42:02.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 -- 172.21.15.122:0/1145040355 wait complete. 2024-09-17T05:42:02.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.691+0000 7f63c2400640 1 Processor -- start 2024-09-17T05:42:02.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 -- start start 2024-09-17T05:42:02.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63bc0cdb30 0x7f63bc0c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0d28e0 0x7f63bc0c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63bc15bea0 0x7f63bc1faba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc168fc0 con 0x7f63bc0d28e0 2024-09-17T05:42:02.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc168e40 con 0x7f63bc0cdb30 2024-09-17T05:42:02.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f63bc169140 con 0x7f63bc15bea0 2024-09-17T05:42:02.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0d28e0 0x7f63bc0c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0d28e0 0x7f63bc0c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58626/0 (socket says 172.21.15.122:58626) 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 -- 172.21.15.122:0/1691391879 learned_addr learned my addr 172.21.15.122:0/1691391879 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63bb400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63bc0cdb30 0x7f63bc0c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63bbe00640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63bc15bea0 0x7f63bc1faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 -- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63bc15bea0 msgr2=0x7f63bc1faba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63bc15bea0 0x7f63bc1faba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 -- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63bc0cdb30 msgr2=0x7f63bc0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63bc0cdb30 0x7f63bc0c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 -- 172.21.15.122:0/1691391879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63bc0c79f0 con 0x7f63bc0d28e0 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b2a00640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0d28e0 0x7f63bc0c5ad0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f63a4019440 tx=0x7f63a40a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b3e00640 1 -- 172.21.15.122:0/1691391879 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a40b2040 con 0x7f63bc0d28e0 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b3e00640 1 -- 172.21.15.122:0/1691391879 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63a4004420 con 0x7f63bc0d28e0 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63b3e00640 1 -- 172.21.15.122:0/1691391879 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4005290 con 0x7f63bc0d28e0 2024-09-17T05:42:02.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63bc0c7cd0 con 0x7f63bc0d28e0 2024-09-17T05:42:02.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63bc1596a0 con 0x7f63bc0d28e0 2024-09-17T05:42:02.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.695+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63bc0cc9d0 con 0x7f63bc0d28e0 2024-09-17T05:42:02.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.699+0000 7f63b3e00640 1 -- 172.21.15.122:0/1691391879 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f63a4005430 con 0x7f63bc0d28e0 2024-09-17T05:42:02.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.699+0000 7f63b3e00640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63980759f0 0x7f6398077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:02.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.699+0000 7f63b3e00640 1 -- 172.21.15.122:0/1691391879 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f63a412d370 con 0x7f63bc0d28e0 2024-09-17T05:42:02.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.699+0000 7f63bb400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63980759f0 0x7f6398077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:02.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.699+0000 7f63bb400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63980759f0 0x7f6398077eb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f63ac004630 tx=0x7f63ac00b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:02.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.703+0000 7f63b3e00640 1 -- 172.21.15.122:0/1691391879 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f63a40c4060 con 0x7f63bc0d28e0 2024-09-17T05:42:02.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.867+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63bc0762f0 con 0x7f63980759f0 2024-09-17T05:42:02.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.867+0000 7f63b3e00640 1 -- 172.21.15.122:0/1691391879 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13546 (secure 0 0 0) 0x7f63bc0762f0 con 0x7f63980759f0 2024-09-17T05:42:02.863 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:42:02.863 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:42:02.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63980759f0 msgr2=0x7f6398077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63980759f0 0x7f6398077eb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f63ac004630 tx=0x7f63ac00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0d28e0 msgr2=0x7f63bc0c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:02.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0d28e0 0x7f63bc0c5ad0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f63a4019440 tx=0x7f63a40a5710 comp rx=0 tx=0).stop 2024-09-17T05:42:02.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 shutdown_connections 2024-09-17T05:42:02.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f63980759f0 0x7f6398077eb0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63bc15bea0 0x7f63bc1faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63bc0d28e0 0x7f63bc0c5ad0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 --2- 172.21.15.122:0/1691391879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63bc0cdb30 0x7f63bc0c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:02.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 >> 172.21.15.122:0/1691391879 conn(0x7f63bc0d1dc0 msgr2=0x7f63bc06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:02.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 shutdown_connections 2024-09-17T05:42:02.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:02.871+0000 7f63c2400640 1 -- 172.21.15.122:0/1691391879 wait complete. 2024-09-17T05:42:02.885 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":659,"stamp":"2024-09-17T05:42:02.795526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":1143240,"kb_used_data":19604,"kb_used_omap":12,"kb_used_meta":1123443,"kb_avail":753831480,"statfs":{"total":773094113280,"available":771923435520,"internally_reserved":0,"allocated":20074496,"data_stored":11705569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1150406198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-295549409,"num_objects":-93,"num_object_clones":0,"num_object_copies":-186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-93,"num_whiteouts":0,"num_read":-3724,"num_read_kb":-14811244,"num_write":-9381,"num_write_kb":-17819810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":-12582912,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.635907"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":358,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T05:41:58.681897+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:41:58.681897+0000","last_peered":"2024-09-17T05:41:58.681897+0000","last_clean":"2024-09-17T05:41:58.681897+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:41:58.681897+0000","last_undegraded":"2024-09-17T05:41:58.681897+0000","last_fullsized":"2024-09-17T05:41:58.681897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509648,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123192,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248648,"statfs":{"total":96636764160,"available":96510615552,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509648,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163452,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208388,"statfs":{"total":96636764160,"available":96469389312,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509648,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":147296,"kb_used_data":3724,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94224544,"statfs":{"total":96636764160,"available":96485933056,"internally_reserved":0,"allocated":3813376,"data_stored":2764186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509647,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509648,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509649,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137680,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234160,"statfs":{"total":96636764160,"available":96495779840,"internally_reserved":0,"allocated":2482176,"data_stored":1426591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509647,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141244,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230596,"statfs":{"total":96636764160,"available":96492130304,"internally_reserved":0,"allocated":1871872,"data_stored":830512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":142735819},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147888,"kb_used_data":4316,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94223952,"statfs":{"total":96636764160,"available":96485326848,"internally_reserved":0,"allocated":4419584,"data_stored":3362232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:42:02.885 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:42:03.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.031+0000 7f5792a00640 1 -- 172.21.15.122:0/4236523403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c073d10 msgr2=0x7f578c0cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:03.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.031+0000 7f5792a00640 1 --2- 172.21.15.122:0/4236523403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c073d10 0x7f578c0cd100 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f578c06fcf0 tx=0x7f577401c820 comp rx=0 tx=0).stop 2024-09-17T05:42:03.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- 172.21.15.122:0/4236523403 shutdown_connections 2024-09-17T05:42:03.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 --2- 172.21.15.122:0/4236523403 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f578c0b49e0 0x7f578c06e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 --2- 172.21.15.122:0/4236523403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c073d10 0x7f578c0cd100 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- 172.21.15.122:0/4236523403 >> 172.21.15.122:0/4236523403 conn(0x7f578c0b18f0 msgr2=0x7f578c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:03.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- 172.21.15.122:0/4236523403 shutdown_connections 2024-09-17T05:42:03.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- 172.21.15.122:0/4236523403 wait complete. 2024-09-17T05:42:03.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 Processor -- start 2024-09-17T05:42:03.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- start start 2024-09-17T05:42:03.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f578c073d10 0x7f578c09bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:03.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c0b49e0 0x7f578c09c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f578c15a660 0x7f578c143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f578c167310 con 0x7f578c073d10 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f578c167190 con 0x7f578c0b49e0 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f578c167490 con 0x7f578c15a660 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f578c073d10 0x7f578c09bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f578c073d10 0x7f578c09bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58630/0 (socket says 172.21.15.122:58630) 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 -- 172.21.15.122:0/1859229113 learned_addr learned my addr 172.21.15.122:0/1859229113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5783400640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c0b49e0 0x7f578c09c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5790a00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f578c15a660 0x7f578c143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:03.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 -- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f578c15a660 msgr2=0x7f578c143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f578c15a660 0x7f578c143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 -- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c0b49e0 msgr2=0x7f578c09c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c0b49e0 0x7f578c09c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 -- 172.21.15.122:0/1859229113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c09c730 con 0x7f578c073d10 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5783400640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c0b49e0 0x7f578c09c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f578be00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f578c073d10 0x7f578c09bbf0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c410 tx=0x7f577c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5788c00640 1 -- 172.21.15.122:0/1859229113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c011790 con 0x7f578c073d10 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5788c00640 1 -- 172.21.15.122:0/1859229113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f577c017070 con 0x7f578c073d10 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5788c00640 1 -- 172.21.15.122:0/1859229113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c011ce0 con 0x7f578c073d10 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.035+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578c09ca10 con 0x7f578c073d10 2024-09-17T05:42:03.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.039+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578c09cfd0 con 0x7f578c073d10 2024-09-17T05:42:03.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.039+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578c069410 con 0x7f578c073d10 2024-09-17T05:42:03.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.039+0000 7f5788c00640 1 -- 172.21.15.122:0/1859229113 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f577c01c460 con 0x7f578c073d10 2024-09-17T05:42:03.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.039+0000 7f5788c00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f57680759f0 0x7f5768077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:03.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.039+0000 7f5788c00640 1 -- 172.21.15.122:0/1859229113 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f577c093320 con 0x7f578c073d10 2024-09-17T05:42:03.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.039+0000 7f5783400640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f57680759f0 0x7f5768077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:03.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.043+0000 7f5783400640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f57680759f0 0x7f5768077eb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f57740ae000 tx=0x7f577401b060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:03.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.043+0000 7f5788c00640 1 -- 172.21.15.122:0/1859229113 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f577c05fe30 con 0x7f578c073d10 2024-09-17T05:42:03.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.215+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f578c0cd100 con 0x7f578c073d10 2024-09-17T05:42:03.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.215+0000 7f5788c00640 1 -- 172.21.15.122:0/1859229113 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+13281 (secure 0 0 0) 0x7f577c05f730 con 0x7f578c073d10 2024-09-17T05:42:03.210 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:42:03.211 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":151,"fsid":"51547930-c98d-41c6-b8ce-f14d7623b707","created":"2024-09-17T05:24:39.149312+0000","modified":"2024-09-17T05:41:58.365453+0000","last_up_change":"2024-09-17T05:24:53.730497+0000","last_in_change":"2024-09-17T05:24:42.838555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:24:54.466798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"de145692-b14c-4df1-9dd6-975e867d1f5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6809","nonce":448781951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6811","nonce":448781951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6815","nonce":448781951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":448781951},{"type":"v1","addr":"172.21.15.122:6813","nonce":448781951}]},"public_addr":"172.21.15.122:6809/448781951","cluster_addr":"172.21.15.122:6811/448781951","heartbeat_back_addr":"172.21.15.122:6815/448781951","heartbeat_front_addr":"172.21.15.122:6813/448781951","state":["exists","up"]},{"osd":1,"uuid":"53a4f200-92b6-4a22-9c59-14a46e26f354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6825","nonce":2252207329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6827","nonce":2252207329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6831","nonce":2252207329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2252207329},{"type":"v1","addr":"172.21.15.122:6829","nonce":2252207329}]},"public_addr":"172.21.15.122:6825/2252207329","cluster_addr":"172.21.15.122:6827/2252207329","heartbeat_back_addr":"172.21.15.122:6831/2252207329","heartbeat_front_addr":"172.21.15.122:6829/2252207329","state":["exists","up"]},{"osd":2,"uuid":"68b85d73-8530-454d-94a0-ceeb46b20285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6817","nonce":1463445108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6819","nonce":1463445108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6823","nonce":1463445108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1463445108},{"type":"v1","addr":"172.21.15.122:6821","nonce":1463445108}]},"public_addr":"172.21.15.122:6817/1463445108","cluster_addr":"172.21.15.122:6819/1463445108","heartbeat_back_addr":"172.21.15.122:6823/1463445108","heartbeat_front_addr":"172.21.15.122:6821/1463445108","state":["exists","up"]},{"osd":3,"uuid":"27f0e2a8-c1d2-43ad-a0c3-a43e2cf9fd2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6801","nonce":3418036108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6803","nonce":3418036108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6807","nonce":3418036108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":3418036108},{"type":"v1","addr":"172.21.15.122:6805","nonce":3418036108}]},"public_addr":"172.21.15.122:6801/3418036108","cluster_addr":"172.21.15.122:6803/3418036108","heartbeat_back_addr":"172.21.15.122:6807/3418036108","heartbeat_front_addr":"172.21.15.122:6805/3418036108","state":["exists","up"]},{"osd":4,"uuid":"633269aa-0f64-4cf3-93a1-39fc33040a34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6811","nonce":1969238728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6813","nonce":1969238728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6817","nonce":1969238728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":1969238728},{"type":"v1","addr":"172.21.15.173:6815","nonce":1969238728}]},"public_addr":"172.21.15.173:6811/1969238728","cluster_addr":"172.21.15.173:6813/1969238728","heartbeat_back_addr":"172.21.15.173:6817/1969238728","heartbeat_front_addr":"172.21.15.173:6815/1969238728","state":["exists","up"]},{"osd":5,"uuid":"40d2464e-0cf8-448c-841a-794fc74d7d72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6803","nonce":2545281431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6805","nonce":2545281431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6809","nonce":2545281431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2545281431},{"type":"v1","addr":"172.21.15.173:6807","nonce":2545281431}]},"public_addr":"172.21.15.173:6803/2545281431","cluster_addr":"172.21.15.173:6805/2545281431","heartbeat_back_addr":"172.21.15.173:6809/2545281431","heartbeat_front_addr":"172.21.15.173:6807/2545281431","state":["exists","up"]},{"osd":6,"uuid":"97f6f4e4-4eae-445b-b8ea-5ec911516ef2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6827","nonce":2596490949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6829","nonce":2596490949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6833","nonce":2596490949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2596490949},{"type":"v1","addr":"172.21.15.173:6831","nonce":2596490949}]},"public_addr":"172.21.15.173:6827/2596490949","cluster_addr":"172.21.15.173:6829/2596490949","heartbeat_back_addr":"172.21.15.173:6833/2596490949","heartbeat_front_addr":"172.21.15.173:6831/2596490949","state":["exists","up"]},{"osd":7,"uuid":"8c603b20-eccf-447c-9117-dba8ff763577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6819","nonce":1087122755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6821","nonce":1087122755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6825","nonce":1087122755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1087122755},{"type":"v1","addr":"172.21.15.173:6823","nonce":1087122755}]},"public_addr":"172.21.15.173:6819/1087122755","cluster_addr":"172.21.15.173:6821/1087122755","heartbeat_back_addr":"172.21.15.173:6825/1087122755","heartbeat_front_addr":"172.21.15.173:6823/1087122755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.134142+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.955710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.120811+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.090091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.735618+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:49.839962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.195454+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:24:50.299712+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6835/4149068254":"2024-09-18T05:41:55.155690+0000","172.21.15.122:6833/4149068254":"2024-09-18T05:41:55.155690+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:42:03.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f57680759f0 msgr2=0x7f5768077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:03.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f57680759f0 0x7f5768077eb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f57740ae000 tx=0x7f577401b060 comp rx=0 tx=0).stop 2024-09-17T05:42:03.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f578c073d10 msgr2=0x7f578c09bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:03.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f578c073d10 0x7f578c09bbf0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f577c00c410 tx=0x7f577c00c8d0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 shutdown_connections 2024-09-17T05:42:03.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f57680759f0 0x7f5768077eb0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f578c15a660 0x7f578c143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f578c0b49e0 0x7f578c09c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 --2- 172.21.15.122:0/1859229113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f578c073d10 0x7f578c09bbf0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:03.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 >> 172.21.15.122:0/1859229113 conn(0x7f578c0b18f0 msgr2=0x7f578c155160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:03.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 shutdown_connections 2024-09-17T05:42:03.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:03.219+0000 7f5792a00640 1 -- 172.21.15.122:0/1859229113 wait complete. 2024-09-17T05:42:04.234 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T05:42:04.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:42:04.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.395+0000 7f8cf3c00640 1 -- 172.21.15.122:0/1154520909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec0d28e0 msgr2=0x7f8cec0d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.395+0000 7f8cf3c00640 1 --2- 172.21.15.122:0/1154520909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec0d28e0 0x7f8cec0d0340 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009900 tx=0x7f8cd801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 -- 172.21.15.122:0/1154520909 shutdown_connections 2024-09-17T05:42:04.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 --2- 172.21.15.122:0/1154520909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec0d28e0 0x7f8cec0d0340 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 -- 172.21.15.122:0/1154520909 >> 172.21.15.122:0/1154520909 conn(0x7f8cec0d1dc0 msgr2=0x7f8cec0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:04.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 -- 172.21.15.122:0/1154520909 shutdown_connections 2024-09-17T05:42:04.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 -- 172.21.15.122:0/1154520909 wait complete. 2024-09-17T05:42:04.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 Processor -- start 2024-09-17T05:42:04.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 -- start start 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cec0af850 0x7f8cec14e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cec0d28e0 0x7f8cec1f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec15f390 0x7f8cec14e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec169f00 con 0x7f8cec15f390 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec169d80 con 0x7f8cec0af850 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf3c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8cec16a080 con 0x7f8cec0d28e0 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec15f390 0x7f8cec14e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec15f390 0x7f8cec14e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58642/0 (socket says 172.21.15.122:58642) 2024-09-17T05:42:04.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 -- 172.21.15.122:0/4111651643 learned_addr learned my addr 172.21.15.122:0/4111651643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cebe00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cec0d28e0 0x7f8cec1f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1200640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cec0af850 0x7f8cec14e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 -- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cec0d28e0 msgr2=0x7f8cec1f88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cec0d28e0 0x7f8cec1f88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 -- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cec0af850 msgr2=0x7f8cec14e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cec0af850 0x7f8cec14e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 -- 172.21.15.122:0/4111651643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec0b5d50 con 0x7f8cec15f390 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.399+0000 7f8cf1c00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec15f390 0x7f8cec14e870 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400c7f0 tx=0x7f8ce400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4003b80 con 0x7f8cec15f390 2024-09-17T05:42:04.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ce4003d20 con 0x7f8cec15f390 2024-09-17T05:42:04.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400aab0 con 0x7f8cec15f390 2024-09-17T05:42:04.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cec0b6030 con 0x7f8cec15f390 2024-09-17T05:42:04.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cec0b65f0 con 0x7f8cec15f390 2024-09-17T05:42:04.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cec0afc30 con 0x7f8cec15f390 2024-09-17T05:42:04.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f8ce400ac50 con 0x7f8cec15f390 2024-09-17T05:42:04.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ccc0759f0 0x7f8ccc077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f8ce4093d80 con 0x7f8cec15f390 2024-09-17T05:42:04.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7f8ccc07b910 0x7f8ccc07dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 --> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ce4019e80 con 0x7f8ccc07b910 2024-09-17T05:42:04.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f8ce4094130 con 0x7f8cec15f390 2024-09-17T05:42:04.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8cf1200640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ccc0759f0 0x7f8ccc077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.403+0000 7f8cebe00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7f8ccc07b910 0x7f8ccc07dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.407+0000 7f8cebe00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7f8ccc07b910 0x7f8ccc07dd90 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:04.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.407+0000 7f8cf1200640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ccc0759f0 0x7f8ccc077eb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0007fd0 tx=0x7f8ce0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:04.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.407+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 <== osd.0 v2:172.21.15.122:6808/448781951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ce4019e80 con 0x7f8ccc07b910 2024-09-17T05:42:04.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 --> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8cec0b5a90 con 0x7f8ccc07b910 2024-09-17T05:42:04.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8ce8c00640 1 -- 172.21.15.122:0/4111651643 <== osd.0 v2:172.21.15.122:6808/448781951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8cec0b5a90 con 0x7f8ccc07b910 2024-09-17T05:42:04.414 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:42:04.414 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:42:04.414 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7f8ccc07b910 msgr2=0x7f8ccc07dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.122:6808/448781951,v1:172.21.15.122:6809/448781951] conn(0x7f8ccc07b910 0x7f8ccc07dd90 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ccc0759f0 msgr2=0x7f8ccc077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f8ccc0759f0 0x7f8ccc077eb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0007fd0 tx=0x7f8ce0009040 comp rx=0 tx=0).stop 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec15f390 msgr2=0x7f8cec14e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 --2- 172.21.15.122:0/4111651643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cec15f390 0x7f8cec14e870 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400c7f0 tx=0x7f8ce400ccb0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf1c00640 1 -- 172.21.15.122:0/4111651643 reap_dead start 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 shutdown_connections 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 >> 172.21.15.122:0/4111651643 conn(0x7f8cec0d1dc0 msgr2=0x7f8cec0b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:04.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 shutdown_connections 2024-09-17T05:42:04.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.419+0000 7f8cf3c00640 1 -- 172.21.15.122:0/4111651643 wait complete. 2024-09-17T05:42:04.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T05:42:04.538 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:42:04.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:42:04.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.583+0000 7fb32f200640 1 -- 172.21.15.122:0/441384251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb314005290 msgr2=0x7fb31400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.583+0000 7fb32f200640 1 --2- 172.21.15.122:0/441384251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb314005290 0x7fb31400b9d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c640 tx=0x7fb32801d260 comp rx=0 tx=0).stop 2024-09-17T05:42:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.583+0000 7fb32f200640 1 -- 172.21.15.122:0/441384251 shutdown_connections 2024-09-17T05:42:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.583+0000 7fb32f200640 1 --2- 172.21.15.122:0/441384251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb314005290 0x7fb31400b9d0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.583+0000 7fb32f200640 1 -- 172.21.15.122:0/441384251 >> 172.21.15.122:0/441384251 conn(0x7fb33009aa10 msgr2=0x7fb33009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.583+0000 7fb32f200640 1 -- 172.21.15.122:0/441384251 shutdown_connections 2024-09-17T05:42:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.583+0000 7fb32f200640 1 -- 172.21.15.122:0/441384251 wait complete. 2024-09-17T05:42:04.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 Processor -- start 2024-09-17T05:42:04.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 -- start start 2024-09-17T05:42:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb314005290 0x7fb3300c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb330071480 0x7fb3300c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33015aca0 0x7fb3301f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3300cf390 con 0x7fb33015aca0 2024-09-17T05:42:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3300cf210 con 0x7fb330071480 2024-09-17T05:42:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb3300b29e0 con 0x7fb314005290 2024-09-17T05:42:04.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33015aca0 0x7fb3301f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33015aca0 0x7fb3301f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58658/0 (socket says 172.21.15.122:58658) 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 -- 172.21.15.122:0/1358530792 learned_addr learned my addr 172.21.15.122:0/1358530792 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32de00640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb314005290 0x7fb3300c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32d400640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb330071480 0x7fb3300c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 -- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb314005290 msgr2=0x7fb3300c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb314005290 0x7fb3300c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 -- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb330071480 msgr2=0x7fb3300c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb330071480 0x7fb3300c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 -- 172.21.15.122:0/1358530792 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3301f54e0 con 0x7fb33015aca0 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32e800640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33015aca0 0x7fb3301f7b70 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fb32801db00 tx=0x7fb328004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb326000640 1 -- 172.21.15.122:0/1358530792 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328001880 con 0x7fb33015aca0 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb326000640 1 -- 172.21.15.122:0/1358530792 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb328001b10 con 0x7fb33015aca0 2024-09-17T05:42:04.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb326000640 1 -- 172.21.15.122:0/1358530792 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb328003be0 con 0x7fb33015aca0 2024-09-17T05:42:04.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3301f4960 con 0x7fb33015aca0 2024-09-17T05:42:04.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3301f4f20 con 0x7fb33015aca0 2024-09-17T05:42:04.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.587+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3300cc5b0 con 0x7fb33015aca0 2024-09-17T05:42:04.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.591+0000 7fb326000640 1 -- 172.21.15.122:0/1358530792 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fb328006070 con 0x7fb33015aca0 2024-09-17T05:42:04.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.591+0000 7fb326000640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb3100759f0 0x7fb310077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.591+0000 7fb326000640 1 -- 172.21.15.122:0/1358530792 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fb32812cf80 con 0x7fb33015aca0 2024-09-17T05:42:04.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.591+0000 7fb32de00640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb3100759f0 0x7fb310077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.591+0000 7fb32de00640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb3100759f0 0x7fb310077eb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb320000c10 tx=0x7fb320006b20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:04.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.595+0000 7fb326000640 1 -- 172.21.15.122:0/1358530792 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb3280c0030 con 0x7fb33015aca0 2024-09-17T05:42:04.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.763+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fb3300cc1b0 con 0x7fb3100759f0 2024-09-17T05:42:04.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.763+0000 7fb326000640 1 -- 172.21.15.122:0/1358530792 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb3300cc1b0 con 0x7fb3100759f0 2024-09-17T05:42:04.759 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb3100759f0 msgr2=0x7fb310077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb3100759f0 0x7fb310077eb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb320000c10 tx=0x7fb320006b20 comp rx=0 tx=0).stop 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33015aca0 msgr2=0x7fb3301f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33015aca0 0x7fb3301f7b70 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fb32801db00 tx=0x7fb328004e60 comp rx=0 tx=0).stop 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 shutdown_connections 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb3100759f0 0x7fb310077eb0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb33015aca0 0x7fb3301f7b70 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb330071480 0x7fb3300c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 --2- 172.21.15.122:0/1358530792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb314005290 0x7fb3300c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 >> 172.21.15.122:0/1358530792 conn(0x7fb33009aa10 msgr2=0x7fb33015e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:04.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 shutdown_connections 2024-09-17T05:42:04.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.767+0000 7fb32f200640 1 -- 172.21.15.122:0/1358530792 wait complete. 2024-09-17T05:42:04.782 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T05:42:04.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:42:04.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.931+0000 7f52c2c00640 1 -- 172.21.15.122:0/1535923852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0b0b60 msgr2=0x7f52bc166eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.931+0000 7f52c2c00640 1 --2- 172.21.15.122:0/1535923852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0b0b60 0x7f52bc166eb0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009bc0 tx=0x7f52b401f3b0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.931+0000 7f52c2c00640 1 -- 172.21.15.122:0/1535923852 shutdown_connections 2024-09-17T05:42:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.931+0000 7f52c2c00640 1 --2- 172.21.15.122:0/1535923852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc0b0b60 0x7f52bc166eb0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.931+0000 7f52c2c00640 1 -- 172.21.15.122:0/1535923852 >> 172.21.15.122:0/1535923852 conn(0x7f52bc0d1da0 msgr2=0x7f52bc0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.931+0000 7f52c2c00640 1 -- 172.21.15.122:0/1535923852 shutdown_connections 2024-09-17T05:42:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.931+0000 7f52c2c00640 1 -- 172.21.15.122:0/1535923852 wait complete. 2024-09-17T05:42:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 Processor -- start 2024-09-17T05:42:04.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 -- start start 2024-09-17T05:42:04.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc0b0b60 0x7f52bc14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc15be10 0x7f52bc159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc15c1f0 0x7f52bc0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1673f0 con 0x7f52bc15be10 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc0cddf0 con 0x7f52bc0b0b60 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f52bc167570 con 0x7f52bc15c1f0 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc15be10 0x7f52bc159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc15be10 0x7f52bc159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58670/0 (socket says 172.21.15.122:58670) 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 -- 172.21.15.122:0/2747640738 learned_addr learned my addr 172.21.15.122:0/2747640738 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52bbe00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc0b0b60 0x7f52bc14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c0c00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc15c1f0 0x7f52bc0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 -- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc15c1f0 msgr2=0x7f52bc0ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52bc15c1f0 0x7f52bc0ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 -- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc0b0b60 msgr2=0x7f52bc14eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52bc0b0b60 0x7f52bc14eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 -- 172.21.15.122:0/2747640738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc15a350 con 0x7f52bc15be10 2024-09-17T05:42:04.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b3400640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc15be10 0x7f52bc159e10 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c430 tx=0x7f52a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:04.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4011790 con 0x7f52bc15be10 2024-09-17T05:42:04.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a4017070 con 0x7f52bc15be10 2024-09-17T05:42:04.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4011ce0 con 0x7f52bc15be10 2024-09-17T05:42:04.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc15a5f0 con 0x7f52bc15be10 2024-09-17T05:42:04.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.935+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc15abb0 con 0x7f52bc15be10 2024-09-17T05:42:04.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52bc0ce700 con 0x7f52bc15be10 2024-09-17T05:42:04.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f52a401c460 con 0x7f52bc15be10 2024-09-17T05:42:04.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52b8c00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52980759f0 0x7f5298077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f52a4093d40 con 0x7f52bc15be10 2024-09-17T05:42:04.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52b8c00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7f529807b910 0x7f529807dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:04.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 --> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f529807e460 con 0x7f529807b910 2024-09-17T05:42:04.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f52a40940f0 con 0x7f52bc15be10 2024-09-17T05:42:04.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52bbe00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52980759f0 0x7f5298077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52c0c00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7f529807b910 0x7f529807dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:04.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52bbe00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52980759f0 0x7f5298077eb0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0c5420 tx=0x7f52ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.939+0000 7f52c0c00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7f529807b910 0x7f529807dd90 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.943+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 <== osd.1 v2:172.21.15.122:6824/2252207329 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f529807e460 con 0x7f529807b910 2024-09-17T05:42:04.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 --> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f52bc070350 con 0x7f529807b910 2024-09-17T05:42:04.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52b8c00640 1 -- 172.21.15.122:0/2747640738 <== osd.1 v2:172.21.15.122:6824/2252207329 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f52bc070350 con 0x7f529807b910 2024-09-17T05:42:04.949 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:42:04.950 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:42:04.950 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:42:04.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7f529807b910 msgr2=0x7f529807dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.122:6824/2252207329,v1:172.21.15.122:6825/2252207329] conn(0x7f529807b910 0x7f529807dd90 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52980759f0 msgr2=0x7f5298077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52980759f0 0x7f5298077eb0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0c5420 tx=0x7f52ac009040 comp rx=0 tx=0).stop 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc15be10 msgr2=0x7f52bc159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 --2- 172.21.15.122:0/2747640738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52bc15be10 0x7f52bc159e10 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c430 tx=0x7f52a400c8f0 comp rx=0 tx=0).stop 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c0c00640 1 -- 172.21.15.122:0/2747640738 reap_dead start 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 shutdown_connections 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 >> 172.21.15.122:0/2747640738 conn(0x7f52bc0d1da0 msgr2=0x7f52bc0b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 shutdown_connections 2024-09-17T05:42:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:04.955+0000 7f52c2c00640 1 -- 172.21.15.122:0/2747640738 wait complete. 2024-09-17T05:42:04.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T05:42:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.123+0000 7fa087800640 1 -- 172.21.15.122:0/1127798329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 msgr2=0x7fa064005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.123+0000 7fa087800640 1 --2- 172.21.15.122:0/1127798329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 0x7fa064005240 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa07800c640 tx=0x7fa07801d230 comp rx=0 tx=0).stop 2024-09-17T05:42:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.123+0000 7fa085800640 1 -- 172.21.15.122:0/1127798329 reap_dead start 2024-09-17T05:42:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.123+0000 7fa087800640 1 -- 172.21.15.122:0/1127798329 shutdown_connections 2024-09-17T05:42:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.123+0000 7fa087800640 1 -- 172.21.15.122:0/1127798329 >> 172.21.15.122:0/1127798329 conn(0x7fa0800d1e30 msgr2=0x7fa0800d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.123+0000 7fa087800640 1 -- 172.21.15.122:0/1127798329 shutdown_connections 2024-09-17T05:42:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.123+0000 7fa087800640 1 -- 172.21.15.122:0/1127798329 wait complete. 2024-09-17T05:42:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.123+0000 7fa087800640 1 Processor -- start 2024-09-17T05:42:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 -- start start 2024-09-17T05:42:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 0x7fa0801fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa080151c40 0x7fa08006e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa08006e690 0x7fa080159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa080167e00 con 0x7fa064004dc0 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa080167c80 con 0x7fa08006e690 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa080167f80 con 0x7fa080151c40 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 0x7fa0801fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 0x7fa0801fb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58686/0 (socket says 172.21.15.122:58686) 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 -- 172.21.15.122:0/4149359100 learned_addr learned my addr 172.21.15.122:0/4149359100 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa077e00640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa080151c40 0x7fa08006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa085800640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa08006e690 0x7fa080159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 -- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa080151c40 msgr2=0x7fa08006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa080151c40 0x7fa08006e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 -- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa08006e690 msgr2=0x7fa080159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa08006e690 0x7fa080159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 -- 172.21.15.122:0/4149359100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0801f7660 con 0x7fa064004dc0 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa085800640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa08006e690 0x7fa080159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa084e00640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 0x7fa0801fb6b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa070003ea0 tx=0x7fa070003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa07d600640 1 -- 172.21.15.122:0/4149359100 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070015070 con 0x7fa064004dc0 2024-09-17T05:42:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa07d600640 1 -- 172.21.15.122:0/4149359100 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa070011920 con 0x7fa064004dc0 2024-09-17T05:42:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa07d600640 1 -- 172.21.15.122:0/4149359100 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070011d70 con 0x7fa064004dc0 2024-09-17T05:42:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08015b930 con 0x7fa064004dc0 2024-09-17T05:42:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0801f7c10 con 0x7fa064004dc0 2024-09-17T05:42:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.127+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08006eaa0 con 0x7fa064004dc0 2024-09-17T05:42:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.131+0000 7fa07d600640 1 -- 172.21.15.122:0/4149359100 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fa070018070 con 0x7fa064004dc0 2024-09-17T05:42:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.131+0000 7fa07d600640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c075a20 0x7fa05c077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.131+0000 7fa07d600640 1 -- 172.21.15.122:0/4149359100 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fa0700942a0 con 0x7fa064004dc0 2024-09-17T05:42:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.131+0000 7fa077e00640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c075a20 0x7fa05c077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.131+0000 7fa077e00640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c075a20 0x7fa05c077ee0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa0800c86a0 tx=0x7fa0680091e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:05.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.135+0000 7fa07d600640 1 -- 172.21.15.122:0/4149359100 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa08006eaa0 con 0x7fa064004dc0 2024-09-17T05:42:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.299+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fa0800b0130 con 0x7fa05c075a20 2024-09-17T05:42:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.299+0000 7fa07d600640 1 -- 172.21.15.122:0/4149359100 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa0800b0130 con 0x7fa05c075a20 2024-09-17T05:42:05.295 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T05:42:05.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c075a20 msgr2=0x7fa05c077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c075a20 0x7fa05c077ee0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa0800c86a0 tx=0x7fa0680091e0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 msgr2=0x7fa0801fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 0x7fa0801fb6b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa070003ea0 tx=0x7fa070003f90 comp rx=0 tx=0).stop 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 shutdown_connections 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa05c075a20 0x7fa05c077ee0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa08006e690 0x7fa080159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa080151c40 0x7fa08006e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 --2- 172.21.15.122:0/4149359100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa064004dc0 0x7fa0801fb6b0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 >> 172.21.15.122:0/4149359100 conn(0x7fa0800d1e30 msgr2=0x7fa08006f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 shutdown_connections 2024-09-17T05:42:05.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.303+0000 7fa087800640 1 -- 172.21.15.122:0/4149359100 wait complete. 2024-09-17T05:42:05.317 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T05:42:05.317 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:42:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- 172.21.15.122:0/664484603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 msgr2=0x7f9d6c15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 --2- 172.21.15.122:0/664484603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 0x7f9d6c15f410 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c06dc00 tx=0x7f9d6401cd50 comp rx=0 tx=0).stop 2024-09-17T05:42:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- 172.21.15.122:0/664484603 shutdown_connections 2024-09-17T05:42:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 --2- 172.21.15.122:0/664484603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 0x7f9d6c15f410 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 --2- 172.21.15.122:0/664484603 >> v2:172.21.15.173:3300/0 conn(0x7f9d6c0d09f0 0x7f9d6c0d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- 172.21.15.122:0/664484603 >> 172.21.15.122:0/664484603 conn(0x7f9d6c0d1da0 msgr2=0x7f9d6c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:05.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- 172.21.15.122:0/664484603 shutdown_connections 2024-09-17T05:42:05.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- 172.21.15.122:0/664484603 wait complete. 2024-09-17T05:42:05.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 Processor -- start 2024-09-17T05:42:05.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- start start 2024-09-17T05:42:05.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d6c0cff00 0x7f9d6c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 0x7f9d6c0c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6c0ccae0 0x7f9d6c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c0b0bd0 con 0x7f9d6c15eff0 2024-09-17T05:42:05.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6c0b0a50 con 0x7f9d6c0cff00 2024-09-17T05:42:05.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6be00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6c15c1b0 con 0x7f9d6c0ccae0 2024-09-17T05:42:05.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 0x7f9d6c0c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 0x7f9d6c0c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58698/0 (socket says 172.21.15.122:58698) 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6a000640 1 -- 172.21.15.122:0/100134142 learned_addr learned my addr 172.21.15.122:0/100134142 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6b400640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6c0ccae0 0x7f9d6c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6aa00640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d6c0cff00 0x7f9d6c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6a000640 1 -- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6c0ccae0 msgr2=0x7f9d6c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6a000640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6c0ccae0 0x7f9d6c153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6a000640 1 -- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d6c0cff00 msgr2=0x7f9d6c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6a000640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d6c0cff00 0x7f9d6c0ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6a000640 1 -- 172.21.15.122:0/100134142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6c0ceea0 con 0x7f9d6c15eff0 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6b400640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6c0ccae0 0x7f9d6c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:42:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.471+0000 7f9d6aa00640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d6c0cff00 0x7f9d6c0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:42:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d6a000640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 0x7f9d6c0c5e50 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c970 tx=0x7f9d5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58007c70 con 0x7f9d6c15eff0 2024-09-17T05:42:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d5800f450 con 0x7f9d6c15eff0 2024-09-17T05:42:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6c0cf180 con 0x7f9d6c15eff0 2024-09-17T05:42:05.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800f8a0 con 0x7f9d6c15eff0 2024-09-17T05:42:05.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6c0ccf50 con 0x7f9d6c15eff0 2024-09-17T05:42:05.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d6c0c67d0 con 0x7f9d6c15eff0 2024-09-17T05:42:05.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f9d58018400 con 0x7f9d6c15eff0 2024-09-17T05:42:05.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d4c0759f0 0x7f9d4c077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f9d58094600 con 0x7f9d6c15eff0 2024-09-17T05:42:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7f9d4c07b910 0x7f9d4c07dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d6aa00640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d4c0759f0 0x7f9d4c077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 --> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d4c07e460 con 0x7f9d4c07b910 2024-09-17T05:42:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.475+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f9d58094d60 con 0x7f9d6c15eff0 2024-09-17T05:42:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.479+0000 7f9d6b400640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7f9d4c07b910 0x7f9d4c07dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.479+0000 7f9d6aa00640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d4c0759f0 0x7f9d4c077eb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c001750 tx=0x7f9d5c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.479+0000 7f9d6b400640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7f9d4c07b910 0x7f9d4c07dd90 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.479+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 <== osd.2 v2:172.21.15.122:6816/1463445108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9d4c07e460 con 0x7f9d4c07b910 2024-09-17T05:42:05.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 --> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9d6c154310 con 0x7f9d4c07b910 2024-09-17T05:42:05.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d63400640 1 -- 172.21.15.122:0/100134142 <== osd.2 v2:172.21.15.122:6816/1463445108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9d6c154310 con 0x7f9d4c07b910 2024-09-17T05:42:05.486 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:42:05.486 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:42:05.486 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7f9d4c07b910 msgr2=0x7f9d4c07dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.122:6816/1463445108,v1:172.21.15.122:6817/1463445108] conn(0x7f9d4c07b910 0x7f9d4c07dd90 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d4c0759f0 msgr2=0x7f9d4c077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d4c0759f0 0x7f9d4c077eb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c001750 tx=0x7f9d5c009040 comp rx=0 tx=0).stop 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 msgr2=0x7f9d6c0c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 --2- 172.21.15.122:0/100134142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6c15eff0 0x7f9d6c0c5e50 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c970 tx=0x7f9d5800ce30 comp rx=0 tx=0).stop 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6b400640 1 -- 172.21.15.122:0/100134142 reap_dead start 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 shutdown_connections 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 >> 172.21.15.122:0/100134142 conn(0x7f9d6c0d1da0 msgr2=0x7f9d6c071100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 shutdown_connections 2024-09-17T05:42:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.491+0000 7f9d6be00640 1 -- 172.21.15.122:0/100134142 wait complete. 2024-09-17T05:42:05.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T05:42:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.651+0000 7f12e9c00640 1 -- 172.21.15.122:0/3151704575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 msgr2=0x7f12c8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.651+0000 7f12e9c00640 1 --2- 172.21.15.122:0/3151704575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 0x7f12c8004810 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f12d0006f70 tx=0x7f12d001cec0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.651+0000 7f12e3400640 1 -- 172.21.15.122:0/3151704575 reap_dead start 2024-09-17T05:42:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.651+0000 7f12e9c00640 1 -- 172.21.15.122:0/3151704575 shutdown_connections 2024-09-17T05:42:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.651+0000 7f12e9c00640 1 -- 172.21.15.122:0/3151704575 >> 172.21.15.122:0/3151704575 conn(0x7f12e40d1dc0 msgr2=0x7f12e40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.651+0000 7f12e9c00640 1 -- 172.21.15.122:0/3151704575 shutdown_connections 2024-09-17T05:42:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.651+0000 7f12e9c00640 1 -- 172.21.15.122:0/3151704575 wait complete. 2024-09-17T05:42:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 Processor -- start 2024-09-17T05:42:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 -- start start 2024-09-17T05:42:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 0x7f12e40b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12e40ca0f0 0x7f12e41660e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12e40ca570 0x7f12e4166620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4167d00 con 0x7f12c8003f80 2024-09-17T05:42:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4167b80 con 0x7f12e40ca0f0 2024-09-17T05:42:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f12e4167e80 con 0x7f12e40ca570 2024-09-17T05:42:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 0x7f12e40b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 0x7f12e40b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58718/0 (socket says 172.21.15.122:58718) 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 -- 172.21.15.122:0/3317382755 learned_addr learned my addr 172.21.15.122:0/3317382755 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2000640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12e40ca0f0 0x7f12e41660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e3400640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12e40ca570 0x7f12e4166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 -- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12e40ca570 msgr2=0x7f12e4166620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12e40ca570 0x7f12e4166620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 -- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12e40ca0f0 msgr2=0x7f12e41660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12e40ca0f0 0x7f12e41660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 -- 172.21.15.122:0/3317382755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e4165360 con 0x7f12c8003f80 2024-09-17T05:42:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e2a00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 0x7f12e40b6b20 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f12d4011b10 tx=0x7f12d4009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:05.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12daa00640 1 -- 172.21.15.122:0/3317382755 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d401e070 con 0x7f12c8003f80 2024-09-17T05:42:05.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12daa00640 1 -- 172.21.15.122:0/3317382755 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12d4004050 con 0x7f12c8003f80 2024-09-17T05:42:05.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12daa00640 1 -- 172.21.15.122:0/3317382755 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d400fea0 con 0x7f12c8003f80 2024-09-17T05:42:05.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e41fba80 con 0x7f12c8003f80 2024-09-17T05:42:05.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.655+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e41658e0 con 0x7f12c8003f80 2024-09-17T05:42:05.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.659+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e40cc990 con 0x7f12c8003f80 2024-09-17T05:42:05.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.659+0000 7f12daa00640 1 -- 172.21.15.122:0/3317382755 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f12d4021070 con 0x7f12c8003f80 2024-09-17T05:42:05.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.659+0000 7f12daa00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12c4075a20 0x7f12c4077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.659+0000 7f12daa00640 1 -- 172.21.15.122:0/3317382755 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f12d4093f50 con 0x7f12c8003f80 2024-09-17T05:42:05.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.659+0000 7f12e2000640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12c4075a20 0x7f12c4077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.659+0000 7f12e2000640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12c4075a20 0x7f12c4077ee0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f12d00ab000 tx=0x7f12d0019d00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:05.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.663+0000 7f12daa00640 1 -- 172.21.15.122:0/3317382755 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f12d40604d0 con 0x7f12c8003f80 2024-09-17T05:42:05.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.831+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f12e40b5350 con 0x7f12c4075a20 2024-09-17T05:42:05.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.831+0000 7f12daa00640 1 -- 172.21.15.122:0/3317382755 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f12e40b5350 con 0x7f12c4075a20 2024-09-17T05:42:05.826 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T05:42:05.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12c4075a20 msgr2=0x7f12c4077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12c4075a20 0x7f12c4077ee0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f12d00ab000 tx=0x7f12d0019d00 comp rx=0 tx=0).stop 2024-09-17T05:42:05.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 msgr2=0x7f12e40b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 0x7f12e40b6b20 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f12d4011b10 tx=0x7f12d4009ea0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 shutdown_connections 2024-09-17T05:42:05.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f12c4075a20 0x7f12c4077ee0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12e40ca570 0x7f12e4166620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12e40ca0f0 0x7f12e41660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 --2- 172.21.15.122:0/3317382755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c8003f80 0x7f12e40b6b20 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 >> 172.21.15.122:0/3317382755 conn(0x7f12e40d1dc0 msgr2=0x7f12e4167200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:05.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 shutdown_connections 2024-09-17T05:42:05.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.835+0000 7f12e9c00640 1 -- 172.21.15.122:0/3317382755 wait complete. 2024-09-17T05:42:05.848 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T05:42:05.848 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:42:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 -- 172.21.15.122:0/1271004791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 msgr2=0x7f93640d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 --2- 172.21.15.122:0/1271004791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 0x7f93640d28c0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f9354009900 tx=0x7f935401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 -- 172.21.15.122:0/1271004791 shutdown_connections 2024-09-17T05:42:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 --2- 172.21.15.122:0/1271004791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93640d02e0 0x7f93640d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 --2- 172.21.15.122:0/1271004791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 0x7f93640d28c0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 -- 172.21.15.122:0/1271004791 >> 172.21.15.122:0/1271004791 conn(0x7f93640d1da0 msgr2=0x7f93640d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 -- 172.21.15.122:0/1271004791 shutdown_connections 2024-09-17T05:42:05.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 -- 172.21.15.122:0/1271004791 wait complete. 2024-09-17T05:42:05.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 Processor -- start 2024-09-17T05:42:05.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:05.999+0000 7f9369200640 1 -- start start 2024-09-17T05:42:05.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 0x7f93641fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93640d02e0 0x7f9364154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f936415eff0 0x7f936415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:05.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93641690f0 con 0x7f93640cff00 2024-09-17T05:42:05.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9364168f70 con 0x7f936415eff0 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9364169270 con 0x7f93640d02e0 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 0x7f93641fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 0x7f93641fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58726/0 (socket says 172.21.15.122:58726) 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 -- 172.21.15.122:0/126556899 learned_addr learned my addr 172.21.15.122:0/126556899 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 -- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93640d02e0 msgr2=0x7f9364154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93640d02e0 0x7f9364154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 -- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f936415eff0 msgr2=0x7f936415b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f936415eff0 0x7f936415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 -- 172.21.15.122:0/126556899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93641517b0 con 0x7f93640cff00 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9363400640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 0x7f93641fba20 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f93540b0040 tx=0x7f9354005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93540b2040 con 0x7f93640cff00 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9354003600 con 0x7f93640cff00 2024-09-17T05:42:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93540039f0 con 0x7f93640cff00 2024-09-17T05:42:05.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93641f8510 con 0x7f93640cff00 2024-09-17T05:42:05.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93641fb500 con 0x7f93640cff00 2024-09-17T05:42:05.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.003+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93640d28c0 con 0x7f93640cff00 2024-09-17T05:42:05.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f93540bf020 con 0x7f93640cff00 2024-09-17T05:42:06.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f935b400640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93440759f0 0x7f9344077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f935412ea40 con 0x7f93640cff00 2024-09-17T05:42:06.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f935b400640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7f934407b910 0x7f934407dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 --> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93540cae50 con 0x7f934407b910 2024-09-17T05:42:06.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f935412edf0 con 0x7f93640cff00 2024-09-17T05:42:06.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f9362a00640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93440759f0 0x7f9344077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f9363e00640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7f934407b910 0x7f934407dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f9363e00640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7f934407b910 0x7f934407dd90 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f9362a00640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93440759f0 0x7f9344077eb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f93500058d0 tx=0x7f9350005860 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.007+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 <== osd.3 v2:172.21.15.122:6800/3418036108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93540cae50 con 0x7f934407b910 2024-09-17T05:42:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.019+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 --> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f93641fb000 con 0x7f934407b910 2024-09-17T05:42:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.019+0000 7f935b400640 1 -- 172.21.15.122:0/126556899 <== osd.3 v2:172.21.15.122:6800/3418036108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f93641fb000 con 0x7f934407b910 2024-09-17T05:42:06.015 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:42:06.015 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:42:06.015 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:42:06.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7f934407b910 msgr2=0x7f934407dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.122:6800/3418036108,v1:172.21.15.122:6801/3418036108] conn(0x7f934407b910 0x7f934407dd90 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93440759f0 msgr2=0x7f9344077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f93440759f0 0x7f9344077eb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f93500058d0 tx=0x7f9350005860 comp rx=0 tx=0).stop 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 msgr2=0x7f93641fba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 --2- 172.21.15.122:0/126556899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cff00 0x7f93641fba20 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f93540b0040 tx=0x7f9354005de0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9363e00640 1 -- 172.21.15.122:0/126556899 reap_dead start 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 shutdown_connections 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 >> 172.21.15.122:0/126556899 conn(0x7f93640d1da0 msgr2=0x7f93640af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 shutdown_connections 2024-09-17T05:42:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.023+0000 7f9369200640 1 -- 172.21.15.122:0/126556899 wait complete. 2024-09-17T05:42:06.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T05:42:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.179+0000 7f9d4da00640 1 -- 172.21.15.122:0/153540881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 msgr2=0x7f9d480d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.179+0000 7f9d4da00640 1 --2- 172.21.15.122:0/153540881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 0x7f9d480d0260 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9d480b33d0 tx=0x7f9d3001c7c0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.179+0000 7f9d4da00640 1 -- 172.21.15.122:0/153540881 shutdown_connections 2024-09-17T05:42:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.179+0000 7f9d4da00640 1 --2- 172.21.15.122:0/153540881 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d480b2d00 0x7f9d481605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.179+0000 7f9d4da00640 1 --2- 172.21.15.122:0/153540881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 0x7f9d480d0260 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.179+0000 7f9d4da00640 1 -- 172.21.15.122:0/153540881 >> 172.21.15.122:0/153540881 conn(0x7f9d480d1e40 msgr2=0x7f9d480d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.179+0000 7f9d4da00640 1 -- 172.21.15.122:0/153540881 shutdown_connections 2024-09-17T05:42:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.179+0000 7f9d4da00640 1 -- 172.21.15.122:0/153540881 wait complete. 2024-09-17T05:42:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 Processor -- start 2024-09-17T05:42:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 -- start start 2024-09-17T05:42:06.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 0x7f9d480b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d480b2d00 0x7f9d4815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d480cfe80 0x7f9d4815bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480b0940 con 0x7f9d480b2760 2024-09-17T05:42:06.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480b07c0 con 0x7f9d480cfe80 2024-09-17T05:42:06.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9d480b0ac0 con 0x7f9d480b2d00 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 0x7f9d480b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 0x7f9d480b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58736/0 (socket says 172.21.15.122:58736) 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 -- 172.21.15.122:0/600521793 learned_addr learned my addr 172.21.15.122:0/600521793 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d3e000640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d480b2d00 0x7f9d4815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d47400640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d480cfe80 0x7f9d4815bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 -- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d480b2d00 msgr2=0x7f9d4815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d480b2d00 0x7f9d4815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 -- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d480cfe80 msgr2=0x7f9d4815bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d480cfe80 0x7f9d4815bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 -- 172.21.15.122:0/600521793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d480b6b30 con 0x7f9d480b2760 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d3e000640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d480b2d00 0x7f9d4815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d46a00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 0x7f9d480b6230 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c3f0 tx=0x7f9d3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d3f400640 1 -- 172.21.15.122:0/600521793 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38017070 con 0x7f9d480b2760 2024-09-17T05:42:06.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d3f400640 1 -- 172.21.15.122:0/600521793 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d380053c0 con 0x7f9d480b2760 2024-09-17T05:42:06.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d481f5750 con 0x7f9d480b2760 2024-09-17T05:42:06.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d3f400640 1 -- 172.21.15.122:0/600521793 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38003860 con 0x7f9d480b2760 2024-09-17T05:42:06.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.183+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d481f5d10 con 0x7f9d480b2760 2024-09-17T05:42:06.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.187+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d481f5f10 con 0x7f9d480b2760 2024-09-17T05:42:06.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.187+0000 7f9d3f400640 1 -- 172.21.15.122:0/600521793 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f9d3801a070 con 0x7f9d480b2760 2024-09-17T05:42:06.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.187+0000 7f9d3f400640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d240759f0 0x7f9d24077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.187+0000 7f9d3f400640 1 -- 172.21.15.122:0/600521793 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f9d380928c0 con 0x7f9d480b2760 2024-09-17T05:42:06.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.187+0000 7f9d3e000640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d240759f0 0x7f9d24077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.187+0000 7f9d3e000640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d240759f0 0x7f9d24077eb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9d300ae000 tx=0x7f9d3001b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.191+0000 7f9d3f400640 1 -- 172.21.15.122:0/600521793 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f9d481f5f10 con 0x7f9d480b2760 2024-09-17T05:42:06.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f9d480d0260 con 0x7f9d240759f0 2024-09-17T05:42:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d3f400640 1 -- 172.21.15.122:0/600521793 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9d480d0260 con 0x7f9d240759f0 2024-09-17T05:42:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T05:42:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d240759f0 msgr2=0x7f9d24077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d240759f0 0x7f9d24077eb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9d300ae000 tx=0x7f9d3001b000 comp rx=0 tx=0).stop 2024-09-17T05:42:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 msgr2=0x7f9d480b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 0x7f9d480b6230 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c3f0 tx=0x7f9d3800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 shutdown_connections 2024-09-17T05:42:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f9d240759f0 0x7f9d24077eb0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d480cfe80 0x7f9d4815bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d480b2d00 0x7f9d4815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 --2- 172.21.15.122:0/600521793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d480b2760 0x7f9d480b6230 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 >> 172.21.15.122:0/600521793 conn(0x7f9d480d1e40 msgr2=0x7f9d48159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 shutdown_connections 2024-09-17T05:42:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.359+0000 7f9d4da00640 1 -- 172.21.15.122:0/600521793 wait complete. 2024-09-17T05:42:06.373 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T05:42:06.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:42:06.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- 172.21.15.122:0/3816895955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 msgr2=0x7f67600ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 --2- 172.21.15.122:0/3816895955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 0x7f67600ae490 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f674c009840 tx=0x7f674c01c790 comp rx=0 tx=0).stop 2024-09-17T05:42:06.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- 172.21.15.122:0/3816895955 shutdown_connections 2024-09-17T05:42:06.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 --2- 172.21.15.122:0/3816895955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 0x7f67600ae490 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 --2- 172.21.15.122:0/3816895955 >> v2:172.21.15.173:3300/0 conn(0x7f676015e360 0x7f67600adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- 172.21.15.122:0/3816895955 >> 172.21.15.122:0/3816895955 conn(0x7f67600cfbc0 msgr2=0x7f67600cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:06.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- 172.21.15.122:0/3816895955 shutdown_connections 2024-09-17T05:42:06.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- 172.21.15.122:0/3816895955 wait complete. 2024-09-17T05:42:06.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 Processor -- start 2024-09-17T05:42:06.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- start start 2024-09-17T05:42:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67600ae9d0 0x7f676014b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 0x7f67601fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676015e360 0x7f67601fb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67601680b0 con 0x7f676015e360 2024-09-17T05:42:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6760167f30 con 0x7f67600ae9d0 2024-09-17T05:42:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6767800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6760168230 con 0x7f67600aef60 2024-09-17T05:42:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6765800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676015e360 0x7f67601fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f675fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 0x7f67601fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f675fe00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 0x7f67601fb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.122:34958/0 (socket says 172.21.15.122:34958) 2024-09-17T05:42:06.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6765800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676015e360 0x7f67601fb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58748/0 (socket says 172.21.15.122:58748) 2024-09-17T05:42:06.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f675fe00640 1 -- 172.21.15.122:0/1955836988 learned_addr learned my addr 172.21.15.122:0/1955836988 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:06.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.527+0000 7f6764e00640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67600ae9d0 0x7f676014b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675fe00640 1 -- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67600ae9d0 msgr2=0x7f676014b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675fe00640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67600ae9d0 0x7f676014b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675fe00640 1 -- 172.21.15.122:0/1955836988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676015e360 msgr2=0x7f67601fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675fe00640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676015e360 0x7f67601fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675fe00640 1 -- 172.21.15.122:0/1955836988 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676009b890 con 0x7f67600aef60 2024-09-17T05:42:06.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675fe00640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 0x7f67601fb0d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f674c000c10 tx=0x7f674c0a6f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0ac070 con 0x7f67600aef60 2024-09-17T05:42:06.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f674c004310 con 0x7f67600aef60 2024-09-17T05:42:06.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c005290 con 0x7f67600aef60 2024-09-17T05:42:06.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676009d250 con 0x7f67600aef60 2024-09-17T05:42:06.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f676009cf40 con 0x7f67600aef60 2024-09-17T05:42:06.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6760072870 con 0x7f67600aef60 2024-09-17T05:42:06.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f674c0af070 con 0x7f67600aef60 2024-09-17T05:42:06.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.531+0000 7f675cc00640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67400759f0 0x7f6740077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f674c12d9c0 con 0x7f67600aef60 2024-09-17T05:42:06.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f675cc00640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7f674007b910 0x7f674007dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f6764e00640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67400759f0 0x7f6740077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 --> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f674c00b0b0 con 0x7f674007b910 2024-09-17T05:42:06.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f674c12e100 con 0x7f67600aef60 2024-09-17T05:42:06.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f6765800640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7f674007b910 0x7f674007dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f6764e00640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67400759f0 0x7f6740077eb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6750009ec0 tx=0x7f6750009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f6765800640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7f674007b910 0x7f674007dd90 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.535+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 <== osd.4 v2:172.21.15.173:6810/1969238728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f674c00b0b0 con 0x7f674007b910 2024-09-17T05:42:06.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.547+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 --> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f67600d0850 con 0x7f674007b910 2024-09-17T05:42:06.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f675cc00640 1 -- 172.21.15.122:0/1955836988 <== osd.4 v2:172.21.15.173:6810/1969238728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f67600d0850 con 0x7f674007b910 2024-09-17T05:42:06.544 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:42:06.544 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:42:06.544 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:42:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7f674007b910 msgr2=0x7f674007dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6810/1969238728,v1:172.21.15.173:6811/1969238728] conn(0x7f674007b910 0x7f674007dd90 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67400759f0 msgr2=0x7f6740077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f67400759f0 0x7f6740077eb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6750009ec0 tx=0x7f6750009500 comp rx=0 tx=0).stop 2024-09-17T05:42:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 msgr2=0x7f67601fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 --2- 172.21.15.122:0/1955836988 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67600aef60 0x7f67601fb0d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f674c000c10 tx=0x7f674c0a6f90 comp rx=0 tx=0).stop 2024-09-17T05:42:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6765800640 1 -- 172.21.15.122:0/1955836988 reap_dead start 2024-09-17T05:42:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 shutdown_connections 2024-09-17T05:42:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 >> 172.21.15.122:0/1955836988 conn(0x7f67600cfbc0 msgr2=0x7f6760163e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 shutdown_connections 2024-09-17T05:42:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.551+0000 7f6767800640 1 -- 172.21.15.122:0/1955836988 wait complete. 2024-09-17T05:42:06.562 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T05:42:06.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 -- 172.21.15.122:0/2601354180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb118003a10 msgr2=0x7fb118003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 --2- 172.21.15.122:0/2601354180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb118003a10 0x7fb118003df0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb124009b60 tx=0x7fb12401d270 comp rx=0 tx=0).stop 2024-09-17T05:42:06.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 -- 172.21.15.122:0/2601354180 shutdown_connections 2024-09-17T05:42:06.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 --2- 172.21.15.122:0/2601354180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb118003a10 0x7fb118003df0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 -- 172.21.15.122:0/2601354180 >> 172.21.15.122:0/2601354180 conn(0x7fb1340d1da0 msgr2=0x7fb1340d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:06.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 -- 172.21.15.122:0/2601354180 shutdown_connections 2024-09-17T05:42:06.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 -- 172.21.15.122:0/2601354180 wait complete. 2024-09-17T05:42:06.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 Processor -- start 2024-09-17T05:42:06.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.711+0000 7fb13a000640 1 -- start start 2024-09-17T05:42:06.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb118003a10 0x7fb1340cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb13414b030 0x7fb13406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1341f5900 0x7fb1341f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1340ce870 con 0x7fb1341f5900 2024-09-17T05:42:06.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1340ce6f0 con 0x7fb118003a10 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb1340ce9f0 con 0x7fb13414b030 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1341f5900 0x7fb1341f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1341f5900 0x7fb1341f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58760/0 (socket says 172.21.15.122:58760) 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 -- 172.21.15.122:0/3889277188 learned_addr learned my addr 172.21.15.122:0/3889277188 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133400640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb118003a10 0x7fb1340cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb12aa00640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb13414b030 0x7fb13406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 -- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb13414b030 msgr2=0x7fb13406efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb13414b030 0x7fb13406efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 -- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb118003a10 msgr2=0x7fb1340cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb118003a10 0x7fb1340cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 -- 172.21.15.122:0/3889277188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1340c83d0 con 0x7fb1341f5900 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb12aa00640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb13414b030 0x7fb13406efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:42:06.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb133e00640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1341f5900 0x7fb1341f5d90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb1340c58a0 tx=0x7fb12c006770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb12be00640 1 -- 172.21.15.122:0/3889277188 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c003d90 con 0x7fb1341f5900 2024-09-17T05:42:06.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb12be00640 1 -- 172.21.15.122:0/3889277188 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb12c011840 con 0x7fb1341f5900 2024-09-17T05:42:06.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1340c86b0 con 0x7fb1341f5900 2024-09-17T05:42:06.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb12be00640 1 -- 172.21.15.122:0/3889277188 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c011c70 con 0x7fb1341f5900 2024-09-17T05:42:06.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13406eac0 con 0x7fb1341f5900 2024-09-17T05:42:06.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.715+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13406ecc0 con 0x7fb1341f5900 2024-09-17T05:42:06.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.719+0000 7fb12be00640 1 -- 172.21.15.122:0/3889277188 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fb12c005070 con 0x7fb1341f5900 2024-09-17T05:42:06.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.719+0000 7fb12be00640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb110075a20 0x7fb110077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:06.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.719+0000 7fb12be00640 1 -- 172.21.15.122:0/3889277188 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fb12c097aa0 con 0x7fb1341f5900 2024-09-17T05:42:06.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.719+0000 7fb133400640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb110075a20 0x7fb110077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:06.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.719+0000 7fb133400640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb110075a20 0x7fb110077ee0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb1240ae000 tx=0x7fb12401a010 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:06.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.719+0000 7fb12be00640 1 -- 172.21.15.122:0/3889277188 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fb13406ecc0 con 0x7fb1341f5900 2024-09-17T05:42:06.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.891+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fb1340b6620 con 0x7fb110075a20 2024-09-17T05:42:06.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.891+0000 7fb12be00640 1 -- 172.21.15.122:0/3889277188 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb1340b6620 con 0x7fb110075a20 2024-09-17T05:42:06.886 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb110075a20 msgr2=0x7fb110077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb110075a20 0x7fb110077ee0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb1240ae000 tx=0x7fb12401a010 comp rx=0 tx=0).stop 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1341f5900 msgr2=0x7fb1341f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1341f5900 0x7fb1341f5d90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb1340c58a0 tx=0x7fb12c006770 comp rx=0 tx=0).stop 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 shutdown_connections 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fb110075a20 0x7fb110077ee0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1341f5900 0x7fb1341f5d90 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb13414b030 0x7fb13406efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 --2- 172.21.15.122:0/3889277188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb118003a10 0x7fb1340cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 >> 172.21.15.122:0/3889277188 conn(0x7fb1340d1da0 msgr2=0x7fb134155b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 shutdown_connections 2024-09-17T05:42:06.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:06.895+0000 7fb13a000640 1 -- 172.21.15.122:0/3889277188 wait complete. 2024-09-17T05:42:06.908 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T05:42:06.908 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:42:07.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.059+0000 7fc511c00640 1 -- 172.21.15.122:0/3623870612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c0b0b60 msgr2=0x7fc50c0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.059+0000 7fc511c00640 1 --2- 172.21.15.122:0/3623870612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c0b0b60 0x7fc50c0ccbd0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fc504009c80 tx=0x7fc50401f5a0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.059+0000 7fc511c00640 1 -- 172.21.15.122:0/3623870612 shutdown_connections 2024-09-17T05:42:07.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.059+0000 7fc511c00640 1 --2- 172.21.15.122:0/3623870612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c0b0b60 0x7fc50c0ccbd0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.059+0000 7fc511c00640 1 --2- 172.21.15.122:0/3623870612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc50c15c1f0 0x7fc50c0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.059+0000 7fc511c00640 1 -- 172.21.15.122:0/3623870612 >> 172.21.15.122:0/3623870612 conn(0x7fc50c0d1da0 msgr2=0x7fc50c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:07.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.059+0000 7fc511c00640 1 -- 172.21.15.122:0/3623870612 shutdown_connections 2024-09-17T05:42:07.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.059+0000 7fc511c00640 1 -- 172.21.15.122:0/3623870612 wait complete. 2024-09-17T05:42:07.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 Processor -- start 2024-09-17T05:42:07.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 -- start start 2024-09-17T05:42:07.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc50c0b0b60 0x7fc50c0c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c15be10 0x7fc50c156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc50c15c1f0 0x7fc50c1f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c0cd320 con 0x7fc50c15be10 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c0cf590 con 0x7fc50c0b0b60 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c0cd4a0 con 0x7fc50c15c1f0 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c15be10 0x7fc50c156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c15be10 0x7fc50c156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58776/0 (socket says 172.21.15.122:58776) 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 -- 172.21.15.122:0/1755179241 learned_addr learned my addr 172.21.15.122:0/1755179241 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc50aa00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc50c0b0b60 0x7fc50c0c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc50b400640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc50c15c1f0 0x7fc50c1f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 -- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc50c15c1f0 msgr2=0x7fc50c1f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc50c15c1f0 0x7fc50c1f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 -- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc50c0b0b60 msgr2=0x7fc50c0c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc50c0b0b60 0x7fc50c0c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 -- 172.21.15.122:0/1755179241 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c0c78d0 con 0x7fc50c15be10 2024-09-17T05:42:07.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc503e00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c15be10 0x7fc50c156c00 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400c9f0 tx=0x7fc4f400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4007bb0 con 0x7fc50c15be10 2024-09-17T05:42:07.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4f4007e40 con 0x7fc50c15be10 2024-09-17T05:42:07.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f400f690 con 0x7fc50c15be10 2024-09-17T05:42:07.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50c1f3130 con 0x7fc50c15be10 2024-09-17T05:42:07.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50c1f2df0 con 0x7fc50c15be10 2024-09-17T05:42:07.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.063+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc50c1f37f0 con 0x7fc50c15be10 2024-09-17T05:42:07.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fc4f4018400 con 0x7fc50c15be10 2024-09-17T05:42:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc502a00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4e80759f0 0x7fc4e8077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fc4f40943c0 con 0x7fc50c15be10 2024-09-17T05:42:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc502a00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc4e807b910 0x7fc4e807dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 --> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4f400de50 con 0x7fc4e807b910 2024-09-17T05:42:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7fc4f4094770 con 0x7fc50c15be10 2024-09-17T05:42:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc50aa00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4e80759f0 0x7fc4e8077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc50b400640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc4e807b910 0x7fc4e807dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc50aa00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4e80759f0 0x7fc4e8077eb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc004e30 tx=0x7fc4fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.067+0000 7fc50b400640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc4e807b910 0x7fc4e807dd90 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.071+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 <== osd.5 v2:172.21.15.173:6802/2545281431 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4f400de50 con 0x7fc4e807b910 2024-09-17T05:42:07.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 --> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc50c0ccbd0 con 0x7fc4e807b910 2024-09-17T05:42:07.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc502a00640 1 -- 172.21.15.122:0/1755179241 <== osd.5 v2:172.21.15.173:6802/2545281431 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc50c0ccbd0 con 0x7fc4e807b910 2024-09-17T05:42:07.077 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:42:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:42:07.078 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc4e807b910 msgr2=0x7fc4e807dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6802/2545281431,v1:172.21.15.173:6803/2545281431] conn(0x7fc4e807b910 0x7fc4e807dd90 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4e80759f0 msgr2=0x7fc4e8077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc4e80759f0 0x7fc4e8077eb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc004e30 tx=0x7fc4fc009040 comp rx=0 tx=0).stop 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c15be10 msgr2=0x7fc50c156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 --2- 172.21.15.122:0/1755179241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc50c15be10 0x7fc50c156c00 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400c9f0 tx=0x7fc4f400ceb0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc50b400640 1 -- 172.21.15.122:0/1755179241 reap_dead start 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 shutdown_connections 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 >> 172.21.15.122:0/1755179241 conn(0x7fc50c0d1da0 msgr2=0x7fc50c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 shutdown_connections 2024-09-17T05:42:07.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.083+0000 7fc511c00640 1 -- 172.21.15.122:0/1755179241 wait complete. 2024-09-17T05:42:07.095 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T05:42:07.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- 172.21.15.122:0/1892282464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0cebf0 msgr2=0x7f70ac0b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 --2- 172.21.15.122:0/1892282464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0cebf0 0x7f70ac0b2bf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f7094009900 tx=0x7f709401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- 172.21.15.122:0/1892282464 shutdown_connections 2024-09-17T05:42:07.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 --2- 172.21.15.122:0/1892282464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac0cebf0 0x7f70ac0b2bf0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- 172.21.15.122:0/1892282464 >> 172.21.15.122:0/1892282464 conn(0x7f70ac0b18f0 msgr2=0x7f70ac0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:07.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- 172.21.15.122:0/1892282464 shutdown_connections 2024-09-17T05:42:07.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- 172.21.15.122:0/1892282464 wait complete. 2024-09-17T05:42:07.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 Processor -- start 2024-09-17T05:42:07.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- start start 2024-09-17T05:42:07.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3130 0x7f70ac142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac15e260 0x7f70ac1fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac14e450 0x7f70ac1f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac168ec0 con 0x7f70ac14e450 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac168d40 con 0x7f70ac15e260 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70b0c00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f70ac169040 con 0x7f70ac0b3130 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70aa800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac14e450 0x7f70ac1f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70aa800640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac14e450 0x7f70ac1f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58800/0 (socket says 172.21.15.122:58800) 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70aa800640 1 -- 172.21.15.122:0/2922938779 learned_addr learned my addr 172.21.15.122:0/2922938779 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70a1400640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac15e260 0x7f70ac1fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70aa800640 1 -- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3130 msgr2=0x7f70ac142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70aa800640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3130 0x7f70ac142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70aa800640 1 -- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac15e260 msgr2=0x7f70ac1fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70aa800640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac15e260 0x7f70ac1fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70aa800640 1 -- 172.21.15.122:0/2922938779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac14ecf0 con 0x7f70ac14e450 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.247+0000 7f70a1400640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac15e260 0x7f70ac1fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:42:07.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70aa800640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac14e450 0x7f70ac1f8c50 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c7b0 tx=0x7f70a400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70a2a00640 1 -- 172.21.15.122:0/2922938779 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4004b70 con 0x7f70ac14e450 2024-09-17T05:42:07.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70a2a00640 1 -- 172.21.15.122:0/2922938779 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70a4004e00 con 0x7f70ac14e450 2024-09-17T05:42:07.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70a2a00640 1 -- 172.21.15.122:0/2922938779 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a400aab0 con 0x7f70ac14e450 2024-09-17T05:42:07.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac074880 con 0x7f70ac14e450 2024-09-17T05:42:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac0745a0 con 0x7f70ac14e450 2024-09-17T05:42:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ac069410 con 0x7f70ac14e450 2024-09-17T05:42:07.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70a2a00640 1 -- 172.21.15.122:0/2922938779 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f70a400ac50 con 0x7f70ac14e450 2024-09-17T05:42:07.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70a2a00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880759f0 0x7f7088077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70a2a00640 1 -- 172.21.15.122:0/2922938779 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f70a4093930 con 0x7f70ac14e450 2024-09-17T05:42:07.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.251+0000 7f70a9e00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880759f0 0x7f7088077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.255+0000 7f70a9e00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880759f0 0x7f7088077eb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f709c000f70 tx=0x7f709c0079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.255+0000 7f70a2a00640 1 -- 172.21.15.122:0/2922938779 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f70a4020070 con 0x7f70ac14e450 2024-09-17T05:42:07.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.431+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f70ac0b2bf0 con 0x7f70880759f0 2024-09-17T05:42:07.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.431+0000 7f70a2a00640 1 -- 172.21.15.122:0/2922938779 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f70ac0b2bf0 con 0x7f70880759f0 2024-09-17T05:42:07.426 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880759f0 msgr2=0x7f7088077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880759f0 0x7f7088077eb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f709c000f70 tx=0x7f709c0079c0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac14e450 msgr2=0x7f70ac1f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac14e450 0x7f70ac1f8c50 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c7b0 tx=0x7f70a400cc70 comp rx=0 tx=0).stop 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 shutdown_connections 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f70880759f0 0x7f7088077eb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac14e450 0x7f70ac1f8c50 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70ac15e260 0x7f70ac1fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 --2- 172.21.15.122:0/2922938779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70ac0b3130 0x7f70ac142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 >> 172.21.15.122:0/2922938779 conn(0x7f70ac0b18f0 msgr2=0x7f70ac0adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 shutdown_connections 2024-09-17T05:42:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.435+0000 7f70b0c00640 1 -- 172.21.15.122:0/2922938779 wait complete. 2024-09-17T05:42:07.449 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T05:42:07.449 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:42:07.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.599+0000 7f2f0d000640 1 -- 172.21.15.122:0/1122175094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080d09f0 msgr2=0x7f2f080d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.599+0000 7f2f0d000640 1 --2- 172.21.15.122:0/1122175094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080d09f0 0x7f2f080d28c0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2f0806dc00 tx=0x7f2ef001c890 comp rx=0 tx=0).stop 2024-09-17T05:42:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.599+0000 7f2f0d000640 1 -- 172.21.15.122:0/1122175094 shutdown_connections 2024-09-17T05:42:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.599+0000 7f2f0d000640 1 --2- 172.21.15.122:0/1122175094 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f0815eff0 0x7f2f0815f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.599+0000 7f2f0d000640 1 --2- 172.21.15.122:0/1122175094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080d09f0 0x7f2f080d28c0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.599+0000 7f2f0d000640 1 -- 172.21.15.122:0/1122175094 >> 172.21.15.122:0/1122175094 conn(0x7f2f080d1da0 msgr2=0x7f2f080d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.599+0000 7f2f0d000640 1 -- 172.21.15.122:0/1122175094 shutdown_connections 2024-09-17T05:42:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.599+0000 7f2f0d000640 1 -- 172.21.15.122:0/1122175094 wait complete. 2024-09-17T05:42:07.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 Processor -- start 2024-09-17T05:42:07.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 -- start start 2024-09-17T05:42:07.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080cff00 0x7f2f080b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f0815eff0 0x7f2f080cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f08151610 0x7f2f080cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f080b0c20 con 0x7f2f080cff00 2024-09-17T05:42:07.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2f080b0aa0 con 0x7f2f08151610 2024-09-17T05:42:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2f08168bf0 con 0x7f2f0815eff0 2024-09-17T05:42:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080cff00 0x7f2f080b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080cff00 0x7f2f080b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58828/0 (socket says 172.21.15.122:58828) 2024-09-17T05:42:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 -- 172.21.15.122:0/4091081935 learned_addr learned my addr 172.21.15.122:0/4091081935 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07e00640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f08151610 0x7f2f080cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2efea00640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f0815eff0 0x7f2f080cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 -- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f0815eff0 msgr2=0x7f2f080cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f0815eff0 0x7f2f080cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 -- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f08151610 msgr2=0x7f2f080cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f08151610 0x7f2f080cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 -- 172.21.15.122:0/4091081935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f081532c0 con 0x7f2f080cff00 2024-09-17T05:42:07.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f07400640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080cff00 0x7f2f080b73d0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800c480 tx=0x7f2ef800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8012830 con 0x7f2f080cff00 2024-09-17T05:42:07.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ef8017070 con 0x7f2f080cff00 2024-09-17T05:42:07.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f08152a20 con 0x7f2f080cff00 2024-09-17T05:42:07.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8012d80 con 0x7f2f080cff00 2024-09-17T05:42:07.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f08152fe0 con 0x7f2f080cff00 2024-09-17T05:42:07.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.603+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f080b7910 con 0x7f2f080cff00 2024-09-17T05:42:07.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f2ef801c460 con 0x7f2f080cff00 2024-09-17T05:42:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2effe00640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2ee40759f0 0x7f2ee4077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f2ef8093d20 con 0x7f2f080cff00 2024-09-17T05:42:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2effe00640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7f2ee407b910 0x7f2ee407dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 --> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ef8023e50 con 0x7f2ee407b910 2024-09-17T05:42:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f2ef80940d0 con 0x7f2f080cff00 2024-09-17T05:42:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2efea00640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2ee40759f0 0x7f2ee4077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2f07e00640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7f2ee407b910 0x7f2ee407dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2efea00640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2ee40759f0 0x7f2ee4077eb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00af000 tx=0x7f2ef001b0d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.607+0000 7f2f07e00640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7f2ee407b910 0x7f2ee407dd90 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.611+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 <== osd.6 v2:172.21.15.173:6826/2596490949 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ef8023e50 con 0x7f2ee407b910 2024-09-17T05:42:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 --> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2f080d28c0 con 0x7f2ee407b910 2024-09-17T05:42:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2effe00640 1 -- 172.21.15.122:0/4091081935 <== osd.6 v2:172.21.15.173:6826/2596490949 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2f080d28c0 con 0x7f2ee407b910 2024-09-17T05:42:07.617 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:42:07.617 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:42:07.617 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:42:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7f2ee407b910 msgr2=0x7f2ee407dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6826/2596490949,v1:172.21.15.173:6827/2596490949] conn(0x7f2ee407b910 0x7f2ee407dd90 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2ee40759f0 msgr2=0x7f2ee4077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f2ee40759f0 0x7f2ee4077eb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00af000 tx=0x7f2ef001b0d0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080cff00 msgr2=0x7f2f080b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 --2- 172.21.15.122:0/4091081935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080cff00 0x7f2f080b73d0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800c480 tx=0x7f2ef800c940 comp rx=0 tx=0).stop 2024-09-17T05:42:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f07e00640 1 -- 172.21.15.122:0/4091081935 reap_dead start 2024-09-17T05:42:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 shutdown_connections 2024-09-17T05:42:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 >> 172.21.15.122:0/4091081935 conn(0x7f2f080d1da0 msgr2=0x7f2f080711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 shutdown_connections 2024-09-17T05:42:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.623+0000 7f2f0d000640 1 -- 172.21.15.122:0/4091081935 wait complete. 2024-09-17T05:42:07.635 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T05:42:07.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.783+0000 7fc92d600640 1 -- 172.21.15.122:0/50333181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc928164c60 msgr2=0x7fc928167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.783+0000 7fc92d600640 1 --2- 172.21.15.122:0/50333181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc928164c60 0x7fc928167050 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc920009bc0 tx=0x7fc92001ca00 comp rx=0 tx=0).stop 2024-09-17T05:42:07.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.783+0000 7fc92d600640 1 -- 172.21.15.122:0/50333181 shutdown_connections 2024-09-17T05:42:07.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.783+0000 7fc92d600640 1 --2- 172.21.15.122:0/50333181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc928164c60 0x7fc928167050 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.783+0000 7fc92d600640 1 -- 172.21.15.122:0/50333181 >> 172.21.15.122:0/50333181 conn(0x7fc9280d1e30 msgr2=0x7fc9280d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:07.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.783+0000 7fc92d600640 1 -- 172.21.15.122:0/50333181 shutdown_connections 2024-09-17T05:42:07.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.783+0000 7fc92d600640 1 -- 172.21.15.122:0/50333181 wait complete. 2024-09-17T05:42:07.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 Processor -- start 2024-09-17T05:42:07.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 -- start start 2024-09-17T05:42:07.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9280b2e80 0x7fc92815b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc92815f490 0x7fc9280b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928164c60 0x7fc9280b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc928169f30 con 0x7fc928164c60 2024-09-17T05:42:07.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc928169db0 con 0x7fc92815f490 2024-09-17T05:42:07.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc92816a0b0 con 0x7fc9280b2e80 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928164c60 0x7fc9280b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928164c60 0x7fc9280b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58836/0 (socket says 172.21.15.122:58836) 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 -- 172.21.15.122:0/4011781673 learned_addr learned my addr 172.21.15.122:0/4011781673 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc926a00640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9280b2e80 0x7fc92815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc91e000640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc92815f490 0x7fc9280b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 -- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9280b2e80 msgr2=0x7fc92815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9280b2e80 0x7fc92815b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 -- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc92815f490 msgr2=0x7fc9280b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc92815f490 0x7fc9280b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 -- 172.21.15.122:0/4011781673 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92815b750 con 0x7fc928164c60 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc927400640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928164c60 0x7fc9280b1c10 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fc920019610 tx=0x7fc920009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc91f400640 1 -- 172.21.15.122:0/4011781673 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9200b2040 con 0x7fc928164c60 2024-09-17T05:42:07.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc91f400640 1 -- 172.21.15.122:0/4011781673 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc920003c30 con 0x7fc928164c60 2024-09-17T05:42:07.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc928149400 con 0x7fc928164c60 2024-09-17T05:42:07.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc91f400640 1 -- 172.21.15.122:0/4011781673 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9200042a0 con 0x7fc928164c60 2024-09-17T05:42:07.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9281499c0 con 0x7fc928164c60 2024-09-17T05:42:07.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.787+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9280b14c0 con 0x7fc928164c60 2024-09-17T05:42:07.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.791+0000 7fc91f400640 1 -- 172.21.15.122:0/4011781673 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fc9200bd030 con 0x7fc928164c60 2024-09-17T05:42:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.791+0000 7fc91f400640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9040759f0 0x7fc904077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:07.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.791+0000 7fc91f400640 1 -- 172.21.15.122:0/4011781673 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fc92012dc00 con 0x7fc928164c60 2024-09-17T05:42:07.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.791+0000 7fc926a00640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9040759f0 0x7fc904077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:07.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.791+0000 7fc926a00640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9040759f0 0x7fc904077eb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc9280ce300 tx=0x7fc918009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:07.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.795+0000 7fc91f400640 1 -- 172.21.15.122:0/4011781673 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fc9200c4020 con 0x7fc928164c60 2024-09-17T05:42:07.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.963+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fc928166110 con 0x7fc9040759f0 2024-09-17T05:42:07.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.963+0000 7fc91f400640 1 -- 172.21.15.122:0/4011781673 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc928166110 con 0x7fc9040759f0 2024-09-17T05:42:07.958 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9040759f0 msgr2=0x7fc904077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9040759f0 0x7fc904077eb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc9280ce300 tx=0x7fc918009040 comp rx=0 tx=0).stop 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928164c60 msgr2=0x7fc9280b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928164c60 0x7fc9280b1c10 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fc920019610 tx=0x7fc920009290 comp rx=0 tx=0).stop 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 shutdown_connections 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fc9040759f0 0x7fc904077eb0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc928164c60 0x7fc9280b1c10 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc92815f490 0x7fc9280b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 --2- 172.21.15.122:0/4011781673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9280b2e80 0x7fc92815b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:07.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 >> 172.21.15.122:0/4011781673 conn(0x7fc9280d1e30 msgr2=0x7fc9280717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:07.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 shutdown_connections 2024-09-17T05:42:07.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:07.967+0000 7fc92d600640 1 -- 172.21.15.122:0/4011781673 wait complete. 2024-09-17T05:42:07.981 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T05:42:07.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:42:08.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 -- 172.21.15.122:0/3166429231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53000cfff0 msgr2=0x7f53000d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 --2- 172.21.15.122:0/3166429231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53000cfff0 0x7f53000d0430 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0098f0 tx=0x7f52ec01c850 comp rx=0 tx=0).stop 2024-09-17T05:42:08.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 -- 172.21.15.122:0/3166429231 shutdown_connections 2024-09-17T05:42:08.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 --2- 172.21.15.122:0/3166429231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53000cfff0 0x7f53000d0430 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 --2- 172.21.15.122:0/3166429231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f530015f2a0 0x7f53000cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 -- 172.21.15.122:0/3166429231 >> 172.21.15.122:0/3166429231 conn(0x7f53000d1cd0 msgr2=0x7f53000d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:08.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 -- 172.21.15.122:0/3166429231 shutdown_connections 2024-09-17T05:42:08.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 -- 172.21.15.122:0/3166429231 wait complete. 2024-09-17T05:42:08.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 Processor -- start 2024-09-17T05:42:08.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.131+0000 7f5306400640 1 -- start start 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53000ce340 0x7f53000b7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53000cfff0 0x7f5300150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f530015f2a0 0x7f5300159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53001693b0 con 0x7f530015f2a0 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5300169230 con 0x7f53000cfff0 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5300169530 con 0x7f53000ce340 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f530015f2a0 0x7f5300159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f530015f2a0 0x7f5300159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58856/0 (socket says 172.21.15.122:58856) 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 -- 172.21.15.122:0/3836083656 learned_addr learned my addr 172.21.15.122:0/3836083656 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305000640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53000ce340 0x7f53000b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f52ffe00640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53000cfff0 0x7f5300150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 -- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53000ce340 msgr2=0x7f53000b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53000ce340 0x7f53000b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 -- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53000cfff0 msgr2=0x7f5300150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53000cfff0 0x7f5300150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 -- 172.21.15.122:0/3836083656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5300150db0 con 0x7f530015f2a0 2024-09-17T05:42:08.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5305a00640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f530015f2a0 0x7f5300159f10 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f52f800c7f0 tx=0x7f52f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:08.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8003b80 con 0x7f530015f2a0 2024-09-17T05:42:08.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52f8003d20 con 0x7f530015f2a0 2024-09-17T05:42:08.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f800aab0 con 0x7f530015f2a0 2024-09-17T05:42:08.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53000b70d0 con 0x7f530015f2a0 2024-09-17T05:42:08.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530015a450 con 0x7f530015f2a0 2024-09-17T05:42:08.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.135+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53000b5530 con 0x7f530015f2a0 2024-09-17T05:42:08.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f52f800ac50 con 0x7f530015f2a0 2024-09-17T05:42:08.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52fcc00640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52e00759f0 0x7f52e0077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f52f8093d20 con 0x7f530015f2a0 2024-09-17T05:42:08.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52fcc00640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f52e007b910 0x7f52e007dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 --> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52e007e460 con 0x7f52e007b910 2024-09-17T05:42:08.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f52f80940d0 con 0x7f530015f2a0 2024-09-17T05:42:08.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f5305000640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52e00759f0 0x7f52e0077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52ffe00640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f52e007b910 0x7f52e007dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f5305000640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52e00759f0 0x7f52e0077eb0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f52f400bb50 tx=0x7f52f400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:08.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52ffe00640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f52e007b910 0x7f52e007dd90 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:08.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.139+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 <== osd.7 v2:172.21.15.173:6818/1087122755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52e007e460 con 0x7f52e007b910 2024-09-17T05:42:08.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 --> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f53000d0430 con 0x7f52e007b910 2024-09-17T05:42:08.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f52fcc00640 1 -- 172.21.15.122:0/3836083656 <== osd.7 v2:172.21.15.173:6818/1087122755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f53000d0430 con 0x7f52e007b910 2024-09-17T05:42:08.148 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-17T05:42:08.148 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:42:08.148 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-17T05:42:08.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f52e007b910 msgr2=0x7f52e007dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6818/1087122755,v1:172.21.15.173:6819/1087122755] conn(0x7f52e007b910 0x7f52e007dd90 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52e00759f0 msgr2=0x7f52e0077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f52e00759f0 0x7f52e0077eb0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f52f400bb50 tx=0x7f52f400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f530015f2a0 msgr2=0x7f5300159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 --2- 172.21.15.122:0/3836083656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f530015f2a0 0x7f5300159f10 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f52f800c7f0 tx=0x7f52f800ccb0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5305a00640 1 -- 172.21.15.122:0/3836083656 reap_dead start 2024-09-17T05:42:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 shutdown_connections 2024-09-17T05:42:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 >> 172.21.15.122:0/3836083656 conn(0x7f53000d1cd0 msgr2=0x7f530015e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 shutdown_connections 2024-09-17T05:42:08.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.155+0000 7f5306400640 1 -- 172.21.15.122:0/3836083656 wait complete. 2024-09-17T05:42:08.166 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- 172.21.15.122:0/3212224551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 msgr2=0x7fa7581603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 --2- 172.21.15.122:0/3212224551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 0x7fa7581603e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fa7580739c0 tx=0x7fa75001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- 172.21.15.122:0/3212224551 shutdown_connections 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 --2- 172.21.15.122:0/3212224551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 0x7fa7581603e0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 --2- 172.21.15.122:0/3212224551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7580d2940 0x7fa75815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 --2- 172.21.15.122:0/3212224551 >> v2:172.21.15.173:3300/0 conn(0x7fa75815bb90 0x7fa75815f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- 172.21.15.122:0/3212224551 >> 172.21.15.122:0/3212224551 conn(0x7fa7580d1d90 msgr2=0x7fa7580d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- 172.21.15.122:0/3212224551 shutdown_connections 2024-09-17T05:42:08.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- 172.21.15.122:0/3212224551 wait complete. 2024-09-17T05:42:08.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 Processor -- start 2024-09-17T05:42:08.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- start start 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7580d2940 0x7fa75814efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75815bb90 0x7fa75806e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 0x7fa758152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580b5770 con 0x7fa75815bf70 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580b55f0 con 0x7fa75815bb90 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa760600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa7580b58f0 con 0x7fa7580d2940 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa75e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 0x7fa758152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa75e600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 0x7fa758152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58882/0 (socket says 172.21.15.122:58882) 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.315+0000 7fa75e600640 1 -- 172.21.15.122:0/665591729 learned_addr learned my addr 172.21.15.122:0/665591729 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75d200640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75815bb90 0x7fa75806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75dc00640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7580d2940 0x7fa75814efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75e600640 1 -- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7580d2940 msgr2=0x7fa75814efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75e600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7580d2940 0x7fa75814efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75e600640 1 -- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75815bb90 msgr2=0x7fa75806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75e600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75815bb90 0x7fa75806e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75e600640 1 -- 172.21.15.122:0/665591729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75814f4e0 con 0x7fa75815bf70 2024-09-17T05:42:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75d200640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75815bb90 0x7fa75806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:42:08.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa75e600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 0x7fa758152530 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fa750009850 tx=0x7fa7500a5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa755600640 1 -- 172.21.15.122:0/665591729 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500bb070 con 0x7fa75815bf70 2024-09-17T05:42:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa755600640 1 -- 172.21.15.122:0/665591729 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa750004490 con 0x7fa75815bf70 2024-09-17T05:42:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75806fc80 con 0x7fa75815bf70 2024-09-17T05:42:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa755600640 1 -- 172.21.15.122:0/665591729 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7500b2040 con 0x7fa75815bf70 2024-09-17T05:42:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75806eec0 con 0x7fa75815bf70 2024-09-17T05:42:08.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7580b5e80 con 0x7fa75815bf70 2024-09-17T05:42:08.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.319+0000 7fa755600640 1 -- 172.21.15.122:0/665591729 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fa7500be070 con 0x7fa75815bf70 2024-09-17T05:42:08.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.323+0000 7fa755600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7380759f0 0x7fa738077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.323+0000 7fa755600640 1 -- 172.21.15.122:0/665591729 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fa75012cf30 con 0x7fa75815bf70 2024-09-17T05:42:08.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.323+0000 7fa75dc00640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7380759f0 0x7fa738077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.323+0000 7fa75dc00640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7380759f0 0x7fa738077eb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005c50 tx=0x7fa74c005be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:08.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.323+0000 7fa755600640 1 -- 172.21.15.122:0/665591729 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fa7500164b0 con 0x7fa75815bf70 2024-09-17T05:42:08.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.495+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa758070270 con 0x7fa7380759f0 2024-09-17T05:42:08.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa755600640 1 -- 172.21.15.122:0/665591729 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa758070270 con 0x7fa7380759f0 2024-09-17T05:42:08.492 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7380759f0 msgr2=0x7fa738077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7380759f0 0x7fa738077eb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005c50 tx=0x7fa74c005be0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 msgr2=0x7fa758152530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 0x7fa758152530 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fa750009850 tx=0x7fa7500a5530 comp rx=0 tx=0).stop 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 shutdown_connections 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fa7380759f0 0x7fa738077eb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa75815bf70 0x7fa758152530 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa75815bb90 0x7fa75806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 --2- 172.21.15.122:0/665591729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7580d2940 0x7fa75814efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 >> 172.21.15.122:0/665591729 conn(0x7fa7580d1d90 msgr2=0x7fa7580b6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 shutdown_connections 2024-09-17T05:42:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.499+0000 7fa760600640 1 -- 172.21.15.122:0/665591729 wait complete. 2024-09-17T05:42:08.513 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:42:08.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- 172.21.15.122:0/721157990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbd4004810 msgr2=0x7fdbd4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 --2- 172.21.15.122:0/721157990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbd4004810 0x7fdbd4004c90 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009eb0 tx=0x7fdbd801d560 comp rx=0 tx=0).stop 2024-09-17T05:42:08.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- 172.21.15.122:0/721157990 shutdown_connections 2024-09-17T05:42:08.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 --2- 172.21.15.122:0/721157990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbd4004810 0x7fdbd4004c90 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- 172.21.15.122:0/721157990 >> 172.21.15.122:0/721157990 conn(0x7fdbf00d1da0 msgr2=0x7fdbf00d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:08.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- 172.21.15.122:0/721157990 shutdown_connections 2024-09-17T05:42:08.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- 172.21.15.122:0/721157990 wait complete. 2024-09-17T05:42:08.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 Processor -- start 2024-09-17T05:42:08.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- start start 2024-09-17T05:42:08.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd4004810 0x7fdbf015d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbf01510f0 0x7fdbf0158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf0158560 0x7fdbf015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf00ce4e0 con 0x7fdbf0158560 2024-09-17T05:42:08.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf00d2b50 con 0x7fdbd4004810 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbf6200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf00ce660 con 0x7fdbf01510f0 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbefe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf0158560 0x7fdbf015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbefe00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf0158560 0x7fdbf015e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58908/0 (socket says 172.21.15.122:58908) 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbefe00640 1 -- 172.21.15.122:0/2358442343 learned_addr learned my addr 172.21.15.122:0/2358442343 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbef400640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd4004810 0x7fdbf015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbe6a00640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbf01510f0 0x7fdbf0158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbefe00640 1 -- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbf01510f0 msgr2=0x7fdbf0158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbefe00640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbf01510f0 0x7fdbf0158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbefe00640 1 -- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd4004810 msgr2=0x7fdbf015d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbefe00640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd4004810 0x7fdbf015d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbefe00640 1 -- 172.21.15.122:0/2358442343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf015c4e0 con 0x7fdbf0158560 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.663+0000 7fdbef400640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd4004810 0x7fdbf015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:42:08.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbefe00640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf0158560 0x7fdbf015e250 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0073b00 tx=0x7fdbe8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:08.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbe7e00640 1 -- 172.21.15.122:0/2358442343 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80107d0 con 0x7fdbf0158560 2024-09-17T05:42:08.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbe7e00640 1 -- 172.21.15.122:0/2358442343 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbe8016070 con 0x7fdbf0158560 2024-09-17T05:42:08.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbe7e00640 1 -- 172.21.15.122:0/2358442343 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8010ce0 con 0x7fdbf0158560 2024-09-17T05:42:08.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf015c7c0 con 0x7fdbf0158560 2024-09-17T05:42:08.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf015cd80 con 0x7fdbf0158560 2024-09-17T05:42:08.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf00b53f0 con 0x7fdbf0158560 2024-09-17T05:42:08.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbe7e00640 1 -- 172.21.15.122:0/2358442343 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7fdbe801b460 con 0x7fdbf0158560 2024-09-17T05:42:08.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbe7e00640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbcc075a20 0x7fdbcc077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:08.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.667+0000 7fdbe7e00640 1 -- 172.21.15.122:0/2358442343 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7fdbe8093c10 con 0x7fdbf0158560 2024-09-17T05:42:08.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.671+0000 7fdbef400640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbcc075a20 0x7fdbcc077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:08.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.671+0000 7fdbef400640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbcc075a20 0x7fdbcc077ee0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0001340 tx=0x7fdbe00068b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:08.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.671+0000 7fdbe7e00640 1 -- 172.21.15.122:0/2358442343 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7fdbe80607a0 con 0x7fdbf0158560 2024-09-17T05:42:08.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdbf00b7a30 con 0x7fdbcc075a20 2024-09-17T05:42:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbe7e00640 1 -- 172.21.15.122:0/2358442343 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13420 (secure 0 0 0) 0x7fdbf00b7a30 con 0x7fdbcc075a20 2024-09-17T05:42:08.833 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:42:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:42:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbcc075a20 msgr2=0x7fdbcc077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbcc075a20 0x7fdbcc077ee0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0001340 tx=0x7fdbe00068b0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf0158560 msgr2=0x7fdbf015e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf0158560 0x7fdbf015e250 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0073b00 tx=0x7fdbe8001640 comp rx=0 tx=0).stop 2024-09-17T05:42:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 shutdown_connections 2024-09-17T05:42:08.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7fdbcc075a20 0x7fdbcc077ee0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf0158560 0x7fdbf015e250 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdbf01510f0 0x7fdbf0158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 --2- 172.21.15.122:0/2358442343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd4004810 0x7fdbf015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 >> 172.21.15.122:0/2358442343 conn(0x7fdbf00d1da0 msgr2=0x7fdbf0071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 shutdown_connections 2024-09-17T05:42:08.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:08.839+0000 7fdbf6200640 1 -- 172.21.15.122:0/2358442343 wait complete. 2024-09-17T05:42:08.855 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":662,"stamp":"2024-09-17T05:42:08.796840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1139536,"kb_used_data":15912,"kb_used_omap":12,"kb_used_meta":1123443,"kb_avail":753835184,"statfs":{"total":773094113280,"available":771927228416,"internally_reserved":0,"allocated":16293888,"data_stored":7870520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1150406198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.417388"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":358,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T05:41:58.681897+0000","last_change":"2024-09-17T05:24:55.940559+0000","last_active":"2024-09-17T05:41:58.681897+0000","last_peered":"2024-09-17T05:41:58.681897+0000","last_clean":"2024-09-17T05:41:58.681897+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:41:58.681897+0000","last_undegraded":"2024-09-17T05:41:58.681897+0000","last_fullsized":"2024-09-17T05:41:58.681897+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:24:54.732667+0000","last_clean_scrub_stamp":"2024-09-17T05:24:54.732667+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:54.455896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509649,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123212,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248628,"statfs":{"total":96636764160,"available":96510595072,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509649,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163464,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208376,"statfs":{"total":96636764160,"available":96469377024,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509649,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145420,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226420,"statfs":{"total":96636764160,"available":96487854080,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141256,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230584,"statfs":{"total":96636764160,"available":96492118016,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509649,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141256,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230584,"statfs":{"total":96636764160,"available":96492118016,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509650,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137680,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234160,"statfs":{"total":96636764160,"available":96495779840,"internally_reserved":0,"allocated":2482176,"data_stored":1426591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141256,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230584,"statfs":{"total":96636764160,"available":96492118016,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":142735819},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542355,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145992,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225848,"statfs":{"total":96636764160,"available":96487268352,"internally_reserved":0,"allocated":2482176,"data_stored":1426591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:42:08.858 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T05:24:54.732667+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=5, tm_min=24, tm_sec=54, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=5, tm_min=42, tm_sec=3, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T05:42:08.858 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T05:42:28.859 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:42:29.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 -- 172.21.15.122:0/3310156947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e4073690 msgr2=0x7f78e4073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 --2- 172.21.15.122:0/3310156947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e4073690 0x7f78e4073b20 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f78e40cf100 tx=0x7f78d001c850 comp rx=0 tx=0).stop 2024-09-17T05:42:29.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 -- 172.21.15.122:0/3310156947 shutdown_connections 2024-09-17T05:42:29.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 --2- 172.21.15.122:0/3310156947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e4073690 0x7f78e4073b20 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 -- 172.21.15.122:0/3310156947 >> 172.21.15.122:0/3310156947 conn(0x7f78e406dd30 msgr2=0x7f78e406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:29.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 -- 172.21.15.122:0/3310156947 shutdown_connections 2024-09-17T05:42:29.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 -- 172.21.15.122:0/3310156947 wait complete. 2024-09-17T05:42:29.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 Processor -- start 2024-09-17T05:42:29.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.014+0000 7f78e9600640 1 -- start start 2024-09-17T05:42:29.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78e4073690 0x7f78e414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78e40c6250 0x7f78e40b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e40cf6f0 0x7f78e415b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4169920 con 0x7f78e40cf6f0 2024-09-17T05:42:29.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41697a0 con 0x7f78e40c6250 2024-09-17T05:42:29.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f78e4169aa0 con 0x7f78e4073690 2024-09-17T05:42:29.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e40cf6f0 0x7f78e415b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e40cf6f0 0x7f78e415b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42774/0 (socket says 172.21.15.122:42774) 2024-09-17T05:42:29.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 -- 172.21.15.122:0/2978127450 learned_addr learned my addr 172.21.15.122:0/2978127450 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 -- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78e4073690 msgr2=0x7f78e414efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78e4073690 0x7f78e414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 -- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78e40c6250 msgr2=0x7f78e40b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e3400640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78e40c6250 0x7f78e40b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78e40c6250 0x7f78e40b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 -- 172.21.15.122:0/2978127450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e414f4e0 con 0x7f78e40cf6f0 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e3400640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78e40c6250 0x7f78e40b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e8c00640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e40cf6f0 0x7f78e415b550 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00b970 tx=0x7f78dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78dbe00640 1 -- 172.21.15.122:0/2978127450 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc019070 con 0x7f78e40cf6f0 2024-09-17T05:42:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78dbe00640 1 -- 172.21.15.122:0/2978127450 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78dc00cc30 con 0x7f78e40cf6f0 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e40b3a90 con 0x7f78e40cf6f0 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78dbe00640 1 -- 172.21.15.122:0/2978127450 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc0146a0 con 0x7f78e40cf6f0 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e40b4050 con 0x7f78e40cf6f0 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.018+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e4073b20 con 0x7f78e40cf6f0 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.022+0000 7f78dbe00640 1 -- 172.21.15.122:0/2978127450 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f78dc01c070 con 0x7f78e40cf6f0 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.022+0000 7f78dbe00640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78c4075a20 0x7f78c4077ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.022+0000 7f78dbe00640 1 -- 172.21.15.122:0/2978127450 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f78dc093140 con 0x7f78e40cf6f0 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.022+0000 7f78e3e00640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78c4075a20 0x7f78c4077ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.022+0000 7f78e3e00640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78c4075a20 0x7f78c4077ee0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f78d4000f70 tx=0x7f78d40079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:29.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.026+0000 7f78dbe00640 1 -- 172.21.15.122:0/2978127450 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f78dc05fc50 con 0x7f78e40cf6f0 2024-09-17T05:42:29.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 --> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78e406ebd0 con 0x7f78c4075a20 2024-09-17T05:42:29.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78dbe00640 1 -- 172.21.15.122:0/2978127450 <== mgr.4108 v2:172.21.15.173:6800/3859572675 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7f78e406ebd0 con 0x7f78c4075a20 2024-09-17T05:42:29.189 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:42:29.189 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78c4075a20 msgr2=0x7f78c4077ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78c4075a20 0x7f78c4077ee0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f78d4000f70 tx=0x7f78d40079c0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e40cf6f0 msgr2=0x7f78e415b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e40cf6f0 0x7f78e415b550 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00b970 tx=0x7f78dc00be30 comp rx=0 tx=0).stop 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 shutdown_connections 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f78c4075a20 0x7f78c4077ee0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78e40cf6f0 0x7f78e415b550 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78e40c6250 0x7f78e40b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 --2- 172.21.15.122:0/2978127450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78e4073690 0x7f78e414efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 >> 172.21.15.122:0/2978127450 conn(0x7f78e406dd30 msgr2=0x7f78e4154100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 shutdown_connections 2024-09-17T05:42:29.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.194+0000 7f78e9600640 1 -- 172.21.15.122:0/2978127450 wait complete. 2024-09-17T05:42:29.210 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":672,"stamp":"2024-09-17T05:42:28.801177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1139528,"kb_used_data":15912,"kb_used_omap":12,"kb_used_meta":1123443,"kb_avail":753835192,"statfs":{"total":773094113280,"available":771927236608,"internally_reserved":0,"allocated":16293888,"data_stored":7870520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1150406198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002519"},"pg_stats":[{"pgid":"1.0","version":"151'74","reported_seq":368,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T05:42:05.986190+0000","last_change":"2024-09-17T05:42:05.986190+0000","last_active":"2024-09-17T05:42:05.986190+0000","last_peered":"2024-09-17T05:42:05.986190+0000","last_clean":"2024-09-17T05:42:05.986190+0000","last_became_active":"2024-09-17T05:24:55.939522+0000","last_became_peered":"2024-09-17T05:24:55.939522+0000","last_unstale":"2024-09-17T05:42:05.986190+0000","last_undegraded":"2024-09-17T05:42:05.986190+0000","last_fullsized":"2024-09-17T05:42:05.986190+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'74","last_scrub_stamp":"2024-09-17T05:42:05.986118+0000","last_deep_scrub":"151'74","last_deep_scrub_stamp":"2024-09-17T05:42:05.986118+0000","last_clean_scrub_stamp":"2024-09-17T05:42:05.986118+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:33.469826+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":123212,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248628,"statfs":{"total":96636764160,"available":96510595072,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163464,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208376,"statfs":{"total":96636764160,"available":96469377024,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165476806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145420,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226420,"statfs":{"total":96636764160,"available":96487854080,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141256,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230584,"statfs":{"total":96636764160,"available":96492118016,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141256,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230584,"statfs":{"total":96636764160,"available":96492118016,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509654,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137672,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234168,"statfs":{"total":96636764160,"available":96495788032,"internally_reserved":0,"allocated":2482176,"data_stored":1426591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":141256,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230584,"statfs":{"total":96636764160,"available":96492118016,"internally_reserved":0,"allocated":1888256,"data_stored":836223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":142735819},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542359,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145992,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94225848,"statfs":{"total":96636764160,"available":96487268352,"internally_reserved":0,"allocated":2482176,"data_stored":1426591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:42:29.211 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T05:42:29.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- 172.21.15.122:0/855148746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b1015bd40 msgr2=0x7f3b100d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 --2- 172.21.15.122:0/855148746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b1015bd40 0x7f3b100d0240 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009900 tx=0x7f3b0001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- 172.21.15.122:0/855148746 shutdown_connections 2024-09-17T05:42:29.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 --2- 172.21.15.122:0/855148746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b100b0d10 0x7f3b100d1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 --2- 172.21.15.122:0/855148746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b1015bd40 0x7f3b100d0240 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- 172.21.15.122:0/855148746 >> 172.21.15.122:0/855148746 conn(0x7f3b1009aa10 msgr2=0x7f3b1009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:29.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- 172.21.15.122:0/855148746 shutdown_connections 2024-09-17T05:42:29.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- 172.21.15.122:0/855148746 wait complete. 2024-09-17T05:42:29.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 Processor -- start 2024-09-17T05:42:29.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- start start 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b100b0d10 0x7f3b1014e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b100d20f0 0x7f3b1015e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1015bd40 0x7f3b100b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3b10168d00 con 0x7f3b100d20f0 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3b10168b80 con 0x7f3b1015bd40 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0f200640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3b10168e80 con 0x7f3b100b0d10 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b100d20f0 0x7f3b1015e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0d400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b100d20f0 0x7f3b1015e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42782/0 (socket says 172.21.15.122:42782) 2024-09-17T05:42:29.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0d400640 1 -- 172.21.15.122:0/2603559146 learned_addr learned my addr 172.21.15.122:0/2603559146 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:29.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0de00640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b100b0d10 0x7f3b1014e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.358+0000 7f3b0e800640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1015bd40 0x7f3b100b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0d400640 1 -- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b100b0d10 msgr2=0x7f3b1014e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0d400640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b100b0d10 0x7f3b1014e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0d400640 1 -- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1015bd40 msgr2=0x7f3b100b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0d400640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1015bd40 0x7f3b100b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0d400640 1 -- 172.21.15.122:0/2603559146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b100b5bf0 con 0x7f3b100d20f0 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0d400640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b100d20f0 0x7f3b1015e560 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00b580 tx=0x7f3afc00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b06000640 1 -- 172.21.15.122:0/2603559146 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc014070 con 0x7f3b100d20f0 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b06000640 1 -- 172.21.15.122:0/2603559146 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3afc010980 con 0x7f3b100d20f0 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b06000640 1 -- 172.21.15.122:0/2603559146 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc010dd0 con 0x7f3b100d20f0 2024-09-17T05:42:29.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b100b5ed0 con 0x7f3b100d20f0 2024-09-17T05:42:29.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b100b6490 con 0x7f3b100d20f0 2024-09-17T05:42:29.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b100d1bb0 con 0x7f3b100d20f0 2024-09-17T05:42:29.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b06000640 1 -- 172.21.15.122:0/2603559146 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f3afc017070 con 0x7f3b100d20f0 2024-09-17T05:42:29.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.362+0000 7f3b06000640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3af00759f0 0x7f3af0077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.366+0000 7f3b06000640 1 -- 172.21.15.122:0/2603559146 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f3afc0933c0 con 0x7f3b100d20f0 2024-09-17T05:42:29.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.366+0000 7f3b0de00640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3af00759f0 0x7f3af0077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.366+0000 7f3b0de00640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3af00759f0 0x7f3af0077eb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3b000ae000 tx=0x7f3b0001b120 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:29.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.366+0000 7f3b06000640 1 -- 172.21.15.122:0/2603559146 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f3afc05f800 con 0x7f3b100d20f0 2024-09-17T05:42:29.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.538+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f3b100cf9d0 con 0x7f3b100d20f0 2024-09-17T05:42:29.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.542+0000 7f3b06000640 1 -- 172.21.15.122:0/2603559146 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v58)=0 v58) ==== 123+0+0 (secure 0 0 0) 0x7f3afc064230 con 0x7f3b100d20f0 2024-09-17T05:42:29.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.542+0000 7f3b06000640 1 -- 172.21.15.122:0/2603559146 <== mon.0 v2:172.21.15.122:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f3afc05f310 con 0x7f3b100d20f0 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3af00759f0 msgr2=0x7f3af0077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3af00759f0 0x7f3af0077eb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3b000ae000 tx=0x7f3b0001b120 comp rx=0 tx=0).stop 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b100d20f0 msgr2=0x7f3b1015e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b100d20f0 0x7f3b1015e560 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00b580 tx=0x7f3afc00ba40 comp rx=0 tx=0).stop 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 shutdown_connections 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f3af00759f0 0x7f3af0077eb0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1015bd40 0x7f3b100b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b100d20f0 0x7f3b1015e560 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 --2- 172.21.15.122:0/2603559146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b100b0d10 0x7f3b1014e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 >> 172.21.15.122:0/2603559146 conn(0x7f3b1009aa10 msgr2=0x7f3b1015ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:29.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 shutdown_connections 2024-09-17T05:42:29.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.546+0000 7f3b0f200640 1 -- 172.21.15.122:0/2603559146 wait complete. 2024-09-17T05:42:29.560 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:42:29.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:42:29.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.706+0000 7f5a21a00640 1 -- 172.21.15.122:0/518729335 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a1c0d29e0 msgr2=0x7f5a1c0b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.706+0000 7f5a21a00640 1 --2- 172.21.15.122:0/518729335 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a1c0d29e0 0x7f5a1c0b6f80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009c80 tx=0x7f5a1401cd70 comp rx=0 tx=0).stop 2024-09-17T05:42:29.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.706+0000 7f5a21a00640 1 -- 172.21.15.122:0/518729335 shutdown_connections 2024-09-17T05:42:29.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.706+0000 7f5a21a00640 1 --2- 172.21.15.122:0/518729335 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a1c0d29e0 0x7f5a1c0b6f80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.706+0000 7f5a21a00640 1 --2- 172.21.15.122:0/518729335 >> v2:172.21.15.173:3300/0 conn(0x7f5a1c1617b0 0x7f5a1c0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.706+0000 7f5a21a00640 1 -- 172.21.15.122:0/518729335 >> 172.21.15.122:0/518729335 conn(0x7f5a1c0d3e00 msgr2=0x7f5a1c0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:29.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.706+0000 7f5a21a00640 1 -- 172.21.15.122:0/518729335 shutdown_connections 2024-09-17T05:42:29.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.706+0000 7f5a21a00640 1 -- 172.21.15.122:0/518729335 wait complete. 2024-09-17T05:42:29.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 Processor -- start 2024-09-17T05:42:29.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 -- start start 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a1c0d29e0 0x7f5a1c15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1c1613d0 0x7f5a1c159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a1c1617b0 0x7f5a1c0c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c16acc0 con 0x7f5a1c1613d0 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c16ae40 con 0x7f5a1c0d29e0 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1c16afc0 con 0x7f5a1c1617b0 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1c1613d0 0x7f5a1c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1c1613d0 0x7f5a1c159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42788/0 (socket says 172.21.15.122:42788) 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 -- 172.21.15.122:0/4106336734 learned_addr learned my addr 172.21.15.122:0/4106336734 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21000640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a1c1617b0 0x7f5a1c0c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1be00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a1c0d29e0 0x7f5a1c15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 -- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a1c1617b0 msgr2=0x7f5a1c0c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a1c1617b0 0x7f5a1c0c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 -- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a1c0d29e0 msgr2=0x7f5a1c15cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a1c0d29e0 0x7f5a1c15cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 -- 172.21.15.122:0/4106336734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c15d560 con 0x7f5a1c1613d0 2024-09-17T05:42:29.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1be00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a1c0d29e0 0x7f5a1c15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:42:29.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a1b400640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1c1613d0 0x7f5a1c159210 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800e7a0 tx=0x7f5a0800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a13e00640 1 -- 172.21.15.122:0/4106336734 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800ccc0 con 0x7f5a1c1613d0 2024-09-17T05:42:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a13e00640 1 -- 172.21.15.122:0/4106336734 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f5a08004020 con 0x7f5a1c1613d0 2024-09-17T05:42:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a13e00640 1 -- 172.21.15.122:0/4106336734 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08004470 con 0x7f5a1c1613d0 2024-09-17T05:42:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c15d840 con 0x7f5a1c1613d0 2024-09-17T05:42:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.710+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c0c7cc0 con 0x7f5a1c1613d0 2024-09-17T05:42:29.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.714+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c0b6f80 con 0x7f5a1c1613d0 2024-09-17T05:42:29.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.714+0000 7f5a13e00640 1 -- 172.21.15.122:0/4106336734 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 60) ==== 100855+0+0 (secure 0 0 0) 0x7f5a08003600 con 0x7f5a1c1613d0 2024-09-17T05:42:29.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.714+0000 7f5a13e00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc0759f0 0x7f59fc077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:42:29.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.714+0000 7f5a13e00640 1 -- 172.21.15.122:0/4106336734 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5055+0+0 (secure 0 0 0) 0x7f5a08093f10 con 0x7f5a1c1613d0 2024-09-17T05:42:29.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.714+0000 7f5a1be00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc0759f0 0x7f59fc077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:42:29.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.714+0000 7f5a1be00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc0759f0 0x7f59fc077eb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c001750 tx=0x7f5a0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:42:29.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.718+0000 7f5a13e00640 1 -- 172.21.15.122:0/4106336734 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188675 (secure 0 0 0) 0x7f5a08060a20 con 0x7f5a1c1613d0 2024-09-17T05:42:29.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.918+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a1c0b5240 con 0x7f5a1c1613d0 2024-09-17T05:42:29.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.918+0000 7f5a13e00640 1 -- 172.21.15.122:0/4106336734 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+3917 (secure 0 0 0) 0x7f5a08060320 con 0x7f5a1c1613d0 2024-09-17T05:42:29.915 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-17T05:42:29.915 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":99,"btime":"2024-09-17T05:41:56:157897+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/223748651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":223748651},{"type":"v1","addr":"172.21.15.173:6837","nonce":223748651}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":8440,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/822031950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":822031950},{"type":"v1","addr":"172.21.15.173:6836","nonce":822031950}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6834/2541492660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2541492660},{"type":"v1","addr":"172.21.15.122:6834","nonce":2541492660}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91},{"gid":9948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/4249833259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6833","nonce":4249833259},{"type":"v1","addr":"172.21.15.122:6835","nonce":4249833259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[]} 2024-09-17T05:42:29.915 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 99 2024-09-17T05:42:29.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc0759f0 msgr2=0x7f59fc077eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc0759f0 0x7f59fc077eb0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c001750 tx=0x7f5a0c009040 comp rx=0 tx=0).stop 2024-09-17T05:42:29.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1c1613d0 msgr2=0x7f5a1c159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:42:29.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1c1613d0 0x7f5a1c159210 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800e7a0 tx=0x7f5a0800ec60 comp rx=0 tx=0).stop 2024-09-17T05:42:29.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 shutdown_connections 2024-09-17T05:42:29.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:6800/3859572675,v1:172.21.15.173:6801/3859572675] conn(0x7f59fc0759f0 0x7f59fc077eb0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a1c1617b0 0x7f5a1c0c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a1c1613d0 0x7f5a1c159210 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 --2- 172.21.15.122:0/4106336734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a1c0d29e0 0x7f5a1c15cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:42:29.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 >> 172.21.15.122:0/4106336734 conn(0x7f5a1c0d3e00 msgr2=0x7f5a1c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:42:29.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 shutdown_connections 2024-09-17T05:42:29.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-17T05:42:29.922+0000 7f5a21a00640 1 -- 172.21.15.122:0/4106336734 wait complete. 2024-09-17T05:42:29.937 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-17T05:42:29.937 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T05:42:29.937 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:42:29.937 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:29.949 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:42:29.949 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:42:29.949 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:30.002 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:42:30.002 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:42:30.002 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:30.017 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:42:30.017 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:42:30.017 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:30.072 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:42:30.073 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T05:42:30.073 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T05:42:30.073 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:30.166 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T05:42:30.166 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T05:42:30.166 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:34.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-17T05:42:34.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:42:34.603 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:42:36.267 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T05:42:36.268 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T05:42:36.268 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:36.351 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T05:42:36.352 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T05:42:36.352 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:40.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-17T05:42:40.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-17T05:42:42.453 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T05:42:42.454 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T05:42:42.454 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:42.538 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T05:42:42.539 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T05:42:42.539 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:45.984 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-17T05:42:45.984 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-17T05:42:48.640 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T05:42:48.641 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T05:42:48.641 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:51.590 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-17T05:42:51.590 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-17T05:42:54.743 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T05:42:54.743 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T05:42:54.743 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:54.820 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T05:42:54.821 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T05:42:54.821 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T05:42:54.821 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:54.882 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T05:42:54.882 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T05:42:54.882 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:42:56.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-17T05:42:56.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-17T05:43:00.984 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T05:43:00.984 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T05:43:00.985 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T05:43:00.985 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:43:01.006 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T05:43:01.006 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T05:43:01.007 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:43:01.025 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T05:43:01.026 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T05:43:01.026 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:43:01.089 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T05:43:01.089 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T05:43:01.090 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-17T05:43:01.135 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:43:01.135 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T05:43:01.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:43:01.383 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T05:43:01.578 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:43:01.578 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T05:43:01.795 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:43:01.795 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T05:43:01.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-17T05:43:01.988 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:43:01.988 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T05:43:02.199 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:43:02.200 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T05:43:02.412 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:43:02.412 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T05:43:02.619 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:43:02.619 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T05:43:02.808 INFO:tasks.ceph:Archiving mon data... 2024-09-17T05:43:02.810 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/data/mon.a.tgz 2024-09-17T05:43:02.810 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-17T05:43:02.819 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.9vVHuNy1Wm 2024-09-17T05:43:02.819 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.9vVHuNy1Wm 2024-09-17T05:43:03.163 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.9vVHuNy1Wm 2024-09-17T05:43:03.235 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.9vVHuNy1Wm is 2MB 2024-09-17T05:43:03.314 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.9vVHuNy1Wm 2024-09-17T05:43:03.323 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/data/mon.b.tgz 2024-09-17T05:43:03.323 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-09-17T05:43:03.332 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.a7diLE0x33 2024-09-17T05:43:03.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.a7diLE0x33 2024-09-17T05:43:03.684 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.a7diLE0x33 2024-09-17T05:43:03.759 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.a7diLE0x33 is 2MB 2024-09-17T05:43:03.859 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.a7diLE0x33 2024-09-17T05:43:03.868 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/data/mon.c.tgz 2024-09-17T05:43:03.869 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-09-17T05:43:03.917 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.HrXkKdCunR 2024-09-17T05:43:03.918 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HrXkKdCunR 2024-09-17T05:43:04.265 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.HrXkKdCunR 2024-09-17T05:43:04.342 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.HrXkKdCunR is 2MB 2024-09-17T05:43:04.422 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.HrXkKdCunR 2024-09-17T05:43:04.430 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T05:43:04.431 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T05:43:04.435 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T05:43:04.505 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T05:43:04.546 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T05:43:04.546 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T05:43:04.548 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/remote/smithi122/crash 2024-09-17T05:43:04.548 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T05:43:04.569 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/crash to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/remote/smithi173/crash 2024-09-17T05:43:04.569 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T05:43:04.613 INFO:tasks.ceph:Shutting down logrotate 2024-09-17T05:43:04.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:43:04.617 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:43:04.702 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T05:43:04.722 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T05:43:04.760 INFO:tasks.ceph:Compressing logs... 2024-09-17T05:43:04.760 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T05:43:04.765 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T05:43:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42813.log 2024-09-17T05:43:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47905.log 2024-09-17T05:43:04.798 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40637.log 2024-09-17T05:43:04.798 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42813.log.gz 2024-09-17T05:43:04.799 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40020.log 2024-09-17T05:43:04.799 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47905.log.gz 2024-09-17T05:43:04.800 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56684.log 2024-09-17T05:43:04.800 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40637.log.gz 2024-09-17T05:43:04.801 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40020.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40020.log.gz 2024-09-17T05:43:04.801 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42921.log 2024-09-17T05:43:04.801 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41645.log 2024-09-17T05:43:04.802 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56684.log.gz 2024-09-17T05:43:04.802 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49351.log 2024-09-17T05:43:04.802 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42921.log.gz 2024-09-17T05:43:04.803 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49960.log 2024-09-17T05:43:04.804 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41645.log.gz 2024-09-17T05:43:04.804 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41975.log 2024-09-17T05:43:04.804 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.49351.log.gz 2024-09-17T05:43:04.805 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43473.log 2024-09-17T05:43:04.805 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49960.log.gz 2024-09-17T05:43:04.807 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45643.log 2024-09-17T05:43:04.807 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41975.log.gz 2024-09-17T05:43:04.807 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51670.log 2024-09-17T05:43:04.807 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43473.log.gz 2024-09-17T05:43:04.807 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54400.log 2024-09-17T05:43:04.808 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45643.log.gz 2024-09-17T05:43:04.808 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53118.log 2024-09-17T05:43:04.808 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51670.log.gz 2024-09-17T05:43:04.809 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55739.log 2024-09-17T05:43:04.809 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54400.log.gz 2024-09-17T05:43:04.810 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55163.log 2024-09-17T05:43:04.810 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53118.log.gz 2024-09-17T05:43:04.811 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56528.log 2024-09-17T05:43:04.811 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55739.log.gz 2024-09-17T05:43:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37406.log 2024-09-17T05:43:04.812 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55163.log.gz 2024-09-17T05:43:04.812 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39313.log 2024-09-17T05:43:04.812 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56528.log.gz 2024-09-17T05:43:04.813 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47499.log 2024-09-17T05:43:04.813 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37406.log.gz 2024-09-17T05:43:04.814 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39664.log 2024-09-17T05:43:04.814 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39313.log.gz 2024-09-17T05:43:04.815 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50991.log 2024-09-17T05:43:04.815 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47499.log.gz 2024-09-17T05:43:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2024-09-17T05:43:04.816 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39664.log.gz 2024-09-17T05:43:04.816 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51696.log 2024-09-17T05:43:04.817 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50991.log.gz 2024-09-17T05:43:04.817 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38970.log 2024-09-17T05:43:04.817 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2024-09-17T05:43:04.818 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47840.log 2024-09-17T05:43:04.818 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51696.log.gz 2024-09-17T05:43:04.819 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49049.log 2024-09-17T05:43:04.820 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38970.log.gz 2024-09-17T05:43:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52515.log 2024-09-17T05:43:04.820 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47840.log.gz 2024-09-17T05:43:04.821 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53066.log 2024-09-17T05:43:04.821 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49049.log.gz 2024-09-17T05:43:04.821 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44525.log 2024-09-17T05:43:04.822 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52515.log.gz 2024-09-17T05:43:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45014.log 2024-09-17T05:43:04.822 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53066.log.gz 2024-09-17T05:43:04.823 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-09-17T05:43:04.823 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44525.log.gz 2024-09-17T05:43:04.824 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42869.log 2024-09-17T05:43:04.824 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45014.log.gz 2024-09-17T05:43:04.825 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35380.log 2024-09-17T05:43:04.825 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-09-17T05:43:04.825 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41084.log 2024-09-17T05:43:04.826 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42869.log.gz 2024-09-17T05:43:04.826 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50064.log 2024-09-17T05:43:04.826 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35380.log.gz 2024-09-17T05:43:04.827 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34247.log 2024-09-17T05:43:04.828 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41084.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T05:43:04.828 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41084.log.gz 2024-09-17T05:43:04.829 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50064.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.50064.log.gz --verbose 2024-09-17T05:43:04.829 INFO:teuthology.orchestra.run.smithi122.stderr: -- /var/log/ceph/ceph-client.admin.49647.log 2024-09-17T05:43:04.829 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43324.log 2024-09-17T05:43:04.830 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34265.log 2024-09-17T05:43:04.830 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.34247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29014.log 2024-09-17T05:43:04.830 INFO:teuthology.orchestra.run.smithi173.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34247.log.gz 2024-09-17T05:43:04.830 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.34265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34265.log.gz 2024-09-17T05:43:04.830 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T05:43:04.830 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.29014.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T05:43:04.830 INFO:teuthology.orchestra.run.smithi173.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29014.log.gz 2024-09-17T05:43:04.833 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29069.log 2024-09-17T05:43:04.833 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T05:43:04.833 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.29069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29117.log 2024-09-17T05:43:04.833 INFO:teuthology.orchestra.run.smithi173.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29069.log.gz 2024-09-17T05:43:04.833 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29038.log 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42199.log 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49647.log.gz 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35100.log 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43324.log.gz 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34788.log 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42199.log.gz 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29476.log 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35100.log.gz 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48373.log 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34788.log.gz 2024-09-17T05:43:04.835 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37511.log 2024-09-17T05:43:04.836 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29476.log.gz 2024-09-17T05:43:04.836 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44040.log 2024-09-17T05:43:04.838 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.29117.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29117.log.gz 2024-09-17T05:43:04.839 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T05:43:04.839 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.29038.log: /var/log/ceph/ceph.log 2024-09-17T05:43:04.839 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mgr.x.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29038.log.gz 2024-09-17T05:43:04.839 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T05:43:04.839 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T05:43:04.839 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-17T05:43:04.840 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48373.log.gz 2024-09-17T05:43:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39157.log 2024-09-17T05:43:04.840 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37511.log.gz 2024-09-17T05:43:04.840 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45318.log 2024-09-17T05:43:04.840 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44040.log.gz 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35606.log 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39157.log.gz 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45318.log.gz 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52768.log 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35606.log.gz 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50090.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36628.log 2024-09-17T05:43:04.841 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gz 2024-09-17T05:43:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.52768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52768.log.gz 2024-09-17T05:43:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54244.log 2024-09-17T05:43:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36732.log 2024-09-17T05:43:04.843 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36628.log.gz 2024-09-17T05:43:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51722.log 2024-09-17T05:43:04.843 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54244.log.gz 2024-09-17T05:43:04.843 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43246.log 2024-09-17T05:43:04.844 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28942.log 2024-09-17T05:43:04.845 INFO:teuthology.orchestra.run.smithi173.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T05:43:04.845 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36732.log.gz 2024-09-17T05:43:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51722.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.51722.log.gz --verbose 2024-09-17T05:43:04.845 INFO:teuthology.orchestra.run.smithi122.stderr: -- /var/log/ceph/ceph-client.admin.42895.log 2024-09-17T05:43:04.845 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54981.log 2024-09-17T05:43:04.846 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.28942.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28942.log.gz 2024-09-17T05:43:04.846 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34354.log 2024-09-17T05:43:04.847 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43246.log.gz 2024-09-17T05:43:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52303.log 2024-09-17T05:43:04.847 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42895.log.gz 2024-09-17T05:43:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50116.log 2024-09-17T05:43:04.847 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54981.log.gz 2024-09-17T05:43:04.847 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52463.log 2024-09-17T05:43:04.850 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52303.log.gz 2024-09-17T05:43:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44988.log 2024-09-17T05:43:04.850 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50116.log.gz 2024-09-17T05:43:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2024-09-17T05:43:04.850 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52463.log.gz 2024-09-17T05:43:04.850 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44300.log 2024-09-17T05:43:04.850 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44988.log.gz 2024-09-17T05:43:04.851 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28990.log 2024-09-17T05:43:04.851 INFO:teuthology.orchestra.run.smithi173.stderr: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T05:43:04.851 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.3.34354.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.3.34354.log.gz -5 2024-09-17T05:43:04.851 INFO:teuthology.orchestra.run.smithi173.stderr: --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-17T05:43:04.851 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.28990.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T05:43:04.851 INFO:teuthology.orchestra.run.smithi173.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28990.log.gz 2024-09-17T05:43:04.852 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55635.log 2024-09-17T05:43:04.852 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2024-09-17T05:43:04.852 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53706.log 2024-09-17T05:43:04.852 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44300.log.gz 2024-09-17T05:43:04.852 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42511.log 2024-09-17T05:43:04.853 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55635.log.gz 2024-09-17T05:43:04.853 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46313.log 2024-09-17T05:43:04.853 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53706.log.gz 2024-09-17T05:43:04.854 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36294.log 2024-09-17T05:43:04.854 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42511.log.gz 2024-09-17T05:43:04.854 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39209.log 2024-09-17T05:43:04.854 INFO:teuthology.orchestra.run.smithi122.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46313.log.gz 2024-09-17T05:43:04.854 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2024-09-17T05:43:04.855 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36294.log.gz 2024-09-17T05:43:04.855 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51471.log 2024-09-17T05:43:04.856 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39209.log.gz 2024-09-17T05:43:04.856 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45069.log 2024-09-17T05:43:04.856 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz 2024-09-17T05:43:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38522.log 2024-09-17T05:43:04.857 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51471.log.gz 2024-09-17T05:43:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49247.log 2024-09-17T05:43:04.857 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45069.log.gz 2024-09-17T05:43:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42999.log 2024-09-17T05:43:04.859 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38522.log.gz 2024-09-17T05:43:04.861 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T05:43:04.861 INFO:teuthology.orchestra.run.smithi173.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-17T05:43:04.862 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55347.log 2024-09-17T05:43:04.862 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49247.log.gz 2024-09-17T05:43:04.862 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T05:43:04.863 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42999.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42999.log.gz/var/log/ceph/ceph-client.admin.55347.log: 2024-09-17T05:43:04.863 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48470.log 2024-09-17T05:43:04.863 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55347.log.gz 2024-09-17T05:43:04.863 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37942.log 2024-09-17T05:43:04.863 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55269.log 2024-09-17T05:43:04.863 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48470.log.gz 2024-09-17T05:43:04.865 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28966.log 2024-09-17T05:43:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49731.log 2024-09-17T05:43:04.866 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37942.log.gz 2024-09-17T05:43:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48522.log 2024-09-17T05:43:04.866 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55269.log.gz 2024-09-17T05:43:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56606.log 2024-09-17T05:43:04.866 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49731.log.gz 2024-09-17T05:43:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2024-09-17T05:43:04.867 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48522.log.gz 2024-09-17T05:43:04.868 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34349.log 2024-09-17T05:43:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41294.log 2024-09-17T05:43:04.869 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56606.log.gz 2024-09-17T05:43:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50797.log 2024-09-17T05:43:04.869 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gz 2024-09-17T05:43:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39024.log 2024-09-17T05:43:04.869 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41294.log.gz 2024-09-17T05:43:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47551.log 2024-09-17T05:43:04.869 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50797.log.gz 2024-09-17T05:43:04.870 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.28966.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28966.log.gz 2024-09-17T05:43:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40455.log 2024-09-17T05:43:04.871 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39024.log.gz 2024-09-17T05:43:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44326.log 2024-09-17T05:43:04.871 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47551.log.gz 2024-09-17T05:43:04.872 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29093.log 2024-09-17T05:43:04.872 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35022.log 2024-09-17T05:43:04.872 INFO:teuthology.orchestra.run.smithi122.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40455.log.gz 2024-09-17T05:43:04.872 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37090.log 2024-09-17T05:43:04.873 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44326.log.gz 2024-09-17T05:43:04.873 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.3.34349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.34349.log.gz 2024-09-17T05:43:04.874 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49075.log 2024-09-17T05:43:04.874 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35022.log.gz 2024-09-17T05:43:04.874 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42277.log 2024-09-17T05:43:04.874 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37090.log.gz 2024-09-17T05:43:04.875 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50901.log 2024-09-17T05:43:04.875 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49075.log.gz 2024-09-17T05:43:04.875 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45292.log 2024-09-17T05:43:04.875 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42277.log.gz 2024-09-17T05:43:04.876 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43499.log 2024-09-17T05:43:04.876 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50901.log.gz 2024-09-17T05:43:04.877 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.29093.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29093.log.gz 2024-09-17T05:43:04.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55609.log 2024-09-17T05:43:04.877 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45292.log.gz 2024-09-17T05:43:04.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47785.log 2024-09-17T05:43:04.878 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43499.log.gz 2024-09-17T05:43:04.879 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37168.log 2024-09-17T05:43:04.879 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55609.log.gz 2024-09-17T05:43:04.880 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39638.log 2024-09-17T05:43:04.880 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47785.log.gz 2024-09-17T05:43:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41709.log 2024-09-17T05:43:04.881 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37168.log.gz 2024-09-17T05:43:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39586.log 2024-09-17T05:43:04.882 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39638.log.gz 2024-09-17T05:43:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51077.log 2024-09-17T05:43:04.882 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41709.log.gz 2024-09-17T05:43:04.883 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39586.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39586.log.gz 2024-09-17T05:43:04.883 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51129.log 2024-09-17T05:43:04.884 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45370.log 2024-09-17T05:43:04.884 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51077.log.gz 2024-09-17T05:43:04.885 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39994.log 2024-09-17T05:43:04.885 INFO:teuthology.orchestra.run.smithi122.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51129.log.gz 2024-09-17T05:43:04.886 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40871.log 2024-09-17T05:43:04.886 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45370.log.gz 2024-09-17T05:43:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53362.log 2024-09-17T05:43:04.887 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39994.log.gz 2024-09-17T05:43:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49559.log 2024-09-17T05:43:04.888 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40871.log.gz 2024-09-17T05:43:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-09-17T05:43:04.888 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53362.log.gz 2024-09-17T05:43:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46778.log 2024-09-17T05:43:04.889 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.49559.log.gz 2024-09-17T05:43:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42947.log 2024-09-17T05:43:04.890 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-09-17T05:43:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45446.log 2024-09-17T05:43:04.891 INFO:teuthology.orchestra.run.smithi122.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.46778.log.gz 2024-09-17T05:43:04.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42947.log: gzip 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42947.log.gz -5 2024-09-17T05:43:04.892 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35179.log 2024-09-17T05:43:04.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37738.log 2024-09-17T05:43:04.893 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45446.log.gz 2024-09-17T05:43:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38887.log 2024-09-17T05:43:04.893 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35179.log.gz 2024-09-17T05:43:04.894 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.37738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37738.log.gz 2024-09-17T05:43:04.894 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54929.log 2024-09-17T05:43:04.895 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56016.log 2024-09-17T05:43:04.895 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38887.log.gz 2024-09-17T05:43:04.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38246.log 2024-09-17T05:43:04.896 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54929.log.gz 2024-09-17T05:43:04.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36450.log 2024-09-17T05:43:04.897 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56016.log.gz 2024-09-17T05:43:04.897 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44551.log 2024-09-17T05:43:04.897 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38246.log.gz 2024-09-17T05:43:04.898 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56736.log 2024-09-17T05:43:04.898 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36450.log.gz 2024-09-17T05:43:04.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37194.log 2024-09-17T05:43:04.899 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44551.log.gz 2024-09-17T05:43:04.900 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37302.log 2024-09-17T05:43:04.900 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56736.log.gz 2024-09-17T05:43:04.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49809.log 2024-09-17T05:43:04.901 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37194.log.gz 2024-09-17T05:43:04.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51560.log 2024-09-17T05:43:04.902 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37302.log.gz 2024-09-17T05:43:04.902 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50849.log 2024-09-17T05:43:04.902 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49809.log.gz 2024-09-17T05:43:04.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50355.log 2024-09-17T05:43:04.903 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51560.log.gz 2024-09-17T05:43:04.904 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54270.log 2024-09-17T05:43:04.904 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50849.log.gz 2024-09-17T05:43:04.905 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55059.log 2024-09-17T05:43:04.905 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50355.log.gz 2024-09-17T05:43:04.905 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2024-09-17T05:43:04.906 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54270.log.gz 2024-09-17T05:43:04.906 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49023.log 2024-09-17T05:43:04.907 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.55059.log.gz 2024-09-17T05:43:04.907 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47983.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2024-09-17T05:43:04.907 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38913.log 2024-09-17T05:43:04.908 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39339.log 2024-09-17T05:43:04.909 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49023.log.gz 2024-09-17T05:43:04.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48087.log 2024-09-17T05:43:04.909 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38913.log.gz 2024-09-17T05:43:04.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39966.log 2024-09-17T05:43:04.910 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39339.log.gz 2024-09-17T05:43:04.910 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38116.log 2024-09-17T05:43:04.910 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48087.log.gz 2024-09-17T05:43:04.911 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54623.log 2024-09-17T05:43:04.911 INFO:teuthology.orchestra.run.smithi122.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.39966.log.gz 2024-09-17T05:43:04.912 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51261.log 2024-09-17T05:43:04.912 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38116.log.gz 2024-09-17T05:43:04.913 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44066.log 2024-09-17T05:43:04.913 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54623.log.gz 2024-09-17T05:43:04.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51103.log 2024-09-17T05:43:04.914 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51261.log.gz 2024-09-17T05:43:04.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-09-17T05:43:04.914 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44066.log.gz 2024-09-17T05:43:04.915 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38444.log 2024-09-17T05:43:04.916 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51103.log.gz 2024-09-17T05:43:04.916 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49861.log 2024-09-17T05:43:04.916 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-09-17T05:43:04.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48314.log 2024-09-17T05:43:04.917 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38444.log.gz 2024-09-17T05:43:04.918 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54649.log 2024-09-17T05:43:04.918 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49861.log.gz 2024-09-17T05:43:04.919 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43718.log 2024-09-17T05:43:04.919 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48314.log.gz 2024-09-17T05:43:04.920 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54162.log 2024-09-17T05:43:04.920 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54649.log.gz 2024-09-17T05:43:04.920 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43718.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24016.log 2024-09-17T05:43:04.921 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43718.log.gz 2024-09-17T05:43:04.921 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50272.log 2024-09-17T05:43:04.921 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54162.log.gz 2024-09-17T05:43:04.922 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.tmp-client.admin.24016.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24016.log.gz 2024-09-17T05:43:04.922 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45396.log 2024-09-17T05:43:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48009.log 2024-09-17T05:43:04.923 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50272.log.gz 2024-09-17T05:43:04.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56191.log 2024-09-17T05:43:04.924 INFO:teuthology.orchestra.run.smithi122.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45396.log.gz 2024-09-17T05:43:04.924 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56398.log 2024-09-17T05:43:04.925 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48009.log.gz 2024-09-17T05:43:04.925 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35165.log 2024-09-17T05:43:04.925 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56191.log.gz 2024-09-17T05:43:04.926 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51949.log 2024-09-17T05:43:04.926 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56398.log.gz 2024-09-17T05:43:04.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48579.log 2024-09-17T05:43:04.927 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35165.log.gz 2024-09-17T05:43:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50012.log 2024-09-17T05:43:04.928 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51949.log.gz 2024-09-17T05:43:04.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45132.log 2024-09-17T05:43:04.929 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48579.log.gz 2024-09-17T05:43:04.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35591.log 2024-09-17T05:43:04.929 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50012.log.gz 2024-09-17T05:43:04.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40127.log 2024-09-17T05:43:04.930 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45132.log.gz 2024-09-17T05:43:04.931 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50628.log 2024-09-17T05:43:04.931 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35591.log.gz 2024-09-17T05:43:04.932 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37142.log 2024-09-17T05:43:04.932 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.40127.log.gz 2024-09-17T05:43:04.932 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53505.log 2024-09-17T05:43:04.933 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50628.log.gz 2024-09-17T05:43:04.933 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51233.log 2024-09-17T05:43:04.933 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37142.log.gz 2024-09-17T05:43:04.934 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53862.log 2024-09-17T05:43:04.934 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53505.log.gz 2024-09-17T05:43:04.935 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34996.log 2024-09-17T05:43:04.935 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51233.log.gz 2024-09-17T05:43:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50823.log 2024-09-17T05:43:04.936 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53862.log.gz 2024-09-17T05:43:04.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46753.log 2024-09-17T05:43:04.937 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34996.log.gz 2024-09-17T05:43:04.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55853.log 2024-09-17T05:43:04.937 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50823.log.gz 2024-09-17T05:43:04.938 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29057.log 2024-09-17T05:43:04.938 INFO:teuthology.orchestra.run.smithi122.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.46753.log.gz 2024-09-17T05:43:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43168.log 2024-09-17T05:43:04.939 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55853.log.gz 2024-09-17T05:43:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54897.log 2024-09-17T05:43:04.940 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29057.log.gz 2024-09-17T05:43:04.941 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40806.log 2024-09-17T05:43:04.941 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43168.log.gz 2024-09-17T05:43:04.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-09-17T05:43:04.942 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54897.log.gz 2024-09-17T05:43:04.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52814.log 2024-09-17T05:43:04.942 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40806.log.gz 2024-09-17T05:43:04.943 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43148.log 2024-09-17T05:43:04.943 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-09-17T05:43:04.944 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51526.log 2024-09-17T05:43:04.944 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52814.log.gz 2024-09-17T05:43:04.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47629.log 2024-09-17T05:43:04.945 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43148.log.gz 2024-09-17T05:43:04.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36265.log 2024-09-17T05:43:04.945 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51526.log.gz 2024-09-17T05:43:04.946 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48698.log 2024-09-17T05:43:04.947 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47629.log.gz 2024-09-17T05:43:04.947 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.2.36265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.36265.log.gz 2024-09-17T05:43:04.947 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T05:43:04.948 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36398.log 2024-09-17T05:43:04.948 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48698.log.gz 2024-09-17T05:43:04.949 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51419.log 2024-09-17T05:43:04.949 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53302.log 2024-09-17T05:43:04.950 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36398.log.gz 2024-09-17T05:43:04.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45097.log 2024-09-17T05:43:04.950 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51419.log.gz 2024-09-17T05:43:04.951 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46829.log 2024-09-17T05:43:04.952 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53302.log.gz 2024-09-17T05:43:04.952 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45097.log: gzip -5 87.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45097.log.gz -- 2024-09-17T05:43:04.952 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-client.admin.55425.log 2024-09-17T05:43:04.953 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41214.log 2024-09-17T05:43:04.953 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46829.log.gz 2024-09-17T05:43:04.954 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34597.log 2024-09-17T05:43:04.954 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55425.log.gz 2024-09-17T05:43:04.955 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51592.log 2024-09-17T05:43:04.955 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41214.log.gz 2024-09-17T05:43:04.956 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35367.log 2024-09-17T05:43:04.956 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34597.log.gz 2024-09-17T05:43:04.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48340.log 2024-09-17T05:43:04.957 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51592.log.gz 2024-09-17T05:43:04.958 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37116.log 2024-09-17T05:43:04.958 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35367.log.gz 2024-09-17T05:43:04.959 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34762.log 2024-09-17T05:43:04.959 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48340.log.gz 2024-09-17T05:43:04.960 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42303.log 2024-09-17T05:43:04.960 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37116.log.gz 2024-09-17T05:43:04.960 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35173.log 2024-09-17T05:43:04.961 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34762.log.gz 2024-09-17T05:43:04.961 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37010.log 2024-09-17T05:43:04.962 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42303.log.gz 2024-09-17T05:43:04.962 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41032.log 2024-09-17T05:43:04.963 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35173.log.gz 2024-09-17T05:43:04.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35958.log 2024-09-17T05:43:04.964 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37010.log.gz 2024-09-17T05:43:04.964 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44421.log 2024-09-17T05:43:04.964 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41032.log.gz 2024-09-17T05:43:04.965 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40293.log 2024-09-17T05:43:04.966 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44421.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.35958.log.gz 2024-09-17T05:43:04.966 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44421.log.gz 2024-09-17T05:43:04.966 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56165.log 2024-09-17T05:43:04.967 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46415.log 2024-09-17T05:43:04.968 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40293.log: /var/log/ceph/ceph-client.admin.56165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48614.log 2024-09-17T05:43:04.968 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40293.log.gz 2024-09-17T05:43:04.968 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56165.log.gz 2024-09-17T05:43:04.969 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42407.log 2024-09-17T05:43:04.969 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46415.log.gz 2024-09-17T05:43:04.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35384.log 2024-09-17T05:43:04.970 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48614.log.gz 2024-09-17T05:43:04.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35388.log 2024-09-17T05:43:04.971 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42407.log.gz 2024-09-17T05:43:04.972 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49705.log 2024-09-17T05:43:04.972 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35384.log.gz 2024-09-17T05:43:04.972 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50927.log 2024-09-17T05:43:04.973 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35388.log.gz 2024-09-17T05:43:04.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43631.log 2024-09-17T05:43:04.974 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49705.log.gz 2024-09-17T05:43:04.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2024-09-17T05:43:04.974 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50927.log.gz 2024-09-17T05:43:04.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43631.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz /var/log/ceph/ceph-client.admin.41923.log 2024-09-17T05:43:04.976 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-17T05:43:04.976 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2024-09-17T05:43:04.976 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2024-09-17T05:43:04.977 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45945.log 2024-09-17T05:43:04.977 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41923.log.gz 2024-09-17T05:43:04.978 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.44092.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45669.log 2024-09-17T05:43:04.979 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2024-09-17T05:43:04.979 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45945.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T05:43:04.979 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45945.log.gz 2024-09-17T05:43:04.980 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45266.log 87.2% 2024-09-17T05:43:04.980 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45669.log.gz 2024-09-17T05:43:04.981 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37743.log 2024-09-17T05:43:04.982 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54101.log 2024-09-17T05:43:04.982 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45266.log.gz 2024-09-17T05:43:04.983 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.37743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37743.log.gz 2024-09-17T05:43:04.983 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55557.log 2024-09-17T05:43:04.984 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43551.log 2024-09-17T05:43:04.985 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54101.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54101.log.gz 2024-09-17T05:43:04.985 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38757.log 2024-09-17T05:43:04.985 INFO:teuthology.orchestra.run.smithi122.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.55557.log.gz 2024-09-17T05:43:04.986 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55033.log 2024-09-17T05:43:04.986 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43551.log.gz 2024-09-17T05:43:04.987 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36598.log 2024-09-17T05:43:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38757.log: /var/log/ceph/ceph-client.admin.55033.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38757.log.gz 2024-09-17T05:43:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34531.log 2024-09-17T05:43:04.988 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55033.log.gz 2024-09-17T05:43:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36598.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36598.log.gz 2024-09-17T05:43:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41541.log 2024-09-17T05:43:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-09-17T05:43:04.991 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34531.log: /var/log/ceph/ceph-client.admin.41541.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34531.log.gz 2024-09-17T05:43:04.991 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44858.log 2024-09-17T05:43:04.991 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41541.log.gz 2024-09-17T05:43:04.992 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35592.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-09-17T05:43:04.993 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48132.log 2024-09-17T05:43:04.993 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42329.log 2024-09-17T05:43:04.994 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44858.log: /var/log/ceph/ceph-client.admin.48132.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44858.log.gz 2024-09-17T05:43:04.994 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37036.log 2024-09-17T05:43:04.994 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48132.log.gz 2024-09-17T05:43:04.995 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38220.log 2024-09-17T05:43:04.996 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42329.log.gz 2024-09-17T05:43:04.996 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37563.log 2024-09-17T05:43:04.997 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37036.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37036.log.gz 2024-09-17T05:43:04.997 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54603.log 2024-09-17T05:43:04.998 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38220.log.gz 2024-09-17T05:43:04.998 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37563.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37563.log.gz 2024-09-17T05:43:04.999 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35580.log 2024-09-17T05:43:04.999 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37537.log 2024-09-17T05:43:05.000 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.54603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54603.log.gz 2024-09-17T05:43:05.000 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53810.log 2024-09-17T05:43:05.001 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35580.log.gz 2024-09-17T05:43:05.002 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47681.log 2024-09-17T05:43:05.002 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37537.log.gz 2024-09-17T05:43:05.002 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52489.log 2024-09-17T05:43:05.003 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53810.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53810.log.gz 2024-09-17T05:43:05.004 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53836.log 2024-09-17T05:43:05.005 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47681.log.gz 2024-09-17T05:43:05.005 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38090.log 2024-09-17T05:43:05.006 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52489.log.gz 2024-09-17T05:43:05.006 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47873.log 2024-09-17T05:43:05.006 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53836.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53836.log.gz/var/log/ceph/ceph-client.admin.38090.log: 2024-09-17T05:43:05.007 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log 2024-09-17T05:43:05.007 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38090.log.gz 2024-09-17T05:43:05.008 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52645.log 2024-09-17T05:43:05.008 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47873.log.gz 2024-09-17T05:43:05.009 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36569.log 2024-09-17T05:43:05.009 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47759.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-09-17T05:43:05.010 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51644.log 2024-09-17T05:43:05.010 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52645.log.gz 2024-09-17T05:43:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54426.log 2024-09-17T05:43:05.011 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36569.log.gz 2024-09-17T05:43:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36758.log 2024-09-17T05:43:05.012 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51644.log.gz 2024-09-17T05:43:05.013 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46141.log 2024-09-17T05:43:05.013 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54426.log.gz 2024-09-17T05:43:05.014 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36758.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.48035.log -- replaced with /var/log/ceph/ceph-client.admin.36758.log.gz 2024-09-17T05:43:05.014 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-17T05:43:05.015 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50390.log/var/log/ceph/ceph-client.admin.46141.log: 2024-09-17T05:43:05.015 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46141.log.gz 2024-09-17T05:43:05.016 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56838.log 2024-09-17T05:43:05.016 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48035.log.gz 2024-09-17T05:43:05.017 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50327.log 2024-09-17T05:43:05.017 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50390.log.gz 2024-09-17T05:43:05.018 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-09-17T05:43:05.018 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56838.log.gz 2024-09-17T05:43:05.019 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52918.log 2024-09-17T05:43:05.019 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50327.log.gz 2024-09-17T05:43:05.020 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47707.log: gzip 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-09-17T05:43:05.020 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52541.log 2024-09-17T05:43:05.021 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.52918.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54044.log 2024-09-17T05:43:05.021 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52918.log.gz 2024-09-17T05:43:05.022 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41469.log 2024-09-17T05:43:05.022 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52541.log.gz 2024-09-17T05:43:05.023 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50149.log 2024-09-17T05:43:05.023 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54044.log.gz 2024-09-17T05:43:05.024 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35383.log 2024-09-17T05:43:05.024 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.41469.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41469.log.gz 2024-09-17T05:43:05.025 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38998.log 2024-09-17T05:43:05.025 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50149.log.gz 2024-09-17T05:43:05.026 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29059.log 2024-09-17T05:43:05.026 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35383.log.gz 2024-09-17T05:43:05.027 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46855.log 2024-09-17T05:43:05.027 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38998.log.gz 2024-09-17T05:43:05.028 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38418.log 2024-09-17T05:43:05.028 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29059.log.gz 2024-09-17T05:43:05.029 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41793.log 2024-09-17T05:43:05.030 INFO:teuthology.orchestra.run.smithi122.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.46855.log.gz 2024-09-17T05:43:05.030 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.38418.log: -- /var/log/ceph/ceph-client.admin.49679.log 2024-09-17T05:43:05.031 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38418.log.gz 2024-09-17T05:43:05.031 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41489.log 2024-09-17T05:43:05.032 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41793.log.gz 2024-09-17T05:43:05.032 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49679.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49679.log.gz 2024-09-17T05:43:05.033 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42752.log 2024-09-17T05:43:05.033 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51819.log 2024-09-17T05:43:05.034 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41489.log.gz 2024-09-17T05:43:05.034 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37890.log 2024-09-17T05:43:05.035 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42752.log.gz 2024-09-17T05:43:05.035 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55373.log 2024-09-17T05:43:05.036 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51819.log.gz 2024-09-17T05:43:05.036 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36320.log 2024-09-17T05:43:05.037 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37890.log: /var/log/ceph/ceph-client.admin.55373.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37890.log.gz 2024-09-17T05:43:05.038 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40741.log 2024-09-17T05:43:05.038 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55373.log.gz 2024-09-17T05:43:05.039 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36320.log: gzip -5 --verbose 87.9% -- -- replaced with /var/log/ceph/ceph-client.admin.36320.log.gz /var/log/ceph/ceph-client.admin.40507.log 2024-09-17T05:43:05.039 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-17T05:43:05.040 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52159.log 2024-09-17T05:43:05.040 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.40741.log.gz 2024-09-17T05:43:05.041 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38194.log 2024-09-17T05:43:05.041 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2024-09-17T05:43:05.042 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52159.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52159.log.gz 2024-09-17T05:43:05.042 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44700.log 2024-09-17T05:43:05.043 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52027.log 2024-09-17T05:43:05.043 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38194.log: /var/log/ceph/ceph-client.admin.44700.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38194.log.gz 2024-09-17T05:43:05.044 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44700.log.gz 2024-09-17T05:43:05.044 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53927.log 2024-09-17T05:43:05.045 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43930.log 2024-09-17T05:43:05.045 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52027.log.gz 2024-09-17T05:43:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42225.log 2024-09-17T05:43:05.046 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53927.log.gz 2024-09-17T05:43:05.047 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41567.log 2024-09-17T05:43:05.047 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43930.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.43930.log.gz 2024-09-17T05:43:05.048 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44832.log 2024-09-17T05:43:05.048 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42225.log.gz 2024-09-17T05:43:05.049 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43194.log 2024-09-17T05:43:05.049 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41567.log.gz 2024-09-17T05:43:05.050 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-09-17T05:43:05.050 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44832.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44832.log.gz 2024-09-17T05:43:05.051 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46219.log 2024-09-17T05:43:05.051 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43194.log.gz 2024-09-17T05:43:05.052 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39104.log 2024-09-17T05:43:05.052 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2024-09-17T05:43:05.053 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.46219.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48724.log 2024-09-17T05:43:05.053 INFO:teuthology.orchestra.run.smithi122.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.46219.log.gz 2024-09-17T05:43:05.054 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42696.log 2024-09-17T05:43:05.054 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39104.log.gz 2024-09-17T05:43:05.055 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45695.log 2024-09-17T05:43:05.055 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48724.log.gz 2024-09-17T05:43:05.056 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40715.log 2024-09-17T05:43:05.056 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42696.log.gz 2024-09-17T05:43:05.057 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36504.log 2024-09-17T05:43:05.057 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45695.log.gz 2024-09-17T05:43:05.058 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40715.log: gzip -5 88.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.40715.log.gz -- 2024-09-17T05:43:05.058 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-client.admin.50875.log 2024-09-17T05:43:05.059 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40481.log 2024-09-17T05:43:05.060 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36504.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36504.log.gz 2024-09-17T05:43:05.060 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40611.log 2024-09-17T05:43:05.061 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50875.log.gz 2024-09-17T05:43:05.062 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36654.log 2024-09-17T05:43:05.062 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40481.log.gz 2024-09-17T05:43:05.062 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2024-09-17T05:43:05.063 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40611.log.gz 2024-09-17T05:43:05.064 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45158.log 2024-09-17T05:43:05.064 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36654.log.gz 2024-09-17T05:43:05.064 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47811.log 2024-09-17T05:43:05.065 INFO:teuthology.orchestra.run.smithi122.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2024-09-17T05:43:05.065 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37589.log 2024-09-17T05:43:05.066 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45158.log: /var/log/ceph/ceph-client.admin.47811.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45158.log.gz 2024-09-17T05:43:05.066 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53395.log 2024-09-17T05:43:05.067 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47811.log.gz 2024-09-17T05:43:05.067 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33362.log 87.5% 2024-09-17T05:43:05.068 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37589.log.gz 2024-09-17T05:43:05.068 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55885.log 2024-09-17T05:43:05.069 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53395.log: 87.7%/var/log/ceph/ceph-client.admin.33362.log: -- replaced with /var/log/ceph/ceph-client.admin.53395.log.gz 2024-09-17T05:43:05.070 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49377.log 2024-09-17T05:43:05.070 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33362.log.gz 2024-09-17T05:43:05.071 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55885.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55885.log.gz 2024-09-17T05:43:05.071 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55111.log 2024-09-17T05:43:05.072 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38653.log 2024-09-17T05:43:05.072 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49377.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.49377.log.gz 2024-09-17T05:43:05.073 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51155.log 2024-09-17T05:43:05.073 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55111.log.gz 2024-09-17T05:43:05.074 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42121.log 2024-09-17T05:43:05.074 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38653.log.gz 2024-09-17T05:43:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56139.log 2024-09-17T05:43:05.075 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42121.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51155.log.gz 2024-09-17T05:43:05.076 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38496.log 2024-09-17T05:43:05.076 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42121.log.gz 2024-09-17T05:43:05.077 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56139.log: gzip -5 --verbose -- 87.0% /var/log/ceph/ceph-client.admin.46054.log -- replaced with /var/log/ceph/ceph-client.admin.56139.log.gz 2024-09-17T05:43:05.077 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-17T05:43:05.078 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.38496.log: -- /var/log/ceph/ceph-client.admin.55007.log 2024-09-17T05:43:05.078 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38496.log.gz 2024-09-17T05:43:05.079 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37458.log 2024-09-17T05:43:05.079 INFO:teuthology.orchestra.run.smithi122.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.46054.log.gz 2024-09-17T05:43:05.080 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39835.log 2024-09-17T05:43:05.080 INFO:teuthology.orchestra.run.smithi122.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.55007.log.gz 2024-09-17T05:43:05.081 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45788.log 2024-09-17T05:43:05.081 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37458.log.gz 2024-09-17T05:43:05.082 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39488.log 2024-09-17T05:43:05.082 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39835.log.gz 2024-09-17T05:43:05.083 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52437.log 2024-09-17T05:43:05.083 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45788.log.gz 2024-09-17T05:43:05.084 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.39488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38142.log 2024-09-17T05:43:05.084 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.39488.log.gz 2024-09-17T05:43:05.085 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37354.log 2024-09-17T05:43:05.085 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52437.log.gz 2024-09-17T05:43:05.086 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50704.log 2024-09-17T05:43:05.086 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38142.log.gz 2024-09-17T05:43:05.087 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41346.log 2024-09-17T05:43:05.087 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37354.log.gz 2024-09-17T05:43:05.088 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38575.log 2024-09-17T05:43:05.089 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50704.log.gz 2024-09-17T05:43:05.089 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38705.log 2024-09-17T05:43:05.089 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41346.log.gz 2024-09-17T05:43:05.090 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41949.log 2024-09-17T05:43:05.091 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38575.log.gz 2024-09-17T05:43:05.091 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50472.log 2024-09-17T05:43:05.091 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38705.log.gz 2024-09-17T05:43:05.092 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41949.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41949.log.gz 2024-09-17T05:43:05.092 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45840.log 2024-09-17T05:43:05.093 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35802.log/var/log/ceph/ceph-client.admin.50472.log: 2024-09-17T05:43:05.094 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50472.log.gz 2024-09-17T05:43:05.094 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47037.log 2024-09-17T05:43:05.094 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45840.log.gz 2024-09-17T05:43:05.095 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35802.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35802.log.gz 2024-09-17T05:43:05.096 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44248.log 2024-09-17T05:43:05.097 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44603.log 2024-09-17T05:43:05.097 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47037.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.47037.log.gz 2024-09-17T05:43:05.098 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49932.log 2024-09-17T05:43:05.098 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44248.log.gz 2024-09-17T05:43:05.099 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48930.log 2024-09-17T05:43:05.099 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44603.log.gz 2024-09-17T05:43:05.100 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53427.log 2024-09-17T05:43:05.100 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49932.log.gz 2024-09-17T05:43:05.101 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35169.log 2024-09-17T05:43:05.101 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48930.log.gz 2024-09-17T05:43:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53427.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53427.log.gz 2024-09-17T05:43:05.102 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T05:43:05.103 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43683.log 2024-09-17T05:43:05.104 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35169.log.gz 2024-09-17T05:43:05.104 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53992.log 2024-09-17T05:43:05.105 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36270.log 2024-09-17T05:43:05.106 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43683.log.gz 2024-09-17T05:43:05.106 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45866.log 2024-09-17T05:43:05.107 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53992.log.gz 2024-09-17T05:43:05.108 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.2.36270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39940.log 2024-09-17T05:43:05.108 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.36270.log.gz 2024-09-17T05:43:05.108 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44923.log 2024-09-17T05:43:05.109 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45866.log.gz 2024-09-17T05:43:05.110 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41767.log 2024-09-17T05:43:05.110 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39940.log.gz 2024-09-17T05:43:05.111 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40774.log 2024-09-17T05:43:05.111 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44923.log.gz 2024-09-17T05:43:05.112 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37758.log 2024-09-17T05:43:05.112 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41767.log.gz 2024-09-17T05:43:05.113 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47957.log 2024-09-17T05:43:05.114 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40774.log.gz 2024-09-17T05:43:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55451.log 2024-09-17T05:43:05.115 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37758.log.gz 2024-09-17T05:43:05.115 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44674.log 2024-09-17T05:43:05.116 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47957.log.gz 2024-09-17T05:43:05.117 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37328.log 2024-09-17T05:43:05.117 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55451.log.gz 2024-09-17T05:43:05.118 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29020.log 2024-09-17T05:43:05.118 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44674.log.gz 2024-09-17T05:43:05.119 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45513.log 2024-09-17T05:43:05.119 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37328.log.gz 2024-09-17T05:43:05.120 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40046.log 2024-09-17T05:43:05.120 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29020.log.gz 2024-09-17T05:43:05.121 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46586.log 2024-09-17T05:43:05.121 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45513.log.gz 2024-09-17T05:43:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39235.log 2024-09-17T05:43:05.123 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40046.log.gz 2024-09-17T05:43:05.123 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37691.log 2024-09-17T05:43:05.123 INFO:teuthology.orchestra.run.smithi122.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.46586.log.gz 2024-09-17T05:43:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48880.log 2024-09-17T05:43:05.125 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39235.log.gz 2024-09-17T05:43:05.126 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39560.log 2024-09-17T05:43:05.126 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37691.log.gz 2024-09-17T05:43:05.126 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40663.log 2024-09-17T05:43:05.127 INFO:teuthology.orchestra.run.smithi122.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48880.log.gz 2024-09-17T05:43:05.128 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37064.log 2024-09-17T05:43:05.128 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39560.log.gz 2024-09-17T05:43:05.129 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45240.log 2024-09-17T05:43:05.129 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40663.log.gz 2024-09-17T05:43:05.130 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29061.log 2024-09-17T05:43:05.130 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37064.log.gz 2024-09-17T05:43:05.131 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56068.log 2024-09-17T05:43:05.131 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45240.log.gz 2024-09-17T05:43:05.132 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45893.log 2024-09-17T05:43:05.132 INFO:teuthology.orchestra.run.smithi122.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29061.log.gz 2024-09-17T05:43:05.134 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56580.log 2024-09-17T05:43:05.134 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56068.log.gz 2024-09-17T05:43:05.135 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55661.log 2024-09-17T05:43:05.135 INFO:teuthology.orchestra.run.smithi122.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45893.log.gz 2024-09-17T05:43:05.135 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28903.log 2024-09-17T05:43:05.136 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56580.log.gz 2024-09-17T05:43:05.137 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51497.log 2024-09-17T05:43:05.137 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55661.log.gz 2024-09-17T05:43:05.138 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39493.log 2024-09-17T05:43:05.138 INFO:teuthology.orchestra.run.smithi122.stderr: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.28903.log.gz 2024-09-17T05:43:05.139 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51845.log 2024-09-17T05:43:05.139 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51497.log.gz 2024-09-17T05:43:05.141 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.39493.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.39493.log.gz -5 2024-09-17T05:43:05.141 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35932.log 2024-09-17T05:43:05.141 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46080.log 2024-09-17T05:43:05.142 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51845.log.gz 2024-09-17T05:43:05.142 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2024-09-17T05:43:05.143 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35932.log.gz 2024-09-17T05:43:05.144 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53654.log 2024-09-17T05:43:05.144 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46080.log.gz 2024-09-17T05:43:05.145 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56292.log 2024-09-17T05:43:05.145 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-09-17T05:43:05.146 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47317.log 2024-09-17T05:43:05.146 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53654.log.gz 2024-09-17T05:43:05.147 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51774.log 2024-09-17T05:43:05.147 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56292.log.gz 2024-09-17T05:43:05.148 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52970.log 2024-09-17T05:43:05.148 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47317.log.gz 2024-09-17T05:43:05.149 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47291.log 2024-09-17T05:43:05.149 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51774.log.gz 2024-09-17T05:43:05.150 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48184.log 2024-09-17T05:43:05.150 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52970.log.gz 2024-09-17T05:43:05.151 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45814.log 2024-09-17T05:43:05.152 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47291.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47291.log.gz 2024-09-17T05:43:05.153 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50498.log 2024-09-17T05:43:05.153 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48184.log.gz 2024-09-17T05:43:05.154 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43298.log 2024-09-17T05:43:05.154 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45814.log.gz 2024-09-17T05:43:05.154 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44170.log 2024-09-17T05:43:05.155 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50498.log.gz 2024-09-17T05:43:05.156 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34654.log 2024-09-17T05:43:05.156 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43298.log.gz 2024-09-17T05:43:05.157 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45617.log 2024-09-17T05:43:05.158 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44170.log.gz 2024-09-17T05:43:05.158 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48551.log 2024-09-17T05:43:05.159 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34654.log.gz 2024-09-17T05:43:05.160 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53092.log 2024-09-17T05:43:05.160 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45617.log.gz 2024-09-17T05:43:05.161 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41515.log 2024-09-17T05:43:05.161 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48551.log.gz 2024-09-17T05:43:05.162 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41871.log 2024-09-17T05:43:05.162 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.53092.log.gz 2024-09-17T05:43:05.163 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39753.log 2024-09-17T05:43:05.163 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41515.log.gz 2024-09-17T05:43:05.164 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46728.log 2024-09-17T05:43:05.164 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41871.log.gz 2024-09-17T05:43:05.165 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36862.log 2024-09-17T05:43:05.165 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39753.log.gz 2024-09-17T05:43:05.166 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47237.log 2024-09-17T05:43:05.166 INFO:teuthology.orchestra.run.smithi122.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.46728.log.gz 2024-09-17T05:43:05.167 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52001.log 2024-09-17T05:43:05.167 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36862.log.gz 2024-09-17T05:43:05.168 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29087.log 2024-09-17T05:43:05.168 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47237.log.gz 2024-09-17T05:43:05.169 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38366.log 2024-09-17T05:43:05.170 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52001.log.gz 2024-09-17T05:43:05.171 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29087.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T05:43:05.171 INFO:teuthology.orchestra.run.smithi122.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29087.log.gz 2024-09-17T05:43:05.171 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53328.log 2024-09-17T05:43:05.172 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38366.log.gz 2024-09-17T05:43:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41619.log 2024-09-17T05:43:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36984.log 2024-09-17T05:43:05.174 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53328.log.gz 2024-09-17T05:43:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34840.log 2024-09-17T05:43:05.175 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41619.log.gz 2024-09-17T05:43:05.176 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36984.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36984.log.gz 2024-09-17T05:43:05.177 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51181.log 2024-09-17T05:43:05.177 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34840.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34840.log.gz 2024-09-17T05:43:05.178 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40533.log 2024-09-17T05:43:05.179 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37864.log 2024-09-17T05:43:05.179 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51181.log.gz 2024-09-17T05:43:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40533.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40533.log.gz 2024-09-17T05:43:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38548.log 2024-09-17T05:43:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55791.log 2024-09-17T05:43:05.182 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37864.log.gz 2024-09-17T05:43:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29039.log 2024-09-17T05:43:05.183 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38548.log.gz 2024-09-17T05:43:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44726.log 2024-09-17T05:43:05.184 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55791.log.gz 2024-09-17T05:43:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56344.log 2024-09-17T05:43:05.185 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29039.log.gz 2024-09-17T05:43:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36784.log 2024-09-17T05:43:05.187 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44726.log.gz 2024-09-17T05:43:05.188 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55989.log 2024-09-17T05:43:05.188 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56344.log.gz 2024-09-17T05:43:05.189 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38005.log 2024-09-17T05:43:05.189 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36784.log.gz 2024-09-17T05:43:05.190 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39050.log 2024-09-17T05:43:05.190 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55989.log.gz 2024-09-17T05:43:05.191 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53784.log 2024-09-17T05:43:05.192 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38005.log.gz 2024-09-17T05:43:05.193 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41162.log 2024-09-17T05:43:05.193 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39050.log.gz 2024-09-17T05:43:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40429.log 2024-09-17T05:43:05.194 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53784.log.gz 2024-09-17T05:43:05.195 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40351.log 2024-09-17T05:43:05.195 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41162.log.gz 2024-09-17T05:43:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40429.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40429.log.gz 2024-09-17T05:43:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55687.log 2024-09-17T05:43:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40351.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40351.log.gz 2024-09-17T05:43:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53224.log 2024-09-17T05:43:05.199 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54018.log 2024-09-17T05:43:05.199 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55687.log.gz 2024-09-17T05:43:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39862.log 2024-09-17T05:43:05.200 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53224.log.gz 2024-09-17T05:43:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51315.log 2024-09-17T05:43:05.202 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54018.log.gz 2024-09-17T05:43:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56788.log 2024-09-17T05:43:05.203 INFO:teuthology.orchestra.run.smithi122.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.39862.log.gz 2024-09-17T05:43:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51051.log 2024-09-17T05:43:05.204 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51315.log.gz 2024-09-17T05:43:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-09-17T05:43:05.205 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56788.log.gz 2024-09-17T05:43:05.206 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35126.log 2024-09-17T05:43:05.207 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51051.log.gz 2024-09-17T05:43:05.208 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44356.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-09-17T05:43:05.209 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35126.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35126.log.gz 2024-09-17T05:43:05.209 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56318.log 2024-09-17T05:43:05.210 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46803.log 2024-09-17T05:43:05.212 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53557.log 2024-09-17T05:43:05.212 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56318.log.gz 2024-09-17T05:43:05.213 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46803.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46803.log.gz 2024-09-17T05:43:05.213 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55963.log 2024-09-17T05:43:05.215 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53557.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53557.log.gz 2024-09-17T05:43:05.215 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49217.log 2024-09-17T05:43:05.216 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49533.log 2024-09-17T05:43:05.216 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55963.log.gz 2024-09-17T05:43:05.217 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52355.log 2024-09-17T05:43:05.217 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49217.log.gz 2024-09-17T05:43:05.218 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49533.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49533.log.gz 2024-09-17T05:43:05.219 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-09-17T05:43:05.220 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52355.log.gz 2024-09-17T05:43:05.220 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36155.log 2024-09-17T05:43:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48668.log 2024-09-17T05:43:05.222 INFO:teuthology.orchestra.run.smithi122.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-09-17T05:43:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.36155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36155.log.gz 2024-09-17T05:43:05.223 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50298.log 2024-09-17T05:43:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49783.log 2024-09-17T05:43:05.224 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48668.log.gz 2024-09-17T05:43:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42030.log 2024-09-17T05:43:05.225 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50298.log.gz 2024-09-17T05:43:05.226 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50602.log 2024-09-17T05:43:05.226 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49783.log.gz 2024-09-17T05:43:05.228 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42030.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42030.log.gz 2024-09-17T05:43:05.228 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52411.log 2024-09-17T05:43:05.229 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50602.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50602.log.gz 2024-09-17T05:43:05.229 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34918.log 2024-09-17T05:43:05.231 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46254.log 2024-09-17T05:43:05.231 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52411.log.gz 2024-09-17T05:43:05.232 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34918.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34918.log.gz 2024-09-17T05:43:05.233 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46254.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46254.log.gz 2024-09-17T05:43:05.233 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49455.log 2024-09-17T05:43:05.235 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54608.log 2024-09-17T05:43:05.236 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51618.log 2024-09-17T05:43:05.237 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49455.log.gz 2024-09-17T05:43:05.237 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.54608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54608.log.gz 2024-09-17T05:43:05.238 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51618.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51618.log.gz 2024-09-17T05:43:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34866.log 2024-09-17T05:43:05.240 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42839.log 2024-09-17T05:43:05.241 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48997.log 2024-09-17T05:43:05.241 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34866.log.gz 2024-09-17T05:43:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42839.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42839.log.gz 2024-09-17T05:43:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48997.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48997.log.gz 2024-09-17T05:43:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43905.log 2024-09-17T05:43:05.245 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35176.log 2024-09-17T05:43:05.246 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46193.log 2024-09-17T05:43:05.246 INFO:teuthology.orchestra.run.smithi122.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.43905.log.gz 2024-09-17T05:43:05.247 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36018.log 2024-09-17T05:43:05.247 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35176.log.gz 2024-09-17T05:43:05.249 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46193.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.46193.log.gz 2024-09-17T05:43:05.249 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39783.log 2024-09-17T05:43:05.249 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36018.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36018.log.gz 2024-09-17T05:43:05.250 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47185.log 2024-09-17T05:43:05.251 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34814.log 2024-09-17T05:43:05.252 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39783.log.gz 2024-09-17T05:43:05.253 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47185.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47185.log.gz 2024-09-17T05:43:05.253 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34814.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34814.log.gz 2024-09-17T05:43:05.254 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41474.log 2024-09-17T05:43:05.255 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54480.log 2024-09-17T05:43:05.256 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.41474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44389.log 2024-09-17T05:43:05.256 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41474.log.gz 2024-09-17T05:43:05.257 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54480.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54480.log.gz 2024-09-17T05:43:05.258 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44389.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44389.log.gz 2024-09-17T05:43:05.259 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52211.log 2024-09-17T05:43:05.260 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48977.log 2024-09-17T05:43:05.262 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28840.log 2024-09-17T05:43:05.262 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52211.log.gz 2024-09-17T05:43:05.262 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.48977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48977.log.gz 2024-09-17T05:43:05.262 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54753.log 2024-09-17T05:43:05.264 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28840.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.28840.log.gz -5 2024-09-17T05:43:05.264 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35590.log 2024-09-17T05:43:05.265 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52329.log 2024-09-17T05:43:05.265 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54753.log.gz 2024-09-17T05:43:05.266 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35590.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35590.log.gz 2024-09-17T05:43:05.267 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37222.log 2024-09-17T05:43:05.268 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52329.log.gz 2024-09-17T05:43:05.268 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47733.log 2024-09-17T05:43:05.270 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44752.log 2024-09-17T05:43:05.270 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37222.log.gz 2024-09-17T05:43:05.271 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47733.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47733.log.gz 2024-09-17T05:43:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44752.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44752.log.gz 2024-09-17T05:43:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53479.log 2024-09-17T05:43:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35371.log 2024-09-17T05:43:05.275 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43391.log 2024-09-17T05:43:05.275 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53479.log.gz 2024-09-17T05:43:05.276 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35371.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35371.log.gz 2024-09-17T05:43:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43391.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43391.log.gz 2024-09-17T05:43:05.277 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44473.log 2024-09-17T05:43:05.278 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35577.log 2024-09-17T05:43:05.280 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56266.log 2024-09-17T05:43:05.280 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44473.log.gz 2024-09-17T05:43:05.281 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35577.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35577.log.gz 2024-09-17T05:43:05.282 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56266.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56266.log.gz 2024-09-17T05:43:05.282 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44577.log 2024-09-17T05:43:05.283 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38731.log 2024-09-17T05:43:05.285 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46167.log 2024-09-17T05:43:05.285 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44577.log.gz 2024-09-17T05:43:05.286 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38731.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38731.log.gz 2024-09-17T05:43:05.287 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46167.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46167.log.gz 2024-09-17T05:43:05.287 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51207.log 2024-09-17T05:43:05.288 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56240.log 2024-09-17T05:43:05.290 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40923.log 2024-09-17T05:43:05.290 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51207.log.gz 2024-09-17T05:43:05.291 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56240.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56240.log.gz 2024-09-17T05:43:05.292 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40923.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40923.log.gz 2024-09-17T05:43:05.292 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T05:43:05.293 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47473.log 2024-09-17T05:43:05.296 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51871.log 2024-09-17T05:43:05.296 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47473.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47473.log.gz 2024-09-17T05:43:05.297 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51871.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51871.log.gz 2024-09-17T05:43:05.297 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52062.log 2024-09-17T05:43:05.298 INFO:teuthology.orchestra.run.smithi122.stderr: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T05:43:05.299 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46106.log 2024-09-17T05:43:05.300 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52062.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52062.log.gz 2024-09-17T05:43:05.301 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46106.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46106.log.gz 2024-09-17T05:43:05.301 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50442.log 2024-09-17T05:43:05.302 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-09-17T05:43:05.304 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28940.log 2024-09-17T05:43:05.304 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50442.log.gz 2024-09-17T05:43:05.305 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48802.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-09-17T05:43:05.306 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28940.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.28940.log.gz 2024-09-17T05:43:05.306 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53583.log 2024-09-17T05:43:05.307 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52094.log 2024-09-17T05:43:05.309 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41320.log 2024-09-17T05:43:05.309 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53583.log.gz 2024-09-17T05:43:05.310 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45919.log 2024-09-17T05:43:05.310 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52094.log.gz 2024-09-17T05:43:05.312 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41320.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41320.log.gz 2024-09-17T05:43:05.312 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35579.log 2024-09-17T05:43:05.312 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45919.log.gz 2024-09-17T05:43:05.313 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56812.log 2024-09-17T05:43:05.315 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55321.log 2024-09-17T05:43:05.315 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35579.log.gz 2024-09-17T05:43:05.316 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56812.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56812.log.gz 2024-09-17T05:43:05.317 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55321.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55321.log.gz 2024-09-17T05:43:05.317 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41422.log 2024-09-17T05:43:05.318 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46611.log 2024-09-17T05:43:05.320 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52240.log 2024-09-17T05:43:05.320 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41422.log.gz 2024-09-17T05:43:05.321 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46611.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.46611.log.gz 2024-09-17T05:43:05.322 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52240.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52240.log.gz 2024-09-17T05:43:05.322 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35378.log 2024-09-17T05:43:05.323 INFO:teuthology.orchestra.run.smithi173.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T05:43:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54136.log 2024-09-17T05:43:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28881.log 2024-09-17T05:43:05.325 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35378.log.gz 2024-09-17T05:43:05.326 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54136.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54136.log.gz 2024-09-17T05:43:05.326 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28881.log: 75.8% -- replaced with /var/log/ceph/ceph-client.admin.28881.log.gz 2024-09-17T05:43:05.327 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54675.log 2024-09-17T05:43:05.328 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52866.log 2024-09-17T05:43:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35180.log 2024-09-17T05:43:05.330 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54675.log.gz 2024-09-17T05:43:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39261.log 2024-09-17T05:43:05.332 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52866.log.gz 2024-09-17T05:43:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35180.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35180.log.gz 2024-09-17T05:43:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29072.log 2024-09-17T05:43:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39261.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39261.log.gz 2024-09-17T05:43:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49127.log 2024-09-17T05:43:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45493.log 2024-09-17T05:43:05.336 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29072.log.gz 2024-09-17T05:43:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49127.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49127.log.gz 2024-09-17T05:43:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.45493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45493.log.gz 2024-09-17T05:43:05.338 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-09-17T05:43:05.339 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45591.log 2024-09-17T05:43:05.340 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-09-17T05:43:05.340 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-09-17T05:43:05.342 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46983.log 2024-09-17T05:43:05.342 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45591.log.gz 2024-09-17T05:43:05.343 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50550.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-09-17T05:43:05.343 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47343.log 2024-09-17T05:43:05.344 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46983.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46983.log.gz 2024-09-17T05:43:05.344 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55505.log 2024-09-17T05:43:05.346 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49101.log 2024-09-17T05:43:05.346 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47343.log.gz 2024-09-17T05:43:05.347 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55505.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55505.log.gz 2024-09-17T05:43:05.348 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49101.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49101.log.gz 2024-09-17T05:43:05.348 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38470.log 2024-09-17T05:43:05.349 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41240.log 2024-09-17T05:43:05.351 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39914.log 2024-09-17T05:43:05.351 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38470.log.gz 2024-09-17T05:43:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41240.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41240.log.gz 2024-09-17T05:43:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39914.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39914.log.gz 2024-09-17T05:43:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55911.log 2024-09-17T05:43:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35074.log 2024-09-17T05:43:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2024-09-17T05:43:05.357 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55911.log.gz 2024-09-17T05:43:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35074.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35074.log.gz 2024-09-17T05:43:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44118.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2024-09-17T05:43:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54348.log 2024-09-17T05:43:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53144.log 2024-09-17T05:43:05.361 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51287.log 2024-09-17T05:43:05.361 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54348.log.gz 2024-09-17T05:43:05.362 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53144.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53144.log.gz 2024-09-17T05:43:05.363 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51287.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51287.log.gz 2024-09-17T05:43:05.364 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56762.log 2024-09-17T05:43:05.364 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38392.log 2024-09-17T05:43:05.366 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53453.log 2024-09-17T05:43:05.366 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56762.log.gz 2024-09-17T05:43:05.367 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48776.log 2024-09-17T05:43:05.367 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38392.log.gz 2024-09-17T05:43:05.368 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53453.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53453.log.gz 2024-09-17T05:43:05.369 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44891.log 2024-09-17T05:43:05.369 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48776.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48776.log.gz 2024-09-17T05:43:05.369 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49835.log 2024-09-17T05:43:05.371 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37484.log 2024-09-17T05:43:05.372 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44891.log.gz 2024-09-17T05:43:05.372 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49835.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49835.log.gz 2024-09-17T05:43:05.373 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37484.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37484.log.gz 2024-09-17T05:43:05.374 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 2024-09-17T05:43:05.374 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48496.log 2024-09-17T05:43:05.376 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42001.log 2024-09-17T05:43:05.376 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz 2024-09-17T05:43:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48496.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48496.log.gz 2024-09-17T05:43:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42001.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42001.log.gz 2024-09-17T05:43:05.379 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53732.log 2024-09-17T05:43:05.380 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45758.log 2024-09-17T05:43:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41110.log 2024-09-17T05:43:05.382 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53732.log.gz 2024-09-17T05:43:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45758.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45758.log.gz 2024-09-17T05:43:05.383 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41110.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41110.log.gz 2024-09-17T05:43:05.384 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45997.log 2024-09-17T05:43:05.385 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49403.log 2024-09-17T05:43:05.386 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 2024-09-17T05:43:05.386 INFO:teuthology.orchestra.run.smithi122.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.45997.log.gz 2024-09-17T05:43:05.387 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40980.log 2024-09-17T05:43:05.387 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49403.log.gz 2024-09-17T05:43:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37380.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2024-09-17T05:43:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34736.log 2024-09-17T05:43:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40980.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40980.log.gz 2024-09-17T05:43:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43051.log 2024-09-17T05:43:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38064.log 2024-09-17T05:43:05.391 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34736.log.gz 2024-09-17T05:43:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43051.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43051.log.gz/var/log/ceph/ceph-client.admin.38064.log: 2024-09-17T05:43:05.393 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38064.log.gz 2024-09-17T05:43:05.394 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36920.log 2024-09-17T05:43:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46389.log 2024-09-17T05:43:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45539.log 2024-09-17T05:43:05.397 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36920.log.gz 2024-09-17T05:43:05.398 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41266.log 2024-09-17T05:43:05.398 INFO:teuthology.orchestra.run.smithi122.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.46389.log.gz 2024-09-17T05:43:05.399 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45539.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45539.log.gz 2024-09-17T05:43:05.400 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52788.log 2024-09-17T05:43:05.400 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41266.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41266.log.gz 2024-09-17T05:43:05.401 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35776.log 2024-09-17T05:43:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46886.log 2024-09-17T05:43:05.402 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52788.log.gz 2024-09-17T05:43:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35776.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35776.log.gz 2024-09-17T05:43:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35177.log 2024-09-17T05:43:05.404 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46886.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46886.log.gz 2024-09-17T05:43:05.405 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35177.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35177.log.gz 2024-09-17T05:43:05.406 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39076.log 2024-09-17T05:43:05.407 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51017.log 2024-09-17T05:43:05.408 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44499.log 2024-09-17T05:43:05.409 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39076.log.gz 2024-09-17T05:43:05.409 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54374.log 2024-09-17T05:43:05.410 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51017.log.gz 2024-09-17T05:43:05.411 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44499.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44499.log.gz 2024-09-17T05:43:05.411 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37615.log 2024-09-17T05:43:05.412 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54374.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54374.log.gz 2024-09-17T05:43:05.412 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34944.log 2024-09-17T05:43:05.414 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53170.log 2024-09-17T05:43:05.414 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37615.log.gz 2024-09-17T05:43:05.415 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34944.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34944.log.gz 2024-09-17T05:43:05.416 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53170.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53170.log.gz 2024-09-17T05:43:05.417 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44447.log 2024-09-17T05:43:05.418 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50771.log 2024-09-17T05:43:05.419 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41593.log 2024-09-17T05:43:05.419 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44447.log.gz 2024-09-17T05:43:05.420 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50771.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50771.log.gz 2024-09-17T05:43:05.421 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41593.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41593.log.gz 2024-09-17T05:43:05.421 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49614.log 2024-09-17T05:43:05.423 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28927.log 2024-09-17T05:43:05.424 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49614.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49614.log.gz 2024-09-17T05:43:05.424 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54779.log 2024-09-17T05:43:05.425 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28927.log: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.28927.log.gz 2024-09-17T05:43:05.427 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54779.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54779.log.gz 2024-09-17T05:43:05.427 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39809.log 2024-09-17T05:43:05.428 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56632.log 2024-09-17T05:43:05.430 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55583.log 2024-09-17T05:43:05.430 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39809.log.gz 2024-09-17T05:43:05.430 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35048.log 2024-09-17T05:43:05.431 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56632.log.gz 2024-09-17T05:43:05.432 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55583.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55583.log.gz 2024-09-17T05:43:05.433 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29098.log 2024-09-17T05:43:05.433 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35048.log.gz 2024-09-17T05:43:05.433 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29062.log 2024-09-17T05:43:05.435 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45971.log 2024-09-17T05:43:05.435 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.29098.log.gz 2024-09-17T05:43:05.436 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29062.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29062.log.gz 2024-09-17T05:43:05.437 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45971.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45971.log.gz 2024-09-17T05:43:05.437 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43220.log 2024-09-17T05:43:05.439 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48405.log 2024-09-17T05:43:05.440 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49273.log 2024-09-17T05:43:05.440 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43220.log.gz 2024-09-17T05:43:05.441 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48405.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48405.log.gz 2024-09-17T05:43:05.442 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49273.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.49273.log.gz 2024-09-17T05:43:05.443 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37432.log 2024-09-17T05:43:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55085.log 2024-09-17T05:43:05.445 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55215.log 2024-09-17T05:43:05.445 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37432.log.gz 2024-09-17T05:43:05.446 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37810.log 2024-09-17T05:43:05.447 INFO:teuthology.orchestra.run.smithi122.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.55085.log.gz 2024-09-17T05:43:05.448 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55215.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.55215.log.gz 2024-09-17T05:43:05.449 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45732.log 2024-09-17T05:43:05.449 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37810.log.gz 2024-09-17T05:43:05.449 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55295.log 2024-09-17T05:43:05.451 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50751.log 2024-09-17T05:43:05.451 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45732.log.gz 2024-09-17T05:43:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55295.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55295.log.gz 2024-09-17T05:43:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.50751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50751.log.gz 2024-09-17T05:43:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40403.log 2024-09-17T05:43:05.455 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45184.log 2024-09-17T05:43:05.456 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40403.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40403.log.gz 2024-09-17T05:43:05.457 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45184.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45184.log.gz 2024-09-17T05:43:05.457 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49325.log 2024-09-17T05:43:05.459 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35160.log 2024-09-17T05:43:05.460 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48236.log 2024-09-17T05:43:05.460 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49325.log.gz 2024-09-17T05:43:05.461 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35160.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35160.log.gz 2024-09-17T05:43:05.462 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48236.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48236.log.gz 2024-09-17T05:43:05.463 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46280.log 2024-09-17T05:43:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39365.log 2024-09-17T05:43:05.465 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46938.log 2024-09-17T05:43:05.466 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46280.log.gz 2024-09-17T05:43:05.466 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39365.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39365.log.gz 2024-09-17T05:43:05.467 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46938.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46938.log.gz 2024-09-17T05:43:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47009.log 2024-09-17T05:43:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52567.log 2024-09-17T05:43:05.470 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35906.log 2024-09-17T05:43:05.471 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47009.log.gz 2024-09-17T05:43:05.471 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54188.log 2024-09-17T05:43:05.472 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52567.log.gz 2024-09-17T05:43:05.472 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35906.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35906.log.gz 2024-09-17T05:43:05.473 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28915.log 2024-09-17T05:43:05.476 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54188.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54188.log.gz 2024-09-17T05:43:05.476 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39183.log 2024-09-17T05:43:05.477 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51748.log 2024-09-17T05:43:05.477 INFO:teuthology.orchestra.run.smithi122.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.28915.log.gz 2024-09-17T05:43:05.477 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39183.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39183.log.gz 2024-09-17T05:43:05.477 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41735.log 2024-09-17T05:43:05.478 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51748.log.gz 2024-09-17T05:43:05.478 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47421.log 2024-09-17T05:43:05.480 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41735.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T05:43:05.480 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41735.log.gz 2024-09-17T05:43:05.481 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47421.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47421.log.gz 2024-09-17T05:43:05.483 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40231.log 2024-09-17T05:43:05.485 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44778.log 2024-09-17T05:43:05.486 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40231.log: 87.3% 90.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T05:43:05.486 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40231.log.gz 2024-09-17T05:43:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44778.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44778.log.gz 2024-09-17T05:43:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36179.log 2024-09-17T05:43:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-17T05:43:05.490 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36346.log 2024-09-17T05:43:05.490 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36179.log.gz 2024-09-17T05:43:05.492 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.36346.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36346.log.gz 2024-09-17T05:43:05.492 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43603.log 2024-09-17T05:43:05.494 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56450.log 2024-09-17T05:43:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43603.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43603.log.gz 2024-09-17T05:43:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56450.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56450.log.gz 2024-09-17T05:43:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44806.log 2024-09-17T05:43:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56113.log 2024-09-17T05:43:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44806.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44806.log.gz 2024-09-17T05:43:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56113.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56113.log.gz 2024-09-17T05:43:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34970.log 2024-09-17T05:43:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53680.log 2024-09-17T05:43:05.503 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34970.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34970.log.gz 2024-09-17T05:43:05.504 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53680.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53680.log.gz 2024-09-17T05:43:05.505 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41819.log 2024-09-17T05:43:05.506 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52892.log 2024-09-17T05:43:05.510 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41819.log: gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.41819.log.gz --verbose 2024-09-17T05:43:05.510 INFO:teuthology.orchestra.run.smithi122.stderr: -- /var/log/ceph/ceph-client.admin.37250.log 2024-09-17T05:43:05.510 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52892.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52892.log.gz 2024-09-17T05:43:05.510 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.37250.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35828.log 2024-09-17T05:43:05.510 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37250.log.gz 2024-09-17T05:43:05.512 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35828.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35828.log.gz 2024-09-17T05:43:05.512 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43101.log 2024-09-17T05:43:05.514 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-17T05:43:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43101.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43101.log.gz 2024-09-17T05:43:05.519 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56710.log 2024-09-17T05:43:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29055.log 2024-09-17T05:43:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56710.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56710.log.gz 2024-09-17T05:43:05.523 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48288.log 2024-09-17T05:43:05.524 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54073.log 2024-09-17T05:43:05.524 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29055.log.gz 2024-09-17T05:43:05.525 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48288.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48288.log.gz 2024-09-17T05:43:05.527 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5/var/log/ceph/ceph-client.admin.54073.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54073.log.gz 2024-09-17T05:43:05.527 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41188.log 2024-09-17T05:43:05.530 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43988.log 2024-09-17T05:43:05.530 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41188.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41188.log.gz 2024-09-17T05:43:05.532 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44196.log 2024-09-17T05:43:05.533 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43988.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43988.log.gz 2024-09-17T05:43:05.535 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54727.log 2024-09-17T05:43:05.536 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44196.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44196.log.gz 2024-09-17T05:43:05.538 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54727.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54727.log.gz 2024-09-17T05:43:05.539 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55189.log 2024-09-17T05:43:05.542 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55189.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55189.log.gz 2024-09-17T05:43:05.542 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46912.log 2024-09-17T05:43:05.548 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46912.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46912.log.gz 2024-09-17T05:43:05.548 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-09-17T05:43:05.548 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48828.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gz 2024-09-17T05:43:05.548 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38275.log 2024-09-17T05:43:05.554 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38275.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38275.log.gz 2024-09-17T05:43:05.554 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49986.log 2024-09-17T05:43:05.554 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49986.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49986.log.gz 2024-09-17T05:43:05.554 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43153.log 2024-09-17T05:43:05.557 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.43153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43153.log.gz 2024-09-17T05:43:05.557 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34682.log 2024-09-17T05:43:05.563 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34682.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34682.log.gz 2024-09-17T05:43:05.563 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51367.log 2024-09-17T05:43:05.563 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51367.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51367.log.gz 2024-09-17T05:43:05.563 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56658.log 2024-09-17T05:43:05.566 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56658.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56658.log.gz 2024-09-17T05:43:05.566 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48982.log 2024-09-17T05:43:05.567 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38790.log 2024-09-17T05:43:05.569 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.48982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48982.log.gz 2024-09-17T05:43:05.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38790.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38790.log.gz 2024-09-17T05:43:05.572 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40153.log 2024-09-17T05:43:05.573 INFO:teuthology.orchestra.run.smithi173.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T05:43:05.575 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40153.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40153.log.gz 2024-09-17T05:43:05.575 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55713.log 2024-09-17T05:43:05.578 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55713.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55713.log.gz 2024-09-17T05:43:05.578 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40325.log 2024-09-17T05:43:05.580 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34892.log 2024-09-17T05:43:05.581 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40325.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40325.log.gz 2024-09-17T05:43:05.583 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34892.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34892.log.gz 2024-09-17T05:43:05.584 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43834.log 2024-09-17T05:43:05.586 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43834.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.43834.log.gz 2024-09-17T05:43:05.586 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28891.log 2024-09-17T05:43:05.589 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28891.log: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.28891.log.gz 2024-09-17T05:43:05.589 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35568.log 2024-09-17T05:43:05.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48640.log 2024-09-17T05:43:05.592 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35568.log.gz 2024-09-17T05:43:05.596 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.48640.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48640.log.gz 2024-09-17T05:43:05.596 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43025.log 2024-09-17T05:43:05.598 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43025.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43025.log.gz 2024-09-17T05:43:05.599 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40205.log 2024-09-17T05:43:05.601 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51975.log 2024-09-17T05:43:05.601 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40205.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40205.log.gz 2024-09-17T05:43:05.604 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51975.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51975.log.gz 2024-09-17T05:43:05.604 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41006.log 2024-09-17T05:43:05.607 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41006.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41006.log.gz 2024-09-17T05:43:05.607 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54556.log 2024-09-17T05:43:05.610 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54556.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54556.log.gz 2024-09-17T05:43:05.610 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42544.log 2024-09-17T05:43:05.612 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48158.log 2024-09-17T05:43:05.613 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42544.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42544.log.gz 2024-09-17T05:43:05.615 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2024-09-17T05:43:05.615 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48158.log.gz 2024-09-17T05:43:05.616 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55399.log 2024-09-17T05:43:05.621 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49153.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2024-09-17T05:43:05.621 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55399.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55399.log.gz 2024-09-17T05:43:05.621 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49507.log 2024-09-17T05:43:05.624 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49507.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49507.log.gz 2024-09-17T05:43:05.624 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54452.log 2024-09-17T05:43:05.627 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54452.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54452.log.gz 2024-09-17T05:43:05.627 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42787.log 2024-09-17T05:43:05.630 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42787.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42787.log.gz 2024-09-17T05:43:05.630 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50756.log 2024-09-17T05:43:05.635 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45214.log 2024-09-17T05:43:05.636 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.50756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50756.log.gz 2024-09-17T05:43:05.636 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45214.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45214.log.gz 2024-09-17T05:43:05.636 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39508.log 2024-09-17T05:43:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50576.log 2024-09-17T05:43:05.639 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39508.log.gz 2024-09-17T05:43:05.641 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52268.log 2024-09-17T05:43:05.642 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50576.log.gz 2024-09-17T05:43:05.643 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54218.log 2024-09-17T05:43:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52268.log: /var/log/ceph/ceph-client.admin.54218.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54218.log.gz 2024-09-17T05:43:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36888.log 2024-09-17T05:43:05.648 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52268.log.gz 2024-09-17T05:43:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43272.log 2024-09-17T05:43:05.653 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36888.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36888.log.gz 2024-09-17T05:43:05.653 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56476.log 2024-09-17T05:43:05.653 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43272.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43272.log.gz 2024-09-17T05:43:05.654 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50038.log 2024-09-17T05:43:05.658 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56476.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56476.log.gz 2024-09-17T05:43:05.658 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50038.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50038.log.gz 2024-09-17T05:43:05.658 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55477.log 2024-09-17T05:43:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55477.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55477.log.gz 2024-09-17T05:43:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52944.log 2024-09-17T05:43:05.664 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53758.log 2024-09-17T05:43:05.664 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52944.log.gz 2024-09-17T05:43:05.665 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40377.log 2024-09-17T05:43:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53758.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53758.log.gz 2024-09-17T05:43:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40377.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40377.log.gz 2024-09-17T05:43:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40075.log 2024-09-17T05:43:05.673 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40075.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40075.log.gz 2024-09-17T05:43:05.673 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36680.log 2024-09-17T05:43:05.675 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36680.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36680.log.gz 2024-09-17T05:43:05.675 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40952.log 2024-09-17T05:43:05.678 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40952.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40952.log.gz 2024-09-17T05:43:05.678 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43962.log 2024-09-17T05:43:05.681 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43962.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43962.log.gz 2024-09-17T05:43:05.681 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56424.log 2024-09-17T05:43:05.682 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48750.log 2024-09-17T05:43:05.685 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56424.log: /var/log/ceph/ceph-client.admin.48750.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56424.log.gz 2024-09-17T05:43:05.685 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41897.log 2024-09-17T05:43:05.685 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48750.log.gz 2024-09-17T05:43:05.688 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41897.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41897.log.gz 2024-09-17T05:43:05.688 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47107.log 2024-09-17T05:43:05.691 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47107.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.47107.log.gz 2024-09-17T05:43:05.691 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45344.log 2024-09-17T05:43:05.693 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45344.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45344.log.gz 2024-09-17T05:43:05.693 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47211.log 2024-09-17T05:43:05.695 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51393.log 2024-09-17T05:43:05.697 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47211.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47211.log.gz 2024-09-17T05:43:05.699 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-09-17T05:43:05.699 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51393.log.gz 2024-09-17T05:43:05.700 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52619.log 2024-09-17T05:43:05.705 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50181.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-09-17T05:43:05.705 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52619.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52619.log.gz 2024-09-17T05:43:05.705 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38168.log 2024-09-17T05:43:05.708 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38168.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38168.log.gz 2024-09-17T05:43:05.708 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44274.log 2024-09-17T05:43:05.711 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44274.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.44274.log.gz 2024-09-17T05:43:05.711 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48262.log 2024-09-17T05:43:05.714 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48262.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48262.log.gz 2024-09-17T05:43:05.714 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38340.log 2024-09-17T05:43:05.717 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45040.log 2024-09-17T05:43:05.717 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38340.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38340.log.gz 2024-09-17T05:43:05.720 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45040.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45040.log.gz 2024-09-17T05:43:05.720 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42598.log 2024-09-17T05:43:05.723 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42598.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42598.log.gz 2024-09-17T05:43:05.723 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40559.log 2024-09-17T05:43:05.726 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40559.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40559.log.gz 2024-09-17T05:43:05.726 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36478.log 2024-09-17T05:43:05.729 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51897.log 2024-09-17T05:43:05.729 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36478.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36478.log.gz 2024-09-17T05:43:05.731 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51897.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51897.log.gz 2024-09-17T05:43:05.731 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48061.log 2024-09-17T05:43:05.735 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48061.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48061.log.gz 2024-09-17T05:43:05.735 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41058.log 2024-09-17T05:43:05.737 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41058.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41058.log.gz 2024-09-17T05:43:05.737 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46363.log 2024-09-17T05:43:05.739 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56042.log 2024-09-17T05:43:05.741 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46363.log: /var/log/ceph/ceph-client.admin.56042.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.46363.log.gz 2024-09-17T05:43:05.741 INFO:teuthology.orchestra.run.smithi122.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56042.log.gz 2024-09-17T05:43:05.742 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39391.log 2024-09-17T05:43:05.745 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39391.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39391.log.gz 2024-09-17T05:43:05.745 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43417.log 2024-09-17T05:43:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43417.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43417.log.gz 2024-09-17T05:43:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46028.log 2024-09-17T05:43:05.751 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42724.log 2024-09-17T05:43:05.751 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46028.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.46028.log.gz 2024-09-17T05:43:05.754 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42724.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42724.log.gz 2024-09-17T05:43:05.754 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54322.log 2024-09-17T05:43:05.757 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54322.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54322.log.gz 2024-09-17T05:43:05.757 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41136.log 2024-09-17T05:43:05.760 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41136.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41136.log.gz 2024-09-17T05:43:05.760 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49481.log 2024-09-17T05:43:05.763 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49481.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49481.log.gz 2024-09-17T05:43:05.763 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39130.log 2024-09-17T05:43:05.766 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39727.log 2024-09-17T05:43:05.766 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39130.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39130.log.gz 2024-09-17T05:43:05.769 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39727.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39727.log.gz 2024-09-17T05:43:05.769 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44648.log 2024-09-17T05:43:05.772 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44648.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44648.log.gz 2024-09-17T05:43:05.772 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34628.log 2024-09-17T05:43:05.775 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34628.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34628.log.gz 2024-09-17T05:43:05.775 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50953.log 2024-09-17T05:43:05.779 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36836.log 2024-09-17T05:43:05.779 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50953.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50953.log.gz 2024-09-17T05:43:05.780 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36836.log: -- /var/log/ceph/ceph-client.admin.53033.log 2024-09-17T05:43:05.780 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36836.log.gz 2024-09-17T05:43:05.783 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53033.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53033.log.gz 2024-09-17T05:43:05.783 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29335.log 2024-09-17T05:43:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29335.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29335.log.gz 2024-09-17T05:43:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42355.log 2024-09-17T05:43:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42355.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42355.log.gz 2024-09-17T05:43:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53531.log 2024-09-17T05:43:05.793 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50654.log 2024-09-17T05:43:05.793 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53531.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53531.log.gz 2024-09-17T05:43:05.793 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38308.log 2024-09-17T05:43:05.799 INFO:teuthology.orchestra.run.smithi122.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50654.log.gz 2024-09-17T05:43:05.799 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38308.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38308.log.gz 2024-09-17T05:43:05.799 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38822.log 2024-09-17T05:43:05.799 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38822.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38822.log.gz 2024-09-17T05:43:05.799 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43525.log 2024-09-17T05:43:05.801 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43525.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.43525.log.gz 2024-09-17T05:43:05.801 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-09-17T05:43:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36706.log 2024-09-17T05:43:05.805 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-09-17T05:43:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56554.log 2024-09-17T05:43:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36706.log: /var/log/ceph/ceph-client.admin.56554.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36424.log 2024-09-17T05:43:05.811 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36706.log.gz 2024-09-17T05:43:05.812 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56554.log.gz 2024-09-17T05:43:05.813 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47265.log 2024-09-17T05:43:05.814 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36424.log.gz 2024-09-17T05:43:05.815 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29388.log 2024-09-17T05:43:05.817 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47265.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47265.log.gz 2024-09-17T05:43:05.819 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40897.log 2024-09-17T05:43:05.821 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29388.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29388.log.gz 2024-09-17T05:43:05.823 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49585.log 2024-09-17T05:43:05.824 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40897.log.gz 2024-09-17T05:43:05.826 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41845.log 2024-09-17T05:43:05.827 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49585.log.gz 2024-09-17T05:43:05.830 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.41845.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41845.log.gz 2024-09-17T05:43:05.830 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47577.log 2024-09-17T05:43:05.833 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47577.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47577.log.gz 2024-09-17T05:43:05.833 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42147.log 2024-09-17T05:43:05.839 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42147.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42147.log.gz 2024-09-17T05:43:05.839 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38601.log 2024-09-17T05:43:05.839 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38601.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38601.log.gz 2024-09-17T05:43:05.839 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53250.log 2024-09-17T05:43:05.842 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53250.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53250.log.gz 2024-09-17T05:43:05.842 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42251.log 2024-09-17T05:43:05.845 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42251.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42251.log.gz 2024-09-17T05:43:05.845 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-09-17T05:43:05.848 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49757.log 2024-09-17T05:43:05.848 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37641.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2024-09-17T05:43:05.851 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49757.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49757.log.gz 2024-09-17T05:43:05.851 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39441.log 2024-09-17T05:43:05.853 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39441.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39441.log.gz 2024-09-17T05:43:05.854 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40689.log 2024-09-17T05:43:05.857 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40689.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40689.log.gz 2024-09-17T05:43:05.857 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43577.log 2024-09-17T05:43:05.859 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43577.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43577.log.gz 2024-09-17T05:43:05.860 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-09-17T05:43:05.862 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49906.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-09-17T05:43:05.862 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37916.log 2024-09-17T05:43:05.867 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41372.log 2024-09-17T05:43:05.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37916.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37916.log.gz 2024-09-17T05:43:05.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55820.log 2024-09-17T05:43:05.868 INFO:teuthology.orchestra.run.smithi122.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41372.log.gz 2024-09-17T05:43:05.869 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38031.log 2024-09-17T05:43:05.870 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55820.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55820.log.gz 2024-09-17T05:43:05.873 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38031.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38031.log.gz 2024-09-17T05:43:05.874 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44014.log 2024-09-17T05:43:05.876 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49299.log 2024-09-17T05:43:05.877 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44014.log.gz 2024-09-17T05:43:05.879 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48854.log 2024-09-17T05:43:05.880 INFO:teuthology.orchestra.run.smithi122.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49299.log.gz 2024-09-17T05:43:05.883 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5/var/log/ceph/ceph-client.admin.48854.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48854.log.gz 2024-09-17T05:43:05.883 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39888.log 2024-09-17T05:43:05.886 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39888.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39888.log.gz 2024-09-17T05:43:05.887 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50246.log 2024-09-17T05:43:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50246.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50246.log.gz 2024-09-17T05:43:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53895.log 2024-09-17T05:43:05.891 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43447.log 2024-09-17T05:43:05.893 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53895.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53895.log.gz 2024-09-17T05:43:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43447.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43447.log.gz 2024-09-17T05:43:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49191.log 2024-09-17T05:43:05.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49191.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49191.log.gz 2024-09-17T05:43:05.899 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54805.log 2024-09-17T05:43:05.903 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45498.log 2024-09-17T05:43:05.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54805.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54805.log.gz 2024-09-17T05:43:05.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.45498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45498.log.gz 2024-09-17T05:43:05.903 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37276.log 2024-09-17T05:43:05.906 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37276.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37276.log.gz 2024-09-17T05:43:05.907 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39287.log 2024-09-17T05:43:05.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42095.log 2024-09-17T05:43:05.909 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39287.log.gz 2024-09-17T05:43:05.911 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54506.log 2024-09-17T05:43:05.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42095.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42095.log.gz 2024-09-17T05:43:05.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54506.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.54506.log.gz 2024-09-17T05:43:05.917 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55241.log 2024-09-17T05:43:05.919 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55241.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.55241.log.gz 2024-09-17T05:43:05.920 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43350.log 2024-09-17T05:43:05.922 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43350.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43350.log.gz 2024-09-17T05:43:05.922 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47655.log 2024-09-17T05:43:05.927 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35854.log 2024-09-17T05:43:05.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47655.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.47655.log.gz 2024-09-17T05:43:05.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35854.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35854.log.gz 2024-09-17T05:43:05.928 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47369.log 2024-09-17T05:43:05.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47369.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47369.log.gz 2024-09-17T05:43:05.931 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47395.log 2024-09-17T05:43:05.934 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47395.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47395.log.gz 2024-09-17T05:43:05.934 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40179.log 2024-09-17T05:43:05.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40179.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40179.log.gz 2024-09-17T05:43:05.937 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53198.log 2024-09-17T05:43:05.938 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36810.log 2024-09-17T05:43:05.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53198.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53198.log.gz 2024-09-17T05:43:05.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36810.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36810.log.gz 2024-09-17T05:43:05.942 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45565.log 2024-09-17T05:43:05.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45565.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45565.log.gz 2024-09-17T05:43:05.945 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55531.log 2024-09-17T05:43:05.947 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55531.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55531.log.gz 2024-09-17T05:43:05.947 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54871.log 2024-09-17T05:43:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54871.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54871.log.gz 2024-09-17T05:43:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47447.log 2024-09-17T05:43:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47447.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47447.log.gz 2024-09-17T05:43:05.953 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46636.log 2024-09-17T05:43:05.957 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55137.log 2024-09-17T05:43:05.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46636.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.46636.log.gz 2024-09-17T05:43:05.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42381.log 2024-09-17T05:43:05.957 INFO:teuthology.orchestra.run.smithi122.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55137.log.gz 2024-09-17T05:43:05.960 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42381.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42381.log.gz 2024-09-17T05:43:05.960 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42433.log 2024-09-17T05:43:05.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42433.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42433.log.gz 2024-09-17T05:43:05.963 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39612.log 2024-09-17T05:43:05.967 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52773.log 2024-09-17T05:43:05.967 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39612.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39612.log.gz 2024-09-17T05:43:05.967 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.52773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38627.log 2024-09-17T05:43:05.967 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52773.log.gz 2024-09-17T05:43:05.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38627.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38627.log.gz 2024-09-17T05:43:05.971 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40585.log 2024-09-17T05:43:05.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52671.log 2024-09-17T05:43:05.974 INFO:teuthology.orchestra.run.smithi122.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40585.log.gz 2024-09-17T05:43:05.975 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28999.log 2024-09-17T05:43:05.978 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52671.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52671.log.gz 2024-09-17T05:43:05.980 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.28999.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47525.log 2024-09-17T05:43:05.981 INFO:teuthology.orchestra.run.smithi122.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.28999.log.gz 2024-09-17T05:43:05.983 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42644.log 2024-09-17T05:43:05.984 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47525.log.gz 2024-09-17T05:43:05.986 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54296.log 2024-09-17T05:43:05.988 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35373.log 2024-09-17T05:43:05.989 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54296.log: -- replaced with /var/log/ceph/ceph-client.admin.42644.log.gz 2024-09-17T05:43:05.990 INFO:teuthology.orchestra.run.smithi122.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54296.log.gz 2024-09-17T05:43:05.991 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54955.log 2024-09-17T05:43:05.995 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54955.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54955.log.gz 2024-09-17T05:43:05.995 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35373.log.gz 2024-09-17T05:43:05.995 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52185.log 2024-09-17T05:43:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52185.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52185.log.gz 2024-09-17T05:43:05.998 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34710.log 2024-09-17T05:43:06.001 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34710.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34710.log.gz 2024-09-17T05:43:06.001 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37784.log 2024-09-17T05:43:06.003 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-09-17T05:43:06.005 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37784.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37784.log.gz 2024-09-17T05:43:06.005 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5/var/log/ceph/ceph-client.admin.47159.log: --verbose -- /var/log/ceph/ceph-client.admin.52721.log 2024-09-17T05:43:06.008 INFO:teuthology.orchestra.run.smithi122.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-09-17T05:43:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52721.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52721.log.gz 2024-09-17T05:43:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55937.log 2024-09-17T05:43:06.011 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55937.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55937.log.gz 2024-09-17T05:43:06.012 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44222.log 2024-09-17T05:43:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51341.log 2024-09-17T05:43:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44222.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44222.log.gz 2024-09-17T05:43:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51341.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51341.log.gz 2024-09-17T05:43:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53007.log 2024-09-17T05:43:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.53007.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53007.log.gz 2024-09-17T05:43:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42670.log 2024-09-17T05:43:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42670.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42670.log.gz 2024-09-17T05:43:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54831.log 2024-09-17T05:43:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54831.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54831.log.gz 2024-09-17T05:43:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47603.log 2024-09-17T05:43:06.027 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42173.log 2024-09-17T05:43:06.029 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47603.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47603.log.gz 2024-09-17T05:43:06.032 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42173.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42173.log.gz 2024-09-17T05:43:06.032 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52593.log 2024-09-17T05:43:06.035 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52593.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52593.log.gz 2024-09-17T05:43:06.035 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51923.log 2024-09-17T05:43:06.038 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51923.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51923.log.gz 2024-09-17T05:43:06.038 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56370.log 2024-09-17T05:43:06.041 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54701.log 2024-09-17T05:43:06.041 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56370.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56370.log.gz 2024-09-17T05:43:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.54701.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54701.log.gz 2024-09-17T05:43:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40260.log 2024-09-17T05:43:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40260.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40260.log.gz 2024-09-17T05:43:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34498.log 2024-09-17T05:43:06.051 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40101.log 2024-09-17T05:43:06.051 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34498.log: /var/log/ceph/ceph-client.admin.40101.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40101.log.gz 2024-09-17T05:43:06.052 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56502.log 2024-09-17T05:43:06.052 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34498.log.gz 2024-09-17T05:43:06.054 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.56502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39534.log 2024-09-17T05:43:06.055 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56502.log.gz 2024-09-17T05:43:06.056 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36538.log 2024-09-17T05:43:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39534.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39534.log.gz 2024-09-17T05:43:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47133.log 2024-09-17T05:43:06.062 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36538.log.gz 2024-09-17T05:43:06.064 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55765.log 2024-09-17T05:43:06.066 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47133.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47133.log.gz 2024-09-17T05:43:06.068 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.55765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42459.log 2024-09-17T05:43:06.069 INFO:teuthology.orchestra.run.smithi122.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55765.log.gz 2024-09-17T05:43:06.071 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46560.log 2024-09-17T05:43:06.075 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46560.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46560.log.gz 2024-09-17T05:43:06.075 INFO:teuthology.orchestra.run.smithi122.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42459.log.gz 2024-09-17T05:43:06.075 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50524.log 2024-09-17T05:43:06.078 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50524.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50524.log.gz 2024-09-17T05:43:06.078 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36372.log 2024-09-17T05:43:06.081 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2024-09-17T05:43:06.081 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36372.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36372.log.gz 2024-09-17T05:43:06.084 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52840.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2024-09-17T05:43:06.084 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36137.log 2024-09-17T05:43:06.087 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.36137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36137.log.gz 2024-09-17T05:43:06.087 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35880.log 2024-09-17T05:43:06.090 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35880.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35880.log.gz 2024-09-17T05:43:06.090 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52385.log 2024-09-17T05:43:06.091 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38679.log 2024-09-17T05:43:06.095 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52385.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52385.log.gz 2024-09-17T05:43:06.095 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38679.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38679.log.gz 2024-09-17T05:43:06.206 INFO:teuthology.orchestra.run.smithi122.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T05:43:06.462 INFO:teuthology.orchestra.run.smithi122.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-17T05:43:07.018 INFO:teuthology.orchestra.run.smithi173.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-17T05:43:08.524 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T05:43:09.191 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T05:43:09.201 INFO:teuthology.orchestra.run.smithi122.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T05:43:09.428 INFO:teuthology.orchestra.run.smithi122.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T05:43:09.610 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T05:43:09.837 INFO:teuthology.orchestra.run.smithi173.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T05:43:09.998 INFO:teuthology.orchestra.run.smithi173.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T05:43:10.000 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-09-17T05:43:10.000 INFO:teuthology.orchestra.run.smithi173.stderr:real 0m5.191s 2024-09-17T05:43:10.000 INFO:teuthology.orchestra.run.smithi173.stderr:user 0m24.413s 2024-09-17T05:43:10.000 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m2.009s 2024-09-17T05:43:10.060 INFO:teuthology.orchestra.run.smithi122.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T05:43:10.161 INFO:teuthology.orchestra.run.smithi122.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T05:43:10.845 INFO:teuthology.orchestra.run.smithi122.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-17T05:43:10.847 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-17T05:43:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m6.078s 2024-09-17T05:43:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m25.319s 2024-09-17T05:43:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m3.133s 2024-09-17T05:43:10.848 INFO:tasks.ceph:Archiving logs... 2024-09-17T05:43:10.848 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/remote/smithi122/log 2024-09-17T05:43:10.849 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T05:43:14.057 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/remote/smithi173/log 2024-09-17T05:43:14.058 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T05:43:16.602 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T05:43:16.613 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T05:43:16.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T05:43:16.616 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T05:43:16.696 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T05:43:16.705 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T05:43:16.705 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:43:16.709 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:43:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:43:16.721 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-17T05:43:16.721 INFO:teuthology.orchestra.run.smithi122.stdout:+hv01.front.sepi 67.205.162.81 3 u 19 64 377 0.093 -9.242 0.775 2024-09-17T05:43:16.721 INFO:teuthology.orchestra.run.smithi122.stdout:+hv02.front.sepi 63.231.80.2 3 u 12 64 377 0.082 -6.712 1.401 2024-09-17T05:43:16.721 INFO:teuthology.orchestra.run.smithi122.stdout:*hv03.front.sepi 74.6.168.72 3 u 3 64 377 0.077 -6.386 0.617 2024-09-17T05:43:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-17T05:43:16.724 INFO:teuthology.orchestra.run.smithi173.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:43:16.724 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================== 2024-09-17T05:43:16.725 INFO:teuthology.orchestra.run.smithi173.stdout:*hv01.front.sepi 67.205.162.81 3 u 7 64 377 0.082 -6.872 1.545 2024-09-17T05:43:16.725 INFO:teuthology.orchestra.run.smithi173.stdout:+hv02.front.sepi 63.231.80.2 3 u 3 64 377 0.134 -7.604 1.634 2024-09-17T05:43:16.725 INFO:teuthology.orchestra.run.smithi173.stdout:+hv03.front.sepi 74.6.168.72 3 u 14 64 377 0.110 -5.994 1.437 2024-09-17T05:43:16.725 INFO:teuthology.orchestra.run.smithi173.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-17T05:43:16.728 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T05:43:16.737 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T05:43:16.737 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T05:43:16.767 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T05:43:16.801 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T05:43:16.834 INFO:teuthology.task.internal:Duration was 1821.098841 seconds 2024-09-17T05:43:16.835 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T05:43:16.868 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T05:43:16.868 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T05:43:16.872 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T05:43:16.965 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T05:43:16.965 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-17T05:43:16.965 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T05:43:16.985 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-09-17T05:43:16.986 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T05:43:17.007 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T05:43:17.007 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T05:43:17.034 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T05:43:17.094 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T05:43:17.094 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T05:43:17.118 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T05:43:17.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T05:43:17.226 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T05:43:17.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T05:43:17.246 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T05:43:17.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T05:43:17.289 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:43:17.292 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:43:17.306 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-17T05:43:17.327 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-09-17T05:43:17.345 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:43:17.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:43:17.378 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:43:17.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:43:17.397 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T05:43:17.408 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T05:43:17.409 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/remote/smithi122 2024-09-17T05:43:17.410 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T05:43:17.445 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/avan-2024-09-17_05:02:16-fs:volumes-wip-athakkar-testing-2024-09-16-2351-distro-default-smithi/7908689/remote/smithi173 2024-09-17T05:43:17.445 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T05:43:17.471 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T05:43:17.472 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T05:43:17.489 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T05:43:17.519 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T05:43:17.534 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T05:43:17.534 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T05:43:17.561 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T05:43:17.561 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T05:43:17.564 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T05:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: 658143 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 05:43 /home/ubuntu/cephtest 2024-09-17T05:43:17.572 INFO:teuthology.orchestra.run.smithi173.stdout: 655428 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 05:43 /home/ubuntu/cephtest 2024-09-17T05:43:17.574 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T05:43:17.605 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T05:43:17.691 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 1821.098840713501 flavor: default owner: scheduled_avan@teuthology success: true 2024-09-17T05:43:17.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T05:43:17.787 INFO:teuthology.run:pass